-- Logs begin at Sat 2022-10-15 20:01:13 UTC, end at Sat 2022-10-15 22:33:42 UTC. -- Oct 15 20:31:12.493951 ncdv-master-1 systemd[1]: Starting Devstack devstack@keystone.service... Oct 15 20:31:12.502808 ncdv-master-1 devstack@keystone.service[99087]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 15 20:31:12.505757 ncdv-master-1 devstack@keystone.service[99087]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Oct 15 20:31:12 2022] *** Oct 15 20:31:12.505958 ncdv-master-1 devstack@keystone.service[99087]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 15 20:31:12.506121 ncdv-master-1 devstack@keystone.service[99087]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 15 20:31:12.506274 ncdv-master-1 devstack@keystone.service[99087]: nodename: ncdv-master-1 Oct 15 20:31:12.506430 ncdv-master-1 devstack@keystone.service[99087]: machine: x86_64 Oct 15 20:31:12.506590 ncdv-master-1 devstack@keystone.service[99087]: clock source: unix Oct 15 20:31:12.506745 ncdv-master-1 devstack@keystone.service[99087]: pcre jit disabled Oct 15 20:31:12.506903 ncdv-master-1 devstack@keystone.service[99087]: detected number of CPU cores: 4 Oct 15 20:31:12.507071 ncdv-master-1 devstack@keystone.service[99087]: current working directory: / Oct 15 20:31:12.507224 ncdv-master-1 devstack@keystone.service[99087]: detected binary path: /usr/bin/uwsgi-core Oct 15 20:31:12.507382 ncdv-master-1 devstack@keystone.service[99087]: your processes number limit is 47680 Oct 15 20:31:12.507535 ncdv-master-1 devstack@keystone.service[99087]: your memory page size is 4096 bytes Oct 15 20:31:12.507688 ncdv-master-1 devstack@keystone.service[99087]: detected max file descriptor number: 2048 Oct 15 20:31:12.507856 ncdv-master-1 devstack@keystone.service[99087]: lock engine: pthread robust mutexes Oct 15 20:31:12.508164 ncdv-master-1 devstack@keystone.service[99087]: thunder lock: enabled Oct 15 20:31:12.508508 ncdv-master-1 devstack@keystone.service[99087]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 15 20:31:12.508668 ncdv-master-1 devstack@keystone.service[99087]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 15 20:31:12.554929 ncdv-master-1 devstack@keystone.service[99087]: Python main interpreter initialized at 0x55c1834679c0 Oct 15 20:31:12.554929 ncdv-master-1 devstack@keystone.service[99087]: python threads support enabled Oct 15 20:31:12.554929 ncdv-master-1 devstack@keystone.service[99087]: your server socket listen backlog is limited to 100 connections Oct 15 20:31:12.554929 ncdv-master-1 devstack@keystone.service[99087]: your mercy for graceful operations on workers is 90 seconds Oct 15 20:31:12.555320 ncdv-master-1 devstack@keystone.service[99087]: mapped 403077 bytes (393 KB) for 2 cores Oct 15 20:31:12.555395 ncdv-master-1 devstack@keystone.service[99087]: *** Operational MODE: preforking *** Oct 15 20:31:12.555594 ncdv-master-1 devstack@keystone.service[99087]: *** uWSGI is running in multiple interpreter mode *** Oct 15 20:31:12.555594 ncdv-master-1 devstack@keystone.service[99087]: spawned uWSGI master process (pid: 99087) Oct 15 20:31:12.555803 ncdv-master-1 systemd[1]: Started Devstack devstack@keystone.service. Oct 15 20:31:12.556893 ncdv-master-1 devstack@keystone.service[99087]: spawned uWSGI worker 1 (pid: 99099, cores: 1) Oct 15 20:31:12.556893 ncdv-master-1 devstack@keystone.service[99087]: spawned uWSGI worker 2 (pid: 99100, cores: 1) Oct 15 20:31:12.556893 ncdv-master-1 devstack@keystone.service[99087]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 15 20:31:17.201985 ncdv-master-1 devstack@keystone.service[99100]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:31:17.201985 ncdv-master-1 devstack@keystone.service[99100]: from cryptography.utils import int_from_bytes Oct 15 20:31:17.202696 ncdv-master-1 devstack@keystone.service[99100]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:31:17.202696 ncdv-master-1 devstack@keystone.service[99100]: from cryptography.utils import int_from_bytes Oct 15 20:31:17.204268 ncdv-master-1 devstack@keystone.service[99099]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:31:17.204268 ncdv-master-1 devstack@keystone.service[99099]: from cryptography.utils import int_from_bytes Oct 15 20:31:17.204712 ncdv-master-1 devstack@keystone.service[99099]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:31:17.204712 ncdv-master-1 devstack@keystone.service[99099]: from cryptography.utils import int_from_bytes Oct 15 20:31:17.251848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 20:31:17.252044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 20:31:17.252233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] command line args: [] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 20:31:17.252437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 20:31:17.252531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 20:31:17.252633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 20:31:17.252712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ================================================================================ {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 20:31:17.252830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] command line args: [] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 20:31:17.252914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] admin_token = **** {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.253130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 20:31:17.253213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] config_dir = [] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.253294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ================================================================================ {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 20:31:17.253365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.253492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] config_source = [] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.253580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] admin_token = **** {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.253673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] debug = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.253784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] config_dir = [] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.253860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.254170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.254170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] config_source = [] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.254372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.254467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] debug = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.254564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] insecure_debug = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.254666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.254808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.254907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.255003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.255098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] insecure_debug = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.255199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] list_limit = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.255294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.255400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] log_config_append = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.255502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.255611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.255611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] log_dir = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.255801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] list_limit = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.255904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] log_file = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.256015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] log_config_append = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.256015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.256233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.256316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] log_dir = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.256433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.256514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] log_file = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.256594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.256717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.256798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.256877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.257006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.257154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.257242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.257361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.257443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.257559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.257640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.257769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.257853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.257972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] notification_format = cadf {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.258987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.259186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.259438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] notification_format = cadf {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.259575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.259706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.259783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] publish_errors = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.259988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.260194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.260284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.260362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] publish_errors = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.260433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.260598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.260680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.260829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.260907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.260998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.261094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] standard_threads = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.261269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.261355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] strict_password_check = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.261504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.261581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.261723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] use_eventlog = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.261803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] standard_threads = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.261924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] use_journal = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.262004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] strict_password_check = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.262081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] use_json = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.262189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.262265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] use_stderr = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.262366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] use_eventlog = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.262456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] use_syslog = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.263152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] watch_log_file = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.263258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] use_journal = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.263334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] use_json = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.263404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.263496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] use_stderr = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.263639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.263717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] use_syslog = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.263898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.263974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] watch_log_file = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:31:17.264140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.264218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.264382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.264464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.264617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.264692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.264863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.264939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.265124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.265217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.265365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.265448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.265607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.265689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.265845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.265921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.266074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.266155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.266319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.266396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.266554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.266632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.266788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.266863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.267030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.267119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.267265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.267350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.267510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.267586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.267759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.267834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.267982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.268055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.268223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.268305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.268471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.268553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.268717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.268794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.268956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.269068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.269227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.269312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.269456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.269532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.269684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.269760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.269909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.269996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.270141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.270215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.270384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.270450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.270626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.270704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.270850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.270916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.271078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.271143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.271320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.271399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.271551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.271616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.271783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.271849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.272024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.272100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.272251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.272323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.272483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.272555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.272714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.272788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.272940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.273007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.273189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.273257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.273417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.273555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.273643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.273709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.273874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.273939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.274105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.274181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.274332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.274813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.274884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.274943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.274999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.275964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.276130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.276209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.276362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.276428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.276594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.276659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.276824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.276904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.277090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.277163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.277308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.277376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.277538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.277605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.277768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.277839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.277994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.278069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.278288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.278365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.278528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.278597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.278762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.278845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.279023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.279091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.279262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.279330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.279493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.279559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.279731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.279798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.279969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.280037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.280211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.280284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.280467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.280549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.280688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.280757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.280922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.280988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.281192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.enabled = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.281272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.281427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.281525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.281661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.281739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.281900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.281978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.enabled = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.282134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.282219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.282368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.282450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.282600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.282676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.282832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.282911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.283064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.283141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.283298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.283375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.283528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.283605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.283765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.283842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.284007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.284085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.284244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.284328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.284478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.284555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.284712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.284791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.284935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.285009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.285194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.285275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.285426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.285503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.285658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.285746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.285884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.285963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.286112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.286190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.286337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.286423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.286566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.286642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.286788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.286863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.287016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.287101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.287246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.287323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.287473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.287549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.287711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.287799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.287945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.288022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.288195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.288273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.288430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.288514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.288657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] auth.external = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.288735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.288883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] auth.mapped = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.288971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.289182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.289262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.289416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] auth.external = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.289504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.289641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] auth.password = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.289719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] auth.mapped = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.289867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] auth.token = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.289944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.290102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.290188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.290336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] catalog.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.290421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] auth.password = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.290584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.290662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] auth.token = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.290828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.290960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.291047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] catalog.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.291125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.291282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.291361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.291507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.291592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.291740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] credential.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.291819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.291983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] credential.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.292063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.292213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.292298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.292452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] credential.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.292539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.292698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.292774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] credential.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.292937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.293024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.293195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.293275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.293439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.293517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.293684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.294261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.294438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.294582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.294730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.294814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.294944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.295032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.295183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.295263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.295438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.295518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.295726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.295813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.295973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.296052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.296211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] federation.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.296288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.296445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.296532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.296689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] federation.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.296768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.296933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.297011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] federation.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.297187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.297285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.297433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] federation.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.297511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.297666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.297744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.297901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.297986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.298137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.298215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.298377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.298454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.298607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.298686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.298850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.298928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.299082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.299162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.299326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.299409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.299569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.299648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.299801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.299879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.300032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.300112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.300274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.300352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.300501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.300585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.300733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.300814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.300980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.301078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.301241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.301320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.301467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.301547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.301696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.301783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.301926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.302005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.302168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.302257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.302398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.302475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.302628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.302707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.302875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.302956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.303116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.303195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.303364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.303441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.303602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.303681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.303837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.303915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.304065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.304143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.304298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.304378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.304525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.304603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.304758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.304836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.304993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.305092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.305253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.305331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.305493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.305571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.305717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.305796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.305959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.306038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.306201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.306279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.306429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.306507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.306668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.306745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.306909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.306987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.307136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.307215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.307368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.307446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.307594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.password = **** {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.307672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.307826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.307906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.308056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.308139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.308283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.password = **** {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.308361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.308514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.308593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.308749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.308834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.308987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.309115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.309250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.309331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.310042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.310136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.310331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.310425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.310581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.310661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.310814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.310893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.311046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.311133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.311298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.311378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.311530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.311607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.311757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.311845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.311994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.312073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.312229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.312306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.312461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.312548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.312701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.312779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.312947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.313025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.313202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.313299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.313434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.313512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.313663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.313740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.313895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.313982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.314128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.314205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.314358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.314436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.314590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.314715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.314831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.314906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.315069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.315147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.315300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.315387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.315540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.315648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.315783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.315863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.316009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.316097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.316252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.316331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.316485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.316564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.316718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.316812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.316946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.317031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.317201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.317280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.317443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.317534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.317679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.317774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.317910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.317988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.318154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.318240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] policy.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.318379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.318456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.318620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.318698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.318853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.318940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] policy.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.319086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] receipt.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.319165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.319319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.319397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.319553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.319639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.319789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.319866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] receipt.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.320016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.320094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.320250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.320337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.320490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] resource.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.320568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.320720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.320797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.320960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.321101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] resource.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.321217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.321295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] resource.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.321462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.321541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.321710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] resource.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.321800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.321940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] revoke.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.322018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.322183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.322261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.322416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.322569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.322645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] revoke.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.322714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] role.cache_time = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.322870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] role.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.322947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.323105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.323190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] role.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.323328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] role.list_limit = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.323406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] role.cache_time = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.323570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] role.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.323698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.323813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.323902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] role.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.324043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] role.list_limit = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.324120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.324286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.324364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.324529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.324610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.324770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.324849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.325080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.325169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.325321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.325401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.325571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.325755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.325833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.326005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.326093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.326252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.326331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.326496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.326574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.326739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.326826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.326985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.327137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.327235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.327372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.327617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.327703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.327863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.327942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.328107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.328186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.328353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.328440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.328582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.328660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.328830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.328909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.329083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.329177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.329333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.329410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.329560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.329638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.329786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.329866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.330013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.330091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.330243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.330320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.330474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.330554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.330726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.330804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.330954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.331032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.331193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.331274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.331419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] token.cache_time = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.331497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.331651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] token.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.331730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.331879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] token.cache_time = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.331957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.332118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] token.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.332196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] token.provider = fernet {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.332347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.332424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.332592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] token.provider = fernet {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.332671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.332829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.332908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.333095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.333176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.333326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.333407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.333568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.333647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.333808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.333884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] trust.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.334038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.334116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.334273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.334351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] trust.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.334505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.334583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.334741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.334820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.334978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.335057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.335224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.335302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.335459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.335555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.335719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.335799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.335958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.336046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.connection = **** {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.336199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.336276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.336441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.336529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.connection = **** {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.336673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.336752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.336909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.336989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.337165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.337259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.337398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.337477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.337627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.337705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.337856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.337945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.338089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.338168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.338317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.338395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.338555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.338643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.338785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.338865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.339027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.339108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.339254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.339340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.339494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.339573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.339719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.339799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.339954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.340041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.340181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.340259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.340344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 20:31:17.340422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.340679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:31:17.340852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 20:31:17.370006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.370232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.370461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.373496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.373724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.373955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.379415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.382412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.407964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.408190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.409476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.409707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.460535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.460847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.509575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.509798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.510461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.510694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.575270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.575492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.575707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.575825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.575914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.576056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.576141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.576278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.576363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.576497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.576712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.576931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.599353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.599585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.599761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.599847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.599995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.600093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.600201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.600283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.600407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.600489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.600613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.600702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.600834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.601065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.612189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.612495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.625192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.625417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.625637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.625719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.625861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.625999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.626085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.626161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.626265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.626349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.626471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.626563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.626726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.626941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 20:31:17.643659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.643863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 20:31:17.643994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.644202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.644283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 20:31:17.644397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 20:31:17.644541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.644670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.644761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 20:31:17.644930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.645009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.645214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.645295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.645469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.645549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.645766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.655146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.655417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.655499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.655681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.655790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.655957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.656079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.656333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.662970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.663321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.665858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.666140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.666251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.666420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.666535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.666701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.666817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.666981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.667097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.667239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.667379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.667505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.667640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.667776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.667910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.668041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.668182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.668310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.668457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.668577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.668793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.669287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.693252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.694206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.694700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.694979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.695672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.695958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.699632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.699913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.700251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.700840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.701152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.701439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.706725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.707005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.707886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.708174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.711153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.711396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.711683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.711968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.712243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.712320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.712511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.712588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.712789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.712868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.713091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.713172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.713464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.713749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.714032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.714336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.729194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.729466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.730729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.731011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.733440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.735093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.736635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.736924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.738334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.738642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.741175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.742901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.744288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.746338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.748300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.748594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.748873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.749182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.749460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.749744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.750257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.750560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.750844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.751140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.751417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.751704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.765662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.765943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.766204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.766462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.767680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.767969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.768236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.768499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.773289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.774803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.775395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.776339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.776610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.776933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.779116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.779389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.779887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.780168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.780459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.780741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.781150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.782094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.782374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.782665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.782957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.783342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.792289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.792567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.792841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.793152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.793539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.793911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.794255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.794692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.794978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.795266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.795546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.795933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.796223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.796503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.810111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.812596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.813124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.815534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.816065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.817523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.818495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.818985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.819262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.819650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.819935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.821438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.821720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.822113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.826040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.827483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.828908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.830364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.830557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.831595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.831891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.832166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.833503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.834754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.835056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.835348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.840014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.840297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.840575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.843789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.844081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.844382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.848009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=99100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.848290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.848566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.848839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.849228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.849506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.849785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.850062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.850336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.850601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.850865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.851127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.851296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=99099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 20:31:17.851387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.851587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.851670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=99100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.851885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.852140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.852414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.852738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.853011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.853318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.853596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.853866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.854137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.854413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.854681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.854942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=99099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 20:31:17.885749 ncdv-master-1 devstack@keystone.service[99100]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c1834679c0 pid: 99100 (default app) Oct 15 20:31:17.888904 ncdv-master-1 devstack@keystone.service[99099]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c1834679c0 pid: 99099 (default app) Oct 15 20:31:17.889016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8987da-34fe-47a3-bfee-55bbda8878be None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:17.889286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8987da-34fe-47a3-bfee-55bbda8878be None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:17.889550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8987da-34fe-47a3-bfee-55bbda8878be None None] PATH_INFO: `/v3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:17.893665 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Sat Oct 15 20:31:17 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:25.130400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfceaf1-2514-404e-9655-3ef7566cbd7b None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:25.130968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfceaf1-2514-404e-9655-3ef7566cbd7b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:25.130968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfceaf1-2514-404e-9655-3ef7566cbd7b None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:25.135225 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:31:25.140869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:25.141454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:25.141454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:25.222082 ncdv-master-1 devstack@keystone.service[99100]: INFO dbcounter [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] Registered counter for database keystone Oct 15 20:31:25.241019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG oslo_db.sqlalchemy.engines [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99100) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:31:25.247909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writer thread running {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:31:25.424816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG passlib.handlers.bcrypt [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=99100) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 15 20:31:25.425228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG passlib.handlers.bcrypt [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=99100) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 15 20:31:25.482941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:25.484518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:25.611309 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 20:31:25.948729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfceaf1-2514-404e-9655-3ef7566cbd7b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:26.026060 ncdv-master-1 devstack@keystone.service[99099]: INFO dbcounter [None req-4cfceaf1-2514-404e-9655-3ef7566cbd7b None None] Registered counter for database keystone Oct 15 20:31:26.048275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG oslo_db.sqlalchemy.engines [None req-4cfceaf1-2514-404e-9655-3ef7566cbd7b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99099) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:31:26.054515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writer thread running {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:31:26.177775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfceaf1-2514-404e-9655-3ef7566cbd7b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:26.182079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edf904-83c2-4c64-b394-0aed7cfdd241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:26.182695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edf904-83c2-4c64-b394-0aed7cfdd241 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:26.182971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edf904-83c2-4c64-b394-0aed7cfdd241 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:26.183237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edf904-83c2-4c64-b394-0aed7cfdd241 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:26.191707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2edf904-83c2-4c64-b394-0aed7cfdd241 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:26.245078 ncdv-master-1 devstack@keystone.service[99099]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e2edf904-83c2-4c64-b394-0aed7cfdd241 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 15 20:31:26.245411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2edf904-83c2-4c64-b394-0aed7cfdd241 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:26.255436 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Sat Oct 15 20:31:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:31:26.261377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:26.279032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b06e8-337b-48fb-8390-701d744c729f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:26.283369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02674654-e394-4323-ab39-ff98727b5f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:26.284455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674654-e394-4323-ab39-ff98727b5f6c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:26.284727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674654-e394-4323-ab39-ff98727b5f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:26.284994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674654-e394-4323-ab39-ff98727b5f6c admin admin] PATH_INFO: `/v3/projects/38e66271254b4380bfa7d7e1ccea54cc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:26.286462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02674654-e394-4323-ab39-ff98727b5f6c admin admin] RBAC: Authorizing `identity:get_project(project_id=38e66271254b4380bfa7d7e1ccea54cc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:26.337056 ncdv-master-1 devstack@keystone.service[99100]: WARNING keystone.common.rbac_enforcer.enforcer [None req-02674654-e394-4323-ab39-ff98727b5f6c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 15 20:31:26.337408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02674654-e394-4323-ab39-ff98727b5f6c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:26.342595 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Sat Oct 15 20:31:26 2022] GET /identity/v3/projects/38e66271254b4380bfa7d7e1ccea54cc => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:28.692255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a311-b2e2-41db-8752-fbb3c0643f15 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:28.692799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a311-b2e2-41db-8752-fbb3c0643f15 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:28.692799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a311-b2e2-41db-8752-fbb3c0643f15 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:28.697116 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:31:28.702642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca3074-cfe4-47e5-b131-77b5cda096b8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:28.703017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca3074-cfe4-47e5-b131-77b5cda096b8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:28.703172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca3074-cfe4-47e5-b131-77b5cda096b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:28.722811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-37ca3074-cfe4-47e5-b131-77b5cda096b8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:28.724297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-37ca3074-cfe4-47e5-b131-77b5cda096b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:28.732579 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 20:31:29.054253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a311-b2e2-41db-8752-fbb3c0643f15 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:29.067040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a311-b2e2-41db-8752-fbb3c0643f15 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:29.071221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d8614-61e8-4883-bb85-aaaff5e0620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:29.071802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d8614-61e8-4883-bb85-aaaff5e0620a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:29.072074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d8614-61e8-4883-bb85-aaaff5e0620a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:29.072338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d8614-61e8-4883-bb85-aaaff5e0620a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:29.080499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d8614-61e8-4883-bb85-aaaff5e0620a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:29.082277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d8614-61e8-4883-bb85-aaaff5e0620a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:29.090980 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Sat Oct 15 20:31:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:31:29.096172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca3074-cfe4-47e5-b131-77b5cda096b8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:29.105148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca3074-cfe4-47e5-b131-77b5cda096b8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:29.109428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347df9c-b3de-4bca-8e42-ca756569660f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:29.110033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347df9c-b3de-4bca-8e42-ca756569660f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:29.110303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347df9c-b3de-4bca-8e42-ca756569660f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:29.110572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347df9c-b3de-4bca-8e42-ca756569660f admin admin] PATH_INFO: `/v3/users/f9ecef8e2dba4f429d37c67479eb320e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:29.111850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347df9c-b3de-4bca-8e42-ca756569660f admin admin] RBAC: Authorizing `identity:get_user(user_id=f9ecef8e2dba4f429d37c67479eb320e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:29.113740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347df9c-b3de-4bca-8e42-ca756569660f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:29.115218 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Sat Oct 15 20:31:29 2022] GET /identity/v3/users/f9ecef8e2dba4f429d37c67479eb320e => generated 247 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:31.546408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad36f2-ad7e-4d41-a82e-d3e5fa996351 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:31.546908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad36f2-ad7e-4d41-a82e-d3e5fa996351 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:31.546908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad36f2-ad7e-4d41-a82e-d3e5fa996351 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:31.548185 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:31.553390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d5b2e-2aad-4d8a-b6ff-86037aed65d1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:31.553786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d5b2e-2aad-4d8a-b6ff-86037aed65d1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:31.554056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d5b2e-2aad-4d8a-b6ff-86037aed65d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:31.573495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a35d5b2e-2aad-4d8a-b6ff-86037aed65d1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:31.575002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a35d5b2e-2aad-4d8a-b6ff-86037aed65d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:31.619547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e563c-95b1-4525-8973-925302f77a23 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:31.619834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e563c-95b1-4525-8973-925302f77a23 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:31.620102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e563c-95b1-4525-8973-925302f77a23 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:31.622310 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:31:31.628324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090c95e-28a2-49e2-a8db-238c250df179 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:31.628602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090c95e-28a2-49e2-a8db-238c250df179 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:31.628871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090c95e-28a2-49e2-a8db-238c250df179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:31.674430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG passlib.handlers.bcrypt [None req-e090c95e-28a2-49e2-a8db-238c250df179 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=99099) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 15 20:31:31.674824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG passlib.handlers.bcrypt [None req-e090c95e-28a2-49e2-a8db-238c250df179 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=99099) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 15 20:31:31.724220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e090c95e-28a2-49e2-a8db-238c250df179 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:31.725874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e090c95e-28a2-49e2-a8db-238c250df179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:31.764061 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:31:31.826297 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:31:32.083826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d5b2e-2aad-4d8a-b6ff-86037aed65d1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:32.113109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:32.158266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090c95e-28a2-49e2-a8db-238c250df179 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:32.199851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d5b2e-2aad-4d8a-b6ff-86037aed65d1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:32.236257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1655a-8db3-4238-9688-a963e3ea5b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:32.236894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1655a-8db3-4238-9688-a963e3ea5b6f None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:32.237199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1655a-8db3-4238-9688-a963e3ea5b6f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:32.237473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1655a-8db3-4238-9688-a963e3ea5b6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:32.248747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1655a-8db3-4238-9688-a963e3ea5b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-59c1655a-8db3-4238-9688-a963e3ea5b6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:32.257556 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:32.258522 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:31:32.259457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:31:32.260107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090c95e-28a2-49e2-a8db-238c250df179 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:32.260174 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:31:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:31:32.262957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:32.264971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1655a-8db3-4238-9688-a963e3ea5b6f None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:32.299571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845a2cc-86f0-4429-be2f-5feb07795cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:32.300430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845a2cc-86f0-4429-be2f-5feb07795cdc None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:32.300708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845a2cc-86f0-4429-be2f-5feb07795cdc None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:32.300973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845a2cc-86f0-4429-be2f-5feb07795cdc None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:32.301973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845a2cc-86f0-4429-be2f-5feb07795cdc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:32.303403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845a2cc-86f0-4429-be2f-5feb07795cdc None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:32.315079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.sql.core [None req-e845a2cc-86f0-4429-be2f-5feb07795cdc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 20:31:32.315079 ncdv-master-1 devstack@keystone.service[99099]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 20:31:32.315079 ncdv-master-1 devstack@keystone.service[99099]: [parameters: {'id': 'd8756f3e2e4e4aa4ae50f6c7d8c5af56', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 20:31:32.315079 ncdv-master-1 devstack@keystone.service[99099]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99099) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-e845a2cc-86f0-4429-be2f-5feb07795cdc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application conn.query(q) Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:31:32.335786 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application result.read() Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 20:31:32.336663 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application next(self.gen) Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application yield resource Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application next(self.gen) Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise self.value Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application session.commit() Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 20:31:32.337644 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.session.flush() Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application compat.raise_( Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 20:31:32.338479 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 20:31:32.339322 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application conn.query(q) Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application result.read() Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 20:31:32.341779 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [parameters: {'id': 'd8756f3e2e4e4aa4ae50f6c7d8c5af56', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 20:31:32.342948 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:32.343996 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 8/15] 10.222.0.16 () {60 vars in 1137 bytes} [Sat Oct 15 20:31:32 2022] POST /identity/v3/domains => generated 190 bytes in 179 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:31:32.345204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845a2cc-86f0-4429-be2f-5feb07795cdc None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:32.351272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:32.370342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1655a-8db3-4238-9688-a963e3ea5b6f None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:32.406822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560d4a7-0daa-4918-bdd0-051c6f803df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:32.407422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560d4a7-0daa-4918-bdd0-051c6f803df8 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:32.407696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560d4a7-0daa-4918-bdd0-051c6f803df8 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:32.407965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560d4a7-0daa-4918-bdd0-051c6f803df8 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:32.408624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560d4a7-0daa-4918-bdd0-051c6f803df8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:32.408922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560d4a7-0daa-4918-bdd0-051c6f803df8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:32.410359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560d4a7-0daa-4918-bdd0-051c6f803df8 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:32.422500 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 8/16] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:31:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:31:32.428242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560d4a7-0daa-4918-bdd0-051c6f803df8 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:32.444157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845a2cc-86f0-4429-be2f-5feb07795cdc None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:32.477821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e9e7d-93c4-40bc-b941-3c084eb58cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:32.478638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e9e7d-93c4-40bc-b941-3c084eb58cf1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:32.478911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e9e7d-93c4-40bc-b941-3c084eb58cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:32.479177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e9e7d-93c4-40bc-b941-3c084eb58cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:32.500566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e9e7d-93c4-40bc-b941-3c084eb58cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:32.502059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e9e7d-93c4-40bc-b941-3c084eb58cf1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-169e9e7d-93c4-40bc-b941-3c084eb58cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:31:32.517306 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:32.518142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:31:32.518901 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:31:32.518901 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:31:32.518901 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:32.518901 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:31:32.524282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e9e7d-93c4-40bc-b941-3c084eb58cf1 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:32.528388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560d4a7-0daa-4918-bdd0-051c6f803df8 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:32.565024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20afb5-bd08-481b-87a9-4580bf7040b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:32.565707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20afb5-bd08-481b-87a9-4580bf7040b5 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:32.565951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20afb5-bd08-481b-87a9-4580bf7040b5 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:32.566215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20afb5-bd08-481b-87a9-4580bf7040b5 None admin] PATH_INFO: `/v3/users/f9ecef8e2dba4f429d37c67479eb320e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:32.567412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20afb5-bd08-481b-87a9-4580bf7040b5 None admin] RBAC: Authorizing `identity:get_user(user_id=f9ecef8e2dba4f429d37c67479eb320e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:32.568907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20afb5-bd08-481b-87a9-4580bf7040b5 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:32.570567 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:32 2022] GET /identity/v3/users/f9ecef8e2dba4f429d37c67479eb320e => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:32.576839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20afb5-bd08-481b-87a9-4580bf7040b5 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:32.593142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:32.655360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:32.684569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20afb5-bd08-481b-87a9-4580bf7040b5 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:32.717554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2164e9f-6f37-4eb5-a1b6-9b52b9da635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:32.718175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2164e9f-6f37-4eb5-a1b6-9b52b9da635b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:32.718452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2164e9f-6f37-4eb5-a1b6-9b52b9da635b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:32.718719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2164e9f-6f37-4eb5-a1b6-9b52b9da635b None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:32.719808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2164e9f-6f37-4eb5-a1b6-9b52b9da635b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:32.721313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2164e9f-6f37-4eb5-a1b6-9b52b9da635b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:32.722850 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:31:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:31:32.728299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2164e9f-6f37-4eb5-a1b6-9b52b9da635b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:32.734717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e9e7d-93c4-40bc-b941-3c084eb58cf1 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:32.768531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe6676-cfcc-4d22-bdaa-25f106175e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:32.769154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe6676-cfcc-4d22-bdaa-25f106175e51 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:32.769425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe6676-cfcc-4d22-bdaa-25f106175e51 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:32.769688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe6676-cfcc-4d22-bdaa-25f106175e51 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:32.770319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe6676-cfcc-4d22-bdaa-25f106175e51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:32.770608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe6676-cfcc-4d22-bdaa-25f106175e51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:32.771964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe6676-cfcc-4d22-bdaa-25f106175e51 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:32.789357 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:31:32.809477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:32.831152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2164e9f-6f37-4eb5-a1b6-9b52b9da635b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:32.868617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581ab34-887f-4fe9-bb5f-5d3218e0e5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:32.869493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581ab34-887f-4fe9-bb5f-5d3218e0e5f7 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:32.869773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581ab34-887f-4fe9-bb5f-5d3218e0e5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:32.870048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581ab34-887f-4fe9-bb5f-5d3218e0e5f7 None admin] PATH_INFO: `/v3/domains/default/users/f9ecef8e2dba4f429d37c67479eb320e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:32.873139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581ab34-887f-4fe9-bb5f-5d3218e0e5f7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f9ecef8e2dba4f429d37c67479eb320e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:32.874646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581ab34-887f-4fe9-bb5f-5d3218e0e5f7 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:33.000836 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 11/21] 10.222.0.16 () {58 vars in 1361 bytes} [Sat Oct 15 20:31:32 2022] PUT /identity/v3/domains/default/users/f9ecef8e2dba4f429d37c67479eb320e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:31:35.893667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8092252-05ad-4e21-8855-b71dc63bebb0 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:35.894207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8092252-05ad-4e21-8855-b71dc63bebb0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:35.894207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8092252-05ad-4e21-8855-b71dc63bebb0 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:35.895508 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:35.901499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b53ee-3a10-47a6-89de-e974c479d291 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:35.901898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b53ee-3a10-47a6-89de-e974c479d291 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:35.902113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b53ee-3a10-47a6-89de-e974c479d291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:35.930191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ee2b53ee-3a10-47a6-89de-e974c479d291 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:35.931722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ee2b53ee-3a10-47a6-89de-e974c479d291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:36.012109 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 12/23] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:36.099846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af759d-ed0d-4111-8e37-90121574e1c1 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:36.100166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af759d-ed0d-4111-8e37-90121574e1c1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:36.100376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af759d-ed0d-4111-8e37-90121574e1c1 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:36.101724 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 12/24] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:36.118721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e648772-fa1e-43a3-8392-0d7e8986a4e5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:36.119000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e648772-fa1e-43a3-8392-0d7e8986a4e5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:36.119525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e648772-fa1e-43a3-8392-0d7e8986a4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:36.143656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5e648772-fa1e-43a3-8392-0d7e8986a4e5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:36.147550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5e648772-fa1e-43a3-8392-0d7e8986a4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:36.244631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:36.251960 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 13/25] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:36.438772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af759d-ed0d-4111-8e37-90121574e1c1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:36.469839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326d4c6-7c00-4338-bf4f-5274d2fc80bd None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:36.470157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326d4c6-7c00-4338-bf4f-5274d2fc80bd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:36.470387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326d4c6-7c00-4338-bf4f-5274d2fc80bd None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:36.474811 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 14/26] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:36.488024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c662e-3cdb-434f-9c56-cfc11455ed24 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:36.488297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c662e-3cdb-434f-9c56-cfc11455ed24 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:36.488574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c662e-3cdb-434f-9c56-cfc11455ed24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:36.520999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-180c662e-3cdb-434f-9c56-cfc11455ed24 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:36.522201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-180c662e-3cdb-434f-9c56-cfc11455ed24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:36.573247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:36.626734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af759d-ed0d-4111-8e37-90121574e1c1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:36.645386 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 15/27] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:36.680477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873c00f-ac34-400e-917e-0e5873811507 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:36.680769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873c00f-ac34-400e-917e-0e5873811507 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:36.681017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873c00f-ac34-400e-917e-0e5873811507 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:36.682394 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 16/28] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:31:36.687631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0012e8-ace8-46aa-a12b-5b6301fd7c6b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:36.687913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0012e8-ace8-46aa-a12b-5b6301fd7c6b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:36.688182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0012e8-ace8-46aa-a12b-5b6301fd7c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:36.702143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb4a1a-bef2-444c-9dc1-d7f0361aa0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:36.702744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb4a1a-bef2-444c-9dc1-d7f0361aa0de None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:36.703013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb4a1a-bef2-444c-9dc1-d7f0361aa0de None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:36.703352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb4a1a-bef2-444c-9dc1-d7f0361aa0de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:36.717477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0d0012e8-ace8-46aa-a12b-5b6301fd7c6b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:36.718958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0d0012e8-ace8-46aa-a12b-5b6301fd7c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:36.731068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb4a1a-bef2-444c-9dc1-d7f0361aa0de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-eccb4a1a-bef2-444c-9dc1-d7f0361aa0de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:36.738500 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:36.739293 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:31:36.740043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:36.742534 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 13/29] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:31:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:31:36.743498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb4a1a-bef2-444c-9dc1-d7f0361aa0de None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:36.846059 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 17/30] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:31:36.846966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0012e8-ace8-46aa-a12b-5b6301fd7c6b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:36.947277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb4a1a-bef2-444c-9dc1-d7f0361aa0de None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:36.963850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:37.012852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fdda9-e707-472e-bf1f-d85911ef3d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:37.013853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fdda9-e707-472e-bf1f-d85911ef3d11 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.014349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fdda9-e707-472e-bf1f-d85911ef3d11 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.014949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fdda9-e707-472e-bf1f-d85911ef3d11 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.015913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fdda9-e707-472e-bf1f-d85911ef3d11 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:37.017699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fdda9-e707-472e-bf1f-d85911ef3d11 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:37.018483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0012e8-ace8-46aa-a12b-5b6301fd7c6b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:37.034294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:37.091196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5451163-257b-4d36-8bb4-03f18ac8f7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:37.091793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5451163-257b-4d36-8bb4-03f18ac8f7b1 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.092068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5451163-257b-4d36-8bb4-03f18ac8f7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.092341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5451163-257b-4d36-8bb4-03f18ac8f7b1 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.092999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5451163-257b-4d36-8bb4-03f18ac8f7b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:37.093319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5451163-257b-4d36-8bb4-03f18ac8f7b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:37.094832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5451163-257b-4d36-8bb4-03f18ac8f7b1 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:37.116434 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 18/31] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:31:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:37.117322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5451163-257b-4d36-8bb4-03f18ac8f7b1 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:37.164472 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 14/32] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Oct 15 20:31:36 2022] POST /identity/v3/roles => generated 225 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:31:37.165826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fdda9-e707-472e-bf1f-d85911ef3d11 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:37.345333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5451163-257b-4d36-8bb4-03f18ac8f7b1 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:37.392719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fdda9-e707-472e-bf1f-d85911ef3d11 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:37.393424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:37.418242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bc2c7-7f91-4ed7-ac0a-5ef220819f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:37.418873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bc2c7-7f91-4ed7-ac0a-5ef220819f66 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.419153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bc2c7-7f91-4ed7-ac0a-5ef220819f66 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.419418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bc2c7-7f91-4ed7-ac0a-5ef220819f66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.433147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bc2c7-7f91-4ed7-ac0a-5ef220819f66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:37.435225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bc2c7-7f91-4ed7-ac0a-5ef220819f66 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:37.445216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4adbc-d1a5-4d0d-a186-a0c90694f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:37.446099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4adbc-d1a5-4d0d-a186-a0c90694f04e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.446466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4adbc-d1a5-4d0d-a186-a0c90694f04e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.446810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4adbc-d1a5-4d0d-a186-a0c90694f04e None admin] PATH_INFO: `/v3/users/f9ecef8e2dba4f429d37c67479eb320e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.448554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4adbc-d1a5-4d0d-a186-a0c90694f04e None admin] RBAC: Authorizing `identity:get_user(user_id=f9ecef8e2dba4f429d37c67479eb320e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:37.451002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4adbc-d1a5-4d0d-a186-a0c90694f04e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:37.452928 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 15/33] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:37 2022] GET /identity/v3/users/f9ecef8e2dba4f429d37c67479eb320e => generated 247 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:37.453835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4adbc-d1a5-4d0d-a186-a0c90694f04e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-250bc2c7-7f91-4ed7-ac0a-5ef220819f66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:31:37.457936 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:37.458775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:31:37.459555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:31:37.459555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:31:37.459555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:31:37.461447 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 19/34] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:31:37.462914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c6e60-07bd-4ec9-a184-6fe8bd36d2a7 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.463360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c6e60-07bd-4ec9-a184-6fe8bd36d2a7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.463791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c6e60-07bd-4ec9-a184-6fe8bd36d2a7 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.465971 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 20/35] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:37.470789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c6e60-07bd-4ec9-a184-6fe8bd36d2a7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:37.530873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:37.569091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4adbc-d1a5-4d0d-a186-a0c90694f04e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:37.590624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c6e60-07bd-4ec9-a184-6fe8bd36d2a7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:37.604353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12775726-aa05-48d4-93ff-1ae2ccb45d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:37.604899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12775726-aa05-48d4-93ff-1ae2ccb45d40 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.605734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12775726-aa05-48d4-93ff-1ae2ccb45d40 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.606007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12775726-aa05-48d4-93ff-1ae2ccb45d40 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.606662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12775726-aa05-48d4-93ff-1ae2ccb45d40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:37.608032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12775726-aa05-48d4-93ff-1ae2ccb45d40 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:37.622637 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 16/36] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Oct 15 20:31:37 2022] POST /identity/v3/roles => generated 219 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:31:37.623433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12775726-aa05-48d4-93ff-1ae2ccb45d40 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:37.628706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77762d1-2582-4b99-955b-c70f88393aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:37.629497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77762d1-2582-4b99-955b-c70f88393aca None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.629840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77762d1-2582-4b99-955b-c70f88393aca None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.630181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77762d1-2582-4b99-955b-c70f88393aca None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.631457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77762d1-2582-4b99-955b-c70f88393aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:37.633067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77762d1-2582-4b99-955b-c70f88393aca None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:37.634877 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 21/37] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:31:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:37.636293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad66c3-80fc-454b-8c24-aea2ab044637 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.637198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad66c3-80fc-454b-8c24-aea2ab044637 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.637545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad66c3-80fc-454b-8c24-aea2ab044637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.658344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ecad66c3-80fc-454b-8c24-aea2ab044637 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:37.659620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ecad66c3-80fc-454b-8c24-aea2ab044637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:37.698898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:37.736813 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 22/38] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:37.737947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e1cf0-8085-4a89-8396-759183cd7e74 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.738232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e1cf0-8085-4a89-8396-759183cd7e74 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.738499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e1cf0-8085-4a89-8396-759183cd7e74 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.739953 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 23/39] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:37.740513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e1cf0-8085-4a89-8396-759183cd7e74 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:37.753063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12775726-aa05-48d4-93ff-1ae2ccb45d40 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:37.785513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:37.795650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676aba7-f882-4f87-b481-f5f13dda5704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:37.796555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676aba7-f882-4f87-b481-f5f13dda5704 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.797218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676aba7-f882-4f87-b481-f5f13dda5704 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.797781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676aba7-f882-4f87-b481-f5f13dda5704 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.798649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676aba7-f882-4f87-b481-f5f13dda5704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:37.799953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676aba7-f882-4f87-b481-f5f13dda5704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:37.801510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676aba7-f882-4f87-b481-f5f13dda5704 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:37.821870 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 17/40] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:37.823196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba096b-0d21-4330-a6cb-7305c83f6157 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.823551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba096b-0d21-4330-a6cb-7305c83f6157 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.823901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba096b-0d21-4330-a6cb-7305c83f6157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.845578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3dba096b-0d21-4330-a6cb-7305c83f6157 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:37.847208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3dba096b-0d21-4330-a6cb-7305c83f6157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:37.885409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e1cf0-8085-4a89-8396-759183cd7e74 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:37.927846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb58651-40af-4361-bf7c-c4c8d062a919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:37.928622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb58651-40af-4361-bf7c-c4c8d062a919 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:37.928999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb58651-40af-4361-bf7c-c4c8d062a919 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:37.929398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb58651-40af-4361-bf7c-c4c8d062a919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:37.930222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb58651-40af-4361-bf7c-c4c8d062a919 None admin] RBAC: Adding query filter params (role.id=fab66a1a6ba64c9583182c78978b218e, scope.domain.id=default, user.id=f9ecef8e2dba4f429d37c67479eb320e) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:37.930685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb58651-40af-4361-bf7c-c4c8d062a919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:37.932185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb58651-40af-4361-bf7c-c4c8d062a919 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:37.940963 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 24/41] 10.222.0.16 () {56 vars in 1322 bytes} [Sat Oct 15 20:31:37 2022] GET /identity/v3/role_assignments?user.id=f9ecef8e2dba4f429d37c67479eb320e&scope.domain.id=default&role.id=fab66a1a6ba64c9583182c78978b218e => generated 530 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:37.942031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb58651-40af-4361-bf7c-c4c8d062a919 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:37.943567 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 18/42] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:37.994573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:38.074960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb58651-40af-4361-bf7c-c4c8d062a919 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:38.114799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba096b-0d21-4330-a6cb-7305c83f6157 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:38.115378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644799df-8d69-4c95-884e-89a857a809e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:38.116146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644799df-8d69-4c95-884e-89a857a809e3 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:38.116557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644799df-8d69-4c95-884e-89a857a809e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:38.116994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644799df-8d69-4c95-884e-89a857a809e3 None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:38.118203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644799df-8d69-4c95-884e-89a857a809e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:38.121701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644799df-8d69-4c95-884e-89a857a809e3 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:38.162589 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 25/43] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:31:37 2022] POST /identity/v3/projects => generated 301 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:31:38.251348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba096b-0d21-4330-a6cb-7305c83f6157 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:38.274965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:38.287621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644799df-8d69-4c95-884e-89a857a809e3 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:38.298621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8c5a1-9016-4645-8be8-b28f192c81e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:38.299493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8c5a1-9016-4645-8be8-b28f192c81e5 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:38.299929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8c5a1-9016-4645-8be8-b28f192c81e5 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:38.300336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8c5a1-9016-4645-8be8-b28f192c81e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:38.301680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8c5a1-9016-4645-8be8-b28f192c81e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:38.303153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8c5a1-9016-4645-8be8-b28f192c81e5 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:38.304946 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 19/44] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:31:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:31:38.310635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8c5a1-9016-4645-8be8-b28f192c81e5 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:38.398548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:38.416148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644799df-8d69-4c95-884e-89a857a809e3 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:38.434490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8c5a1-9016-4645-8be8-b28f192c81e5 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:38.455892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227de875-9e09-4b12-8903-ef0d2b60ab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:38.456493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227de875-9e09-4b12-8903-ef0d2b60ab1b None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:38.456778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227de875-9e09-4b12-8903-ef0d2b60ab1b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:38.457061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227de875-9e09-4b12-8903-ef0d2b60ab1b None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:38.457802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227de875-9e09-4b12-8903-ef0d2b60ab1b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:38.459205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227de875-9e09-4b12-8903-ef0d2b60ab1b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:38.470183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747dfb1-61ee-4a5c-b925-733caecd8825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:38.470861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747dfb1-61ee-4a5c-b925-733caecd8825 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:38.471134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747dfb1-61ee-4a5c-b925-733caecd8825 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:38.471401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747dfb1-61ee-4a5c-b925-733caecd8825 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:38.472032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747dfb1-61ee-4a5c-b925-733caecd8825 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:38.473498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747dfb1-61ee-4a5c-b925-733caecd8825 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:38.474798 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 26/45] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Oct 15 20:31:38 2022] POST /identity/v3/roles => generated 223 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:31:38.507235 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 20/46] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:31:38 2022] POST /identity/v3/projects => generated 312 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:31:40.912979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd40d-5670-47b4-90ec-61d8618e0c2a None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:40.913540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd40d-5670-47b4-90ec-61d8618e0c2a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:40.913629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd40d-5670-47b4-90ec-61d8618e0c2a None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:40.914904 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 27/47] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:40.920670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc2b08-9220-4a63-a782-973fdc0ae913 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:40.921203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc2b08-9220-4a63-a782-973fdc0ae913 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:40.921306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc2b08-9220-4a63-a782-973fdc0ae913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:40.939356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5fbc2b08-9220-4a63-a782-973fdc0ae913 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:40.940562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5fbc2b08-9220-4a63-a782-973fdc0ae913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:40.970076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:41.008577 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 21/48] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:41.329943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bd40d-5670-47b4-90ec-61d8618e0c2a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:41.430129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bd40d-5670-47b4-90ec-61d8618e0c2a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:41.462955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a624b-4a75-424f-b441-f3abba7c073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:41.463511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a624b-4a75-424f-b441-f3abba7c073d None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:41.463786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a624b-4a75-424f-b441-f3abba7c073d None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:41.464054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a624b-4a75-424f-b441-f3abba7c073d None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:41.465309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a624b-4a75-424f-b441-f3abba7c073d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:41.466776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a624b-4a75-424f-b441-f3abba7c073d None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:41.468321 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 28/49] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:31:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:41.473801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc2b08-9220-4a63-a782-973fdc0ae913 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:41.569056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc2b08-9220-4a63-a782-973fdc0ae913 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:41.602176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2cb4e-f0b9-4f35-bc4a-58e7ecf336e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:41.602766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2cb4e-f0b9-4f35-bc4a-58e7ecf336e1 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:41.603041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2cb4e-f0b9-4f35-bc4a-58e7ecf336e1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:41.603310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2cb4e-f0b9-4f35-bc4a-58e7ecf336e1 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:41.603951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2cb4e-f0b9-4f35-bc4a-58e7ecf336e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:41.605454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2cb4e-f0b9-4f35-bc4a-58e7ecf336e1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:41.630243 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 22/50] 10.222.0.16 () {60 vars in 1140 bytes} [Sat Oct 15 20:31:41 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:31:44.000120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78389f-0ce5-4d61-b3d2-b1c816eb9b7d None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:44.000652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78389f-0ce5-4d61-b3d2-b1c816eb9b7d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:44.000652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78389f-0ce5-4d61-b3d2-b1c816eb9b7d None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:44.002028 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 29/51] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:44.007252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e70d7-bcc7-4bb7-89f4-11c65b2e486d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:44.007615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e70d7-bcc7-4bb7-89f4-11c65b2e486d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:44.007793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e70d7-bcc7-4bb7-89f4-11c65b2e486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:44.026354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ac6e70d7-bcc7-4bb7-89f4-11c65b2e486d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:44.027681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ac6e70d7-bcc7-4bb7-89f4-11c65b2e486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:44.041289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:44.092952 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 23/52] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:44.417821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78389f-0ce5-4d61-b3d2-b1c816eb9b7d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:44.436604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:44.517440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78389f-0ce5-4d61-b3d2-b1c816eb9b7d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:44.550328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd69db-b8d8-4ccf-8231-f5e2c42253b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:44.550934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd69db-b8d8-4ccf-8231-f5e2c42253b7 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:44.551205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd69db-b8d8-4ccf-8231-f5e2c42253b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:44.551468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd69db-b8d8-4ccf-8231-f5e2c42253b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:44.552541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd69db-b8d8-4ccf-8231-f5e2c42253b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:44.554010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd69db-b8d8-4ccf-8231-f5e2c42253b7 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:44.555529 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 30/53] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:31:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:44.561609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e70d7-bcc7-4bb7-89f4-11c65b2e486d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:44.656446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e70d7-bcc7-4bb7-89f4-11c65b2e486d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:44.689286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69964bd7-86b1-47cb-9279-76524d409b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:44.689818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69964bd7-86b1-47cb-9279-76524d409b97 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:44.690099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69964bd7-86b1-47cb-9279-76524d409b97 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:44.690367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69964bd7-86b1-47cb-9279-76524d409b97 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:44.691005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69964bd7-86b1-47cb-9279-76524d409b97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:44.692425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69964bd7-86b1-47cb-9279-76524d409b97 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:44.718791 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 24/54] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:31:44 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:31:48.348094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12743a5-0dac-4a98-9c58-1feaa0e54a80 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:48.349121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12743a5-0dac-4a98-9c58-1feaa0e54a80 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:48.349605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12743a5-0dac-4a98-9c58-1feaa0e54a80 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:48.351516 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 31/55] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:48.356990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f947-437a-4504-ae97-09f8fc133176 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:48.357428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f947-437a-4504-ae97-09f8fc133176 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:48.357601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f947-437a-4504-ae97-09f8fc133176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:48.373896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:48.385413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4c55f947-437a-4504-ae97-09f8fc133176 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:48.387212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4c55f947-437a-4504-ae97-09f8fc133176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:48.472648 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 25/56] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:48.478570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b865a-8de3-4a82-897a-8f6848384a44 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:48.479091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b865a-8de3-4a82-897a-8f6848384a44 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:48.479648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b865a-8de3-4a82-897a-8f6848384a44 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:48.481221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 32/57] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:48.486526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1e652-58ca-409e-9787-b245abe11ee6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:48.487046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1e652-58ca-409e-9787-b245abe11ee6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:48.487513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1e652-58ca-409e-9787-b245abe11ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:48.510225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0df1e652-58ca-409e-9787-b245abe11ee6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:48.512446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0df1e652-58ca-409e-9787-b245abe11ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:48.585123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a8fe0-c010-4673-bbff-540ad01f10f3 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:48.585735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a8fe0-c010-4673-bbff-540ad01f10f3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:48.586205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a8fe0-c010-4673-bbff-540ad01f10f3 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:48.587739 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 33/58] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:31:48.593137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab865-faa4-4333-999b-348163668ede None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:48.593650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab865-faa4-4333-999b-348163668ede None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:48.594114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab865-faa4-4333-999b-348163668ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:48.614548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-19cab865-faa4-4333-999b-348163668ede None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:48.616170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-19cab865-faa4-4333-999b-348163668ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:48.622482 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 26/59] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:48.688287 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 34/60] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:48.746738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63584e17-62de-4ea2-9f22-819189b98cca None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:48.747339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63584e17-62de-4ea2-9f22-819189b98cca None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:48.747797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63584e17-62de-4ea2-9f22-819189b98cca None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:48.749377 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 27/61] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:48.754968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f33d69-3dbd-49d7-8843-083db0fed2ff None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:48.755255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f33d69-3dbd-49d7-8843-083db0fed2ff None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:48.755527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f33d69-3dbd-49d7-8843-083db0fed2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:48.779020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c8f33d69-3dbd-49d7-8843-083db0fed2ff None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:48.780206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c8f33d69-3dbd-49d7-8843-083db0fed2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:48.793395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:48.820199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63584e17-62de-4ea2-9f22-819189b98cca None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:48.882709 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 35/62] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:48.893956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:48.935349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63584e17-62de-4ea2-9f22-819189b98cca None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:48.971233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e006e5d-20ba-43f8-8430-a54b00a415e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:48.972185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e006e5d-20ba-43f8-8430-a54b00a415e7 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:48.972677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e006e5d-20ba-43f8-8430-a54b00a415e7 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:48.973167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e006e5d-20ba-43f8-8430-a54b00a415e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:48.984417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e006e5d-20ba-43f8-8430-a54b00a415e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-8e006e5d-20ba-43f8-8430-a54b00a415e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:48.989722 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:48.990530 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:31:48.991278 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:48.993509 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 28/63] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:31:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:31:49.000107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f33d69-3dbd-49d7-8843-083db0fed2ff None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.052736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e006e5d-20ba-43f8-8430-a54b00a415e7 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.156433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e006e5d-20ba-43f8-8430-a54b00a415e7 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.186790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f33d69-3dbd-49d7-8843-083db0fed2ff None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.189490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:49.206772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72e6e0-66be-43f8-b440-c8c2fd63b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:49.207705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72e6e0-66be-43f8-b440-c8c2fd63b01b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.208175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72e6e0-66be-43f8-b440-c8c2fd63b01b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.208634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72e6e0-66be-43f8-b440-c8c2fd63b01b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.221470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c72e6e0-66be-43f8-b440-c8c2fd63b01b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-4c72e6e0-66be-43f8-b440-c8c2fd63b01b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:49.226562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:49.227351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:31:49.228116 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:49.231151 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 29/64] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:31:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:31:49.239852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72e6e0-66be-43f8-b440-c8c2fd63b01b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.266220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc304e-fe08-4ef1-a629-a53d5471379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:49.266744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc304e-fe08-4ef1-a629-a53d5471379e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.267015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc304e-fe08-4ef1-a629-a53d5471379e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.267280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc304e-fe08-4ef1-a629-a53d5471379e None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.268198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc304e-fe08-4ef1-a629-a53d5471379e None admin] RBAC: Adding query filter params (name=member) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:49.268497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc304e-fe08-4ef1-a629-a53d5471379e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:49.269913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc304e-fe08-4ef1-a629-a53d5471379e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:49.286436 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 36/65] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:31:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:49.287259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc304e-fe08-4ef1-a629-a53d5471379e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.300786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:49.368807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72e6e0-66be-43f8-b440-c8c2fd63b01b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.393029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc304e-fe08-4ef1-a629-a53d5471379e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.404572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8f291-6386-46d5-9c22-6ab4efa18629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:49.405190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8f291-6386-46d5-9c22-6ab4efa18629 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.405464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8f291-6386-46d5-9c22-6ab4efa18629 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.405728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8f291-6386-46d5-9c22-6ab4efa18629 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.406374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8f291-6386-46d5-9c22-6ab4efa18629 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:49.406668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8f291-6386-46d5-9c22-6ab4efa18629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:49.408039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8f291-6386-46d5-9c22-6ab4efa18629 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:49.419578 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 30/66] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:31:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:49.420357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8f291-6386-46d5-9c22-6ab4efa18629 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.426722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3a7fe-3e4b-4274-be0e-f6fa5543ea73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:49.427338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3a7fe-3e4b-4274-be0e-f6fa5543ea73 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.427609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3a7fe-3e4b-4274-be0e-f6fa5543ea73 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.427874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3a7fe-3e4b-4274-be0e-f6fa5543ea73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.437762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3a7fe-3e4b-4274-be0e-f6fa5543ea73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-baf3a7fe-3e4b-4274-be0e-f6fa5543ea73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:49.443545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:49.444432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 37/67] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:31:49.450242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3a7fe-3e4b-4274-be0e-f6fa5543ea73 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.469365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:49.514261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:49.522051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8f291-6386-46d5-9c22-6ab4efa18629 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.548849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3a7fe-3e4b-4274-be0e-f6fa5543ea73 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.554817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3a147-48f8-4367-84ea-af5542070618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:49.555437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3a147-48f8-4367-84ea-af5542070618 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.555709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3a147-48f8-4367-84ea-af5542070618 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.555973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3a147-48f8-4367-84ea-af5542070618 None admin] PATH_INFO: `/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.570290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3a147-48f8-4367-84ea-af5542070618 None admin] RBAC: Authorizing `identity:get_user(user_id=22b1d8706e1e41dbb9984cbc4c9d7be5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:49.571766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3a147-48f8-4367-84ea-af5542070618 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:49.574077 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 31/68] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:49 2022] GET /identity/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:49.575007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a025fbb-d29c-4174-bb74-8905b8579733 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.575287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a025fbb-d29c-4174-bb74-8905b8579733 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.575554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a025fbb-d29c-4174-bb74-8905b8579733 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.576869 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 32/69] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:49.577440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a025fbb-d29c-4174-bb74-8905b8579733 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.586053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec46c90-8b4f-435e-a30e-e5ec54377ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:49.586680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec46c90-8b4f-435e-a30e-e5ec54377ff1 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.586955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec46c90-8b4f-435e-a30e-e5ec54377ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.587221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec46c90-8b4f-435e-a30e-e5ec54377ff1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.597932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec46c90-8b4f-435e-a30e-e5ec54377ff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-8ec46c90-8b4f-435e-a30e-e5ec54377ff1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:49.602775 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:49.603592 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:31:49.604336 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 38/70] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:31:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:31:49.605824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec46c90-8b4f-435e-a30e-e5ec54377ff1 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.682087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a025fbb-d29c-4174-bb74-8905b8579733 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.689288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:49.703953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec46c90-8b4f-435e-a30e-e5ec54377ff1 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.716265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9c203-18c5-4baa-8312-153a1d175350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:49.716878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9c203-18c5-4baa-8312-153a1d175350 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.717236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9c203-18c5-4baa-8312-153a1d175350 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.717503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9c203-18c5-4baa-8312-153a1d175350 None admin] PATH_INFO: `/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.718602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9c203-18c5-4baa-8312-153a1d175350 None admin] RBAC: Authorizing `identity:get_user(user_id=22b1d8706e1e41dbb9984cbc4c9d7be5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:49.720032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9c203-18c5-4baa-8312-153a1d175350 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:49.721692 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 33/71] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:49 2022] GET /identity/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:49.722446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9c203-18c5-4baa-8312-153a1d175350 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.732713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:49.742051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7097dd8-7ec4-4a3f-8318-13d8b2f7ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:49.742646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7097dd8-7ec4-4a3f-8318-13d8b2f7ea70 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.742921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7097dd8-7ec4-4a3f-8318-13d8b2f7ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.743187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7097dd8-7ec4-4a3f-8318-13d8b2f7ea70 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.743838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7097dd8-7ec4-4a3f-8318-13d8b2f7ea70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:49.744387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7097dd8-7ec4-4a3f-8318-13d8b2f7ea70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:49.746157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7097dd8-7ec4-4a3f-8318-13d8b2f7ea70 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:49.754172 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 39/72] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:31:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:49.755300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7a728-b7db-4a41-99b8-4c019abf5c65 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.755586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7a728-b7db-4a41-99b8-4c019abf5c65 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.755856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7a728-b7db-4a41-99b8-4c019abf5c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.773491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-42a7a728-b7db-4a41-99b8-4c019abf5c65 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:49.774617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-42a7a728-b7db-4a41-99b8-4c019abf5c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:49.821752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9c203-18c5-4baa-8312-153a1d175350 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.842470 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 40/73] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:49.843295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7a728-b7db-4a41-99b8-4c019abf5c65 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.856443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e952b7b-3f76-496e-8236-1a5544d2f452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:49.857081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e952b7b-3f76-496e-8236-1a5544d2f452 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:49.857358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e952b7b-3f76-496e-8236-1a5544d2f452 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:49.857622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e952b7b-3f76-496e-8236-1a5544d2f452 None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:49.858734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e952b7b-3f76-496e-8236-1a5544d2f452 None admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:49.860152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e952b7b-3f76-496e-8236-1a5544d2f452 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:49.861799 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 34/74] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:31:49 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:49.862545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e952b7b-3f76-496e-8236-1a5544d2f452 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:49.910399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:49.969696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e952b7b-3f76-496e-8236-1a5544d2f452 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.970315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7a728-b7db-4a41-99b8-4c019abf5c65 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:49.987779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:50.006494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131be48-fd23-4419-bbb1-bafa77f59984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:50.006648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c480-4e18-4546-ad09-77a583b3fb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:50.007096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131be48-fd23-4419-bbb1-bafa77f59984 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:50.007196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c480-4e18-4546-ad09-77a583b3fb82 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:50.007365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131be48-fd23-4419-bbb1-bafa77f59984 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:50.007457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c480-4e18-4546-ad09-77a583b3fb82 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:50.007631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131be48-fd23-4419-bbb1-bafa77f59984 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:50.007722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c480-4e18-4546-ad09-77a583b3fb82 None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:50.008315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131be48-fd23-4419-bbb1-bafa77f59984 None admin] RBAC: Adding query filter params (name=member) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:50.008615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131be48-fd23-4419-bbb1-bafa77f59984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:50.008764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61c480-4e18-4546-ad09-77a583b3fb82 None admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:50.010034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131be48-fd23-4419-bbb1-bafa77f59984 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:50.010222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61c480-4e18-4546-ad09-77a583b3fb82 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:50.017004 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 35/75] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:31:49 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:50.017004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c480-4e18-4546-ad09-77a583b3fb82 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:50.024558 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 41/76] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:31:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:50.032902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131be48-fd23-4419-bbb1-bafa77f59984 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:50.128208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c480-4e18-4546-ad09-77a583b3fb82 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:50.138882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131be48-fd23-4419-bbb1-bafa77f59984 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:50.139896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:50.165293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46291759-e601-4a8a-a0ab-dc82d4804998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:50.165909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46291759-e601-4a8a-a0ab-dc82d4804998 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:50.166181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46291759-e601-4a8a-a0ab-dc82d4804998 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:50.166446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46291759-e601-4a8a-a0ab-dc82d4804998 None admin] PATH_INFO: `/v3/users/f9ecef8e2dba4f429d37c67479eb320e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:50.167682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46291759-e601-4a8a-a0ab-dc82d4804998 None admin] RBAC: Authorizing `identity:get_user(user_id=f9ecef8e2dba4f429d37c67479eb320e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:50.169157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46291759-e601-4a8a-a0ab-dc82d4804998 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:50.170730 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 36/77] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:50 2022] GET /identity/v3/users/f9ecef8e2dba4f429d37c67479eb320e => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:50.171469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46291759-e601-4a8a-a0ab-dc82d4804998 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:50.175174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179b1fc-d645-4ca2-ad1f-73eff4a72c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:50.175825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179b1fc-d645-4ca2-ad1f-73eff4a72c73 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:50.176102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179b1fc-d645-4ca2-ad1f-73eff4a72c73 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:50.176377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179b1fc-d645-4ca2-ad1f-73eff4a72c73 None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/users/22b1d8706e1e41dbb9984cbc4c9d7be5/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:50.178299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179b1fc-d645-4ca2-ad1f-73eff4a72c73 None admin] RBAC: Authorizing `identity:create_grant(project_id=abb9dbb5eb664a45a354501eb93cb0cd, user_id=22b1d8706e1e41dbb9984cbc4c9d7be5, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:50.179758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179b1fc-d645-4ca2-ad1f-73eff4a72c73 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:50.193065 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 42/78] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:31:50 2022] PUT /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/users/22b1d8706e1e41dbb9984cbc4c9d7be5/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:31:50.194154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179b1fc-d645-4ca2-ad1f-73eff4a72c73 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:50.245795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:50.287714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46291759-e601-4a8a-a0ab-dc82d4804998 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:50.314409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179b1fc-d645-4ca2-ad1f-73eff4a72c73 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:50.326258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c2034-e379-43b1-a528-dbedd088eb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:50.326926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c2034-e379-43b1-a528-dbedd088eb87 None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:50.327201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c2034-e379-43b1-a528-dbedd088eb87 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:50.327467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c2034-e379-43b1-a528-dbedd088eb87 None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/users/22b1d8706e1e41dbb9984cbc4c9d7be5/roles/25b2c5582f194703afe45b5d8cc79171` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:50.329726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c2034-e379-43b1-a528-dbedd088eb87 None admin] RBAC: Authorizing `identity:create_grant(project_id=abb9dbb5eb664a45a354501eb93cb0cd, user_id=22b1d8706e1e41dbb9984cbc4c9d7be5, role_id=25b2c5582f194703afe45b5d8cc79171)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:50.331193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c2034-e379-43b1-a528-dbedd088eb87 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:50.350132 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 37/79] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:31:50 2022] PUT /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/users/22b1d8706e1e41dbb9984cbc4c9d7be5/roles/25b2c5582f194703afe45b5d8cc79171 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:31:50.351445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c2034-e379-43b1-a528-dbedd088eb87 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:50.366388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d89181-3128-4458-a4e0-66e2c263e51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:50.367103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d89181-3128-4458-a4e0-66e2c263e51f None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:50.367454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d89181-3128-4458-a4e0-66e2c263e51f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:50.367918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d89181-3128-4458-a4e0-66e2c263e51f None admin] PATH_INFO: `/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:50.370045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d89181-3128-4458-a4e0-66e2c263e51f None admin] RBAC: Authorizing `identity:get_user(user_id=22b1d8706e1e41dbb9984cbc4c9d7be5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:50.371726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d89181-3128-4458-a4e0-66e2c263e51f None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:50.373571 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 43/80] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:50 2022] GET /identity/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:50.379360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d89181-3128-4458-a4e0-66e2c263e51f None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:50.423021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:50.503424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d89181-3128-4458-a4e0-66e2c263e51f None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:50.514642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:50.515343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c2034-e379-43b1-a528-dbedd088eb87 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:50.544881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2199c59-a0e3-45ad-90ec-4b33de4d2403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:50.545583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2199c59-a0e3-45ad-90ec-4b33de4d2403 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:50.545930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2199c59-a0e3-45ad-90ec-4b33de4d2403 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:50.546272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2199c59-a0e3-45ad-90ec-4b33de4d2403 None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:50.547627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2199c59-a0e3-45ad-90ec-4b33de4d2403 None admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:50.549346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2199c59-a0e3-45ad-90ec-4b33de4d2403 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:50.551305 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 44/81] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:31:50 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:50.552020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2199c59-a0e3-45ad-90ec-4b33de4d2403 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:50.568542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772dde4-c166-4180-a68e-55027eb43653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:50.569791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772dde4-c166-4180-a68e-55027eb43653 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:50.570276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772dde4-c166-4180-a68e-55027eb43653 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:50.571281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772dde4-c166-4180-a68e-55027eb43653 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:50.572940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772dde4-c166-4180-a68e-55027eb43653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:50.575905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772dde4-c166-4180-a68e-55027eb43653 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:50.577762 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 38/82] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:31:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:50.578713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772dde4-c166-4180-a68e-55027eb43653 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:50.698133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2199c59-a0e3-45ad-90ec-4b33de4d2403 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:50.700008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772dde4-c166-4180-a68e-55027eb43653 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:50.710155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:50.747400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c0731-f408-4e6a-ac93-79b0301f357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:50.748080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c0731-f408-4e6a-ac93-79b0301f357a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:50.748367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c0731-f408-4e6a-ac93-79b0301f357a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:50.748582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c82a9a-1069-4bb1-b554-b7a549e53280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:50.748689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c0731-f408-4e6a-ac93-79b0301f357a None admin] PATH_INFO: `/v3/projects/325db256cf5f41a1baf8f901ccdf2716` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:50.749585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c82a9a-1069-4bb1-b554-b7a549e53280 None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:50.749781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c0731-f408-4e6a-ac93-79b0301f357a None admin] RBAC: Authorizing `identity:get_project(project_id=325db256cf5f41a1baf8f901ccdf2716)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:50.749873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c82a9a-1069-4bb1-b554-b7a549e53280 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:50.750125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c82a9a-1069-4bb1-b554-b7a549e53280 None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/users/f9ecef8e2dba4f429d37c67479eb320e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:50.751246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c0731-f408-4e6a-ac93-79b0301f357a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:50.752052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c82a9a-1069-4bb1-b554-b7a549e53280 None admin] RBAC: Authorizing `identity:create_grant(project_id=abb9dbb5eb664a45a354501eb93cb0cd, user_id=f9ecef8e2dba4f429d37c67479eb320e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:50.752987 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 45/83] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:31:50 2022] GET /identity/v3/projects/325db256cf5f41a1baf8f901ccdf2716 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:50.753529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c82a9a-1069-4bb1-b554-b7a549e53280 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:50.753755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c0731-f408-4e6a-ac93-79b0301f357a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:50.768585 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 39/84] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:31:50 2022] PUT /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/users/f9ecef8e2dba4f429d37c67479eb320e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:31:50.769520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c82a9a-1069-4bb1-b554-b7a549e53280 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:50.835443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:50.960294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c0731-f408-4e6a-ac93-79b0301f357a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:50.962409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c82a9a-1069-4bb1-b554-b7a549e53280 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:51.001057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346627f8-6fcd-4826-b571-aa49bcb82812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:51.001739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346627f8-6fcd-4826-b571-aa49bcb82812 None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:51.002019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346627f8-6fcd-4826-b571-aa49bcb82812 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:51.002290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346627f8-6fcd-4826-b571-aa49bcb82812 None admin] PATH_INFO: `/v3/projects/325db256cf5f41a1baf8f901ccdf2716/users/22b1d8706e1e41dbb9984cbc4c9d7be5/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:51.004287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346627f8-6fcd-4826-b571-aa49bcb82812 None admin] RBAC: Authorizing `identity:create_grant(project_id=325db256cf5f41a1baf8f901ccdf2716, user_id=22b1d8706e1e41dbb9984cbc4c9d7be5, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:51.007936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346627f8-6fcd-4826-b571-aa49bcb82812 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:51.024311 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 40/85] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:31:50 2022] PUT /identity/v3/projects/325db256cf5f41a1baf8f901ccdf2716/users/22b1d8706e1e41dbb9984cbc4c9d7be5/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:31:51.033814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb19fc-8fc5-448b-abe0-9f77ea614e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:51.034694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb19fc-8fc5-448b-abe0-9f77ea614e26 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:51.034909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb19fc-8fc5-448b-abe0-9f77ea614e26 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:51.035215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb19fc-8fc5-448b-abe0-9f77ea614e26 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:51.036021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb19fc-8fc5-448b-abe0-9f77ea614e26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:51.038317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb19fc-8fc5-448b-abe0-9f77ea614e26 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:51.082224 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 46/86] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:31:50 2022] POST /identity/v3/users => generated 289 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:31:52.846144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c4e9-5e16-443d-b8f4-4a3030d48f75 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:52.847149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c4e9-5e16-443d-b8f4-4a3030d48f75 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:52.847753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c4e9-5e16-443d-b8f4-4a3030d48f75 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:52.849616 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 41/87] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:52.856248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19f66b-19cf-4311-a173-488efd1f7253 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:52.857206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19f66b-19cf-4311-a173-488efd1f7253 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:52.857693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19f66b-19cf-4311-a173-488efd1f7253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:52.876931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5d19f66b-19cf-4311-a173-488efd1f7253 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:52.878463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5d19f66b-19cf-4311-a173-488efd1f7253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:52.947711 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 47/88] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:53.627975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272c4e9-5e16-443d-b8f4-4a3030d48f75 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:53.630832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a8ff8-d3c1-4e41-aa82-fbbe4d628490 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:53.631335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a8ff8-d3c1-4e41-aa82-fbbe4d628490 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:53.631795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a8ff8-d3c1-4e41-aa82-fbbe4d628490 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:53.633353 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 48/89] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:53.639531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622517f-21e2-4576-93a1-fb5356e6e0fe None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:53.640094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622517f-21e2-4576-93a1-fb5356e6e0fe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:53.640568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622517f-21e2-4576-93a1-fb5356e6e0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:53.661651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:53.669949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3622517f-21e2-4576-93a1-fb5356e6e0fe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:53.671493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3622517f-21e2-4576-93a1-fb5356e6e0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:53.709571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:53.808719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272c4e9-5e16-443d-b8f4-4a3030d48f75 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:53.822253 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 49/90] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:31:53.870794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186cc4-0121-4fd3-b394-fab7601cabd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:53.871406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186cc4-0121-4fd3-b394-fab7601cabd7 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:53.871681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186cc4-0121-4fd3-b394-fab7601cabd7 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:53.871950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186cc4-0121-4fd3-b394-fab7601cabd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:53.893925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9186cc4-0121-4fd3-b394-fab7601cabd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-c9186cc4-0121-4fd3-b394-fab7601cabd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:53.898938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:53.899815 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:53.900959 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 42/91] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:31:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:31:53.907052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622517f-21e2-4576-93a1-fb5356e6e0fe None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:54.040906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622517f-21e2-4576-93a1-fb5356e6e0fe None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:54.082280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e3fa8-2036-44ef-8f74-4c32c0db53b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:54.083332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e3fa8-2036-44ef-8f74-4c32c0db53b9 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:54.083836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e3fa8-2036-44ef-8f74-4c32c0db53b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:54.084332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e3fa8-2036-44ef-8f74-4c32c0db53b9 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:54.085251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e3fa8-2036-44ef-8f74-4c32c0db53b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:54.085755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e3fa8-2036-44ef-8f74-4c32c0db53b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:54.087528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e3fa8-2036-44ef-8f74-4c32c0db53b9 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:54.099935 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 50/92] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:31:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:54.106285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186cc4-0121-4fd3-b394-fab7601cabd7 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:54.242440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:54.259090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186cc4-0121-4fd3-b394-fab7601cabd7 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:54.301011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472653c8-cfe0-4f6b-a61f-200933d9b9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:54.301668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472653c8-cfe0-4f6b-a61f-200933d9b9e3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:54.301945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472653c8-cfe0-4f6b-a61f-200933d9b9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:54.302211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472653c8-cfe0-4f6b-a61f-200933d9b9e3 None admin] PATH_INFO: `/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:54.303333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472653c8-cfe0-4f6b-a61f-200933d9b9e3 None admin] RBAC: Authorizing `identity:get_user(user_id=22b1d8706e1e41dbb9984cbc4c9d7be5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:54.304781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472653c8-cfe0-4f6b-a61f-200933d9b9e3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:54.306396 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 43/93] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:54 2022] GET /identity/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:54.312179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e3fa8-2036-44ef-8f74-4c32c0db53b9 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:54.343589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:54.412484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472653c8-cfe0-4f6b-a61f-200933d9b9e3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:54.458434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e3fa8-2036-44ef-8f74-4c32c0db53b9 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:54.520672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c3a3e-4fe1-40a2-8357-ca409ba05290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:54.522533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c3a3e-4fe1-40a2-8357-ca409ba05290 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:54.522996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c3a3e-4fe1-40a2-8357-ca409ba05290 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:54.524401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c3a3e-4fe1-40a2-8357-ca409ba05290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:54.590995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472653c8-cfe0-4f6b-a61f-200933d9b9e3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:54.645718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac524392-9076-492d-abb1-ce01d87dc1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:54.646662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac524392-9076-492d-abb1-ce01d87dc1b7 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:54.647748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac524392-9076-492d-abb1-ce01d87dc1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:54.648363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac524392-9076-492d-abb1-ce01d87dc1b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:54.664520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c3a3e-4fe1-40a2-8357-ca409ba05290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:54.666337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c3a3e-4fe1-40a2-8357-ca409ba05290 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:54.672402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac524392-9076-492d-abb1-ce01d87dc1b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-ac524392-9076-492d-abb1-ce01d87dc1b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:54.681439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:54.682403 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:31:54.683824 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:31:54.690304 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 44/94] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:31:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:31:54.691825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f903299-5487-4dee-ac17-08ff143ffe4a None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:54.693537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f903299-5487-4dee-ac17-08ff143ffe4a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:54.693968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f903299-5487-4dee-ac17-08ff143ffe4a None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:54.695821 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 45/95] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:54.697063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f903299-5487-4dee-ac17-08ff143ffe4a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:54.720130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:54.723152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:54.815021 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 51/96] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:31:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:31:54.820767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe4d6d-41ad-4265-b90a-2c4ecf80d9df None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:54.821198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe4d6d-41ad-4265-b90a-2c4ecf80d9df None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:54.821479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe4d6d-41ad-4265-b90a-2c4ecf80d9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:54.840677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7afe4d6d-41ad-4265-b90a-2c4ecf80d9df None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:54.840857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f903299-5487-4dee-ac17-08ff143ffe4a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:54.841932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7afe4d6d-41ad-4265-b90a-2c4ecf80d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:54.883566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2d69f-e318-442b-93d7-15a1cb97547a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:54.884199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2d69f-e318-442b-93d7-15a1cb97547a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:54.884611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2d69f-e318-442b-93d7-15a1cb97547a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:54.884953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2d69f-e318-442b-93d7-15a1cb97547a None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:54.885748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2d69f-e318-442b-93d7-15a1cb97547a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:54.886109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2d69f-e318-442b-93d7-15a1cb97547a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:54.888001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2d69f-e318-442b-93d7-15a1cb97547a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:54.898973 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 46/97] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:31:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:54.900284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78429a2-6c6a-4e02-8de8-d09f33ad98b5 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:54.900632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78429a2-6c6a-4e02-8de8-d09f33ad98b5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:54.901447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78429a2-6c6a-4e02-8de8-d09f33ad98b5 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:54.903074 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 47/98] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:54.903994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78429a2-6c6a-4e02-8de8-d09f33ad98b5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:54.916100 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 52/99] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:54.918144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b0657-3fb6-4d50-878d-71c8bf2d0ff4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:54.918474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b0657-3fb6-4d50-878d-71c8bf2d0ff4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:54.918752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b0657-3fb6-4d50-878d-71c8bf2d0ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:54.938793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fd1b0657-3fb6-4d50-878d-71c8bf2d0ff4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:54.940123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fd1b0657-3fb6-4d50-878d-71c8bf2d0ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:54.996326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:55.002630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:55.034527 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 53/100] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:55.035845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b0657-3fb6-4d50-878d-71c8bf2d0ff4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:55.050300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78429a2-6c6a-4e02-8de8-d09f33ad98b5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:55.108001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28427caf-ea1c-4b58-8fec-f1072090e1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:55.109750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28427caf-ea1c-4b58-8fec-f1072090e1bf None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.110032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28427caf-ea1c-4b58-8fec-f1072090e1bf None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.110300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28427caf-ea1c-4b58-8fec-f1072090e1bf None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.111567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28427caf-ea1c-4b58-8fec-f1072090e1bf None admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:55.113359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28427caf-ea1c-4b58-8fec-f1072090e1bf None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:55.115682 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 48/101] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:31:54 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:55.116925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8e55e-c133-4e97-a62b-876993cc39ba None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.121287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8e55e-c133-4e97-a62b-876993cc39ba None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.121287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8e55e-c133-4e97-a62b-876993cc39ba None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.121287 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 49/102] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:55.121287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2ebff-eb3e-441d-bee8-ecd76a9e7d9a None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.121589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2ebff-eb3e-441d-bee8-ecd76a9e7d9a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.121589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2ebff-eb3e-441d-bee8-ecd76a9e7d9a None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.122783 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 50/103] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:55.128245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0945d5-6a37-4aab-997f-81613ea08231 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.128523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0945d5-6a37-4aab-997f-81613ea08231 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.128786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0945d5-6a37-4aab-997f-81613ea08231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.147756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8f0945d5-6a37-4aab-997f-81613ea08231 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:55.149280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f0945d5-6a37-4aab-997f-81613ea08231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:55.213151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b0657-3fb6-4d50-878d-71c8bf2d0ff4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:55.222147 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 51/104] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:55.223621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275196ad-584b-4ab7-9395-f19f4695e8fb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.223964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275196ad-584b-4ab7-9395-f19f4695e8fb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.224340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275196ad-584b-4ab7-9395-f19f4695e8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.243658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-275196ad-584b-4ab7-9395-f19f4695e8fb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:55.245261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-275196ad-584b-4ab7-9395-f19f4695e8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:55.256522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ea95f-c1b0-41e8-aaa4-af687154ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:55.258237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ea95f-c1b0-41e8-aaa4-af687154ccad None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.258734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ea95f-c1b0-41e8-aaa4-af687154ccad None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.259209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ea95f-c1b0-41e8-aaa4-af687154ccad None admin] PATH_INFO: `/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.261497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ea95f-c1b0-41e8-aaa4-af687154ccad None admin] RBAC: Authorizing `identity:get_user(user_id=22b1d8706e1e41dbb9984cbc4c9d7be5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:55.263787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ea95f-c1b0-41e8-aaa4-af687154ccad None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:55.266334 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 54/105] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:55 2022] GET /identity/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:55.267837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ea95f-c1b0-41e8-aaa4-af687154ccad None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:55.318922 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 52/106] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:55.319867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275196ad-584b-4ab7-9395-f19f4695e8fb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:55.334867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:55.335959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:55.408503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ea95f-c1b0-41e8-aaa4-af687154ccad None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:55.466001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929cd1b-ffa0-4314-b787-6eefce2e6b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:55.466960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929cd1b-ffa0-4314-b787-6eefce2e6b9e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.467410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929cd1b-ffa0-4314-b787-6eefce2e6b9e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.467889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929cd1b-ffa0-4314-b787-6eefce2e6b9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.468900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929cd1b-ffa0-4314-b787-6eefce2e6b9e None admin] RBAC: Adding query filter params (role.id=b5b286eb4fb94285914766a260e99693, scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd, user.id=22b1d8706e1e41dbb9984cbc4c9d7be5) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:55.469520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929cd1b-ffa0-4314-b787-6eefce2e6b9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:55.471126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929cd1b-ffa0-4314-b787-6eefce2e6b9e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:55.472164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275196ad-584b-4ab7-9395-f19f4695e8fb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:55.477922 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 55/107] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:31:55 2022] GET /identity/v3/role_assignments?user.id=22b1d8706e1e41dbb9984cbc4c9d7be5&scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd&role.id=b5b286eb4fb94285914766a260e99693 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:55.486988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929cd1b-ffa0-4314-b787-6eefce2e6b9e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:55.534408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbe0db-f561-4971-872e-af41912ca6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:55.535529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbe0db-f561-4971-872e-af41912ca6b1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.535529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbe0db-f561-4971-872e-af41912ca6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.535798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbe0db-f561-4971-872e-af41912ca6b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.670359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbe0db-f561-4971-872e-af41912ca6b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:55.672192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbe0db-f561-4971-872e-af41912ca6b1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:55.675095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929cd1b-ffa0-4314-b787-6eefce2e6b9e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:55.675266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:55.694537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:55.721236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb04a3-2375-452d-b070-94df09f3839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:55.721600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb04a3-2375-452d-b070-94df09f3839b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.721980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb04a3-2375-452d-b070-94df09f3839b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.722352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb04a3-2375-452d-b070-94df09f3839b None admin] PATH_INFO: `/v3/roles/member` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.735842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb04a3-2375-452d-b070-94df09f3839b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-93fb04a3-2375-452d-b070-94df09f3839b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:55.741526 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:55.742361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:31:55.743150 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 56/108] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:31:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:31:55.743754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb04a3-2375-452d-b070-94df09f3839b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:55.798160 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 53/109] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:31:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:31:55.798935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbe0db-f561-4971-872e-af41912ca6b1 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:55.850495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb04a3-2375-452d-b070-94df09f3839b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:55.883731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb8955-e16c-4774-8666-2d34e8546dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:55.884341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb8955-e16c-4774-8666-2d34e8546dc4 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.884617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb8955-e16c-4774-8666-2d34e8546dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.884881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb8955-e16c-4774-8666-2d34e8546dc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.895634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb8955-e16c-4774-8666-2d34e8546dc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:55.898566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbe0db-f561-4971-872e-af41912ca6b1 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:55.900783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-e3eb8955-e16c-4774-8666-2d34e8546dc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:55.900901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:55.901985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 57/110] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:31:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:31:55.902898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb8955-e16c-4774-8666-2d34e8546dc4 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:55.934333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fbc18-7dd4-48f3-920c-f89933b51fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:55.934942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fbc18-7dd4-48f3-920c-f89933b51fa1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:55.935213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fbc18-7dd4-48f3-920c-f89933b51fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:55.935476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fbc18-7dd4-48f3-920c-f89933b51fa1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:55.936328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:55.936770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fbc18-7dd4-48f3-920c-f89933b51fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:55.938002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fbc18-7dd4-48f3-920c-f89933b51fa1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:55.939550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 54/111] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:31:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:55.940379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fbc18-7dd4-48f3-920c-f89933b51fa1 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.003208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb8955-e16c-4774-8666-2d34e8546dc4 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.035986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcab4eb-35f7-49cd-ac8f-ae85a23b82e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.036608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcab4eb-35f7-49cd-ac8f-ae85a23b82e8 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.036874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcab4eb-35f7-49cd-ac8f-ae85a23b82e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.037183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcab4eb-35f7-49cd-ac8f-ae85a23b82e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.039838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fbc18-7dd4-48f3-920c-f89933b51fa1 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.047505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcab4eb-35f7-49cd-ac8f-ae85a23b82e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-dfcab4eb-35f7-49cd-ac8f-ae85a23b82e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:31:56.053701 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:31:56.054722 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:31:56.055709 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 58/112] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:31:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:31:56.058403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcab4eb-35f7-49cd-ac8f-ae85a23b82e8 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.076279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e44882-a7af-4a5b-9682-f831f4a91fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.076872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e44882-a7af-4a5b-9682-f831f4a91fba None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.077179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e44882-a7af-4a5b-9682-f831f4a91fba None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.077442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e44882-a7af-4a5b-9682-f831f4a91fba None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.078098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e44882-a7af-4a5b-9682-f831f4a91fba None admin] RBAC: Adding query filter params (name=member) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:56.078417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e44882-a7af-4a5b-9682-f831f4a91fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:56.079810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e44882-a7af-4a5b-9682-f831f4a91fba None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:56.087997 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 55/113] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:31:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:56.088901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e44882-a7af-4a5b-9682-f831f4a91fba None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.123792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:56.155143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:56.161720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcab4eb-35f7-49cd-ac8f-ae85a23b82e8 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.190999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e44882-a7af-4a5b-9682-f831f4a91fba None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.199887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98fc65-5b3c-452a-a828-6e247d6c404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.200587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98fc65-5b3c-452a-a828-6e247d6c404e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.200860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98fc65-5b3c-452a-a828-6e247d6c404e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.201161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98fc65-5b3c-452a-a828-6e247d6c404e None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.202279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98fc65-5b3c-452a-a828-6e247d6c404e None admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:56.203740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98fc65-5b3c-452a-a828-6e247d6c404e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:56.205389 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 59/114] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:31:56 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:56.206118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98fc65-5b3c-452a-a828-6e247d6c404e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.225367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5acbd-8c4a-43fb-8a76-cfd84a235d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.225966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5acbd-8c4a-43fb-8a76-cfd84a235d57 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.226251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5acbd-8c4a-43fb-8a76-cfd84a235d57 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.226523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5acbd-8c4a-43fb-8a76-cfd84a235d57 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.227180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5acbd-8c4a-43fb-8a76-cfd84a235d57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:56.227486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5acbd-8c4a-43fb-8a76-cfd84a235d57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:56.228854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5acbd-8c4a-43fb-8a76-cfd84a235d57 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:56.237540 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 56/115] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:31:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:56.238320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5acbd-8c4a-43fb-8a76-cfd84a235d57 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.306993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98fc65-5b3c-452a-a828-6e247d6c404e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.332233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:56.338258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5acbd-8c4a-43fb-8a76-cfd84a235d57 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.340281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7452c0-9769-48f3-8305-414099fb2a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.340860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7452c0-9769-48f3-8305-414099fb2a54 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.341168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7452c0-9769-48f3-8305-414099fb2a54 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.341439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7452c0-9769-48f3-8305-414099fb2a54 None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.342080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7452c0-9769-48f3-8305-414099fb2a54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:56.343524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7452c0-9769-48f3-8305-414099fb2a54 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:56.370992 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 60/116] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:31:56 2022] POST /identity/v3/projects => generated 302 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:31:56.372040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7452c0-9769-48f3-8305-414099fb2a54 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.377956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecad3c-1b4d-45a0-a8dd-2548e09f7d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.378584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecad3c-1b4d-45a0-a8dd-2548e09f7d0d None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.378857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecad3c-1b4d-45a0-a8dd-2548e09f7d0d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.379126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecad3c-1b4d-45a0-a8dd-2548e09f7d0d None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.379850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecad3c-1b4d-45a0-a8dd-2548e09f7d0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:56.380147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecad3c-1b4d-45a0-a8dd-2548e09f7d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:56.381672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecad3c-1b4d-45a0-a8dd-2548e09f7d0d None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:56.394829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:56.398488 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 57/117] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:31:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:56.404372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecad3c-1b4d-45a0-a8dd-2548e09f7d0d None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.494760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7452c0-9769-48f3-8305-414099fb2a54 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.523243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecad3c-1b4d-45a0-a8dd-2548e09f7d0d None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.532679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49812b3d-1750-4eaa-8f9a-e65549829f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.533356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49812b3d-1750-4eaa-8f9a-e65549829f3a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.533637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49812b3d-1750-4eaa-8f9a-e65549829f3a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.533903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49812b3d-1750-4eaa-8f9a-e65549829f3a None admin] PATH_INFO: `/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.535054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49812b3d-1750-4eaa-8f9a-e65549829f3a None admin] RBAC: Authorizing `identity:get_user(user_id=22b1d8706e1e41dbb9984cbc4c9d7be5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:56.536563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49812b3d-1750-4eaa-8f9a-e65549829f3a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:56.543545 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 61/118] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:56 2022] GET /identity/v3/users/22b1d8706e1e41dbb9984cbc4c9d7be5 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:56.543545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49812b3d-1750-4eaa-8f9a-e65549829f3a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.569621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b83866-6800-471c-8b72-e9301e513646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.570241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b83866-6800-471c-8b72-e9301e513646 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.570587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b83866-6800-471c-8b72-e9301e513646 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.570922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b83866-6800-471c-8b72-e9301e513646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.571893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b83866-6800-471c-8b72-e9301e513646 None admin] RBAC: Adding query filter params (role.id=25b2c5582f194703afe45b5d8cc79171, scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd, user.id=22b1d8706e1e41dbb9984cbc4c9d7be5) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:56.572583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b83866-6800-471c-8b72-e9301e513646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:56.575055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b83866-6800-471c-8b72-e9301e513646 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:56.585197 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 58/119] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:31:56 2022] GET /identity/v3/role_assignments?user.id=22b1d8706e1e41dbb9984cbc4c9d7be5&scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd&role.id=25b2c5582f194703afe45b5d8cc79171 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:56.586155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b83866-6800-471c-8b72-e9301e513646 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.624423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:56.639275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:56.660133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49812b3d-1750-4eaa-8f9a-e65549829f3a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.708269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a342b6e-520b-43a4-8bcd-142225faa482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.709606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a342b6e-520b-43a4-8bcd-142225faa482 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.710253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a342b6e-520b-43a4-8bcd-142225faa482 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.710622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a342b6e-520b-43a4-8bcd-142225faa482 None admin] PATH_INFO: `/v3/users/f9ecef8e2dba4f429d37c67479eb320e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.712108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a342b6e-520b-43a4-8bcd-142225faa482 None admin] RBAC: Authorizing `identity:get_user(user_id=f9ecef8e2dba4f429d37c67479eb320e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:56.714584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a342b6e-520b-43a4-8bcd-142225faa482 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:56.716886 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 62/120] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:56 2022] GET /identity/v3/users/f9ecef8e2dba4f429d37c67479eb320e => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:56.717766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a342b6e-520b-43a4-8bcd-142225faa482 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.737271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b83866-6800-471c-8b72-e9301e513646 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.781829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96035b4-357b-4bb7-b842-83cf57fb39e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.782651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96035b4-357b-4bb7-b842-83cf57fb39e1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.783026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96035b4-357b-4bb7-b842-83cf57fb39e1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.783395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96035b4-357b-4bb7-b842-83cf57fb39e1 None admin] PATH_INFO: `/v3/users/5e63c046c57a4c89a4adec19b6c68425` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.801540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96035b4-357b-4bb7-b842-83cf57fb39e1 None admin] RBAC: Authorizing `identity:get_user(user_id=5e63c046c57a4c89a4adec19b6c68425)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:56.803152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96035b4-357b-4bb7-b842-83cf57fb39e1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:56.804986 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 59/121] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:31:56 2022] GET /identity/v3/users/5e63c046c57a4c89a4adec19b6c68425 => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:56.805949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96035b4-357b-4bb7-b842-83cf57fb39e1 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:56.857830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a342b6e-520b-43a4-8bcd-142225faa482 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.902727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6286a7d-6d4f-4ff2-bd74-c7fff5d97bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.903335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6286a7d-6d4f-4ff2-bd74-c7fff5d97bad None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.903609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6286a7d-6d4f-4ff2-bd74-c7fff5d97bad None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.903876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6286a7d-6d4f-4ff2-bd74-c7fff5d97bad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.911526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:56.924248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:56.934106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96035b4-357b-4bb7-b842-83cf57fb39e1 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:56.971558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c19fc3-d314-46aa-995e-33064b3050e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:56.972167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c19fc3-d314-46aa-995e-33064b3050e3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:56.972446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c19fc3-d314-46aa-995e-33064b3050e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:56.972723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c19fc3-d314-46aa-995e-33064b3050e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:56.975970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6286a7d-6d4f-4ff2-bd74-c7fff5d97bad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:56.977504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6286a7d-6d4f-4ff2-bd74-c7fff5d97bad None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:57.050175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c19fc3-d314-46aa-995e-33064b3050e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:57.051614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c19fc3-d314-46aa-995e-33064b3050e3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:57.083432 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 63/122] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:31:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:31:57.084424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6286a7d-6d4f-4ff2-bd74-c7fff5d97bad None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:57.139058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:57.143405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:57.165770 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 60/123] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:31:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:31:57.165770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c19fc3-d314-46aa-995e-33064b3050e3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:57.207790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6286a7d-6d4f-4ff2-bd74-c7fff5d97bad None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:57.245932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a0053-b9a5-49e2-9508-c9424ed7eb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:57.246565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a0053-b9a5-49e2-9508-c9424ed7eb23 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:57.246844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a0053-b9a5-49e2-9508-c9424ed7eb23 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:57.247112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a0053-b9a5-49e2-9508-c9424ed7eb23 None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:57.248696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a0053-b9a5-49e2-9508-c9424ed7eb23 None admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:57.250576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a0053-b9a5-49e2-9508-c9424ed7eb23 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:57.252548 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 64/124] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:31:57 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:57.253331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a0053-b9a5-49e2-9508-c9424ed7eb23 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:57.281544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c19fc3-d314-46aa-995e-33064b3050e3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:57.317716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b09be3-f822-48f5-b952-669f79b9f249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:57.318550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b09be3-f822-48f5-b952-669f79b9f249 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:57.318963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b09be3-f822-48f5-b952-669f79b9f249 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:57.319393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b09be3-f822-48f5-b952-669f79b9f249 None admin] PATH_INFO: `/v3/projects/325db256cf5f41a1baf8f901ccdf2716` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:57.320639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b09be3-f822-48f5-b952-669f79b9f249 None admin] RBAC: Authorizing `identity:get_project(project_id=325db256cf5f41a1baf8f901ccdf2716)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:57.322245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b09be3-f822-48f5-b952-669f79b9f249 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:57.324656 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 61/125] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:31:57 2022] GET /identity/v3/projects/325db256cf5f41a1baf8f901ccdf2716 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:57.325418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b09be3-f822-48f5-b952-669f79b9f249 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:57.361096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a0053-b9a5-49e2-9508-c9424ed7eb23 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:57.378739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:57.384021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:57.399172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7efe64-bfd8-453f-a148-20108b9dd38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:57.399806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7efe64-bfd8-453f-a148-20108b9dd38e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:57.400223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7efe64-bfd8-453f-a148-20108b9dd38e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:57.400495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7efe64-bfd8-453f-a148-20108b9dd38e None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:57.402167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7efe64-bfd8-453f-a148-20108b9dd38e None admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:57.403765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7efe64-bfd8-453f-a148-20108b9dd38e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:57.410280 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 65/126] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:31:57 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:57.410280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7efe64-bfd8-453f-a148-20108b9dd38e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:57.446148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b09be3-f822-48f5-b952-669f79b9f249 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:57.481135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ff0f7-6251-4222-b703-b1e4ac0eca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:57.481791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ff0f7-6251-4222-b703-b1e4ac0eca2d None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:57.482070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ff0f7-6251-4222-b703-b1e4ac0eca2d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:57.482341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ff0f7-6251-4222-b703-b1e4ac0eca2d None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/users/5e63c046c57a4c89a4adec19b6c68425/roles/c303443fad7a458f8aa8aadd7bfc77e0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:57.484293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ff0f7-6251-4222-b703-b1e4ac0eca2d None admin] RBAC: Authorizing `identity:create_grant(project_id=abb9dbb5eb664a45a354501eb93cb0cd, user_id=5e63c046c57a4c89a4adec19b6c68425, role_id=c303443fad7a458f8aa8aadd7bfc77e0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:57.485783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ff0f7-6251-4222-b703-b1e4ac0eca2d None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:57.509069 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 62/127] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:31:57 2022] PUT /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/users/5e63c046c57a4c89a4adec19b6c68425/roles/c303443fad7a458f8aa8aadd7bfc77e0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:31:57.509069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ff0f7-6251-4222-b703-b1e4ac0eca2d None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:57.528499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7efe64-bfd8-453f-a148-20108b9dd38e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:57.581658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f8b56-cd10-4ed4-a57b-56cd0f9143f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:57.582636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8b56-cd10-4ed4-a57b-56cd0f9143f5 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:57.583443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8b56-cd10-4ed4-a57b-56cd0f9143f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:57.583794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8b56-cd10-4ed4-a57b-56cd0f9143f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:57.584903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f8b56-cd10-4ed4-a57b-56cd0f9143f5 None admin] RBAC: Adding query filter params (role.id=b5b286eb4fb94285914766a260e99693, scope.project.id=325db256cf5f41a1baf8f901ccdf2716, user.id=22b1d8706e1e41dbb9984cbc4c9d7be5) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:57.585352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f8b56-cd10-4ed4-a57b-56cd0f9143f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:57.587259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f8b56-cd10-4ed4-a57b-56cd0f9143f5 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:57.596423 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 66/128] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:31:57 2022] GET /identity/v3/role_assignments?user.id=22b1d8706e1e41dbb9984cbc4c9d7be5&scope.project.id=325db256cf5f41a1baf8f901ccdf2716&role.id=b5b286eb4fb94285914766a260e99693 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:57.635212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ff0f7-6251-4222-b703-b1e4ac0eca2d None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:57.647306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:57.681108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20917f19-65bc-43fd-bfc8-60cf82804c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:57.681797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20917f19-65bc-43fd-bfc8-60cf82804c84 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:57.682080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20917f19-65bc-43fd-bfc8-60cf82804c84 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:57.682998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20917f19-65bc-43fd-bfc8-60cf82804c84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:57.683809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20917f19-65bc-43fd-bfc8-60cf82804c84 None admin] RBAC: Adding query filter params (role.id=fab66a1a6ba64c9583182c78978b218e, scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd, user.id=f9ecef8e2dba4f429d37c67479eb320e) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:31:57.684125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20917f19-65bc-43fd-bfc8-60cf82804c84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:57.685720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20917f19-65bc-43fd-bfc8-60cf82804c84 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:57.693525 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 63/129] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:31:57 2022] GET /identity/v3/role_assignments?user.id=f9ecef8e2dba4f429d37c67479eb320e&scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd&role.id=fab66a1a6ba64c9583182c78978b218e => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:58.884189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbb704-707c-447b-a672-90aa24219b05 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:58.884739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbb704-707c-447b-a672-90aa24219b05 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:58.884739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbb704-707c-447b-a672-90aa24219b05 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:58.886705 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 67/130] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:58.891767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb03499-f7a4-47aa-bb98-c5edfb2cbe6c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:58.892184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb03499-f7a4-47aa-bb98-c5edfb2cbe6c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:58.892456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb03499-f7a4-47aa-bb98-c5edfb2cbe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:58.911122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1fb03499-f7a4-47aa-bb98-c5edfb2cbe6c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:58.912279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1fb03499-f7a4-47aa-bb98-c5edfb2cbe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:31:58.975991 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 64/131] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:31:59.296195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbb704-707c-447b-a672-90aa24219b05 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:59.346299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:59.397878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbb704-707c-447b-a672-90aa24219b05 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:59.430969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f587c89-051b-4820-8261-656b229d7193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:59.431589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f587c89-051b-4820-8261-656b229d7193 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:59.431867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f587c89-051b-4820-8261-656b229d7193 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:59.432779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f587c89-051b-4820-8261-656b229d7193 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:59.433911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f587c89-051b-4820-8261-656b229d7193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:59.435382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f587c89-051b-4820-8261-656b229d7193 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:59.436947 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 68/132] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:31:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:31:59.442489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb03499-f7a4-47aa-bb98-c5edfb2cbe6c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:31:59.544181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb03499-f7a4-47aa-bb98-c5edfb2cbe6c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:31:59.545729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:31:59.579285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb780c73-45bb-45fe-8e3c-0b907f8633f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:31:59.579892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb780c73-45bb-45fe-8e3c-0b907f8633f0 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:59.580168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb780c73-45bb-45fe-8e3c-0b907f8633f0 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:59.580437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb780c73-45bb-45fe-8e3c-0b907f8633f0 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:59.581116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb780c73-45bb-45fe-8e3c-0b907f8633f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:31:59.582554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb780c73-45bb-45fe-8e3c-0b907f8633f0 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:31:59.604142 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 65/133] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:31:59 2022] POST /identity/v3/users => generated 283 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:31:59.954832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd6fba-d9e1-435a-b66b-2e1c35ff5448 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:59.955345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd6fba-d9e1-435a-b66b-2e1c35ff5448 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:59.955418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd6fba-d9e1-435a-b66b-2e1c35ff5448 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:59.956897 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 69/134] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:31:59.962357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad3e16-fee4-4172-b41a-508410539b73 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:31:59.962780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad3e16-fee4-4172-b41a-508410539b73 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:31:59.962877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad3e16-fee4-4172-b41a-508410539b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:31:59.981893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3aad3e16-fee4-4172-b41a-508410539b73 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:31:59.983059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3aad3e16-fee4-4172-b41a-508410539b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:00.064231 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 66/135] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:00.395825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd6fba-d9e1-435a-b66b-2e1c35ff5448 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:00.504282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd6fba-d9e1-435a-b66b-2e1c35ff5448 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:00.525381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:00.539937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d5ab4-e237-4003-b48a-12a9cdceee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:00.540556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d5ab4-e237-4003-b48a-12a9cdceee79 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:00.540826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d5ab4-e237-4003-b48a-12a9cdceee79 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:00.541121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d5ab4-e237-4003-b48a-12a9cdceee79 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:00.551424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d5ab4-e237-4003-b48a-12a9cdceee79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-014d5ab4-e237-4003-b48a-12a9cdceee79 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:00.556584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:00.558103 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:00.558913 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 70/136] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:00.565327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad3e16-fee4-4172-b41a-508410539b73 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:00.658543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:00.672247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad3e16-fee4-4172-b41a-508410539b73 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:00.708782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8bd3-b2e6-4af4-b615-abde278563a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:00.709416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8bd3-b2e6-4af4-b615-abde278563a2 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:00.709690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8bd3-b2e6-4af4-b615-abde278563a2 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:00.709957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8bd3-b2e6-4af4-b615-abde278563a2 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:00.710609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8bd3-b2e6-4af4-b615-abde278563a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:00.710906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8bd3-b2e6-4af4-b615-abde278563a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:00.712266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8bd3-b2e6-4af4-b615-abde278563a2 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:00.721095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 67/137] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:00.726728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d5ab4-e237-4003-b48a-12a9cdceee79 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:00.834048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d5ab4-e237-4003-b48a-12a9cdceee79 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:00.869289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774875b-dbc1-406d-b26e-0cc0ca1a813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:00.869967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774875b-dbc1-406d-b26e-0cc0ca1a813e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:00.870240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774875b-dbc1-406d-b26e-0cc0ca1a813e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:00.870506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774875b-dbc1-406d-b26e-0cc0ca1a813e None admin] PATH_INFO: `/v3/users/5e63c046c57a4c89a4adec19b6c68425` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:00.871626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774875b-dbc1-406d-b26e-0cc0ca1a813e None admin] RBAC: Authorizing `identity:get_user(user_id=5e63c046c57a4c89a4adec19b6c68425)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:00.873182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774875b-dbc1-406d-b26e-0cc0ca1a813e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:00.874741 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 71/138] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:00 2022] GET /identity/v3/users/5e63c046c57a4c89a4adec19b6c68425 => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:00.880273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8bd3-b2e6-4af4-b615-abde278563a2 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:00.987146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8bd3-b2e6-4af4-b615-abde278563a2 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:01.026345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b43cf-c860-4bd8-b488-fe4894a52889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:01.027355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b43cf-c860-4bd8-b488-fe4894a52889 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:01.027826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b43cf-c860-4bd8-b488-fe4894a52889 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:01.028284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b43cf-c860-4bd8-b488-fe4894a52889 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:01.048747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:01.108677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b43cf-c860-4bd8-b488-fe4894a52889 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:01.110273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b43cf-c860-4bd8-b488-fe4894a52889 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:01.216228 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 68/139] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:01.222314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774875b-dbc1-406d-b26e-0cc0ca1a813e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:01.272966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:01.330736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774875b-dbc1-406d-b26e-0cc0ca1a813e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:01.367033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048d5e7-0917-43da-9850-a2df2ccc2f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:01.368008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048d5e7-0917-43da-9850-a2df2ccc2f38 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:01.368493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048d5e7-0917-43da-9850-a2df2ccc2f38 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:01.369239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048d5e7-0917-43da-9850-a2df2ccc2f38 None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:01.371170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048d5e7-0917-43da-9850-a2df2ccc2f38 None admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:01.372810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048d5e7-0917-43da-9850-a2df2ccc2f38 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:01.374641 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 72/140] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:01 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:01.380879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b43cf-c860-4bd8-b488-fe4894a52889 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:01.433836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:01.483910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b43cf-c860-4bd8-b488-fe4894a52889 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:01.518945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f12e1-4679-4f0a-bde0-8b41fa6a599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:01.519872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f12e1-4679-4f0a-bde0-8b41fa6a599e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:01.520338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f12e1-4679-4f0a-bde0-8b41fa6a599e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:01.520837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f12e1-4679-4f0a-bde0-8b41fa6a599e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:01.521782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f12e1-4679-4f0a-bde0-8b41fa6a599e None admin] RBAC: Adding query filter params (role.id=c303443fad7a458f8aa8aadd7bfc77e0, scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd, user.id=5e63c046c57a4c89a4adec19b6c68425) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:01.522279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f12e1-4679-4f0a-bde0-8b41fa6a599e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:01.523947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f12e1-4679-4f0a-bde0-8b41fa6a599e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:01.530275 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 69/141] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:32:01 2022] GET /identity/v3/role_assignments?user.id=5e63c046c57a4c89a4adec19b6c68425&scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd&role.id=c303443fad7a458f8aa8aadd7bfc77e0 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:02.190525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219b315-7365-4554-b325-b1320ae8f20b None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:02.191194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219b315-7365-4554-b325-b1320ae8f20b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:02.191669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219b315-7365-4554-b325-b1320ae8f20b None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:02.193253 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 73/142] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:02.202096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b486adf-d2a7-4141-8112-6dcd8f1f6ab0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:02.202569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b486adf-d2a7-4141-8112-6dcd8f1f6ab0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:02.202643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b486adf-d2a7-4141-8112-6dcd8f1f6ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:02.223366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3b486adf-d2a7-4141-8112-6dcd8f1f6ab0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:02.223366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3b486adf-d2a7-4141-8112-6dcd8f1f6ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:02.290944 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 70/143] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:02.298727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0bc52-89e4-4648-9916-9e9f1c0c3a18 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:02.299195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0bc52-89e4-4648-9916-9e9f1c0c3a18 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:02.299270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0bc52-89e4-4648-9916-9e9f1c0c3a18 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:02.300742 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 74/144] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:02.305869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a2765-3a59-4583-8342-9a669cec0dfb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:02.306159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a2765-3a59-4583-8342-9a669cec0dfb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:02.306442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a2765-3a59-4583-8342-9a669cec0dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:02.325241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a93a2765-3a59-4583-8342-9a669cec0dfb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:02.326405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a93a2765-3a59-4583-8342-9a669cec0dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:02.368450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:02.387640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854e0f-6333-4fc4-a858-40098d922246 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:02.387991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854e0f-6333-4fc4-a858-40098d922246 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:02.388463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854e0f-6333-4fc4-a858-40098d922246 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:02.389924 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 75/145] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:02.394832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1370f-61bb-482a-b594-809c3abf0c1e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:02.395114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1370f-61bb-482a-b594-809c3abf0c1e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:02.395504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1370f-61bb-482a-b594-809c3abf0c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:02.396550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 71/146] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:02.414714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d7f1370f-61bb-482a-b594-809c3abf0c1e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:02.415970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d7f1370f-61bb-482a-b594-809c3abf0c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:02.483503 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 76/147] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:02.622033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a2765-3a59-4583-8342-9a669cec0dfb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:02.721480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a2765-3a59-4583-8342-9a669cec0dfb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:02.724057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1370f-61bb-482a-b594-809c3abf0c1e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:02.760828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de89374-3c73-4c1d-be64-355b99468b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:02.761414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de89374-3c73-4c1d-be64-355b99468b5b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:02.761674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de89374-3c73-4c1d-be64-355b99468b5b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:02.761935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de89374-3c73-4c1d-be64-355b99468b5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:02.772320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de89374-3c73-4c1d-be64-355b99468b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:02.772931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-0de89374-3c73-4c1d-be64-355b99468b5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:02.777279 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:02.778101 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:02.779049 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 72/148] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:32:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:02.783967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de89374-3c73-4c1d-be64-355b99468b5b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:02.818313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:02.832953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1370f-61bb-482a-b594-809c3abf0c1e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:02.867494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d5242-d331-45fa-9515-3a1fae250f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:02.868335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d5242-d331-45fa-9515-3a1fae250f1c None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:02.868442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d5242-d331-45fa-9515-3a1fae250f1c None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:02.868916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d5242-d331-45fa-9515-3a1fae250f1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:02.879140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d5242-d331-45fa-9515-3a1fae250f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:02.882774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de89374-3c73-4c1d-be64-355b99468b5b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-f73d5242-d331-45fa-9515-3a1fae250f1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:02.883911 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:02.884717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:02.885570 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 77/149] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:32:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:32:02.886254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d5242-d331-45fa-9515-3a1fae250f1c None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:02.918711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e605fe1-1454-4686-b3dd-de4798a29acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:02.919301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e605fe1-1454-4686-b3dd-de4798a29acb None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:02.919570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e605fe1-1454-4686-b3dd-de4798a29acb None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:02.919833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e605fe1-1454-4686-b3dd-de4798a29acb None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:02.920472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e605fe1-1454-4686-b3dd-de4798a29acb None admin] RBAC: Adding query filter params (name=admin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:02.920767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e605fe1-1454-4686-b3dd-de4798a29acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:02.922180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e605fe1-1454-4686-b3dd-de4798a29acb None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:02.930165 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 73/150] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:02.935614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e605fe1-1454-4686-b3dd-de4798a29acb None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:02.985402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d5242-d331-45fa-9515-3a1fae250f1c None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:02.987790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:03.020358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30bfae-01c9-4dcf-9cbe-5b53e83d7769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:03.020899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30bfae-01c9-4dcf-9cbe-5b53e83d7769 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:03.021203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30bfae-01c9-4dcf-9cbe-5b53e83d7769 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:03.021476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30bfae-01c9-4dcf-9cbe-5b53e83d7769 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:03.022569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30bfae-01c9-4dcf-9cbe-5b53e83d7769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:03.024035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30bfae-01c9-4dcf-9cbe-5b53e83d7769 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:03.024694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:03.026136 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 78/151] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:03.026552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30bfae-01c9-4dcf-9cbe-5b53e83d7769 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:03.034939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e605fe1-1454-4686-b3dd-de4798a29acb None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:03.068924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3c0b6-9f31-44fb-9f97-665860c3af87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:03.069538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3c0b6-9f31-44fb-9f97-665860c3af87 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:03.069815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3c0b6-9f31-44fb-9f97-665860c3af87 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:03.070078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3c0b6-9f31-44fb-9f97-665860c3af87 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:03.070723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3c0b6-9f31-44fb-9f97-665860c3af87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:03.071016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3c0b6-9f31-44fb-9f97-665860c3af87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:03.072368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3c0b6-9f31-44fb-9f97-665860c3af87 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:03.080231 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 74/152] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:03.080995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3c0b6-9f31-44fb-9f97-665860c3af87 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:03.125857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30bfae-01c9-4dcf-9cbe-5b53e83d7769 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:03.158118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe42ed-dd28-4355-9791-1be76196b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:03.158725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe42ed-dd28-4355-9791-1be76196b801 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:03.158992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe42ed-dd28-4355-9791-1be76196b801 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:03.159256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe42ed-dd28-4355-9791-1be76196b801 None admin] PATH_INFO: `/v3/users/29f3c0b867c54de99920611eb980d35f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:03.172708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe42ed-dd28-4355-9791-1be76196b801 None admin] RBAC: Authorizing `identity:get_user(user_id=29f3c0b867c54de99920611eb980d35f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:03.174220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe42ed-dd28-4355-9791-1be76196b801 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:03.175822 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 79/153] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:03 2022] GET /identity/v3/users/29f3c0b867c54de99920611eb980d35f => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:03.176585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe42ed-dd28-4355-9791-1be76196b801 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:03.185148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3c0b6-9f31-44fb-9f97-665860c3af87 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:03.209198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:03.220659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85502225-e7d4-44aa-b1a9-ea011b276300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:03.221294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85502225-e7d4-44aa-b1a9-ea011b276300 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:03.221572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85502225-e7d4-44aa-b1a9-ea011b276300 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:03.221867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85502225-e7d4-44aa-b1a9-ea011b276300 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:03.222486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85502225-e7d4-44aa-b1a9-ea011b276300 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:03.223910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85502225-e7d4-44aa-b1a9-ea011b276300 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:03.245881 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 75/154] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:32:03 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:32:03.254119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85502225-e7d4-44aa-b1a9-ea011b276300 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:03.272439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:03.290983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe42ed-dd28-4355-9791-1be76196b801 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:03.329955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cce2ee-4d6b-4e19-8a69-a2c3f1096721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:03.330651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cce2ee-4d6b-4e19-8a69-a2c3f1096721 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:03.330949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cce2ee-4d6b-4e19-8a69-a2c3f1096721 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:03.331220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cce2ee-4d6b-4e19-8a69-a2c3f1096721 None admin] PATH_INFO: `/v3/users/29f3c0b867c54de99920611eb980d35f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:03.332417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cce2ee-4d6b-4e19-8a69-a2c3f1096721 None admin] RBAC: Authorizing `identity:get_user(user_id=29f3c0b867c54de99920611eb980d35f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:03.333911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cce2ee-4d6b-4e19-8a69-a2c3f1096721 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:03.335562 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 80/155] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:03 2022] GET /identity/v3/users/29f3c0b867c54de99920611eb980d35f => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:03.341318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cce2ee-4d6b-4e19-8a69-a2c3f1096721 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:03.381222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85502225-e7d4-44aa-b1a9-ea011b276300 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:03.427215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a43ee-64a5-4a32-b323-020e165484e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:03.428337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a43ee-64a5-4a32-b323-020e165484e3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:03.428793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a43ee-64a5-4a32-b323-020e165484e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:03.429402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a43ee-64a5-4a32-b323-020e165484e3 None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:03.430613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a43ee-64a5-4a32-b323-020e165484e3 None admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:03.432664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a43ee-64a5-4a32-b323-020e165484e3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:03.439781 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 76/156] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:03 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:03.440886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a43ee-64a5-4a32-b323-020e165484e3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:03.454647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:03.460903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cce2ee-4d6b-4e19-8a69-a2c3f1096721 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:03.503958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fcf1d8-4ac4-47c7-8011-9185c1fe8bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:03.504583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fcf1d8-4ac4-47c7-8011-9185c1fe8bab None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:03.504860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fcf1d8-4ac4-47c7-8011-9185c1fe8bab None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:03.505165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fcf1d8-4ac4-47c7-8011-9185c1fe8bab None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:03.507235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fcf1d8-4ac4-47c7-8011-9185c1fe8bab None admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:03.508702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fcf1d8-4ac4-47c7-8011-9185c1fe8bab None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:03.510467 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 81/157] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:03 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:03.516746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fcf1d8-4ac4-47c7-8011-9185c1fe8bab None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:03.533785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:03.556376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a43ee-64a5-4a32-b323-020e165484e3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:03.607057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8d54f-74a6-4bc3-bdb7-04ada026c7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:03.607715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8d54f-74a6-4bc3-bdb7-04ada026c7fc None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:03.607987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8d54f-74a6-4bc3-bdb7-04ada026c7fc None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:03.608250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8d54f-74a6-4bc3-bdb7-04ada026c7fc None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b/users/29f3c0b867c54de99920611eb980d35f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:03.610184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8d54f-74a6-4bc3-bdb7-04ada026c7fc None admin] RBAC: Authorizing `identity:create_grant(project_id=6d040a120d1b4214a79694db0340276b, user_id=29f3c0b867c54de99920611eb980d35f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:03.611620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8d54f-74a6-4bc3-bdb7-04ada026c7fc None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:03.633140 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 77/158] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:32:03 2022] PUT /identity/v3/projects/6d040a120d1b4214a79694db0340276b/users/29f3c0b867c54de99920611eb980d35f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:32:03.654309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fcf1d8-4ac4-47c7-8011-9185c1fe8bab None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:03.696673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4ef54-956d-47df-9a2c-1ae151544048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:03.697392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4ef54-956d-47df-9a2c-1ae151544048 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:03.697736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4ef54-956d-47df-9a2c-1ae151544048 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:03.698015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4ef54-956d-47df-9a2c-1ae151544048 None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b/users/29f3c0b867c54de99920611eb980d35f/roles/25b2c5582f194703afe45b5d8cc79171` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:03.700260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4ef54-956d-47df-9a2c-1ae151544048 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d040a120d1b4214a79694db0340276b, user_id=29f3c0b867c54de99920611eb980d35f, role_id=25b2c5582f194703afe45b5d8cc79171)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:03.701805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4ef54-956d-47df-9a2c-1ae151544048 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:03.721159 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 82/159] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:32:03 2022] PUT /identity/v3/projects/6d040a120d1b4214a79694db0340276b/users/29f3c0b867c54de99920611eb980d35f/roles/25b2c5582f194703afe45b5d8cc79171 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:32:05.896266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720288a4-ecbd-4ea6-abd4-ae668557241e None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:05.896775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720288a4-ecbd-4ea6-abd4-ae668557241e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:05.896775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720288a4-ecbd-4ea6-abd4-ae668557241e None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:05.898167 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 78/160] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:05.904572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de47b12-6648-4ecd-bc56-29020b7fffa3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:05.905017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de47b12-6648-4ecd-bc56-29020b7fffa3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:05.905218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de47b12-6648-4ecd-bc56-29020b7fffa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:05.924808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3de47b12-6648-4ecd-bc56-29020b7fffa3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:05.926023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3de47b12-6648-4ecd-bc56-29020b7fffa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:05.947888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:05.980562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c670aca-722a-44a9-b1b7-6a31c8f5a140 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:05.980923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c670aca-722a-44a9-b1b7-6a31c8f5a140 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:05.981293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c670aca-722a-44a9-b1b7-6a31c8f5a140 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:05.982632 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 79/161] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:32:05.987885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1234d7-ed0c-4fd9-9c41-792a5b9a68cc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:05.988166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1234d7-ed0c-4fd9-9c41-792a5b9a68cc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:05.988442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1234d7-ed0c-4fd9-9c41-792a5b9a68cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.003877 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 83/162] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:06.009371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-dc1234d7-ed0c-4fd9-9c41-792a5b9a68cc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:06.011331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dc1234d7-ed0c-4fd9-9c41-792a5b9a68cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:06.067074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042c5c-2d7e-4686-b4b8-49136f3650e3 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:06.067374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042c5c-2d7e-4686-b4b8-49136f3650e3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:06.067613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042c5c-2d7e-4686-b4b8-49136f3650e3 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.069189 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 84/163] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:32:06.074875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc315ee-a97a-417b-8523-d06b85c373b7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:06.075160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc315ee-a97a-417b-8523-d06b85c373b7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:06.075440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc315ee-a97a-417b-8523-d06b85c373b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.087173 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 80/164] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:06.097759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fcc315ee-a97a-417b-8523-d06b85c373b7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:06.098948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fcc315ee-a97a-417b-8523-d06b85c373b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:06.167643 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 85/165] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:06.242356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf47d1-75cd-4891-baeb-a54eef596632 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:06.242639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf47d1-75cd-4891-baeb-a54eef596632 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:06.242906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf47d1-75cd-4891-baeb-a54eef596632 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.244743 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 81/166] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:06.250075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ef08a-f211-4897-b44c-6939525235ec None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:06.250554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ef08a-f211-4897-b44c-6939525235ec None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:06.250992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ef08a-f211-4897-b44c-6939525235ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.277485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8f6ef08a-f211-4897-b44c-6939525235ec None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:06.278656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8f6ef08a-f211-4897-b44c-6939525235ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:06.346302 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 86/167] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:06.351806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf47d1-75cd-4891-baeb-a54eef596632 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:06.428621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:06.444117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ef08a-f211-4897-b44c-6939525235ec None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:06.463531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf47d1-75cd-4891-baeb-a54eef596632 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:06.464188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:06.500150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b211f2-5350-4381-85fc-abefe05a3621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:06.500820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b211f2-5350-4381-85fc-abefe05a3621 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:06.501119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b211f2-5350-4381-85fc-abefe05a3621 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:06.501382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b211f2-5350-4381-85fc-abefe05a3621 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.512941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b211f2-5350-4381-85fc-abefe05a3621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-c4b211f2-5350-4381-85fc-abefe05a3621 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:06.519218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:06.520163 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:06.520918 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 82/168] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:06.522078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b211f2-5350-4381-85fc-abefe05a3621 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:06.561328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ef08a-f211-4897-b44c-6939525235ec None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:06.595342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0446d6c-2906-4a72-a696-6ca7712b23a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:06.595963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0446d6c-2906-4a72-a696-6ca7712b23a6 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:06.596236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0446d6c-2906-4a72-a696-6ca7712b23a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:06.596502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0446d6c-2906-4a72-a696-6ca7712b23a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.597616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0446d6c-2906-4a72-a696-6ca7712b23a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:06.599245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0446d6c-2906-4a72-a696-6ca7712b23a6 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:06.601014 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 87/169] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:06.606018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0446d6c-2906-4a72-a696-6ca7712b23a6 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:06.626867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b211f2-5350-4381-85fc-abefe05a3621 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:06.657931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:06.663655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b396b-ef11-4188-b8eb-89ab9940cc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:06.664348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b396b-ef11-4188-b8eb-89ab9940cc00 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:06.664629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b396b-ef11-4188-b8eb-89ab9940cc00 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:06.664896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b396b-ef11-4188-b8eb-89ab9940cc00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.676517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b396b-ef11-4188-b8eb-89ab9940cc00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-da9b396b-ef11-4188-b8eb-89ab9940cc00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:06.681554 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:06.683162 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 83/170] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:32:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:06.683928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b396b-ef11-4188-b8eb-89ab9940cc00 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:06.700623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:06.717123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0446d6c-2906-4a72-a696-6ca7712b23a6 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:06.751218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e43fc7-3913-47e3-acb6-16db815e1308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:06.751892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e43fc7-3913-47e3-acb6-16db815e1308 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:06.752163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e43fc7-3913-47e3-acb6-16db815e1308 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:06.752434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e43fc7-3913-47e3-acb6-16db815e1308 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.753113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e43fc7-3913-47e3-acb6-16db815e1308 None admin] RBAC: Adding query filter params (name=member) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:06.753416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e43fc7-3913-47e3-acb6-16db815e1308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:06.754789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e43fc7-3913-47e3-acb6-16db815e1308 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:06.768061 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 88/171] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:06.768061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e43fc7-3913-47e3-acb6-16db815e1308 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:06.785302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b396b-ef11-4188-b8eb-89ab9940cc00 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:06.819539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac67eb-45a9-4557-b2e8-795bad387c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:06.820130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac67eb-45a9-4557-b2e8-795bad387c90 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:06.820399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac67eb-45a9-4557-b2e8-795bad387c90 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:06.820658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac67eb-45a9-4557-b2e8-795bad387c90 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.821340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac67eb-45a9-4557-b2e8-795bad387c90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:06.822764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac67eb-45a9-4557-b2e8-795bad387c90 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:06.842730 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 84/172] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:32:06 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:32:06.843546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac67eb-45a9-4557-b2e8-795bad387c90 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:06.869947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e43fc7-3913-47e3-acb6-16db815e1308 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:06.916107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:06.924426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaaae7-1e38-460a-98a8-a81af64489e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:06.925127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaaae7-1e38-460a-98a8-a81af64489e3 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:06.925394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaaae7-1e38-460a-98a8-a81af64489e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:06.925664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaaae7-1e38-460a-98a8-a81af64489e3 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:06.926336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaaae7-1e38-460a-98a8-a81af64489e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:06.926632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaaae7-1e38-460a-98a8-a81af64489e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:06.928034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaaae7-1e38-460a-98a8-a81af64489e3 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:06.944356 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 89/173] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:06.944356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaaae7-1e38-460a-98a8-a81af64489e3 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:06.958703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:07.003464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac67eb-45a9-4557-b2e8-795bad387c90 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:07.049102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b83d4a-36bc-4774-a548-0c712fb2ea8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:07.049102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b83d4a-36bc-4774-a548-0c712fb2ea8e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:07.049102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b83d4a-36bc-4774-a548-0c712fb2ea8e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:07.049102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b83d4a-36bc-4774-a548-0c712fb2ea8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:07.057773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b83d4a-36bc-4774-a548-0c712fb2ea8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-75b83d4a-36bc-4774-a548-0c712fb2ea8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:07.066787 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:07.067681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:07.068453 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 85/174] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:32:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:07.069878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b83d4a-36bc-4774-a548-0c712fb2ea8e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:07.086746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaaae7-1e38-460a-98a8-a81af64489e3 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:07.133235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca97f1-d386-47d8-924c-da60bfaefa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:07.133866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca97f1-d386-47d8-924c-da60bfaefa5a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:07.134133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca97f1-d386-47d8-924c-da60bfaefa5a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:07.134395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca97f1-d386-47d8-924c-da60bfaefa5a None admin] PATH_INFO: `/v3/users/d133be6edc784cd28a805762b06c5087` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:07.158072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca97f1-d386-47d8-924c-da60bfaefa5a None admin] RBAC: Authorizing `identity:get_user(user_id=d133be6edc784cd28a805762b06c5087)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:07.159516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca97f1-d386-47d8-924c-da60bfaefa5a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:07.162433 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 90/175] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:06 2022] GET /identity/v3/users/d133be6edc784cd28a805762b06c5087 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:07.163227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca97f1-d386-47d8-924c-da60bfaefa5a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:07.254158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b83d4a-36bc-4774-a548-0c712fb2ea8e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:07.266847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:07.278512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:07.315095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6fb63-313b-43da-8c5e-c46c42d93875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:07.316596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fb63-313b-43da-8c5e-c46c42d93875 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:07.316596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fb63-313b-43da-8c5e-c46c42d93875 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:07.319668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fb63-313b-43da-8c5e-c46c42d93875 None admin] PATH_INFO: `/v3/users/29f3c0b867c54de99920611eb980d35f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:07.322661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6fb63-313b-43da-8c5e-c46c42d93875 None admin] RBAC: Authorizing `identity:get_user(user_id=29f3c0b867c54de99920611eb980d35f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:07.324189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6fb63-313b-43da-8c5e-c46c42d93875 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:07.325859 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 86/176] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:07 2022] GET /identity/v3/users/29f3c0b867c54de99920611eb980d35f => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:07.326692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6fb63-313b-43da-8c5e-c46c42d93875 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:07.343227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca97f1-d386-47d8-924c-da60bfaefa5a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:07.380181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ee9d7-9392-4d6d-af56-f543adae6c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:07.381100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ee9d7-9392-4d6d-af56-f543adae6c04 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:07.381546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ee9d7-9392-4d6d-af56-f543adae6c04 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:07.382026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ee9d7-9392-4d6d-af56-f543adae6c04 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:07.382854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ee9d7-9392-4d6d-af56-f543adae6c04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:07.383321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ee9d7-9392-4d6d-af56-f543adae6c04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:07.384916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ee9d7-9392-4d6d-af56-f543adae6c04 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:07.393866 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 91/177] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:07.395107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ee9d7-9392-4d6d-af56-f543adae6c04 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:07.435881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6fb63-313b-43da-8c5e-c46c42d93875 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:07.472726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de5b20-7be9-40f2-ba99-e52890158d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:07.473357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de5b20-7be9-40f2-ba99-e52890158d55 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:07.473627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de5b20-7be9-40f2-ba99-e52890158d55 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:07.473889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de5b20-7be9-40f2-ba99-e52890158d55 None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:07.475816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de5b20-7be9-40f2-ba99-e52890158d55 None admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:07.476628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de5b20-7be9-40f2-ba99-e52890158d55 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:07.483748 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 87/178] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:07 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:07.483748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de5b20-7be9-40f2-ba99-e52890158d55 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:07.518802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ee9d7-9392-4d6d-af56-f543adae6c04 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:07.549048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:07.553635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:07.560306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887bde8-c786-4ebc-8e65-fbd7affd944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:07.560915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887bde8-c786-4ebc-8e65-fbd7affd944c None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:07.561237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887bde8-c786-4ebc-8e65-fbd7affd944c None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:07.561506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887bde8-c786-4ebc-8e65-fbd7affd944c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:07.636328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de5b20-7be9-40f2-ba99-e52890158d55 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:07.639130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887bde8-c786-4ebc-8e65-fbd7affd944c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:07.640835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887bde8-c786-4ebc-8e65-fbd7affd944c None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:07.683232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab621b-d618-4ee5-ab71-eeda415b5aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:07.684190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab621b-d618-4ee5-ab71-eeda415b5aa8 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:07.684647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab621b-d618-4ee5-ab71-eeda415b5aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:07.685568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab621b-d618-4ee5-ab71-eeda415b5aa8 None admin] PATH_INFO: `/v3/users/29f3c0b867c54de99920611eb980d35f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:07.687295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab621b-d618-4ee5-ab71-eeda415b5aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=29f3c0b867c54de99920611eb980d35f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:07.689474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab621b-d618-4ee5-ab71-eeda415b5aa8 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:07.692026 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 88/179] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:07 2022] GET /identity/v3/users/29f3c0b867c54de99920611eb980d35f => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:07.693103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab621b-d618-4ee5-ab71-eeda415b5aa8 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:07.756011 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 92/180] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:07.756619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887bde8-c786-4ebc-8e65-fbd7affd944c None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:07.787940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:07.830901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab621b-d618-4ee5-ab71-eeda415b5aa8 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:07.846971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:07.870923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b08e2-5183-4d27-acd5-4fb0b87d1e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:07.871817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b08e2-5183-4d27-acd5-4fb0b87d1e4f None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:07.872167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b08e2-5183-4d27-acd5-4fb0b87d1e4f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:07.872507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b08e2-5183-4d27-acd5-4fb0b87d1e4f None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b/users/d133be6edc784cd28a805762b06c5087/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:07.874681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b08e2-5183-4d27-acd5-4fb0b87d1e4f None admin] RBAC: Authorizing `identity:create_grant(project_id=6d040a120d1b4214a79694db0340276b, user_id=d133be6edc784cd28a805762b06c5087, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:07.876310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b08e2-5183-4d27-acd5-4fb0b87d1e4f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:07.878475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887bde8-c786-4ebc-8e65-fbd7affd944c None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:07.897074 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 89/181] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:32:07 2022] PUT /identity/v3/projects/6d040a120d1b4214a79694db0340276b/users/d133be6edc784cd28a805762b06c5087/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:32:07.897074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b08e2-5183-4d27-acd5-4fb0b87d1e4f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:07.954891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389d4bb-5205-4d01-9d49-274369a00716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:07.957137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389d4bb-5205-4d01-9d49-274369a00716 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:07.957429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389d4bb-5205-4d01-9d49-274369a00716 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:07.957699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389d4bb-5205-4d01-9d49-274369a00716 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:08.128154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389d4bb-5205-4d01-9d49-274369a00716 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:08.129999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389d4bb-5205-4d01-9d49-274369a00716 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:08.131003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b08e2-5183-4d27-acd5-4fb0b87d1e4f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:08.172788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9b52d-4441-4d54-97fd-a63a6b84eb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:08.173460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9b52d-4441-4d54-97fd-a63a6b84eb5a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:08.173686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:08.173786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9b52d-4441-4d54-97fd-a63a6b84eb5a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:08.173989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9b52d-4441-4d54-97fd-a63a6b84eb5a None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:08.175220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9b52d-4441-4d54-97fd-a63a6b84eb5a None admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:08.176640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9b52d-4441-4d54-97fd-a63a6b84eb5a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:08.179189 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 90/182] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:07 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:08.186507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9b52d-4441-4d54-97fd-a63a6b84eb5a None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:08.261389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:08.290379 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 93/183] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:08.298023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389d4bb-5205-4d01-9d49-274369a00716 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:08.360808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9b52d-4441-4d54-97fd-a63a6b84eb5a None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:08.404485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffe80b-c5bd-4bd8-b2e4-8f6d5d1064b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:08.405090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe80b-c5bd-4bd8-b2e4-8f6d5d1064b3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:08.405371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe80b-c5bd-4bd8-b2e4-8f6d5d1064b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:08.405636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe80b-c5bd-4bd8-b2e4-8f6d5d1064b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:08.406349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffe80b-c5bd-4bd8-b2e4-8f6d5d1064b3 None admin] RBAC: Adding query filter params (role.id=fab66a1a6ba64c9583182c78978b218e, scope.project.id=6d040a120d1b4214a79694db0340276b, user.id=29f3c0b867c54de99920611eb980d35f) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:08.406649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffe80b-c5bd-4bd8-b2e4-8f6d5d1064b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:08.408086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffe80b-c5bd-4bd8-b2e4-8f6d5d1064b3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:08.416484 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 91/184] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:32:08 2022] GET /identity/v3/role_assignments?user.id=29f3c0b867c54de99920611eb980d35f&scope.project.id=6d040a120d1b4214a79694db0340276b&role.id=fab66a1a6ba64c9583182c78978b218e => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:08.417336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389d4bb-5205-4d01-9d49-274369a00716 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:08.449056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:08.458180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95715ce-0d60-411d-a3cd-7582751d7390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:08.458811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95715ce-0d60-411d-a3cd-7582751d7390 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:08.459088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95715ce-0d60-411d-a3cd-7582751d7390 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:08.459356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95715ce-0d60-411d-a3cd-7582751d7390 None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:08.460492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95715ce-0d60-411d-a3cd-7582751d7390 None admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:08.462024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95715ce-0d60-411d-a3cd-7582751d7390 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:08.464402 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 94/185] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:08 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:08.470300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffe80b-c5bd-4bd8-b2e4-8f6d5d1064b3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:08.585995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffe80b-c5bd-4bd8-b2e4-8f6d5d1064b3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:08.588885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:08.624545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a901fd-8f01-4164-a727-e61bff670a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:08.625188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a901fd-8f01-4164-a727-e61bff670a3d None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:08.625461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a901fd-8f01-4164-a727-e61bff670a3d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:08.625725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a901fd-8f01-4164-a727-e61bff670a3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:08.626448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a901fd-8f01-4164-a727-e61bff670a3d None admin] RBAC: Adding query filter params (role.id=25b2c5582f194703afe45b5d8cc79171, scope.project.id=6d040a120d1b4214a79694db0340276b, user.id=29f3c0b867c54de99920611eb980d35f) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:08.626747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a901fd-8f01-4164-a727-e61bff670a3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:08.628185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a901fd-8f01-4164-a727-e61bff670a3d None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:08.634469 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 92/186] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:32:08 2022] GET /identity/v3/role_assignments?user.id=29f3c0b867c54de99920611eb980d35f&scope.project.id=6d040a120d1b4214a79694db0340276b&role.id=25b2c5582f194703afe45b5d8cc79171 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:09.538564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec644-94e2-4004-9ee2-28d6b8614213 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:09.539061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec644-94e2-4004-9ee2-28d6b8614213 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:09.539061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec644-94e2-4004-9ee2-28d6b8614213 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:09.540342 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 95/187] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:09.545661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe09971-16af-4140-8ebd-f02c63eb3684 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:09.546082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe09971-16af-4140-8ebd-f02c63eb3684 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:09.546271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe09971-16af-4140-8ebd-f02c63eb3684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:09.564468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3fe09971-16af-4140-8ebd-f02c63eb3684 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:09.565675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3fe09971-16af-4140-8ebd-f02c63eb3684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:09.633362 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 93/188] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:09.773183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d260-9671-41a5-9648-83a3068b1119 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:09.773538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d260-9671-41a5-9648-83a3068b1119 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:09.773729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d260-9671-41a5-9648-83a3068b1119 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:09.775094 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 96/189] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:09.780833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66676994-0018-4d16-a445-edc85ddea58a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:09.781154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66676994-0018-4d16-a445-edc85ddea58a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:09.781426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66676994-0018-4d16-a445-edc85ddea58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:09.800628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-66676994-0018-4d16-a445-edc85ddea58a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:09.801761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-66676994-0018-4d16-a445-edc85ddea58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:09.871049 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 94/190] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:09.957429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8d260-9671-41a5-9648-83a3068b1119 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:10.070786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8d260-9671-41a5-9648-83a3068b1119 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:10.107958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426fdbe-229d-4322-8413-2b055577f8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:10.108930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426fdbe-229d-4322-8413-2b055577f8a7 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:10.109437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426fdbe-229d-4322-8413-2b055577f8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:10.109913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426fdbe-229d-4322-8413-2b055577f8a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:10.120854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426fdbe-229d-4322-8413-2b055577f8a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-6426fdbe-229d-4322-8413-2b055577f8a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:10.126014 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:10.126865 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:10.127619 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:10.129876 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 97/191] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:10.136301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66676994-0018-4d16-a445-edc85ddea58a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:10.165054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:10.198300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426fdbe-229d-4322-8413-2b055577f8a7 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:10.253365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66676994-0018-4d16-a445-edc85ddea58a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:10.258687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:10.292961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec6a19-f99a-4ad8-a0e3-827d99f3271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:10.293701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec6a19-f99a-4ad8-a0e3-827d99f3271b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:10.294044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec6a19-f99a-4ad8-a0e3-827d99f3271b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:10.294404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec6a19-f99a-4ad8-a0e3-827d99f3271b None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:10.295044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec6a19-f99a-4ad8-a0e3-827d99f3271b None admin] RBAC: Adding query filter params (name=reader) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:10.295335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec6a19-f99a-4ad8-a0e3-827d99f3271b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:10.296721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec6a19-f99a-4ad8-a0e3-827d99f3271b None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:10.304533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426fdbe-229d-4322-8413-2b055577f8a7 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:10.305068 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 95/192] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:10.311139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec6a19-f99a-4ad8-a0e3-827d99f3271b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:10.346613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929a36c-e0f0-4c61-8123-3eab6c411016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:10.347201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929a36c-e0f0-4c61-8123-3eab6c411016 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:10.347408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929a36c-e0f0-4c61-8123-3eab6c411016 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:10.347666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929a36c-e0f0-4c61-8123-3eab6c411016 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:10.348737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929a36c-e0f0-4c61-8123-3eab6c411016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:10.350635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929a36c-e0f0-4c61-8123-3eab6c411016 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:10.352261 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 98/193] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:10.357637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929a36c-e0f0-4c61-8123-3eab6c411016 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:10.410186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:10.421237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec6a19-f99a-4ad8-a0e3-827d99f3271b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:10.457343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f7cde-0fb7-484b-b2b6-93bed247e5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:10.458123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f7cde-0fb7-484b-b2b6-93bed247e5ba None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:10.458712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f7cde-0fb7-484b-b2b6-93bed247e5ba None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:10.458985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f7cde-0fb7-484b-b2b6-93bed247e5ba None admin] PATH_INFO: `/v3/users/9e81f61715034000a467131443c503c8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:10.467742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929a36c-e0f0-4c61-8123-3eab6c411016 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:10.475179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f7cde-0fb7-484b-b2b6-93bed247e5ba None admin] RBAC: Authorizing `identity:get_user(user_id=9e81f61715034000a467131443c503c8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:10.477102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f7cde-0fb7-484b-b2b6-93bed247e5ba None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:10.479837 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 96/194] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:10 2022] GET /identity/v3/users/9e81f61715034000a467131443c503c8 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:10.481135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa0b2e-fc55-484c-976b-025100afc6c8 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:10.481415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa0b2e-fc55-484c-976b-025100afc6c8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:10.481682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa0b2e-fc55-484c-976b-025100afc6c8 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:10.483065 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 97/195] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:10.484825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa0b2e-fc55-484c-976b-025100afc6c8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:10.495575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:10.508515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2934f-e6cc-4709-99be-0a7b1b7fcec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:10.509138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2934f-e6cc-4709-99be-0a7b1b7fcec9 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:10.509408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2934f-e6cc-4709-99be-0a7b1b7fcec9 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:10.510273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2934f-e6cc-4709-99be-0a7b1b7fcec9 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:10.510916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2934f-e6cc-4709-99be-0a7b1b7fcec9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:10.512350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2934f-e6cc-4709-99be-0a7b1b7fcec9 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:10.532633 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 99/196] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:32:10 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:32:10.533861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b358-e0ea-43ae-b7b3-a17c40dacdf9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:10.534143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b358-e0ea-43ae-b7b3-a17c40dacdf9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:10.534411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b358-e0ea-43ae-b7b3-a17c40dacdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:10.567201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3950b358-e0ea-43ae-b7b3-a17c40dacdf9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:10.568499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3950b358-e0ea-43ae-b7b3-a17c40dacdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:10.599602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa0b2e-fc55-484c-976b-025100afc6c8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:10.640783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286d91f-5bb9-4f8f-8dd0-df18e2e8b1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:10.641442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286d91f-5bb9-4f8f-8dd0-df18e2e8b1c6 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:10.641714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286d91f-5bb9-4f8f-8dd0-df18e2e8b1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:10.641978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286d91f-5bb9-4f8f-8dd0-df18e2e8b1c6 None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:10.643130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286d91f-5bb9-4f8f-8dd0-df18e2e8b1c6 None admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:10.644040 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 100/197] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:10.645830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286d91f-5bb9-4f8f-8dd0-df18e2e8b1c6 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:10.652842 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 98/198] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:10 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:10.653899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b358-e0ea-43ae-b7b3-a17c40dacdf9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:10.762775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b358-e0ea-43ae-b7b3-a17c40dacdf9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:10.773945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:10.804746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e359aa4-94be-4b97-b943-94906b0e1974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:10.805482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e359aa4-94be-4b97-b943-94906b0e1974 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:10.805752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e359aa4-94be-4b97-b943-94906b0e1974 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:10.806016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e359aa4-94be-4b97-b943-94906b0e1974 None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b/users/9e81f61715034000a467131443c503c8/roles/c303443fad7a458f8aa8aadd7bfc77e0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:10.808140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e359aa4-94be-4b97-b943-94906b0e1974 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d040a120d1b4214a79694db0340276b, user_id=9e81f61715034000a467131443c503c8, role_id=c303443fad7a458f8aa8aadd7bfc77e0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:10.809665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e359aa4-94be-4b97-b943-94906b0e1974 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:10.823107 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 101/199] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:32:10 2022] PUT /identity/v3/projects/6d040a120d1b4214a79694db0340276b/users/9e81f61715034000a467131443c503c8/roles/c303443fad7a458f8aa8aadd7bfc77e0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:32:10.979662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286d91f-5bb9-4f8f-8dd0-df18e2e8b1c6 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:11.000733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:11.093230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286d91f-5bb9-4f8f-8dd0-df18e2e8b1c6 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:11.128929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa0ea0-e53b-4ea9-9b0d-6532faf60c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:11.129837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa0ea0-e53b-4ea9-9b0d-6532faf60c6b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:11.130248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa0ea0-e53b-4ea9-9b0d-6532faf60c6b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:11.130659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa0ea0-e53b-4ea9-9b0d-6532faf60c6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:11.141627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa0ea0-e53b-4ea9-9b0d-6532faf60c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-09fa0ea0-e53b-4ea9-9b0d-6532faf60c6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:11.147005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:11.148043 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:11.149000 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:11.151743 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 99/200] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:11.157342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e359aa4-94be-4b97-b943-94906b0e1974 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:11.270727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e359aa4-94be-4b97-b943-94906b0e1974 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:11.309055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87952dc4-00d8-4f73-8358-da4c7a607a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:11.309657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87952dc4-00d8-4f73-8358-da4c7a607a05 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:11.309927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87952dc4-00d8-4f73-8358-da4c7a607a05 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:11.310192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87952dc4-00d8-4f73-8358-da4c7a607a05 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:11.310913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87952dc4-00d8-4f73-8358-da4c7a607a05 None admin] RBAC: Adding query filter params (name=member) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:11.311209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87952dc4-00d8-4f73-8358-da4c7a607a05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:11.312642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87952dc4-00d8-4f73-8358-da4c7a607a05 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:11.321862 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 102/201] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:11.327816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa0ea0-e53b-4ea9-9b0d-6532faf60c6b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:11.412130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:11.437002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa0ea0-e53b-4ea9-9b0d-6532faf60c6b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:11.473339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f677767-33c3-4460-9096-b176e80408a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:11.473968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f677767-33c3-4460-9096-b176e80408a1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:11.474238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f677767-33c3-4460-9096-b176e80408a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:11.474518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f677767-33c3-4460-9096-b176e80408a1 None admin] PATH_INFO: `/v3/users/d133be6edc784cd28a805762b06c5087` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:11.475627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f677767-33c3-4460-9096-b176e80408a1 None admin] RBAC: Authorizing `identity:get_user(user_id=d133be6edc784cd28a805762b06c5087)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:11.477087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f677767-33c3-4460-9096-b176e80408a1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:11.479160 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 100/202] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:11 2022] GET /identity/v3/users/d133be6edc784cd28a805762b06c5087 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:11.484385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87952dc4-00d8-4f73-8358-da4c7a607a05 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:11.518023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:11.591812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87952dc4-00d8-4f73-8358-da4c7a607a05 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:11.626788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01895d-aecf-4ff2-adaf-955bf22f35d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:11.627383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01895d-aecf-4ff2-adaf-955bf22f35d6 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:11.627651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01895d-aecf-4ff2-adaf-955bf22f35d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:11.627910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01895d-aecf-4ff2-adaf-955bf22f35d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:11.702593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01895d-aecf-4ff2-adaf-955bf22f35d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:11.704373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01895d-aecf-4ff2-adaf-955bf22f35d6 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:11.734749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:11.805472 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 103/203] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:11.817066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f677767-33c3-4460-9096-b176e80408a1 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:11.923374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f677767-33c3-4460-9096-b176e80408a1 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:11.958230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e919c-9a6a-48e2-ab14-0045a9ae2242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:11.959123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e919c-9a6a-48e2-ab14-0045a9ae2242 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:11.959553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e919c-9a6a-48e2-ab14-0045a9ae2242 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:11.959964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e919c-9a6a-48e2-ab14-0045a9ae2242 None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:11.961218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e919c-9a6a-48e2-ab14-0045a9ae2242 None admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:11.963305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e919c-9a6a-48e2-ab14-0045a9ae2242 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:11.965144 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 101/204] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:11 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:11.973193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01895d-aecf-4ff2-adaf-955bf22f35d6 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:12.084023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01895d-aecf-4ff2-adaf-955bf22f35d6 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:12.116170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:12.123451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50edb5-5bea-4590-9675-ed2b200e5985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:12.124040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50edb5-5bea-4590-9675-ed2b200e5985 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:12.124374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50edb5-5bea-4590-9675-ed2b200e5985 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:12.124643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50edb5-5bea-4590-9675-ed2b200e5985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:12.125395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50edb5-5bea-4590-9675-ed2b200e5985 None admin] RBAC: Adding query filter params (role.id=b5b286eb4fb94285914766a260e99693, scope.project.id=6d040a120d1b4214a79694db0340276b, user.id=d133be6edc784cd28a805762b06c5087) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:12.125695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50edb5-5bea-4590-9675-ed2b200e5985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:12.127141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50edb5-5bea-4590-9675-ed2b200e5985 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:12.133358 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 104/205] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:32:11 2022] GET /identity/v3/role_assignments?user.id=d133be6edc784cd28a805762b06c5087&scope.project.id=6d040a120d1b4214a79694db0340276b&role.id=b5b286eb4fb94285914766a260e99693 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:13.225186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c3e28-4abc-4f60-a0e9-521adf6ac88b None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:13.225676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c3e28-4abc-4f60-a0e9-521adf6ac88b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:13.225888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c3e28-4abc-4f60-a0e9-521adf6ac88b None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:13.227259 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 102/206] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:13.232678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4d35-23ef-4e5c-8db8-010919be18f1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:13.233088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4d35-23ef-4e5c-8db8-010919be18f1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:13.233304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4d35-23ef-4e5c-8db8-010919be18f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:13.252864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a4ed4d35-23ef-4e5c-8db8-010919be18f1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:13.254080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a4ed4d35-23ef-4e5c-8db8-010919be18f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:13.292416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe16ecc-c410-4b8e-9ca6-13579d887bf0 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:13.292720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe16ecc-c410-4b8e-9ca6-13579d887bf0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:13.292963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe16ecc-c410-4b8e-9ca6-13579d887bf0 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:13.294463 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 103/207] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:32:13.299397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ac8d3-c942-41e9-b77e-0473fd7adf5c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:13.299681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ac8d3-c942-41e9-b77e-0473fd7adf5c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:13.299946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ac8d3-c942-41e9-b77e-0473fd7adf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:13.320297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6b7ac8d3-c942-41e9-b77e-0473fd7adf5c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:13.321828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6b7ac8d3-c942-41e9-b77e-0473fd7adf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:13.325515 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 105/208] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:13.330633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:13.370150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c9ece-b44f-4153-94ec-f16c601d24c5 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:13.370453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c9ece-b44f-4153-94ec-f16c601d24c5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:13.370683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c9ece-b44f-4153-94ec-f16c601d24c5 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:13.372011 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 106/209] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:32:13.376965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa9afe-9e58-48ae-b2a4-ed6ce6375a7c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:13.377265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa9afe-9e58-48ae-b2a4-ed6ce6375a7c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:13.377523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa9afe-9e58-48ae-b2a4-ed6ce6375a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:13.396332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-72aa9afe-9e58-48ae-b2a4-ed6ce6375a7c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:13.397569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-72aa9afe-9e58-48ae-b2a4-ed6ce6375a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:13.401689 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 104/210] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:13.466078 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 107/211] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:13.675865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ac8d3-c942-41e9-b77e-0473fd7adf5c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:13.739753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa9afe-9e58-48ae-b2a4-ed6ce6375a7c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:13.784426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ac8d3-c942-41e9-b77e-0473fd7adf5c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:13.801431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:13.818773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f9471-93c3-4333-80d5-1b9fd18b1244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:13.819395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f9471-93c3-4333-80d5-1b9fd18b1244 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:13.819670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f9471-93c3-4333-80d5-1b9fd18b1244 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:13.819935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f9471-93c3-4333-80d5-1b9fd18b1244 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:13.830235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f9471-93c3-4333-80d5-1b9fd18b1244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-5d5f9471-93c3-4333-80d5-1b9fd18b1244 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:13.834998 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:13.836018 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 105/212] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:13.836980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f9471-93c3-4333-80d5-1b9fd18b1244 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:13.843753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:13.849348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa9afe-9e58-48ae-b2a4-ed6ce6375a7c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:13.887387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e7871-ef38-4bb7-9b8e-363217440d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:13.888020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7871-ef38-4bb7-9b8e-363217440d95 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:13.888294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7871-ef38-4bb7-9b8e-363217440d95 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:13.888557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7871-ef38-4bb7-9b8e-363217440d95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:13.889794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e7871-ef38-4bb7-9b8e-363217440d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:13.891250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e7871-ef38-4bb7-9b8e-363217440d95 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:13.892990 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 108/213] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:13.893625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e7871-ef38-4bb7-9b8e-363217440d95 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:13.953029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f9471-93c3-4333-80d5-1b9fd18b1244 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:13.990784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1819b6-f60a-4154-9cbf-73801ddfd9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:13.991324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1819b6-f60a-4154-9cbf-73801ddfd9ce None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:13.991593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1819b6-f60a-4154-9cbf-73801ddfd9ce None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:13.991858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1819b6-f60a-4154-9cbf-73801ddfd9ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:14.002453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1819b6-f60a-4154-9cbf-73801ddfd9ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-7f1819b6-f60a-4154-9cbf-73801ddfd9ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:14.007640 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:14.008669 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:14.009478 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 106/214] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:14.010111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1819b6-f60a-4154-9cbf-73801ddfd9ce None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:14.021233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e7871-ef38-4bb7-9b8e-363217440d95 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:14.046516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:14.059927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21797b49-6b23-4d23-8a29-33230f0cac38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:14.060536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21797b49-6b23-4d23-8a29-33230f0cac38 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:14.060805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21797b49-6b23-4d23-8a29-33230f0cac38 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:14.061152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21797b49-6b23-4d23-8a29-33230f0cac38 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:14.061770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21797b49-6b23-4d23-8a29-33230f0cac38 None admin] RBAC: Adding query filter params (name=member) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:14.062067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21797b49-6b23-4d23-8a29-33230f0cac38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:14.063464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21797b49-6b23-4d23-8a29-33230f0cac38 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:14.072462 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 109/215] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:14.075358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:14.078104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21797b49-6b23-4d23-8a29-33230f0cac38 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:14.120223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1819b6-f60a-4154-9cbf-73801ddfd9ce None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:14.157512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a17ae9-eb42-4ec7-9f82-9a4a858ecc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:14.158110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a17ae9-eb42-4ec7-9f82-9a4a858ecc06 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:14.158377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a17ae9-eb42-4ec7-9f82-9a4a858ecc06 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:14.158639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a17ae9-eb42-4ec7-9f82-9a4a858ecc06 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:14.159274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a17ae9-eb42-4ec7-9f82-9a4a858ecc06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:14.160693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a17ae9-eb42-4ec7-9f82-9a4a858ecc06 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:14.182472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21797b49-6b23-4d23-8a29-33230f0cac38 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:14.183995 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 107/216] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:32:14 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:32:14.189452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a17ae9-eb42-4ec7-9f82-9a4a858ecc06 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:14.227190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58673d-2706-4018-af7d-948b5c67bf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:14.228870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58673d-2706-4018-af7d-948b5c67bf80 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:14.229359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58673d-2706-4018-af7d-948b5c67bf80 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:14.229768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58673d-2706-4018-af7d-948b5c67bf80 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:14.230749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58673d-2706-4018-af7d-948b5c67bf80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:14.231291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58673d-2706-4018-af7d-948b5c67bf80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:14.232879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58673d-2706-4018-af7d-948b5c67bf80 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:14.250847 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 110/217] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:14.252007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58673d-2706-4018-af7d-948b5c67bf80 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:14.310481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:14.324274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a17ae9-eb42-4ec7-9f82-9a4a858ecc06 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:14.346796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:14.369789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000eee5-f330-493f-aee0-fdb91269daf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:14.370569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000eee5-f330-493f-aee0-fdb91269daf3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:14.371012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000eee5-f330-493f-aee0-fdb91269daf3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:14.371584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000eee5-f330-493f-aee0-fdb91269daf3 None admin] PATH_INFO: `/v3/users/4de71d4c56b1460aacd8a642ae5d8211` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:14.380452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58673d-2706-4018-af7d-948b5c67bf80 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:14.391554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000eee5-f330-493f-aee0-fdb91269daf3 None admin] RBAC: Authorizing `identity:get_user(user_id=4de71d4c56b1460aacd8a642ae5d8211)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:14.393135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000eee5-f330-493f-aee0-fdb91269daf3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:14.395026 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 108/218] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:14 2022] GET /identity/v3/users/4de71d4c56b1460aacd8a642ae5d8211 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:14.400869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000eee5-f330-493f-aee0-fdb91269daf3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:14.427203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3455aa-ad2d-461b-918d-dc2528f07668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:14.427833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3455aa-ad2d-461b-918d-dc2528f07668 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:14.428107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3455aa-ad2d-461b-918d-dc2528f07668 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:14.428983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3455aa-ad2d-461b-918d-dc2528f07668 None admin] PATH_INFO: `/v3/users/9e81f61715034000a467131443c503c8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:14.429529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3455aa-ad2d-461b-918d-dc2528f07668 None admin] RBAC: Authorizing `identity:get_user(user_id=9e81f61715034000a467131443c503c8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:14.432227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3455aa-ad2d-461b-918d-dc2528f07668 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:14.434067 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 111/219] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:14 2022] GET /identity/v3/users/9e81f61715034000a467131443c503c8 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:14.441509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3455aa-ad2d-461b-918d-dc2528f07668 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:14.535518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000eee5-f330-493f-aee0-fdb91269daf3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:14.576435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3455aa-ad2d-461b-918d-dc2528f07668 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:14.583637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb826e-3961-4963-8a15-9c1f2cd503b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:14.584394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb826e-3961-4963-8a15-9c1f2cd503b6 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:14.584664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb826e-3961-4963-8a15-9c1f2cd503b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:14.585261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb826e-3961-4963-8a15-9c1f2cd503b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:14.585865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb826e-3961-4963-8a15-9c1f2cd503b6 None admin] RBAC: Adding query filter params (role.id=b5b286eb4fb94285914766a260e99693, user.id=4de71d4c56b1460aacd8a642ae5d8211, scope.system=all) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:14.586169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb826e-3961-4963-8a15-9c1f2cd503b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:14.588163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb826e-3961-4963-8a15-9c1f2cd503b6 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:14.594831 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 109/220] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Oct 15 20:32:14 2022] GET /identity/v3/role_assignments?user.id=4de71d4c56b1460aacd8a642ae5d8211&scope.system=all&role.id=b5b286eb4fb94285914766a260e99693 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:14.597542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:14.619810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21b979-0381-4dc7-9f4f-1540af2a09fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:14.620441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21b979-0381-4dc7-9f4f-1540af2a09fc None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:14.620714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21b979-0381-4dc7-9f4f-1540af2a09fc None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:14.620982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21b979-0381-4dc7-9f4f-1540af2a09fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:14.705702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21b979-0381-4dc7-9f4f-1540af2a09fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:14.707232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21b979-0381-4dc7-9f4f-1540af2a09fc None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:14.824586 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 112/221] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:14.830664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb826e-3961-4963-8a15-9c1f2cd503b6 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:14.879173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:14.936907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb826e-3961-4963-8a15-9c1f2cd503b6 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:14.971463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbe107-de5e-4953-8651-b2626185af19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:14.972079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbe107-de5e-4953-8651-b2626185af19 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:14.972349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbe107-de5e-4953-8651-b2626185af19 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:14.972612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbe107-de5e-4953-8651-b2626185af19 None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:14.973734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbe107-de5e-4953-8651-b2626185af19 None admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:14.975160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbe107-de5e-4953-8651-b2626185af19 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:14.981225 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 110/222] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:14 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:14.986147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21b979-0381-4dc7-9f4f-1540af2a09fc None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:15.006021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:15.110746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21b979-0381-4dc7-9f4f-1540af2a09fc None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:15.186825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148a077-2e67-4092-bde8-b30192b6c75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:15.187317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148a077-2e67-4092-bde8-b30192b6c75f None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:15.187592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148a077-2e67-4092-bde8-b30192b6c75f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:15.187857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148a077-2e67-4092-bde8-b30192b6c75f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:15.188615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148a077-2e67-4092-bde8-b30192b6c75f None admin] RBAC: Adding query filter params (role.id=c303443fad7a458f8aa8aadd7bfc77e0, scope.project.id=6d040a120d1b4214a79694db0340276b, user.id=9e81f61715034000a467131443c503c8) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:15.188945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148a077-2e67-4092-bde8-b30192b6c75f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:15.200935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148a077-2e67-4092-bde8-b30192b6c75f None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:15.213206 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 113/223] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:32:14 2022] GET /identity/v3/role_assignments?user.id=9e81f61715034000a467131443c503c8&scope.project.id=6d040a120d1b4214a79694db0340276b&role.id=c303443fad7a458f8aa8aadd7bfc77e0 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:16.888679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd400f22-dd35-45ae-adf1-278e3e8dc910 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:16.889238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd400f22-dd35-45ae-adf1-278e3e8dc910 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:16.889238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd400f22-dd35-45ae-adf1-278e3e8dc910 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:16.890163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8943ff-e9a0-43af-842e-f58de21ec309 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:16.890566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8943ff-e9a0-43af-842e-f58de21ec309 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:16.890661 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 111/224] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:16.890814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8943ff-e9a0-43af-842e-f58de21ec309 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:16.891983 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 114/225] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:16.897007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d15ecc-dd6c-4a2d-8288-eabef204e4b1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:16.897238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddcfc7-3e19-478d-8268-4a8bf1ba4ffa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:16.897354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d15ecc-dd6c-4a2d-8288-eabef204e4b1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:16.897554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddcfc7-3e19-478d-8268-4a8bf1ba4ffa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:16.897662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d15ecc-dd6c-4a2d-8288-eabef204e4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:16.897807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddcfc7-3e19-478d-8268-4a8bf1ba4ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:16.915923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a0d15ecc-dd6c-4a2d-8288-eabef204e4b1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:16.917130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a0d15ecc-dd6c-4a2d-8288-eabef204e4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:16.917408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f8ddcfc7-3e19-478d-8268-4a8bf1ba4ffa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:16.918555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f8ddcfc7-3e19-478d-8268-4a8bf1ba4ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:16.986810 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 112/226] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:16.992837 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 115/227] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:17.140777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a046bd-9564-439f-a6ea-98facba2f7ec None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:17.140957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a046bd-9564-439f-a6ea-98facba2f7ec None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:17.141267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a046bd-9564-439f-a6ea-98facba2f7ec None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:17.142754 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 113/228] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:17.147646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c001-73d2-40a0-bfbf-e3ea0a9221ea None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:17.147934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c001-73d2-40a0-bfbf-e3ea0a9221ea None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:17.148206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c001-73d2-40a0-bfbf-e3ea0a9221ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:17.166996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1db8c001-73d2-40a0-bfbf-e3ea0a9221ea None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:17.168175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1db8c001-73d2-40a0-bfbf-e3ea0a9221ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:17.171796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:17.240512 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 116/229] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:17.306718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a046bd-9564-439f-a6ea-98facba2f7ec None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:17.346254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8c001-73d2-40a0-bfbf-e3ea0a9221ea None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:17.349344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:17.419830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a046bd-9564-439f-a6ea-98facba2f7ec None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:17.452955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8c001-73d2-40a0-bfbf-e3ea0a9221ea None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:17.454338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef3f0c-9a7f-4d79-b9f9-fb82a1be1c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:17.454956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef3f0c-9a7f-4d79-b9f9-fb82a1be1c11 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:17.455232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef3f0c-9a7f-4d79-b9f9-fb82a1be1c11 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:17.455496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef3f0c-9a7f-4d79-b9f9-fb82a1be1c11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:17.456591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef3f0c-9a7f-4d79-b9f9-fb82a1be1c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:17.458041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef3f0c-9a7f-4d79-b9f9-fb82a1be1c11 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:17.459854 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 114/230] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:17.465105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef3f0c-9a7f-4d79-b9f9-fb82a1be1c11 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:17.488739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99252fef-3221-4aeb-b565-40cb30ef3dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:17.489470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99252fef-3221-4aeb-b565-40cb30ef3dc3 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:17.489741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99252fef-3221-4aeb-b565-40cb30ef3dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:17.490003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99252fef-3221-4aeb-b565-40cb30ef3dc3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:17.497949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:17.502451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99252fef-3221-4aeb-b565-40cb30ef3dc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-99252fef-3221-4aeb-b565-40cb30ef3dc3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:17.507626 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:17.508498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:17.514005 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 117/231] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:17.515561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99252fef-3221-4aeb-b565-40cb30ef3dc3 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:17.576233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef3f0c-9a7f-4d79-b9f9-fb82a1be1c11 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:17.583152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:17.612812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7c70d-56ba-4116-8321-877075453be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:17.613434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7c70d-56ba-4116-8321-877075453be9 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:17.613705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7c70d-56ba-4116-8321-877075453be9 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:17.613969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7c70d-56ba-4116-8321-877075453be9 None admin] PATH_INFO: `/v3/groups` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:17.614578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7c70d-56ba-4116-8321-877075453be9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:17.615990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7c70d-56ba-4116-8321-877075453be9 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:17.621706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99252fef-3221-4aeb-b565-40cb30ef3dc3 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:17.634057 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 115/232] 10.222.0.16 () {60 vars in 1134 bytes} [Sat Oct 15 20:32:17 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:32:17.635360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7c70d-56ba-4116-8321-877075453be9 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:17.676290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2161e9-cb24-4dcf-b3cf-8ab7aa5a070b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:17.676812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2161e9-cb24-4dcf-b3cf-8ab7aa5a070b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:17.677121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2161e9-cb24-4dcf-b3cf-8ab7aa5a070b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:17.677387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2161e9-cb24-4dcf-b3cf-8ab7aa5a070b None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:17.678266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2161e9-cb24-4dcf-b3cf-8ab7aa5a070b None admin] RBAC: Adding query filter params (name=reader) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:17.679485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2161e9-cb24-4dcf-b3cf-8ab7aa5a070b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:17.680919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2161e9-cb24-4dcf-b3cf-8ab7aa5a070b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:17.690325 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 118/233] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:17.696463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2161e9-cb24-4dcf-b3cf-8ab7aa5a070b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:17.762938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:17.765618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7c70d-56ba-4116-8321-877075453be9 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:17.806072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2161e9-cb24-4dcf-b3cf-8ab7aa5a070b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:17.811242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1fa5-dae4-427a-97e9-0078f953a2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:17.812291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1fa5-dae4-427a-97e9-0078f953a2ea None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:17.812748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1fa5-dae4-427a-97e9-0078f953a2ea None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:17.813221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1fa5-dae4-427a-97e9-0078f953a2ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:17.826075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf1fa5-dae4-427a-97e9-0078f953a2ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-4edf1fa5-dae4-427a-97e9-0078f953a2ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:17.832439 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:17.833379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:17.834142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:17.836437 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 116/234] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:17.841400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa24c35-27c4-4895-943a-725725812ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:17.842034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa24c35-27c4-4895-943a-725725812ba5 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:17.842307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa24c35-27c4-4895-943a-725725812ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:17.842558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa24c35-27c4-4895-943a-725725812ba5 None admin] PATH_INFO: `/v3/users/cc3228f89238446a90ed65bfad206297` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:17.842871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1fa5-dae4-427a-97e9-0078f953a2ea None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:17.859916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa24c35-27c4-4895-943a-725725812ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=cc3228f89238446a90ed65bfad206297)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:17.861709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa24c35-27c4-4895-943a-725725812ba5 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:17.864619 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 119/235] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:17 2022] GET /identity/v3/users/cc3228f89238446a90ed65bfad206297 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:17.871201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa24c35-27c4-4895-943a-725725812ba5 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:17.886625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:17.970588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1fa5-dae4-427a-97e9-0078f953a2ea None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:17.984029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa24c35-27c4-4895-943a-725725812ba5 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:18.006551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f4406-8852-41bb-84d1-09add3807d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:18.007029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:18.007150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f4406-8852-41bb-84d1-09add3807d7b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:18.007414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f4406-8852-41bb-84d1-09add3807d7b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:18.007710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f4406-8852-41bb-84d1-09add3807d7b None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:18.008714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f4406-8852-41bb-84d1-09add3807d7b None admin] RBAC: Adding query filter params (name=member) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:18.008714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f4406-8852-41bb-84d1-09add3807d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:18.010492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f4406-8852-41bb-84d1-09add3807d7b None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:18.019775 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 117/236] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:18.022637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b394da7-e5b3-4ac0-a3bc-5116643b67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:18.023261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b394da7-e5b3-4ac0-a3bc-5116643b67b8 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:18.023533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b394da7-e5b3-4ac0-a3bc-5116643b67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:18.023804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b394da7-e5b3-4ac0-a3bc-5116643b67b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:18.024614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b394da7-e5b3-4ac0-a3bc-5116643b67b8 None admin] RBAC: Adding query filter params (role.id=c303443fad7a458f8aa8aadd7bfc77e0, user.id=cc3228f89238446a90ed65bfad206297, scope.system=all) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:18.024885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b394da7-e5b3-4ac0-a3bc-5116643b67b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:18.025181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f4406-8852-41bb-84d1-09add3807d7b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:18.026364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b394da7-e5b3-4ac0-a3bc-5116643b67b8 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:18.031706 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 120/237] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Oct 15 20:32:17 2022] GET /identity/v3/role_assignments?user.id=cc3228f89238446a90ed65bfad206297&scope.system=all&role.id=c303443fad7a458f8aa8aadd7bfc77e0 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:18.135561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:18.149553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f4406-8852-41bb-84d1-09add3807d7b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:18.187303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3893fe-e44a-4de4-b992-734a50477d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:18.187948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3893fe-e44a-4de4-b992-734a50477d94 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:18.188219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3893fe-e44a-4de4-b992-734a50477d94 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:18.188483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3893fe-e44a-4de4-b992-734a50477d94 None admin] PATH_INFO: `/v3/users/4de71d4c56b1460aacd8a642ae5d8211` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:18.195305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3893fe-e44a-4de4-b992-734a50477d94 None admin] RBAC: Authorizing `identity:get_user(user_id=4de71d4c56b1460aacd8a642ae5d8211)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:18.195305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3893fe-e44a-4de4-b992-734a50477d94 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:18.195305 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 118/238] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:18 2022] GET /identity/v3/users/4de71d4c56b1460aacd8a642ae5d8211 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:18.199633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b394da7-e5b3-4ac0-a3bc-5116643b67b8 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:18.308075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b394da7-e5b3-4ac0-a3bc-5116643b67b8 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:18.342486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8166570-5f76-4d74-8961-dd32101567e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:18.343138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8166570-5f76-4d74-8961-dd32101567e6 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:18.343401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8166570-5f76-4d74-8961-dd32101567e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:18.343668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8166570-5f76-4d74-8961-dd32101567e6 None admin] PATH_INFO: `/v3/system/users/4de71d4c56b1460aacd8a642ae5d8211/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:18.345090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8166570-5f76-4d74-8961-dd32101567e6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4de71d4c56b1460aacd8a642ae5d8211, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:18.346485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8166570-5f76-4d74-8961-dd32101567e6 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:18.355913 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 121/239] 10.222.0.16 () {58 vars in 1334 bytes} [Sat Oct 15 20:32:18 2022] PUT /identity/v3/system/users/4de71d4c56b1460aacd8a642ae5d8211/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:32:20.058570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9aa68-8b63-403a-a6f6-501fb8aa1692 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:20.059084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9aa68-8b63-403a-a6f6-501fb8aa1692 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:20.059084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9aa68-8b63-403a-a6f6-501fb8aa1692 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:20.060554 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 119/240] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:32:20.065544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d7f88-aa4e-4aae-9a0f-093017028ec7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:20.065921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d7f88-aa4e-4aae-9a0f-093017028ec7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:20.066093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d7f88-aa4e-4aae-9a0f-093017028ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:20.085538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c38d7f88-aa4e-4aae-9a0f-093017028ec7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:20.086549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c38d7f88-aa4e-4aae-9a0f-093017028ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:20.134297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:20.161678 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 122/241] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:20.440666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62df8f-fc38-465b-ad7a-2424f1568573 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:20.440939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62df8f-fc38-465b-ad7a-2424f1568573 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:20.441442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62df8f-fc38-465b-ad7a-2424f1568573 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:20.442785 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 120/242] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:20.448806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d90cf-dcaf-4f82-b21d-aceae87094c9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:20.449110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d90cf-dcaf-4f82-b21d-aceae87094c9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:20.449384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d90cf-dcaf-4f82-b21d-aceae87094c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:20.468295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3b1d90cf-dcaf-4f82-b21d-aceae87094c9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:20.469481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3b1d90cf-dcaf-4f82-b21d-aceae87094c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:20.496031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62df8f-fc38-465b-ad7a-2424f1568573 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:20.544467 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 123/243] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:20.607460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62df8f-fc38-465b-ad7a-2424f1568573 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:20.641590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc458f34-ba3e-4411-ae02-12cc947b1223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:20.642213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc458f34-ba3e-4411-ae02-12cc947b1223 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:20.642484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc458f34-ba3e-4411-ae02-12cc947b1223 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:20.642750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc458f34-ba3e-4411-ae02-12cc947b1223 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:20.643828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc458f34-ba3e-4411-ae02-12cc947b1223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:20.645290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc458f34-ba3e-4411-ae02-12cc947b1223 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:20.646997 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 121/244] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:20.653129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d90cf-dcaf-4f82-b21d-aceae87094c9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:20.744348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fabbe3-6c5f-42de-bdbe-4bfcaf13fd08 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:20.744630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fabbe3-6c5f-42de-bdbe-4bfcaf13fd08 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:20.744790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fabbe3-6c5f-42de-bdbe-4bfcaf13fd08 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:20.746272 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 122/245] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:32:20.750972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc1352-f66f-4159-b003-29dee8d29eb5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:20.751227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc1352-f66f-4159-b003-29dee8d29eb5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:20.751493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc1352-f66f-4159-b003-29dee8d29eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:20.752746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d90cf-dcaf-4f82-b21d-aceae87094c9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:20.762500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:20.769566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8cfc1352-f66f-4159-b003-29dee8d29eb5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:20.770732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8cfc1352-f66f-4159-b003-29dee8d29eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:20.788993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:20.791381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef9ae3-1403-4e85-91ef-764c10d3ffc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:20.792081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef9ae3-1403-4e85-91ef-764c10d3ffc9 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:20.792355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef9ae3-1403-4e85-91ef-764c10d3ffc9 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:20.792621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef9ae3-1403-4e85-91ef-764c10d3ffc9 None admin] PATH_INFO: `/v3/groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:20.793276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef9ae3-1403-4e85-91ef-764c10d3ffc9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:20.794732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef9ae3-1403-4e85-91ef-764c10d3ffc9 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:20.813072 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 124/246] 10.222.0.16 () {60 vars in 1134 bytes} [Sat Oct 15 20:32:20 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:32:20.857562 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 123/247] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:20.873025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef9ae3-1403-4e85-91ef-764c10d3ffc9 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:20.982713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef9ae3-1403-4e85-91ef-764c10d3ffc9 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:21.019029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5880e-aa93-4183-8f19-51212e1524da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:21.019569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5880e-aa93-4183-8f19-51212e1524da None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:21.019843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5880e-aa93-4183-8f19-51212e1524da None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:21.020109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5880e-aa93-4183-8f19-51212e1524da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:21.032077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5880e-aa93-4183-8f19-51212e1524da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-f6d5880e-aa93-4183-8f19-51212e1524da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:21.037725 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:21.038569 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:21.039376 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 125/248] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:21.046581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc1352-f66f-4159-b003-29dee8d29eb5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:21.154423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc1352-f66f-4159-b003-29dee8d29eb5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:21.173855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5880e-aa93-4183-8f19-51212e1524da None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:21.200395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7c0c3-a75b-49e4-bdce-da1633b3a63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:21.200904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7c0c3-a75b-49e4-bdce-da1633b3a63b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:21.201260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7c0c3-a75b-49e4-bdce-da1633b3a63b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:21.201528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7c0c3-a75b-49e4-bdce-da1633b3a63b None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:21.202384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7c0c3-a75b-49e4-bdce-da1633b3a63b None admin] RBAC: Adding query filter params (name=reader) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:21.205944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7c0c3-a75b-49e4-bdce-da1633b3a63b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:21.205944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7c0c3-a75b-49e4-bdce-da1633b3a63b None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:21.214583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:21.215178 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 124/249] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:21.220962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7c0c3-a75b-49e4-bdce-da1633b3a63b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:21.243884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:21.312605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5880e-aa93-4183-8f19-51212e1524da None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:21.353291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7c0c3-a75b-49e4-bdce-da1633b3a63b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:21.354161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59208501-c296-4891-8428-372a7bb1e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:21.354951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208501-c296-4891-8428-372a7bb1e7be None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:21.355368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208501-c296-4891-8428-372a7bb1e7be None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:21.355805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208501-c296-4891-8428-372a7bb1e7be None admin] PATH_INFO: `/v3/roles/member` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:21.367501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59208501-c296-4891-8428-372a7bb1e7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-59208501-c296-4891-8428-372a7bb1e7be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:21.373260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:21.374180 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:21.374960 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:21.377376 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 126/250] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:32:21.383776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59208501-c296-4891-8428-372a7bb1e7be None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:21.403241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9861a6-943a-4a8b-9b76-c8b62f73e66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:21.404099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9861a6-943a-4a8b-9b76-c8b62f73e66a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:21.404410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9861a6-943a-4a8b-9b76-c8b62f73e66a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:21.404679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9861a6-943a-4a8b-9b76-c8b62f73e66a None admin] PATH_INFO: `/v3/users/cc3228f89238446a90ed65bfad206297` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:21.406637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9861a6-943a-4a8b-9b76-c8b62f73e66a None admin] RBAC: Authorizing `identity:get_user(user_id=cc3228f89238446a90ed65bfad206297)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:21.408158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9861a6-943a-4a8b-9b76-c8b62f73e66a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:21.411665 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 125/251] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:21 2022] GET /identity/v3/users/cc3228f89238446a90ed65bfad206297 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:21.420862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9861a6-943a-4a8b-9b76-c8b62f73e66a None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:21.529816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:21.530472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:21.546458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59208501-c296-4891-8428-372a7bb1e7be None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:21.549408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9861a6-943a-4a8b-9b76-c8b62f73e66a None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:21.588796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5241f-9024-461f-990a-f934e9ee3caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:21.589479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5241f-9024-461f-990a-f934e9ee3caf None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:21.589848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5241f-9024-461f-990a-f934e9ee3caf None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:21.590192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5241f-9024-461f-990a-f934e9ee3caf None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:21.590924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5241f-9024-461f-990a-f934e9ee3caf None admin] RBAC: Adding query filter params (name=member) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:21.591291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5241f-9024-461f-990a-f934e9ee3caf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:21.592593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff0c39-8e7f-42d4-870e-d9e49be4b697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:21.592758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5241f-9024-461f-990a-f934e9ee3caf None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:21.593337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff0c39-8e7f-42d4-870e-d9e49be4b697 None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:21.593675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff0c39-8e7f-42d4-870e-d9e49be4b697 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:21.593950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff0c39-8e7f-42d4-870e-d9e49be4b697 None admin] PATH_INFO: `/v3/system/users/cc3228f89238446a90ed65bfad206297/roles/c303443fad7a458f8aa8aadd7bfc77e0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:21.595957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff0c39-8e7f-42d4-870e-d9e49be4b697 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cc3228f89238446a90ed65bfad206297, role_id=c303443fad7a458f8aa8aadd7bfc77e0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:21.597347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff0c39-8e7f-42d4-870e-d9e49be4b697 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:21.604171 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 127/252] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:21.610347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5241f-9024-461f-990a-f934e9ee3caf None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:21.615610 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 126/253] 10.222.0.16 () {58 vars in 1334 bytes} [Sat Oct 15 20:32:21 2022] PUT /identity/v3/system/users/cc3228f89238446a90ed65bfad206297/roles/c303443fad7a458f8aa8aadd7bfc77e0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:32:21.744938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5241f-9024-461f-990a-f934e9ee3caf None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:21.784729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c18006-365c-4c92-99a4-8a916dced611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:21.785426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18006-365c-4c92-99a4-8a916dced611 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:21.785696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18006-365c-4c92-99a4-8a916dced611 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:21.785960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18006-365c-4c92-99a4-8a916dced611 None admin] PATH_INFO: `/v3/users/4de71d4c56b1460aacd8a642ae5d8211` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:21.787237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c18006-365c-4c92-99a4-8a916dced611 None admin] RBAC: Authorizing `identity:get_user(user_id=4de71d4c56b1460aacd8a642ae5d8211)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:21.788727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c18006-365c-4c92-99a4-8a916dced611 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:21.790444 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 128/254] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:21 2022] GET /identity/v3/users/4de71d4c56b1460aacd8a642ae5d8211 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:21.796485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff0c39-8e7f-42d4-870e-d9e49be4b697 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:21.924430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff0c39-8e7f-42d4-870e-d9e49be4b697 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:21.967921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0482a-b6e1-4f67-bb32-17558066f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:21.968522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0482a-b6e1-4f67-bb32-17558066f8e3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:21.968794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0482a-b6e1-4f67-bb32-17558066f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:21.969145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0482a-b6e1-4f67-bb32-17558066f8e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:21.969818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0482a-b6e1-4f67-bb32-17558066f8e3 None admin] RBAC: Adding query filter params (role.id=b5b286eb4fb94285914766a260e99693, user.id=4de71d4c56b1460aacd8a642ae5d8211, scope.system=all) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:21.970123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0482a-b6e1-4f67-bb32-17558066f8e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:21.971559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0482a-b6e1-4f67-bb32-17558066f8e3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:21.986382 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 127/255] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Oct 15 20:32:21 2022] GET /identity/v3/role_assignments?user.id=4de71d4c56b1460aacd8a642ae5d8211&scope.system=all&role.id=b5b286eb4fb94285914766a260e99693 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:23.965397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2a6c3-e607-4bbb-8706-099a5b977738 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:23.965915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2a6c3-e607-4bbb-8706-099a5b977738 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:23.965915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2a6c3-e607-4bbb-8706-099a5b977738 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:23.967511 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 129/256] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:23.973092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b8326-5323-4ebf-bee8-86ce1826a56f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:23.973497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b8326-5323-4ebf-bee8-86ce1826a56f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:23.973677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b8326-5323-4ebf-bee8-86ce1826a56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:23.991018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:23.998600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e92b8326-5323-4ebf-bee8-86ce1826a56f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:23.999855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e92b8326-5323-4ebf-bee8-86ce1826a56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:24.082768 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 128/257] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 20:32:24.707932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc51e8-bd95-4cc8-b3ef-b60260a57b45 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:24.708720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc51e8-bd95-4cc8-b3ef-b60260a57b45 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:24.709222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc51e8-bd95-4cc8-b3ef-b60260a57b45 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:24.711008 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 130/258] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:24.717717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2931-7a90-46fd-ab44-c3adbaeabc8a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:24.718238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2931-7a90-46fd-ab44-c3adbaeabc8a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:24.718714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2931-7a90-46fd-ab44-c3adbaeabc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:24.735999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17b2ec-e270-435c-b511-54d411d9084c None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:24.736573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17b2ec-e270-435c-b511-54d411d9084c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:24.737023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17b2ec-e270-435c-b511-54d411d9084c None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:24.738597 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 131/259] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:24.743996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abb753-25b1-457e-8edb-b107fe24fd88 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:24.744512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abb753-25b1-457e-8edb-b107fe24fd88 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:24.745021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abb753-25b1-457e-8edb-b107fe24fd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:24.759557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e51c2931-7a90-46fd-ab44-c3adbaeabc8a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:24.761227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e51c2931-7a90-46fd-ab44-c3adbaeabc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:24.776171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-91abb753-25b1-457e-8edb-b107fe24fd88 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:24.779110 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 129/260] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 20:32:24.781393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-91abb753-25b1-457e-8edb-b107fe24fd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:24.781806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c2931-7a90-46fd-ab44-c3adbaeabc8a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:24.800344 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 132/261] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 15 20:32:24.800850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c2931-7a90-46fd-ab44-c3adbaeabc8a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:24.808534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ed8f-0f5f-4e60-a545-474d22c7d109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:24.809566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ed8f-0f5f-4e60-a545-474d22c7d109 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:24.810050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ed8f-0f5f-4e60-a545-474d22c7d109 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:24.810507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ed8f-0f5f-4e60-a545-474d22c7d109 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:24.826736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267ed8f-0f5f-4e60-a545-474d22c7d109 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-b267ed8f-0f5f-4e60-a545-474d22c7d109 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:24.831798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:24.832617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:24.833411 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:24.835596 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 130/262] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Oct 15 20:32:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:24.845756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abb753-25b1-457e-8edb-b107fe24fd88 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:24.861355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abb753-25b1-457e-8edb-b107fe24fd88 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:24.874572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5cd78-669c-4294-9e8f-02fd0e365067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:24.875181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5cd78-669c-4294-9e8f-02fd0e365067 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:24.875450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5cd78-669c-4294-9e8f-02fd0e365067 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:24.875707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5cd78-669c-4294-9e8f-02fd0e365067 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:24.876442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5cd78-669c-4294-9e8f-02fd0e365067 admin admin] RBAC: Adding query filter params (name=member) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:24.876741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5cd78-669c-4294-9e8f-02fd0e365067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:24.878565 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-00c5cd78-669c-4294-9e8f-02fd0e365067 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:32:24.878565 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 20:32:24.878565 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 20:32:24.879485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5cd78-669c-4294-9e8f-02fd0e365067 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:24.894085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 133/263] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:32:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:24.901184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ed8f-0f5f-4e60-a545-474d22c7d109 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:24.917842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ed8f-0f5f-4e60-a545-474d22c7d109 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:24.929677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbdc00-8146-41b3-bb60-e0f768d9d211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:24.930310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbdc00-8146-41b3-bb60-e0f768d9d211 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:24.930667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbdc00-8146-41b3-bb60-e0f768d9d211 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:24.931012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbdc00-8146-41b3-bb60-e0f768d9d211 admin admin] PATH_INFO: `/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:24.943516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbdc00-8146-41b3-bb60-e0f768d9d211 admin admin] RBAC: Authorizing `identity:get_group(group_id=ad081b3cc6514a99a9d97d5cd2779b97)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:24.945309 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-0bfbdc00-8146-41b3-bb60-e0f768d9d211 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:32:24.945309 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 20:32:24.945309 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 20:32:24.947203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbdc00-8146-41b3-bb60-e0f768d9d211 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:24.950393 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 131/264] 10.222.0.16 () {56 vars in 1201 bytes} [Sat Oct 15 20:32:24 2022] GET /identity/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97 => generated 226 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:24.955016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5cd78-669c-4294-9e8f-02fd0e365067 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:24.961417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:24.970307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5cd78-669c-4294-9e8f-02fd0e365067 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:24.981327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e49378-cd0f-4b8e-af2b-06ba714f5a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:24.981967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49378-cd0f-4b8e-af2b-06ba714f5a70 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:24.982240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49378-cd0f-4b8e-af2b-06ba714f5a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:24.982506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49378-cd0f-4b8e-af2b-06ba714f5a70 admin admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:24.983651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e49378-cd0f-4b8e-af2b-06ba714f5a70 admin admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:24.985758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e49378-cd0f-4b8e-af2b-06ba714f5a70 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:24.987751 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 134/265] 10.222.0.16 () {56 vars in 1207 bytes} [Sat Oct 15 20:32:24 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:24.993569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbdc00-8146-41b3-bb60-e0f768d9d211 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.006192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbdc00-8146-41b3-bb60-e0f768d9d211 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.013587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a1e2a-3adb-457d-8050-0cf6e775f396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.014243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a1e2a-3adb-457d-8050-0cf6e775f396 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.014514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a1e2a-3adb-457d-8050-0cf6e775f396 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.014778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a1e2a-3adb-457d-8050-0cf6e775f396 admin admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/groups/ad081b3cc6514a99a9d97d5cd2779b97/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.022376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a1e2a-3adb-457d-8050-0cf6e775f396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb9dbb5eb664a45a354501eb93cb0cd, group_id=ad081b3cc6514a99a9d97d5cd2779b97, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:25.023394 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-700a1e2a-3adb-457d-8050-0cf6e775f396 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:32:25.023394 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 20:32:25.023394 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 20:32:25.026325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a1e2a-3adb-457d-8050-0cf6e775f396 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:25.049212 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 132/266] 10.222.0.16 () {58 vars in 1463 bytes} [Sat Oct 15 20:32:24 2022] PUT /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/groups/ad081b3cc6514a99a9d97d5cd2779b97/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:32:25.111392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7d9f0-eb05-456b-9d9b-d56daba5bba2 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.112111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7d9f0-eb05-456b-9d9b-d56daba5bba2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.112581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7d9f0-eb05-456b-9d9b-d56daba5bba2 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.114502 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 135/267] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:25.120452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763e4b7-6dc0-4256-ae87-801f3802ad7f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.120729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763e4b7-6dc0-4256-ae87-801f3802ad7f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.121060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763e4b7-6dc0-4256-ae87-801f3802ad7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.162090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7763e4b7-6dc0-4256-ae87-801f3802ad7f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:25.163303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7763e4b7-6dc0-4256-ae87-801f3802ad7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:25.272747 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 133/268] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:25.298483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7d9f0-eb05-456b-9d9b-d56daba5bba2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.384896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7d9f0-eb05-456b-9d9b-d56daba5bba2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.389707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f3441-e0f9-46b8-b7d8-0010f90e6c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.390578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f3441-e0f9-46b8-b7d8-0010f90e6c50 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.391054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f3441-e0f9-46b8-b7d8-0010f90e6c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.391529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f3441-e0f9-46b8-b7d8-0010f90e6c50 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.407491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f3441-e0f9-46b8-b7d8-0010f90e6c50 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:25.409287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763e4b7-6dc0-4256-ae87-801f3802ad7f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-950f3441-e0f9-46b8-b7d8-0010f90e6c50 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:25.414136 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:25.415435 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:25.416303 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:25.419294 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 136/269] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Oct 15 20:32:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 121 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:25.424549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f3441-e0f9-46b8-b7d8-0010f90e6c50 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.432739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763e4b7-6dc0-4256-ae87-801f3802ad7f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.437135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f3441-e0f9-46b8-b7d8-0010f90e6c50 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.443071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ad111-58e2-467c-b2c4-b98a7bb81580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.444051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ad111-58e2-467c-b2c4-b98a7bb81580 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.444570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ad111-58e2-467c-b2c4-b98a7bb81580 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.449897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd8d51-db1f-4cb5-9fad-598acf1eac86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.450573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd8d51-db1f-4cb5-9fad-598acf1eac86 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.450847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd8d51-db1f-4cb5-9fad-598acf1eac86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.451111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd8d51-db1f-4cb5-9fad-598acf1eac86 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.451744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ad111-58e2-467c-b2c4-b98a7bb81580 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.452559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ad111-58e2-467c-b2c4-b98a7bb81580 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:25.452857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ad111-58e2-467c-b2c4-b98a7bb81580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:25.454764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ad111-58e2-467c-b2c4-b98a7bb81580 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:25.470713 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 137/270] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Oct 15 20:32:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:25.474788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ad111-58e2-467c-b2c4-b98a7bb81580 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.476331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd8d51-db1f-4cb5-9fad-598acf1eac86 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-f0cd8d51-db1f-4cb5-9fad-598acf1eac86 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:25.481833 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:25.482721 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:25.483466 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:25.492530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ad111-58e2-467c-b2c4-b98a7bb81580 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.493589 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 134/271] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Oct 15 20:32:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 84 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:25.494770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd8d51-db1f-4cb5-9fad-598acf1eac86 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.509083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8664127-1836-49c4-bdfc-0debeba543c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.509954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8664127-1836-49c4-bdfc-0debeba543c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.510267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8664127-1836-49c4-bdfc-0debeba543c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.510477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8664127-1836-49c4-bdfc-0debeba543c6 admin admin] PATH_INFO: `/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.511780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8664127-1836-49c4-bdfc-0debeba543c6 admin admin] RBAC: Authorizing `identity:get_group(group_id=ad081b3cc6514a99a9d97d5cd2779b97)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:25.512815 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-e8664127-1836-49c4-bdfc-0debeba543c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:32:25.512815 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 20:32:25.512815 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 20:32:25.514052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8664127-1836-49c4-bdfc-0debeba543c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:25.515802 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 138/272] 10.222.0.16 () {56 vars in 1201 bytes} [Sat Oct 15 20:32:25 2022] GET /identity/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97 => generated 226 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:25.518049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd8d51-db1f-4cb5-9fad-598acf1eac86 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.521339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8664127-1836-49c4-bdfc-0debeba543c6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.533606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d80af-8a5a-46b5-9a90-2b9cf7e93c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.534157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d80af-8a5a-46b5-9a90-2b9cf7e93c35 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.534411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d80af-8a5a-46b5-9a90-2b9cf7e93c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.534675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d80af-8a5a-46b5-9a90-2b9cf7e93c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.535338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d80af-8a5a-46b5-9a90-2b9cf7e93c35 admin admin] RBAC: Adding query filter params (name=member) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:25.535629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d80af-8a5a-46b5-9a90-2b9cf7e93c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:25.536634 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-407d80af-8a5a-46b5-9a90-2b9cf7e93c35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:32:25.536634 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 20:32:25.536634 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 20:32:25.537727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d80af-8a5a-46b5-9a90-2b9cf7e93c35 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:25.549674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8664127-1836-49c4-bdfc-0debeba543c6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.555019 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 135/273] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:32:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:25.560627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d80af-8a5a-46b5-9a90-2b9cf7e93c35 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.565825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0737a-eff6-41d6-8e36-b5656566f023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.566444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0737a-eff6-41d6-8e36-b5656566f023 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.566720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0737a-eff6-41d6-8e36-b5656566f023 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.566986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0737a-eff6-41d6-8e36-b5656566f023 admin admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.568152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0737a-eff6-41d6-8e36-b5656566f023 admin admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:25.570294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0737a-eff6-41d6-8e36-b5656566f023 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:25.572523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d80af-8a5a-46b5-9a90-2b9cf7e93c35 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.572940 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 139/274] 10.222.0.16 () {56 vars in 1207 bytes} [Sat Oct 15 20:32:25 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:25.577596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32611ebf-5d7d-4a7e-8103-46e7390430c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.578398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32611ebf-5d7d-4a7e-8103-46e7390430c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.578580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0737a-eff6-41d6-8e36-b5656566f023 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.578966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32611ebf-5d7d-4a7e-8103-46e7390430c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.579378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32611ebf-5d7d-4a7e-8103-46e7390430c3 admin admin] PATH_INFO: `/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.581831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32611ebf-5d7d-4a7e-8103-46e7390430c3 admin admin] RBAC: Authorizing `identity:get_group(group_id=ad081b3cc6514a99a9d97d5cd2779b97)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:25.583915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32611ebf-5d7d-4a7e-8103-46e7390430c3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:25.585966 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 136/275] 10.222.0.16 () {56 vars in 1201 bytes} [Sat Oct 15 20:32:25 2022] GET /identity/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97 => generated 226 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:25.591726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32611ebf-5d7d-4a7e-8103-46e7390430c3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.594420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0737a-eff6-41d6-8e36-b5656566f023 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.601651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c90e51-94be-4a56-a619-e1a9d4636773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.602312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c90e51-94be-4a56-a619-e1a9d4636773 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.603116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c90e51-94be-4a56-a619-e1a9d4636773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.603537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c90e51-94be-4a56-a619-e1a9d4636773 admin admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/groups/ad081b3cc6514a99a9d97d5cd2779b97/roles/25b2c5582f194703afe45b5d8cc79171` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.604035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32611ebf-5d7d-4a7e-8103-46e7390430c3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.606038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c90e51-94be-4a56-a619-e1a9d4636773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb9dbb5eb664a45a354501eb93cb0cd, group_id=ad081b3cc6514a99a9d97d5cd2779b97, role_id=25b2c5582f194703afe45b5d8cc79171)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:25.608739 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-f3c90e51-94be-4a56-a619-e1a9d4636773 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:32:25.608739 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 20:32:25.608739 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 20:32:25.609012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c90e51-94be-4a56-a619-e1a9d4636773 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:25.612282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf4af1-d594-495b-b3bc-9af5d0133c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.613175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf4af1-d594-495b-b3bc-9af5d0133c58 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.613657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf4af1-d594-495b-b3bc-9af5d0133c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.614191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf4af1-d594-495b-b3bc-9af5d0133c58 admin admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.616929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf4af1-d594-495b-b3bc-9af5d0133c58 admin admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:25.620612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf4af1-d594-495b-b3bc-9af5d0133c58 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:25.623175 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 137/276] 10.222.0.16 () {56 vars in 1207 bytes} [Sat Oct 15 20:32:25 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:25.629179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf4af1-d594-495b-b3bc-9af5d0133c58 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.639652 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 140/277] 10.222.0.16 () {58 vars in 1463 bytes} [Sat Oct 15 20:32:25 2022] PUT /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd/groups/ad081b3cc6514a99a9d97d5cd2779b97/roles/25b2c5582f194703afe45b5d8cc79171 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:32:25.645748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf4af1-d594-495b-b3bc-9af5d0133c58 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.669292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:25.712143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5f2d6-f5de-4c8a-9727-1562cbc6bb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.713516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5f2d6-f5de-4c8a-9727-1562cbc6bb39 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.714140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5f2d6-f5de-4c8a-9727-1562cbc6bb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.714484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5f2d6-f5de-4c8a-9727-1562cbc6bb39 admin admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b/groups/ad081b3cc6514a99a9d97d5cd2779b97/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.717053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5f2d6-f5de-4c8a-9727-1562cbc6bb39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d040a120d1b4214a79694db0340276b, group_id=ad081b3cc6514a99a9d97d5cd2779b97, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:25.720105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5f2d6-f5de-4c8a-9727-1562cbc6bb39 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:25.742301 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 138/278] 10.222.0.16 () {58 vars in 1463 bytes} [Sat Oct 15 20:32:25 2022] PUT /identity/v3/projects/6d040a120d1b4214a79694db0340276b/groups/ad081b3cc6514a99a9d97d5cd2779b97/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:32:25.775684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c90e51-94be-4a56-a619-e1a9d4636773 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:25.900537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c90e51-94be-4a56-a619-e1a9d4636773 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:25.913743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:25.917016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99544271-aa08-4d32-aaca-6a0d47989b95 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.917360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99544271-aa08-4d32-aaca-6a0d47989b95 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.917629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99544271-aa08-4d32-aaca-6a0d47989b95 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.918982 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 139/279] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:25.924811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52767-4b32-4fa0-afa4-bcfa2938c069 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.925159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52767-4b32-4fa0-afa4-bcfa2938c069 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.925429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52767-4b32-4fa0-afa4-bcfa2938c069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.948793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3fa52767-4b32-4fa0-afa4-bcfa2938c069 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:25.950207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3fa52767-4b32-4fa0-afa4-bcfa2938c069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:25.962358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ddb7e-0e85-40ab-89ce-093567dc4b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:25.963293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ddb7e-0e85-40ab-89ce-093567dc4b0b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:25.963569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ddb7e-0e85-40ab-89ce-093567dc4b0b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:25.963834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ddb7e-0e85-40ab-89ce-093567dc4b0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:25.982108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ddb7e-0e85-40ab-89ce-093567dc4b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-4c8ddb7e-0e85-40ab-89ce-093567dc4b0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:25.987439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:25.988294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:25.989089 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 141/280] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:25.996780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ddb7e-0e85-40ab-89ce-093567dc4b0b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:26.020861 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 140/281] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 20:32:26.161499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ddb7e-0e85-40ab-89ce-093567dc4b0b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:26.206574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71063e01-86eb-45f2-a328-2cefecb4e9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:26.207191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71063e01-86eb-45f2-a328-2cefecb4e9ca None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:26.207464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71063e01-86eb-45f2-a328-2cefecb4e9ca None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:26.207731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71063e01-86eb-45f2-a328-2cefecb4e9ca None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:26.208412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71063e01-86eb-45f2-a328-2cefecb4e9ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:26.208712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71063e01-86eb-45f2-a328-2cefecb4e9ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:26.210155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71063e01-86eb-45f2-a328-2cefecb4e9ca None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:26.222729 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 142/282] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:26.231541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52767-4b32-4fa0-afa4-bcfa2938c069 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:26.371924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:26.377709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52767-4b32-4fa0-afa4-bcfa2938c069 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:26.416179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68a791-3cd0-4aa0-80db-340e070c239c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:26.416723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68a791-3cd0-4aa0-80db-340e070c239c None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:26.416997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68a791-3cd0-4aa0-80db-340e070c239c None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:26.417293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68a791-3cd0-4aa0-80db-340e070c239c None admin] PATH_INFO: `/v3/users/cc3228f89238446a90ed65bfad206297` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:26.418419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68a791-3cd0-4aa0-80db-340e070c239c None admin] RBAC: Authorizing `identity:get_user(user_id=cc3228f89238446a90ed65bfad206297)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:26.419986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68a791-3cd0-4aa0-80db-340e070c239c None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:26.421934 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 141/283] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:32:26 2022] GET /identity/v3/users/cc3228f89238446a90ed65bfad206297 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:26.430364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71063e01-86eb-45f2-a328-2cefecb4e9ca None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:26.493772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68a791-3cd0-4aa0-80db-340e070c239c None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:26.497693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:26.513489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68a791-3cd0-4aa0-80db-340e070c239c None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:26.523386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d04bd1-344e-496a-99b4-776a7dca7b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:26.524036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d04bd1-344e-496a-99b4-776a7dca7b9a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:26.524507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d04bd1-344e-496a-99b4-776a7dca7b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:26.524889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d04bd1-344e-496a-99b4-776a7dca7b9a admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:26.546664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d04bd1-344e-496a-99b4-776a7dca7b9a admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-61d04bd1-344e-496a-99b4-776a7dca7b9a admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:26.551535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:26.552406 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:26.553211 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 142/284] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Oct 15 20:32:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 60 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:32:26.563086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d04bd1-344e-496a-99b4-776a7dca7b9a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:26.580650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d04bd1-344e-496a-99b4-776a7dca7b9a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:26.593746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dff738-47f1-445f-99a9-82477f1765f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:26.594359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dff738-47f1-445f-99a9-82477f1765f6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:26.594635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dff738-47f1-445f-99a9-82477f1765f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:26.594900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dff738-47f1-445f-99a9-82477f1765f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:26.595572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dff738-47f1-445f-99a9-82477f1765f6 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:26.595867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dff738-47f1-445f-99a9-82477f1765f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:26.597503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dff738-47f1-445f-99a9-82477f1765f6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:26.611205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 143/285] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Oct 15 20:32:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:26.617376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dff738-47f1-445f-99a9-82477f1765f6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:26.630850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dff738-47f1-445f-99a9-82477f1765f6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:26.635226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71063e01-86eb-45f2-a328-2cefecb4e9ca None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:26.650176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714c569-00e6-4ef4-b1e0-24acbd958c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:26.650834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714c569-00e6-4ef4-b1e0-24acbd958c5a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:26.651113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714c569-00e6-4ef4-b1e0-24acbd958c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:26.651390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714c569-00e6-4ef4-b1e0-24acbd958c5a admin admin] PATH_INFO: `/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:26.652599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714c569-00e6-4ef4-b1e0-24acbd958c5a admin admin] RBAC: Authorizing `identity:get_group(group_id=ad081b3cc6514a99a9d97d5cd2779b97)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:26.654564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714c569-00e6-4ef4-b1e0-24acbd958c5a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:26.656142 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 144/286] 10.222.0.16 () {56 vars in 1201 bytes} [Sat Oct 15 20:32:26 2022] GET /identity/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97 => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:26.662624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714c569-00e6-4ef4-b1e0-24acbd958c5a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:26.678970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714c569-00e6-4ef4-b1e0-24acbd958c5a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:26.691207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f164d-722c-446c-854e-7680bc1ab338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:26.691828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f164d-722c-446c-854e-7680bc1ab338 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:26.692135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f164d-722c-446c-854e-7680bc1ab338 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:26.692367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f164d-722c-446c-854e-7680bc1ab338 admin admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:26.696359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35798492-b2b2-4c4d-b465-bc8cd4d32a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:26.697321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798492-b2b2-4c4d-b465-bc8cd4d32a28 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:26.697600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798492-b2b2-4c4d-b465-bc8cd4d32a28 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:26.697866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798492-b2b2-4c4d-b465-bc8cd4d32a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:26.697969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f164d-722c-446c-854e-7680bc1ab338 admin admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:26.698595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35798492-b2b2-4c4d-b465-bc8cd4d32a28 None admin] RBAC: Adding query filter params (role.id=c303443fad7a458f8aa8aadd7bfc77e0, user.id=cc3228f89238446a90ed65bfad206297, scope.system=all) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:26.698896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35798492-b2b2-4c4d-b465-bc8cd4d32a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:26.699988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f164d-722c-446c-854e-7680bc1ab338 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:26.700343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35798492-b2b2-4c4d-b465-bc8cd4d32a28 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:26.701926 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 145/287] 10.222.0.16 () {56 vars in 1207 bytes} [Sat Oct 15 20:32:26 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:26.707959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f164d-722c-446c-854e-7680bc1ab338 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:26.722707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f164d-722c-446c-854e-7680bc1ab338 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:26.724734 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 143/288] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Oct 15 20:32:26 2022] GET /identity/v3/role_assignments?user.id=cc3228f89238446a90ed65bfad206297&scope.system=all&role.id=c303443fad7a458f8aa8aadd7bfc77e0 => generated 510 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:26.734465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf173f7-bcf8-4fba-9915-c312f61a36f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:26.735139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf173f7-bcf8-4fba-9915-c312f61a36f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:26.735416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf173f7-bcf8-4fba-9915-c312f61a36f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:26.735684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf173f7-bcf8-4fba-9915-c312f61a36f4 admin admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b/groups/ad081b3cc6514a99a9d97d5cd2779b97/roles/25b2c5582f194703afe45b5d8cc79171` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:26.743824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf173f7-bcf8-4fba-9915-c312f61a36f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d040a120d1b4214a79694db0340276b, group_id=ad081b3cc6514a99a9d97d5cd2779b97, role_id=25b2c5582f194703afe45b5d8cc79171)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:26.746512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf173f7-bcf8-4fba-9915-c312f61a36f4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:26.771350 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 146/289] 10.222.0.16 () {58 vars in 1463 bytes} [Sat Oct 15 20:32:26 2022] PUT /identity/v3/projects/6d040a120d1b4214a79694db0340276b/groups/ad081b3cc6514a99a9d97d5cd2779b97/roles/25b2c5582f194703afe45b5d8cc79171 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:32:27.210979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647795d-b2fc-404b-9913-327299c61630 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:27.211498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647795d-b2fc-404b-9913-327299c61630 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:27.211498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647795d-b2fc-404b-9913-327299c61630 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:27.214359 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 144/290] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:27.218672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc67bd-1378-4b31-a61b-02a1dc00ae70 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:27.219115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc67bd-1378-4b31-a61b-02a1dc00ae70 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:27.219195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc67bd-1378-4b31-a61b-02a1dc00ae70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:27.250524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-20cc67bd-1378-4b31-a61b-02a1dc00ae70 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:27.251734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-20cc67bd-1378-4b31-a61b-02a1dc00ae70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:27.322290 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 147/291] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 20:32:27.730391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647795d-b2fc-404b-9913-327299c61630 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:27.746315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647795d-b2fc-404b-9913-327299c61630 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:27.753649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d06d8-a283-4797-853f-ccd727beea62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:27.754278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d06d8-a283-4797-853f-ccd727beea62 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:27.754549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d06d8-a283-4797-853f-ccd727beea62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:27.754818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d06d8-a283-4797-853f-ccd727beea62 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:27.770437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d06d8-a283-4797-853f-ccd727beea62 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-f98d06d8-a283-4797-853f-ccd727beea62 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:27.777216 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:27.778066 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:32:27.778822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:27.779401 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 145/292] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Oct 15 20:32:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:27.784016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc67bd-1378-4b31-a61b-02a1dc00ae70 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:27.798123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc67bd-1378-4b31-a61b-02a1dc00ae70 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:27.802644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2748e-bbe4-42a5-93d9-c75ecc4e96b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:27.803235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2748e-bbe4-42a5-93d9-c75ecc4e96b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:27.803504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2748e-bbe4-42a5-93d9-c75ecc4e96b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:27.803764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2748e-bbe4-42a5-93d9-c75ecc4e96b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:27.804419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2748e-bbe4-42a5-93d9-c75ecc4e96b3 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:27.804712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2748e-bbe4-42a5-93d9-c75ecc4e96b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:27.806713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2748e-bbe4-42a5-93d9-c75ecc4e96b3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:27.820705 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 148/293] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Oct 15 20:32:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:27.827143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d06d8-a283-4797-853f-ccd727beea62 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:27.837673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d06d8-a283-4797-853f-ccd727beea62 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:27.844971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454524f9-f938-4c27-8d49-2c58e627d46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:27.845616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454524f9-f938-4c27-8d49-2c58e627d46d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:27.845889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454524f9-f938-4c27-8d49-2c58e627d46d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:27.846150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454524f9-f938-4c27-8d49-2c58e627d46d admin admin] PATH_INFO: `/v3/groups/72428733dd174d4e91a967a8b789fd2e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:27.854531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454524f9-f938-4c27-8d49-2c58e627d46d admin admin] RBAC: Authorizing `identity:get_group(group_id=72428733dd174d4e91a967a8b789fd2e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:27.856491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454524f9-f938-4c27-8d49-2c58e627d46d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:27.858197 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 146/294] 10.222.0.16 () {56 vars in 1201 bytes} [Sat Oct 15 20:32:27 2022] GET /identity/v3/groups/72428733dd174d4e91a967a8b789fd2e => generated 229 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:27.863756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2748e-bbe4-42a5-93d9-c75ecc4e96b3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:27.878035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2748e-bbe4-42a5-93d9-c75ecc4e96b3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:27.890163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f442f1-22ea-450f-9a7f-fa82a20a357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:27.890788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f442f1-22ea-450f-9a7f-fa82a20a357a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:27.891059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f442f1-22ea-450f-9a7f-fa82a20a357a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:27.891322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f442f1-22ea-450f-9a7f-fa82a20a357a admin admin] PATH_INFO: `/v3/projects/38e66271254b4380bfa7d7e1ccea54cc` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:27.892445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f442f1-22ea-450f-9a7f-fa82a20a357a admin admin] RBAC: Authorizing `identity:get_project(project_id=38e66271254b4380bfa7d7e1ccea54cc)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:27.894241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f442f1-22ea-450f-9a7f-fa82a20a357a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:27.895910 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 149/295] 10.222.0.16 () {56 vars in 1207 bytes} [Sat Oct 15 20:32:27 2022] GET /identity/v3/projects/38e66271254b4380bfa7d7e1ccea54cc => generated 344 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:27.902152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454524f9-f938-4c27-8d49-2c58e627d46d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:27.919441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454524f9-f938-4c27-8d49-2c58e627d46d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:27.929486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621b23f-90cd-4584-9608-7d972a150085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:27.930157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621b23f-90cd-4584-9608-7d972a150085 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:27.930691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621b23f-90cd-4584-9608-7d972a150085 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:27.931089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621b23f-90cd-4584-9608-7d972a150085 admin admin] PATH_INFO: `/v3/projects/38e66271254b4380bfa7d7e1ccea54cc/groups/72428733dd174d4e91a967a8b789fd2e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:27.935575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621b23f-90cd-4584-9608-7d972a150085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e66271254b4380bfa7d7e1ccea54cc, group_id=72428733dd174d4e91a967a8b789fd2e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:27.938277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621b23f-90cd-4584-9608-7d972a150085 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:27.954928 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 147/296] 10.222.0.16 () {58 vars in 1463 bytes} [Sat Oct 15 20:32:27 2022] PUT /identity/v3/projects/38e66271254b4380bfa7d7e1ccea54cc/groups/72428733dd174d4e91a967a8b789fd2e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:32:29.146675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bbe3d-27ea-4532-83ae-c39f8ec04e2e None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:29.147190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bbe3d-27ea-4532-83ae-c39f8ec04e2e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:29.147190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bbe3d-27ea-4532-83ae-c39f8ec04e2e None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:29.148536 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 150/297] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:29.167086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb3df1-f7ea-4d89-82f2-c0b7aa0d8e07 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:29.167596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb3df1-f7ea-4d89-82f2-c0b7aa0d8e07 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:29.167596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb3df1-f7ea-4d89-82f2-c0b7aa0d8e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:29.197534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3dcb3df1-f7ea-4d89-82f2-c0b7aa0d8e07 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:29.198729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3dcb3df1-f7ea-4d89-82f2-c0b7aa0d8e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:29.242393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83c540-6944-42f6-b38d-57ef7a2f5de3 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:29.242720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83c540-6944-42f6-b38d-57ef7a2f5de3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:29.243698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83c540-6944-42f6-b38d-57ef7a2f5de3 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:29.245421 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 151/298] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:32:29.250946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3441a61-b1f0-4a98-b772-1bf65957e273 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:29.251231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3441a61-b1f0-4a98-b772-1bf65957e273 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:29.251487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3441a61-b1f0-4a98-b772-1bf65957e273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:29.284184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f3441a61-b1f0-4a98-b772-1bf65957e273 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:29.288696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f3441a61-b1f0-4a98-b772-1bf65957e273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:29.301388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:29.329841 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 148/299] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:29.374988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:29.391545 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 152/300] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:29.440146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503eeb1-1460-49cf-9660-def9682c59b0 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:29.440324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503eeb1-1460-49cf-9660-def9682c59b0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:29.440605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503eeb1-1460-49cf-9660-def9682c59b0 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:29.442007 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 149/301] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:29.448126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e03693-84d2-4e01-8995-4e916c80e81b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:29.448595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e03693-84d2-4e01-8995-4e916c80e81b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:29.449000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e03693-84d2-4e01-8995-4e916c80e81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:29.478730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-45e03693-84d2-4e01-8995-4e916c80e81b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:29.480128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-45e03693-84d2-4e01-8995-4e916c80e81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:29.565593 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 153/302] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:29.783532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503eeb1-1460-49cf-9660-def9682c59b0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:29.879757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e03693-84d2-4e01-8995-4e916c80e81b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:29.921832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503eeb1-1460-49cf-9660-def9682c59b0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:29.974905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d5c47-a029-45b4-a3a7-78bb81d5e3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:29.975538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d5c47-a029-45b4-a3a7-78bb81d5e3f7 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:29.975816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d5c47-a029-45b4-a3a7-78bb81d5e3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:29.976082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d5c47-a029-45b4-a3a7-78bb81d5e3f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:30.004377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d5c47-a029-45b4-a3a7-78bb81d5e3f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-5e1d5c47-a029-45b4-a3a7-78bb81d5e3f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:30.009564 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:30.010428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:30.011593 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:30.012172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e03693-84d2-4e01-8995-4e916c80e81b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:30.012251 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 150/303] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:32:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:30.022020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d5c47-a029-45b4-a3a7-78bb81d5e3f7 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:30.040916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:30.064551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12edff-cabb-44ab-8446-62e86eb30ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:30.065264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12edff-cabb-44ab-8446-62e86eb30ef2 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:30.065538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12edff-cabb-44ab-8446-62e86eb30ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:30.065804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12edff-cabb-44ab-8446-62e86eb30ef2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:30.096511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12edff-cabb-44ab-8446-62e86eb30ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-0c12edff-cabb-44ab-8446-62e86eb30ef2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:30.104290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:30.105222 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:30.105979 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 154/304] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:30.107667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:30.114998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12edff-cabb-44ab-8446-62e86eb30ef2 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:30.187792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d5c47-a029-45b4-a3a7-78bb81d5e3f7 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:30.237259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9c549-126e-47b2-92ab-e8fd59dcea19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:30.237866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9c549-126e-47b2-92ab-e8fd59dcea19 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:30.238141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9c549-126e-47b2-92ab-e8fd59dcea19 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:30.238410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9c549-126e-47b2-92ab-e8fd59dcea19 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:30.242151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9c549-126e-47b2-92ab-e8fd59dcea19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:30.242633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9c549-126e-47b2-92ab-e8fd59dcea19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:30.244931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9c549-126e-47b2-92ab-e8fd59dcea19 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:30.254958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12edff-cabb-44ab-8446-62e86eb30ef2 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:30.257447 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 151/305] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:30.264153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9c549-126e-47b2-92ab-e8fd59dcea19 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:30.298576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c43f7-3052-43de-b8e9-923082f63f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:30.299405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c43f7-3052-43de-b8e9-923082f63f50 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:30.300181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c43f7-3052-43de-b8e9-923082f63f50 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:30.300748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c43f7-3052-43de-b8e9-923082f63f50 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:30.301687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c43f7-3052-43de-b8e9-923082f63f50 None admin] RBAC: Adding query filter params (name=member) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:30.302130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c43f7-3052-43de-b8e9-923082f63f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:30.303645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c43f7-3052-43de-b8e9-923082f63f50 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:30.314612 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 155/306] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:30.315659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c43f7-3052-43de-b8e9-923082f63f50 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:30.361002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:30.385841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9c549-126e-47b2-92ab-e8fd59dcea19 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:30.397636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:30.435004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c43f7-3052-43de-b8e9-923082f63f50 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:30.435482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde9540-94f0-464b-b99b-6c476a7663d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:30.436115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde9540-94f0-464b-b99b-6c476a7663d8 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:30.436441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde9540-94f0-464b-b99b-6c476a7663d8 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:30.436655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde9540-94f0-464b-b99b-6c476a7663d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:30.453295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde9540-94f0-464b-b99b-6c476a7663d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-bdde9540-94f0-464b-b99b-6c476a7663d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:30.458988 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:30.460134 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 20:32:30.461063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:30.463771 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 152/307] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:30.465157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93386d13-64c9-4764-8064-bbc36b810bfe None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:30.465719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93386d13-64c9-4764-8064-bbc36b810bfe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:30.466056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93386d13-64c9-4764-8064-bbc36b810bfe None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:30.467358 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 153/308] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:30.468233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93386d13-64c9-4764-8064-bbc36b810bfe None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:30.480283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85ab1a-0943-4bdd-af18-e5171482c775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:30.480886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85ab1a-0943-4bdd-af18-e5171482c775 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:30.481203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85ab1a-0943-4bdd-af18-e5171482c775 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:30.481470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85ab1a-0943-4bdd-af18-e5171482c775 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:30.572862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85ab1a-0943-4bdd-af18-e5171482c775 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:30.575036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85ab1a-0943-4bdd-af18-e5171482c775 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:30.606714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93386d13-64c9-4764-8064-bbc36b810bfe None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:30.634230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:30.649864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d397d-9ea5-4c5c-b03d-36d2e9a25210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:30.650468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d397d-9ea5-4c5c-b03d-36d2e9a25210 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:30.650743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d397d-9ea5-4c5c-b03d-36d2e9a25210 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:30.651361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d397d-9ea5-4c5c-b03d-36d2e9a25210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:30.690320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:30.710395 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 156/309] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:30.712259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85ab1a-0943-4bdd-af18-e5171482c775 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:30.731486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d397d-9ea5-4c5c-b03d-36d2e9a25210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:30.733063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d397d-9ea5-4c5c-b03d-36d2e9a25210 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:30.843844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85ab1a-0943-4bdd-af18-e5171482c775 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:30.870156 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 154/310] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:30.871807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c21f6-ed58-4471-b237-48a51123705b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:30.872322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c21f6-ed58-4471-b237-48a51123705b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:30.872765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c21f6-ed58-4471-b237-48a51123705b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:30.887058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410281-bb03-43ac-a5c8-4abccdb080b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:30.887662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410281-bb03-43ac-a5c8-4abccdb080b3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:30.887939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410281-bb03-43ac-a5c8-4abccdb080b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:30.888204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410281-bb03-43ac-a5c8-4abccdb080b3 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:30.888864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410281-bb03-43ac-a5c8-4abccdb080b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:30.889194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410281-bb03-43ac-a5c8-4abccdb080b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:30.890706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410281-bb03-43ac-a5c8-4abccdb080b3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:30.894662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c47c21f6-ed58-4471-b237-48a51123705b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:30.895871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c47c21f6-ed58-4471-b237-48a51123705b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:30.902759 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 157/311] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:32:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:30.903832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410281-bb03-43ac-a5c8-4abccdb080b3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:30.907027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:30.983473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:30.991662 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 155/312] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:30.992905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c21f6-ed58-4471-b237-48a51123705b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:31.024627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410281-bb03-43ac-a5c8-4abccdb080b3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:31.062661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72c952-1a02-4fae-8ff4-f02a64474d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:31.063627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72c952-1a02-4fae-8ff4-f02a64474d53 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.064101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72c952-1a02-4fae-8ff4-f02a64474d53 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.064579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72c952-1a02-4fae-8ff4-f02a64474d53 None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.066863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72c952-1a02-4fae-8ff4-f02a64474d53 None admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:31.068333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72c952-1a02-4fae-8ff4-f02a64474d53 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:31.070375 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 158/313] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:30 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:31.072163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72c952-1a02-4fae-8ff4-f02a64474d53 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:31.125410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c21f6-ed58-4471-b237-48a51123705b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:31.156252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:31.176275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882bb63-dfb9-477b-9147-2bc2da2b5624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:31.177269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882bb63-dfb9-477b-9147-2bc2da2b5624 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.177745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882bb63-dfb9-477b-9147-2bc2da2b5624 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.178859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882bb63-dfb9-477b-9147-2bc2da2b5624 None admin] PATH_INFO: `/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.181021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882bb63-dfb9-477b-9147-2bc2da2b5624 None admin] RBAC: Authorizing `identity:get_project(project_id=abb9dbb5eb664a45a354501eb93cb0cd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:31.183581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882bb63-dfb9-477b-9147-2bc2da2b5624 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:31.186941 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 156/314] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:30 2022] GET /identity/v3/projects/abb9dbb5eb664a45a354501eb93cb0cd => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:31.188640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882bb63-dfb9-477b-9147-2bc2da2b5624 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:31.200634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72c952-1a02-4fae-8ff4-f02a64474d53 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:31.247357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9daca-4dc9-4b01-97af-65be15343968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:31.248023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9daca-4dc9-4b01-97af-65be15343968 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.248299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9daca-4dc9-4b01-97af-65be15343968 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.248569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9daca-4dc9-4b01-97af-65be15343968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.280422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:31.318108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882bb63-dfb9-477b-9147-2bc2da2b5624 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:31.331120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9daca-4dc9-4b01-97af-65be15343968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:31.332577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9daca-4dc9-4b01-97af-65be15343968 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:31.358685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6692e6a-8bed-45ef-807a-fae4382b2111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:31.360096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6692e6a-8bed-45ef-807a-fae4382b2111 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.360537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6692e6a-8bed-45ef-807a-fae4382b2111 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.360985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6692e6a-8bed-45ef-807a-fae4382b2111 None admin] PATH_INFO: `/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.362250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6692e6a-8bed-45ef-807a-fae4382b2111 None admin] RBAC: Authorizing `identity:get_group(group_id=ad081b3cc6514a99a9d97d5cd2779b97)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:31.363830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6692e6a-8bed-45ef-807a-fae4382b2111 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:31.366527 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 157/315] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Oct 15 20:32:31 2022] GET /identity/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:31.367496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6692e6a-8bed-45ef-807a-fae4382b2111 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:31.411305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:31.449842 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 159/316] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:31.456698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97636952-dd61-4baa-969a-8a4206151b7a None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.456974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97636952-dd61-4baa-969a-8a4206151b7a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.457265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97636952-dd61-4baa-969a-8a4206151b7a None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.458566 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 160/317] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:31.464122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97636952-dd61-4baa-969a-8a4206151b7a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:31.487837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6692e6a-8bed-45ef-807a-fae4382b2111 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:31.521640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e275438-1a2c-4bdb-ae0e-1f053e1f9015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:31.522243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e275438-1a2c-4bdb-ae0e-1f053e1f9015 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.522518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e275438-1a2c-4bdb-ae0e-1f053e1f9015 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.522795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e275438-1a2c-4bdb-ae0e-1f053e1f9015 None admin] PATH_INFO: `/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.524689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e275438-1a2c-4bdb-ae0e-1f053e1f9015 None admin] RBAC: Authorizing `identity:get_group(group_id=ad081b3cc6514a99a9d97d5cd2779b97)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:31.526164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e275438-1a2c-4bdb-ae0e-1f053e1f9015 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:31.527716 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 158/318] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Oct 15 20:32:31 2022] GET /identity/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:31.528471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e275438-1a2c-4bdb-ae0e-1f053e1f9015 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:31.530929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:31.572776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97636952-dd61-4baa-969a-8a4206151b7a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:31.610078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9403d-a37f-46a7-80ec-e5f239be941f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:31.610675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9403d-a37f-46a7-80ec-e5f239be941f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.610947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9403d-a37f-46a7-80ec-e5f239be941f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.611211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9403d-a37f-46a7-80ec-e5f239be941f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.611918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9403d-a37f-46a7-80ec-e5f239be941f None admin] RBAC: Adding query filter params (group.id=ad081b3cc6514a99a9d97d5cd2779b97, role.id=25b2c5582f194703afe45b5d8cc79171, scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:31.612223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9403d-a37f-46a7-80ec-e5f239be941f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:31.613666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9403d-a37f-46a7-80ec-e5f239be941f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:31.619667 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 161/319] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Oct 15 20:32:31 2022] GET /identity/v3/role_assignments?group.id=ad081b3cc6514a99a9d97d5cd2779b97&scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd&role.id=25b2c5582f194703afe45b5d8cc79171 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:31.620446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9403d-a37f-46a7-80ec-e5f239be941f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:31.652533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e275438-1a2c-4bdb-ae0e-1f053e1f9015 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:31.660630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:31.693891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b7620-3823-4e87-b5cd-0a8fb2043130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:31.694448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b7620-3823-4e87-b5cd-0a8fb2043130 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.694724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b7620-3823-4e87-b5cd-0a8fb2043130 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.694989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b7620-3823-4e87-b5cd-0a8fb2043130 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.706687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b7620-3823-4e87-b5cd-0a8fb2043130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-0f5b7620-3823-4e87-b5cd-0a8fb2043130 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:31.712211 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:31.713470 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 159/320] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:32:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:31.714259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6932-600e-425f-9df2-1aeac133aff9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.714942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6932-600e-425f-9df2-1aeac133aff9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.714942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6932-600e-425f-9df2-1aeac133aff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.734290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-02bb6932-600e-425f-9df2-1aeac133aff9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:31.735552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-02bb6932-600e-425f-9df2-1aeac133aff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:31.753427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9403d-a37f-46a7-80ec-e5f239be941f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:31.799414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79a0c6-4f1b-4e61-b7f0-68d98c540f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:31.800062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79a0c6-4f1b-4e61-b7f0-68d98c540f4b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.800334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79a0c6-4f1b-4e61-b7f0-68d98c540f4b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.800618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79a0c6-4f1b-4e61-b7f0-68d98c540f4b None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.801859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79a0c6-4f1b-4e61-b7f0-68d98c540f4b None admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:31.802374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:31.803312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79a0c6-4f1b-4e61-b7f0-68d98c540f4b None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:31.805741 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 162/321] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:31 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:31.806619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79a0c6-4f1b-4e61-b7f0-68d98c540f4b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:31.820083 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 160/322] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:31.822509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb6932-600e-425f-9df2-1aeac133aff9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:31.931664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79a0c6-4f1b-4e61-b7f0-68d98c540f4b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:31.933788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:31.954914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb6932-600e-425f-9df2-1aeac133aff9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:31.970194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2ea87-a32d-4525-b133-383db89fb598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:31.971173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2ea87-a32d-4525-b133-383db89fb598 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.971706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2ea87-a32d-4525-b133-383db89fb598 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.971973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2ea87-a32d-4525-b133-383db89fb598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.972692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2ea87-a32d-4525-b133-383db89fb598 None admin] RBAC: Adding query filter params (group.id=ad081b3cc6514a99a9d97d5cd2779b97, role.id=b5b286eb4fb94285914766a260e99693, scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:31.972982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2ea87-a32d-4525-b133-383db89fb598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:31.974427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2ea87-a32d-4525-b133-383db89fb598 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:31.981394 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 163/323] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Oct 15 20:32:31 2022] GET /identity/v3/role_assignments?group.id=ad081b3cc6514a99a9d97d5cd2779b97&scope.project.id=abb9dbb5eb664a45a354501eb93cb0cd&role.id=b5b286eb4fb94285914766a260e99693 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:31.982649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2ea87-a32d-4525-b133-383db89fb598 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:31.994804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb04f1-8f28-434e-a240-d2c6bdb38ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:31.995537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb04f1-8f28-434e-a240-d2c6bdb38ffd None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:31.995898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb04f1-8f28-434e-a240-d2c6bdb38ffd None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:31.996317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb04f1-8f28-434e-a240-d2c6bdb38ffd None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:31.997110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb04f1-8f28-434e-a240-d2c6bdb38ffd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:31.997481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb04f1-8f28-434e-a240-d2c6bdb38ffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:31.999085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb04f1-8f28-434e-a240-d2c6bdb38ffd None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:32.010291 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 161/324] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:32.017712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb04f1-8f28-434e-a240-d2c6bdb38ffd None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:32.096067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:32.143092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2ea87-a32d-4525-b133-383db89fb598 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:32.149402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb04f1-8f28-434e-a240-d2c6bdb38ffd None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:32.186093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6794a05-27f2-41d5-91af-c59098e9e1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:32.186719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6794a05-27f2-41d5-91af-c59098e9e1d3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:32.186990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6794a05-27f2-41d5-91af-c59098e9e1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:32.187247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6794a05-27f2-41d5-91af-c59098e9e1d3 None admin] PATH_INFO: `/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:32.188332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6794a05-27f2-41d5-91af-c59098e9e1d3 None admin] RBAC: Authorizing `identity:get_group(group_id=ad081b3cc6514a99a9d97d5cd2779b97)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:32.189797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6794a05-27f2-41d5-91af-c59098e9e1d3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:32.191372 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 164/325] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Oct 15 20:32:31 2022] GET /identity/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:32.192141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6794a05-27f2-41d5-91af-c59098e9e1d3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:32.201167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fd73c-4e60-4a16-b8fe-0222ed45b27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:32.202501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fd73c-4e60-4a16-b8fe-0222ed45b27a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:32.202988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fd73c-4e60-4a16-b8fe-0222ed45b27a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:32.203255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fd73c-4e60-4a16-b8fe-0222ed45b27a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:32.237518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:32.288806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fd73c-4e60-4a16-b8fe-0222ed45b27a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:32.290978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fd73c-4e60-4a16-b8fe-0222ed45b27a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:32.314022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6794a05-27f2-41d5-91af-c59098e9e1d3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:32.354466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:32.360888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9cb4b-7462-48c4-9e4f-039ce9e60425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:32.361658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9cb4b-7462-48c4-9e4f-039ce9e60425 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:32.361932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9cb4b-7462-48c4-9e4f-039ce9e60425 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:32.362197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9cb4b-7462-48c4-9e4f-039ce9e60425 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:32.373724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9cb4b-7462-48c4-9e4f-039ce9e60425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-60f9cb4b-7462-48c4-9e4f-039ce9e60425 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:32.379259 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:32.380191 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:32.380960 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 165/326] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:32:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:32.386137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9cb4b-7462-48c4-9e4f-039ce9e60425 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:32.408144 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 162/327] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:32.408999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fd73c-4e60-4a16-b8fe-0222ed45b27a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:32.509583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:32.518743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9cb4b-7462-48c4-9e4f-039ce9e60425 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:32.528222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fd73c-4e60-4a16-b8fe-0222ed45b27a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:32.561931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ce3b5-d87d-4ea9-9544-0ce00b6a7aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:32.562528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ce3b5-d87d-4ea9-9544-0ce00b6a7aa8 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:32.562801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ce3b5-d87d-4ea9-9544-0ce00b6a7aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:32.563066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ce3b5-d87d-4ea9-9544-0ce00b6a7aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:32.563780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ce3b5-d87d-4ea9-9544-0ce00b6a7aa8 None admin] RBAC: Adding query filter params (group.id=ad081b3cc6514a99a9d97d5cd2779b97, role.id=b5b286eb4fb94285914766a260e99693, scope.project.id=6d040a120d1b4214a79694db0340276b) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:32.564082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ce3b5-d87d-4ea9-9544-0ce00b6a7aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:32.565600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ce3b5-d87d-4ea9-9544-0ce00b6a7aa8 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:32.572388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84966267-0dc7-4f23-a6c5-2c8fa961d3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:32.572928 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 166/328] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Oct 15 20:32:32 2022] GET /identity/v3/role_assignments?group.id=ad081b3cc6514a99a9d97d5cd2779b97&scope.project.id=6d040a120d1b4214a79694db0340276b&role.id=b5b286eb4fb94285914766a260e99693 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:32.573084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966267-0dc7-4f23-a6c5-2c8fa961d3ae None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:32.573322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966267-0dc7-4f23-a6c5-2c8fa961d3ae None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:32.573592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966267-0dc7-4f23-a6c5-2c8fa961d3ae None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:32.573845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ce3b5-d87d-4ea9-9544-0ce00b6a7aa8 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:32.574357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84966267-0dc7-4f23-a6c5-2c8fa961d3ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:32.574662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84966267-0dc7-4f23-a6c5-2c8fa961d3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:32.576071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84966267-0dc7-4f23-a6c5-2c8fa961d3ae None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:32.588010 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 163/329] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:32.595330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84966267-0dc7-4f23-a6c5-2c8fa961d3ae None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:32.626601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:32.699743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ce3b5-d87d-4ea9-9544-0ce00b6a7aa8 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:32.719779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84966267-0dc7-4f23-a6c5-2c8fa961d3ae None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:32.740176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb691a-90e7-40c5-a57d-ecbe0325355f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:32.740811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb691a-90e7-40c5-a57d-ecbe0325355f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:32.741115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb691a-90e7-40c5-a57d-ecbe0325355f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:32.741387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb691a-90e7-40c5-a57d-ecbe0325355f None admin] PATH_INFO: `/v3/projects/6d040a120d1b4214a79694db0340276b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:32.744005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb691a-90e7-40c5-a57d-ecbe0325355f None admin] RBAC: Authorizing `identity:get_project(project_id=6d040a120d1b4214a79694db0340276b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:32.745547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb691a-90e7-40c5-a57d-ecbe0325355f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:32.747479 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 167/330] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:32 2022] GET /identity/v3/projects/6d040a120d1b4214a79694db0340276b => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:32.752855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb691a-90e7-40c5-a57d-ecbe0325355f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:32.758127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e835b-2c9c-43b4-b67e-abf7a6f77528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:32.758793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e835b-2c9c-43b4-b67e-abf7a6f77528 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:32.759064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e835b-2c9c-43b4-b67e-abf7a6f77528 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:32.759325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e835b-2c9c-43b4-b67e-abf7a6f77528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:32.785279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:32.837647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e835b-2c9c-43b4-b67e-abf7a6f77528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:32.839578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e835b-2c9c-43b4-b67e-abf7a6f77528 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:32.863630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:32.882666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb691a-90e7-40c5-a57d-ecbe0325355f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:32.922647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0648bb-252a-412e-8e6d-2d6eccd23ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:32.923407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0648bb-252a-412e-8e6d-2d6eccd23ceb None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:32.923678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0648bb-252a-412e-8e6d-2d6eccd23ceb None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:32.924483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0648bb-252a-412e-8e6d-2d6eccd23ceb None admin] PATH_INFO: `/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:32.925105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0648bb-252a-412e-8e6d-2d6eccd23ceb None admin] RBAC: Authorizing `identity:get_group(group_id=ad081b3cc6514a99a9d97d5cd2779b97)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:32.926676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0648bb-252a-412e-8e6d-2d6eccd23ceb None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:32.928000 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 168/331] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Oct 15 20:32:32 2022] GET /identity/v3/groups/ad081b3cc6514a99a9d97d5cd2779b97 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:32.934638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0648bb-252a-412e-8e6d-2d6eccd23ceb None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:32.962011 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 164/332] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:32.968337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e835b-2c9c-43b4-b67e-abf7a6f77528 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:33.042547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:33.049549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0648bb-252a-412e-8e6d-2d6eccd23ceb None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:33.073584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e835b-2c9c-43b4-b67e-abf7a6f77528 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:33.082329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab7c5a-7c36-41c2-b0b4-96d426af0808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:33.082914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab7c5a-7c36-41c2-b0b4-96d426af0808 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:33.083184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab7c5a-7c36-41c2-b0b4-96d426af0808 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:33.083447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab7c5a-7c36-41c2-b0b4-96d426af0808 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:33.084150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab7c5a-7c36-41c2-b0b4-96d426af0808 None admin] RBAC: Adding query filter params (group.id=ad081b3cc6514a99a9d97d5cd2779b97, role.id=25b2c5582f194703afe45b5d8cc79171, scope.project.id=6d040a120d1b4214a79694db0340276b) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:33.084445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab7c5a-7c36-41c2-b0b4-96d426af0808 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:33.085996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab7c5a-7c36-41c2-b0b4-96d426af0808 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:33.090051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:33.091856 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 169/333] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Oct 15 20:32:32 2022] GET /identity/v3/role_assignments?group.id=ad081b3cc6514a99a9d97d5cd2779b97&scope.project.id=6d040a120d1b4214a79694db0340276b&role.id=25b2c5582f194703afe45b5d8cc79171 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:33.109303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c1295-8152-4deb-8670-8d96816ba138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:33.109948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c1295-8152-4deb-8670-8d96816ba138 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:33.110220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c1295-8152-4deb-8670-8d96816ba138 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:33.110483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c1295-8152-4deb-8670-8d96816ba138 None admin] PATH_INFO: `/v3/projects/38e66271254b4380bfa7d7e1ccea54cc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:33.111662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c1295-8152-4deb-8670-8d96816ba138 None admin] RBAC: Authorizing `identity:get_project(project_id=38e66271254b4380bfa7d7e1ccea54cc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:33.113192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c1295-8152-4deb-8670-8d96816ba138 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:33.114918 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 165/334] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:32:32 2022] GET /identity/v3/projects/38e66271254b4380bfa7d7e1ccea54cc => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:33.121629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab7c5a-7c36-41c2-b0b4-96d426af0808 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:33.229308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab7c5a-7c36-41c2-b0b4-96d426af0808 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:33.264127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3f022-1558-476e-a093-466590623924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:33.264753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3f022-1558-476e-a093-466590623924 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:33.265024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3f022-1558-476e-a093-466590623924 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:33.265319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3f022-1558-476e-a093-466590623924 None admin] PATH_INFO: `/v3/groups/72428733dd174d4e91a967a8b789fd2e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:33.266443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3f022-1558-476e-a093-466590623924 None admin] RBAC: Authorizing `identity:get_group(group_id=72428733dd174d4e91a967a8b789fd2e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:33.267865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3f022-1558-476e-a093-466590623924 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:33.269505 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 170/335] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Oct 15 20:32:33 2022] GET /identity/v3/groups/72428733dd174d4e91a967a8b789fd2e => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:33.276219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c1295-8152-4deb-8670-8d96816ba138 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:33.379365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c1295-8152-4deb-8670-8d96816ba138 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:33.414431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c1c97-3d62-4cb0-8399-9d75f9e30f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:33.415044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c1c97-3d62-4cb0-8399-9d75f9e30f14 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:33.415315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c1c97-3d62-4cb0-8399-9d75f9e30f14 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:33.415579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c1c97-3d62-4cb0-8399-9d75f9e30f14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:33.416295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c1c97-3d62-4cb0-8399-9d75f9e30f14 None admin] RBAC: Adding query filter params (group.id=72428733dd174d4e91a967a8b789fd2e, role.id=fab66a1a6ba64c9583182c78978b218e, scope.project.id=38e66271254b4380bfa7d7e1ccea54cc) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:33.416596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c1c97-3d62-4cb0-8399-9d75f9e30f14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:33.418074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c1c97-3d62-4cb0-8399-9d75f9e30f14 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:33.424105 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 166/336] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Oct 15 20:32:33 2022] GET /identity/v3/role_assignments?group.id=72428733dd174d4e91a967a8b789fd2e&scope.project.id=38e66271254b4380bfa7d7e1ccea54cc&role.id=fab66a1a6ba64c9583182c78978b218e => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:36.596632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a8ae4-6bed-4204-aa41-775808eb00df None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:36.597665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a8ae4-6bed-4204-aa41-775808eb00df None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:36.598143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a8ae4-6bed-4204-aa41-775808eb00df None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:36.599933 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 171/337] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:36.605982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca33ccc-8cd4-4929-905e-04ef5e4b0746 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:36.606844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca33ccc-8cd4-4929-905e-04ef5e4b0746 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:36.607332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca33ccc-8cd4-4929-905e-04ef5e4b0746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:36.630923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7ca33ccc-8cd4-4929-905e-04ef5e4b0746 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:36.631606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fec520-e0bb-4969-b7ad-b93bb6cdc29a None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:36.632115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fec520-e0bb-4969-b7ad-b93bb6cdc29a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:36.632473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7ca33ccc-8cd4-4929-905e-04ef5e4b0746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:36.632780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fec520-e0bb-4969-b7ad-b93bb6cdc29a None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:36.635316 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 172/338] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:36.644258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d5299-0251-4aa9-a037-c86305db6b1d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:36.644850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d5299-0251-4aa9-a037-c86305db6b1d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:36.645426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d5299-0251-4aa9-a037-c86305db6b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:36.672987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5a0d5299-0251-4aa9-a037-c86305db6b1d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:36.674203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5a0d5299-0251-4aa9-a037-c86305db6b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:36.702033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:36.726204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:36.761794 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 167/339] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:36.783458 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 173/340] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:36.997685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9da24-57fc-4253-a80d-fe708db196d9 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:36.998207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9da24-57fc-4253-a80d-fe708db196d9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:36.998621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9da24-57fc-4253-a80d-fe708db196d9 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:37.000132 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 168/341] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:37.005346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd435b6c-ae06-46bf-aa2a-864c5d04369d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:37.005846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd435b6c-ae06-46bf-aa2a-864c5d04369d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:37.006255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd435b6c-ae06-46bf-aa2a-864c5d04369d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:37.033971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bd435b6c-ae06-46bf-aa2a-864c5d04369d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:37.035386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bd435b6c-ae06-46bf-aa2a-864c5d04369d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:37.127592 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 174/342] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:37.129994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9da24-57fc-4253-a80d-fe708db196d9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:37.209668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0fac-8b58-4150-b500-0ca264251465 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:37.209956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0fac-8b58-4150-b500-0ca264251465 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:37.210400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0fac-8b58-4150-b500-0ca264251465 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:37.212264 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 175/343] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:37.214805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de0fac-8b58-4150-b500-0ca264251465 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:37.246715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9da24-57fc-4253-a80d-fe708db196d9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:37.299145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25d826-5984-4e9a-80a3-aa0688b265fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:37.299994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25d826-5984-4e9a-80a3-aa0688b265fb None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:37.300408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25d826-5984-4e9a-80a3-aa0688b265fb None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:37.301271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25d826-5984-4e9a-80a3-aa0688b265fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:37.310103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:37.318648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25d826-5984-4e9a-80a3-aa0688b265fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:37.321120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25d826-5984-4e9a-80a3-aa0688b265fb None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:37.327395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de0fac-8b58-4150-b500-0ca264251465 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-6b25d826-5984-4e9a-80a3-aa0688b265fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:37.341390 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:37.342340 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:37.348389 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:37.348389 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:37.348389 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:37.348389 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 169/344] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:37.348389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990901b-4b93-45fc-b9f9-1fb6f0594d6f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:37.348389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990901b-4b93-45fc-b9f9-1fb6f0594d6f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:37.348389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990901b-4b93-45fc-b9f9-1fb6f0594d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:37.368289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e990901b-4b93-45fc-b9f9-1fb6f0594d6f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:37.370011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e990901b-4b93-45fc-b9f9-1fb6f0594d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:37.372736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde813ca-37a8-4fa5-a693-09b972c73c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:37.373640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde813ca-37a8-4fa5-a693-09b972c73c1e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:37.374105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde813ca-37a8-4fa5-a693-09b972c73c1e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:37.374567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde813ca-37a8-4fa5-a693-09b972c73c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:37.386046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde813ca-37a8-4fa5-a693-09b972c73c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:37.387738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde813ca-37a8-4fa5-a693-09b972c73c1e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:37.400034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-dde813ca-37a8-4fa5-a693-09b972c73c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:37.404071 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:37.404919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:37.405791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:37.405791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:37.405791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:37.407848 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 176/345] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:37.408961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde813ca-37a8-4fa5-a693-09b972c73c1e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:37.460359 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 170/346] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:37.461864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990901b-4b93-45fc-b9f9-1fb6f0594d6f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:37.535194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde813ca-37a8-4fa5-a693-09b972c73c1e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:37.574608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96053b5-ed2a-4fa0-8152-7655c09a494e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:37.575532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96053b5-ed2a-4fa0-8152-7655c09a494e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:37.575995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96053b5-ed2a-4fa0-8152-7655c09a494e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:37.576450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96053b5-ed2a-4fa0-8152-7655c09a494e None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:37.577322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96053b5-ed2a-4fa0-8152-7655c09a494e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:37.577823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96053b5-ed2a-4fa0-8152-7655c09a494e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:37.579431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96053b5-ed2a-4fa0-8152-7655c09a494e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:37.588363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:37.595330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990901b-4b93-45fc-b9f9-1fb6f0594d6f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:37.595692 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 177/347] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:37.601244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96053b5-ed2a-4fa0-8152-7655c09a494e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:37.641549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998875b6-009f-4e86-b7ce-fdc6b130f502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:37.642193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998875b6-009f-4e86-b7ce-fdc6b130f502 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:37.642424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998875b6-009f-4e86-b7ce-fdc6b130f502 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:37.642698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998875b6-009f-4e86-b7ce-fdc6b130f502 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:37.643336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998875b6-009f-4e86-b7ce-fdc6b130f502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:37.643635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998875b6-009f-4e86-b7ce-fdc6b130f502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:37.645464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998875b6-009f-4e86-b7ce-fdc6b130f502 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:37.670361 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 171/348] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:37.676193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998875b6-009f-4e86-b7ce-fdc6b130f502 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:37.705858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:37.728904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96053b5-ed2a-4fa0-8152-7655c09a494e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:37.777575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10c14d-39c0-4860-a4ff-5e4d54cd7a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:37.778886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10c14d-39c0-4860-a4ff-5e4d54cd7a30 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:37.779355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10c14d-39c0-4860-a4ff-5e4d54cd7a30 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:37.780093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10c14d-39c0-4860-a4ff-5e4d54cd7a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:37.795218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10c14d-39c0-4860-a4ff-5e4d54cd7a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:37.797587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10c14d-39c0-4860-a4ff-5e4d54cd7a30 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:37.799938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998875b6-009f-4e86-b7ce-fdc6b130f502 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-3c10c14d-39c0-4860-a4ff-5e4d54cd7a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:37.818328 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:37.819235 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:37.820130 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:37.820130 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:37.820130 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:37.823757 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 178/349] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:37.825413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10c14d-39c0-4860-a4ff-5e4d54cd7a30 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:37.837874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f7e33-d066-4b8d-9b4a-c055958d2be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:37.838224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f7e33-d066-4b8d-9b4a-c055958d2be3 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:37.838491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f7e33-d066-4b8d-9b4a-c055958d2be3 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:37.838753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f7e33-d066-4b8d-9b4a-c055958d2be3 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:37.839378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f7e33-d066-4b8d-9b4a-c055958d2be3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:37.840921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f7e33-d066-4b8d-9b4a-c055958d2be3 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:37.873069 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 172/350] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:32:37 2022] POST /identity/v3/users => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:32:37.873069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f7e33-d066-4b8d-9b4a-c055958d2be3 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:37.900667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:37.963184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10c14d-39c0-4860-a4ff-5e4d54cd7a30 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:37.983903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:38.002890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e72ca-1bfe-4d95-bd49-5b6f4ff0ee68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:38.003494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e72ca-1bfe-4d95-bd49-5b6f4ff0ee68 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:38.003768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e72ca-1bfe-4d95-bd49-5b6f4ff0ee68 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:38.004036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e72ca-1bfe-4d95-bd49-5b6f4ff0ee68 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:38.004690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e72ca-1bfe-4d95-bd49-5b6f4ff0ee68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:38.006200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e72ca-1bfe-4d95-bd49-5b6f4ff0ee68 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:38.011747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f7e33-d066-4b8d-9b4a-c055958d2be3 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:38.030093 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 179/351] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:32:37 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:32:38.034396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e72ca-1bfe-4d95-bd49-5b6f4ff0ee68 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:38.056370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b8ed5-c2db-4434-b141-095206a913cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:38.057261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b8ed5-c2db-4434-b141-095206a913cf None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:38.057668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b8ed5-c2db-4434-b141-095206a913cf None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:38.058098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b8ed5-c2db-4434-b141-095206a913cf None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:38.058925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b8ed5-c2db-4434-b141-095206a913cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:38.059374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b8ed5-c2db-4434-b141-095206a913cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:38.061259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b8ed5-c2db-4434-b141-095206a913cf None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:38.076494 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 173/352] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:38.077960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a17c0d-3b8b-42f3-8afb-f595c080340a None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:38.078287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a17c0d-3b8b-42f3-8afb-f595c080340a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:38.078654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a17c0d-3b8b-42f3-8afb-f595c080340a None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:38.080070 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 174/353] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:38.085610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a17c0d-3b8b-42f3-8afb-f595c080340a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:38.182107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e72ca-1bfe-4d95-bd49-5b6f4ff0ee68 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:38.191558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:38.230515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f0dae-1b54-430b-813f-659a6109c405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:38.231741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f0dae-1b54-430b-813f-659a6109c405 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:38.232236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f0dae-1b54-430b-813f-659a6109c405 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:38.232642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f0dae-1b54-430b-813f-659a6109c405 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:38.240417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a17c0d-3b8b-42f3-8afb-f595c080340a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:38.246455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f0dae-1b54-430b-813f-659a6109c405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:38.248272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f0dae-1b54-430b-813f-659a6109c405 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-3f6f0dae-1b54-430b-813f-659a6109c405 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:38.265742 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:38.266841 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:38.267868 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:38.267868 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:38.267868 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:38.269577 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 180/354] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:38.271546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480f47e-1d5c-4565-8bae-4b5e2f0c1239 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:38.272220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480f47e-1d5c-4565-8bae-4b5e2f0c1239 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:38.272627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480f47e-1d5c-4565-8bae-4b5e2f0c1239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:38.283929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73632eba-409a-440c-b16b-0c539b53ec5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:38.284526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632eba-409a-440c-b16b-0c539b53ec5e None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:38.284795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632eba-409a-440c-b16b-0c539b53ec5e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:38.285078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632eba-409a-440c-b16b-0c539b53ec5e None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:38.285719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73632eba-409a-440c-b16b-0c539b53ec5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:38.287224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73632eba-409a-440c-b16b-0c539b53ec5e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:38.294294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a480f47e-1d5c-4565-8bae-4b5e2f0c1239 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:38.295697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a480f47e-1d5c-4565-8bae-4b5e2f0c1239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:38.298038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:38.311764 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 175/355] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:32:38 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:32:38.314403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73632eba-409a-440c-b16b-0c539b53ec5e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:38.412797 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 181/356] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:38.456292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73632eba-409a-440c-b16b-0c539b53ec5e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:38.510426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa77437-02b5-49a0-897d-52c65de9932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:38.511416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa77437-02b5-49a0-897d-52c65de9932f None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:38.511904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa77437-02b5-49a0-897d-52c65de9932f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:38.512362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa77437-02b5-49a0-897d-52c65de9932f None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:38.513258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa77437-02b5-49a0-897d-52c65de9932f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:38.513770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa77437-02b5-49a0-897d-52c65de9932f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:38.515362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa77437-02b5-49a0-897d-52c65de9932f None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:38.528556 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 176/357] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:38.534302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480f47e-1d5c-4565-8bae-4b5e2f0c1239 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:38.625749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:38.638859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480f47e-1d5c-4565-8bae-4b5e2f0c1239 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:38.676094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e6265-943c-4127-ad5a-298036272a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:38.677014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e6265-943c-4127-ad5a-298036272a33 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:38.677517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e6265-943c-4127-ad5a-298036272a33 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:38.677966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e6265-943c-4127-ad5a-298036272a33 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:38.678796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e6265-943c-4127-ad5a-298036272a33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:38.680960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e6265-943c-4127-ad5a-298036272a33 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:38.704746 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 182/358] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:32:38 2022] POST /identity/v3/users => generated 249 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:32:38.803229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa77437-02b5-49a0-897d-52c65de9932f None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:38.864485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:38.920116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa77437-02b5-49a0-897d-52c65de9932f None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:38.957702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd96c4f-35f8-4b2d-8b6d-618eda855153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:38.958228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd96c4f-35f8-4b2d-8b6d-618eda855153 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:38.958499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd96c4f-35f8-4b2d-8b6d-618eda855153 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:38.958823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd96c4f-35f8-4b2d-8b6d-618eda855153 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:38.959499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd96c4f-35f8-4b2d-8b6d-618eda855153 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:38.960878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd96c4f-35f8-4b2d-8b6d-618eda855153 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:38.970820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.sql.core [None req-fbd96c4f-35f8-4b2d-8b6d-618eda855153 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 20:32:38.970820 ncdv-master-1 devstack@keystone.service[99100]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 15 20:32:38.970820 ncdv-master-1 devstack@keystone.service[99100]: [parameters: {'id': '329abaae758444118ec2674f2c1a7573', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 15 20:32:38.970820 ncdv-master-1 devstack@keystone.service[99100]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99100) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-fbd96c4f-35f8-4b2d-8b6d-618eda855153 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application conn.query(q) Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result.read() Oct 15 20:32:38.987359 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return ref.to_dict() Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 20:32:38.988229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application next(self.gen) Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application yield resource Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application next(self.gen) Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise self.value Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application session.commit() Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 20:32:38.989164 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.session.flush() Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application compat.raise_( Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 20:32:38.989985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:32:38.990837 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application conn.query(q) Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result.read() Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:32:38.991691 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [parameters: {'id': '329abaae758444118ec2674f2c1a7573', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:38.995781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 15 20:32:38.996828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:38.997863 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 177/359] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Oct 15 20:32:38 2022] POST /identity/v3/roles => generated 144 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:38.998120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e6265-943c-4127-ad5a-298036272a33 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:39.122310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e6265-943c-4127-ad5a-298036272a33 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:39.171498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0af8b1-965f-489e-aba4-da3df1d3a4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:39.172523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0af8b1-965f-489e-aba4-da3df1d3a4c1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:39.172992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0af8b1-965f-489e-aba4-da3df1d3a4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:39.173477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0af8b1-965f-489e-aba4-da3df1d3a4c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:39.189892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0af8b1-965f-489e-aba4-da3df1d3a4c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-fd0af8b1-965f-489e-aba4-da3df1d3a4c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:39.197702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:39.198560 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:39.199352 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 183/360] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:32:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:39.205112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd96c4f-35f8-4b2d-8b6d-618eda855153 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:39.318872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd96c4f-35f8-4b2d-8b6d-618eda855153 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:39.336746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:39.356063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86477903-002f-45bd-8a30-5a76a1961aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:39.356787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86477903-002f-45bd-8a30-5a76a1961aa3 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:39.357144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86477903-002f-45bd-8a30-5a76a1961aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:39.357391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86477903-002f-45bd-8a30-5a76a1961aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:39.358401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86477903-002f-45bd-8a30-5a76a1961aa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:39.359119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86477903-002f-45bd-8a30-5a76a1961aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:39.360540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86477903-002f-45bd-8a30-5a76a1961aa3 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:39.369627 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 178/361] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:40.884366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35db96-07d3-4ce5-82df-8fa4e8ea1eca None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:40.884915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35db96-07d3-4ce5-82df-8fa4e8ea1eca None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:40.884915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35db96-07d3-4ce5-82df-8fa4e8ea1eca None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:40.886500 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 184/362] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:40.898564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f784736-1e03-4d7e-8608-851c8ef9f31c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:40.899267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f784736-1e03-4d7e-8608-851c8ef9f31c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:40.899267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f784736-1e03-4d7e-8608-851c8ef9f31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:40.924995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4f784736-1e03-4d7e-8608-851c8ef9f31c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:40.926197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4f784736-1e03-4d7e-8608-851c8ef9f31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:41.006114 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 179/363] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:41.111644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9081b-08ae-4014-a56b-3a993a01eea4 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:41.111998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9081b-08ae-4014-a56b-3a993a01eea4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:41.112180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9081b-08ae-4014-a56b-3a993a01eea4 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:41.113846 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 185/364] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:41.119567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09898eb-6e69-4474-8cee-cc2b2c9aec24 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:41.119851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09898eb-6e69-4474-8cee-cc2b2c9aec24 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:41.120113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09898eb-6e69-4474-8cee-cc2b2c9aec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:41.145510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a09898eb-6e69-4474-8cee-cc2b2c9aec24 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:41.146635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a09898eb-6e69-4474-8cee-cc2b2c9aec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:41.243348 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 180/365] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:41.406650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba5214-779e-44d4-8043-6f45bb0674d3 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:41.406984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba5214-779e-44d4-8043-6f45bb0674d3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:41.407096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba5214-779e-44d4-8043-6f45bb0674d3 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:41.408847 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 186/366] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:41.413920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c204a-589f-4265-b8e7-7b080b3b78ed None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:41.414196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c204a-589f-4265-b8e7-7b080b3b78ed None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:41.414465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c204a-589f-4265-b8e7-7b080b3b78ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:41.435487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c61c204a-589f-4265-b8e7-7b080b3b78ed None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:41.436649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c61c204a-589f-4265-b8e7-7b080b3b78ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:41.460845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba5214-779e-44d4-8043-6f45bb0674d3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:41.470127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:41.489389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:41.542305 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 181/367] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:41.624913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba5214-779e-44d4-8043-6f45bb0674d3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:41.674668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886bf6b-22d1-452b-b49b-785800cf8fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:41.675217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886bf6b-22d1-452b-b49b-785800cf8fed None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:41.675491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886bf6b-22d1-452b-b49b-785800cf8fed None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:41.675753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886bf6b-22d1-452b-b49b-785800cf8fed None admin] PATH_INFO: `/v3/roles/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:41.685237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c204a-589f-4265-b8e7-7b080b3b78ed None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:41.699243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886bf6b-22d1-452b-b49b-785800cf8fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-3886bf6b-22d1-452b-b49b-785800cf8fed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:41.704060 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:41.704914 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:41.705741 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 187/368] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:32:41.714345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886bf6b-22d1-452b-b49b-785800cf8fed None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:41.813991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:41.864185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c204a-589f-4265-b8e7-7b080b3b78ed None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:41.911658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886bf6b-22d1-452b-b49b-785800cf8fed None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:41.914547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0902f4a-c0f2-480f-a14c-e7b02a9f3261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:41.915277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902f4a-c0f2-480f-a14c-e7b02a9f3261 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:41.915550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902f4a-c0f2-480f-a14c-e7b02a9f3261 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:41.915815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902f4a-c0f2-480f-a14c-e7b02a9f3261 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:41.938562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0902f4a-c0f2-480f-a14c-e7b02a9f3261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-e0902f4a-c0f2-480f-a14c-e7b02a9f3261 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:41.944500 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:41.945653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:41.946465 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:41.947062 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 182/369] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:32:41.948006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0902f4a-c0f2-480f-a14c-e7b02a9f3261 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:41.958255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940987a5-7a22-4da6-8bd7-e31c8a5cd80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:41.959457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940987a5-7a22-4da6-8bd7-e31c8a5cd80a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:41.959652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940987a5-7a22-4da6-8bd7-e31c8a5cd80a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:41.959904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940987a5-7a22-4da6-8bd7-e31c8a5cd80a None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:41.961473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940987a5-7a22-4da6-8bd7-e31c8a5cd80a None admin] RBAC: Adding query filter params (name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:41.962029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940987a5-7a22-4da6-8bd7-e31c8a5cd80a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:41.963433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940987a5-7a22-4da6-8bd7-e31c8a5cd80a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:41.973975 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 188/370] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:32:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:41.974893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940987a5-7a22-4da6-8bd7-e31c8a5cd80a None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:41.986675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:42.065617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0902f4a-c0f2-480f-a14c-e7b02a9f3261 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.080499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940987a5-7a22-4da6-8bd7-e31c8a5cd80a None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.104881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a7fb6-86c9-4da0-8933-5918ee1f062e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.106064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a7fb6-86c9-4da0-8933-5918ee1f062e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.106543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a7fb6-86c9-4da0-8933-5918ee1f062e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.107061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a7fb6-86c9-4da0-8933-5918ee1f062e None admin] PATH_INFO: `/v3/roles/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.109678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:42.120249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24337107-bdbf-4ddc-a0fc-69351806e1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.120616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24337107-bdbf-4ddc-a0fc-69351806e1f5 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.120894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24337107-bdbf-4ddc-a0fc-69351806e1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.121198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24337107-bdbf-4ddc-a0fc-69351806e1f5 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.121937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24337107-bdbf-4ddc-a0fc-69351806e1f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:42.122238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24337107-bdbf-4ddc-a0fc-69351806e1f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:42.123830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24337107-bdbf-4ddc-a0fc-69351806e1f5 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:42.125493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a7fb6-86c9-4da0-8933-5918ee1f062e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-e18a7fb6-86c9-4da0-8933-5918ee1f062e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:42.131240 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:42.132235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:42.133584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:42.135014 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 189/371] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:32:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:42.140846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24337107-bdbf-4ddc-a0fc-69351806e1f5 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:42.141232 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 183/372] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:42.142365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a7fb6-86c9-4da0-8933-5918ee1f062e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:42.241405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:42.253674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24337107-bdbf-4ddc-a0fc-69351806e1f5 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.255708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a7fb6-86c9-4da0-8933-5918ee1f062e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.291108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3ee0f-c0e4-41dd-a249-1dba63b88d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.291574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bc1b6-909a-4686-a5bb-a2b5dd7fc696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.291763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3ee0f-c0e4-41dd-a249-1dba63b88d55 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.292006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3ee0f-c0e4-41dd-a249-1dba63b88d55 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.292183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bc1b6-909a-4686-a5bb-a2b5dd7fc696 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.292277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3ee0f-c0e4-41dd-a249-1dba63b88d55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.292458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bc1b6-909a-4686-a5bb-a2b5dd7fc696 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.292706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bc1b6-909a-4686-a5bb-a2b5dd7fc696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.303965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3ee0f-c0e4-41dd-a249-1dba63b88d55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:42.304207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bc1b6-909a-4686-a5bb-a2b5dd7fc696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:42.305750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3ee0f-c0e4-41dd-a249-1dba63b88d55 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:42.305861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bc1b6-909a-4686-a5bb-a2b5dd7fc696 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-d92bc1b6-909a-4686-a5bb-a2b5dd7fc696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:42.321485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:42.322392 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:42.323213 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:42.323213 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:42.323213 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-09b3ee0f-c0e4-41dd-a249-1dba63b88d55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:42.323370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:42.324704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:42.325819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:42.325819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:42.325819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:42.325819 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 184/373] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:42.325819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3ee0f-c0e4-41dd-a249-1dba63b88d55 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:42.326574 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 190/374] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:42.329147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eefff-ea2c-4fb1-96ff-c6fa025dd076 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.329606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eefff-ea2c-4fb1-96ff-c6fa025dd076 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.330067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eefff-ea2c-4fb1-96ff-c6fa025dd076 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.331508 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 191/375] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:42.332479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eefff-ea2c-4fb1-96ff-c6fa025dd076 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:42.379291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:42.437880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3ee0f-c0e4-41dd-a249-1dba63b88d55 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.448090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eefff-ea2c-4fb1-96ff-c6fa025dd076 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.474755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8e8a9-5e7f-4cc0-837d-163eafe5e847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.475348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e8a9-5e7f-4cc0-837d-163eafe5e847 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.475617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e8a9-5e7f-4cc0-837d-163eafe5e847 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.475873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e8a9-5e7f-4cc0-837d-163eafe5e847 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.476665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8e8a9-5e7f-4cc0-837d-163eafe5e847 None admin] RBAC: Adding query filter params (name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:42.476961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8e8a9-5e7f-4cc0-837d-163eafe5e847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:42.478458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8e8a9-5e7f-4cc0-837d-163eafe5e847 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:42.484045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33619d-ae3e-4762-b0d2-76f254e54f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.484704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33619d-ae3e-4762-b0d2-76f254e54f60 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.484907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33619d-ae3e-4762-b0d2-76f254e54f60 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.485218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33619d-ae3e-4762-b0d2-76f254e54f60 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.485844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33619d-ae3e-4762-b0d2-76f254e54f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:42.486136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33619d-ae3e-4762-b0d2-76f254e54f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:42.486584 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 185/376] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:32:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:42.487336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8e8a9-5e7f-4cc0-837d-163eafe5e847 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:42.489976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:42.491875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33619d-ae3e-4762-b0d2-76f254e54f60 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:42.509980 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 192/377] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:42.509980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df5bc7-76df-468f-af8b-d1fd666c28a3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.509980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df5bc7-76df-468f-af8b-d1fd666c28a3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.509980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df5bc7-76df-468f-af8b-d1fd666c28a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.525769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-86df5bc7-76df-468f-af8b-d1fd666c28a3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:42.526981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-86df5bc7-76df-468f-af8b-d1fd666c28a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:42.594781 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 193/378] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:42.595622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df5bc7-76df-468f-af8b-d1fd666c28a3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:42.600681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8e8a9-5e7f-4cc0-837d-163eafe5e847 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.632506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:42.637745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12882363-2c8a-4ea5-98a3-e4f0459d87c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.638376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12882363-2c8a-4ea5-98a3-e4f0459d87c4 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.638650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12882363-2c8a-4ea5-98a3-e4f0459d87c4 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.639515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12882363-2c8a-4ea5-98a3-e4f0459d87c4 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.639515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12882363-2c8a-4ea5-98a3-e4f0459d87c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:42.639833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12882363-2c8a-4ea5-98a3-e4f0459d87c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:42.641243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12882363-2c8a-4ea5-98a3-e4f0459d87c4 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:42.653891 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 186/379] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:42.658877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12882363-2c8a-4ea5-98a3-e4f0459d87c4 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:42.706911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df5bc7-76df-468f-af8b-d1fd666c28a3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.726695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:42.747928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4529202-f950-4ca9-98af-b79749fc8564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.748561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4529202-f950-4ca9-98af-b79749fc8564 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.748838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4529202-f950-4ca9-98af-b79749fc8564 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.749127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4529202-f950-4ca9-98af-b79749fc8564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.760807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4529202-f950-4ca9-98af-b79749fc8564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:42.762801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4529202-f950-4ca9-98af-b79749fc8564 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:42.773591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12882363-2c8a-4ea5-98a3-e4f0459d87c4 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-c4529202-f950-4ca9-98af-b79749fc8564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:42.778990 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:42.779890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:42.780726 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:42.780726 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:42.780726 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:42.784687 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 194/380] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:42.785936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57fb3-d3ba-4ba0-9f98-a98355828834 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.786202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57fb3-d3ba-4ba0-9f98-a98355828834 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.786467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57fb3-d3ba-4ba0-9f98-a98355828834 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.787719 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 195/381] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:42.788374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff57fb3-d3ba-4ba0-9f98-a98355828834 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:42.814562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b3e86-ceff-4ea6-bff5-305e91ce4a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.815177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b3e86-ceff-4ea6-bff5-305e91ce4a35 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.815658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b3e86-ceff-4ea6-bff5-305e91ce4a35 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.815763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b3e86-ceff-4ea6-bff5-305e91ce4a35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.825338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b3e86-ceff-4ea6-bff5-305e91ce4a35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:42.827029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b3e86-ceff-4ea6-bff5-305e91ce4a35 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-c53b3e86-ceff-4ea6-bff5-305e91ce4a35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:42.843776 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:42.844676 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:42.845635 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 187/382] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Oct 15 20:32:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:42.846835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b3e86-ceff-4ea6-bff5-305e91ce4a35 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:42.886633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:42.910297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff57fb3-d3ba-4ba0-9f98-a98355828834 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.946344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c85ce-dd2d-4ac4-8c79-14d71a95949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.946972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c85ce-dd2d-4ac4-8c79-14d71a95949e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.947215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c85ce-dd2d-4ac4-8c79-14d71a95949e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.947479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c85ce-dd2d-4ac4-8c79-14d71a95949e None admin] PATH_INFO: `/v3/users/glance` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.952368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b3e86-ceff-4ea6-bff5-305e91ce4a35 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:42.954391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c85ce-dd2d-4ac4-8c79-14d71a95949e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:42.955819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c85ce-dd2d-4ac4-8c79-14d71a95949e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-fa1c85ce-dd2d-4ac4-8c79-14d71a95949e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:42.969099 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:42.970045 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:42.970944 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 196/383] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:42.973201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d03c5f-b01a-4da5-a825-e6fa9b62633b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.973483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d03c5f-b01a-4da5-a825-e6fa9b62633b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.973754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d03c5f-b01a-4da5-a825-e6fa9b62633b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.975531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:42.987456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9db9d9-cd73-4260-91a6-d4e21f60f94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:42.988051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9db9d9-cd73-4260-91a6-d4e21f60f94b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:42.988328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9db9d9-cd73-4260-91a6-d4e21f60f94b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:42.988595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9db9d9-cd73-4260-91a6-d4e21f60f94b None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:42.989261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9db9d9-cd73-4260-91a6-d4e21f60f94b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:42.989560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9db9d9-cd73-4260-91a6-d4e21f60f94b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:42.990933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9db9d9-cd73-4260-91a6-d4e21f60f94b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:42.991390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-02d03c5f-b01a-4da5-a825-e6fa9b62633b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:42.992609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-02d03c5f-b01a-4da5-a825-e6fa9b62633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:43.003256 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 188/384] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:43.004083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9db9d9-cd73-4260-91a6-d4e21f60f94b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.065196 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 197/385] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:43.066035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d03c5f-b01a-4da5-a825-e6fa9b62633b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.113575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9db9d9-cd73-4260-91a6-d4e21f60f94b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.139889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:43.150726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73eb59-9910-486f-bd02-2b001e079c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.151326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73eb59-9910-486f-bd02-2b001e079c59 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:43.151599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73eb59-9910-486f-bd02-2b001e079c59 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:43.151865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73eb59-9910-486f-bd02-2b001e079c59 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:43.152588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73eb59-9910-486f-bd02-2b001e079c59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:43.152889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73eb59-9910-486f-bd02-2b001e079c59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:43.154348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73eb59-9910-486f-bd02-2b001e079c59 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:43.173509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d03c5f-b01a-4da5-a825-e6fa9b62633b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.195640 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 189/386] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:43.195640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73eb59-9910-486f-bd02-2b001e079c59 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.217674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e96a9-adcd-4b08-a9ce-83623dd27052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.218326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e96a9-adcd-4b08-a9ce-83623dd27052 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:43.218599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e96a9-adcd-4b08-a9ce-83623dd27052 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:43.218863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e96a9-adcd-4b08-a9ce-83623dd27052 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:43.229907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e96a9-adcd-4b08-a9ce-83623dd27052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-aa4e96a9-adcd-4b08-a9ce-83623dd27052 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:43.234576 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:43.235571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:43.236376 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 198/387] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:43.237120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e96a9-adcd-4b08-a9ce-83623dd27052 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.245211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:43.307092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73eb59-9910-486f-bd02-2b001e079c59 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.342955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecff6f-a074-476d-80d9-4ad588377c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.343556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecff6f-a074-476d-80d9-4ad588377c70 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:43.343826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecff6f-a074-476d-80d9-4ad588377c70 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:43.344090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecff6f-a074-476d-80d9-4ad588377c70 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:43.344797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecff6f-a074-476d-80d9-4ad588377c70 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:43.345120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecff6f-a074-476d-80d9-4ad588377c70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:43.346588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecff6f-a074-476d-80d9-4ad588377c70 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:43.351095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e96a9-adcd-4b08-a9ce-83623dd27052 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.363907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 190/388] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:43.364753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecff6f-a074-476d-80d9-4ad588377c70 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.391370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:43.397277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4680c6-fa20-4207-bccc-7dcbe07f2f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.397898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4680c6-fa20-4207-bccc-7dcbe07f2f8f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:43.398800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4680c6-fa20-4207-bccc-7dcbe07f2f8f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:43.399062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4680c6-fa20-4207-bccc-7dcbe07f2f8f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:43.407384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4680c6-fa20-4207-bccc-7dcbe07f2f8f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:43.409567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4680c6-fa20-4207-bccc-7dcbe07f2f8f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-af4680c6-fa20-4207-bccc-7dcbe07f2f8f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:43.421007 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:43.421919 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:43.422816 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 199/389] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:43.423363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4680c6-fa20-4207-bccc-7dcbe07f2f8f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.465970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:43.469927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecff6f-a074-476d-80d9-4ad588377c70 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.503675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69efa-8d03-47fe-b8eb-b4bf5bdfd107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.504223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69efa-8d03-47fe-b8eb-b4bf5bdfd107 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:43.504493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69efa-8d03-47fe-b8eb-b4bf5bdfd107 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:43.504755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69efa-8d03-47fe-b8eb-b4bf5bdfd107 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:43.514678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c69efa-8d03-47fe-b8eb-b4bf5bdfd107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:43.516160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c69efa-8d03-47fe-b8eb-b4bf5bdfd107 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:43.524645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4680c6-fa20-4207-bccc-7dcbe07f2f8f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-09c69efa-8d03-47fe-b8eb-b4bf5bdfd107 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:43.530580 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:43.531429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:43.532237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:43.532237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:43.532237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:43.532237 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 191/390] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:43.532441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69efa-8d03-47fe-b8eb-b4bf5bdfd107 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.560085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed2939-4bcb-47d4-9c31-e51e17998d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.560676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed2939-4bcb-47d4-9c31-e51e17998d2b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:43.560950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed2939-4bcb-47d4-9c31-e51e17998d2b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:43.561245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed2939-4bcb-47d4-9c31-e51e17998d2b None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:43.561895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed2939-4bcb-47d4-9c31-e51e17998d2b None admin] RBAC: Adding query filter params (name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:43.562190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed2939-4bcb-47d4-9c31-e51e17998d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:43.563544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed2939-4bcb-47d4-9c31-e51e17998d2b None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:43.571755 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 200/391] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:43.572529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed2939-4bcb-47d4-9c31-e51e17998d2b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.629576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:43.639465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69efa-8d03-47fe-b8eb-b4bf5bdfd107 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.673315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53081947-e5be-479a-b4fb-5a331628b383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.673929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53081947-e5be-479a-b4fb-5a331628b383 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:43.674201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53081947-e5be-479a-b4fb-5a331628b383 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:43.674465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53081947-e5be-479a-b4fb-5a331628b383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:43.675612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed2939-4bcb-47d4-9c31-e51e17998d2b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.684872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53081947-e5be-479a-b4fb-5a331628b383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:43.686364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53081947-e5be-479a-b4fb-5a331628b383 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-53081947-e5be-479a-b4fb-5a331628b383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:43.700750 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:43.701729 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:43.702545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:43.702545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:43.702545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:43.702545 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 192/392] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:43.703212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53081947-e5be-479a-b4fb-5a331628b383 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.705532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:43.713417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539735c-6053-4b50-bd2a-5df4d5d34ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.714036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539735c-6053-4b50-bd2a-5df4d5d34ee1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:43.714310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539735c-6053-4b50-bd2a-5df4d5d34ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:43.714595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539735c-6053-4b50-bd2a-5df4d5d34ee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:43.725064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539735c-6053-4b50-bd2a-5df4d5d34ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:43.727155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539735c-6053-4b50-bd2a-5df4d5d34ee1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-8539735c-6053-4b50-bd2a-5df4d5d34ee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:43.741001 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:43.742654 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:43.743757 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:43.743757 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:43.743757 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:43.743757 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 201/393] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:43.744148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539735c-6053-4b50-bd2a-5df4d5d34ee1 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.811122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53081947-e5be-479a-b4fb-5a331628b383 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.842946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539735c-6053-4b50-bd2a-5df4d5d34ee1 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.844667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec426df-de83-4301-bc76-e8c7ccddc178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.845288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec426df-de83-4301-bc76-e8c7ccddc178 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:43.845558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec426df-de83-4301-bc76-e8c7ccddc178 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:43.845818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec426df-de83-4301-bc76-e8c7ccddc178 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:43.846521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec426df-de83-4301-bc76-e8c7ccddc178 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:43.846812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec426df-de83-4301-bc76-e8c7ccddc178 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:43.848224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec426df-de83-4301-bc76-e8c7ccddc178 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:43.849750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:43.865386 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 193/394] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:43.866181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec426df-de83-4301-bc76-e8c7ccddc178 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.881432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c98568-39de-4043-804c-eb1b52558a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.882024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c98568-39de-4043-804c-eb1b52558a0c None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:43.882295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c98568-39de-4043-804c-eb1b52558a0c None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:43.883066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c98568-39de-4043-804c-eb1b52558a0c None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:43.883823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c98568-39de-4043-804c-eb1b52558a0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:43.884118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c98568-39de-4043-804c-eb1b52558a0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:43.885520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c98568-39de-4043-804c-eb1b52558a0c None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:43.896394 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 202/395] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:43.897211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c98568-39de-4043-804c-eb1b52558a0c None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:43.926164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:43.966339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec426df-de83-4301-bc76-e8c7ccddc178 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.996417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c98568-39de-4043-804c-eb1b52558a0c None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:43.999373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d630098-72a6-4011-adbd-054d31b6de37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:43.999976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d630098-72a6-4011-adbd-054d31b6de37 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.000244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d630098-72a6-4011-adbd-054d31b6de37 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.000505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d630098-72a6-4011-adbd-054d31b6de37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.010869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d630098-72a6-4011-adbd-054d31b6de37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.012337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d630098-72a6-4011-adbd-054d31b6de37 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-8d630098-72a6-4011-adbd-054d31b6de37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:44.027560 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:44.028496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:44.029370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:44.029370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:44.029370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:44.029370 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 194/396] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:44.029579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d630098-72a6-4011-adbd-054d31b6de37 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.032592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd47b3e-0c78-48c8-96d5-a7ffe6a0ac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.033228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd47b3e-0c78-48c8-96d5-a7ffe6a0ac6d None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.033508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd47b3e-0c78-48c8-96d5-a7ffe6a0ac6d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.033775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd47b3e-0c78-48c8-96d5-a7ffe6a0ac6d None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.034412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd47b3e-0c78-48c8-96d5-a7ffe6a0ac6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:44.034713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd47b3e-0c78-48c8-96d5-a7ffe6a0ac6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.036056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd47b3e-0c78-48c8-96d5-a7ffe6a0ac6d None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.048446 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 203/397] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:44.049291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd47b3e-0c78-48c8-96d5-a7ffe6a0ac6d None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.078629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:44.129939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d630098-72a6-4011-adbd-054d31b6de37 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:44.142509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:44.152814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd47b3e-0c78-48c8-96d5-a7ffe6a0ac6d None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:44.164793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabd472-ebe6-4493-8443-96a7ee67661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.165408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabd472-ebe6-4493-8443-96a7ee67661d None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.165678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabd472-ebe6-4493-8443-96a7ee67661d None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.165944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabd472-ebe6-4493-8443-96a7ee67661d None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.166569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabd472-ebe6-4493-8443-96a7ee67661d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:44.166857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabd472-ebe6-4493-8443-96a7ee67661d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.168253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabd472-ebe6-4493-8443-96a7ee67661d None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.180296 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 195/398] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:44.181113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabd472-ebe6-4493-8443-96a7ee67661d None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.188153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc233edd-7886-4134-9200-ddb2ece15206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.188769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc233edd-7886-4134-9200-ddb2ece15206 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.189075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc233edd-7886-4134-9200-ddb2ece15206 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.189351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc233edd-7886-4134-9200-ddb2ece15206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.199738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc233edd-7886-4134-9200-ddb2ece15206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.201221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc233edd-7886-4134-9200-ddb2ece15206 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-cc233edd-7886-4134-9200-ddb2ece15206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:44.216420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:44.217495 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:44.218515 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:44.218515 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:44.218515 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:44.218515 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 204/399] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:44.218515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc233edd-7886-4134-9200-ddb2ece15206 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.292683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabd472-ebe6-4493-8443-96a7ee67661d None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:44.295697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:44.316607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc233edd-7886-4134-9200-ddb2ece15206 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:44.326441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6b442-12b7-46e3-8866-55c9586755db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.327046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6b442-12b7-46e3-8866-55c9586755db None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.327318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6b442-12b7-46e3-8866-55c9586755db None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.327580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6b442-12b7-46e3-8866-55c9586755db None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.334070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6b442-12b7-46e3-8866-55c9586755db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.335455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6b442-12b7-46e3-8866-55c9586755db None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-83e6b442-12b7-46e3-8866-55c9586755db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:44.347336 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:44.348189 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:44.349023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:44.349023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:44.349023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:44.349023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:44.349023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:44.349023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:44.349023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:44.349023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:44.349023 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 196/400] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:44.349655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6b442-12b7-46e3-8866-55c9586755db None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.354460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678734e9-a35f-44e2-a90c-fc4df6f20b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.355214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678734e9-a35f-44e2-a90c-fc4df6f20b7e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.355505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678734e9-a35f-44e2-a90c-fc4df6f20b7e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.355777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678734e9-a35f-44e2-a90c-fc4df6f20b7e None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.357283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678734e9-a35f-44e2-a90c-fc4df6f20b7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:44.357587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678734e9-a35f-44e2-a90c-fc4df6f20b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.358945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678734e9-a35f-44e2-a90c-fc4df6f20b7e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.370676 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 205/401] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:44.371451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678734e9-a35f-44e2-a90c-fc4df6f20b7e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.384593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:44.453168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6b442-12b7-46e3-8866-55c9586755db None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:44.470739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678734e9-a35f-44e2-a90c-fc4df6f20b7e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:44.485867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacb9c0-7df1-4370-b883-960c89160816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.486487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb9c0-7df1-4370-b883-960c89160816 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.486759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb9c0-7df1-4370-b883-960c89160816 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.487022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb9c0-7df1-4370-b883-960c89160816 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.493798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacb9c0-7df1-4370-b883-960c89160816 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.495246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacb9c0-7df1-4370-b883-960c89160816 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.503581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f0ae4-18ea-4a07-9c02-b15ea94b5a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.504176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f0ae4-18ea-4a07-9c02-b15ea94b5a5f None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.504446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f0ae4-18ea-4a07-9c02-b15ea94b5a5f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.504713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f0ae4-18ea-4a07-9c02-b15ea94b5a5f None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.505373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f0ae4-18ea-4a07-9c02-b15ea94b5a5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-4eacb9c0-7df1-4370-b883-960c89160816 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:44.506294 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:44.507218 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:44.508740 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:44.508740 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:44.508740 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:44.508740 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:44.508740 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:44.508740 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:44.508740 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:44.508740 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:44.509833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f0ae4-18ea-4a07-9c02-b15ea94b5a5f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.509933 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 197/402] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:44.509933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacb9c0-7df1-4370-b883-960c89160816 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.517296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:44.528870 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 206/403] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:32:44 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:32:44.530001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f0ae4-18ea-4a07-9c02-b15ea94b5a5f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.612349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:44.623285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacb9c0-7df1-4370-b883-960c89160816 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:44.660614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195ac1f-5413-4468-8fed-2417b882706a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.661362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195ac1f-5413-4468-8fed-2417b882706a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.661637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195ac1f-5413-4468-8fed-2417b882706a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.661899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195ac1f-5413-4468-8fed-2417b882706a None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.662540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195ac1f-5413-4468-8fed-2417b882706a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:44.663348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195ac1f-5413-4468-8fed-2417b882706a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.664970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195ac1f-5413-4468-8fed-2417b882706a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.675327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f0ae4-18ea-4a07-9c02-b15ea94b5a5f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:44.678751 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 198/404] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:44.678924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195ac1f-5413-4468-8fed-2417b882706a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.720472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df253db2-5d0e-4829-8686-c4773633c4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.721110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df253db2-5d0e-4829-8686-c4773633c4d7 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.721382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df253db2-5d0e-4829-8686-c4773633c4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.721642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df253db2-5d0e-4829-8686-c4773633c4d7 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.722324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df253db2-5d0e-4829-8686-c4773633c4d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:44.722620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df253db2-5d0e-4829-8686-c4773633c4d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.724048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df253db2-5d0e-4829-8686-c4773633c4d7 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.743541 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 207/405] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:44.745252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df253db2-5d0e-4829-8686-c4773633c4d7 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.795011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195ac1f-5413-4468-8fed-2417b882706a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:44.810871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:44.831029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b7fbf-3e43-453f-b877-f8f87836d4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.832020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b7fbf-3e43-453f-b877-f8f87836d4d1 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.832497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b7fbf-3e43-453f-b877-f8f87836d4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.832962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b7fbf-3e43-453f-b877-f8f87836d4d1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.843464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b7fbf-3e43-453f-b877-f8f87836d4d1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.845630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b7fbf-3e43-453f-b877-f8f87836d4d1 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.859739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df253db2-5d0e-4829-8686-c4773633c4d7 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-e83b7fbf-3e43-453f-b877-f8f87836d4d1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:44.863428 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:44.864290 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:44.865183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:44.865183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:44.865183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:44.865183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:44.865183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:44.865183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:44.865183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:44.865183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:44.865183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 20:32:44.865183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:44.869188 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 199/406] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:44.870439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b7fbf-3e43-453f-b877-f8f87836d4d1 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.892883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:44.900138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cade54a-bf2e-46d8-8de4-565a06004d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:44.900821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cade54a-bf2e-46d8-8de4-565a06004d02 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:44.901212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cade54a-bf2e-46d8-8de4-565a06004d02 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:44.901553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cade54a-bf2e-46d8-8de4-565a06004d02 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:44.902245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cade54a-bf2e-46d8-8de4-565a06004d02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:44.902544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cade54a-bf2e-46d8-8de4-565a06004d02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:44.904006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cade54a-bf2e-46d8-8de4-565a06004d02 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:44.917476 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 208/407] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:44.917993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cade54a-bf2e-46d8-8de4-565a06004d02 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:44.988578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b7fbf-3e43-453f-b877-f8f87836d4d1 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:45.026519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a175d3-3289-4264-a36b-fd0822dfb9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:45.027141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a175d3-3289-4264-a36b-fd0822dfb9ed None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:45.027415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a175d3-3289-4264-a36b-fd0822dfb9ed None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:45.027514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cade54a-bf2e-46d8-8de4-565a06004d02 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:45.027968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a175d3-3289-4264-a36b-fd0822dfb9ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:45.034595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a175d3-3289-4264-a36b-fd0822dfb9ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:45.036054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a175d3-3289-4264-a36b-fd0822dfb9ed None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:45.042062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-f2a175d3-3289-4264-a36b-fd0822dfb9ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:45.048069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:45.048902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:45.049798 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:45.049798 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:45.049798 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:45.049798 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:45.049798 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:45.049798 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:45.049798 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:45.049798 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:45.049798 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 200/408] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:45.050238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a175d3-3289-4264-a36b-fd0822dfb9ed None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:45.070618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69f0ef-5a3f-4049-9755-b88f1ce8f08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:45.071268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f0ef-5a3f-4049-9755-b88f1ce8f08e None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:45.071536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f0ef-5a3f-4049-9755-b88f1ce8f08e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:45.071796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f0ef-5a3f-4049-9755-b88f1ce8f08e None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/6225c81d2620463886d8a936b59784df/roles/492ce6c3fedb4109ac31de9467e8ef3b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:45.087667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69f0ef-5a3f-4049-9755-b88f1ce8f08e None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=6225c81d2620463886d8a936b59784df, role_id=492ce6c3fedb4109ac31de9467e8ef3b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:45.089234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69f0ef-5a3f-4049-9755-b88f1ce8f08e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:45.103692 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 209/409] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:32:44 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/6225c81d2620463886d8a936b59784df/roles/492ce6c3fedb4109ac31de9467e8ef3b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:32:45.104478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69f0ef-5a3f-4049-9755-b88f1ce8f08e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:45.135948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:45.175757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a175d3-3289-4264-a36b-fd0822dfb9ed None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:45.224456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7866d-86e6-4d5d-928d-0ad6641a5614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:45.225410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7866d-86e6-4d5d-928d-0ad6641a5614 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:45.225867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7866d-86e6-4d5d-928d-0ad6641a5614 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:45.226457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7866d-86e6-4d5d-928d-0ad6641a5614 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:45.227620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7866d-86e6-4d5d-928d-0ad6641a5614 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:45.228172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7866d-86e6-4d5d-928d-0ad6641a5614 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:45.229940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7866d-86e6-4d5d-928d-0ad6641a5614 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:45.250605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69f0ef-5a3f-4049-9755-b88f1ce8f08e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:45.255787 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 201/410] 10.222.0.16 () {56 vars in 1139 bytes} [Sat Oct 15 20:32:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:45.257177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7866d-86e6-4d5d-928d-0ad6641a5614 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:45.299369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4c3aa-3905-4d61-ba48-2a3fa602bd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:45.300043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4c3aa-3905-4d61-ba48-2a3fa602bd71 None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:45.300314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4c3aa-3905-4d61-ba48-2a3fa602bd71 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:45.300577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4c3aa-3905-4d61-ba48-2a3fa602bd71 None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/af8591ebfa174c91853b673360326a9c/roles/492ce6c3fedb4109ac31de9467e8ef3b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:45.319836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4c3aa-3905-4d61-ba48-2a3fa602bd71 None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=af8591ebfa174c91853b673360326a9c, role_id=492ce6c3fedb4109ac31de9467e8ef3b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:45.322583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4c3aa-3905-4d61-ba48-2a3fa602bd71 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:45.339650 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 210/411] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:32:45 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/af8591ebfa174c91853b673360326a9c/roles/492ce6c3fedb4109ac31de9467e8ef3b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:32:45.340782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4c3aa-3905-4d61-ba48-2a3fa602bd71 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:45.368098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:45.385797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7866d-86e6-4d5d-928d-0ad6641a5614 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:45.419959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:45.438446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e9e72-b04a-4f66-bc5e-136480b6c827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:45.439114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e9e72-b04a-4f66-bc5e-136480b6c827 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:45.439446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e9e72-b04a-4f66-bc5e-136480b6c827 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:45.439713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e9e72-b04a-4f66-bc5e-136480b6c827 None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:45.440457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e9e72-b04a-4f66-bc5e-136480b6c827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:45.440771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e9e72-b04a-4f66-bc5e-136480b6c827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:45.442243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e9e72-b04a-4f66-bc5e-136480b6c827 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:45.474513 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 202/412] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:45.483728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e9e72-b04a-4f66-bc5e-136480b6c827 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:45.506798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4c3aa-3905-4d61-ba48-2a3fa602bd71 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:45.578019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58008f53-ec81-4570-81a7-5168c3e40014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:45.578662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58008f53-ec81-4570-81a7-5168c3e40014 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:45.578936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58008f53-ec81-4570-81a7-5168c3e40014 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:45.579200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58008f53-ec81-4570-81a7-5168c3e40014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:45.601586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58008f53-ec81-4570-81a7-5168c3e40014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:45.603191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58008f53-ec81-4570-81a7-5168c3e40014 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-58008f53-ec81-4570-81a7-5168c3e40014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:45.627610 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:45.628536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:45.629379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:45.629379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:45.629379 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:45.631259 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 211/413] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:45.638181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58008f53-ec81-4570-81a7-5168c3e40014 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:45.660818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e9e72-b04a-4f66-bc5e-136480b6c827 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:45.700973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad773067-76ad-4bc2-ac0c-b537caac75d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:45.701585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773067-76ad-4bc2-ac0c-b537caac75d1 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:45.701861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773067-76ad-4bc2-ac0c-b537caac75d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:45.702131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773067-76ad-4bc2-ac0c-b537caac75d1 None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/aa85b647a4be4d81b966bb50691bd3e0/roles/492ce6c3fedb4109ac31de9467e8ef3b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:45.719389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad773067-76ad-4bc2-ac0c-b537caac75d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=aa85b647a4be4d81b966bb50691bd3e0, role_id=492ce6c3fedb4109ac31de9467e8ef3b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:45.720918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad773067-76ad-4bc2-ac0c-b537caac75d1 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:45.736098 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 203/414] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:32:45 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/aa85b647a4be4d81b966bb50691bd3e0/roles/492ce6c3fedb4109ac31de9467e8ef3b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:32:45.757751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:45.771486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58008f53-ec81-4570-81a7-5168c3e40014 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:45.810632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639779b-4645-409d-b75e-f91656d3ef77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:45.811236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639779b-4645-409d-b75e-f91656d3ef77 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:45.811509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639779b-4645-409d-b75e-f91656d3ef77 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:45.811776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639779b-4645-409d-b75e-f91656d3ef77 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:45.812417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639779b-4645-409d-b75e-f91656d3ef77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:45.812711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639779b-4645-409d-b75e-f91656d3ef77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:45.814119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639779b-4645-409d-b75e-f91656d3ef77 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:45.828128 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 212/415] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:45.834053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad773067-76ad-4bc2-ac0c-b537caac75d1 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:45.905888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:45.993562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad773067-76ad-4bc2-ac0c-b537caac75d1 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:46.032138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95fd6a-777e-4484-9594-e4c2378df507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:46.032785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95fd6a-777e-4484-9594-e4c2378df507 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:46.033079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95fd6a-777e-4484-9594-e4c2378df507 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:46.033354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95fd6a-777e-4484-9594-e4c2378df507 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:46.042112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95fd6a-777e-4484-9594-e4c2378df507 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:46.043619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95fd6a-777e-4484-9594-e4c2378df507 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-de95fd6a-777e-4484-9594-e4c2378df507 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:46.057357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:46.058242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:46.059063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:46.059063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:46.059063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:46.059063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:46.059063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:46.059063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:46.059063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:46.059063 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:46.059063 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 204/416] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:46.065109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639779b-4645-409d-b75e-f91656d3ef77 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:46.185990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639779b-4645-409d-b75e-f91656d3ef77 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:46.216352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:46.227345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd2152-3190-4327-87a8-88bc45abdb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:46.228218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2152-3190-4327-87a8-88bc45abdb50 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:46.228664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2152-3190-4327-87a8-88bc45abdb50 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:46.229132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2152-3190-4327-87a8-88bc45abdb50 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:46.229994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd2152-3190-4327-87a8-88bc45abdb50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:46.230465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd2152-3190-4327-87a8-88bc45abdb50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:46.232079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd2152-3190-4327-87a8-88bc45abdb50 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:46.248144 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 213/417] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:46.254122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95fd6a-777e-4484-9594-e4c2378df507 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:46.410974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:46.414940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95fd6a-777e-4484-9594-e4c2378df507 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:46.462381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d5d7d-e474-4b7d-8468-e350114b39f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:46.463032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d5d7d-e474-4b7d-8468-e350114b39f7 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:46.463306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d5d7d-e474-4b7d-8468-e350114b39f7 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:46.463573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d5d7d-e474-4b7d-8468-e350114b39f7 None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/176e98e72ba840c4a6546519b4c98d90/roles/492ce6c3fedb4109ac31de9467e8ef3b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:46.480700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d5d7d-e474-4b7d-8468-e350114b39f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=176e98e72ba840c4a6546519b4c98d90, role_id=492ce6c3fedb4109ac31de9467e8ef3b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:46.482170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d5d7d-e474-4b7d-8468-e350114b39f7 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:46.505599 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 205/418] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:32:46 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/176e98e72ba840c4a6546519b4c98d90/roles/492ce6c3fedb4109ac31de9467e8ef3b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:32:47.260645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa939ea-f226-426a-90da-69c8ae20adf7 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:47.261611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa939ea-f226-426a-90da-69c8ae20adf7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:47.262102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa939ea-f226-426a-90da-69c8ae20adf7 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:47.263854 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 214/419] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:47.269557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d86840-dd15-46b8-9587-4912134d23c3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:47.270358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d86840-dd15-46b8-9587-4912134d23c3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:47.270822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d86840-dd15-46b8-9587-4912134d23c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:47.290610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-08d86840-dd15-46b8-9587-4912134d23c3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:47.292143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-08d86840-dd15-46b8-9587-4912134d23c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:47.363673 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 206/420] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:47.705109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa939ea-f226-426a-90da-69c8ae20adf7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:47.842945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa939ea-f226-426a-90da-69c8ae20adf7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:47.890964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b4790-f647-4f31-a374-dd3b01de412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:47.891973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b4790-f647-4f31-a374-dd3b01de412d None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:47.892479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b4790-f647-4f31-a374-dd3b01de412d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:47.892934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b4790-f647-4f31-a374-dd3b01de412d None admin] PATH_INFO: `/v3/roles/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:47.908144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b4790-f647-4f31-a374-dd3b01de412d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-3a8b4790-f647-4f31-a374-dd3b01de412d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:47.912898 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:47.914341 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:47.915150 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 215/421] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:47.922185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d86840-dd15-46b8-9587-4912134d23c3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:48.053482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:48.079752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d86840-dd15-46b8-9587-4912134d23c3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:48.126021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615dc5b-8cee-4d42-b89b-875e913b0181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:48.127116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615dc5b-8cee-4d42-b89b-875e913b0181 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:48.127636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615dc5b-8cee-4d42-b89b-875e913b0181 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:48.128128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615dc5b-8cee-4d42-b89b-875e913b0181 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:48.129127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615dc5b-8cee-4d42-b89b-875e913b0181 None admin] RBAC: Adding query filter params (name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:48.129663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615dc5b-8cee-4d42-b89b-875e913b0181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:48.131295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615dc5b-8cee-4d42-b89b-875e913b0181 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:48.143043 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 207/422] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:32:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:48.149009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b4790-f647-4f31-a374-dd3b01de412d None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:48.151881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae86d27-58b4-4f18-8213-928d3823e7d1 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:48.152154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae86d27-58b4-4f18-8213-928d3823e7d1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:48.152417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae86d27-58b4-4f18-8213-928d3823e7d1 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:48.154109 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 208/423] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:32:48.160712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defedddb-768d-4cf4-8b21-7a625e8ba23e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:48.161026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defedddb-768d-4cf4-8b21-7a625e8ba23e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:48.161287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defedddb-768d-4cf4-8b21-7a625e8ba23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:48.191572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-defedddb-768d-4cf4-8b21-7a625e8ba23e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:48.192774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-defedddb-768d-4cf4-8b21-7a625e8ba23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:48.243085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:48.301240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b4790-f647-4f31-a374-dd3b01de412d None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:48.323609 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 209/424] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:48.391083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6141-94bf-4aed-abfb-2d31133cb690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:48.391708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6141-94bf-4aed-abfb-2d31133cb690 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:48.391980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6141-94bf-4aed-abfb-2d31133cb690 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:48.392436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6141-94bf-4aed-abfb-2d31133cb690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:48.414294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f6141-94bf-4aed-abfb-2d31133cb690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:48.415750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f6141-94bf-4aed-abfb-2d31133cb690 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-515f6141-94bf-4aed-abfb-2d31133cb690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:48.440811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:48.441844 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:48.442697 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:48.442697 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:48.442697 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:48.442697 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 216/425] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:48.447763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defedddb-768d-4cf4-8b21-7a625e8ba23e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:48.544838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:48.600828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defedddb-768d-4cf4-8b21-7a625e8ba23e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:48.639903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0be8a-c6ee-4fdb-8fc0-73171e2e868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:48.640493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0be8a-c6ee-4fdb-8fc0-73171e2e868a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:48.640755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0be8a-c6ee-4fdb-8fc0-73171e2e868a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:48.641048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0be8a-c6ee-4fdb-8fc0-73171e2e868a None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:48.641677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0be8a-c6ee-4fdb-8fc0-73171e2e868a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:48.641968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0be8a-c6ee-4fdb-8fc0-73171e2e868a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:48.643338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0be8a-c6ee-4fdb-8fc0-73171e2e868a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:48.656265 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 210/426] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:48.661535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6141-94bf-4aed-abfb-2d31133cb690 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:48.710211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c37a46-c65f-45a3-80f5-2eac04c8da30 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:48.710997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c37a46-c65f-45a3-80f5-2eac04c8da30 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:48.711270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c37a46-c65f-45a3-80f5-2eac04c8da30 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:48.712818 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 211/427] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:48.719265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d955ba1-c755-403f-9fb1-fac4405a5d7f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:48.719867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d955ba1-c755-403f-9fb1-fac4405a5d7f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:48.720137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d955ba1-c755-403f-9fb1-fac4405a5d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:48.742229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3d955ba1-c755-403f-9fb1-fac4405a5d7f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:48.743935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3d955ba1-c755-403f-9fb1-fac4405a5d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:48.829903 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 212/428] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:48.831125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d955ba1-c755-403f-9fb1-fac4405a5d7f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:48.840278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:48.848526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6141-94bf-4aed-abfb-2d31133cb690 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:48.922648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f5df7-bcbc-4169-8a53-34c62fda62c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:48.923277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f5df7-bcbc-4169-8a53-34c62fda62c4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:48.923549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f5df7-bcbc-4169-8a53-34c62fda62c4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:48.923813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f5df7-bcbc-4169-8a53-34c62fda62c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:48.941794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f5df7-bcbc-4169-8a53-34c62fda62c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:48.943247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f5df7-bcbc-4169-8a53-34c62fda62c4 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:48.950124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-385f5df7-bcbc-4169-8a53-34c62fda62c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:48.966418 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:48.967343 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:48.968175 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 217/429] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:32:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:48.973246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f5df7-bcbc-4169-8a53-34c62fda62c4 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:48.994505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d955ba1-c755-403f-9fb1-fac4405a5d7f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:49.042245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416c31c-c573-4162-bbe0-c8109749da4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:49.043049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416c31c-c573-4162-bbe0-c8109749da4a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.043331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416c31c-c573-4162-bbe0-c8109749da4a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.043592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416c31c-c573-4162-bbe0-c8109749da4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.071217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416c31c-c573-4162-bbe0-c8109749da4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-a416c31c-c573-4162-bbe0-c8109749da4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:49.077605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:49.078642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:49.079599 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:49.083350 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 213/430] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:49.088039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416c31c-c573-4162-bbe0-c8109749da4a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:49.142443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f5df7-bcbc-4169-8a53-34c62fda62c4 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:49.180838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb80e5-d4f9-4181-89a0-cd994fa5a85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:49.181884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb80e5-d4f9-4181-89a0-cd994fa5a85d None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.182392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb80e5-d4f9-4181-89a0-cd994fa5a85d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.182845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb80e5-d4f9-4181-89a0-cd994fa5a85d None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.183839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb80e5-d4f9-4181-89a0-cd994fa5a85d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:49.184325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb80e5-d4f9-4181-89a0-cd994fa5a85d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:49.186025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb80e5-d4f9-4181-89a0-cd994fa5a85d None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:49.214873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416c31c-c573-4162-bbe0-c8109749da4a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:49.235079 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 218/431] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Oct 15 20:32:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:49.236624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a9851-9c45-4782-b9a4-f18506b4aa91 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.237157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a9851-9c45-4782-b9a4-f18506b4aa91 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.237492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a9851-9c45-4782-b9a4-f18506b4aa91 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.243686 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 219/432] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:49.243686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a9851-9c45-4782-b9a4-f18506b4aa91 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:49.244345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:49.257126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45da08d-139c-48f7-89f1-b12250928a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:49.258060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45da08d-139c-48f7-89f1-b12250928a6b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.258502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45da08d-139c-48f7-89f1-b12250928a6b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.258921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45da08d-139c-48f7-89f1-b12250928a6b None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.259582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45da08d-139c-48f7-89f1-b12250928a6b None admin] RBAC: Adding query filter params (name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:49.259886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45da08d-139c-48f7-89f1-b12250928a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:49.261316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45da08d-139c-48f7-89f1-b12250928a6b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:49.261746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:49.270060 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 214/433] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:32:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:49.270842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45da08d-139c-48f7-89f1-b12250928a6b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:49.360349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a9851-9c45-4782-b9a4-f18506b4aa91 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:49.378489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45da08d-139c-48f7-89f1-b12250928a6b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:49.397130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57687d9-ebcf-47ef-bef2-ce527bddb3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:49.397746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57687d9-ebcf-47ef-bef2-ce527bddb3e0 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.398683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57687d9-ebcf-47ef-bef2-ce527bddb3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.398951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57687d9-ebcf-47ef-bef2-ce527bddb3e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.409140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57687d9-ebcf-47ef-bef2-ce527bddb3e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-c57687d9-ebcf-47ef-bef2-ce527bddb3e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:49.413932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:49.414791 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:49.418030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:49.418617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286e252-8451-4f29-9c17-750fd7c389e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:49.418617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e252-8451-4f29-9c17-750fd7c389e1 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.418617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e252-8451-4f29-9c17-750fd7c389e1 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.418617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e252-8451-4f29-9c17-750fd7c389e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.418940 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 220/434] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:49.418940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7575be1-0f2b-467e-85be-c50e9df00359 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.418940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7575be1-0f2b-467e-85be-c50e9df00359 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.418940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7575be1-0f2b-467e-85be-c50e9df00359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.431023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286e252-8451-4f29-9c17-750fd7c389e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:49.432402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286e252-8451-4f29-9c17-750fd7c389e1 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:49.436533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c7575be1-0f2b-467e-85be-c50e9df00359 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:49.437778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c7575be1-0f2b-467e-85be-c50e9df00359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-6286e252-8451-4f29-9c17-750fd7c389e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:49.446803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:49.447710 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:49.453227 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:49.453227 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:49.453227 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:49.453227 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 215/435] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:49.453227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286e252-8451-4f29-9c17-750fd7c389e1 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:49.486465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:49.511824 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 221/436] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:49.512662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7575be1-0f2b-467e-85be-c50e9df00359 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:49.520750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:49.564593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286e252-8451-4f29-9c17-750fd7c389e1 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:49.598844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b878fa0-c023-4eb7-bca2-02a220567591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:49.599461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b878fa0-c023-4eb7-bca2-02a220567591 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.599737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b878fa0-c023-4eb7-bca2-02a220567591 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.600010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b878fa0-c023-4eb7-bca2-02a220567591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.611475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b878fa0-c023-4eb7-bca2-02a220567591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:49.612928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b878fa0-c023-4eb7-bca2-02a220567591 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:49.626547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7575be1-0f2b-467e-85be-c50e9df00359 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-4b878fa0-c023-4eb7-bca2-02a220567591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:49.628782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:49.629692 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:49.630533 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:49.630533 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:49.630533 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:49.630533 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 216/437] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:49.630753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b878fa0-c023-4eb7-bca2-02a220567591 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:49.669261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8be3cb-5680-46cf-bf3b-d049cbb6f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:49.669857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8be3cb-5680-46cf-bf3b-d049cbb6f5a1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.670127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8be3cb-5680-46cf-bf3b-d049cbb6f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.670387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8be3cb-5680-46cf-bf3b-d049cbb6f5a1 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.671196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8be3cb-5680-46cf-bf3b-d049cbb6f5a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:49.671488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8be3cb-5680-46cf-bf3b-d049cbb6f5a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:49.673641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8be3cb-5680-46cf-bf3b-d049cbb6f5a1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:49.683455 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 222/438] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:32:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:49.684737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9466-f1d3-4c68-88cd-0130035fe9c1 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.685019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9466-f1d3-4c68-88cd-0130035fe9c1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.685310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9466-f1d3-4c68-88cd-0130035fe9c1 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.686784 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 223/439] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:49.689771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce9466-f1d3-4c68-88cd-0130035fe9c1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:49.728074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:49.741882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b878fa0-c023-4eb7-bca2-02a220567591 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:49.763089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:49.778498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2181dd1-df23-4a17-ac0c-e8e0e9560711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:49.779100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2181dd1-df23-4a17-ac0c-e8e0e9560711 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.779436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2181dd1-df23-4a17-ac0c-e8e0e9560711 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.779704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2181dd1-df23-4a17-ac0c-e8e0e9560711 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.780396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2181dd1-df23-4a17-ac0c-e8e0e9560711 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:49.780688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2181dd1-df23-4a17-ac0c-e8e0e9560711 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:49.782112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2181dd1-df23-4a17-ac0c-e8e0e9560711 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:49.796047 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 217/440] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:49.796777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2181dd1-df23-4a17-ac0c-e8e0e9560711 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:49.804807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce9466-f1d3-4c68-88cd-0130035fe9c1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:49.847336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002d7ae-f223-4530-be66-02ac81683c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:49.847943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d7ae-f223-4530-be66-02ac81683c6c None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.848216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d7ae-f223-4530-be66-02ac81683c6c None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.848654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d7ae-f223-4530-be66-02ac81683c6c None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.849327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002d7ae-f223-4530-be66-02ac81683c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:49.849621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002d7ae-f223-4530-be66-02ac81683c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:49.851159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002d7ae-f223-4530-be66-02ac81683c6c None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:49.862847 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 224/441] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:49.865095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59bab-1eec-41a0-be52-17c32897f430 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.865095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59bab-1eec-41a0-be52-17c32897f430 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.865095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59bab-1eec-41a0-be52-17c32897f430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.882827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-17f59bab-1eec-41a0-be52-17c32897f430 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:49.884107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-17f59bab-1eec-41a0-be52-17c32897f430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:49.903424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2181dd1-df23-4a17-ac0c-e8e0e9560711 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:49.936430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aa128-80f7-41fe-83c8-1d3033652831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:49.936966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aa128-80f7-41fe-83c8-1d3033652831 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:49.937269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aa128-80f7-41fe-83c8-1d3033652831 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:49.937530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aa128-80f7-41fe-83c8-1d3033652831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:49.947792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aa128-80f7-41fe-83c8-1d3033652831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:49.949266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aa128-80f7-41fe-83c8-1d3033652831 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:49.949765 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 225/442] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:49.950640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f59bab-1eec-41a0-be52-17c32897f430 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:49.959612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-c77aa128-80f7-41fe-83c8-1d3033652831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:49.965779 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:49.966647 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:49.967457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:49.967457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:49.967457 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:49.967457 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 218/443] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:49.968117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aa128-80f7-41fe-83c8-1d3033652831 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.020453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:50.064572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f59bab-1eec-41a0-be52-17c32897f430 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.073420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aa128-80f7-41fe-83c8-1d3033652831 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.100023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772b8a8-07e9-4b0c-94b0-c572b9e9f962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.100636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772b8a8-07e9-4b0c-94b0-c572b9e9f962 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.101041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772b8a8-07e9-4b0c-94b0-c572b9e9f962 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.101483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772b8a8-07e9-4b0c-94b0-c572b9e9f962 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.108495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772b8a8-07e9-4b0c-94b0-c572b9e9f962 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:50.108908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9a984-be3b-4a50-b13d-4610655a1ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.109557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9a984-be3b-4a50-b13d-4610655a1ccf None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.109836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9a984-be3b-4a50-b13d-4610655a1ccf None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.109930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772b8a8-07e9-4b0c-94b0-c572b9e9f962 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:50.110102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9a984-be3b-4a50-b13d-4610655a1ccf None admin] PATH_INFO: `/v3/roles/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.121272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9a984-be3b-4a50-b13d-4610655a1ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-a772b8a8-07e9-4b0c-94b0-c572b9e9f962 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:50.122117 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:50.122879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:50.123720 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:50.123720 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:50.123720 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:50.123720 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:50.123720 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:50.123720 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:50.123720 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:50.123720 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-cdb9a984-be3b-4a50-b13d-4610655a1ccf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:50.128060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:50.128891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:50.134008 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:50.134671 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 226/444] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:50.134671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772b8a8-07e9-4b0c-94b0-c572b9e9f962 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.134789 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 219/445] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:50.134789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9a984-be3b-4a50-b13d-4610655a1ccf None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.202503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:50.245099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772b8a8-07e9-4b0c-94b0-c572b9e9f962 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.247351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9a984-be3b-4a50-b13d-4610655a1ccf None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.262164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:50.280050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7bca6-8c6e-4efd-bd53-45f7bb426d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.280669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7bca6-8c6e-4efd-bd53-45f7bb426d83 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.280939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7bca6-8c6e-4efd-bd53-45f7bb426d83 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.281255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7bca6-8c6e-4efd-bd53-45f7bb426d83 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.283651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7506ef-30f1-48b2-bd41-511752c82da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.284238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7506ef-30f1-48b2-bd41-511752c82da0 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.284508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7506ef-30f1-48b2-bd41-511752c82da0 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.284770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7506ef-30f1-48b2-bd41-511752c82da0 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.285431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7506ef-30f1-48b2-bd41-511752c82da0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:50.285726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7506ef-30f1-48b2-bd41-511752c82da0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:50.287104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7506ef-30f1-48b2-bd41-511752c82da0 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:50.289332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7bca6-8c6e-4efd-bd53-45f7bb426d83 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:50.290739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7bca6-8c6e-4efd-bd53-45f7bb426d83 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:50.299132 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 220/446] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:50.299946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7506ef-30f1-48b2-bd41-511752c82da0 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-eca7bca6-8c6e-4efd-bd53-45f7bb426d83 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:50.301627 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:50.302461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:50.303314 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 227/447] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:50.304397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7bca6-8c6e-4efd-bd53-45f7bb426d83 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.403162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7506ef-30f1-48b2-bd41-511752c82da0 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.403690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7bca6-8c6e-4efd-bd53-45f7bb426d83 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.423159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:50.437859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a10b22-acc8-4a37-92fc-e9d4085298b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.438050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7725db-8219-4146-9e97-6eb787400e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.438450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a10b22-acc8-4a37-92fc-e9d4085298b4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.438637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7725db-8219-4146-9e97-6eb787400e3c None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.438735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a10b22-acc8-4a37-92fc-e9d4085298b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.438912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7725db-8219-4146-9e97-6eb787400e3c None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.439009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a10b22-acc8-4a37-92fc-e9d4085298b4 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.439180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7725db-8219-4146-9e97-6eb787400e3c None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.439597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a10b22-acc8-4a37-92fc-e9d4085298b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:50.439861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7725db-8219-4146-9e97-6eb787400e3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:50.439947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a10b22-acc8-4a37-92fc-e9d4085298b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:50.440159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7725db-8219-4146-9e97-6eb787400e3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:50.441257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a10b22-acc8-4a37-92fc-e9d4085298b4 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:50.441621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7725db-8219-4146-9e97-6eb787400e3c None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:50.450014 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 228/448] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:50.450715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a10b22-acc8-4a37-92fc-e9d4085298b4 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.463478 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 221/449] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:50.463478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7725db-8219-4146-9e97-6eb787400e3c None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.491747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:50.549732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a10b22-acc8-4a37-92fc-e9d4085298b4 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.558990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7725db-8219-4146-9e97-6eb787400e3c None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.581712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df5628-5a01-4fea-87fc-54b52a94a0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.582314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df5628-5a01-4fea-87fc-54b52a94a0a4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.582579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df5628-5a01-4fea-87fc-54b52a94a0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.582842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df5628-5a01-4fea-87fc-54b52a94a0a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.592101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973a3d5-5e6b-4659-8be6-cd3eaaa614ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.592490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df5628-5a01-4fea-87fc-54b52a94a0a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:50.592709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973a3d5-5e6b-4659-8be6-cd3eaaa614ad None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.592982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973a3d5-5e6b-4659-8be6-cd3eaaa614ad None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.593280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973a3d5-5e6b-4659-8be6-cd3eaaa614ad None admin] PATH_INFO: `/v3/users/glance` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-57df5628-5a01-4fea-87fc-54b52a94a0a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:50.597542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:50.598551 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:50.599358 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 229/450] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:50.600003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df5628-5a01-4fea-87fc-54b52a94a0a4 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.601936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973a3d5-5e6b-4659-8be6-cd3eaaa614ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:50.603382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973a3d5-5e6b-4659-8be6-cd3eaaa614ad None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-2973a3d5-5e6b-4659-8be6-cd3eaaa614ad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:50.614613 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:50.615509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:50.616370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:50.616370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:50.616370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:50.616370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:50.616370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:50.616370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:50.616370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:50.616370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:50.616370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 15 20:32:50.616370 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:50.620773 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 222/451] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:50.621489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973a3d5-5e6b-4659-8be6-cd3eaaa614ad None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.661914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:50.703232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:50.706489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df5628-5a01-4fea-87fc-54b52a94a0a4 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.725968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973a3d5-5e6b-4659-8be6-cd3eaaa614ad None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.739203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562bd4d-4934-4ed4-8e1c-cda8784340e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.739788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562bd4d-4934-4ed4-8e1c-cda8784340e0 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.740055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562bd4d-4934-4ed4-8e1c-cda8784340e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.740319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562bd4d-4934-4ed4-8e1c-cda8784340e0 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.741018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562bd4d-4934-4ed4-8e1c-cda8784340e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:50.741345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562bd4d-4934-4ed4-8e1c-cda8784340e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:50.743591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562bd4d-4934-4ed4-8e1c-cda8784340e0 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:50.760917 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 230/452] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:50.761138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792f3b3-fb1a-4f29-a293-42408765e22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.761463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792f3b3-fb1a-4f29-a293-42408765e22c None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.761563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562bd4d-4934-4ed4-8e1c-cda8784340e0 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.761731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792f3b3-fb1a-4f29-a293-42408765e22c None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.761992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792f3b3-fb1a-4f29-a293-42408765e22c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.775614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792f3b3-fb1a-4f29-a293-42408765e22c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:50.777083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792f3b3-fb1a-4f29-a293-42408765e22c None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-e792f3b3-fb1a-4f29-a293-42408765e22c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:50.791126 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:50.791998 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:50.792995 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:50.792995 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:50.792995 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:50.792995 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 223/453] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:50.792995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792f3b3-fb1a-4f29-a293-42408765e22c None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.864212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562bd4d-4934-4ed4-8e1c-cda8784340e0 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.886890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:50.895898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792f3b3-fb1a-4f29-a293-42408765e22c None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:50.900417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126000c2-6c3c-458f-8159-4a5aeb214e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.901097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126000c2-6c3c-458f-8159-4a5aeb214e8e None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.901375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126000c2-6c3c-458f-8159-4a5aeb214e8e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.901644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126000c2-6c3c-458f-8159-4a5aeb214e8e None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/abfea6b853134fed8ea5e32e5406bd5f/roles/492ce6c3fedb4109ac31de9467e8ef3b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.916532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126000c2-6c3c-458f-8159-4a5aeb214e8e None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=abfea6b853134fed8ea5e32e5406bd5f, role_id=492ce6c3fedb4109ac31de9467e8ef3b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:50.917993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126000c2-6c3c-458f-8159-4a5aeb214e8e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:50.930279 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 231/454] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:32:50 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/abfea6b853134fed8ea5e32e5406bd5f/roles/492ce6c3fedb4109ac31de9467e8ef3b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:32:50.931057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126000c2-6c3c-458f-8159-4a5aeb214e8e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:50.933020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2865a-11d2-4cf1-9407-29ad173ad449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:50.933834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2865a-11d2-4cf1-9407-29ad173ad449 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:50.934294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2865a-11d2-4cf1-9407-29ad173ad449 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:50.934564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2865a-11d2-4cf1-9407-29ad173ad449 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:50.935300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2865a-11d2-4cf1-9407-29ad173ad449 None admin] RBAC: Adding query filter params (name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:50.935600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2865a-11d2-4cf1-9407-29ad173ad449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:50.937091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2865a-11d2-4cf1-9407-29ad173ad449 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:50.943444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:50.947375 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 224/455] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:50.953439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2865a-11d2-4cf1-9407-29ad173ad449 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:51.045061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126000c2-6c3c-458f-8159-4a5aeb214e8e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:51.068292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2865a-11d2-4cf1-9407-29ad173ad449 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:51.083293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57e5e7-4304-4d95-87b2-2848b6064e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:51.083879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57e5e7-4304-4d95-87b2-2848b6064e7d None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:51.084152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57e5e7-4304-4d95-87b2-2848b6064e7d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:51.084416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57e5e7-4304-4d95-87b2-2848b6064e7d None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:51.085258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57e5e7-4304-4d95-87b2-2848b6064e7d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:51.086073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57e5e7-4304-4d95-87b2-2848b6064e7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:51.087502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57e5e7-4304-4d95-87b2-2848b6064e7d None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:51.107664 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 232/456] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:51.108590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bb2f7-a681-4279-9416-f7584d7e8a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:51.108967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57e5e7-4304-4d95-87b2-2848b6064e7d None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:51.109236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb2f7-a681-4279-9416-f7584d7e8a96 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:51.109511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb2f7-a681-4279-9416-f7584d7e8a96 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:51.109829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb2f7-a681-4279-9416-f7584d7e8a96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:51.138389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:51.183769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:51.187454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bb2f7-a681-4279-9416-f7584d7e8a96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:51.192595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bb2f7-a681-4279-9416-f7584d7e8a96 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:51.223257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57e5e7-4304-4d95-87b2-2848b6064e7d None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:51.258859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e12a5-e852-484f-ace0-633d4e0238ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:51.259457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e12a5-e852-484f-ace0-633d4e0238ae None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:51.259728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e12a5-e852-484f-ace0-633d4e0238ae None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:51.259993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e12a5-e852-484f-ace0-633d4e0238ae None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:51.260619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e12a5-e852-484f-ace0-633d4e0238ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:51.260914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e12a5-e852-484f-ace0-633d4e0238ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:51.262387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e12a5-e852-484f-ace0-633d4e0238ae None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:51.274634 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 233/457] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:51.275437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e12a5-e852-484f-ace0-633d4e0238ae None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:51.302667 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 225/458] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:51.303954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bb2f7-a681-4279-9416-f7584d7e8a96 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:51.360876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:51.390679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e12a5-e852-484f-ace0-633d4e0238ae None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:51.408146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bb2f7-a681-4279-9416-f7584d7e8a96 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:51.418410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:51.428225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32720fd5-a7bf-4b1d-aa59-5c926d36f854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:51.428844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32720fd5-a7bf-4b1d-aa59-5c926d36f854 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:51.429675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32720fd5-a7bf-4b1d-aa59-5c926d36f854 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:51.429930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32720fd5-a7bf-4b1d-aa59-5c926d36f854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:51.441852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32720fd5-a7bf-4b1d-aa59-5c926d36f854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:51.443324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32720fd5-a7bf-4b1d-aa59-5c926d36f854 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:51.445087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1c597-11bf-44a8-90f0-d452808d7782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:51.445287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1c597-11bf-44a8-90f0-d452808d7782 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:51.445693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1c597-11bf-44a8-90f0-d452808d7782 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:51.445961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1c597-11bf-44a8-90f0-d452808d7782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-32720fd5-a7bf-4b1d-aa59-5c926d36f854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:51.460932 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:51.462097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:51.462906 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:51.462906 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:51.462906 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:51.462906 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 234/459] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:51.462906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32720fd5-a7bf-4b1d-aa59-5c926d36f854 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:51.518037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1c597-11bf-44a8-90f0-d452808d7782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:51.519412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1c597-11bf-44a8-90f0-d452808d7782 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:51.572529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:51.573599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32720fd5-a7bf-4b1d-aa59-5c926d36f854 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:51.608279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dbe08-f8a6-4577-9349-08aef2a9cb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:51.608922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dbe08-f8a6-4577-9349-08aef2a9cb0f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:51.609222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dbe08-f8a6-4577-9349-08aef2a9cb0f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:51.609505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dbe08-f8a6-4577-9349-08aef2a9cb0f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:51.617962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dbe08-f8a6-4577-9349-08aef2a9cb0f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:51.619394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dbe08-f8a6-4577-9349-08aef2a9cb0f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:51.624172 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 226/460] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:51.625191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1c597-11bf-44a8-90f0-d452808d7782 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-f78dbe08-f8a6-4577-9349-08aef2a9cb0f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:51.634415 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:51.635323 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 235/461] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:51.636562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dbe08-f8a6-4577-9349-08aef2a9cb0f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:51.678493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:51.738952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1c597-11bf-44a8-90f0-d452808d7782 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:51.745414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dbe08-f8a6-4577-9349-08aef2a9cb0f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:51.775764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d875d7-5454-4445-8c44-d016cf321984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:51.776452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d875d7-5454-4445-8c44-d016cf321984 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:51.776822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d875d7-5454-4445-8c44-d016cf321984 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:51.777256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d875d7-5454-4445-8c44-d016cf321984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:51.782019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fdc38-3b55-4657-9777-712a6447f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:51.782609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fdc38-3b55-4657-9777-712a6447f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:51.782879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fdc38-3b55-4657-9777-712a6447f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:51.783144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fdc38-3b55-4657-9777-712a6447f5d6 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:51.783773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fdc38-3b55-4657-9777-712a6447f5d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:51.784067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fdc38-3b55-4657-9777-712a6447f5d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:51.785452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fdc38-3b55-4657-9777-712a6447f5d6 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:51.788832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d875d7-5454-4445-8c44-d016cf321984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:51.790570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d875d7-5454-4445-8c44-d016cf321984 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:51.797434 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 236/462] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:51.798222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fdc38-3b55-4657-9777-712a6447f5d6 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-59d875d7-5454-4445-8c44-d016cf321984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:51.810756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:51.811841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:51.812814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:51.812814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:51.812814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:51.812814 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 227/463] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:51.813865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d875d7-5454-4445-8c44-d016cf321984 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:51.824187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:51.914202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:51.918882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fdc38-3b55-4657-9777-712a6447f5d6 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:51.932090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d875d7-5454-4445-8c44-d016cf321984 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:51.955478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaedaf0-1a23-45ec-b1a8-093777125edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:51.956087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaedaf0-1a23-45ec-b1a8-093777125edc None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:51.956385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaedaf0-1a23-45ec-b1a8-093777125edc None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:51.956628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaedaf0-1a23-45ec-b1a8-093777125edc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:51.967217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00467ec3-9ce8-4df6-af4d-f2918788ce17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:51.967357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaedaf0-1a23-45ec-b1a8-093777125edc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:51.968069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00467ec3-9ce8-4df6-af4d-f2918788ce17 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:51.968508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00467ec3-9ce8-4df6-af4d-f2918788ce17 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:51.968732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaedaf0-1a23-45ec-b1a8-093777125edc None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:51.969101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00467ec3-9ce8-4df6-af4d-f2918788ce17 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:51.969817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00467ec3-9ce8-4df6-af4d-f2918788ce17 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:51.970119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00467ec3-9ce8-4df6-af4d-f2918788ce17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:51.971777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00467ec3-9ce8-4df6-af4d-f2918788ce17 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-3aaedaf0-1a23-45ec-b1a8-093777125edc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:51.983309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:51.984221 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:51.985061 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:51.985061 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:51.985061 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:51.985061 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 237/464] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:51.985274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaedaf0-1a23-45ec-b1a8-093777125edc None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.000900 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 228/465] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Oct 15 20:32:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:52.000900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00467ec3-9ce8-4df6-af4d-f2918788ce17 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.066717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:52.101525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaedaf0-1a23-45ec-b1a8-093777125edc None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:52.109228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00467ec3-9ce8-4df6-af4d-f2918788ce17 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:52.136486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853f8d5-ceeb-4167-98be-7c0dc2a3a561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:52.137363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853f8d5-ceeb-4167-98be-7c0dc2a3a561 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:52.137763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853f8d5-ceeb-4167-98be-7c0dc2a3a561 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:52.138879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853f8d5-ceeb-4167-98be-7c0dc2a3a561 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:52.147011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf7818-4b2a-4cc8-926c-eb1903d89e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:52.147618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf7818-4b2a-4cc8-926c-eb1903d89e37 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:52.148101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf7818-4b2a-4cc8-926c-eb1903d89e37 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:52.148371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf7818-4b2a-4cc8-926c-eb1903d89e37 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:52.148638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853f8d5-ceeb-4167-98be-7c0dc2a3a561 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:52.149007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf7818-4b2a-4cc8-926c-eb1903d89e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:52.149339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf7818-4b2a-4cc8-926c-eb1903d89e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:52.150545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853f8d5-ceeb-4167-98be-7c0dc2a3a561 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:52.150734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf7818-4b2a-4cc8-926c-eb1903d89e37 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-b853f8d5-ceeb-4167-98be-7c0dc2a3a561 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:52.162207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:52.163276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:52.164276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:52.164276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:52.164276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:52.164276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:52.164276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:52.164276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:52.164276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:52.164276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:52.164276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 20:32:52.164276 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:52.166069 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 238/466] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:52.166962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853f8d5-ceeb-4167-98be-7c0dc2a3a561 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.167744 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 229/467] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:52.168470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf7818-4b2a-4cc8-926c-eb1903d89e37 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.184062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:52.275206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf7818-4b2a-4cc8-926c-eb1903d89e37 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:52.291121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853f8d5-ceeb-4167-98be-7c0dc2a3a561 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:52.298503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:52.312602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39461c03-8df3-42f0-a2c0-002d70f2ec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:52.313277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39461c03-8df3-42f0-a2c0-002d70f2ec48 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:52.313550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39461c03-8df3-42f0-a2c0-002d70f2ec48 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:52.313814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39461c03-8df3-42f0-a2c0-002d70f2ec48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:52.328848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d6c8e-efbb-4def-ad8f-a8db912879a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:52.329414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d6c8e-efbb-4def-ad8f-a8db912879a6 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:52.329686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d6c8e-efbb-4def-ad8f-a8db912879a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:52.329943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d6c8e-efbb-4def-ad8f-a8db912879a6 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:52.330582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d6c8e-efbb-4def-ad8f-a8db912879a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:52.330878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d6c8e-efbb-4def-ad8f-a8db912879a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:52.332232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d6c8e-efbb-4def-ad8f-a8db912879a6 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:52.344536 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 239/468] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:52.345185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d6c8e-efbb-4def-ad8f-a8db912879a6 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.391603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39461c03-8df3-42f0-a2c0-002d70f2ec48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:52.393242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39461c03-8df3-42f0-a2c0-002d70f2ec48 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:52.418887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:52.453863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d6c8e-efbb-4def-ad8f-a8db912879a6 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:52.489959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc7f7b-800e-4b40-8c91-b99a82d23bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:52.490571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc7f7b-800e-4b40-8c91-b99a82d23bb5 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:52.490844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc7f7b-800e-4b40-8c91-b99a82d23bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:52.491105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc7f7b-800e-4b40-8c91-b99a82d23bb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:52.498069 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 230/469] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:52.498069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39461c03-8df3-42f0-a2c0-002d70f2ec48 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.504659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc7f7b-800e-4b40-8c91-b99a82d23bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:52.506223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc7f7b-800e-4b40-8c91-b99a82d23bb5 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-5ffc7f7b-800e-4b40-8c91-b99a82d23bb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:52.518698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:52.519573 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:52.525435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:52.525435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:52.525435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:52.525435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:52.525435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:52.525435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:52.525435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:52.525435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:52.525435 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 240/470] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:52.525435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc7f7b-800e-4b40-8c91-b99a82d23bb5 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.528517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:52.611299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39461c03-8df3-42f0-a2c0-002d70f2ec48 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:52.631429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc7f7b-800e-4b40-8c91-b99a82d23bb5 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:52.647235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61e576-a052-475a-ba33-37a8b96312a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:52.648531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61e576-a052-475a-ba33-37a8b96312a5 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:52.648909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61e576-a052-475a-ba33-37a8b96312a5 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:52.649089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61e576-a052-475a-ba33-37a8b96312a5 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:52.649809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61e576-a052-475a-ba33-37a8b96312a5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:52.650223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61e576-a052-475a-ba33-37a8b96312a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:52.651528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61e576-a052-475a-ba33-37a8b96312a5 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:52.663156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:52.670654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a149daa-f9d1-4a84-8957-7592ea2566d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:52.671119 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 231/471] 10.222.0.16 () {56 vars in 1139 bytes} [Sat Oct 15 20:32:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:52.671889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61e576-a052-475a-ba33-37a8b96312a5 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.673620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a149daa-f9d1-4a84-8957-7592ea2566d8 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:52.674385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a149daa-f9d1-4a84-8957-7592ea2566d8 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:52.674849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a149daa-f9d1-4a84-8957-7592ea2566d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:52.684208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a149daa-f9d1-4a84-8957-7592ea2566d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:52.686038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a149daa-f9d1-4a84-8957-7592ea2566d8 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-5a149daa-f9d1-4a84-8957-7592ea2566d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:52.698832 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:52.699681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:52.700507 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:52.700507 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:52.700507 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:52.700507 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:52.700507 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:52.700507 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:52.700507 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:52.700507 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:52.700507 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 241/472] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:52.700954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a149daa-f9d1-4a84-8957-7592ea2566d8 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.769381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:52.789476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61e576-a052-475a-ba33-37a8b96312a5 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:52.808263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a149daa-f9d1-4a84-8957-7592ea2566d8 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:52.825368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f29004-5f09-4ead-9f15-0d2b2bb5071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:52.826076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f29004-5f09-4ead-9f15-0d2b2bb5071f None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:52.826363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f29004-5f09-4ead-9f15-0d2b2bb5071f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:52.826627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f29004-5f09-4ead-9f15-0d2b2bb5071f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:52.837868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f29004-5f09-4ead-9f15-0d2b2bb5071f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:52.839609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f29004-5f09-4ead-9f15-0d2b2bb5071f None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:52.843579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bb135-e089-4f23-b252-40d8ca8a837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:52.844174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bb135-e089-4f23-b252-40d8ca8a837a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:52.844446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bb135-e089-4f23-b252-40d8ca8a837a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:52.844709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bb135-e089-4f23-b252-40d8ca8a837a None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:52.845418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bb135-e089-4f23-b252-40d8ca8a837a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:52.845713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bb135-e089-4f23-b252-40d8ca8a837a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:52.847117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bb135-e089-4f23-b252-40d8ca8a837a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-74f29004-5f09-4ead-9f15-0d2b2bb5071f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:52.855955 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:52.856891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:52.858642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:52.858642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:52.858642 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:52.858642 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 232/473] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:52.858642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f29004-5f09-4ead-9f15-0d2b2bb5071f None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.865863 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 242/474] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:52.870463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bb135-e089-4f23-b252-40d8ca8a837a None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:52.924466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:52.978763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bb135-e089-4f23-b252-40d8ca8a837a None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:52.999028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f29004-5f09-4ead-9f15-0d2b2bb5071f None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:53.017419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d26f9e-3355-4389-904f-eda432c6e89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:53.018317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d26f9e-3355-4389-904f-eda432c6e89c None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.018788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d26f9e-3355-4389-904f-eda432c6e89c None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.019250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d26f9e-3355-4389-904f-eda432c6e89c None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.020162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d26f9e-3355-4389-904f-eda432c6e89c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:53.020562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d26f9e-3355-4389-904f-eda432c6e89c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:53.022032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d26f9e-3355-4389-904f-eda432c6e89c None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:53.034495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e21fe-b245-47c0-9432-e60a42eebddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:53.040205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e21fe-b245-47c0-9432-e60a42eebddb None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.040449 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 243/475] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:53.040622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e21fe-b245-47c0-9432-e60a42eebddb None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.040622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e21fe-b245-47c0-9432-e60a42eebddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.040765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d26f9e-3355-4389-904f-eda432c6e89c None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:53.040831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:53.110963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e21fe-b245-47c0-9432-e60a42eebddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:53.112419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e21fe-b245-47c0-9432-e60a42eebddb None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:53.149000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d26f9e-3355-4389-904f-eda432c6e89c None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:53.160184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:53.186327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227389ed-1dae-4c59-afd4-120ccd94fa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:53.186919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227389ed-1dae-4c59-afd4-120ccd94fa93 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.187191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227389ed-1dae-4c59-afd4-120ccd94fa93 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.187453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227389ed-1dae-4c59-afd4-120ccd94fa93 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.188082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227389ed-1dae-4c59-afd4-120ccd94fa93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:53.188376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227389ed-1dae-4c59-afd4-120ccd94fa93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:53.195108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227389ed-1dae-4c59-afd4-120ccd94fa93 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:53.206988 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 244/476] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:53.212519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227389ed-1dae-4c59-afd4-120ccd94fa93 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:53.225869 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 233/477] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:53.226968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e21fe-b245-47c0-9432-e60a42eebddb None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:53.261698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:53.329413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227389ed-1dae-4c59-afd4-120ccd94fa93 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:53.333650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e21fe-b245-47c0-9432-e60a42eebddb None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:53.365916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20822b-8002-41e0-be6b-a67cfd9fd663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:53.366591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20822b-8002-41e0-be6b-a67cfd9fd663 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.367084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20822b-8002-41e0-be6b-a67cfd9fd663 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.367339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20822b-8002-41e0-be6b-a67cfd9fd663 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.368313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20822b-8002-41e0-be6b-a67cfd9fd663 None admin] RBAC: Adding query filter params (role.id=492ce6c3fedb4109ac31de9467e8ef3b, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=6225c81d2620463886d8a936b59784df) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:53.368614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20822b-8002-41e0-be6b-a67cfd9fd663 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:53.369479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbb6c6-65b8-4249-bd06-d9f44aa560a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:53.370052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20822b-8002-41e0-be6b-a67cfd9fd663 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:53.370161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbb6c6-65b8-4249-bd06-d9f44aa560a0 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.370324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbb6c6-65b8-4249-bd06-d9f44aa560a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.370594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbb6c6-65b8-4249-bd06-d9f44aa560a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.371295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbb6c6-65b8-4249-bd06-d9f44aa560a0 None admin] RBAC: Adding query filter params (role.id=492ce6c3fedb4109ac31de9467e8ef3b, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=af8591ebfa174c91853b673360326a9c) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:53.371588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbb6c6-65b8-4249-bd06-d9f44aa560a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:53.373014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbb6c6-65b8-4249-bd06-d9f44aa560a0 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:53.375893 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 245/478] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:32:53 2022] GET /identity/v3/role_assignments?user.id=6225c81d2620463886d8a936b59784df&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=492ce6c3fedb4109ac31de9467e8ef3b => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:53.376661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20822b-8002-41e0-be6b-a67cfd9fd663 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:53.381078 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 234/479] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:32:53 2022] GET /identity/v3/role_assignments?user.id=af8591ebfa174c91853b673360326a9c&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=492ce6c3fedb4109ac31de9467e8ef3b => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:53.381078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbb6c6-65b8-4249-bd06-d9f44aa560a0 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:53.434466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:53.516268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:53.521404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbb6c6-65b8-4249-bd06-d9f44aa560a0 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:53.526441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20822b-8002-41e0-be6b-a67cfd9fd663 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:53.566527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127db2cd-235b-4929-891c-e404a1e849d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:53.567601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127db2cd-235b-4929-891c-e404a1e849d3 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.568135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127db2cd-235b-4929-891c-e404a1e849d3 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.568653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127db2cd-235b-4929-891c-e404a1e849d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.570780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50524cbb-0321-45f9-b637-7643c9b67990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:53.571690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50524cbb-0321-45f9-b637-7643c9b67990 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.572058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50524cbb-0321-45f9-b637-7643c9b67990 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.572417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50524cbb-0321-45f9-b637-7643c9b67990 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.581241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50524cbb-0321-45f9-b637-7643c9b67990 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:53.582897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50524cbb-0321-45f9-b637-7643c9b67990 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:53.583927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127db2cd-235b-4929-891c-e404a1e849d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:53.586067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127db2cd-235b-4929-891c-e404a1e849d3 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-50524cbb-0321-45f9-b637-7643c9b67990 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:53.597499 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:53.598590 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:53.599628 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:53.599628 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:53.599628 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:53.599628 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:53.599628 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:53.599628 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:53.599628 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:53.599628 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:53.602421 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 246/480] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:53.603934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8678cec-c030-4cbd-9372-0b25794222bf None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.604308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8678cec-c030-4cbd-9372-0b25794222bf None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.604669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8678cec-c030-4cbd-9372-0b25794222bf None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.606320 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 247/481] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:53.607593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8678cec-c030-4cbd-9372-0b25794222bf None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-127db2cd-235b-4929-891c-e404a1e849d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:53.608673 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:53.609743 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:53.610772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:53.610772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:53.610772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:53.612850 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 235/482] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:53.614601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292ba9e-889e-41ea-a805-5c82d4316c77 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.614881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292ba9e-889e-41ea-a805-5c82d4316c77 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.615151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292ba9e-889e-41ea-a805-5c82d4316c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.639768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f292ba9e-889e-41ea-a805-5c82d4316c77 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:53.640401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f292ba9e-889e-41ea-a805-5c82d4316c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:53.723519 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 236/483] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:53.724464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292ba9e-889e-41ea-a805-5c82d4316c77 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:53.735193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:53.748294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8678cec-c030-4cbd-9372-0b25794222bf None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:53.787204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3034c6-ec82-4b02-bb12-c127fed048ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:53.788168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3034c6-ec82-4b02-bb12-c127fed048ca None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.788635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3034c6-ec82-4b02-bb12-c127fed048ca None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.789059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3034c6-ec82-4b02-bb12-c127fed048ca None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.789985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3034c6-ec82-4b02-bb12-c127fed048ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:53.790765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3034c6-ec82-4b02-bb12-c127fed048ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:53.793062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3034c6-ec82-4b02-bb12-c127fed048ca None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:53.813645 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 248/484] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:53.815225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:53.816018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3034c6-ec82-4b02-bb12-c127fed048ca None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:53.837598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292ba9e-889e-41ea-a805-5c82d4316c77 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:53.885501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc54cc-7fe5-4ece-b298-54ff60111a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:53.886138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc54cc-7fe5-4ece-b298-54ff60111a56 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:53.886410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc54cc-7fe5-4ece-b298-54ff60111a56 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:53.886677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc54cc-7fe5-4ece-b298-54ff60111a56 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:53.887321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc54cc-7fe5-4ece-b298-54ff60111a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:53.887616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc54cc-7fe5-4ece-b298-54ff60111a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:53.889057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc54cc-7fe5-4ece-b298-54ff60111a56 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:53.902294 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 237/485] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:53.907673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc54cc-7fe5-4ece-b298-54ff60111a56 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:53.955480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3034c6-ec82-4b02-bb12-c127fed048ca None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:53.999822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d5940-0de0-41a7-872c-afeeb09ddd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:54.000753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d5940-0de0-41a7-872c-afeeb09ddd8a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:54.001247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d5940-0de0-41a7-872c-afeeb09ddd8a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:54.001707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d5940-0de0-41a7-872c-afeeb09ddd8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:54.002636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d5940-0de0-41a7-872c-afeeb09ddd8a None admin] RBAC: Adding query filter params (role.id=492ce6c3fedb4109ac31de9467e8ef3b, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=aa85b647a4be4d81b966bb50691bd3e0) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:54.003159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d5940-0de0-41a7-872c-afeeb09ddd8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:54.005336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d5940-0de0-41a7-872c-afeeb09ddd8a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:54.015058 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 249/486] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:32:53 2022] GET /identity/v3/role_assignments?user.id=aa85b647a4be4d81b966bb50691bd3e0&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=492ce6c3fedb4109ac31de9467e8ef3b => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:54.047678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc54cc-7fe5-4ece-b298-54ff60111a56 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:54.098447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07340e28-4015-45ed-8902-c22a114b7aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:54.099134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07340e28-4015-45ed-8902-c22a114b7aa0 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:54.099405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07340e28-4015-45ed-8902-c22a114b7aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:54.099669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07340e28-4015-45ed-8902-c22a114b7aa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:54.108671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07340e28-4015-45ed-8902-c22a114b7aa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:54.110179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07340e28-4015-45ed-8902-c22a114b7aa0 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:54.112968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-07340e28-4015-45ed-8902-c22a114b7aa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:54.124717 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:54.125650 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:54.126466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:54.126466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:54.126466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:54.126466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:54.126466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:54.126466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:54.126466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:54.126466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:54.130728 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 238/487] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:54.136999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d5940-0de0-41a7-872c-afeeb09ddd8a None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:54.167005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:54.250019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07340e28-4015-45ed-8902-c22a114b7aa0 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:54.274325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d5940-0de0-41a7-872c-afeeb09ddd8a None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:54.313901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2be883-087e-40f6-9382-8389f63626e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:54.314911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2be883-087e-40f6-9382-8389f63626e3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:54.315361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2be883-087e-40f6-9382-8389f63626e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:54.315996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2be883-087e-40f6-9382-8389f63626e3 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:54.317822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2be883-087e-40f6-9382-8389f63626e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:54.318436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2be883-087e-40f6-9382-8389f63626e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:54.320865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2be883-087e-40f6-9382-8389f63626e3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:54.336722 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 250/488] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:54.344528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2be883-087e-40f6-9382-8389f63626e3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:54.422679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07340e28-4015-45ed-8902-c22a114b7aa0 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:54.443977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:54.470732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2be883-087e-40f6-9382-8389f63626e3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:54.472442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e247e-b4f4-4628-99b0-70b1aa30cd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:54.474245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e247e-b4f4-4628-99b0-70b1aa30cd85 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:54.474778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e247e-b4f4-4628-99b0-70b1aa30cd85 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:54.475236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e247e-b4f4-4628-99b0-70b1aa30cd85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:54.488891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e247e-b4f4-4628-99b0-70b1aa30cd85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-a67e247e-b4f4-4628-99b0-70b1aa30cd85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:54.495651 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:54.496699 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:32:54.497652 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:54.500203 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 239/489] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:32:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:54.505603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e247e-b4f4-4628-99b0-70b1aa30cd85 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:54.512675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9890b-7bc1-447d-98fd-bb6fe67b9f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:54.513307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9890b-7bc1-447d-98fd-bb6fe67b9f09 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:54.513580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9890b-7bc1-447d-98fd-bb6fe67b9f09 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:54.513842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9890b-7bc1-447d-98fd-bb6fe67b9f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:54.514627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9890b-7bc1-447d-98fd-bb6fe67b9f09 None admin] RBAC: Adding query filter params (role.id=492ce6c3fedb4109ac31de9467e8ef3b, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=176e98e72ba840c4a6546519b4c98d90) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:54.514931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9890b-7bc1-447d-98fd-bb6fe67b9f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:54.516393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9890b-7bc1-447d-98fd-bb6fe67b9f09 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:54.523280 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 251/490] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:32:54 2022] GET /identity/v3/role_assignments?user.id=176e98e72ba840c4a6546519b4c98d90&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=492ce6c3fedb4109ac31de9467e8ef3b => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:54.594703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:54.654872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e247e-b4f4-4628-99b0-70b1aa30cd85 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:54.704946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fd461-076f-4b29-b657-ed30203546e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:54.705878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fd461-076f-4b29-b657-ed30203546e7 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:54.706380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fd461-076f-4b29-b657-ed30203546e7 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:54.706808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fd461-076f-4b29-b657-ed30203546e7 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:54.707595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fd461-076f-4b29-b657-ed30203546e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:54.708026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fd461-076f-4b29-b657-ed30203546e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:54.709569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fd461-076f-4b29-b657-ed30203546e7 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:54.718826 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 240/491] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:32:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:54.724699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9890b-7bc1-447d-98fd-bb6fe67b9f09 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:54.842122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9890b-7bc1-447d-98fd-bb6fe67b9f09 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:54.880596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03e32c-5c4f-48e1-8d93-d9ab7a6b31ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:54.881271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03e32c-5c4f-48e1-8d93-d9ab7a6b31ca None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:54.881538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03e32c-5c4f-48e1-8d93-d9ab7a6b31ca None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:54.882326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03e32c-5c4f-48e1-8d93-d9ab7a6b31ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:54.894868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03e32c-5c4f-48e1-8d93-d9ab7a6b31ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:54.896363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03e32c-5c4f-48e1-8d93-d9ab7a6b31ca None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:54.898700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-bd03e32c-5c4f-48e1-8d93-d9ab7a6b31ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:54.913617 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:54.914682 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:54.916161 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:54.916161 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:54.916161 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:54.918682 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 252/492] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:54.924095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fd461-076f-4b29-b657-ed30203546e7 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:55.039936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fd461-076f-4b29-b657-ed30203546e7 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:55.063475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:55.076461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7a406-66cd-435f-8dc8-1ccc97b7636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:55.077074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7a406-66cd-435f-8dc8-1ccc97b7636d None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:55.077354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7a406-66cd-435f-8dc8-1ccc97b7636d None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:55.077615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7a406-66cd-435f-8dc8-1ccc97b7636d None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:55.078251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7a406-66cd-435f-8dc8-1ccc97b7636d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:55.078545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7a406-66cd-435f-8dc8-1ccc97b7636d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:55.079915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7a406-66cd-435f-8dc8-1ccc97b7636d None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:55.092708 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 241/493] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:55.097842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03e32c-5c4f-48e1-8d93-d9ab7a6b31ca None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:55.287813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03e32c-5c4f-48e1-8d93-d9ab7a6b31ca None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:55.328980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c1f92-db21-434b-8eed-99fd692e39ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:55.333907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c1f92-db21-434b-8eed-99fd692e39ee None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:55.334343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c1f92-db21-434b-8eed-99fd692e39ee None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:55.334761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c1f92-db21-434b-8eed-99fd692e39ee None admin] PATH_INFO: `/v3/users/swift` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:55.343172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c1f92-db21-434b-8eed-99fd692e39ee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:55.344816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c1f92-db21-434b-8eed-99fd692e39ee None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-a60c1f92-db21-434b-8eed-99fd692e39ee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:55.358488 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:55.359596 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:55.360770 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:55.360770 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:55.360770 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:55.360770 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:55.360770 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:55.360770 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:55.360770 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:55.360770 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:55.360770 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 20:32:55.360770 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:55.363026 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 253/494] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:32:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:55.369229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7a406-66cd-435f-8dc8-1ccc97b7636d None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:55.498462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7a406-66cd-435f-8dc8-1ccc97b7636d None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:55.546619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38928779-2cf5-4b7e-8eac-fddf1c66bfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:55.547221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928779-2cf5-4b7e-8eac-fddf1c66bfe9 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:55.547499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928779-2cf5-4b7e-8eac-fddf1c66bfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:55.547769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928779-2cf5-4b7e-8eac-fddf1c66bfe9 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:55.548488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38928779-2cf5-4b7e-8eac-fddf1c66bfe9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:55.548786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38928779-2cf5-4b7e-8eac-fddf1c66bfe9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:55.550238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38928779-2cf5-4b7e-8eac-fddf1c66bfe9 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:55.574441 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 242/495] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Oct 15 20:32:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:55.579780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c1f92-db21-434b-8eed-99fd692e39ee None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:55.683912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:55.749584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c1f92-db21-434b-8eed-99fd692e39ee None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:55.814149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef1deb-4480-4d41-b6c6-69f947b0e375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:55.814755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1deb-4480-4d41-b6c6-69f947b0e375 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:55.815029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1deb-4480-4d41-b6c6-69f947b0e375 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:55.815297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1deb-4480-4d41-b6c6-69f947b0e375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:55.899170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef1deb-4480-4d41-b6c6-69f947b0e375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:55.900830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef1deb-4480-4d41-b6c6-69f947b0e375 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:55.982439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:56.030239 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 254/496] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:32:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 20:32:56.038427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38928779-2cf5-4b7e-8eac-fddf1c66bfe9 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:56.092610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:56.170243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38928779-2cf5-4b7e-8eac-fddf1c66bfe9 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:56.245150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b842a-3e68-4ab7-b37f-9582a7374dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:56.245711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b842a-3e68-4ab7-b37f-9582a7374dd8 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:56.245987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b842a-3e68-4ab7-b37f-9582a7374dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:56.246258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b842a-3e68-4ab7-b37f-9582a7374dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:56.264718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b842a-3e68-4ab7-b37f-9582a7374dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:56.266212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b842a-3e68-4ab7-b37f-9582a7374dd8 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-bd1b842a-3e68-4ab7-b37f-9582a7374dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:56.289680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:56.290772 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:56.291658 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:56.291658 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:56.291658 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:56.291836 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 243/497] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:56.296415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef1deb-4480-4d41-b6c6-69f947b0e375 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:56.418532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ff728-a248-4cf0-8145-8a9bc4397c38 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:56.419194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ff728-a248-4cf0-8145-8a9bc4397c38 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:56.419670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ff728-a248-4cf0-8145-8a9bc4397c38 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:56.421493 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 244/498] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:56.426843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6736-95dc-49f9-bbc2-468f732ab194 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:56.427378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6736-95dc-49f9-bbc2-468f732ab194 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:56.427850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6736-95dc-49f9-bbc2-468f732ab194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:56.449289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef1deb-4480-4d41-b6c6-69f947b0e375 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:56.461734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-debd6736-95dc-49f9-bbc2-468f732ab194 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:56.463238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-debd6736-95dc-49f9-bbc2-468f732ab194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:56.507489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc9deb-6c81-4901-8e67-8e8122b5f49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:56.508001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc9deb-6c81-4901-8e67-8e8122b5f49f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:56.508275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc9deb-6c81-4901-8e67-8e8122b5f49f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:56.508544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc9deb-6c81-4901-8e67-8e8122b5f49f None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:56.509214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc9deb-6c81-4901-8e67-8e8122b5f49f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:56.509523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc9deb-6c81-4901-8e67-8e8122b5f49f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:56.510870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc9deb-6c81-4901-8e67-8e8122b5f49f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:56.522976 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 255/499] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:56.524371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce6b31-5e1c-4358-b6d2-cc817edae4be None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:56.524652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce6b31-5e1c-4358-b6d2-cc817edae4be None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:56.524922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce6b31-5e1c-4358-b6d2-cc817edae4be None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:56.530375 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 256/500] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:56.531118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce6b31-5e1c-4358-b6d2-cc817edae4be None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:56.548943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:56.562294 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 245/501] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:32:56.566415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8581d-d7b6-4cee-b3d3-80df4b64fcfa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:56.566693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8581d-d7b6-4cee-b3d3-80df4b64fcfa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:56.566959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8581d-d7b6-4cee-b3d3-80df4b64fcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:56.587192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e0d8581d-d7b6-4cee-b3d3-80df4b64fcfa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:56.593058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e0d8581d-d7b6-4cee-b3d3-80df4b64fcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:56.614413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:56.673071 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 246/502] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:56.738580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce6b31-5e1c-4358-b6d2-cc817edae4be None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:56.788895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066880f5-a7a2-49c5-ba52-210f9c6243e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:56.789465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066880f5-a7a2-49c5-ba52-210f9c6243e7 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:56.789737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066880f5-a7a2-49c5-ba52-210f9c6243e7 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:56.789997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066880f5-a7a2-49c5-ba52-210f9c6243e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:56.802084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066880f5-a7a2-49c5-ba52-210f9c6243e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:56.803508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066880f5-a7a2-49c5-ba52-210f9c6243e7 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-066880f5-a7a2-49c5-ba52-210f9c6243e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:56.818909 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:56.820474 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:56.822233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:56.822233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:56.822233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:56.822233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:56.822233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:56.822233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:56.822233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:56.822233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:56.822233 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 257/503] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:56.825786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8581d-d7b6-4cee-b3d3-80df4b64fcfa None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:56.948161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066880f5-a7a2-49c5-ba52-210f9c6243e7 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:56.977812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8581d-d7b6-4cee-b3d3-80df4b64fcfa None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:57.039045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb810c-c8ad-449f-aadb-6da8a45ddcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:57.039675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb810c-c8ad-449f-aadb-6da8a45ddcfb None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.039950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb810c-c8ad-449f-aadb-6da8a45ddcfb None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.040219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb810c-c8ad-449f-aadb-6da8a45ddcfb None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.040913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb810c-c8ad-449f-aadb-6da8a45ddcfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:57.041235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb810c-c8ad-449f-aadb-6da8a45ddcfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:57.042665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb810c-c8ad-449f-aadb-6da8a45ddcfb None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:57.067451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:57.075497 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 247/504] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:57.081445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb810c-c8ad-449f-aadb-6da8a45ddcfb None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:57.095460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066880f5-a7a2-49c5-ba52-210f9c6243e7 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:57.123191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:57.132775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d2c65-c747-4a60-a612-26a7dba2becb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:57.133482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d2c65-c747-4a60-a612-26a7dba2becb None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.133762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d2c65-c747-4a60-a612-26a7dba2becb None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.134034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d2c65-c747-4a60-a612-26a7dba2becb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.145762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d2c65-c747-4a60-a612-26a7dba2becb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:57.147473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d2c65-c747-4a60-a612-26a7dba2becb None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-fb1d2c65-c747-4a60-a612-26a7dba2becb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:57.163591 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:57.164553 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:57.165441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:57.165441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:57.165441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:57.166051 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 258/505] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:57.166861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d2c65-c747-4a60-a612-26a7dba2becb None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:57.229415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb810c-c8ad-449f-aadb-6da8a45ddcfb None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:57.270549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b1e47-d0e1-4be2-a11e-99f1bedceba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:57.271243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b1e47-d0e1-4be2-a11e-99f1bedceba8 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.271576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b1e47-d0e1-4be2-a11e-99f1bedceba8 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.271842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b1e47-d0e1-4be2-a11e-99f1bedceba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.272716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b1e47-d0e1-4be2-a11e-99f1bedceba8 None admin] RBAC: Adding query filter params (role.id=492ce6c3fedb4109ac31de9467e8ef3b, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=abfea6b853134fed8ea5e32e5406bd5f) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:57.273024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b1e47-d0e1-4be2-a11e-99f1bedceba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:57.274675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b1e47-d0e1-4be2-a11e-99f1bedceba8 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:57.287847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d2c65-c747-4a60-a612-26a7dba2becb None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:57.289554 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 248/506] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:32:57 2022] GET /identity/v3/role_assignments?user.id=abfea6b853134fed8ea5e32e5406bd5f&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=492ce6c3fedb4109ac31de9467e8ef3b => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:57.290631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b1e47-d0e1-4be2-a11e-99f1bedceba8 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:57.317470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:57.327522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe71c6-bdc0-41a4-a07e-01dceea8d2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:57.328256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe71c6-bdc0-41a4-a07e-01dceea8d2a1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.328602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe71c6-bdc0-41a4-a07e-01dceea8d2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.328942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe71c6-bdc0-41a4-a07e-01dceea8d2a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.342574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe71c6-bdc0-41a4-a07e-01dceea8d2a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-45fe71c6-bdc0-41a4-a07e-01dceea8d2a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:32:57.348404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:57.350349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:57.352693 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 259/507] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:32:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:57.358563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe71c6-bdc0-41a4-a07e-01dceea8d2a1 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:57.442579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:57.478050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b1e47-d0e1-4be2-a11e-99f1bedceba8 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:57.516409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe71c6-bdc0-41a4-a07e-01dceea8d2a1 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:57.526650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567499a-e9f5-4ae3-847d-cd027085e0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:57.527273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567499a-e9f5-4ae3-847d-cd027085e0de None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.527545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567499a-e9f5-4ae3-847d-cd027085e0de None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.527810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567499a-e9f5-4ae3-847d-cd027085e0de None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.528459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567499a-e9f5-4ae3-847d-cd027085e0de None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:57.528754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567499a-e9f5-4ae3-847d-cd027085e0de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:57.530720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567499a-e9f5-4ae3-847d-cd027085e0de None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:57.549208 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 249/508] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:57.550735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246714a-aca3-4270-a0f6-2b52bd585105 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.551014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246714a-aca3-4270-a0f6-2b52bd585105 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.551280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246714a-aca3-4270-a0f6-2b52bd585105 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.552726 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 250/509] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:57.554976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246714a-aca3-4270-a0f6-2b52bd585105 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:57.564684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3431ae-a4b9-4299-b592-d0fc0fb6da06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:57.565466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3431ae-a4b9-4299-b592-d0fc0fb6da06 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.565748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3431ae-a4b9-4299-b592-d0fc0fb6da06 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.566013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3431ae-a4b9-4299-b592-d0fc0fb6da06 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.566676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3431ae-a4b9-4299-b592-d0fc0fb6da06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:57.566977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3431ae-a4b9-4299-b592-d0fc0fb6da06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:57.568326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3431ae-a4b9-4299-b592-d0fc0fb6da06 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:57.577711 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 260/510] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:32:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:32:57.579358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcd302-5fe8-450b-b6be-80435cc62b0d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.579634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcd302-5fe8-450b-b6be-80435cc62b0d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.579915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcd302-5fe8-450b-b6be-80435cc62b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.601016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-37dcd302-5fe8-450b-b6be-80435cc62b0d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:57.602243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-37dcd302-5fe8-450b-b6be-80435cc62b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:57.664582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:57.687020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246714a-aca3-4270-a0f6-2b52bd585105 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:57.694750 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 261/511] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:57.695602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcd302-5fe8-450b-b6be-80435cc62b0d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:57.726641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cf943-f082-423e-9095-48dd7ac58a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:57.727236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cf943-f082-423e-9095-48dd7ac58a39 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.727508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cf943-f082-423e-9095-48dd7ac58a39 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.727773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cf943-f082-423e-9095-48dd7ac58a39 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.728410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7cf943-f082-423e-9095-48dd7ac58a39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:57.729929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7cf943-f082-423e-9095-48dd7ac58a39 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:57.739395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:57.761130 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 251/512] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:32:57 2022] POST /identity/v3/users => generated 254 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:32:57.761130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81a50e-2c3b-4811-916d-c48d4e78a5c6 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.761130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81a50e-2c3b-4811-916d-c48d4e78a5c6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.761130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81a50e-2c3b-4811-916d-c48d4e78a5c6 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.761130 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 252/513] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:32:57.763751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce3034-36cf-4229-8411-54a64bb5c548 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.764155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce3034-36cf-4229-8411-54a64bb5c548 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.764518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce3034-36cf-4229-8411-54a64bb5c548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.784102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6fce3034-36cf-4229-8411-54a64bb5c548 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:32:57.785565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6fce3034-36cf-4229-8411-54a64bb5c548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:32:57.861925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcd302-5fe8-450b-b6be-80435cc62b0d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:57.926590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877947c-1634-4725-9e67-4345a665021a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:57.927235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877947c-1634-4725-9e67-4345a665021a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:57.927504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877947c-1634-4725-9e67-4345a665021a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:57.927765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877947c-1634-4725-9e67-4345a665021a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:57.929850 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 253/514] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:32:57.946026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877947c-1634-4725-9e67-4345a665021a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:57.947499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877947c-1634-4725-9e67-4345a665021a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-6877947c-1634-4725-9e67-4345a665021a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:57.969693 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:57.970681 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:57.971561 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:57.971561 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:57.971561 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:57.973230 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 262/515] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:57.978632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce3034-36cf-4229-8411-54a64bb5c548 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:58.086735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877947c-1634-4725-9e67-4345a665021a None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:58.116336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce3034-36cf-4229-8411-54a64bb5c548 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:58.145835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:58.151000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:58.169183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a765d5-1c5f-4310-b74b-9894962de711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:58.170126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a765d5-1c5f-4310-b74b-9894962de711 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:58.170466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a765d5-1c5f-4310-b74b-9894962de711 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:58.170809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a765d5-1c5f-4310-b74b-9894962de711 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:58.171994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a765d5-1c5f-4310-b74b-9894962de711 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:58.172298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a765d5-1c5f-4310-b74b-9894962de711 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:58.174060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a765d5-1c5f-4310-b74b-9894962de711 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:58.196220 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 254/516] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:58.202058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a765d5-1c5f-4310-b74b-9894962de711 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:58.252545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877947c-1634-4725-9e67-4345a665021a None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:58.290335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97cbc7-eaf6-4227-bdc3-ef4f4eb322b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:58.291131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97cbc7-eaf6-4227-bdc3-ef4f4eb322b6 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:58.291498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97cbc7-eaf6-4227-bdc3-ef4f4eb322b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:58.291857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97cbc7-eaf6-4227-bdc3-ef4f4eb322b6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:58.292522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97cbc7-eaf6-4227-bdc3-ef4f4eb322b6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:58.293998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97cbc7-eaf6-4227-bdc3-ef4f4eb322b6 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-1a97cbc7-eaf6-4227-bdc3-ef4f4eb322b6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:58.312329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:58.313526 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:58.314348 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 263/517] 10.222.0.16 () {56 vars in 1129 bytes} [Sat Oct 15 20:32:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:58.320906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97cbc7-eaf6-4227-bdc3-ef4f4eb322b6 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:58.330273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a765d5-1c5f-4310-b74b-9894962de711 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:58.366834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8cc89-845b-4430-b039-26d2b0c6674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:58.368002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8cc89-845b-4430-b039-26d2b0c6674a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:58.368465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8cc89-845b-4430-b039-26d2b0c6674a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:58.368901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8cc89-845b-4430-b039-26d2b0c6674a None admin] PATH_INFO: `/v3/users/nova` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:58.377400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8cc89-845b-4430-b039-26d2b0c6674a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:58.379141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8cc89-845b-4430-b039-26d2b0c6674a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-08a8cc89-845b-4430-b039-26d2b0c6674a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:32:58.391203 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:58.392242 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:58.393174 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:58.393174 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:58.393174 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:32:58.393174 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:32:58.393174 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:32:58.393174 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:32:58.393174 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:32:58.393174 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:32:58.393174 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:32:58.393174 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:58.395379 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 255/518] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Oct 15 20:32:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:32:58.396379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8cc89-845b-4430-b039-26d2b0c6674a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:58.422935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:58.432770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97cbc7-eaf6-4227-bdc3-ef4f4eb322b6 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:58.442899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:58.472718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd873bc-2c04-47c7-ba2f-3ce8b70baa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:58.473700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd873bc-2c04-47c7-ba2f-3ce8b70baa8c None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:58.474110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd873bc-2c04-47c7-ba2f-3ce8b70baa8c None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:58.474553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd873bc-2c04-47c7-ba2f-3ce8b70baa8c None admin] PATH_INFO: `/v3/services/network` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:58.475275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd873bc-2c04-47c7-ba2f-3ce8b70baa8c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:58.476935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd873bc-2c04-47c7-ba2f-3ce8b70baa8c None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-ebd873bc-2c04-47c7-ba2f-3ce8b70baa8c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:58.486905 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:58.487903 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 20:32:58.488702 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:58.496654 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 264/519] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:58 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:58.497922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd873bc-2c04-47c7-ba2f-3ce8b70baa8c None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:58.505212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8cc89-845b-4430-b039-26d2b0c6674a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:58.540961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37a15b-0b5e-4641-8b1c-5dcf9a56ca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:58.541719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37a15b-0b5e-4641-8b1c-5dcf9a56ca33 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:58.542062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37a15b-0b5e-4641-8b1c-5dcf9a56ca33 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:58.542397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37a15b-0b5e-4641-8b1c-5dcf9a56ca33 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:58.543211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37a15b-0b5e-4641-8b1c-5dcf9a56ca33 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:58.543579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37a15b-0b5e-4641-8b1c-5dcf9a56ca33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:58.545114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37a15b-0b5e-4641-8b1c-5dcf9a56ca33 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:58.550607 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 256/520] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Oct 15 20:32:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:58.556263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37a15b-0b5e-4641-8b1c-5dcf9a56ca33 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:58.615729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd873bc-2c04-47c7-ba2f-3ce8b70baa8c None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:58.653397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bef7f-1957-4d7d-93f6-1f892113bac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:58.653997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bef7f-1957-4d7d-93f6-1f892113bac4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:58.654268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bef7f-1957-4d7d-93f6-1f892113bac4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:58.654531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bef7f-1957-4d7d-93f6-1f892113bac4 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:58.655251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bef7f-1957-4d7d-93f6-1f892113bac4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:58.655650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bef7f-1957-4d7d-93f6-1f892113bac4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:58.657097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bef7f-1957-4d7d-93f6-1f892113bac4 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:58.681115 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 265/521] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Oct 15 20:32:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:58.681115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bef7f-1957-4d7d-93f6-1f892113bac4 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:58.689246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37a15b-0b5e-4641-8b1c-5dcf9a56ca33 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:58.698222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:58.701423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:58.743909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc0e2-b5c7-4333-b38d-bd1ea25064ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:58.744512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc0e2-b5c7-4333-b38d-bd1ea25064ee None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:58.744777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc0e2-b5c7-4333-b38d-bd1ea25064ee None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:58.745055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc0e2-b5c7-4333-b38d-bd1ea25064ee None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:58.746761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cc0e2-b5c7-4333-b38d-bd1ea25064ee None admin] RBAC: Adding query filter params (name=network) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:58.747056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cc0e2-b5c7-4333-b38d-bd1ea25064ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:58.748419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cc0e2-b5c7-4333-b38d-bd1ea25064ee None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:58.758660 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 257/522] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:32:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:58.759515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc0e2-b5c7-4333-b38d-bd1ea25064ee None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:58.806017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bef7f-1957-4d7d-93f6-1f892113bac4 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:58.851083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6d0c6-33d8-44b4-9381-c33e8938b84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:58.852022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6d0c6-33d8-44b4-9381-c33e8938b84f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:58.852432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6d0c6-33d8-44b4-9381-c33e8938b84f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:58.852865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6d0c6-33d8-44b4-9381-c33e8938b84f None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:58.853907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6d0c6-33d8-44b4-9381-c33e8938b84f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:58.854396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6d0c6-33d8-44b4-9381-c33e8938b84f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:58.856233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6d0c6-33d8-44b4-9381-c33e8938b84f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:58.863687 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 266/523] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Oct 15 20:32:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:58.865190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6d0c6-33d8-44b4-9381-c33e8938b84f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:58.896605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc0e2-b5c7-4333-b38d-bd1ea25064ee None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:58.959577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0a81c-beee-4413-9aa0-4fad11076e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:58.961508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0a81c-beee-4413-9aa0-4fad11076e04 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:58.962042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0a81c-beee-4413-9aa0-4fad11076e04 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:58.962540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0a81c-beee-4413-9aa0-4fad11076e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:58.974674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0a81c-beee-4413-9aa0-4fad11076e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:58.976634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0a81c-beee-4413-9aa0-4fad11076e04 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-56b0a81c-beee-4413-9aa0-4fad11076e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:32:58.993444 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:58.994432 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:32:58.995286 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:32:58.995286 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:32:58.995286 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:32:58.997451 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 258/524] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:59.003248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0a81c-beee-4413-9aa0-4fad11076e04 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:59.033060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:59.072041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:59.119275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6d0c6-33d8-44b4-9381-c33e8938b84f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:59.155239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0a81c-beee-4413-9aa0-4fad11076e04 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:59.202479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b31b5-eeda-4e56-8779-a38b76fec183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:59.204523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b31b5-eeda-4e56-8779-a38b76fec183 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:59.205836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b31b5-eeda-4e56-8779-a38b76fec183 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:59.207030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b31b5-eeda-4e56-8779-a38b76fec183 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:59.208848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b31b5-eeda-4e56-8779-a38b76fec183 None admin] RBAC: Adding query filter params (type=network) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:59.210692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b31b5-eeda-4e56-8779-a38b76fec183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:59.215017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b31b5-eeda-4e56-8779-a38b76fec183 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:59.221278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc8ff1-bcf8-40a0-b1c4-305084e6815a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:59.222344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc8ff1-bcf8-40a0-b1c4-305084e6815a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:59.222820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc8ff1-bcf8-40a0-b1c4-305084e6815a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:59.223283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc8ff1-bcf8-40a0-b1c4-305084e6815a None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:59.224114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc8ff1-bcf8-40a0-b1c4-305084e6815a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:59.224603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc8ff1-bcf8-40a0-b1c4-305084e6815a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:59.228177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc8ff1-bcf8-40a0-b1c4-305084e6815a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:59.230711 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 267/525] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:32:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:59.259604 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 259/526] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:59.265828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b31b5-eeda-4e56-8779-a38b76fec183 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:59.437027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b31b5-eeda-4e56-8779-a38b76fec183 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:59.491708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436b49-c606-41a8-87fc-ee1e01330260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:59.492912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436b49-c606-41a8-87fc-ee1e01330260 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:59.493408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436b49-c606-41a8-87fc-ee1e01330260 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:59.493865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436b49-c606-41a8-87fc-ee1e01330260 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:59.502303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9436b49-c606-41a8-87fc-ee1e01330260 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:59.504099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9436b49-c606-41a8-87fc-ee1e01330260 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-a9436b49-c606-41a8-87fc-ee1e01330260 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:32:59.517675 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:32:59.518623 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:32:59.519566 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:32:59.519566 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:32:59.519566 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:32:59.519566 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:32:59.519566 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:32:59.519566 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:32:59.519566 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:32:59.519566 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:32:59.522306 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 268/527] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:32:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:32:59.528041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc8ff1-bcf8-40a0-b1c4-305084e6815a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:59.619490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:59.647536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc8ff1-bcf8-40a0-b1c4-305084e6815a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:59.698919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e639ecc-2739-4001-84f9-9e00d92b4b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:59.699522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e639ecc-2739-4001-84f9-9e00d92b4b1e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:59.699793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e639ecc-2739-4001-84f9-9e00d92b4b1e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:59.700057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e639ecc-2739-4001-84f9-9e00d92b4b1e None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:59.700745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e639ecc-2739-4001-84f9-9e00d92b4b1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:32:59.701057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e639ecc-2739-4001-84f9-9e00d92b4b1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:59.702487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e639ecc-2739-4001-84f9-9e00d92b4b1e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:59.720601 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 260/528] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:32:59.726565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436b49-c606-41a8-87fc-ee1e01330260 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:32:59.735433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:32:59.876849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436b49-c606-41a8-87fc-ee1e01330260 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:32:59.923128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5caacf-39c3-497f-b84d-2971c7f020fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:32:59.923837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5caacf-39c3-497f-b84d-2971c7f020fd None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:32:59.924111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5caacf-39c3-497f-b84d-2971c7f020fd None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:32:59.924410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5caacf-39c3-497f-b84d-2971c7f020fd None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/6225c81d2620463886d8a936b59784df/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:32:59.926357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5caacf-39c3-497f-b84d-2971c7f020fd None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=6225c81d2620463886d8a936b59784df, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:32:59.927883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5caacf-39c3-497f-b84d-2971c7f020fd None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:32:59.947563 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 269/529] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:32:59 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/6225c81d2620463886d8a936b59784df/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:33:00.037298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60ecc6-8ced-4f6e-94e2-dd53ebb978e3 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:00.037625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60ecc6-8ced-4f6e-94e2-dd53ebb978e3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:00.038377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60ecc6-8ced-4f6e-94e2-dd53ebb978e3 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:00.040022 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 261/530] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:00.063099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8ed24-4e8d-4860-bc52-60317c54bae0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:00.063428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8ed24-4e8d-4860-bc52-60317c54bae0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:00.063633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8ed24-4e8d-4860-bc52-60317c54bae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:00.094367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bab8ed24-4e8d-4860-bc52-60317c54bae0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:00.095605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bab8ed24-4e8d-4860-bc52-60317c54bae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:00.191646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:00.211531 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 270/531] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:33:00.570603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60ecc6-8ced-4f6e-94e2-dd53ebb978e3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:00.701696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60ecc6-8ced-4f6e-94e2-dd53ebb978e3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:00.751774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8edef0-ab2d-4f50-96e9-4ef74d531eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:00.752806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8edef0-ab2d-4f50-96e9-4ef74d531eff None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:00.753314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8edef0-ab2d-4f50-96e9-4ef74d531eff None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:00.753799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8edef0-ab2d-4f50-96e9-4ef74d531eff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:00.769255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8edef0-ab2d-4f50-96e9-4ef74d531eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-bd8edef0-ab2d-4f50-96e9-4ef74d531eff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:00.775247 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:00.776252 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:33:00.777139 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:00.780223 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 262/532] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:33:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:00.785875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8ed24-4e8d-4860-bc52-60317c54bae0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:00.872902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c362f-eaee-4436-8c53-3d83c54bf500 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:00.873171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c362f-eaee-4436-8c53-3d83c54bf500 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:00.873400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c362f-eaee-4436-8c53-3d83c54bf500 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:00.874999 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 263/533] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:00.880146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53a4f7-eab8-404f-be70-d44138e42504 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:00.880426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53a4f7-eab8-404f-be70-d44138e42504 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:00.880691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53a4f7-eab8-404f-be70-d44138e42504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:00.899195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:00.921124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8ed24-4e8d-4860-bc52-60317c54bae0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:00.922056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8c53a4f7-eab8-404f-be70-d44138e42504 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:00.923255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8c53a4f7-eab8-404f-be70-d44138e42504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:00.966446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a61c3-ed7d-4ee4-9939-7c905ca22448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:00.968398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a61c3-ed7d-4ee4-9939-7c905ca22448 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:00.971531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a61c3-ed7d-4ee4-9939-7c905ca22448 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:00.972029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a61c3-ed7d-4ee4-9939-7c905ca22448 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:00.972893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a61c3-ed7d-4ee4-9939-7c905ca22448 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:00.974065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a61c3-ed7d-4ee4-9939-7c905ca22448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:00.976415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a61c3-ed7d-4ee4-9939-7c905ca22448 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:00.987170 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 271/534] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:33:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:00.992811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a61c3-ed7d-4ee4-9939-7c905ca22448 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:01.073574 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 264/535] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 20:33:01.093782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:01.140898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a61c3-ed7d-4ee4-9939-7c905ca22448 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:01.175629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7d5ec-cdaa-4845-a863-57cbc353f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:01.176578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7d5ec-cdaa-4845-a863-57cbc353f76e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:01.177180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7d5ec-cdaa-4845-a863-57cbc353f76e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:01.177666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7d5ec-cdaa-4845-a863-57cbc353f76e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:01.189540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7d5ec-cdaa-4845-a863-57cbc353f76e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:01.191318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7d5ec-cdaa-4845-a863-57cbc353f76e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-78b7d5ec-cdaa-4845-a863-57cbc353f76e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:01.206706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:01.207613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:01.208518 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:01.208518 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:01.208518 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:01.211220 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 272/536] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:01.218122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53a4f7-eab8-404f-be70-d44138e42504 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:01.340813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53a4f7-eab8-404f-be70-d44138e42504 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:01.372717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:01.378130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e2ab1-0b67-41d5-b03c-3b558a51ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:01.379121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e2ab1-0b67-41d5-b03c-3b558a51ff9a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:01.379589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e2ab1-0b67-41d5-b03c-3b558a51ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:01.380045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e2ab1-0b67-41d5-b03c-3b558a51ff9a None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:01.380867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e2ab1-0b67-41d5-b03c-3b558a51ff9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:01.381554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e2ab1-0b67-41d5-b03c-3b558a51ff9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:01.383124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e2ab1-0b67-41d5-b03c-3b558a51ff9a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:01.395635 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 265/537] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:01.402402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7d5ec-cdaa-4845-a863-57cbc353f76e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:01.452819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e2ab1-0b67-41d5-b03c-3b558a51ff9a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:01.565308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e2ab1-0b67-41d5-b03c-3b558a51ff9a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:01.602448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25dcb0-9b04-47ed-8ec6-13b45171b432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:01.603215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25dcb0-9b04-47ed-8ec6-13b45171b432 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:01.603451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25dcb0-9b04-47ed-8ec6-13b45171b432 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:01.603717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25dcb0-9b04-47ed-8ec6-13b45171b432 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:01.614990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25dcb0-9b04-47ed-8ec6-13b45171b432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-8b25dcb0-9b04-47ed-8ec6-13b45171b432 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:01.621575 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:01.622503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:01.623574 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 266/538] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:33:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:01.626898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7d5ec-cdaa-4845-a863-57cbc353f76e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:01.635725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25dcb0-9b04-47ed-8ec6-13b45171b432 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:01.648201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:01.702323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:01.751494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25dcb0-9b04-47ed-8ec6-13b45171b432 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:01.765771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f84eb-116a-4a62-a186-cc764eb60f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:01.766556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f84eb-116a-4a62-a186-cc764eb60f96 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:01.766947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f84eb-116a-4a62-a186-cc764eb60f96 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:01.767321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f84eb-116a-4a62-a186-cc764eb60f96 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:01.788651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf461bd-2291-41ea-85e4-963b753ca4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:01.789332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf461bd-2291-41ea-85e4-963b753ca4ad None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:01.789607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf461bd-2291-41ea-85e4-963b753ca4ad None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:01.789921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf461bd-2291-41ea-85e4-963b753ca4ad None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:01.790754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf461bd-2291-41ea-85e4-963b753ca4ad None admin] RBAC: Adding query filter params (name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:01.791057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf461bd-2291-41ea-85e4-963b753ca4ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:01.791929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f84eb-116a-4a62-a186-cc764eb60f96 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:01.792569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf461bd-2291-41ea-85e4-963b753ca4ad None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:01.793580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f84eb-116a-4a62-a186-cc764eb60f96 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:01.801641 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 267/539] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:33:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:01.807637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf461bd-2291-41ea-85e4-963b753ca4ad None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-f88f84eb-116a-4a62-a186-cc764eb60f96 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:33:01.826315 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:01.827290 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:01.828183 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:01.828183 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:01.828183 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:33:01.828183 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:33:01.828183 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:33:01.828183 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:33:01.828183 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:33:01.828183 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:33:01.828183 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 20:33:01.828183 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:01.833429 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 273/540] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:33:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:01.840000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f84eb-116a-4a62-a186-cc764eb60f96 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:01.981166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf461bd-2291-41ea-85e4-963b753ca4ad None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:01.996128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:02.017654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1220413-7179-4d11-82ad-ac0e41970e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:02.018185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1220413-7179-4d11-82ad-ac0e41970e89 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.018699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1220413-7179-4d11-82ad-ac0e41970e89 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.019034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1220413-7179-4d11-82ad-ac0e41970e89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.031018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1220413-7179-4d11-82ad-ac0e41970e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:02.031499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f84eb-116a-4a62-a186-cc764eb60f96 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:02.032703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1220413-7179-4d11-82ad-ac0e41970e89 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-c1220413-7179-4d11-82ad-ac0e41970e89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:02.049244 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:02.050365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:02.051239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:02.051239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:02.051239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:02.051696 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 268/541] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:02.057735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1220413-7179-4d11-82ad-ac0e41970e89 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:02.073463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d7d5c-5eb1-4655-8e05-b0f78672ed53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:02.074345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d7d5c-5eb1-4655-8e05-b0f78672ed53 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.074883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d7d5c-5eb1-4655-8e05-b0f78672ed53 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.076907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d7d5c-5eb1-4655-8e05-b0f78672ed53 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.077856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d7d5c-5eb1-4655-8e05-b0f78672ed53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:02.078467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d7d5c-5eb1-4655-8e05-b0f78672ed53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:02.080240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d7d5c-5eb1-4655-8e05-b0f78672ed53 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:02.115658 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 274/542] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Oct 15 20:33:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:02.123162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d7d5c-5eb1-4655-8e05-b0f78672ed53 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:02.180300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:02.236714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1220413-7179-4d11-82ad-ac0e41970e89 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:02.275011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d7d5c-5eb1-4655-8e05-b0f78672ed53 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:02.284130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ada0c-ed93-4b39-8ecc-72530d08f51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:02.285021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ada0c-ed93-4b39-8ecc-72530d08f51e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.285983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ada0c-ed93-4b39-8ecc-72530d08f51e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.286457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ada0c-ed93-4b39-8ecc-72530d08f51e None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.288172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ada0c-ed93-4b39-8ecc-72530d08f51e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:02.289047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ada0c-ed93-4b39-8ecc-72530d08f51e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:02.290849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ada0c-ed93-4b39-8ecc-72530d08f51e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:02.308395 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 269/543] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:02.310698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4086a05-dbc8-476d-b1d6-2bc19da36826 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.311236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4086a05-dbc8-476d-b1d6-2bc19da36826 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.311769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4086a05-dbc8-476d-b1d6-2bc19da36826 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.317366 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 270/544] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:02.318241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4086a05-dbc8-476d-b1d6-2bc19da36826 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:02.320083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc0fe2-7e88-47f1-9ebd-8966ed5789c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:02.320706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc0fe2-7e88-47f1-9ebd-8966ed5789c7 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.321026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc0fe2-7e88-47f1-9ebd-8966ed5789c7 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.321326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc0fe2-7e88-47f1-9ebd-8966ed5789c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.332541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc0fe2-7e88-47f1-9ebd-8966ed5789c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:02.334146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc0fe2-7e88-47f1-9ebd-8966ed5789c7 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-6edc0fe2-7e88-47f1-9ebd-8966ed5789c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:02.351986 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:02.352952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:02.354515 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:02.354515 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:02.354515 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:02.357540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:02.357659 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 275/545] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:02.362585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737efcb-b61f-45e1-bed4-78bfb71762fe None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.362860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737efcb-b61f-45e1-bed4-78bfb71762fe None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.363134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737efcb-b61f-45e1-bed4-78bfb71762fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.384471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5737efcb-b61f-45e1-bed4-78bfb71762fe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:02.385797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5737efcb-b61f-45e1-bed4-78bfb71762fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:02.447960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4086a05-dbc8-476d-b1d6-2bc19da36826 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:02.465177 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 276/546] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:33:02.466236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737efcb-b61f-45e1-bed4-78bfb71762fe None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:02.489167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:02.498376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90441d97-f9ca-4d9f-af89-129d9734e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:02.499571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90441d97-f9ca-4d9f-af89-129d9734e76a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.499990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90441d97-f9ca-4d9f-af89-129d9734e76a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.500894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90441d97-f9ca-4d9f-af89-129d9734e76a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.511295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90441d97-f9ca-4d9f-af89-129d9734e76a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:02.513212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90441d97-f9ca-4d9f-af89-129d9734e76a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-90441d97-f9ca-4d9f-af89-129d9734e76a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:33:02.529766 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:02.530947 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:02.532036 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:02.532036 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:02.532036 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:33:02.532036 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:33:02.532036 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:33:02.532036 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:33:02.532036 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:33:02.532036 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:33:02.532036 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 20:33:02.532036 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:02.536215 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 271/547] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Oct 15 20:33:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:02.537501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c3c6c-b61f-479d-b8d3-0fc00daf858c None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.538328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c3c6c-b61f-479d-b8d3-0fc00daf858c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.539088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c3c6c-b61f-479d-b8d3-0fc00daf858c None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.541067 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 272/548] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:02.542070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c3c6c-b61f-479d-b8d3-0fc00daf858c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:02.621202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737efcb-b61f-45e1-bed4-78bfb71762fe None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:02.642579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:02.661477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7742d3-2181-4e60-a6b7-6e63021f67ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:02.662403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7742d3-2181-4e60-a6b7-6e63021f67ea None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.663461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7742d3-2181-4e60-a6b7-6e63021f67ea None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.663783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7742d3-2181-4e60-a6b7-6e63021f67ea None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.664601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7742d3-2181-4e60-a6b7-6e63021f67ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:02.664964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7742d3-2181-4e60-a6b7-6e63021f67ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:02.665491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c3c6c-b61f-479d-b8d3-0fc00daf858c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:02.666594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7742d3-2181-4e60-a6b7-6e63021f67ea None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:02.686256 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 277/549] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:02.687787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74639a-a4dc-4334-a6de-5c0aeafaaaef None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.688254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74639a-a4dc-4334-a6de-5c0aeafaaaef None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.688674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74639a-a4dc-4334-a6de-5c0aeafaaaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.706241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250c0e9-2142-4316-861d-a39d873f6880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:02.706980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250c0e9-2142-4316-861d-a39d873f6880 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.707316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250c0e9-2142-4316-861d-a39d873f6880 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.707670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250c0e9-2142-4316-861d-a39d873f6880 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.708469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250c0e9-2142-4316-861d-a39d873f6880 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:02.708825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250c0e9-2142-4316-861d-a39d873f6880 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:02.711024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250c0e9-2142-4316-861d-a39d873f6880 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:02.712918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fc74639a-a4dc-4334-a6de-5c0aeafaaaef None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:02.715348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fc74639a-a4dc-4334-a6de-5c0aeafaaaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:02.734391 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 273/550] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Oct 15 20:33:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:02.734391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250c0e9-2142-4316-861d-a39d873f6880 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:02.807896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:02.815097 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 278/551] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:33:02.815097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74639a-a4dc-4334-a6de-5c0aeafaaaef None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:02.842939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250c0e9-2142-4316-861d-a39d873f6880 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:02.884414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128978c1-e8ef-44ce-a110-76b61ea4acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:02.885501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128978c1-e8ef-44ce-a110-76b61ea4acb5 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.885847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128978c1-e8ef-44ce-a110-76b61ea4acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.886575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128978c1-e8ef-44ce-a110-76b61ea4acb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.895604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128978c1-e8ef-44ce-a110-76b61ea4acb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:02.897966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128978c1-e8ef-44ce-a110-76b61ea4acb5 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-128978c1-e8ef-44ce-a110-76b61ea4acb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:33:02.914026 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:02.914945 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:02.915816 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:33:02.915816 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:33:02.915816 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:33:02.915816 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:33:02.915816 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:33:02.915816 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:33:02.915816 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:02.915816 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:02.916750 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 274/552] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:02.916750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128978c1-e8ef-44ce-a110-76b61ea4acb5 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:02.924363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:02.937584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74639a-a4dc-4334-a6de-5c0aeafaaaef None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:02.971301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9654066-c781-4c7d-b58e-abbfd226c344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:02.971912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9654066-c781-4c7d-b58e-abbfd226c344 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:02.972186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9654066-c781-4c7d-b58e-abbfd226c344 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:02.972448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9654066-c781-4c7d-b58e-abbfd226c344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:02.982801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9654066-c781-4c7d-b58e-abbfd226c344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:02.984200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9654066-c781-4c7d-b58e-abbfd226c344 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-f9654066-c781-4c7d-b58e-abbfd226c344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:03.002937 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:03.003910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:03.004798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:03.004798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:03.004798 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:03.004798 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 279/553] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:03.004798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9654066-c781-4c7d-b58e-abbfd226c344 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:03.041305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128978c1-e8ef-44ce-a110-76b61ea4acb5 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:03.063267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:03.076683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87c1dc-cc0e-43c0-9f56-78d0ea066c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:03.077308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87c1dc-cc0e-43c0-9f56-78d0ea066c4a None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:03.077628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87c1dc-cc0e-43c0-9f56-78d0ea066c4a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:03.078106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87c1dc-cc0e-43c0-9f56-78d0ea066c4a None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:03.078654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87c1dc-cc0e-43c0-9f56-78d0ea066c4a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:03.080027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87c1dc-cc0e-43c0-9f56-78d0ea066c4a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:03.095508 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 275/554] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:33:02 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:03.096875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b063f-c6fc-454f-ab42-b4356af36815 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:03.097188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b063f-c6fc-454f-ab42-b4356af36815 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:03.097453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b063f-c6fc-454f-ab42-b4356af36815 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:03.098659 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 276/555] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:03.101283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b063f-c6fc-454f-ab42-b4356af36815 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:03.132080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9654066-c781-4c7d-b58e-abbfd226c344 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:03.195203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:03.197445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12635e-3d14-453d-83c7-40d4d63b509a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:03.198215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12635e-3d14-453d-83c7-40d4d63b509a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:03.198708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12635e-3d14-453d-83c7-40d4d63b509a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:03.198979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12635e-3d14-453d-83c7-40d4d63b509a None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:03.199817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12635e-3d14-453d-83c7-40d4d63b509a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:03.200151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12635e-3d14-453d-83c7-40d4d63b509a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:03.202204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12635e-3d14-453d-83c7-40d4d63b509a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:03.216982 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 280/556] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:33:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:03.223171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001f2d9-f95b-44c7-a4f9-0c3c197539b5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:03.223861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001f2d9-f95b-44c7-a4f9-0c3c197539b5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:03.224109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001f2d9-f95b-44c7-a4f9-0c3c197539b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:03.248928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b063f-c6fc-454f-ab42-b4356af36815 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:03.249877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9001f2d9-f95b-44c7-a4f9-0c3c197539b5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:03.251107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9001f2d9-f95b-44c7-a4f9-0c3c197539b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:03.291175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007f69b-1140-4d32-a64f-97bd8f77087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:03.292145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007f69b-1140-4d32-a64f-97bd8f77087a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:03.292411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007f69b-1140-4d32-a64f-97bd8f77087a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:03.292740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007f69b-1140-4d32-a64f-97bd8f77087a None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:03.293435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007f69b-1140-4d32-a64f-97bd8f77087a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:03.293732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007f69b-1140-4d32-a64f-97bd8f77087a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:03.295120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007f69b-1140-4d32-a64f-97bd8f77087a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:03.313544 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 277/557] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:03.313544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007f69b-1140-4d32-a64f-97bd8f77087a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:03.336366 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 281/558] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 20:33:03.337472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001f2d9-f95b-44c7-a4f9-0c3c197539b5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:03.380100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:03.434827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007f69b-1140-4d32-a64f-97bd8f77087a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:03.463147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001f2d9-f95b-44c7-a4f9-0c3c197539b5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:03.467427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:03.481556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9756638-a582-4014-8ce1-37b4984be1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:03.482572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9756638-a582-4014-8ce1-37b4984be1fc None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:03.482951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9756638-a582-4014-8ce1-37b4984be1fc None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:03.483320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9756638-a582-4014-8ce1-37b4984be1fc None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:03.484222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9756638-a582-4014-8ce1-37b4984be1fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:03.486032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9756638-a582-4014-8ce1-37b4984be1fc None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:03.504979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981272d-9813-4fcf-91bf-b9e8421bd7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:03.505869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981272d-9813-4fcf-91bf-b9e8421bd7a1 None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:03.506402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981272d-9813-4fcf-91bf-b9e8421bd7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:03.506914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981272d-9813-4fcf-91bf-b9e8421bd7a1 None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/abfea6b853134fed8ea5e32e5406bd5f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:03.509019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981272d-9813-4fcf-91bf-b9e8421bd7a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=abfea6b853134fed8ea5e32e5406bd5f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:03.510745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981272d-9813-4fcf-91bf-b9e8421bd7a1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:03.513656 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 278/559] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:33:03 2022] POST /identity/v3/services => generated 240 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:03.515358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9756638-a582-4014-8ce1-37b4984be1fc None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:03.531800 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 282/560] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:33:03 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/abfea6b853134fed8ea5e32e5406bd5f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:33:03.660473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9756638-a582-4014-8ce1-37b4984be1fc None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:03.671248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981272d-9813-4fcf-91bf-b9e8421bd7a1 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:03.707130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22675e4d-b11a-418e-93a4-b85f51ff504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:03.707782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22675e4d-b11a-418e-93a4-b85f51ff504c None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:03.708055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22675e4d-b11a-418e-93a4-b85f51ff504c None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:03.708321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22675e4d-b11a-418e-93a4-b85f51ff504c None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:03.717669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22675e4d-b11a-418e-93a4-b85f51ff504c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:03.719257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22675e4d-b11a-418e-93a4-b85f51ff504c None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-22675e4d-b11a-418e-93a4-b85f51ff504c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:33:03.732823 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:03.733809 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:03.734662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:33:03.734662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:33:03.734662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:33:03.734662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:33:03.734662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:33:03.734662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:33:03.734662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:03.734662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:03.734662 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 279/561] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:03.742760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22675e4d-b11a-418e-93a4-b85f51ff504c None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:03.794962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:03.806014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:03.813876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981272d-9813-4fcf-91bf-b9e8421bd7a1 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:03.854683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45b694-6057-446e-8ccf-2d20adc6953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:03.855374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45b694-6057-446e-8ccf-2d20adc6953f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:03.855797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45b694-6057-446e-8ccf-2d20adc6953f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:03.856072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45b694-6057-446e-8ccf-2d20adc6953f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:03.873092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45b694-6057-446e-8ccf-2d20adc6953f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-6e45b694-6057-446e-8ccf-2d20adc6953f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:03.874890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:03.876010 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:33:03.876945 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:03.877614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22675e4d-b11a-418e-93a4-b85f51ff504c None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:03.877702 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 283/562] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:33:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:03.882941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45b694-6057-446e-8ccf-2d20adc6953f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:03.920396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e42bdd-6e9c-4f82-ad69-44aca6aac4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:03.921626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e42bdd-6e9c-4f82-ad69-44aca6aac4b6 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:03.923001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e42bdd-6e9c-4f82-ad69-44aca6aac4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:03.923475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e42bdd-6e9c-4f82-ad69-44aca6aac4b6 None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:03.924359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e42bdd-6e9c-4f82-ad69-44aca6aac4b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:03.924857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e42bdd-6e9c-4f82-ad69-44aca6aac4b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:03.926526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e42bdd-6e9c-4f82-ad69-44aca6aac4b6 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:03.946342 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 280/563] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:33:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:03.957069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e42bdd-6e9c-4f82-ad69-44aca6aac4b6 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:04.005365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45b694-6057-446e-8ccf-2d20adc6953f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:04.046004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef345370-f949-4856-8e32-b0b6b4136c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:04.046599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef345370-f949-4856-8e32-b0b6b4136c09 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:04.046872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef345370-f949-4856-8e32-b0b6b4136c09 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:04.047139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef345370-f949-4856-8e32-b0b6b4136c09 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:04.047789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef345370-f949-4856-8e32-b0b6b4136c09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:04.048084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef345370-f949-4856-8e32-b0b6b4136c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:04.049502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef345370-f949-4856-8e32-b0b6b4136c09 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:04.058467 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 284/564] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:33:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:04.064033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef345370-f949-4856-8e32-b0b6b4136c09 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:04.085516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:04.088682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:04.094459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e42bdd-6e9c-4f82-ad69-44aca6aac4b6 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:04.142930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f2a34-46e9-463a-8ff8-1901cb38a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:04.144033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f2a34-46e9-463a-8ff8-1901cb38a9d4 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:04.144512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f2a34-46e9-463a-8ff8-1901cb38a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:04.145105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f2a34-46e9-463a-8ff8-1901cb38a9d4 None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/7a9c40e53f72418381298190ca330e14/roles/492ce6c3fedb4109ac31de9467e8ef3b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:04.165527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f2a34-46e9-463a-8ff8-1901cb38a9d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=7a9c40e53f72418381298190ca330e14, role_id=492ce6c3fedb4109ac31de9467e8ef3b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:04.168194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f2a34-46e9-463a-8ff8-1901cb38a9d4 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:04.182930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef345370-f949-4856-8e32-b0b6b4136c09 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:04.186667 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 281/565] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:33:03 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/7a9c40e53f72418381298190ca330e14/roles/492ce6c3fedb4109ac31de9467e8ef3b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:33:04.251643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc387c-4781-4906-84e4-3185ee9adda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:04.252294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc387c-4781-4906-84e4-3185ee9adda4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:04.252568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc387c-4781-4906-84e4-3185ee9adda4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:04.252836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc387c-4781-4906-84e4-3185ee9adda4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:04.271642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc387c-4781-4906-84e4-3185ee9adda4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:04.273854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc387c-4781-4906-84e4-3185ee9adda4 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-4afc387c-4781-4906-84e4-3185ee9adda4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:04.298012 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:04.299002 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:04.299865 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:04.299865 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:04.299865 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:04.299865 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 285/566] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:04.306664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f2a34-46e9-463a-8ff8-1901cb38a9d4 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:04.454970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f2a34-46e9-463a-8ff8-1901cb38a9d4 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:04.505534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c9724-0fb7-47d2-b5dd-01cf771c6f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:04.506141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c9724-0fb7-47d2-b5dd-01cf771c6f60 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:04.506981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c9724-0fb7-47d2-b5dd-01cf771c6f60 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:04.507355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c9724-0fb7-47d2-b5dd-01cf771c6f60 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:04.507997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c9724-0fb7-47d2-b5dd-01cf771c6f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:04.508295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c9724-0fb7-47d2-b5dd-01cf771c6f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:04.509773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c9724-0fb7-47d2-b5dd-01cf771c6f60 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:04.522718 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 282/567] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:04.527896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc387c-4781-4906-84e4-3185ee9adda4 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:04.633000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:04.680512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc387c-4781-4906-84e4-3185ee9adda4 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:04.726642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b548922-4ae0-455b-80a3-66ccac372b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:04.727188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b548922-4ae0-455b-80a3-66ccac372b65 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:04.727460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b548922-4ae0-455b-80a3-66ccac372b65 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:04.727723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b548922-4ae0-455b-80a3-66ccac372b65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:04.741690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b548922-4ae0-455b-80a3-66ccac372b65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:04.743395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b548922-4ae0-455b-80a3-66ccac372b65 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-8b548922-4ae0-455b-80a3-66ccac372b65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:33:04.760535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:04.761517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:04.762441 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 286/568] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Oct 15 20:33:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:04.767622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c9724-0fb7-47d2-b5dd-01cf771c6f60 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:04.783354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:04.902845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c9724-0fb7-47d2-b5dd-01cf771c6f60 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:04.941796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009e119-f242-4bb9-b217-dfa398650f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:04.942396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009e119-f242-4bb9-b217-dfa398650f6f None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:04.942671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009e119-f242-4bb9-b217-dfa398650f6f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:04.942938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009e119-f242-4bb9-b217-dfa398650f6f None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:04.944474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009e119-f242-4bb9-b217-dfa398650f6f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:04.944782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009e119-f242-4bb9-b217-dfa398650f6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:04.946297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009e119-f242-4bb9-b217-dfa398650f6f None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:04.970421 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 283/569] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Oct 15 20:33:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:04.976837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b548922-4ae0-455b-80a3-66ccac372b65 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:05.104235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b548922-4ae0-455b-80a3-66ccac372b65 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:05.140686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:05.147853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e34fa-8323-4e62-a08b-cbf4e0c104e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:05.148857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e34fa-8323-4e62-a08b-cbf4e0c104e4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:05.149454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e34fa-8323-4e62-a08b-cbf4e0c104e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:05.149925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e34fa-8323-4e62-a08b-cbf4e0c104e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:05.235198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e34fa-8323-4e62-a08b-cbf4e0c104e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:05.236644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e34fa-8323-4e62-a08b-cbf4e0c104e4 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:05.361762 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 287/570] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:33:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 20:33:05.367807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009e119-f242-4bb9-b217-dfa398650f6f None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:05.440275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:05.477000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009e119-f242-4bb9-b217-dfa398650f6f None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:05.513101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db3d0-4936-4cb1-8678-48442920648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:05.513646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db3d0-4936-4cb1-8678-48442920648c None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:05.513918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db3d0-4936-4cb1-8678-48442920648c None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:05.514181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db3d0-4936-4cb1-8678-48442920648c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:05.525245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4db3d0-4936-4cb1-8678-48442920648c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:05.526719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4db3d0-4936-4cb1-8678-48442920648c None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-1a4db3d0-4936-4cb1-8678-48442920648c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:05.541974 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:05.543041 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:05.543912 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:05.543912 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:05.543912 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:05.543912 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 284/571] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:05.548950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e34fa-8323-4e62-a08b-cbf4e0c104e4 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:05.598407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:05.710032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e34fa-8323-4e62-a08b-cbf4e0c104e4 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:05.758419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418465cf-c5e2-4c76-9574-d0570cf165c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:05.759024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418465cf-c5e2-4c76-9574-d0570cf165c1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:05.759300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418465cf-c5e2-4c76-9574-d0570cf165c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:05.759572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418465cf-c5e2-4c76-9574-d0570cf165c1 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:05.760210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418465cf-c5e2-4c76-9574-d0570cf165c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:05.760505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418465cf-c5e2-4c76-9574-d0570cf165c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:05.761899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418465cf-c5e2-4c76-9574-d0570cf165c1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:05.782423 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 288/572] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:05.788122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db3d0-4936-4cb1-8678-48442920648c None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:05.949486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db3d0-4936-4cb1-8678-48442920648c None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:05.983961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:06.003653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65531e51-35e6-4dfb-838e-3ba1885c6f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:06.004276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65531e51-35e6-4dfb-838e-3ba1885c6f79 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:06.004549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65531e51-35e6-4dfb-838e-3ba1885c6f79 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:06.004815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65531e51-35e6-4dfb-838e-3ba1885c6f79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:06.017634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65531e51-35e6-4dfb-838e-3ba1885c6f79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:06.019266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65531e51-35e6-4dfb-838e-3ba1885c6f79 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-65531e51-35e6-4dfb-838e-3ba1885c6f79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:33:06.031183 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:06.032090 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:06.032993 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:33:06.032993 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:33:06.032993 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:33:06.032993 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:33:06.032993 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:33:06.032993 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:33:06.032993 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:06.032993 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:06.032993 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 285/573] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:06.042135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418465cf-c5e2-4c76-9574-d0570cf165c1 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:06.187662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:06.213272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418465cf-c5e2-4c76-9574-d0570cf165c1 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:06.261656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b1aa9-a6b1-4307-8870-ab864ef97bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:06.262693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b1aa9-a6b1-4307-8870-ab864ef97bb0 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:06.263186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b1aa9-a6b1-4307-8870-ab864ef97bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:06.263659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b1aa9-a6b1-4307-8870-ab864ef97bb0 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:06.264599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b1aa9-a6b1-4307-8870-ab864ef97bb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:06.269246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b1aa9-a6b1-4307-8870-ab864ef97bb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:06.271226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b1aa9-a6b1-4307-8870-ab864ef97bb0 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:06.288975 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 289/574] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:33:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:06.294893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65531e51-35e6-4dfb-838e-3ba1885c6f79 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:06.321124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b47251-83e5-4277-abf9-4c390ee44433 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:06.321725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b47251-83e5-4277-abf9-4c390ee44433 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:06.322170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b47251-83e5-4277-abf9-4c390ee44433 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:06.324019 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 290/575] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:06.329840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc768a6-5990-40b2-997f-6549995994cc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:06.330365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc768a6-5990-40b2-997f-6549995994cc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:06.330802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc768a6-5990-40b2-997f-6549995994cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:06.366127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5fc768a6-5990-40b2-997f-6549995994cc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:06.368117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5fc768a6-5990-40b2-997f-6549995994cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:06.435456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65531e51-35e6-4dfb-838e-3ba1885c6f79 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:06.478623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d75ee2-623d-4242-949f-ac34874d78ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:06.479224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d75ee2-623d-4242-949f-ac34874d78ac None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:06.479500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d75ee2-623d-4242-949f-ac34874d78ac None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:06.479768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d75ee2-623d-4242-949f-ac34874d78ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:06.480770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d75ee2-623d-4242-949f-ac34874d78ac None admin] RBAC: Adding query filter params (role.id=fab66a1a6ba64c9583182c78978b218e, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=6225c81d2620463886d8a936b59784df) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:06.481106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d75ee2-623d-4242-949f-ac34874d78ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:06.483035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d75ee2-623d-4242-949f-ac34874d78ac None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:06.491063 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 286/576] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:33:06 2022] GET /identity/v3/role_assignments?user.id=6225c81d2620463886d8a936b59784df&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=fab66a1a6ba64c9583182c78978b218e => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:06.503502 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 291/577] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:06.778814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a502a-0709-40eb-b13f-3c43c7d5c72b None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:06.779259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a502a-0709-40eb-b13f-3c43c7d5c72b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:06.779363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a502a-0709-40eb-b13f-3c43c7d5c72b None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:06.781127 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 287/578] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:06.786561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2619853-ebbf-4d77-9a52-82aaaf39fba5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:06.786838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2619853-ebbf-4d77-9a52-82aaaf39fba5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:06.787109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2619853-ebbf-4d77-9a52-82aaaf39fba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:06.822219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a2619853-ebbf-4d77-9a52-82aaaf39fba5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:06.823452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a2619853-ebbf-4d77-9a52-82aaaf39fba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:06.902748 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 292/579] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:06.969827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a502a-0709-40eb-b13f-3c43c7d5c72b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:07.012988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c4cca-cf8d-433d-845d-3059e05eeb21 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:07.013331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c4cca-cf8d-433d-845d-3059e05eeb21 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:07.013559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c4cca-cf8d-433d-845d-3059e05eeb21 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:07.015156 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 293/580] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:07.021489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79952a0-d094-44dd-8105-bf76fb85f82b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:07.022339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79952a0-d094-44dd-8105-bf76fb85f82b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:07.022613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79952a0-d094-44dd-8105-bf76fb85f82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:07.037643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:07.045941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e79952a0-d094-44dd-8105-bf76fb85f82b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:07.047544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e79952a0-d094-44dd-8105-bf76fb85f82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:07.050550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:07.133781 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 294/581] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:07.173750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a502a-0709-40eb-b13f-3c43c7d5c72b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:07.232475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee4369-9f3e-44a2-95f5-7756838a4363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:07.233120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee4369-9f3e-44a2-95f5-7756838a4363 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:07.233397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee4369-9f3e-44a2-95f5-7756838a4363 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:07.233660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee4369-9f3e-44a2-95f5-7756838a4363 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:07.234233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee4369-9f3e-44a2-95f5-7756838a4363 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:07.235606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee4369-9f3e-44a2-95f5-7756838a4363 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:07.250708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79952a0-d094-44dd-8105-bf76fb85f82b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-deee4369-9f3e-44a2-95f5-7756838a4363 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:07.252570 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:07.253548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:07.254403 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 288/582] 10.222.0.16 () {56 vars in 1129 bytes} [Sat Oct 15 20:33:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:07.264125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee4369-9f3e-44a2-95f5-7756838a4363 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:07.407099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79952a0-d094-44dd-8105-bf76fb85f82b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:07.445141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a487a-6970-4703-8d3b-b046ed650ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:07.445766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a487a-6970-4703-8d3b-b046ed650ab4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:07.446035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a487a-6970-4703-8d3b-b046ed650ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:07.446294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a487a-6970-4703-8d3b-b046ed650ab4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:07.457556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:07.462512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a487a-6970-4703-8d3b-b046ed650ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-d12a487a-6970-4703-8d3b-b046ed650ab4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:07.477063 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:07.478065 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:07.478979 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 295/583] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:33:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:07.485066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61daeb-76a8-4d8b-977b-8ca1d6ef1241 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:07.485066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61daeb-76a8-4d8b-977b-8ca1d6ef1241 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:07.485066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61daeb-76a8-4d8b-977b-8ca1d6ef1241 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:07.489060 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 296/584] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:07 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:07.493100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61daeb-76a8-4d8b-977b-8ca1d6ef1241 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:07.510737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee4369-9f3e-44a2-95f5-7756838a4363 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:07.522714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:07.558746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34748fe8-3f68-45e5-a935-f19feb75f659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:07.559358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34748fe8-3f68-45e5-a935-f19feb75f659 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:07.559860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34748fe8-3f68-45e5-a935-f19feb75f659 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:07.559860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34748fe8-3f68-45e5-a935-f19feb75f659 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:07.560492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34748fe8-3f68-45e5-a935-f19feb75f659 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:07.560786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34748fe8-3f68-45e5-a935-f19feb75f659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:07.562225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34748fe8-3f68-45e5-a935-f19feb75f659 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:07.572451 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 289/585] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Oct 15 20:33:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:07.573815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3baada9-a784-4224-ac59-8d61c7aee56a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:07.574341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3baada9-a784-4224-ac59-8d61c7aee56a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:07.574341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3baada9-a784-4224-ac59-8d61c7aee56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:07.592645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c3baada9-a784-4224-ac59-8d61c7aee56a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:07.593838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c3baada9-a784-4224-ac59-8d61c7aee56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:07.621648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61daeb-76a8-4d8b-977b-8ca1d6ef1241 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:07.667365 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 290/586] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:07.668491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3baada9-a784-4224-ac59-8d61c7aee56a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:07.678613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52b23c-a8b9-4d66-bde5-65192fc6ad16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:07.679230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52b23c-a8b9-4d66-bde5-65192fc6ad16 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:07.679507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52b23c-a8b9-4d66-bde5-65192fc6ad16 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:07.679778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52b23c-a8b9-4d66-bde5-65192fc6ad16 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:07.680919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52b23c-a8b9-4d66-bde5-65192fc6ad16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:07.681249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52b23c-a8b9-4d66-bde5-65192fc6ad16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:07.683674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52b23c-a8b9-4d66-bde5-65192fc6ad16 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:07.698602 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 297/587] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:33:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:07.699473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52b23c-a8b9-4d66-bde5-65192fc6ad16 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:07.782157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:07.838381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52b23c-a8b9-4d66-bde5-65192fc6ad16 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:07.856364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3baada9-a784-4224-ac59-8d61c7aee56a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:07.858474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:07.893501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156767f7-1bc4-45bb-9379-ddeb0e7c574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:07.894588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156767f7-1bc4-45bb-9379-ddeb0e7c574b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:07.895104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209893a9-e856-41e5-bcec-b37d21b3c900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:07.895257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156767f7-1bc4-45bb-9379-ddeb0e7c574b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:07.895585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156767f7-1bc4-45bb-9379-ddeb0e7c574b None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:07.896167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209893a9-e856-41e5-bcec-b37d21b3c900 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:07.896404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156767f7-1bc4-45bb-9379-ddeb0e7c574b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:07.897351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209893a9-e856-41e5-bcec-b37d21b3c900 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:07.897523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156767f7-1bc4-45bb-9379-ddeb0e7c574b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:07.897977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209893a9-e856-41e5-bcec-b37d21b3c900 None admin] PATH_INFO: `/v3/services/network` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:07.898769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209893a9-e856-41e5-bcec-b37d21b3c900 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:07.899124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156767f7-1bc4-45bb-9379-ddeb0e7c574b None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:07.900434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209893a9-e856-41e5-bcec-b37d21b3c900 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:07.905083 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 298/588] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Oct 15 20:33:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:07.905978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156767f7-1bc4-45bb-9379-ddeb0e7c574b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-209893a9-e856-41e5-bcec-b37d21b3c900 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:07.910686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:07.911942 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 20:33:07.913548 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:07.916058 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 291/589] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:07 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:07.917218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209893a9-e856-41e5-bcec-b37d21b3c900 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.025355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209893a9-e856-41e5-bcec-b37d21b3c900 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:08.036798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156767f7-1bc4-45bb-9379-ddeb0e7c574b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:08.064513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:08.064785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579b48c-ab4e-4795-8b1c-48b0d5e2cfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:08.065442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579b48c-ab4e-4795-8b1c-48b0d5e2cfa2 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:08.065640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579b48c-ab4e-4795-8b1c-48b0d5e2cfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:08.065886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579b48c-ab4e-4795-8b1c-48b0d5e2cfa2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:08.066582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579b48c-ab4e-4795-8b1c-48b0d5e2cfa2 None admin] RBAC: Adding query filter params (interface=public, service_id=8110d8de107a4c16bdf36c4f3f810c52, region_id=RegionOne) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:08.066890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579b48c-ab4e-4795-8b1c-48b0d5e2cfa2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:08.068262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579b48c-ab4e-4795-8b1c-48b0d5e2cfa2 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:08.075862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f44d24-dbf7-4e08-8a70-8c430fa5ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:08.080664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f44d24-dbf7-4e08-8a70-8c430fa5ac48 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:08.080791 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 292/590] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Oct 15 20:33:07 2022] GET /identity/v3/endpoints?service_id=8110d8de107a4c16bdf36c4f3f810c52&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:08.080884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f44d24-dbf7-4e08-8a70-8c430fa5ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:08.080884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f44d24-dbf7-4e08-8a70-8c430fa5ac48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:08.081071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579b48c-ab4e-4795-8b1c-48b0d5e2cfa2 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.092888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f44d24-dbf7-4e08-8a70-8c430fa5ac48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:08.094362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f44d24-dbf7-4e08-8a70-8c430fa5ac48 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-11f44d24-dbf7-4e08-8a70-8c430fa5ac48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:08.109958 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:08.110908 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:08.111751 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:08.111751 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:08.111751 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:08.111751 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 299/591] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:08.112276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f44d24-dbf7-4e08-8a70-8c430fa5ac48 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.119784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:08.205124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579b48c-ab4e-4795-8b1c-48b0d5e2cfa2 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:08.222252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f44d24-dbf7-4e08-8a70-8c430fa5ac48 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:08.243536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e5d55-26a4-4cc3-aace-2514cee9ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:08.244144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e5d55-26a4-4cc3-aace-2514cee9ab00 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:08.244467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e5d55-26a4-4cc3-aace-2514cee9ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:08.244741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e5d55-26a4-4cc3-aace-2514cee9ab00 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:08.245426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e5d55-26a4-4cc3-aace-2514cee9ab00 None admin] RBAC: Adding query filter params (name=network) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:08.245726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e5d55-26a4-4cc3-aace-2514cee9ab00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:08.247109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e5d55-26a4-4cc3-aace-2514cee9ab00 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:08.252123 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 293/592] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:08.258194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e5d55-26a4-4cc3-aace-2514cee9ab00 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.260556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b7981-156e-4abf-aa50-8ff97521c93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:08.261218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b7981-156e-4abf-aa50-8ff97521c93c None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:08.261494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b7981-156e-4abf-aa50-8ff97521c93c None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:08.261762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b7981-156e-4abf-aa50-8ff97521c93c None admin] PATH_INFO: `/v3/roles/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:08.272786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b7981-156e-4abf-aa50-8ff97521c93c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-f78b7981-156e-4abf-aa50-8ff97521c93c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:08.278361 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:08.279236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 300/593] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:33:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:08.280053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b7981-156e-4abf-aa50-8ff97521c93c None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.326789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:08.363587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:08.371402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e5d55-26a4-4cc3-aace-2514cee9ab00 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:08.394044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b7981-156e-4abf-aa50-8ff97521c93c None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:08.407335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33fe8-5cd8-4581-9377-f002ca177151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:08.407938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33fe8-5cd8-4581-9377-f002ca177151 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:08.408216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33fe8-5cd8-4581-9377-f002ca177151 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:08.408486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33fe8-5cd8-4581-9377-f002ca177151 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:08.409139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f33fe8-5cd8-4581-9377-f002ca177151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:08.410489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f33fe8-5cd8-4581-9377-f002ca177151 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:08.425066 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 294/594] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Oct 15 20:33:08 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:08.425066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33fe8-5cd8-4581-9377-f002ca177151 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.449986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b21543-9e06-471a-950e-701c53e03cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:08.450719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b21543-9e06-471a-950e-701c53e03cce None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:08.451056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b21543-9e06-471a-950e-701c53e03cce None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:08.451384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b21543-9e06-471a-950e-701c53e03cce None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:08.452161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b21543-9e06-471a-950e-701c53e03cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:08.452525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b21543-9e06-471a-950e-701c53e03cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:08.454355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b21543-9e06-471a-950e-701c53e03cce None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:08.470297 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 301/595] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:08.471350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b21543-9e06-471a-950e-701c53e03cce None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.566425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33fe8-5cd8-4581-9377-f002ca177151 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:08.597682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:08.602852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b21543-9e06-471a-950e-701c53e03cce None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:08.624481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86482b-571e-468b-8f0f-a794a74c8179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:08.625843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86482b-571e-468b-8f0f-a794a74c8179 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:08.626303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86482b-571e-468b-8f0f-a794a74c8179 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:08.626743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86482b-571e-468b-8f0f-a794a74c8179 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:08.627534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86482b-571e-468b-8f0f-a794a74c8179 None admin] RBAC: Adding query filter params (type=network) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:08.627987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86482b-571e-468b-8f0f-a794a74c8179 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:08.630445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86482b-571e-468b-8f0f-a794a74c8179 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:08.645070 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 295/596] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:08.645700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86482b-571e-468b-8f0f-a794a74c8179 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.652201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e505753-4e1c-46f2-9954-53ee05ea357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:08.652812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e505753-4e1c-46f2-9954-53ee05ea357a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:08.653104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e505753-4e1c-46f2-9954-53ee05ea357a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:08.653368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e505753-4e1c-46f2-9954-53ee05ea357a None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:08.654018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e505753-4e1c-46f2-9954-53ee05ea357a None admin] RBAC: Adding query filter params (name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:08.654308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e505753-4e1c-46f2-9954-53ee05ea357a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:08.655644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e505753-4e1c-46f2-9954-53ee05ea357a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:08.666259 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 302/597] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:33:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:08.667152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e505753-4e1c-46f2-9954-53ee05ea357a None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.682148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:08.780882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e505753-4e1c-46f2-9954-53ee05ea357a None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:08.790323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86482b-571e-468b-8f0f-a794a74c8179 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:08.817372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626656b9-1b9a-486f-94eb-feafb128db79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:08.817890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626656b9-1b9a-486f-94eb-feafb128db79 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:08.818163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626656b9-1b9a-486f-94eb-feafb128db79 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:08.818455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626656b9-1b9a-486f-94eb-feafb128db79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:08.819723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626656b9-1b9a-486f-94eb-feafb128db79 None admin] RBAC: Adding query filter params (interface=public, service_id=2318852920374e678b951cd54214fa15, region_id=RegionOne) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:08.820029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626656b9-1b9a-486f-94eb-feafb128db79 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:08.821398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626656b9-1b9a-486f-94eb-feafb128db79 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:08.829311 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 303/598] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Oct 15 20:33:08 2022] GET /identity/v3/endpoints?service_id=2318852920374e678b951cd54214fa15&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:08.830110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626656b9-1b9a-486f-94eb-feafb128db79 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.837861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524a6f9-72ce-4917-bed9-6f32618cbd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:08.838816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524a6f9-72ce-4917-bed9-6f32618cbd63 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:08.839298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524a6f9-72ce-4917-bed9-6f32618cbd63 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:08.839925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524a6f9-72ce-4917-bed9-6f32618cbd63 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:08.852440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524a6f9-72ce-4917-bed9-6f32618cbd63 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:08.855020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524a6f9-72ce-4917-bed9-6f32618cbd63 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:08.870880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-b524a6f9-72ce-4917-bed9-6f32618cbd63 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:33:08.872745 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:08.873825 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:08.874781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:08.874781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:08.874781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:33:08.874781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:33:08.874781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:33:08.874781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:33:08.874781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:33:08.874781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:33:08.874781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 20:33:08.874781 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:08.883653 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 296/599] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:33:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:08.884884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524a6f9-72ce-4917-bed9-6f32618cbd63 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:08.964848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626656b9-1b9a-486f-94eb-feafb128db79 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:09.006961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:09.007780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b9f4c-1340-41aa-b2e9-e13f9182b926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:09.008652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b9f4c-1340-41aa-b2e9-e13f9182b926 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:09.008956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b9f4c-1340-41aa-b2e9-e13f9182b926 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:09.009345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b9f4c-1340-41aa-b2e9-e13f9182b926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:09.021716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b9f4c-1340-41aa-b2e9-e13f9182b926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:09.023489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b9f4c-1340-41aa-b2e9-e13f9182b926 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:09.034926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524a6f9-72ce-4917-bed9-6f32618cbd63 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-4c2b9f4c-1340-41aa-b2e9-e13f9182b926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:09.039487 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:09.040605 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:09.041739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:09.041739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:09.041739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:09.042825 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 304/600] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:09.045309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b9f4c-1340-41aa-b2e9-e13f9182b926 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:09.088333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933408d8-1427-42ca-961b-b4d612ba5eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:09.089727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933408d8-1427-42ca-961b-b4d612ba5eed None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:09.090155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933408d8-1427-42ca-961b-b4d612ba5eed None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:09.090572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933408d8-1427-42ca-961b-b4d612ba5eed None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:09.091825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933408d8-1427-42ca-961b-b4d612ba5eed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:09.094156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933408d8-1427-42ca-961b-b4d612ba5eed None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:09.102162 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 297/601] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Oct 15 20:33:08 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:09.103846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933408d8-1427-42ca-961b-b4d612ba5eed None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:09.132722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:09.198514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b9f4c-1340-41aa-b2e9-e13f9182b926 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:09.274059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b572d80-68db-41a8-b17b-dd72ed128458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:09.274714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b572d80-68db-41a8-b17b-dd72ed128458 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:09.276585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b572d80-68db-41a8-b17b-dd72ed128458 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:09.276959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b572d80-68db-41a8-b17b-dd72ed128458 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:09.279538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b572d80-68db-41a8-b17b-dd72ed128458 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:09.281135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b572d80-68db-41a8-b17b-dd72ed128458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:09.282633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b572d80-68db-41a8-b17b-dd72ed128458 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:09.329229 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 305/602] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Oct 15 20:33:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:09.331165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b572d80-68db-41a8-b17b-dd72ed128458 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:09.345445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933408d8-1427-42ca-961b-b4d612ba5eed None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:09.406715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf575e-8351-4641-90b0-78b3e74ec459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:09.407562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf575e-8351-4641-90b0-78b3e74ec459 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:09.408081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf575e-8351-4641-90b0-78b3e74ec459 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:09.408492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf575e-8351-4641-90b0-78b3e74ec459 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:09.409338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf575e-8351-4641-90b0-78b3e74ec459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:09.409787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf575e-8351-4641-90b0-78b3e74ec459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:09.412488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf575e-8351-4641-90b0-78b3e74ec459 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:09.430309 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 298/603] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:09.435508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf575e-8351-4641-90b0-78b3e74ec459 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:09.445748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:09.550489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b572d80-68db-41a8-b17b-dd72ed128458 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:09.560062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf575e-8351-4641-90b0-78b3e74ec459 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:09.594240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636892e-2e2f-46cf-a5ed-3272ff2d186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:09.595295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636892e-2e2f-46cf-a5ed-3272ff2d186f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:09.595866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636892e-2e2f-46cf-a5ed-3272ff2d186f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:09.596227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57712339-a7ae-44aa-8422-b06b9ff68d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:09.596445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636892e-2e2f-46cf-a5ed-3272ff2d186f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:09.597067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57712339-a7ae-44aa-8422-b06b9ff68d98 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:09.597463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57712339-a7ae-44aa-8422-b06b9ff68d98 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:09.597830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57712339-a7ae-44aa-8422-b06b9ff68d98 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:09.600534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:09.606192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57712339-a7ae-44aa-8422-b06b9ff68d98 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:09.607915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57712339-a7ae-44aa-8422-b06b9ff68d98 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-57712339-a7ae-44aa-8422-b06b9ff68d98 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:33:09.620263 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:09.621357 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:09.622476 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 299/604] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Oct 15 20:33:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:09.624333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c4f3-e433-465f-83de-99e91381a8a2 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:09.624706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c4f3-e433-465f-83de-99e91381a8a2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:09.629086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c4f3-e433-465f-83de-99e91381a8a2 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:09.629086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 300/605] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:09.631785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9c4f3-e433-465f-83de-99e91381a8a2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:09.694613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636892e-2e2f-46cf-a5ed-3272ff2d186f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:09.696457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636892e-2e2f-46cf-a5ed-3272ff2d186f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:09.708119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:09.747009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9c4f3-e433-465f-83de-99e91381a8a2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:09.783875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6b8cb-6c55-4fd3-84b3-777a7fc1312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:09.784474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b8cb-6c55-4fd3-84b3-777a7fc1312a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:09.784747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b8cb-6c55-4fd3-84b3-777a7fc1312a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:09.785010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b8cb-6c55-4fd3-84b3-777a7fc1312a None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:09.785859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6b8cb-6c55-4fd3-84b3-777a7fc1312a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:09.786210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6b8cb-6c55-4fd3-84b3-777a7fc1312a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:09.788364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6b8cb-6c55-4fd3-84b3-777a7fc1312a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:09.813486 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 301/607] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Oct 15 20:33:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:09.813486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a199c4-0e03-4fe1-8833-3c519a48810c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:09.813486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a199c4-0e03-4fe1-8833-3c519a48810c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:09.813486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a199c4-0e03-4fe1-8833-3c519a48810c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:09.814200 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 306/607] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:33:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 15 20:33:09.815365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636892e-2e2f-46cf-a5ed-3272ff2d186f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:09.836141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e1a199c4-0e03-4fe1-8833-3c519a48810c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:09.837398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e1a199c4-0e03-4fe1-8833-3c519a48810c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:09.855115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:09.933853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636892e-2e2f-46cf-a5ed-3272ff2d186f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:09.934666 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 302/608] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:09.935626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a199c4-0e03-4fe1-8833-3c519a48810c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:09.971143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e7770-4da1-48a9-a2b2-5a586e88ee4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:09.972078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e7770-4da1-48a9-a2b2-5a586e88ee4c None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:09.972575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e7770-4da1-48a9-a2b2-5a586e88ee4c None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:09.973079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e7770-4da1-48a9-a2b2-5a586e88ee4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:10.007876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:10.056732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a199c4-0e03-4fe1-8833-3c519a48810c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:10.070969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e7770-4da1-48a9-a2b2-5a586e88ee4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:10.072427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e7770-4da1-48a9-a2b2-5a586e88ee4c None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:10.096619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be26e9a-02a8-4504-a25f-5463edae315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:10.097632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be26e9a-02a8-4504-a25f-5463edae315a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:10.098118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be26e9a-02a8-4504-a25f-5463edae315a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:10.098623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be26e9a-02a8-4504-a25f-5463edae315a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:10.110663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be26e9a-02a8-4504-a25f-5463edae315a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:10.112463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be26e9a-02a8-4504-a25f-5463edae315a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:10.125167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-1be26e9a-02a8-4504-a25f-5463edae315a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:10.139758 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:10.140720 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:10.141587 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:10.141587 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:10.141587 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:10.143740 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 303/609] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:10.154862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be26e9a-02a8-4504-a25f-5463edae315a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:10.262094 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 307/610] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:33:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 20:33:10.267524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e7770-4da1-48a9-a2b2-5a586e88ee4c None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:10.304454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be26e9a-02a8-4504-a25f-5463edae315a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:10.312372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:10.361776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81cde2-2988-4a97-b2a2-e8b2603c9414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:10.362724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81cde2-2988-4a97-b2a2-e8b2603c9414 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:10.363239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81cde2-2988-4a97-b2a2-e8b2603c9414 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:10.363726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81cde2-2988-4a97-b2a2-e8b2603c9414 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:10.364565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81cde2-2988-4a97-b2a2-e8b2603c9414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:10.365127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81cde2-2988-4a97-b2a2-e8b2603c9414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:10.367558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81cde2-2988-4a97-b2a2-e8b2603c9414 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:10.377071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e7770-4da1-48a9-a2b2-5a586e88ee4c None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:10.384830 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 304/611] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:10.386248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81cde2-2988-4a97-b2a2-e8b2603c9414 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:10.416230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19e10-29e8-463e-b703-e9c5ad41f797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:10.416954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19e10-29e8-463e-b703-e9c5ad41f797 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:10.417564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19e10-29e8-463e-b703-e9c5ad41f797 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:10.417944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19e10-29e8-463e-b703-e9c5ad41f797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:10.420608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:10.431755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff19e10-29e8-463e-b703-e9c5ad41f797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:10.433383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff19e10-29e8-463e-b703-e9c5ad41f797 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-6ff19e10-29e8-463e-b703-e9c5ad41f797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:10.453874 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:10.455049 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:10.456136 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:10.456136 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:10.456136 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:10.461108 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 308/612] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:10.462096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19e10-29e8-463e-b703-e9c5ad41f797 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:10.509926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81cde2-2988-4a97-b2a2-e8b2603c9414 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:10.561996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d5e0-e700-48c6-853f-e00c680fa3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:10.562652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d5e0-e700-48c6-853f-e00c680fa3d1 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:10.562962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d5e0-e700-48c6-853f-e00c680fa3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:10.563242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d5e0-e700-48c6-853f-e00c680fa3d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:10.563816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06d5e0-e700-48c6-853f-e00c680fa3d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:10.565928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06d5e0-e700-48c6-853f-e00c680fa3d1 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-af06d5e0-e700-48c6-853f-e00c680fa3d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:10.577285 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:10.578411 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 20:33:10.579456 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:10.580437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19e10-29e8-463e-b703-e9c5ad41f797 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:10.581649 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 305/613] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Oct 15 20:33:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:10.582613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d5e0-e700-48c6-853f-e00c680fa3d1 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:10.625599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:10.629967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbb884-2eb3-4bbc-8406-b9334928634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:10.631694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbb884-2eb3-4bbc-8406-b9334928634a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:10.632125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbb884-2eb3-4bbc-8406-b9334928634a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:10.632563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbb884-2eb3-4bbc-8406-b9334928634a None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:10.641370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbb884-2eb3-4bbc-8406-b9334928634a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:10.643458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbb884-2eb3-4bbc-8406-b9334928634a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-cddbb884-2eb3-4bbc-8406-b9334928634a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:33:10.659184 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:10.660295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:10.661390 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:33:10.661390 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:33:10.661390 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:33:10.661390 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:33:10.661390 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:33:10.661390 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:33:10.661390 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:10.661390 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:10.661390 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 309/614] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:10.661390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbb884-2eb3-4bbc-8406-b9334928634a None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:10.690881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d5e0-e700-48c6-853f-e00c680fa3d1 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:10.725379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:10.726301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9c6ec-092b-4ea9-8694-e4848c54b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:10.727001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9c6ec-092b-4ea9-8694-e4848c54b35a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:10.727221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9c6ec-092b-4ea9-8694-e4848c54b35a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:10.727651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9c6ec-092b-4ea9-8694-e4848c54b35a None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:10.728540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9c6ec-092b-4ea9-8694-e4848c54b35a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:10.728901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9c6ec-092b-4ea9-8694-e4848c54b35a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:10.730628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9c6ec-092b-4ea9-8694-e4848c54b35a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:10.750629 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 306/615] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:10.751763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9c6ec-092b-4ea9-8694-e4848c54b35a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:10.804433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbb884-2eb3-4bbc-8406-b9334928634a None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:10.846110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258da64-6e26-462d-b3ce-171cc7a69de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:10.847572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258da64-6e26-462d-b3ce-171cc7a69de3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:10.848409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258da64-6e26-462d-b3ce-171cc7a69de3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:10.848898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258da64-6e26-462d-b3ce-171cc7a69de3 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:10.850793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258da64-6e26-462d-b3ce-171cc7a69de3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:10.851321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258da64-6e26-462d-b3ce-171cc7a69de3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:10.853671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258da64-6e26-462d-b3ce-171cc7a69de3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:10.865806 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 310/616] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:10.866729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258da64-6e26-462d-b3ce-171cc7a69de3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:10.867397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:10.883175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9c6ec-092b-4ea9-8694-e4848c54b35a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:10.919963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac28ed8-db34-4f7e-8a48-056962fb756c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:10.920576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac28ed8-db34-4f7e-8a48-056962fb756c None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:10.920845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac28ed8-db34-4f7e-8a48-056962fb756c None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:10.921151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac28ed8-db34-4f7e-8a48-056962fb756c None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:10.921834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac28ed8-db34-4f7e-8a48-056962fb756c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:10.922128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac28ed8-db34-4f7e-8a48-056962fb756c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:10.923548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac28ed8-db34-4f7e-8a48-056962fb756c None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:10.938010 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 307/617] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:33:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:10.938903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac28ed8-db34-4f7e-8a48-056962fb756c None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:11.012768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258da64-6e26-462d-b3ce-171cc7a69de3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:11.027620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:11.045732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac28ed8-db34-4f7e-8a48-056962fb756c None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:11.063331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d1e00-68bd-473e-83d0-a1b4ca287508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:11.064460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1e00-68bd-473e-83d0-a1b4ca287508 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:11.065332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1e00-68bd-473e-83d0-a1b4ca287508 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:11.065780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1e00-68bd-473e-83d0-a1b4ca287508 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:11.075927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d1e00-68bd-473e-83d0-a1b4ca287508 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:11.078465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d1e00-68bd-473e-83d0-a1b4ca287508 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:11.082930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56156857-17aa-4bc4-8e9f-a5cb8ca9c1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:11.083604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56156857-17aa-4bc4-8e9f-a5cb8ca9c1e6 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:11.084491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56156857-17aa-4bc4-8e9f-a5cb8ca9c1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:11.085014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56156857-17aa-4bc4-8e9f-a5cb8ca9c1e6 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:11.085742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56156857-17aa-4bc4-8e9f-a5cb8ca9c1e6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:11.086112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56156857-17aa-4bc4-8e9f-a5cb8ca9c1e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:11.087577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56156857-17aa-4bc4-8e9f-a5cb8ca9c1e6 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:11.101079 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 308/618] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:11.101079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56156857-17aa-4bc4-8e9f-a5cb8ca9c1e6 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-ba3d1e00-68bd-473e-83d0-a1b4ca287508 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:33:11.101357 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:11.102167 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:11.103030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:33:11.103030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:33:11.103030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:33:11.103030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:33:11.103030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:33:11.103030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:33:11.103030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:11.103030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:11.103030 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 311/619] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:11.106604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d1e00-68bd-473e-83d0-a1b4ca287508 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:11.131994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:11.259436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d1e00-68bd-473e-83d0-a1b4ca287508 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:11.263325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56156857-17aa-4bc4-8e9f-a5cb8ca9c1e6 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:11.299987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54175dd8-4f73-4171-960e-ae46d4897ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:11.300590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54175dd8-4f73-4171-960e-ae46d4897ba7 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:11.300860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54175dd8-4f73-4171-960e-ae46d4897ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:11.301166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54175dd8-4f73-4171-960e-ae46d4897ba7 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:11.301869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54175dd8-4f73-4171-960e-ae46d4897ba7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:11.302169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54175dd8-4f73-4171-960e-ae46d4897ba7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:11.303955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54175dd8-4f73-4171-960e-ae46d4897ba7 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:11.304992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7720d47-8d81-4efe-8ef5-94539af2aebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:11.306019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720d47-8d81-4efe-8ef5-94539af2aebe None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:11.306658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720d47-8d81-4efe-8ef5-94539af2aebe None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:11.307213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720d47-8d81-4efe-8ef5-94539af2aebe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:11.308154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7720d47-8d81-4efe-8ef5-94539af2aebe None admin] RBAC: Adding query filter params (role.id=fab66a1a6ba64c9583182c78978b218e, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=abfea6b853134fed8ea5e32e5406bd5f) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:11.308669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7720d47-8d81-4efe-8ef5-94539af2aebe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:11.310354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7720d47-8d81-4efe-8ef5-94539af2aebe None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:11.317369 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 309/620] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:33:11 2022] GET /identity/v3/role_assignments?user.id=abfea6b853134fed8ea5e32e5406bd5f&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=fab66a1a6ba64c9583182c78978b218e => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:11.319528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d4f85-cceb-4fa4-8cba-d5188a863581 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:11.320021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d4f85-cceb-4fa4-8cba-d5188a863581 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:11.320486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d4f85-cceb-4fa4-8cba-d5188a863581 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:11.322382 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 310/622] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:11.324105 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 312/622] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:33:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:11.325301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d4f85-cceb-4fa4-8cba-d5188a863581 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:11.328438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9f30c-eed1-4c05-847c-7d8f78238f8d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:11.328715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9f30c-eed1-4c05-847c-7d8f78238f8d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:11.328977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9f30c-eed1-4c05-847c-7d8f78238f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:11.361896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-34c9f30c-eed1-4c05-847c-7d8f78238f8d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:11.363359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-34c9f30c-eed1-4c05-847c-7d8f78238f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:11.393488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:11.455167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:11.469614 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 313/623] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:11.490000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d4f85-cceb-4fa4-8cba-d5188a863581 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:11.541950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c477-e8fe-4c8c-bb1d-2e335cb95873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:11.542561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c477-e8fe-4c8c-bb1d-2e335cb95873 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:11.542831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c477-e8fe-4c8c-bb1d-2e335cb95873 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:11.543150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c477-e8fe-4c8c-bb1d-2e335cb95873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:11.543891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4c477-e8fe-4c8c-bb1d-2e335cb95873 None admin] RBAC: Adding query filter params (role.id=492ce6c3fedb4109ac31de9467e8ef3b, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=7a9c40e53f72418381298190ca330e14) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:11.544191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4c477-e8fe-4c8c-bb1d-2e335cb95873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:11.545716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4c477-e8fe-4c8c-bb1d-2e335cb95873 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:11.559042 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 311/624] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:33:11 2022] GET /identity/v3/role_assignments?user.id=7a9c40e53f72418381298190ca330e14&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=492ce6c3fedb4109ac31de9467e8ef3b => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:11.829772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9f30c-eed1-4c05-847c-7d8f78238f8d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:11.938933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9f30c-eed1-4c05-847c-7d8f78238f8d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:11.976380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db94f0-c935-49a1-b3f3-b617b6e85857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:11.977013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db94f0-c935-49a1-b3f3-b617b6e85857 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:11.977323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db94f0-c935-49a1-b3f3-b617b6e85857 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:11.977649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db94f0-c935-49a1-b3f3-b617b6e85857 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:11.978232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db94f0-c935-49a1-b3f3-b617b6e85857 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:11.979584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db94f0-c935-49a1-b3f3-b617b6e85857 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-38db94f0-c935-49a1-b3f3-b617b6e85857 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:11.988505 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:11.989524 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:11.990890 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 314/625] 10.222.0.16 () {56 vars in 1129 bytes} [Sat Oct 15 20:33:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:11.996936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c477-e8fe-4c8c-bb1d-2e335cb95873 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:12.015838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc29be-9a00-4cdb-a754-4f2119203998 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:12.016144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc29be-9a00-4cdb-a754-4f2119203998 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:12.016374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc29be-9a00-4cdb-a754-4f2119203998 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:12.018169 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 315/626] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:12.024714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec37f4e-3699-4a44-ad62-eaa6fee24aff None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:12.024991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec37f4e-3699-4a44-ad62-eaa6fee24aff None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:12.025363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec37f4e-3699-4a44-ad62-eaa6fee24aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:12.058332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-aec37f4e-3699-4a44-ad62-eaa6fee24aff None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:12.059529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aec37f4e-3699-4a44-ad62-eaa6fee24aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:12.105479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:12.195614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c477-e8fe-4c8c-bb1d-2e335cb95873 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:12.206714 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 316/627] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:12.250351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46364bcc-1dbd-4fa1-b7e8-0edde30e44ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:12.250954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46364bcc-1dbd-4fa1-b7e8-0edde30e44ae None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:12.251229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46364bcc-1dbd-4fa1-b7e8-0edde30e44ae None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:12.251500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46364bcc-1dbd-4fa1-b7e8-0edde30e44ae None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:12.252133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46364bcc-1dbd-4fa1-b7e8-0edde30e44ae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:12.252498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46364bcc-1dbd-4fa1-b7e8-0edde30e44ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:12.253995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46364bcc-1dbd-4fa1-b7e8-0edde30e44ae None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:12.259306 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 312/628] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Oct 15 20:33:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:12.265221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec37f4e-3699-4a44-ad62-eaa6fee24aff None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:12.384887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec37f4e-3699-4a44-ad62-eaa6fee24aff None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:12.426696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2eb86-0ed5-467f-8bf9-4fbf57bddbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:12.427349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2eb86-0ed5-467f-8bf9-4fbf57bddbf9 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:12.428068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2eb86-0ed5-467f-8bf9-4fbf57bddbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:12.428711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2eb86-0ed5-467f-8bf9-4fbf57bddbf9 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:12.428988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2eb86-0ed5-467f-8bf9-4fbf57bddbf9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:12.429311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2eb86-0ed5-467f-8bf9-4fbf57bddbf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:12.430656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2eb86-0ed5-467f-8bf9-4fbf57bddbf9 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:12.436052 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 317/629] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Oct 15 20:33:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:12.441697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46364bcc-1dbd-4fa1-b7e8-0edde30e44ae None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:12.460725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:12.590508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46364bcc-1dbd-4fa1-b7e8-0edde30e44ae None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:12.626917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ec8e3-12f0-4752-8f46-ec396eec88ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:12.627860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ec8e3-12f0-4752-8f46-ec396eec88ec None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:12.628368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ec8e3-12f0-4752-8f46-ec396eec88ec None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:12.628829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ec8e3-12f0-4752-8f46-ec396eec88ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:12.629712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ec8e3-12f0-4752-8f46-ec396eec88ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:12.631986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ec8e3-12f0-4752-8f46-ec396eec88ec None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:12.662718 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 313/630] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:33:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:12.672043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2eb86-0ed5-467f-8bf9-4fbf57bddbf9 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:12.710782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:12.838738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2eb86-0ed5-467f-8bf9-4fbf57bddbf9 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:12.895044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc0d3e-4c89-4df0-abf7-72fd8ebb2b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:12.895667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0d3e-4c89-4df0-abf7-72fd8ebb2b9a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:12.895936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0d3e-4c89-4df0-abf7-72fd8ebb2b9a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:12.896198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0d3e-4c89-4df0-abf7-72fd8ebb2b9a None admin] PATH_INFO: `/v3/services/network` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:12.896791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc0d3e-4c89-4df0-abf7-72fd8ebb2b9a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:12.898337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc0d3e-4c89-4df0-abf7-72fd8ebb2b9a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-00fc0d3e-4c89-4df0-abf7-72fd8ebb2b9a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:12.910051 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:12.911132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:12.912503 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 318/631] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:12 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:12.918050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ec8e3-12f0-4752-8f46-ec396eec88ec None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:12.993834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:13.038407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ec8e3-12f0-4752-8f46-ec396eec88ec None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:13.085108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3c246-f950-495a-b676-354c0962ca26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:13.086100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3c246-f950-495a-b676-354c0962ca26 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:13.086601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3c246-f950-495a-b676-354c0962ca26 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:13.087108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3c246-f950-495a-b676-354c0962ca26 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:13.087934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3c246-f950-495a-b676-354c0962ca26 None admin] RBAC: Adding query filter params (name=network) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:13.088428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3c246-f950-495a-b676-354c0962ca26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:13.090112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3c246-f950-495a-b676-354c0962ca26 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:13.101118 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 314/632] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:13.107009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc0d3e-4c89-4df0-abf7-72fd8ebb2b9a None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:13.245088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:13.273284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc0d3e-4c89-4df0-abf7-72fd8ebb2b9a None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:13.319229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69c65e-8200-463b-a496-e0faf56e34c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:13.319828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69c65e-8200-463b-a496-e0faf56e34c3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:13.320097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69c65e-8200-463b-a496-e0faf56e34c3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:13.320360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69c65e-8200-463b-a496-e0faf56e34c3 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:13.320979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69c65e-8200-463b-a496-e0faf56e34c3 None admin] RBAC: Adding query filter params (type=network) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:13.321301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69c65e-8200-463b-a496-e0faf56e34c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:13.322649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69c65e-8200-463b-a496-e0faf56e34c3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:13.329405 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 319/633] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:13.335161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3c246-f950-495a-b676-354c0962ca26 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:13.456503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3c246-f950-495a-b676-354c0962ca26 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:13.488691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:13.501130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be40f2-a92c-4708-8647-1052096c8927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:13.502128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be40f2-a92c-4708-8647-1052096c8927 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:13.502600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be40f2-a92c-4708-8647-1052096c8927 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:13.503064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be40f2-a92c-4708-8647-1052096c8927 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:13.503927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be40f2-a92c-4708-8647-1052096c8927 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:13.505554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be40f2-a92c-4708-8647-1052096c8927 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:13.534251 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 315/634] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:33:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:14.149332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12779223-afc4-4a7a-b827-cc8b8d870c42 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:14.150406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12779223-afc4-4a7a-b827-cc8b8d870c42 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:14.150920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12779223-afc4-4a7a-b827-cc8b8d870c42 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:14.152748 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 320/635] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:14.158204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469b3a6-8e25-4301-b65e-53e72994ddd1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:14.158735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469b3a6-8e25-4301-b65e-53e72994ddd1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:14.159201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469b3a6-8e25-4301-b65e-53e72994ddd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:14.179911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9469b3a6-8e25-4301-b65e-53e72994ddd1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:14.181468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9469b3a6-8e25-4301-b65e-53e72994ddd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:14.269757 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 316/636] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:14.436151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e720-a6c1-4aa1-8f89-f2030c3c64de None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:14.436746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e720-a6c1-4aa1-8f89-f2030c3c64de None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:14.437241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e720-a6c1-4aa1-8f89-f2030c3c64de None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:14.439038 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 321/637] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:14.444781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf46d3-fa84-4d6c-939c-098625369137 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:14.445681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf46d3-fa84-4d6c-939c-098625369137 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:14.446152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf46d3-fa84-4d6c-939c-098625369137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:14.465435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-20cf46d3-fa84-4d6c-939c-098625369137 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:14.466996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-20cf46d3-fa84-4d6c-939c-098625369137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:14.538579 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 317/638] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:14.632799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061e720-a6c1-4aa1-8f89-f2030c3c64de None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:14.743408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061e720-a6c1-4aa1-8f89-f2030c3c64de None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:14.775331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ab177-c9e5-4005-a4cd-7dc56463119b None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:14.775657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ab177-c9e5-4005-a4cd-7dc56463119b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:14.775884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ab177-c9e5-4005-a4cd-7dc56463119b None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:14.777628 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 318/639] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:14.779700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24962c-31de-4c25-8419-bfe0b92ca526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:14.780295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24962c-31de-4c25-8419-bfe0b92ca526 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:14.780563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24962c-31de-4c25-8419-bfe0b92ca526 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:14.780829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24962c-31de-4c25-8419-bfe0b92ca526 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:14.781615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24962c-31de-4c25-8419-bfe0b92ca526 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:14.782640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3e73b-11a1-48a3-b674-da36a59ced5c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:14.783008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24962c-31de-4c25-8419-bfe0b92ca526 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:14.783354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3e73b-11a1-48a3-b674-da36a59ced5c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:14.783861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3e73b-11a1-48a3-b674-da36a59ced5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:14.802257 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 322/640] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:33:14 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:33:14.809562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bdb3e73b-11a1-48a3-b674-da36a59ced5c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:14.811203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bdb3e73b-11a1-48a3-b674-da36a59ced5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:14.856900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:14.869138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24962c-31de-4c25-8419-bfe0b92ca526 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:14.873398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:14.921804 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 319/641] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:14 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:14.993737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24962c-31de-4c25-8419-bfe0b92ca526 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:15.032101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bbb19-4929-479b-9643-d6fb661d10ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:15.032733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bbb19-4929-479b-9643-d6fb661d10ef None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:15.033007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bbb19-4929-479b-9643-d6fb661d10ef None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:15.033305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bbb19-4929-479b-9643-d6fb661d10ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:15.033882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bbb19-4929-479b-9643-d6fb661d10ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:15.035231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bbb19-4929-479b-9643-d6fb661d10ef None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-a24bbb19-4929-479b-9643-d6fb661d10ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:15.045386 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:15.046592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:15.047598 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 323/642] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:15.052786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3e73b-11a1-48a3-b674-da36a59ced5c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:15.172099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3e73b-11a1-48a3-b674-da36a59ced5c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:15.212925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932df819-a925-4cf8-b6a6-ce675d6087dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:15.213837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932df819-a925-4cf8-b6a6-ce675d6087dc None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:15.214301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932df819-a925-4cf8-b6a6-ce675d6087dc None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:15.214740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932df819-a925-4cf8-b6a6-ce675d6087dc None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:15.215700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932df819-a925-4cf8-b6a6-ce675d6087dc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:15.216182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932df819-a925-4cf8-b6a6-ce675d6087dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:15.217754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932df819-a925-4cf8-b6a6-ce675d6087dc None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:15.223153 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 320/643] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:33:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:15.229580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bbb19-4929-479b-9643-d6fb661d10ef None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:15.254834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932df819-a925-4cf8-b6a6-ce675d6087dc None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:15.287312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:15.313604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:15.361836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bbb19-4929-479b-9643-d6fb661d10ef None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:15.372182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932df819-a925-4cf8-b6a6-ce675d6087dc None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:15.412589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835ceeb-312d-44f0-8def-d9e3dc91d5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:15.412589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ceeb-312d-44f0-8def-d9e3dc91d5bb None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:15.412589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ceeb-312d-44f0-8def-d9e3dc91d5bb None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:15.412589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ceeb-312d-44f0-8def-d9e3dc91d5bb None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:15.413003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835ceeb-312d-44f0-8def-d9e3dc91d5bb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:15.413003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835ceeb-312d-44f0-8def-d9e3dc91d5bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:15.414066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835ceeb-312d-44f0-8def-d9e3dc91d5bb None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:15.416344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83617a-2c35-4310-a32b-b0b07d02ffc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:15.417228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83617a-2c35-4310-a32b-b0b07d02ffc0 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:15.417795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83617a-2c35-4310-a32b-b0b07d02ffc0 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:15.418260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83617a-2c35-4310-a32b-b0b07d02ffc0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:15.421112 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 324/644] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:33:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:15.439272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83617a-2c35-4310-a32b-b0b07d02ffc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-2c83617a-2c35-4310-a32b-b0b07d02ffc0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:15.444681 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:15.446128 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:15.446927 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 321/645] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Oct 15 20:33:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:15.454470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835ceeb-312d-44f0-8def-d9e3dc91d5bb None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:15.559494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791df5b-720a-4345-9b0f-2c31a48fb50d None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:15.560112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791df5b-720a-4345-9b0f-2c31a48fb50d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:15.560594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791df5b-720a-4345-9b0f-2c31a48fb50d None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:15.563077 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 322/646] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:15.569575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835ceeb-312d-44f0-8def-d9e3dc91d5bb None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:15.569932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463cd5c-4289-41ac-8c41-a5f9d6094aad None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:15.570215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463cd5c-4289-41ac-8c41-a5f9d6094aad None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:15.570491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463cd5c-4289-41ac-8c41-a5f9d6094aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:15.590111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1463cd5c-4289-41ac-8c41-a5f9d6094aad None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:15.591328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1463cd5c-4289-41ac-8c41-a5f9d6094aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:15.602853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:15.609674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0eed8-618c-43ad-a7ea-65ed7e1cd4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:15.610277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0eed8-618c-43ad-a7ea-65ed7e1cd4ed None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:15.610545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0eed8-618c-43ad-a7ea-65ed7e1cd4ed None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:15.610805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0eed8-618c-43ad-a7ea-65ed7e1cd4ed None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:15.611467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0eed8-618c-43ad-a7ea-65ed7e1cd4ed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:15.611827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0eed8-618c-43ad-a7ea-65ed7e1cd4ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:15.613311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0eed8-618c-43ad-a7ea-65ed7e1cd4ed None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:15.628156 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 325/647] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Oct 15 20:33:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:15.629855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0eed8-618c-43ad-a7ea-65ed7e1cd4ed None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:15.693060 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 323/648] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:15 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:15.744618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0eed8-618c-43ad-a7ea-65ed7e1cd4ed None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:15.781129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060211d-a401-4f0b-999b-3fec5d0df81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:15.782062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060211d-a401-4f0b-999b-3fec5d0df81e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:15.782336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060211d-a401-4f0b-999b-3fec5d0df81e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:15.782623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060211d-a401-4f0b-999b-3fec5d0df81e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:15.795351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060211d-a401-4f0b-999b-3fec5d0df81e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:15.796879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060211d-a401-4f0b-999b-3fec5d0df81e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-5060211d-a401-4f0b-999b-3fec5d0df81e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:15.813274 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:15.814231 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:15.815384 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:15.815384 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:15.815384 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:15.815384 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 326/649] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:15.823302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463cd5c-4289-41ac-8c41-a5f9d6094aad None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:15.846457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:15.950193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463cd5c-4289-41ac-8c41-a5f9d6094aad None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:15.990752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7ddc-7e38-477b-b67d-4a112ae4e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:15.991684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7ddc-7e38-477b-b67d-4a112ae4e4bb None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:15.992383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7ddc-7e38-477b-b67d-4a112ae4e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:15.992825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7ddc-7e38-477b-b67d-4a112ae4e4bb None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:15.993990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7ddc-7e38-477b-b67d-4a112ae4e4bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:15.994495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7ddc-7e38-477b-b67d-4a112ae4e4bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:15.996806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7ddc-7e38-477b-b67d-4a112ae4e4bb None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:16.009932 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 324/650] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:16.015616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060211d-a401-4f0b-999b-3fec5d0df81e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:16.043568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7ddc-7e38-477b-b67d-4a112ae4e4bb None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:16.070398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:16.139025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060211d-a401-4f0b-999b-3fec5d0df81e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:16.150644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:16.180771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7ddc-7e38-477b-b67d-4a112ae4e4bb None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:16.183016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0c5b1-f90e-4f2b-a2f5-5d387b65aaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:16.183731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0c5b1-f90e-4f2b-a2f5-5d387b65aaeb None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:16.184065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0c5b1-f90e-4f2b-a2f5-5d387b65aaeb None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:16.184322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0c5b1-f90e-4f2b-a2f5-5d387b65aaeb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:16.207237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0c5b1-f90e-4f2b-a2f5-5d387b65aaeb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:16.209612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0c5b1-f90e-4f2b-a2f5-5d387b65aaeb None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:16.225538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc17e7-656b-4e2b-8ff0-918c1af0cd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-fbe0c5b1-f90e-4f2b-a2f5-5d387b65aaeb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:33:16.226311 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:16.227218 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:16.228113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:16.228113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:16.228113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:33:16.228113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:33:16.228113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:33:16.228113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:33:16.228113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:33:16.228113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:33:16.228113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 20:33:16.228113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:16.229416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc17e7-656b-4e2b-8ff0-918c1af0cd21 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:16.229893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc17e7-656b-4e2b-8ff0-918c1af0cd21 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:16.230346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc17e7-656b-4e2b-8ff0-918c1af0cd21 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:16.231210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc17e7-656b-4e2b-8ff0-918c1af0cd21 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:16.233097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc17e7-656b-4e2b-8ff0-918c1af0cd21 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:16.233230 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 327/651] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Oct 15 20:33:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:16.234640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0c5b1-f90e-4f2b-a2f5-5d387b65aaeb None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-1bcc17e7-656b-4e2b-8ff0-918c1af0cd21 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:16.246033 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:16.247142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 20:33:16.247969 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:16.251827 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 325/652] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:33:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:16.257369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc17e7-656b-4e2b-8ff0-918c1af0cd21 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:16.351165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0c5b1-f90e-4f2b-a2f5-5d387b65aaeb None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:16.354811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:16.383395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc17e7-656b-4e2b-8ff0-918c1af0cd21 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:16.403579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c0613-cc52-4a51-9056-c69db8d0c262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:16.404187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c0613-cc52-4a51-9056-c69db8d0c262 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:16.404464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c0613-cc52-4a51-9056-c69db8d0c262 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:16.404732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c0613-cc52-4a51-9056-c69db8d0c262 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:16.405766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c0613-cc52-4a51-9056-c69db8d0c262 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:16.406066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c0613-cc52-4a51-9056-c69db8d0c262 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:16.408079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c0613-cc52-4a51-9056-c69db8d0c262 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:16.432735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bf0b7-1636-4517-b76d-b9ae265ef42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:16.433729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bf0b7-1636-4517-b76d-b9ae265ef42d None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:16.434217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bf0b7-1636-4517-b76d-b9ae265ef42d None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:16.434769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bf0b7-1636-4517-b76d-b9ae265ef42d None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:16.435610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bf0b7-1636-4517-b76d-b9ae265ef42d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:16.436100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bf0b7-1636-4517-b76d-b9ae265ef42d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:16.437708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bf0b7-1636-4517-b76d-b9ae265ef42d None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:16.439085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 328/653] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Oct 15 20:33:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:16.440694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c0613-cc52-4a51-9056-c69db8d0c262 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:16.445093 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 326/654] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:33:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:16.451610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bf0b7-1636-4517-b76d-b9ae265ef42d None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:16.466441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:16.568406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c0613-cc52-4a51-9056-c69db8d0c262 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:16.593015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bf0b7-1636-4517-b76d-b9ae265ef42d None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:16.608449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6313c6e-6807-4ac3-b454-3e3808c20536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:16.609130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6313c6e-6807-4ac3-b454-3e3808c20536 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:16.609402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6313c6e-6807-4ac3-b454-3e3808c20536 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:16.609665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6313c6e-6807-4ac3-b454-3e3808c20536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:16.622354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6313c6e-6807-4ac3-b454-3e3808c20536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:16.623778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6313c6e-6807-4ac3-b454-3e3808c20536 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:16.634011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1217884-fbe3-44c2-a81f-d9718bf48610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:16.634859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1217884-fbe3-44c2-a81f-d9718bf48610 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:16.635331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1217884-fbe3-44c2-a81f-d9718bf48610 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:16.635809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1217884-fbe3-44c2-a81f-d9718bf48610 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:16.636617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1217884-fbe3-44c2-a81f-d9718bf48610 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:16.637211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1217884-fbe3-44c2-a81f-d9718bf48610 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:16.638805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1217884-fbe3-44c2-a81f-d9718bf48610 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-e6313c6e-6807-4ac3-b454-3e3808c20536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:16.639337 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:16.640248 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:16.641169 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:16.641169 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:16.641169 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:16.645298 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 329/655] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:16.646299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6313c6e-6807-4ac3-b454-3e3808c20536 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:16.659433 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 327/656] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:33:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:16.677982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:16.764017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6313c6e-6807-4ac3-b454-3e3808c20536 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:16.802908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe3b24-3c56-4e92-abea-758715548641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:16.803566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe3b24-3c56-4e92-abea-758715548641 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:16.803834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe3b24-3c56-4e92-abea-758715548641 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:16.804089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe3b24-3c56-4e92-abea-758715548641 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:16.804718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe3b24-3c56-4e92-abea-758715548641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:16.805006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe3b24-3c56-4e92-abea-758715548641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:16.806458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe3b24-3c56-4e92-abea-758715548641 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:16.819860 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 330/657] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:16.825533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1217884-fbe3-44c2-a81f-d9718bf48610 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:16.907822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:16.942018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1217884-fbe3-44c2-a81f-d9718bf48610 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:16.980452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9b0a1-b6bd-4fc1-a552-00c885c69579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:16.981092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9b0a1-b6bd-4fc1-a552-00c885c69579 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:16.981366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9b0a1-b6bd-4fc1-a552-00c885c69579 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:16.981689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9b0a1-b6bd-4fc1-a552-00c885c69579 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:16.988894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9b0a1-b6bd-4fc1-a552-00c885c69579 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:16.990587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9b0a1-b6bd-4fc1-a552-00c885c69579 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-7bb9b0a1-b6bd-4fc1-a552-00c885c69579 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:33:17.002410 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:17.003316 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:17.004598 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:33:17.004598 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:33:17.004598 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:33:17.004598 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:33:17.004598 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:33:17.004598 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:33:17.004598 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:17.004598 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:17.004598 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 328/658] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:17.009751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe3b24-3c56-4e92-abea-758715548641 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:17.094156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:17.120950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe3b24-3c56-4e92-abea-758715548641 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:17.156307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f781ef-5333-4524-9ab1-e7c2870a46a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:17.156897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f781ef-5333-4524-9ab1-e7c2870a46a8 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:17.157195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f781ef-5333-4524-9ab1-e7c2870a46a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:17.157457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f781ef-5333-4524-9ab1-e7c2870a46a8 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:17.158276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f781ef-5333-4524-9ab1-e7c2870a46a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:17.158517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f781ef-5333-4524-9ab1-e7c2870a46a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:17.160011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f781ef-5333-4524-9ab1-e7c2870a46a8 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:17.173531 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 331/659] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:33:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:17.179514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9b0a1-b6bd-4fc1-a552-00c885c69579 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:17.291158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9b0a1-b6bd-4fc1-a552-00c885c69579 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:17.321285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:17.327682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0df53-d37d-4a0e-b79d-4e97b10f3d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:17.328329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0df53-d37d-4a0e-b79d-4e97b10f3d17 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:17.328599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0df53-d37d-4a0e-b79d-4e97b10f3d17 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:17.328866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0df53-d37d-4a0e-b79d-4e97b10f3d17 None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/7a9c40e53f72418381298190ca330e14/roles/e181e16789c24966b73c9d59d2075672` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:17.331021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0df53-d37d-4a0e-b79d-4e97b10f3d17 None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=7a9c40e53f72418381298190ca330e14, role_id=e181e16789c24966b73c9d59d2075672)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:17.332521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0df53-d37d-4a0e-b79d-4e97b10f3d17 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:17.347204 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 329/660] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:33:17 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/7a9c40e53f72418381298190ca330e14/roles/e181e16789c24966b73c9d59d2075672 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:33:17.512145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277941aa-fa9d-4731-b407-6d09aacb5016 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:17.512649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277941aa-fa9d-4731-b407-6d09aacb5016 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:17.512649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277941aa-fa9d-4731-b407-6d09aacb5016 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:17.514292 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 332/661] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:17.519944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659f619-cab7-46aa-9895-1b39aecf7b1a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:17.520458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659f619-cab7-46aa-9895-1b39aecf7b1a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:17.520458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659f619-cab7-46aa-9895-1b39aecf7b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:17.540858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f659f619-cab7-46aa-9895-1b39aecf7b1a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:17.542092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f659f619-cab7-46aa-9895-1b39aecf7b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:17.614491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 330/662] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:17 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:17.941969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277941aa-fa9d-4731-b407-6d09aacb5016 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:18.046596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277941aa-fa9d-4731-b407-6d09aacb5016 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:18.081394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d4c3-ba96-4b14-be5e-6478649205f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:18.082010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d4c3-ba96-4b14-be5e-6478649205f9 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:18.082277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d4c3-ba96-4b14-be5e-6478649205f9 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:18.082540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d4c3-ba96-4b14-be5e-6478649205f9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:18.083102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5d4c3-ba96-4b14-be5e-6478649205f9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:18.084467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5d4c3-ba96-4b14-be5e-6478649205f9 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:18.086272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-cef5d4c3-ba96-4b14-be5e-6478649205f9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:18.093886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:18.095020 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 20:33:18.096085 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:18.099026 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 333/663] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Oct 15 20:33:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:18.105543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d4c3-ba96-4b14-be5e-6478649205f9 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:18.105847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd031f-030f-4612-ab86-67067c88c007 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:18.106132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd031f-030f-4612-ab86-67067c88c007 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:18.106406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd031f-030f-4612-ab86-67067c88c007 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:18.107961 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 331/664] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:18.113282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1455855-3be0-4c10-a377-5b7a17bfce75 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:18.113563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1455855-3be0-4c10-a377-5b7a17bfce75 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:18.113834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1455855-3be0-4c10-a377-5b7a17bfce75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:18.134326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d1455855-3be0-4c10-a377-5b7a17bfce75 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:18.135629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d1455855-3be0-4c10-a377-5b7a17bfce75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:18.221262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d4c3-ba96-4b14-be5e-6478649205f9 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:18.233884 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 332/665] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:18 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:18.267244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb31711-69f6-48d1-9657-b82d864ec8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:18.267837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb31711-69f6-48d1-9657-b82d864ec8bc None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:18.268104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb31711-69f6-48d1-9657-b82d864ec8bc None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:18.268370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb31711-69f6-48d1-9657-b82d864ec8bc None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:18.268980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb31711-69f6-48d1-9657-b82d864ec8bc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:18.269302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb31711-69f6-48d1-9657-b82d864ec8bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:18.270649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb31711-69f6-48d1-9657-b82d864ec8bc None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:18.275502 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 334/666] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:18.281284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1455855-3be0-4c10-a377-5b7a17bfce75 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:18.335041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:18.388346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1455855-3be0-4c10-a377-5b7a17bfce75 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:18.424653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509279a9-f989-494c-8df0-a35045742d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:18.425273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509279a9-f989-494c-8df0-a35045742d39 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:18.425598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509279a9-f989-494c-8df0-a35045742d39 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:18.425862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509279a9-f989-494c-8df0-a35045742d39 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:18.426496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509279a9-f989-494c-8df0-a35045742d39 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:18.426816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509279a9-f989-494c-8df0-a35045742d39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:18.428279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509279a9-f989-494c-8df0-a35045742d39 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:18.433582 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 333/667] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:18.439460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb31711-69f6-48d1-9657-b82d864ec8bc None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:18.505674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:18.553976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb31711-69f6-48d1-9657-b82d864ec8bc None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:18.589297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2b99a-7911-4d7d-97f0-6d40415a3f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:18.589889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2b99a-7911-4d7d-97f0-6d40415a3f65 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:18.590157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2b99a-7911-4d7d-97f0-6d40415a3f65 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:18.590418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2b99a-7911-4d7d-97f0-6d40415a3f65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:18.591108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2b99a-7911-4d7d-97f0-6d40415a3f65 None admin] RBAC: Adding query filter params (interface=public, service_id=bd84ce95d7344730b919376c4c76ed4f, region_id=RegionOne) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:18.591402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2b99a-7911-4d7d-97f0-6d40415a3f65 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:18.592760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2b99a-7911-4d7d-97f0-6d40415a3f65 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:18.598270 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 335/668] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Oct 15 20:33:18 2022] GET /identity/v3/endpoints?service_id=bd84ce95d7344730b919376c4c76ed4f&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:18.603989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509279a9-f989-494c-8df0-a35045742d39 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:18.607177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2b99a-7911-4d7d-97f0-6d40415a3f65 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:18.724918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2b99a-7911-4d7d-97f0-6d40415a3f65 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:18.740487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509279a9-f989-494c-8df0-a35045742d39 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:18.752455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:18.761546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:18.763481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c51b46-0acc-4c14-a0b6-3ba5de80ac8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:18.764176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51b46-0acc-4c14-a0b6-3ba5de80ac8a None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:18.764516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51b46-0acc-4c14-a0b6-3ba5de80ac8a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:18.764861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51b46-0acc-4c14-a0b6-3ba5de80ac8a None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:18.765613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c51b46-0acc-4c14-a0b6-3ba5de80ac8a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:18.767435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c51b46-0acc-4c14-a0b6-3ba5de80ac8a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:18.785683 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 336/669] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:33:18 2022] POST /identity/v3/services => generated 241 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:33:18.803447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d30414-01af-4a6c-96cb-67e05eb33048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:18.804402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d30414-01af-4a6c-96cb-67e05eb33048 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:18.804887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d30414-01af-4a6c-96cb-67e05eb33048 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:18.805347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d30414-01af-4a6c-96cb-67e05eb33048 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:18.806410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d30414-01af-4a6c-96cb-67e05eb33048 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:18.808027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d30414-01af-4a6c-96cb-67e05eb33048 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:18.815747 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 334/670] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Oct 15 20:33:18 2022] GET /identity/v3/services => generated 1258 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:33:19.384355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26932d-10d1-42ab-aca1-dc636d9bfcae None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:19.385110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26932d-10d1-42ab-aca1-dc636d9bfcae None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:19.385595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26932d-10d1-42ab-aca1-dc636d9bfcae None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:19.387401 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 337/671] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:19.393139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8626-1591-47e6-bd7d-ab694b77ccba None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:19.393653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8626-1591-47e6-bd7d-ab694b77ccba None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:19.394138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8626-1591-47e6-bd7d-ab694b77ccba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:19.413902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9a9e8626-1591-47e6-bd7d-ab694b77ccba None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:19.415417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9a9e8626-1591-47e6-bd7d-ab694b77ccba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:19.501216 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 335/672] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:19 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:19.827042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26932d-10d1-42ab-aca1-dc636d9bfcae None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:19.879318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e21345-c0c8-464e-859c-92a3fccbfa2d None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:19.879835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e21345-c0c8-464e-859c-92a3fccbfa2d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:19.879835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e21345-c0c8-464e-859c-92a3fccbfa2d None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:19.881967 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 336/673] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:19.886676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60eb08-4a47-4753-87b1-12abe1cfe6e5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:19.886956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60eb08-4a47-4753-87b1-12abe1cfe6e5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:19.887229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60eb08-4a47-4753-87b1-12abe1cfe6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:19.906295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ba60eb08-4a47-4753-87b1-12abe1cfe6e5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:19.907585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ba60eb08-4a47-4753-87b1-12abe1cfe6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:19.939868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26932d-10d1-42ab-aca1-dc636d9bfcae None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:19.987035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad79b69-4b76-4790-9930-d678e2405e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:19.987640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad79b69-4b76-4790-9930-d678e2405e92 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:19.987914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad79b69-4b76-4790-9930-d678e2405e92 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:19.988243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad79b69-4b76-4790-9930-d678e2405e92 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:19.988908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad79b69-4b76-4790-9930-d678e2405e92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:19.990887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad79b69-4b76-4790-9930-d678e2405e92 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:19.994235 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 337/674] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:19 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:20.019591 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 338/675] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:33:19 2022] POST /identity/v3/services => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:20.342656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60eb08-4a47-4753-87b1-12abe1cfe6e5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:20.379766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:20.450166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60eb08-4a47-4753-87b1-12abe1cfe6e5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:20.492691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af7ce3-7c0c-4b36-b96d-d9d5fb782760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:20.493358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af7ce3-7c0c-4b36-b96d-d9d5fb782760 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:20.493628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af7ce3-7c0c-4b36-b96d-d9d5fb782760 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:20.493891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af7ce3-7c0c-4b36-b96d-d9d5fb782760 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:20.503977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af7ce3-7c0c-4b36-b96d-d9d5fb782760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-c8af7ce3-7c0c-4b36-b96d-d9d5fb782760 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:20.509385 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:20.510314 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:20.511160 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 338/676] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Oct 15 20:33:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:20.517929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad79b69-4b76-4790-9930-d678e2405e92 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:20.565921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:20.624713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad79b69-4b76-4790-9930-d678e2405e92 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:20.662913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859ac82-3000-4d75-92b0-3f31560e5454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:20.663507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859ac82-3000-4d75-92b0-3f31560e5454 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:20.663776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859ac82-3000-4d75-92b0-3f31560e5454 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:20.664037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859ac82-3000-4d75-92b0-3f31560e5454 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:20.664682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859ac82-3000-4d75-92b0-3f31560e5454 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:20.664985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859ac82-3000-4d75-92b0-3f31560e5454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:20.666408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859ac82-3000-4d75-92b0-3f31560e5454 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:20.674930 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 339/677] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Oct 15 20:33:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:20.680747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af7ce3-7c0c-4b36-b96d-d9d5fb782760 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:20.777520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:20.789269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af7ce3-7c0c-4b36-b96d-d9d5fb782760 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:20.825845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516af3-d087-4daf-830f-c3b2286b19c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:20.826511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516af3-d087-4daf-830f-c3b2286b19c8 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:20.826783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516af3-d087-4daf-830f-c3b2286b19c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:20.827096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516af3-d087-4daf-830f-c3b2286b19c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:20.837967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80516af3-d087-4daf-830f-c3b2286b19c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:20.839430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80516af3-d087-4daf-830f-c3b2286b19c8 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-80516af3-d087-4daf-830f-c3b2286b19c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:20.855114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:20.856085 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:20.856957 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:20.856957 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:20.856957 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:20.856957 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 339/678] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:20.863341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859ac82-3000-4d75-92b0-3f31560e5454 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:20.988300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859ac82-3000-4d75-92b0-3f31560e5454 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:20.999132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:21.026944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a7c57-1f7f-4bda-a0a7-30a08293deb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:21.027599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a7c57-1f7f-4bda-a0a7-30a08293deb8 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:21.027810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a7c57-1f7f-4bda-a0a7-30a08293deb8 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:21.028073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a7c57-1f7f-4bda-a0a7-30a08293deb8 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:21.028706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a7c57-1f7f-4bda-a0a7-30a08293deb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:21.028997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a7c57-1f7f-4bda-a0a7-30a08293deb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:21.030490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a7c57-1f7f-4bda-a0a7-30a08293deb8 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:21.044407 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 340/679] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:21.049766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516af3-d087-4daf-830f-c3b2286b19c8 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:21.156456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516af3-d087-4daf-830f-c3b2286b19c8 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:21.202457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788f28f-0b34-45c1-a517-0c180c5fa62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:21.203085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788f28f-0b34-45c1-a517-0c180c5fa62f None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:21.203621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788f28f-0b34-45c1-a517-0c180c5fa62f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:21.203887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788f28f-0b34-45c1-a517-0c180c5fa62f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:21.212910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788f28f-0b34-45c1-a517-0c180c5fa62f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:21.214424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788f28f-0b34-45c1-a517-0c180c5fa62f None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-7788f28f-0b34-45c1-a517-0c180c5fa62f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:33:21.227864 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:21.228841 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:21.229788 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 340/680] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Oct 15 20:33:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:21.236420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a7c57-1f7f-4bda-a0a7-30a08293deb8 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:21.344456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a7c57-1f7f-4bda-a0a7-30a08293deb8 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:21.382494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba6612-5373-4d62-964a-71ea55138a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:21.383473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba6612-5373-4d62-964a-71ea55138a02 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:21.383955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba6612-5373-4d62-964a-71ea55138a02 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:21.384441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba6612-5373-4d62-964a-71ea55138a02 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:21.385392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba6612-5373-4d62-964a-71ea55138a02 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:21.385891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba6612-5373-4d62-964a-71ea55138a02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:21.387545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba6612-5373-4d62-964a-71ea55138a02 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:21.402216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff22a8-9455-43c7-a837-257b139292d6 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:21.402830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff22a8-9455-43c7-a837-257b139292d6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:21.403296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff22a8-9455-43c7-a837-257b139292d6 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:21.404830 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 341/681] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:21.410130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4607496-16b6-4289-b820-844942e48db9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:21.410403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4607496-16b6-4289-b820-844942e48db9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:21.410668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4607496-16b6-4289-b820-844942e48db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:21.412022 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 341/682] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Oct 15 20:33:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:21.417415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:21.419025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba6612-5373-4d62-964a-71ea55138a02 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:21.430234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d4607496-16b6-4289-b820-844942e48db9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:21.431405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4607496-16b6-4289-b820-844942e48db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:21.450260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:21.520925 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 342/683] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:21.522696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab1eeb-0f9c-4c87-8e94-767b640bf99d None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:21.523034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab1eeb-0f9c-4c87-8e94-767b640bf99d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:21.523307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab1eeb-0f9c-4c87-8e94-767b640bf99d None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:21.529791 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 343/684] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:21.530283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e299bc-6e5e-41dc-8732-a7cba613a7c1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:21.530557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e299bc-6e5e-41dc-8732-a7cba613a7c1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:21.530819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e299bc-6e5e-41dc-8732-a7cba613a7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:21.549653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-76e299bc-6e5e-41dc-8732-a7cba613a7c1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:21.550846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-76e299bc-6e5e-41dc-8732-a7cba613a7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:21.570465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba6612-5373-4d62-964a-71ea55138a02 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:21.629787 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 344/685] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:21.735414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4181f-4c82-4db9-b913-070b0c5e81ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:21.736369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4181f-4c82-4db9-b913-070b0c5e81ea None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:21.736840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4181f-4c82-4db9-b913-070b0c5e81ea None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:21.737347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4181f-4c82-4db9-b913-070b0c5e81ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:21.809687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4181f-4c82-4db9-b913-070b0c5e81ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:21.811453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4181f-4c82-4db9-b913-070b0c5e81ea None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:21.838087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:21.878875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e299bc-6e5e-41dc-8732-a7cba613a7c1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:21.918942 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 342/686] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:33:21 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 20:33:21.924381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4181f-4c82-4db9-b913-070b0c5e81ea None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:21.957877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:22.004159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e299bc-6e5e-41dc-8732-a7cba613a7c1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.032440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4181f-4c82-4db9-b913-070b0c5e81ea None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.041651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7449a8c-40eb-42ca-9cd8-65485161d59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.042275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7449a8c-40eb-42ca-9cd8-65485161d59c None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.042547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7449a8c-40eb-42ca-9cd8-65485161d59c None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.042839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7449a8c-40eb-42ca-9cd8-65485161d59c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.043405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7449a8c-40eb-42ca-9cd8-65485161d59c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.044774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7449a8c-40eb-42ca-9cd8-65485161d59c None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-a7449a8c-40eb-42ca-9cd8-65485161d59c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:22.056344 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:22.057814 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:22.058672 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 345/687] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:22.059486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7449a8c-40eb-42ca-9cd8-65485161d59c None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.067160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:22.076659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097030a-78b0-44a9-94c3-6355e729dd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.077319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097030a-78b0-44a9-94c3-6355e729dd4d None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.077593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097030a-78b0-44a9-94c3-6355e729dd4d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.077886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097030a-78b0-44a9-94c3-6355e729dd4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.088561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097030a-78b0-44a9-94c3-6355e729dd4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.090497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097030a-78b0-44a9-94c3-6355e729dd4d None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-3097030a-78b0-44a9-94c3-6355e729dd4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:22.107930 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:22.108863 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:22.109759 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:22.109759 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:22.109759 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:22.109759 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 343/688] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:22.110416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097030a-78b0-44a9-94c3-6355e729dd4d None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.170353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7449a8c-40eb-42ca-9cd8-65485161d59c None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.204957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:22.216992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98201b2-78af-4d3e-ade3-d35a4c95d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.217655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98201b2-78af-4d3e-ade3-d35a4c95d53f None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.217927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98201b2-78af-4d3e-ade3-d35a4c95d53f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.218214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98201b2-78af-4d3e-ade3-d35a4c95d53f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.218797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98201b2-78af-4d3e-ade3-d35a4c95d53f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.220210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98201b2-78af-4d3e-ade3-d35a4c95d53f None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.228253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097030a-78b0-44a9-94c3-6355e729dd4d None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-d98201b2-78af-4d3e-ade3-d35a4c95d53f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:22.230133 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:22.231235 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:22.234079 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 346/689] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Oct 15 20:33:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:22.234860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98201b2-78af-4d3e-ade3-d35a4c95d53f None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.272372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c793e4d-4ecd-47eb-b399-ed508090f26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.272969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c793e4d-4ecd-47eb-b399-ed508090f26f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.273271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c793e4d-4ecd-47eb-b399-ed508090f26f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.273535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c793e4d-4ecd-47eb-b399-ed508090f26f None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.274151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c793e4d-4ecd-47eb-b399-ed508090f26f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:22.274444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c793e4d-4ecd-47eb-b399-ed508090f26f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.275780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c793e4d-4ecd-47eb-b399-ed508090f26f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.280846 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 344/690] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:33:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:22.281673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c793e4d-4ecd-47eb-b399-ed508090f26f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.327444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:22.341027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98201b2-78af-4d3e-ade3-d35a4c95d53f None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.375513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95bcf7-029a-4902-b952-043c2543cb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.376119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95bcf7-029a-4902-b952-043c2543cb13 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.376386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95bcf7-029a-4902-b952-043c2543cb13 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.376691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95bcf7-029a-4902-b952-043c2543cb13 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.377361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95bcf7-029a-4902-b952-043c2543cb13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:22.377647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95bcf7-029a-4902-b952-043c2543cb13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.379020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95bcf7-029a-4902-b952-043c2543cb13 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.390794 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 347/691] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:22.391596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95bcf7-029a-4902-b952-043c2543cb13 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.392734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c793e4d-4ecd-47eb-b399-ed508090f26f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.431074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97103274-0667-4e10-b0ec-cf1d1641f560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.431675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97103274-0667-4e10-b0ec-cf1d1641f560 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.431948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97103274-0667-4e10-b0ec-cf1d1641f560 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.432208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97103274-0667-4e10-b0ec-cf1d1641f560 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.432821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97103274-0667-4e10-b0ec-cf1d1641f560 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:22.433148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97103274-0667-4e10-b0ec-cf1d1641f560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.434492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97103274-0667-4e10-b0ec-cf1d1641f560 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.444610 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 345/692] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:22.444610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97103274-0667-4e10-b0ec-cf1d1641f560 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.453888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:22.512448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95bcf7-029a-4902-b952-043c2543cb13 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.547751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ef5e2-fe78-4bf0-a00c-7487ddcf3396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.548351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ef5e2-fe78-4bf0-a00c-7487ddcf3396 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.548624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ef5e2-fe78-4bf0-a00c-7487ddcf3396 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.548886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ef5e2-fe78-4bf0-a00c-7487ddcf3396 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.549562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ef5e2-fe78-4bf0-a00c-7487ddcf3396 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:22.549856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ef5e2-fe78-4bf0-a00c-7487ddcf3396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.551225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ef5e2-fe78-4bf0-a00c-7487ddcf3396 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.553553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97103274-0667-4e10-b0ec-cf1d1641f560 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.562146 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 348/693] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:33:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:22.562146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ef5e2-fe78-4bf0-a00c-7487ddcf3396 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.562363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:22.591746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9010baa-f7c2-49d3-8167-8a4b218f1c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.592349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9010baa-f7c2-49d3-8167-8a4b218f1c38 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.592614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9010baa-f7c2-49d3-8167-8a4b218f1c38 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.592873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9010baa-f7c2-49d3-8167-8a4b218f1c38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.600167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9010baa-f7c2-49d3-8167-8a4b218f1c38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.602148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9010baa-f7c2-49d3-8167-8a4b218f1c38 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-d9010baa-f7c2-49d3-8167-8a4b218f1c38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:33:22.612674 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:22.613567 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:22.614498 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:33:22.614498 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:33:22.614498 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:33:22.614498 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:33:22.614498 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:33:22.614498 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:33:22.614498 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:22.614498 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:22.614498 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 346/694] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:22.615917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9010baa-f7c2-49d3-8167-8a4b218f1c38 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.674490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ef5e2-fe78-4bf0-a00c-7487ddcf3396 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.688677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:22.709842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9856c3e-a4e9-4454-9baa-43ee4e67534b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.710466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9856c3e-a4e9-4454-9baa-43ee4e67534b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.710738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9856c3e-a4e9-4454-9baa-43ee4e67534b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.711000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9856c3e-a4e9-4454-9baa-43ee4e67534b None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.711692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9856c3e-a4e9-4454-9baa-43ee4e67534b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:22.711986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9856c3e-a4e9-4454-9baa-43ee4e67534b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.713387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9856c3e-a4e9-4454-9baa-43ee4e67534b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.717872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9010baa-f7c2-49d3-8167-8a4b218f1c38 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.717996 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 349/695] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:22.718744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9856c3e-a4e9-4454-9baa-43ee4e67534b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.751457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffef3e4-9f62-44ee-85a5-ed14e3f84dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.752047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffef3e4-9f62-44ee-85a5-ed14e3f84dd2 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.752312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffef3e4-9f62-44ee-85a5-ed14e3f84dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.752571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffef3e4-9f62-44ee-85a5-ed14e3f84dd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.753273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffef3e4-9f62-44ee-85a5-ed14e3f84dd2 None admin] RBAC: Adding query filter params (interface=public, service_id=6761ee8ed7d8450599960caddd77c3f4, region_id=RegionOne) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:22.753572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffef3e4-9f62-44ee-85a5-ed14e3f84dd2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.754897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffef3e4-9f62-44ee-85a5-ed14e3f84dd2 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.760399 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 347/696] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Oct 15 20:33:22 2022] GET /identity/v3/endpoints?service_id=6761ee8ed7d8450599960caddd77c3f4&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:22.761564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d51aef-a9bd-4569-8de5-557b229d94b8 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.761834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d51aef-a9bd-4569-8de5-557b229d94b8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.762095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d51aef-a9bd-4569-8de5-557b229d94b8 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.763431 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 348/697] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:22.765888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d51aef-a9bd-4569-8de5-557b229d94b8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.796797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:22.821743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9856c3e-a4e9-4454-9baa-43ee4e67534b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.855338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c98002-3c4d-4771-a4a7-39f2e892177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.855933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c98002-3c4d-4771-a4a7-39f2e892177e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.856209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c98002-3c4d-4771-a4a7-39f2e892177e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.856478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c98002-3c4d-4771-a4a7-39f2e892177e None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.857480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c98002-3c4d-4771-a4a7-39f2e892177e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:22.857787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c98002-3c4d-4771-a4a7-39f2e892177e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.859317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c98002-3c4d-4771-a4a7-39f2e892177e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.870079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d51aef-a9bd-4569-8de5-557b229d94b8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:22.871736 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 350/698] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:33:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:22.872539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c98002-3c4d-4771-a4a7-39f2e892177e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:22.909493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f811ef1-845d-4396-be67-5d575c3ac0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:22.910075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f811ef1-845d-4396-be67-5d575c3ac0c2 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.910344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f811ef1-845d-4396-be67-5d575c3ac0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.910606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f811ef1-845d-4396-be67-5d575c3ac0c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.911274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f811ef1-845d-4396-be67-5d575c3ac0c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:22.911939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:22.912696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f811ef1-845d-4396-be67-5d575c3ac0c2 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:22.933828 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 349/699] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:33:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:22.935090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb600b-dec2-4b42-a28b-7960a14225d9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:22.935375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb600b-dec2-4b42-a28b-7960a14225d9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:22.935642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb600b-dec2-4b42-a28b-7960a14225d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:22.955677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-61bb600b-dec2-4b42-a28b-7960a14225d9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:22.956816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-61bb600b-dec2-4b42-a28b-7960a14225d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:22.992196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c98002-3c4d-4771-a4a7-39f2e892177e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:23.038299 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 350/700] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:23.039451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb600b-dec2-4b42-a28b-7960a14225d9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:23.050271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac15250-9788-4094-aa71-d1b1b6de2d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:23.051311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac15250-9788-4094-aa71-d1b1b6de2d3a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:23.051656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac15250-9788-4094-aa71-d1b1b6de2d3a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:23.052578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac15250-9788-4094-aa71-d1b1b6de2d3a None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:23.053316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac15250-9788-4094-aa71-d1b1b6de2d3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:23.054884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac15250-9788-4094-aa71-d1b1b6de2d3a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:23.064745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:23.069744 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 351/701] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Oct 15 20:33:22 2022] GET /identity/v3/services => generated 1497 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:33:23.179000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb600b-dec2-4b42-a28b-7960a14225d9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:23.226639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae42e80-0ad6-41cc-a45b-09d84413e150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:23.227148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae42e80-0ad6-41cc-a45b-09d84413e150 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:23.227414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae42e80-0ad6-41cc-a45b-09d84413e150 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:23.227740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae42e80-0ad6-41cc-a45b-09d84413e150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:23.228471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae42e80-0ad6-41cc-a45b-09d84413e150 None admin] RBAC: Adding query filter params (role.id=e181e16789c24966b73c9d59d2075672, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=7a9c40e53f72418381298190ca330e14) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:23.228766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae42e80-0ad6-41cc-a45b-09d84413e150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:23.230296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae42e80-0ad6-41cc-a45b-09d84413e150 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:23.237466 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 351/702] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:33:23 2022] GET /identity/v3/role_assignments?user.id=7a9c40e53f72418381298190ca330e14&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=e181e16789c24966b73c9d59d2075672 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:23.415890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac15250-9788-4094-aa71-d1b1b6de2d3a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:23.525802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:23.529636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac15250-9788-4094-aa71-d1b1b6de2d3a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:23.564155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e512ba-d91f-4bd5-bca1-3162ea3cab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:23.564776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e512ba-d91f-4bd5-bca1-3162ea3cab14 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:23.565074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e512ba-d91f-4bd5-bca1-3162ea3cab14 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:23.565337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e512ba-d91f-4bd5-bca1-3162ea3cab14 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:23.565902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e512ba-d91f-4bd5-bca1-3162ea3cab14 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:23.567266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e512ba-d91f-4bd5-bca1-3162ea3cab14 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-39e512ba-d91f-4bd5-bca1-3162ea3cab14 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:23.576671 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:23.577694 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:23.578555 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 352/703] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:33:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:23.586850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae42e80-0ad6-41cc-a45b-09d84413e150 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:23.673067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:23.699606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae42e80-0ad6-41cc-a45b-09d84413e150 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:23.735529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307f7ad-3d58-4db2-ad21-045724b17f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:23.736468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307f7ad-3d58-4db2-ad21-045724b17f03 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:23.736976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307f7ad-3d58-4db2-ad21-045724b17f03 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:23.737462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307f7ad-3d58-4db2-ad21-045724b17f03 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:23.738302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307f7ad-3d58-4db2-ad21-045724b17f03 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:23.738808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307f7ad-3d58-4db2-ad21-045724b17f03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:23.741439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307f7ad-3d58-4db2-ad21-045724b17f03 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:23.747140 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 352/704] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:33:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:23.753016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e512ba-d91f-4bd5-bca1-3162ea3cab14 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:23.869830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e512ba-d91f-4bd5-bca1-3162ea3cab14 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:23.906324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfea3e-82b5-487e-97ec-08d4859f1878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:23.906924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfea3e-82b5-487e-97ec-08d4859f1878 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:23.907195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfea3e-82b5-487e-97ec-08d4859f1878 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:23.907462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfea3e-82b5-487e-97ec-08d4859f1878 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:23.908088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfea3e-82b5-487e-97ec-08d4859f1878 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:23.908384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfea3e-82b5-487e-97ec-08d4859f1878 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:23.909792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfea3e-82b5-487e-97ec-08d4859f1878 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:23.915506 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 353/705] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:33:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:23.921245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307f7ad-3d58-4db2-ad21-045724b17f03 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:24.024861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307f7ad-3d58-4db2-ad21-045724b17f03 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:24.060705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eed87a-bba7-472a-b666-8035ec27f768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:24.062117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eed87a-bba7-472a-b666-8035ec27f768 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:24.062615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eed87a-bba7-472a-b666-8035ec27f768 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:24.063074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eed87a-bba7-472a-b666-8035ec27f768 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:24.063959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eed87a-bba7-472a-b666-8035ec27f768 None admin] RBAC: Adding query filter params (interface=public, service_id=5df0ac02cd61453381f1cc10275bf0e1, region_id=RegionOne) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:24.064467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eed87a-bba7-472a-b666-8035ec27f768 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:24.066026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eed87a-bba7-472a-b666-8035ec27f768 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:24.072625 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 353/706] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Oct 15 20:33:23 2022] GET /identity/v3/endpoints?service_id=5df0ac02cd61453381f1cc10275bf0e1&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:24.078533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfea3e-82b5-487e-97ec-08d4859f1878 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:24.108373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:24.186842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfea3e-82b5-487e-97ec-08d4859f1878 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:24.234956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d799a-d82f-45a1-badc-367bdfadb480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:24.235558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d799a-d82f-45a1-badc-367bdfadb480 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:24.235831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d799a-d82f-45a1-badc-367bdfadb480 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:24.236093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d799a-d82f-45a1-badc-367bdfadb480 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:24.236741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d799a-d82f-45a1-badc-367bdfadb480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:24.238144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d799a-d82f-45a1-badc-367bdfadb480 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:24.245106 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 354/707] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Oct 15 20:33:24 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:33:25.500723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1905b48-2416-4ca4-a12e-ea9a0214c9bf None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:25.501277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1905b48-2416-4ca4-a12e-ea9a0214c9bf None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:25.501277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1905b48-2416-4ca4-a12e-ea9a0214c9bf None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:25.502784 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 354/708] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:25.508323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527b5f-622b-4bc3-a85b-b7eede2e49d7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:25.508796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527b5f-622b-4bc3-a85b-b7eede2e49d7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:25.508872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527b5f-622b-4bc3-a85b-b7eede2e49d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:25.527363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b2527b5f-622b-4bc3-a85b-b7eede2e49d7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:25.529007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b2527b5f-622b-4bc3-a85b-b7eede2e49d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:25.602501 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 355/709] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:25.676152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6067b8-4334-408f-b02f-b94f2f756019 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:25.677544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6067b8-4334-408f-b02f-b94f2f756019 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:25.678068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6067b8-4334-408f-b02f-b94f2f756019 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:25.679959 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 355/710] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:25.685700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babca92a-3a5f-4cd8-95e1-2a7e26459ded None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:25.686225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babca92a-3a5f-4cd8-95e1-2a7e26459ded None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:25.686720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babca92a-3a5f-4cd8-95e1-2a7e26459ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:25.706945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-babca92a-3a5f-4cd8-95e1-2a7e26459ded None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:25.708824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-babca92a-3a5f-4cd8-95e1-2a7e26459ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:25.725927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:25.748556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab70f0-7983-4d27-87e1-a8168284039c None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:25.748893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab70f0-7983-4d27-87e1-a8168284039c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:25.749125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab70f0-7983-4d27-87e1-a8168284039c None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:25.750672 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 356/711] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:25.755911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122dad6-1bf8-42ca-8135-5f81f4771948 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:25.756152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122dad6-1bf8-42ca-8135-5f81f4771948 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:25.756388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122dad6-1bf8-42ca-8135-5f81f4771948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:25.763599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:25.779617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a122dad6-1bf8-42ca-8135-5f81f4771948 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:25.780717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a122dad6-1bf8-42ca-8135-5f81f4771948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:25.801085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 356/712] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:25.860372 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 357/713] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:25.932178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babca92a-3a5f-4cd8-95e1-2a7e26459ded None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:26.039463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babca92a-3a5f-4cd8-95e1-2a7e26459ded None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:26.078337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3378e-585e-4755-ba71-8667a11367dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:26.079081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3378e-585e-4755-ba71-8667a11367dd None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.079323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3378e-585e-4755-ba71-8667a11367dd None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.079593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3378e-585e-4755-ba71-8667a11367dd None admin] PATH_INFO: `/v3/services/compute` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.080207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3378e-585e-4755-ba71-8667a11367dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:26.081624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3378e-585e-4755-ba71-8667a11367dd None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-68f3378e-585e-4755-ba71-8667a11367dd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:26.091697 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:26.092663 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:26.093641 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 357/714] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:26.101205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122dad6-1bf8-42ca-8135-5f81f4771948 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:26.174117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3378e-585e-4755-ba71-8667a11367dd None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:26.202280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:26.215531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122dad6-1bf8-42ca-8135-5f81f4771948 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:26.252802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0aff0-ee52-4e2c-9d9a-a42496660b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:26.253499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0aff0-ee52-4e2c-9d9a-a42496660b6a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.253975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0aff0-ee52-4e2c-9d9a-a42496660b6a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.254103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0aff0-ee52-4e2c-9d9a-a42496660b6a None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.254636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0aff0-ee52-4e2c-9d9a-a42496660b6a None admin] RBAC: Adding query filter params (name=compute) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:26.255511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0aff0-ee52-4e2c-9d9a-a42496660b6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:26.256962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0aff0-ee52-4e2c-9d9a-a42496660b6a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:26.258277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:26.262896 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 358/715] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:26.264560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0aff0-ee52-4e2c-9d9a-a42496660b6a None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:26.296087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3378e-585e-4755-ba71-8667a11367dd None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:26.331019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1585de9-f06d-4cb4-adc4-f04c805bc7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:26.331943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1585de9-f06d-4cb4-adc4-f04c805bc7dd None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.332219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1585de9-f06d-4cb4-adc4-f04c805bc7dd None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.332482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1585de9-f06d-4cb4-adc4-f04c805bc7dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.333163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1585de9-f06d-4cb4-adc4-f04c805bc7dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:26.334542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1585de9-f06d-4cb4-adc4-f04c805bc7dd None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-e1585de9-f06d-4cb4-adc4-f04c805bc7dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:26.343328 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:26.344381 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:26.345546 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 358/716] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:26.346316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1585de9-f06d-4cb4-adc4-f04c805bc7dd None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:26.381684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0aff0-ee52-4e2c-9d9a-a42496660b6a None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:26.424836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9a9ab-46c1-4994-bb60-39460efaca78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:26.425651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9a9ab-46c1-4994-bb60-39460efaca78 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.425913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9a9ab-46c1-4994-bb60-39460efaca78 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.426239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9a9ab-46c1-4994-bb60-39460efaca78 None admin] PATH_INFO: `/v3/services/image` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.426929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9a9ab-46c1-4994-bb60-39460efaca78 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:26.428419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9a9ab-46c1-4994-bb60-39460efaca78 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:26.436420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-f2a9a9ab-46c1-4994-bb60-39460efaca78 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:26.437840 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:26.439502 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:26.441660 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 359/717] 10.222.0.16 () {56 vars in 1105 bytes} [Sat Oct 15 20:33:26 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:26.442968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9a9ab-46c1-4994-bb60-39460efaca78 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:26.465624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1585de9-f06d-4cb4-adc4-f04c805bc7dd None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:26.504107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84b64c-8525-4be5-94e0-1637f47a6baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:26.505255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84b64c-8525-4be5-94e0-1637f47a6baa None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.505564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84b64c-8525-4be5-94e0-1637f47a6baa None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.505826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84b64c-8525-4be5-94e0-1637f47a6baa None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.506550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84b64c-8525-4be5-94e0-1637f47a6baa None admin] RBAC: Adding query filter params (type=compute) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:26.506847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84b64c-8525-4be5-94e0-1637f47a6baa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:26.508283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84b64c-8525-4be5-94e0-1637f47a6baa None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:26.521138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:26.521822 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 359/718] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:26.522539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84b64c-8525-4be5-94e0-1637f47a6baa None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:26.572297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9a9ab-46c1-4994-bb60-39460efaca78 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:26.615370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51d096-0b75-4c5a-90c1-dc3b8e6b6c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:26.616538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51d096-0b75-4c5a-90c1-dc3b8e6b6c0e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.617417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51d096-0b75-4c5a-90c1-dc3b8e6b6c0e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.617691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51d096-0b75-4c5a-90c1-dc3b8e6b6c0e None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.618352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51d096-0b75-4c5a-90c1-dc3b8e6b6c0e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:26.619352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51d096-0b75-4c5a-90c1-dc3b8e6b6c0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:26.620754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51d096-0b75-4c5a-90c1-dc3b8e6b6c0e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:26.627738 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 360/719] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:33:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:26.634263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51d096-0b75-4c5a-90c1-dc3b8e6b6c0e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:26.664717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84b64c-8525-4be5-94e0-1637f47a6baa None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:26.708152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c1c28-2e4a-4e31-938d-bbf636483497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:26.709242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c1c28-2e4a-4e31-938d-bbf636483497 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.709729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c1c28-2e4a-4e31-938d-bbf636483497 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.710222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c1c28-2e4a-4e31-938d-bbf636483497 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.711059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c1c28-2e4a-4e31-938d-bbf636483497 None admin] RBAC: Adding query filter params (name=image) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:26.711570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c1c28-2e4a-4e31-938d-bbf636483497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:26.713210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c1c28-2e4a-4e31-938d-bbf636483497 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:26.719318 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 360/720] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:26.729086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c1c28-2e4a-4e31-938d-bbf636483497 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:26.735600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:26.777072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51d096-0b75-4c5a-90c1-dc3b8e6b6c0e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:26.808802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:26.822637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a4483-bb4a-46ad-82af-4861d4e7e418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:26.823311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a4483-bb4a-46ad-82af-4861d4e7e418 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.823673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a4483-bb4a-46ad-82af-4861d4e7e418 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.824039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a4483-bb4a-46ad-82af-4861d4e7e418 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.824843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a4483-bb4a-46ad-82af-4861d4e7e418 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:26.825238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a4483-bb4a-46ad-82af-4861d4e7e418 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:26.827206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a4483-bb4a-46ad-82af-4861d4e7e418 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:26.835026 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 361/721] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:33:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:26.839808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2097c-e642-4ff2-9695-b83e5e3844d6 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.840099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2097c-e642-4ff2-9695-b83e5e3844d6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.840367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2097c-e642-4ff2-9695-b83e5e3844d6 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.841796 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 362/722] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:26.842322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2097c-e642-4ff2-9695-b83e5e3844d6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:26.858400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c1c28-2e4a-4e31-938d-bbf636483497 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:26.894544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58ca40-5877-4b9b-8641-21a2c6c8ad80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:26.895189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58ca40-5877-4b9b-8641-21a2c6c8ad80 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.895906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58ca40-5877-4b9b-8641-21a2c6c8ad80 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.896172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58ca40-5877-4b9b-8641-21a2c6c8ad80 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.896784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58ca40-5877-4b9b-8641-21a2c6c8ad80 None admin] RBAC: Adding query filter params (type=image) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:26.897104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58ca40-5877-4b9b-8641-21a2c6c8ad80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:26.898676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58ca40-5877-4b9b-8641-21a2c6c8ad80 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:26.913064 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 361/723] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:26 2022] GET /identity/v3/services?type=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:26.913064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39180b-51e2-4cbf-a419-3b960f75bc06 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:26.913064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39180b-51e2-4cbf-a419-3b960f75bc06 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:26.913064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39180b-51e2-4cbf-a419-3b960f75bc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:26.931137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0e39180b-51e2-4cbf-a419-3b960f75bc06 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:26.932548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0e39180b-51e2-4cbf-a419-3b960f75bc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:26.972102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2097c-e642-4ff2-9695-b83e5e3844d6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:27.014788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:27.029072 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 362/724] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:27.029308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79171d-db9f-41fd-9b30-208922fbac8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:27.029629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79171d-db9f-41fd-9b30-208922fbac8d None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:27.029983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79171d-db9f-41fd-9b30-208922fbac8d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:27.030421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79171d-db9f-41fd-9b30-208922fbac8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:27.031154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79171d-db9f-41fd-9b30-208922fbac8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:27.032631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79171d-db9f-41fd-9b30-208922fbac8d None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:27.059473 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 363/725] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:33:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:27.368111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39180b-51e2-4cbf-a419-3b960f75bc06 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:27.476081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39180b-51e2-4cbf-a419-3b960f75bc06 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:27.519699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dabb0-fc83-4f95-9816-70e590ebefea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:27.521571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dabb0-fc83-4f95-9816-70e590ebefea None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:27.521869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dabb0-fc83-4f95-9816-70e590ebefea None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:27.522390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dabb0-fc83-4f95-9816-70e590ebefea None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:27.522698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dabb0-fc83-4f95-9816-70e590ebefea None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:27.524075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dabb0-fc83-4f95-9816-70e590ebefea None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-1a1dabb0-fc83-4f95-9816-70e590ebefea None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:27.533163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:27.534200 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:27.535686 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 363/726] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:33:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:27.541224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79171d-db9f-41fd-9b30-208922fbac8d None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:27.597429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:27.658550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79171d-db9f-41fd-9b30-208922fbac8d None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:27.698690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3140e-dc94-4b38-844e-68c13b0e59ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:27.699613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3140e-dc94-4b38-844e-68c13b0e59ce None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:27.700161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3140e-dc94-4b38-844e-68c13b0e59ce None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:27.700599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3140e-dc94-4b38-844e-68c13b0e59ce None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:27.701432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3140e-dc94-4b38-844e-68c13b0e59ce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:27.701913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3140e-dc94-4b38-844e-68c13b0e59ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:27.703428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3140e-dc94-4b38-844e-68c13b0e59ce None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:27.709311 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 364/727] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:33:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:27.714851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dabb0-fc83-4f95-9816-70e590ebefea None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:27.781604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:27.825519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dabb0-fc83-4f95-9816-70e590ebefea None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:27.860583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f299dd-41b0-4889-8141-06da9376a442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:27.861317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f299dd-41b0-4889-8141-06da9376a442 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:27.861487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f299dd-41b0-4889-8141-06da9376a442 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:27.861741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f299dd-41b0-4889-8141-06da9376a442 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:27.862366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f299dd-41b0-4889-8141-06da9376a442 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:27.862655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f299dd-41b0-4889-8141-06da9376a442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:27.864037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f299dd-41b0-4889-8141-06da9376a442 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:27.870065 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 364/728] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Oct 15 20:33:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:27.878194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3140e-dc94-4b38-844e-68c13b0e59ce None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:27.989125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3140e-dc94-4b38-844e-68c13b0e59ce None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:28.000074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:28.024799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5a6bf-606a-4267-b6d8-f724ab3a0ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:28.025418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5a6bf-606a-4267-b6d8-f724ab3a0ec8 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:28.025686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5a6bf-606a-4267-b6d8-f724ab3a0ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:28.025945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5a6bf-606a-4267-b6d8-f724ab3a0ec8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:28.026586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5a6bf-606a-4267-b6d8-f724ab3a0ec8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:28.027919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5a6bf-606a-4267-b6d8-f724ab3a0ec8 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:28.050546 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 365/729] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:33:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:29.066217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911cea7-ceb8-41c4-812d-bd67f3dba66e None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:29.066763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911cea7-ceb8-41c4-812d-bd67f3dba66e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:29.066763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911cea7-ceb8-41c4-812d-bd67f3dba66e None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:29.068531 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 365/730] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:29.073960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cdb16-ce6d-42c6-ba36-1ad9685377e1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:29.074753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cdb16-ce6d-42c6-ba36-1ad9685377e1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:29.075222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cdb16-ce6d-42c6-ba36-1ad9685377e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:29.095032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-201cdb16-ce6d-42c6-ba36-1ad9685377e1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:29.096672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-201cdb16-ce6d-42c6-ba36-1ad9685377e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:29.182114 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 366/731] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:29.526135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911cea7-ceb8-41c4-812d-bd67f3dba66e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:29.543246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fc084-9592-4e3c-9aa0-571953ce12d0 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:29.543579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fc084-9592-4e3c-9aa0-571953ce12d0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:29.543777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fc084-9592-4e3c-9aa0-571953ce12d0 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:29.545348 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 367/732] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:29.551254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed07f-15a5-41a1-a5d9-240368dbfcd2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:29.551563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed07f-15a5-41a1-a5d9-240368dbfcd2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:29.551832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed07f-15a5-41a1-a5d9-240368dbfcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:29.572321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-29bed07f-15a5-41a1-a5d9-240368dbfcd2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:29.573524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-29bed07f-15a5-41a1-a5d9-240368dbfcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:29.639745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911cea7-ceb8-41c4-812d-bd67f3dba66e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:29.647638 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 368/733] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:29.654463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba8752-92d3-491e-af78-47d3aa259819 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:29.654735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba8752-92d3-491e-af78-47d3aa259819 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:29.654993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba8752-92d3-491e-af78-47d3aa259819 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:29.656373 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 369/734] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:29.662008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db303ef-c469-4928-a66a-a58fe7132ef3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:29.662277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db303ef-c469-4928-a66a-a58fe7132ef3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:29.662539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db303ef-c469-4928-a66a-a58fe7132ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:29.671861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:29.684459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e19971-4fd0-4288-ab15-c94ba91462ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:29.685219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e19971-4fd0-4288-ab15-c94ba91462ad None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:29.685604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e19971-4fd0-4288-ab15-c94ba91462ad None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:29.685888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0db303ef-c469-4928-a66a-a58fe7132ef3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:29.686016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e19971-4fd0-4288-ab15-c94ba91462ad None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:29.686761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e19971-4fd0-4288-ab15-c94ba91462ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:29.687453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0db303ef-c469-4928-a66a-a58fe7132ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:29.688363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e19971-4fd0-4288-ab15-c94ba91462ad None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:29.709483 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 366/735] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:33:29 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:29.710699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:29.771550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 370/736] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:29.980032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e19971-4fd0-4288-ab15-c94ba91462ad None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:30.083417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e19971-4fd0-4288-ab15-c94ba91462ad None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:30.111976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db303ef-c469-4928-a66a-a58fe7132ef3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:30.124472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70198204-5098-4720-8d01-bd61f1b2a074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:30.125654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70198204-5098-4720-8d01-bd61f1b2a074 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:30.125654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70198204-5098-4720-8d01-bd61f1b2a074 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:30.125654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70198204-5098-4720-8d01-bd61f1b2a074 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:30.126326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70198204-5098-4720-8d01-bd61f1b2a074 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:30.127759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70198204-5098-4720-8d01-bd61f1b2a074 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-70198204-5098-4720-8d01-bd61f1b2a074 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:30.138640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:30.139698 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:30.140520 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 367/737] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:30.146259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70198204-5098-4720-8d01-bd61f1b2a074 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:30.216632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:30.244596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db303ef-c469-4928-a66a-a58fe7132ef3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:30.277480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70198204-5098-4720-8d01-bd61f1b2a074 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:30.284228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092de70-7bf5-424e-ab02-d8ef876766ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:30.285142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092de70-7bf5-424e-ab02-d8ef876766ba None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:30.285591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092de70-7bf5-424e-ab02-d8ef876766ba None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:30.286715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092de70-7bf5-424e-ab02-d8ef876766ba None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:30.287560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092de70-7bf5-424e-ab02-d8ef876766ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:30.289182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092de70-7bf5-424e-ab02-d8ef876766ba None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:30.317099 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 371/738] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:33:30 2022] POST /identity/v3/services => generated 242 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:30.334724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114a4ff-9d2a-48f7-a65c-805a1531e651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:30.335340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114a4ff-9d2a-48f7-a65c-805a1531e651 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:30.335611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114a4ff-9d2a-48f7-a65c-805a1531e651 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:30.335962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114a4ff-9d2a-48f7-a65c-805a1531e651 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:30.336638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114a4ff-9d2a-48f7-a65c-805a1531e651 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:30.337028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114a4ff-9d2a-48f7-a65c-805a1531e651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:30.338497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114a4ff-9d2a-48f7-a65c-805a1531e651 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:30.350955 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 368/739] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:33:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:30.352515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092de70-7bf5-424e-ab02-d8ef876766ba None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:30.365435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:30.467581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092de70-7bf5-424e-ab02-d8ef876766ba None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:30.508177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe2026-525a-430c-a973-407fdc2b1be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:30.508702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe2026-525a-430c-a973-407fdc2b1be4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:30.508970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe2026-525a-430c-a973-407fdc2b1be4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:30.509266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe2026-525a-430c-a973-407fdc2b1be4 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:30.509902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe2026-525a-430c-a973-407fdc2b1be4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:30.510194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe2026-525a-430c-a973-407fdc2b1be4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:30.511563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe2026-525a-430c-a973-407fdc2b1be4 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:30.522527 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 372/740] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:33:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:30.524839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114a4ff-9d2a-48f7-a65c-805a1531e651 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:30.582263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab177b38-0a27-4ea6-8cbc-ad8141b29115 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:30.583111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab177b38-0a27-4ea6-8cbc-ad8141b29115 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:30.583500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab177b38-0a27-4ea6-8cbc-ad8141b29115 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:30.584963 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 373/741] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:30.590696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e6dc0-e8ee-4da8-b986-665ab4bbcbe3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:30.591425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e6dc0-e8ee-4da8-b986-665ab4bbcbe3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:30.591425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e6dc0-e8ee-4da8-b986-665ab4bbcbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:30.615054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f91e6dc0-e8ee-4da8-b986-665ab4bbcbe3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:30.616361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f91e6dc0-e8ee-4da8-b986-665ab4bbcbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:30.643606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114a4ff-9d2a-48f7-a65c-805a1531e651 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:30.655373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:30.695751 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 374/742] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:30 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:30.702085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adabbb-6676-4f2a-a617-9e09e66a20c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:30.702699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adabbb-6676-4f2a-a617-9e09e66a20c6 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:30.702969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adabbb-6676-4f2a-a617-9e09e66a20c6 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:30.703232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adabbb-6676-4f2a-a617-9e09e66a20c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:30.703947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adabbb-6676-4f2a-a617-9e09e66a20c6 None admin] RBAC: Adding query filter params (interface=admin, service_id=6761ee8ed7d8450599960caddd77c3f4, region_id=RegionOne) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:30.704299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adabbb-6676-4f2a-a617-9e09e66a20c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:30.705763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adabbb-6676-4f2a-a617-9e09e66a20c6 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:30.716522 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 369/743] 10.222.0.16 () {56 vars in 1249 bytes} [Sat Oct 15 20:33:30 2022] GET /identity/v3/endpoints?service_id=6761ee8ed7d8450599960caddd77c3f4&interface=admin®ion_id=RegionOne => generated 193 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:30.717883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e6dc0-e8ee-4da8-b986-665ab4bbcbe3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:30.744685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:30.822637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e6dc0-e8ee-4da8-b986-665ab4bbcbe3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:30.857789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f4b83-0ed0-4160-a411-c27ca38d7d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:30.858441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f4b83-0ed0-4160-a411-c27ca38d7d35 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:30.858646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f4b83-0ed0-4160-a411-c27ca38d7d35 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:30.858903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f4b83-0ed0-4160-a411-c27ca38d7d35 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:30.859560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f4b83-0ed0-4160-a411-c27ca38d7d35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:30.861015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f4b83-0ed0-4160-a411-c27ca38d7d35 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:30.873204 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 375/744] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Oct 15 20:33:30 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:33:31.062190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adabbb-6676-4f2a-a617-9e09e66a20c6 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:31.178870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adabbb-6676-4f2a-a617-9e09e66a20c6 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:31.219770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:31.220783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:31.221327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:31.221809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:31.222753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:31.225138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:31.228241 ncdv-master-1 devstack@keystone.service[99100]: WARNING keystone.server.flask.common [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 15 20:33:31.248082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.sql.core [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 20:33:31.248082 ncdv-master-1 devstack@keystone.service[99100]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 20:33:31.248082 ncdv-master-1 devstack@keystone.service[99100]: [parameters: {'id': '7fde5feb6ef04aeb90a65e3d92e554c8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 15 20:33:31.248082 ncdv-master-1 devstack@keystone.service[99100]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99100) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fde5feb6ef04aeb90a65e3d92e554c8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fde5feb6ef04aeb90a65e3d92e554c8. Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application conn.query(q) Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:33:31.257604 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result.read() Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 20:33:31.258406 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application next(self.gen) Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application yield resource Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application next(self.gen) Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise self.value Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application session.commit() Oct 15 20:33:31.259405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.session.flush() Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application compat.raise_( Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 20:33:31.260417 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 20:33:31.262079 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application conn.query(q) Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result.read() Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:33:31.262977 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [parameters: {'id': '7fde5feb6ef04aeb90a65e3d92e554c8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:31.263840 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:31.264644 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:31.266981 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:31.266981 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 15 20:33:31.266981 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 15 20:33:31.266981 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:31.266981 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:31.266981 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 20:33:31.266981 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 20:33:31.266981 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fde5feb6ef04aeb90a65e3d92e554c8. Oct 15 20:33:31.266981 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:31.270562 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 370/745] 10.222.0.16 () {60 vars in 1140 bytes} [Sat Oct 15 20:33:31 2022] POST /identity/v3/projects => generated 262 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:31.277146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f4b83-0ed0-4160-a411-c27ca38d7d35 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:31.352203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:31.388993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f4b83-0ed0-4160-a411-c27ca38d7d35 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:31.423597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d321fe-6bcd-462d-b4dc-d4030ea07f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:31.424218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d321fe-6bcd-462d-b4dc-d4030ea07f4d None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:31.424489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d321fe-6bcd-462d-b4dc-d4030ea07f4d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:31.424755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d321fe-6bcd-462d-b4dc-d4030ea07f4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:31.432366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d321fe-6bcd-462d-b4dc-d4030ea07f4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:31.433761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d321fe-6bcd-462d-b4dc-d4030ea07f4d None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-46d321fe-6bcd-462d-b4dc-d4030ea07f4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:33:31.445132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:31.446024 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:31.446886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:33:31.446886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:33:31.446886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:33:31.446886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:33:31.446886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:33:31.446886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:33:31.446886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:31.446886 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:31.446886 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 376/746] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:31.454625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:31.489640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:31.571557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81488c8a-dbf5-4cef-b092-463c59275a79 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:31.611665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7aa28-6c19-48df-b30b-cf89bd5db508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:31.612669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7aa28-6c19-48df-b30b-cf89bd5db508 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:31.613219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7aa28-6c19-48df-b30b-cf89bd5db508 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:31.613700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7aa28-6c19-48df-b30b-cf89bd5db508 None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:31.615133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7aa28-6c19-48df-b30b-cf89bd5db508 None admin] RBAC: Adding query filter params (name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:31.615690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7aa28-6c19-48df-b30b-cf89bd5db508 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:31.617884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7aa28-6c19-48df-b30b-cf89bd5db508 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:31.640146 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 371/747] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:32.324388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1ba9c-141a-43a2-a1a3-c36f23541d83 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:32.325436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1ba9c-141a-43a2-a1a3-c36f23541d83 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:32.325943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1ba9c-141a-43a2-a1a3-c36f23541d83 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:32.327768 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 377/748] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:32.333525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b84562-7c85-4c60-894f-68662e6c1b26 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:32.334356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b84562-7c85-4c60-894f-68662e6c1b26 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:32.334833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b84562-7c85-4c60-894f-68662e6c1b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:32.354649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b3b84562-7c85-4c60-894f-68662e6c1b26 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:32.356459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b3b84562-7c85-4c60-894f-68662e6c1b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:32.437192 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 372/749] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:32 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:32.759659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1ba9c-141a-43a2-a1a3-c36f23541d83 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:32.850482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786b238-cf1e-412d-904c-91ce7433b9e6 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:32.850835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786b238-cf1e-412d-904c-91ce7433b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:32.851041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786b238-cf1e-412d-904c-91ce7433b9e6 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:32.852649 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 373/750] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:32.857972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafb22b-aabd-4ab0-904b-f429fc3a4d9c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:32.858467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafb22b-aabd-4ab0-904b-f429fc3a4d9c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:32.858895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafb22b-aabd-4ab0-904b-f429fc3a4d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:32.874725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1ba9c-141a-43a2-a1a3-c36f23541d83 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:32.881586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-acafb22b-aabd-4ab0-904b-f429fc3a4d9c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:32.883136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-acafb22b-aabd-4ab0-904b-f429fc3a4d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:32.899230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:32.908162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:32.919357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9597d7-fff5-4d64-a762-020cee6acee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:32.919974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9597d7-fff5-4d64-a762-020cee6acee1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:32.920246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9597d7-fff5-4d64-a762-020cee6acee1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:32.920510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9597d7-fff5-4d64-a762-020cee6acee1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:32.921335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9597d7-fff5-4d64-a762-020cee6acee1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:32.923283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9597d7-fff5-4d64-a762-020cee6acee1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-dc9597d7-fff5-4d64-a762-020cee6acee1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:32.935641 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:32.936691 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 20:33:32.937536 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:32.942480 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 378/751] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:32.943752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9597d7-fff5-4d64-a762-020cee6acee1 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:32.980047 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 374/752] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:32 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:33.059276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9597d7-fff5-4d64-a762-020cee6acee1 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:33.094603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e3c5f-3684-4d12-91aa-f9533a899c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:33.095213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e3c5f-3684-4d12-91aa-f9533a899c85 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:33.095486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e3c5f-3684-4d12-91aa-f9533a899c85 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:33.095748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e3c5f-3684-4d12-91aa-f9533a899c85 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:33.096371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e3c5f-3684-4d12-91aa-f9533a899c85 None admin] RBAC: Adding query filter params (name=compute) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:33.096668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e3c5f-3684-4d12-91aa-f9533a899c85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:33.098795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e3c5f-3684-4d12-91aa-f9533a899c85 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:33.108651 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 379/753] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:33.110143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafb22b-aabd-4ab0-904b-f429fc3a4d9c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:33.217165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafb22b-aabd-4ab0-904b-f429fc3a4d9c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:33.252962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5af163-9b12-451b-b57e-82346773bf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:33.253590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5af163-9b12-451b-b57e-82346773bf5c None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:33.253864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5af163-9b12-451b-b57e-82346773bf5c None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:33.254130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5af163-9b12-451b-b57e-82346773bf5c None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:33.255347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5af163-9b12-451b-b57e-82346773bf5c None admin] RBAC: Adding query filter params (type=compute) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:33.255589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5af163-9b12-451b-b57e-82346773bf5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:33.256975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5af163-9b12-451b-b57e-82346773bf5c None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:33.266857 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 375/754] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:33.268193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e3c5f-3684-4d12-91aa-f9533a899c85 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:33.326688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:33.333755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5af163-9b12-451b-b57e-82346773bf5c None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:33.350835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:33.381410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e3c5f-3684-4d12-91aa-f9533a899c85 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:33.424677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb14f34-7a22-415e-a553-ac37d87b9507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:33.425858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb14f34-7a22-415e-a553-ac37d87b9507 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:33.426816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb14f34-7a22-415e-a553-ac37d87b9507 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:33.427256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb14f34-7a22-415e-a553-ac37d87b9507 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:33.428430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb14f34-7a22-415e-a553-ac37d87b9507 None admin] RBAC: Adding query filter params (interface=public, service_id=b83053014f71434b8b7232545e4c7916, region_id=RegionOne) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:33.428879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb14f34-7a22-415e-a553-ac37d87b9507 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:33.432223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb14f34-7a22-415e-a553-ac37d87b9507 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:33.437852 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 380/755] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Oct 15 20:33:33 2022] GET /identity/v3/endpoints?service_id=b83053014f71434b8b7232545e4c7916&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:33.440811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a081020-f8ee-433a-893b-e4859efda0cf None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:33.440811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a081020-f8ee-433a-893b-e4859efda0cf None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:33.440811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a081020-f8ee-433a-893b-e4859efda0cf None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:33.440811 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 381/756] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:33.445432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a081020-f8ee-433a-893b-e4859efda0cf None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:33.455313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5af163-9b12-451b-b57e-82346773bf5c None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:33.491671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dab47a-e040-4576-8255-1072d92f80f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:33.492304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dab47a-e040-4576-8255-1072d92f80f0 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:33.492575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dab47a-e040-4576-8255-1072d92f80f0 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:33.492839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dab47a-e040-4576-8255-1072d92f80f0 None admin] PATH_INFO: `/v3/services/image` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:33.493440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dab47a-e040-4576-8255-1072d92f80f0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:33.494932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dab47a-e040-4576-8255-1072d92f80f0 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-20dab47a-e040-4576-8255-1072d92f80f0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:33.505638 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:33.506605 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:33.507445 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 376/757] 10.222.0.16 () {56 vars in 1105 bytes} [Sat Oct 15 20:33:33 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:33.508220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63e38f-f531-4bb4-b72b-69dbc234c125 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:33.508285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63e38f-f531-4bb4-b72b-69dbc234c125 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:33.509027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63e38f-f531-4bb4-b72b-69dbc234c125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:33.529091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6e63e38f-f531-4bb4-b72b-69dbc234c125 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:33.530293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e63e38f-f531-4bb4-b72b-69dbc234c125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:33.564066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a081020-f8ee-433a-893b-e4859efda0cf None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:33.586368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:33.600865 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 377/758] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:33 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:33.604060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13374d-466e-473c-9238-010b8a20ad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:33.604660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13374d-466e-473c-9238-010b8a20ad01 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:33.604931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13374d-466e-473c-9238-010b8a20ad01 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:33.605230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13374d-466e-473c-9238-010b8a20ad01 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:33.605824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13374d-466e-473c-9238-010b8a20ad01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:33.607172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13374d-466e-473c-9238-010b8a20ad01 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:33.609903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63e38f-f531-4bb4-b72b-69dbc234c125 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:33.615894 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 382/759] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Oct 15 20:33:33 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:33:33.619871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:33.727562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63e38f-f531-4bb4-b72b-69dbc234c125 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:33.768995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef62952-4513-405c-ac40-a65d72a03ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:33.769975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef62952-4513-405c-ac40-a65d72a03ef4 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:33.770434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef62952-4513-405c-ac40-a65d72a03ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:33.770849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef62952-4513-405c-ac40-a65d72a03ef4 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:33.771675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef62952-4513-405c-ac40-a65d72a03ef4 None admin] RBAC: Adding query filter params (name=image) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:33.772123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef62952-4513-405c-ac40-a65d72a03ef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:33.773745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef62952-4513-405c-ac40-a65d72a03ef4 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:33.780419 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 378/760] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:33.786785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13374d-466e-473c-9238-010b8a20ad01 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:33.904666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13374d-466e-473c-9238-010b8a20ad01 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:33.940854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca2d17-e974-438d-bfc2-92eca956da8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:33.941725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca2d17-e974-438d-bfc2-92eca956da8b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:33.942323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca2d17-e974-438d-bfc2-92eca956da8b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:33.942728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca2d17-e974-438d-bfc2-92eca956da8b None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:33.943557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca2d17-e974-438d-bfc2-92eca956da8b None admin] RBAC: Adding query filter params (type=image) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:33.943988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca2d17-e974-438d-bfc2-92eca956da8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:33.945495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca2d17-e974-438d-bfc2-92eca956da8b None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:33.951679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef62952-4513-405c-ac40-a65d72a03ef4 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:33.960636 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 383/761] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:33.961910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca2d17-e974-438d-bfc2-92eca956da8b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:34.015774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:34.043425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:34.083318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca2d17-e974-438d-bfc2-92eca956da8b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:34.089697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef62952-4513-405c-ac40-a65d72a03ef4 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:34.118137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c6ebd-ceb3-4643-a6ff-85d26d30cebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:34.118990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c6ebd-ceb3-4643-a6ff-85d26d30cebd None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:34.118990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c6ebd-ceb3-4643-a6ff-85d26d30cebd None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:34.119246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c6ebd-ceb3-4643-a6ff-85d26d30cebd None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:34.119927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c6ebd-ceb3-4643-a6ff-85d26d30cebd None admin] RBAC: Adding query filter params (interface=public, service_id=1475e7aa5bf94a179dab817407fce08b, region_id=RegionOne) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:34.120226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c6ebd-ceb3-4643-a6ff-85d26d30cebd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:34.121602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c6ebd-ceb3-4643-a6ff-85d26d30cebd None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:34.133061 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 384/762] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Oct 15 20:33:33 2022] GET /identity/v3/endpoints?service_id=1475e7aa5bf94a179dab817407fce08b&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:34.135105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c6ebd-ceb3-4643-a6ff-85d26d30cebd None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:34.143403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde3eea-3f8f-4fbd-a193-c090714c114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:34.144027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde3eea-3f8f-4fbd-a193-c090714c114e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:34.144295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde3eea-3f8f-4fbd-a193-c090714c114e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:34.144606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde3eea-3f8f-4fbd-a193-c090714c114e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:34.145805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde3eea-3f8f-4fbd-a193-c090714c114e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:34.147271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde3eea-3f8f-4fbd-a193-c090714c114e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-2dde3eea-3f8f-4fbd-a193-c090714c114e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:34.156523 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:34.157538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 20:33:34.158419 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:34.162762 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 379/763] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Oct 15 20:33:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:34.163831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde3eea-3f8f-4fbd-a193-c090714c114e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:34.257638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c6ebd-ceb3-4643-a6ff-85d26d30cebd None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:34.268126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:34.278686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde3eea-3f8f-4fbd-a193-c090714c114e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:34.294388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfce7f7-2ece-4e4b-a4d2-54f679846c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:34.294979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfce7f7-2ece-4e4b-a4d2-54f679846c5a None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:34.295251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfce7f7-2ece-4e4b-a4d2-54f679846c5a None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:34.295511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfce7f7-2ece-4e4b-a4d2-54f679846c5a None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:34.296097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfce7f7-2ece-4e4b-a4d2-54f679846c5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:34.297473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfce7f7-2ece-4e4b-a4d2-54f679846c5a None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:34.303957 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 385/764] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Oct 15 20:33:34 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:33:34.305098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094de9d-8fd0-4442-846c-dd3248219696 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:34.305375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094de9d-8fd0-4442-846c-dd3248219696 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:34.305637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094de9d-8fd0-4442-846c-dd3248219696 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:34.306994 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 386/765] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:34.314748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa539fab-282f-4a5c-bf1b-54e00c58498d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:34.315054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa539fab-282f-4a5c-bf1b-54e00c58498d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:34.315287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa539fab-282f-4a5c-bf1b-54e00c58498d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:34.327047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22871d-c0e2-4e80-9ccd-55129d6e2064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:34.327671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22871d-c0e2-4e80-9ccd-55129d6e2064 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:34.327942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22871d-c0e2-4e80-9ccd-55129d6e2064 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:34.328204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22871d-c0e2-4e80-9ccd-55129d6e2064 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:34.328843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22871d-c0e2-4e80-9ccd-55129d6e2064 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:34.329241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22871d-c0e2-4e80-9ccd-55129d6e2064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:34.330657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22871d-c0e2-4e80-9ccd-55129d6e2064 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:34.331956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:34.340355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fa539fab-282f-4a5c-bf1b-54e00c58498d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:34.342420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa539fab-282f-4a5c-bf1b-54e00c58498d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:34.343922 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 380/766] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:33:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:34.344908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22871d-c0e2-4e80-9ccd-55129d6e2064 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:34.442227 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 387/767] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:34 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:34.464748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22871d-c0e2-4e80-9ccd-55129d6e2064 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:34.505826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902984-e627-4baf-bb6c-3a9fa77ad60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:34.506923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902984-e627-4baf-bb6c-3a9fa77ad60b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:34.507429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902984-e627-4baf-bb6c-3a9fa77ad60b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:34.507906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902984-e627-4baf-bb6c-3a9fa77ad60b None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:34.508798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9902984-e627-4baf-bb6c-3a9fa77ad60b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:34.509325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9902984-e627-4baf-bb6c-3a9fa77ad60b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:34.510984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9902984-e627-4baf-bb6c-3a9fa77ad60b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:34.517601 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 381/768] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Oct 15 20:33:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:34.525544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa539fab-282f-4a5c-bf1b-54e00c58498d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:34.632376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa539fab-282f-4a5c-bf1b-54e00c58498d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:34.634691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:34.669330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ccf04-7ec0-4623-a514-3b554a7e94c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:34.669979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ccf04-7ec0-4623-a514-3b554a7e94c2 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:34.670246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ccf04-7ec0-4623-a514-3b554a7e94c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:34.670505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ccf04-7ec0-4623-a514-3b554a7e94c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:34.671217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ccf04-7ec0-4623-a514-3b554a7e94c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:34.672568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ccf04-7ec0-4623-a514-3b554a7e94c2 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:34.692819 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 388/769] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:33:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:34.818785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902984-e627-4baf-bb6c-3a9fa77ad60b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:34.891777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:34.936453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902984-e627-4baf-bb6c-3a9fa77ad60b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:34.980352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f8e75-14e4-477e-bf7e-9473e3cd8c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:34.980879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f8e75-14e4-477e-bf7e-9473e3cd8c65 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:34.981210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f8e75-14e4-477e-bf7e-9473e3cd8c65 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:34.981477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f8e75-14e4-477e-bf7e-9473e3cd8c65 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:34.982104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f8e75-14e4-477e-bf7e-9473e3cd8c65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:34.983530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f8e75-14e4-477e-bf7e-9473e3cd8c65 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:35.002478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.sql.core [None req-4b6f8e75-14e4-477e-bf7e-9473e3cd8c65 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 20:33:35.002478 ncdv-master-1 devstack@keystone.service[99100]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 15 20:33:35.002478 ncdv-master-1 devstack@keystone.service[99100]: [parameters: {'user_id': '4fd3885517a241c4b02b33b5380758be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 15 20:33:35.002478 ncdv-master-1 devstack@keystone.service[99100]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99100) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-4b6f8e75-14e4-477e-bf7e-9473e3cd8c65 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application conn.query(q) Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result.read() Oct 15 20:33:35.010939 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 20:33:35.011792 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application next(self.gen) Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application yield resource Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application next(self.gen) Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise self.value Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application session.commit() Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 20:33:35.012709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.session.flush() Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application compat.raise_( Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 20:33:35.013610 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:33:35.014463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application conn.query(q) Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result.read() Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:33:35.015275 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [parameters: {'user_id': '4fd3885517a241c4b02b33b5380758be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:35.016075 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 15 20:33:35.016833 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 15 20:33:35.020549 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 15 20:33:35.020549 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 15 20:33:35.020549 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 20:33:35.020549 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 20:33:35.020549 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 15 20:33:35.020549 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:35.020549 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 382/770] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Oct 15 20:33:34 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:35.025620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ccf04-7ec0-4623-a514-3b554a7e94c2 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:35.136256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ccf04-7ec0-4623-a514-3b554a7e94c2 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:35.180363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726452b-5f49-4131-a3c9-bac5094f97be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:35.181266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726452b-5f49-4131-a3c9-bac5094f97be None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:35.181698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726452b-5f49-4131-a3c9-bac5094f97be None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:35.182128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726452b-5f49-4131-a3c9-bac5094f97be None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:35.190817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726452b-5f49-4131-a3c9-bac5094f97be None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:35.193525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726452b-5f49-4131-a3c9-bac5094f97be None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-8726452b-5f49-4131-a3c9-bac5094f97be None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:33:35.209082 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:35.210310 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:35.211355 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 389/771] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:33:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:35.213711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f8e75-14e4-477e-bf7e-9473e3cd8c65 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:35.325586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f8e75-14e4-477e-bf7e-9473e3cd8c65 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:35.346612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:35.362161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4f6f8-354a-45af-a9f1-a59f74829464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:35.362762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4f6f8-354a-45af-a9f1-a59f74829464 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:35.363041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4f6f8-354a-45af-a9f1-a59f74829464 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:35.363311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4f6f8-354a-45af-a9f1-a59f74829464 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:35.364390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4f6f8-354a-45af-a9f1-a59f74829464 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:35.364700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4f6f8-354a-45af-a9f1-a59f74829464 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:35.366703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4f6f8-354a-45af-a9f1-a59f74829464 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:35.413333 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 383/772] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Oct 15 20:33:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:36.158060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b4a63-1d15-46b3-8ac9-ffbc3dca9002 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:36.159002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b4a63-1d15-46b3-8ac9-ffbc3dca9002 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:36.159493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b4a63-1d15-46b3-8ac9-ffbc3dca9002 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:36.161831 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 390/773] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:36.167336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab6f3c-bec6-4bed-9ef0-399956ae7656 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:36.167617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab6f3c-bec6-4bed-9ef0-399956ae7656 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:36.167883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab6f3c-bec6-4bed-9ef0-399956ae7656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:36.187335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-22ab6f3c-bec6-4bed-9ef0-399956ae7656 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:36.188843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-22ab6f3c-bec6-4bed-9ef0-399956ae7656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:36.258834 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 384/774] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:36 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:36.584181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b4a63-1d15-46b3-8ac9-ffbc3dca9002 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:36.610382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:36.690814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b4a63-1d15-46b3-8ac9-ffbc3dca9002 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:36.733444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481dd3b-fd19-4632-9779-75e79bd852e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:36.734071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481dd3b-fd19-4632-9779-75e79bd852e0 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:36.734344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481dd3b-fd19-4632-9779-75e79bd852e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:36.734611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481dd3b-fd19-4632-9779-75e79bd852e0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:36.735192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481dd3b-fd19-4632-9779-75e79bd852e0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:36.736575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481dd3b-fd19-4632-9779-75e79bd852e0 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-2481dd3b-fd19-4632-9779-75e79bd852e0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:36.747055 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:36.748129 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 20:33:36.748962 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:36.753509 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 391/775] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:36.755124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab6f3c-bec6-4bed-9ef0-399956ae7656 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:36.855930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e94d-ff3b-4418-ba98-07fd355ac4e1 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:36.856144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e94d-ff3b-4418-ba98-07fd355ac4e1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:36.856381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e94d-ff3b-4418-ba98-07fd355ac4e1 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:36.856632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab6f3c-bec6-4bed-9ef0-399956ae7656 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:36.858033 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 392/776] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:36.864502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9101c-2b80-42a8-935f-a183ed76d46d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:36.864804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9101c-2b80-42a8-935f-a183ed76d46d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:36.865064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9101c-2b80-42a8-935f-a183ed76d46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:36.867991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:36.883842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-13a9101c-2b80-42a8-935f-a183ed76d46d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:36.885027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-13a9101c-2b80-42a8-935f-a183ed76d46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:36.895199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae80c0-754d-4682-8a48-4eeff4a378da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:36.895793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae80c0-754d-4682-8a48-4eeff4a378da None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:36.896067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae80c0-754d-4682-8a48-4eeff4a378da None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:36.896336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae80c0-754d-4682-8a48-4eeff4a378da None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:36.896961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae80c0-754d-4682-8a48-4eeff4a378da None admin] RBAC: Adding query filter params (name=compute) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:36.897288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae80c0-754d-4682-8a48-4eeff4a378da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:36.898664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae80c0-754d-4682-8a48-4eeff4a378da None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:36.903652 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 385/777] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:36.909546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae80c0-754d-4682-8a48-4eeff4a378da None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:36.959803 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 393/778] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:36 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:37.013627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae80c0-754d-4682-8a48-4eeff4a378da None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:37.049172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da69814-f106-4a74-9a1c-87c4fa5dde97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:37.049770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da69814-f106-4a74-9a1c-87c4fa5dde97 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:37.050043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da69814-f106-4a74-9a1c-87c4fa5dde97 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:37.050309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da69814-f106-4a74-9a1c-87c4fa5dde97 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:37.050959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da69814-f106-4a74-9a1c-87c4fa5dde97 None admin] RBAC: Adding query filter params (type=compute) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:37.051262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da69814-f106-4a74-9a1c-87c4fa5dde97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:37.052691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da69814-f106-4a74-9a1c-87c4fa5dde97 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:37.057722 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 386/779] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Oct 15 20:33:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:37.063403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9101c-2b80-42a8-935f-a183ed76d46d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:37.086766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:37.172145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9101c-2b80-42a8-935f-a183ed76d46d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:37.175776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9fa11-a48e-4976-bd17-b39407601db8 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:37.176568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9fa11-a48e-4976-bd17-b39407601db8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:37.176840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9fa11-a48e-4976-bd17-b39407601db8 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:37.178327 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 387/780] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:37.183051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e51a1-690a-4eba-adfe-725361e15a1a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:37.183325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e51a1-690a-4eba-adfe-725361e15a1a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:37.183598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e51a1-690a-4eba-adfe-725361e15a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:37.203194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c50e51a1-690a-4eba-adfe-725361e15a1a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:37.204498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c50e51a1-690a-4eba-adfe-725361e15a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:37.215327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ade8a-b37e-4267-98fb-b5f16e90dde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:37.215917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ade8a-b37e-4267-98fb-b5f16e90dde8 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:37.216182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ade8a-b37e-4267-98fb-b5f16e90dde8 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:37.216440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ade8a-b37e-4267-98fb-b5f16e90dde8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:37.217119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ade8a-b37e-4267-98fb-b5f16e90dde8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:37.218459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ade8a-b37e-4267-98fb-b5f16e90dde8 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:37.235325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:37.241646 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 394/781] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:33:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:37.288621 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 388/782] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:37.301863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ade8a-b37e-4267-98fb-b5f16e90dde8 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:37.385634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:37.417100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ade8a-b37e-4267-98fb-b5f16e90dde8 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:37.453477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb52c9-7d11-470c-9d84-378f72cdd80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:37.454034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb52c9-7d11-470c-9d84-378f72cdd80e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:37.454302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb52c9-7d11-470c-9d84-378f72cdd80e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:37.454566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb52c9-7d11-470c-9d84-378f72cdd80e None admin] PATH_INFO: `/v3/services/image` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:37.455141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb52c9-7d11-470c-9d84-378f72cdd80e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:37.456495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb52c9-7d11-470c-9d84-378f72cdd80e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-59cb52c9-7d11-470c-9d84-378f72cdd80e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:37.465517 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:37.466485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:37.467480 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 395/783] 10.222.0.16 () {56 vars in 1105 bytes} [Sat Oct 15 20:33:37 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:37.472628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e51a1-690a-4eba-adfe-725361e15a1a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:37.617503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb52c9-7d11-470c-9d84-378f72cdd80e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:37.703764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e51a1-690a-4eba-adfe-725361e15a1a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:37.731099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb52c9-7d11-470c-9d84-378f72cdd80e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:37.746494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b5b48-5f69-461e-86b1-5371b665c866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:37.747215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b5b48-5f69-461e-86b1-5371b665c866 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:37.747374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b5b48-5f69-461e-86b1-5371b665c866 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:37.747626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b5b48-5f69-461e-86b1-5371b665c866 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:37.748255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b5b48-5f69-461e-86b1-5371b665c866 None admin] RBAC: Adding query filter params (name=image) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:37.748553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b5b48-5f69-461e-86b1-5371b665c866 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:37.749989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b5b48-5f69-461e-86b1-5371b665c866 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:37.755515 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 389/784] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:37.761178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b5b48-5f69-461e-86b1-5371b665c866 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:37.771322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94becd6-eb8b-4635-96b4-5226ee0348b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:37.772162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94becd6-eb8b-4635-96b4-5226ee0348b7 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:37.772958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94becd6-eb8b-4635-96b4-5226ee0348b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:37.773407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94becd6-eb8b-4635-96b4-5226ee0348b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:37.775212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94becd6-eb8b-4635-96b4-5226ee0348b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:37.776776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94becd6-eb8b-4635-96b4-5226ee0348b7 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:37.783131 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 396/785] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:37.784660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94becd6-eb8b-4635-96b4-5226ee0348b7 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:37.794184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:37.795591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:37.881254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b5b48-5f69-461e-86b1-5371b665c866 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:37.896863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94becd6-eb8b-4635-96b4-5226ee0348b7 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:37.917813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f911e6e-4307-4a84-934c-6d1b417c2d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:37.918844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911e6e-4307-4a84-934c-6d1b417c2d35 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:37.919120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911e6e-4307-4a84-934c-6d1b417c2d35 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:37.919543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911e6e-4307-4a84-934c-6d1b417c2d35 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:37.920156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f911e6e-4307-4a84-934c-6d1b417c2d35 None admin] RBAC: Adding query filter params (type=image) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:37.920458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f911e6e-4307-4a84-934c-6d1b417c2d35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:37.921882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f911e6e-4307-4a84-934c-6d1b417c2d35 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:37.931891 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 390/786] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:37.933082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f911e6e-4307-4a84-934c-6d1b417c2d35 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:37.937323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea837d-edcb-42c2-82ab-fbd53a2ad414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:37.937973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea837d-edcb-42c2-82ab-fbd53a2ad414 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:37.938253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea837d-edcb-42c2-82ab-fbd53a2ad414 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:37.938519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea837d-edcb-42c2-82ab-fbd53a2ad414 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:37.939153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea837d-edcb-42c2-82ab-fbd53a2ad414 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:37.940570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea837d-edcb-42c2-82ab-fbd53a2ad414 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:37.977290 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 397/787] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:33:37 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:38.048355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:38.061414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f911e6e-4307-4a84-934c-6d1b417c2d35 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:38.099902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4587c54-f8f5-460b-abc3-6251f1810c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:38.100534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4587c54-f8f5-460b-abc3-6251f1810c15 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:38.100794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4587c54-f8f5-460b-abc3-6251f1810c15 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:38.101082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4587c54-f8f5-460b-abc3-6251f1810c15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:38.101761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4587c54-f8f5-460b-abc3-6251f1810c15 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:38.103157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4587c54-f8f5-460b-abc3-6251f1810c15 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:38.133081 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 391/788] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:33:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:33:39.752072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e527d82-24d5-4135-8bb6-21395ac338d0 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:39.752602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e527d82-24d5-4135-8bb6-21395ac338d0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:39.752602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e527d82-24d5-4135-8bb6-21395ac338d0 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:39.754490 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 398/789] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:39.759912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71d8c-3a37-48eb-a61e-bb551a9c46d0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:39.760313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71d8c-3a37-48eb-a61e-bb551a9c46d0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:39.760505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71d8c-3a37-48eb-a61e-bb551a9c46d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:39.779135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b9d71d8c-3a37-48eb-a61e-bb551a9c46d0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:39.780223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9d71d8c-3a37-48eb-a61e-bb551a9c46d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:39.853975 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 392/790] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:40.175139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e527d82-24d5-4135-8bb6-21395ac338d0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:40.239245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:40.283883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e527d82-24d5-4135-8bb6-21395ac338d0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:40.318061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b4503-41fb-4be9-ac04-fee3bb4f1418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:40.318677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b4503-41fb-4be9-ac04-fee3bb4f1418 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:40.319005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b4503-41fb-4be9-ac04-fee3bb4f1418 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:40.319201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b4503-41fb-4be9-ac04-fee3bb4f1418 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:40.329120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b4503-41fb-4be9-ac04-fee3bb4f1418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-fb6b4503-41fb-4be9-ac04-fee3bb4f1418 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:40.333978 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:40.334902 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:40.335766 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 399/791] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Oct 15 20:33:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:40.342472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d71d8c-3a37-48eb-a61e-bb551a9c46d0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:40.411461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:40.443586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d71d8c-3a37-48eb-a61e-bb551a9c46d0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:40.478399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9224f-3dd6-46e0-86a4-1eb966e9d55e None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:40.478614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675dc5-c8a4-4d40-8e2b-6683ce836e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:40.478722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9224f-3dd6-46e0-86a4-1eb966e9d55e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:40.478964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9224f-3dd6-46e0-86a4-1eb966e9d55e None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:40.479125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675dc5-c8a4-4d40-8e2b-6683ce836e61 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:40.479358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675dc5-c8a4-4d40-8e2b-6683ce836e61 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:40.479622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675dc5-c8a4-4d40-8e2b-6683ce836e61 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:40.480431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca675dc5-c8a4-4d40-8e2b-6683ce836e61 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:40.480531 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 400/792] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:40.480602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca675dc5-c8a4-4d40-8e2b-6683ce836e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:40.481924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca675dc5-c8a4-4d40-8e2b-6683ce836e61 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:40.485258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe3add-d400-49b8-9314-6cd127e435f3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:40.485539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe3add-d400-49b8-9314-6cd127e435f3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:40.485805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe3add-d400-49b8-9314-6cd127e435f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:40.495554 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 393/793] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Oct 15 20:33:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:40.497026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675dc5-c8a4-4d40-8e2b-6683ce836e61 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:40.505169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5abe3add-d400-49b8-9314-6cd127e435f3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:40.506402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5abe3add-d400-49b8-9314-6cd127e435f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:40.575903 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 401/794] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:40.599718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ddd59-d2b0-4079-83ab-c3f9732f5d87 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:40.600006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ddd59-d2b0-4079-83ab-c3f9732f5d87 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:40.600273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ddd59-d2b0-4079-83ab-c3f9732f5d87 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:40.601640 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 402/795] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:40.605503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675dc5-c8a4-4d40-8e2b-6683ce836e61 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:40.606481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ed112-9000-42c1-99da-a6582c107091 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:40.606757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ed112-9000-42c1-99da-a6582c107091 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:40.607025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ed112-9000-42c1-99da-a6582c107091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:40.625499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6b7ed112-9000-42c1-99da-a6582c107091 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:40.626682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6b7ed112-9000-42c1-99da-a6582c107091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:40.642821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f5003-9c34-4b25-88a8-2401166e8538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:40.643436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f5003-9c34-4b25-88a8-2401166e8538 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:40.643706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f5003-9c34-4b25-88a8-2401166e8538 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:40.643965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f5003-9c34-4b25-88a8-2401166e8538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:40.646006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:40.655949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f5003-9c34-4b25-88a8-2401166e8538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:40.657437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f5003-9c34-4b25-88a8-2401166e8538 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-6d2f5003-9c34-4b25-88a8-2401166e8538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:40.672907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:40.674072 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:40.675163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:40.675163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:40.675163 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:40.677376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:40.678700 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 394/796] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:40.679949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f5003-9c34-4b25-88a8-2401166e8538 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:40.704902 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 403/797] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:40.791254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f5003-9c34-4b25-88a8-2401166e8538 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:40.826650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1007d-5714-4d73-b1e2-6efe732f7305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:40.827431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1007d-5714-4d73-b1e2-6efe732f7305 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:40.827848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1007d-5714-4d73-b1e2-6efe732f7305 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:40.828287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1007d-5714-4d73-b1e2-6efe732f7305 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:40.829074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1007d-5714-4d73-b1e2-6efe732f7305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:40.829532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1007d-5714-4d73-b1e2-6efe732f7305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:40.831036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1007d-5714-4d73-b1e2-6efe732f7305 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:40.843658 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 395/798] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:40.849574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ed112-9000-42c1-99da-a6582c107091 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:40.901791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1007d-5714-4d73-b1e2-6efe732f7305 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:40.956206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ed112-9000-42c1-99da-a6582c107091 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:40.957739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:40.993842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f334f-fa1f-43ba-b667-137a693b12f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:40.994519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f334f-fa1f-43ba-b667-137a693b12f3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:40.994727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f334f-fa1f-43ba-b667-137a693b12f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:40.994961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f334f-fa1f-43ba-b667-137a693b12f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.001990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f334f-fa1f-43ba-b667-137a693b12f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.003410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f334f-fa1f-43ba-b667-137a693b12f3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-d32f334f-fa1f-43ba-b667-137a693b12f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:33:41.015373 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:41.016298 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:41.017784 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:41.017784 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:41.017784 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:33:41.017784 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:33:41.017784 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:33:41.017784 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:33:41.017784 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:33:41.017784 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:33:41.017784 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:33:41.017784 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:41.018312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1007d-5714-4d73-b1e2-6efe732f7305 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:41.021996 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 404/799] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Oct 15 20:33:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:41.022657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f334f-fa1f-43ba-b667-137a693b12f3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:41.053344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd46d0-c7ed-42c1-be2a-75fc253838a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:41.053957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd46d0-c7ed-42c1-be2a-75fc253838a1 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:41.054224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd46d0-c7ed-42c1-be2a-75fc253838a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:41.054483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd46d0-c7ed-42c1-be2a-75fc253838a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.055563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd46d0-c7ed-42c1-be2a-75fc253838a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.056984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd46d0-c7ed-42c1-be2a-75fc253838a1 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.058554 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 396/800] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:41.059271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd46d0-c7ed-42c1-be2a-75fc253838a1 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:41.066952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:41.126390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f334f-fa1f-43ba-b667-137a693b12f3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:41.159965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8bc2a-85dd-4bff-baef-e855794083c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:41.160569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8bc2a-85dd-4bff-baef-e855794083c2 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:41.160839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8bc2a-85dd-4bff-baef-e855794083c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:41.161134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8bc2a-85dd-4bff-baef-e855794083c2 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.161841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8bc2a-85dd-4bff-baef-e855794083c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:41.162134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8bc2a-85dd-4bff-baef-e855794083c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.162558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd46d0-c7ed-42c1-be2a-75fc253838a1 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:41.163533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8bc2a-85dd-4bff-baef-e855794083c2 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.177582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:41.181281 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 405/801] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Oct 15 20:33:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:41.181281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8bc2a-85dd-4bff-baef-e855794083c2 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:41.204853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5fae7-7197-4be3-8a85-2dc8d1a649e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:41.205477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5fae7-7197-4be3-8a85-2dc8d1a649e6 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:41.205743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5fae7-7197-4be3-8a85-2dc8d1a649e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:41.206004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5fae7-7197-4be3-8a85-2dc8d1a649e6 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.206703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5fae7-7197-4be3-8a85-2dc8d1a649e6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.208177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5fae7-7197-4be3-8a85-2dc8d1a649e6 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.215882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.sql.core [None req-b7e5fae7-7197-4be3-8a85-2dc8d1a649e6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 20:33:41.215882 ncdv-master-1 devstack@keystone.service[99100]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 20:33:41.215882 ncdv-master-1 devstack@keystone.service[99100]: [parameters: {'id': 'cb9523807af843988dad1441909988a8', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 20:33:41.215882 ncdv-master-1 devstack@keystone.service[99100]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99100) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-b7e5fae7-7197-4be3-8a85-2dc8d1a649e6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application conn.query(q) Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:33:41.223056 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result.read() Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 20:33:41.223891 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application next(self.gen) Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application yield resource Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application next(self.gen) Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise self.value Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application session.commit() Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 20:33:41.224824 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.session.flush() Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application compat.raise_( Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 20:33:41.225708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 20:33:41.226554 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application conn.query(q) Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result.read() Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 20:33:41.227323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [parameters: {'id': 'cb9523807af843988dad1441909988a8', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 20:33:41.230361 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:41.231308 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 397/802] 10.222.0.16 () {60 vars in 1137 bytes} [Sat Oct 15 20:33:41 2022] POST /identity/v3/domains => generated 190 bytes in 165 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:41.232362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5fae7-7197-4be3-8a85-2dc8d1a649e6 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:41.292885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:41.299807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8bc2a-85dd-4bff-baef-e855794083c2 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:41.328713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5fae7-7197-4be3-8a85-2dc8d1a649e6 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:41.333149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fc228-0623-4d10-9447-8470140dea3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:41.333735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fc228-0623-4d10-9447-8470140dea3b None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:41.334001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fc228-0623-4d10-9447-8470140dea3b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:41.334260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fc228-0623-4d10-9447-8470140dea3b None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.334889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fc228-0623-4d10-9447-8470140dea3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.336277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fc228-0623-4d10-9447-8470140dea3b None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.358451 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 406/803] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:33:41 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:41.359406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fc228-0623-4d10-9447-8470140dea3b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:41.369285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae47f75-885c-457f-8e65-fca661ac6856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:41.369937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae47f75-885c-457f-8e65-fca661ac6856 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:41.370207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae47f75-885c-457f-8e65-fca661ac6856 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:41.370468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae47f75-885c-457f-8e65-fca661ac6856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.390282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae47f75-885c-457f-8e65-fca661ac6856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.391767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae47f75-885c-457f-8e65-fca661ac6856 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-aae47f75-885c-457f-8e65-fca661ac6856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:41.407940 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:41.409062 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:41.410113 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:41.410113 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:41.410113 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:41.414065 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 398/804] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:41.414836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae47f75-885c-457f-8e65-fca661ac6856 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:41.450418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:41.481854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fc228-0623-4d10-9447-8470140dea3b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:41.520718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdddd77-f1db-439c-b052-036d685e5428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:41.521546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdddd77-f1db-439c-b052-036d685e5428 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:41.521828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdddd77-f1db-439c-b052-036d685e5428 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:41.522093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdddd77-f1db-439c-b052-036d685e5428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.531621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae47f75-885c-457f-8e65-fca661ac6856 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:41.533901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdddd77-f1db-439c-b052-036d685e5428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.535404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdddd77-f1db-439c-b052-036d685e5428 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-abdddd77-f1db-439c-b052-036d685e5428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:41.550613 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:41.551871 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:41.552766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:41.552766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:41.552766 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:41.552766 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 407/805] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:41.559565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdddd77-f1db-439c-b052-036d685e5428 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:41.564066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:41.571637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34120292-731b-47b8-8a3c-245bf3ba220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:41.572442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34120292-731b-47b8-8a3c-245bf3ba220a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:41.572598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34120292-731b-47b8-8a3c-245bf3ba220a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:41.572793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34120292-731b-47b8-8a3c-245bf3ba220a None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.573426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34120292-731b-47b8-8a3c-245bf3ba220a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:41.573721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34120292-731b-47b8-8a3c-245bf3ba220a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.575080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34120292-731b-47b8-8a3c-245bf3ba220a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.588715 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 399/806] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:41.594618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34120292-731b-47b8-8a3c-245bf3ba220a None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:41.682029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdddd77-f1db-439c-b052-036d685e5428 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:41.698680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:41.709909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34120292-731b-47b8-8a3c-245bf3ba220a None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:41.718582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d6e13-f4d4-4e7c-9223-a4e9b1493f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:41.719184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d6e13-f4d4-4e7c-9223-a4e9b1493f8e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:41.719466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d6e13-f4d4-4e7c-9223-a4e9b1493f8e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:41.719729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d6e13-f4d4-4e7c-9223-a4e9b1493f8e None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.720359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d6e13-f4d4-4e7c-9223-a4e9b1493f8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:41.720656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d6e13-f4d4-4e7c-9223-a4e9b1493f8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.722042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d6e13-f4d4-4e7c-9223-a4e9b1493f8e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.734694 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 408/807] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:41.757603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d413e-8964-4440-8671-d36803ed53db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:41.758382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d413e-8964-4440-8671-d36803ed53db None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:41.758736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d413e-8964-4440-8671-d36803ed53db None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:41.759086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d413e-8964-4440-8671-d36803ed53db None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.767680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d413e-8964-4440-8671-d36803ed53db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.769189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d413e-8964-4440-8671-d36803ed53db None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-bf6d413e-8964-4440-8671-d36803ed53db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:33:41.782476 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:41.783439 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:41.789496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:33:41.789496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:33:41.789496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:33:41.789496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:33:41.789496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:33:41.789496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:33:41.789496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:41.789496 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:41.789496 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 400/808] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:41.791886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d6e13-f4d4-4e7c-9223-a4e9b1493f8e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:41.865502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:41.909156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d6e13-f4d4-4e7c-9223-a4e9b1493f8e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:41.948431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1a6dc-82a5-4a85-8d26-4ed1cb60f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:41.949308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1a6dc-82a5-4a85-8d26-4ed1cb60f155 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:41.949726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1a6dc-82a5-4a85-8d26-4ed1cb60f155 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:41.950162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1a6dc-82a5-4a85-8d26-4ed1cb60f155 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:41.950989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1a6dc-82a5-4a85-8d26-4ed1cb60f155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:41.951444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1a6dc-82a5-4a85-8d26-4ed1cb60f155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:41.952860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1a6dc-82a5-4a85-8d26-4ed1cb60f155 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:41.965511 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 409/809] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:41.971407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d413e-8964-4440-8671-d36803ed53db None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:42.079904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d413e-8964-4440-8671-d36803ed53db None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:42.115922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74a8cf-f046-4641-b094-8e28ee486d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:42.116819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74a8cf-f046-4641-b094-8e28ee486d64 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:42.117123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74a8cf-f046-4641-b094-8e28ee486d64 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:42.117387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74a8cf-f046-4641-b094-8e28ee486d64 None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/6225c81d2620463886d8a936b59784df/roles/e181e16789c24966b73c9d59d2075672` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:42.119567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74a8cf-f046-4641-b094-8e28ee486d64 None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=6225c81d2620463886d8a936b59784df, role_id=e181e16789c24966b73c9d59d2075672)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:42.120988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74a8cf-f046-4641-b094-8e28ee486d64 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:42.134436 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 401/810] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:33:41 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/6225c81d2620463886d8a936b59784df/roles/e181e16789c24966b73c9d59d2075672 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:33:43.801437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86084404-bc41-44ed-bd97-45354aa05118 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:43.801982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86084404-bc41-44ed-bd97-45354aa05118 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:43.801982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86084404-bc41-44ed-bd97-45354aa05118 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:43.803439 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 410/811] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:43.808310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f18338-f58a-4c50-924a-36a0cb8f9ac2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:43.808709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f18338-f58a-4c50-924a-36a0cb8f9ac2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:43.808843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f18338-f58a-4c50-924a-36a0cb8f9ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:43.827512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-74f18338-f58a-4c50-924a-36a0cb8f9ac2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:43.828666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-74f18338-f58a-4c50-924a-36a0cb8f9ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:43.831382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:43.899351 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 402/812] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:44.224498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86084404-bc41-44ed-bd97-45354aa05118 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:44.325283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86084404-bc41-44ed-bd97-45354aa05118 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:44.349465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:44.360366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51510eb3-63bd-4cde-bea4-72469f69f978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:44.360980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51510eb3-63bd-4cde-bea4-72469f69f978 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:44.361282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51510eb3-63bd-4cde-bea4-72469f69f978 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:44.361552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51510eb3-63bd-4cde-bea4-72469f69f978 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:44.371887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51510eb3-63bd-4cde-bea4-72469f69f978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-51510eb3-63bd-4cde-bea4-72469f69f978 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:44.376622 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:44.377582 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:44.378422 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 411/813] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:33:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:44.384478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f18338-f58a-4c50-924a-36a0cb8f9ac2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:44.482905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f18338-f58a-4c50-924a-36a0cb8f9ac2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:44.507394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb773eca-1c46-42a3-bf00-6669bc89d62e None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:44.507665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb773eca-1c46-42a3-bf00-6669bc89d62e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:44.507926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb773eca-1c46-42a3-bf00-6669bc89d62e None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:44.509261 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 412/814] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:33:44.513994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3fd45-33b4-4a93-aede-5d9e1f76e61b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:44.514263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3fd45-33b4-4a93-aede-5d9e1f76e61b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:44.514530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3fd45-33b4-4a93-aede-5d9e1f76e61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:44.516681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a6880-210e-447d-ba40-ed2d4e577b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:44.517309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a6880-210e-447d-ba40-ed2d4e577b6f None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:44.517581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a6880-210e-447d-ba40-ed2d4e577b6f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:44.517851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a6880-210e-447d-ba40-ed2d4e577b6f None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:44.518501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a6880-210e-447d-ba40-ed2d4e577b6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:44.518794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a6880-210e-447d-ba40-ed2d4e577b6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:44.521359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a6880-210e-447d-ba40-ed2d4e577b6f None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:44.529615 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 403/815] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:33:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:44.532799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a9a3fd45-33b4-4a93-aede-5d9e1f76e61b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:44.534017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a9a3fd45-33b4-4a93-aede-5d9e1f76e61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:44.535817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a6880-210e-447d-ba40-ed2d4e577b6f None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:44.538402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:44.600588 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 413/816] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:33:44.636805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a6880-210e-447d-ba40-ed2d4e577b6f None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:44.669008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14a531-a9de-4ecd-8f7d-99dd989fd15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:44.669641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14a531-a9de-4ecd-8f7d-99dd989fd15d None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:44.669919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14a531-a9de-4ecd-8f7d-99dd989fd15d None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:44.670184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14a531-a9de-4ecd-8f7d-99dd989fd15d None admin] PATH_INFO: `/v3/users/6e961d0c020841c8a3f3b03d93799406` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:44.683498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14a531-a9de-4ecd-8f7d-99dd989fd15d None admin] RBAC: Authorizing `identity:get_user(user_id=6e961d0c020841c8a3f3b03d93799406)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:44.684921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14a531-a9de-4ecd-8f7d-99dd989fd15d None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:44.686532 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 404/817] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:33:44 2022] GET /identity/v3/users/6e961d0c020841c8a3f3b03d93799406 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:44.692923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3fd45-33b4-4a93-aede-5d9e1f76e61b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:44.788255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3fd45-33b4-4a93-aede-5d9e1f76e61b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:44.802930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:44.823406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90302205-48b8-47d9-ade9-37c0e393e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:44.824049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90302205-48b8-47d9-ade9-37c0e393e7ae None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:44.824322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90302205-48b8-47d9-ade9-37c0e393e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:44.824589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90302205-48b8-47d9-ade9-37c0e393e7ae None admin] PATH_INFO: `/v3/projects/b54ee3887c794301936349de95d2df39` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:44.825700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90302205-48b8-47d9-ade9-37c0e393e7ae None admin] RBAC: Authorizing `identity:get_project(project_id=b54ee3887c794301936349de95d2df39)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:44.827103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90302205-48b8-47d9-ade9-37c0e393e7ae None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:44.828881 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 414/818] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:33:44 2022] GET /identity/v3/projects/b54ee3887c794301936349de95d2df39 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:44.835097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14a531-a9de-4ecd-8f7d-99dd989fd15d None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:44.894033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:44.923147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90302205-48b8-47d9-ade9-37c0e393e7ae None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:44.934574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14a531-a9de-4ecd-8f7d-99dd989fd15d None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:44.968993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c88dc-3e7a-4453-ad47-a72df378adc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:44.969666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c88dc-3e7a-4453-ad47-a72df378adc3 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:44.969934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c88dc-3e7a-4453-ad47-a72df378adc3 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:44.970200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c88dc-3e7a-4453-ad47-a72df378adc3 None admin] PATH_INFO: `/v3/projects/b54ee3887c794301936349de95d2df39/users/6e961d0c020841c8a3f3b03d93799406/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:44.972071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c88dc-3e7a-4453-ad47-a72df378adc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b54ee3887c794301936349de95d2df39, user_id=6e961d0c020841c8a3f3b03d93799406, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:44.973538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c88dc-3e7a-4453-ad47-a72df378adc3 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:44.986065 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 405/819] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:33:44 2022] PUT /identity/v3/projects/b54ee3887c794301936349de95d2df39/users/6e961d0c020841c8a3f3b03d93799406/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:33:45.034658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90302205-48b8-47d9-ade9-37c0e393e7ae None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:45.070692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd5bad-756e-4270-943b-57f2baa0ccc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:45.071224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd5bad-756e-4270-943b-57f2baa0ccc6 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:45.071500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd5bad-756e-4270-943b-57f2baa0ccc6 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:45.071761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd5bad-756e-4270-943b-57f2baa0ccc6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:45.083731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd5bad-756e-4270-943b-57f2baa0ccc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-37bd5bad-756e-4270-943b-57f2baa0ccc6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:45.088462 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:45.089544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:45.090348 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 415/820] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Oct 15 20:33:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:45.095883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c88dc-3e7a-4453-ad47-a72df378adc3 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:45.209157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c88dc-3e7a-4453-ad47-a72df378adc3 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:45.230823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:45.246880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f15a7a-b014-4021-939a-53c6b15ddbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:45.247473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f15a7a-b014-4021-939a-53c6b15ddbe1 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:45.247743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f15a7a-b014-4021-939a-53c6b15ddbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:45.248002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f15a7a-b014-4021-939a-53c6b15ddbe1 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:45.248649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f15a7a-b014-4021-939a-53c6b15ddbe1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:45.248940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f15a7a-b014-4021-939a-53c6b15ddbe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:45.250335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f15a7a-b014-4021-939a-53c6b15ddbe1 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:45.258474 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 406/821] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Oct 15 20:33:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:45.264147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd5bad-756e-4270-943b-57f2baa0ccc6 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:45.307168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:45.362898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd5bad-756e-4270-943b-57f2baa0ccc6 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:45.396369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c29e9-df64-48b6-ae74-a63eb08850b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:45.396978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c29e9-df64-48b6-ae74-a63eb08850b4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:45.397280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c29e9-df64-48b6-ae74-a63eb08850b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:45.397544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c29e9-df64-48b6-ae74-a63eb08850b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:45.407699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c29e9-df64-48b6-ae74-a63eb08850b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:45.409133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c29e9-df64-48b6-ae74-a63eb08850b4 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-695c29e9-df64-48b6-ae74-a63eb08850b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:45.422548 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:45.423455 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:45.429793 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:45.429793 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:45.429793 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:45.429793 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 416/822] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:33:45.430139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f15a7a-b014-4021-939a-53c6b15ddbe1 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:45.528113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f15a7a-b014-4021-939a-53c6b15ddbe1 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:45.560702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb45c17-bbf6-4734-b96a-99ab612181e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:45.561317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45c17-bbf6-4734-b96a-99ab612181e8 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:45.561584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45c17-bbf6-4734-b96a-99ab612181e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:45.561841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45c17-bbf6-4734-b96a-99ab612181e8 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:45.562464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb45c17-bbf6-4734-b96a-99ab612181e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:45.562753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb45c17-bbf6-4734-b96a-99ab612181e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:45.564103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb45c17-bbf6-4734-b96a-99ab612181e8 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:45.575545 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 407/823] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:45.581167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c29e9-df64-48b6-ae74-a63eb08850b4 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:45.673887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:45.681109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c29e9-df64-48b6-ae74-a63eb08850b4 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:45.713889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a2e05-6299-4ef5-af39-c6292ca0af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:45.714495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a2e05-6299-4ef5-af39-c6292ca0af07 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:45.714764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a2e05-6299-4ef5-af39-c6292ca0af07 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:45.715026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a2e05-6299-4ef5-af39-c6292ca0af07 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:45.722262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a2e05-6299-4ef5-af39-c6292ca0af07 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:45.724362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a2e05-6299-4ef5-af39-c6292ca0af07 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-054a2e05-6299-4ef5-af39-c6292ca0af07 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:33:45.735207 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:45.736109 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:45.737015 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 417/824] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Oct 15 20:33:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:45.742471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb45c17-bbf6-4734-b96a-99ab612181e8 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:45.786202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:45.844603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb45c17-bbf6-4734-b96a-99ab612181e8 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:45.877247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e65dd-a792-48ab-8b10-eba1979598cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:45.877834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e65dd-a792-48ab-8b10-eba1979598cb None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:45.878106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e65dd-a792-48ab-8b10-eba1979598cb None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:45.878364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e65dd-a792-48ab-8b10-eba1979598cb None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:45.879592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e65dd-a792-48ab-8b10-eba1979598cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:45.879888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e65dd-a792-48ab-8b10-eba1979598cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:45.881332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e65dd-a792-48ab-8b10-eba1979598cb None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:45.898605 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 408/825] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Oct 15 20:33:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:45.904330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a2e05-6299-4ef5-af39-c6292ca0af07 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:46.000362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a2e05-6299-4ef5-af39-c6292ca0af07 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:46.031624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede790ec-f6f0-4c52-8c34-fed16ba2fc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:46.032202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede790ec-f6f0-4c52-8c34-fed16ba2fc68 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:46.032470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede790ec-f6f0-4c52-8c34-fed16ba2fc68 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:46.032725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede790ec-f6f0-4c52-8c34-fed16ba2fc68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:46.057637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:46.101115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede790ec-f6f0-4c52-8c34-fed16ba2fc68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:46.102541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede790ec-f6f0-4c52-8c34-fed16ba2fc68 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:46.198981 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 418/826] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:33:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 20:33:46.204521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e65dd-a792-48ab-8b10-eba1979598cb None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:46.301142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:46.304294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e65dd-a792-48ab-8b10-eba1979598cb None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:46.338011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69129dc4-4e8d-4cb7-8cab-5c4f2f629009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:46.338621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69129dc4-4e8d-4cb7-8cab-5c4f2f629009 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:46.338891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69129dc4-4e8d-4cb7-8cab-5c4f2f629009 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:46.339152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69129dc4-4e8d-4cb7-8cab-5c4f2f629009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:46.348957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69129dc4-4e8d-4cb7-8cab-5c4f2f629009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:46.350429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69129dc4-4e8d-4cb7-8cab-5c4f2f629009 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-69129dc4-4e8d-4cb7-8cab-5c4f2f629009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:33:46.365543 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:46.366819 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:33:46.367665 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:33:46.367665 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:33:46.367665 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:46.367665 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 409/827] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:46.372787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede790ec-f6f0-4c52-8c34-fed16ba2fc68 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:46.427164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:46.472804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede790ec-f6f0-4c52-8c34-fed16ba2fc68 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:46.506433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459afba-37c4-4663-a5f3-112a1b9121b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:46.507013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459afba-37c4-4663-a5f3-112a1b9121b6 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:46.507281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459afba-37c4-4663-a5f3-112a1b9121b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:46.507545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459afba-37c4-4663-a5f3-112a1b9121b6 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:46.508165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459afba-37c4-4663-a5f3-112a1b9121b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:46.508459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459afba-37c4-4663-a5f3-112a1b9121b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:46.509840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459afba-37c4-4663-a5f3-112a1b9121b6 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:46.521848 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 419/828] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:46.527493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69129dc4-4e8d-4cb7-8cab-5c4f2f629009 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:46.626045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69129dc4-4e8d-4cb7-8cab-5c4f2f629009 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:46.658734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c41b54-5aff-489c-b0a4-08f4196d8a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:46.659341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c41b54-5aff-489c-b0a4-08f4196d8a0e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:46.659610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c41b54-5aff-489c-b0a4-08f4196d8a0e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:46.660524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c41b54-5aff-489c-b0a4-08f4196d8a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:46.667020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c41b54-5aff-489c-b0a4-08f4196d8a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:46.668526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c41b54-5aff-489c-b0a4-08f4196d8a0e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-20c41b54-5aff-489c-b0a4-08f4196d8a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:33:46.679992 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:46.681009 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:46.681931 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:33:46.681931 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:33:46.681931 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:33:46.681931 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:33:46.681931 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:33:46.681931 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:33:46.681931 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:33:46.681931 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:33:46.681931 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 410/829] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:33:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:46.686859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459afba-37c4-4663-a5f3-112a1b9121b6 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:46.783680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459afba-37c4-4663-a5f3-112a1b9121b6 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:46.797705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:46.818815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958c91b-e41e-490c-9f5b-78b2ae4e9ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:46.819408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958c91b-e41e-490c-9f5b-78b2ae4e9ed1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:46.819681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958c91b-e41e-490c-9f5b-78b2ae4e9ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:46.819953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958c91b-e41e-490c-9f5b-78b2ae4e9ed1 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:46.820636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958c91b-e41e-490c-9f5b-78b2ae4e9ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:46.820934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958c91b-e41e-490c-9f5b-78b2ae4e9ed1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:46.822376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958c91b-e41e-490c-9f5b-78b2ae4e9ed1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:46.834691 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 420/830] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:33:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:46.840593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c41b54-5aff-489c-b0a4-08f4196d8a0e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:46.856338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:46.941990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c41b54-5aff-489c-b0a4-08f4196d8a0e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:46.976546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cb280-270a-4631-bca2-56a85048678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:46.977178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cb280-270a-4631-bca2-56a85048678a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:46.977450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cb280-270a-4631-bca2-56a85048678a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:46.977713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cb280-270a-4631-bca2-56a85048678a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:46.978442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cb280-270a-4631-bca2-56a85048678a None admin] RBAC: Adding query filter params (role.id=e181e16789c24966b73c9d59d2075672, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=6225c81d2620463886d8a936b59784df) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:46.978740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cb280-270a-4631-bca2-56a85048678a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:46.980250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cb280-270a-4631-bca2-56a85048678a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:46.986913 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 411/831] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:33:46 2022] GET /identity/v3/role_assignments?user.id=6225c81d2620463886d8a936b59784df&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=e181e16789c24966b73c9d59d2075672 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:47.434379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e6bf8-8b27-4389-a7fe-b309aaa4abe0 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:47.434884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e6bf8-8b27-4389-a7fe-b309aaa4abe0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:47.434884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e6bf8-8b27-4389-a7fe-b309aaa4abe0 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:47.436216 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 421/832] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:47.441729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16d854-c441-4b13-8e73-40b727b29c71 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:47.441876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16d854-c441-4b13-8e73-40b727b29c71 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:47.442141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16d854-c441-4b13-8e73-40b727b29c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:47.460570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cd16d854-c441-4b13-8e73-40b727b29c71 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:47.461698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cd16d854-c441-4b13-8e73-40b727b29c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:47.535378 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 412/833] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:47.858585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e6bf8-8b27-4389-a7fe-b309aaa4abe0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:47.957662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e6bf8-8b27-4389-a7fe-b309aaa4abe0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:47.990166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94181c-d1a2-4f93-9bef-25c37826a495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:47.990766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94181c-d1a2-4f93-9bef-25c37826a495 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:47.991279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94181c-d1a2-4f93-9bef-25c37826a495 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:47.991538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94181c-d1a2-4f93-9bef-25c37826a495 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:48.001225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94181c-d1a2-4f93-9bef-25c37826a495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-cc94181c-d1a2-4f93-9bef-25c37826a495 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:33:48.006509 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:33:48.007570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:33:48.008570 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 422/834] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:33:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:33:48.013713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16d854-c441-4b13-8e73-40b727b29c71 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:48.022265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:48.109962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16d854-c441-4b13-8e73-40b727b29c71 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:48.142286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c7c4e-8a4b-4d29-b66a-280cf9ee7b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:48.142861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c7c4e-8a4b-4d29-b66a-280cf9ee7b84 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:48.143129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c7c4e-8a4b-4d29-b66a-280cf9ee7b84 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:48.143391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c7c4e-8a4b-4d29-b66a-280cf9ee7b84 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:48.144657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c7c4e-8a4b-4d29-b66a-280cf9ee7b84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:48.145005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c7c4e-8a4b-4d29-b66a-280cf9ee7b84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:48.146382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c7c4e-8a4b-4d29-b66a-280cf9ee7b84 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:48.154006 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 413/835] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:33:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:48.159976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94181c-d1a2-4f93-9bef-25c37826a495 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:48.194126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:48.258511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94181c-d1a2-4f93-9bef-25c37826a495 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:48.291334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be23fb-25e2-488f-9b53-07bed19d3862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:48.291974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be23fb-25e2-488f-9b53-07bed19d3862 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:48.292248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be23fb-25e2-488f-9b53-07bed19d3862 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:48.292512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be23fb-25e2-488f-9b53-07bed19d3862 None admin] PATH_INFO: `/v3/users/6e961d0c020841c8a3f3b03d93799406` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:48.293633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be23fb-25e2-488f-9b53-07bed19d3862 None admin] RBAC: Authorizing `identity:get_user(user_id=6e961d0c020841c8a3f3b03d93799406)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:48.295049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be23fb-25e2-488f-9b53-07bed19d3862 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:48.296600 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 423/836] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:33:48 2022] GET /identity/v3/users/6e961d0c020841c8a3f3b03d93799406 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:48.302668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c7c4e-8a4b-4d29-b66a-280cf9ee7b84 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:48.366828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:48.400523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c7c4e-8a4b-4d29-b66a-280cf9ee7b84 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:48.433010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326834fc-330e-48c8-b8d5-eab5b03d53e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:48.433621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326834fc-330e-48c8-b8d5-eab5b03d53e8 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:48.433888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326834fc-330e-48c8-b8d5-eab5b03d53e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:48.434148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326834fc-330e-48c8-b8d5-eab5b03d53e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:48.499097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326834fc-330e-48c8-b8d5-eab5b03d53e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:48.500519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326834fc-330e-48c8-b8d5-eab5b03d53e8 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:48.561891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:48.594699 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 414/837] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:33:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 20:33:48.600634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be23fb-25e2-488f-9b53-07bed19d3862 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:48.698237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be23fb-25e2-488f-9b53-07bed19d3862 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:48.699682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:48.731162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a457f26-7995-4df6-bad1-ac98d805cebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:48.731757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a457f26-7995-4df6-bad1-ac98d805cebd None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:48.732026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a457f26-7995-4df6-bad1-ac98d805cebd None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:48.732286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a457f26-7995-4df6-bad1-ac98d805cebd None admin] PATH_INFO: `/v3/projects/b54ee3887c794301936349de95d2df39` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:48.733390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a457f26-7995-4df6-bad1-ac98d805cebd None admin] RBAC: Authorizing `identity:get_project(project_id=b54ee3887c794301936349de95d2df39)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:48.734785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a457f26-7995-4df6-bad1-ac98d805cebd None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:48.736359 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 424/838] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:33:48 2022] GET /identity/v3/projects/b54ee3887c794301936349de95d2df39 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:48.743419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326834fc-330e-48c8-b8d5-eab5b03d53e8 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:48.841235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326834fc-330e-48c8-b8d5-eab5b03d53e8 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:48.873640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee608b1-c061-425c-a06e-fa044d73989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:48.874218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee608b1-c061-425c-a06e-fa044d73989e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:48.874489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee608b1-c061-425c-a06e-fa044d73989e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:48.874747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee608b1-c061-425c-a06e-fa044d73989e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:48.875448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee608b1-c061-425c-a06e-fa044d73989e None admin] RBAC: Adding query filter params (role.id=fab66a1a6ba64c9583182c78978b218e, scope.project.id=b54ee3887c794301936349de95d2df39, user.id=6e961d0c020841c8a3f3b03d93799406) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:48.875751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee608b1-c061-425c-a06e-fa044d73989e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:48.877194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee608b1-c061-425c-a06e-fa044d73989e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:48.883287 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 415/839] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:33:48 2022] GET /identity/v3/role_assignments?user.id=6e961d0c020841c8a3f3b03d93799406&scope.project.id=b54ee3887c794301936349de95d2df39&role.id=fab66a1a6ba64c9583182c78978b218e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:33:51.267117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3bc93-699d-4f3e-8633-9f591a5fe74d None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:51.267626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3bc93-699d-4f3e-8633-9f591a5fe74d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:51.267626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3bc93-699d-4f3e-8633-9f591a5fe74d None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:51.269137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 425/840] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:51.274336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f324f-05a6-421d-b211-029f54c86c3d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:51.274732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f324f-05a6-421d-b211-029f54c86c3d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:51.274878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f324f-05a6-421d-b211-029f54c86c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:51.293316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1e6f324f-05a6-421d-b211-029f54c86c3d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:51.294495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e6f324f-05a6-421d-b211-029f54c86c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:51.320349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:51.360726 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 416/841] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:51.683819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3bc93-699d-4f3e-8633-9f591a5fe74d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:51.780242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3bc93-699d-4f3e-8633-9f591a5fe74d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:51.814272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da445603-39ee-4918-a604-ba8ea293ed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:51.814874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da445603-39ee-4918-a604-ba8ea293ed12 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:51.815145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da445603-39ee-4918-a604-ba8ea293ed12 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:51.815408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da445603-39ee-4918-a604-ba8ea293ed12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:51.816735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da445603-39ee-4918-a604-ba8ea293ed12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:51.818180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da445603-39ee-4918-a604-ba8ea293ed12 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:51.819707 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 426/842] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:33:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:51.825407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f324f-05a6-421d-b211-029f54c86c3d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:51.924623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f324f-05a6-421d-b211-029f54c86c3d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:51.958838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d556d9-9708-486c-a203-c50f7a960708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:51.959354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d556d9-9708-486c-a203-c50f7a960708 None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:51.959626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d556d9-9708-486c-a203-c50f7a960708 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:51.959890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d556d9-9708-486c-a203-c50f7a960708 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:51.960528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d556d9-9708-486c-a203-c50f7a960708 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:51.961984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d556d9-9708-486c-a203-c50f7a960708 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:51.982976 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 417/843] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:33:51 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:33:54.403197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74ef92-7ffd-4366-a152-dfb9806cd7da None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:54.403751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74ef92-7ffd-4366-a152-dfb9806cd7da None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:54.403751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74ef92-7ffd-4366-a152-dfb9806cd7da None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:54.405030 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 427/844] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:54.410309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd4dca-d22c-4566-ad7b-66109108d4b1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:54.410686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd4dca-d22c-4566-ad7b-66109108d4b1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:54.410849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd4dca-d22c-4566-ad7b-66109108d4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:54.429098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-56dd4dca-d22c-4566-ad7b-66109108d4b1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:54.430276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-56dd4dca-d22c-4566-ad7b-66109108d4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:54.443995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:54.497613 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 418/845] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:54.822087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74ef92-7ffd-4366-a152-dfb9806cd7da None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:54.864011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:54.922662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74ef92-7ffd-4366-a152-dfb9806cd7da None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:54.955457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3db711-758a-4aed-b9e7-46e5ea66ce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:54.956055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3db711-758a-4aed-b9e7-46e5ea66ce02 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:54.956321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3db711-758a-4aed-b9e7-46e5ea66ce02 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:54.956578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3db711-758a-4aed-b9e7-46e5ea66ce02 None admin] PATH_INFO: `/v3/roles/25b2c5582f194703afe45b5d8cc79171` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:54.957934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3db711-758a-4aed-b9e7-46e5ea66ce02 None admin] RBAC: Authorizing `identity:get_role(role_id=25b2c5582f194703afe45b5d8cc79171)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:54.959259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3db711-758a-4aed-b9e7-46e5ea66ce02 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:54.960543 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 428/846] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:33:54 2022] GET /identity/v3/roles/25b2c5582f194703afe45b5d8cc79171 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:54.966340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd4dca-d22c-4566-ad7b-66109108d4b1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:55.062179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd4dca-d22c-4566-ad7b-66109108d4b1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:55.095127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872be29-1ebc-4e5d-b99d-ff5399e6383e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:55.095668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872be29-1ebc-4e5d-b99d-ff5399e6383e None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:55.095934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872be29-1ebc-4e5d-b99d-ff5399e6383e None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:55.096195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872be29-1ebc-4e5d-b99d-ff5399e6383e None admin] PATH_INFO: `/v3/users/1849b5501fba4649899ab6606c5e3b61` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:55.109234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872be29-1ebc-4e5d-b99d-ff5399e6383e None admin] RBAC: Authorizing `identity:get_user(user_id=1849b5501fba4649899ab6606c5e3b61)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:55.110657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872be29-1ebc-4e5d-b99d-ff5399e6383e None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:55.112974 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 419/847] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:33:54 2022] GET /identity/v3/users/1849b5501fba4649899ab6606c5e3b61 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:55.118864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3db711-758a-4aed-b9e7-46e5ea66ce02 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:55.216385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3db711-758a-4aed-b9e7-46e5ea66ce02 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:55.224417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:55.251423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247716d2-2353-4d7b-879c-13629d293e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:55.252040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247716d2-2353-4d7b-879c-13629d293e89 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:55.252311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247716d2-2353-4d7b-879c-13629d293e89 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:55.252577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247716d2-2353-4d7b-879c-13629d293e89 None admin] PATH_INFO: `/v3/projects/b54ee3887c794301936349de95d2df39` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:55.253709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247716d2-2353-4d7b-879c-13629d293e89 None admin] RBAC: Authorizing `identity:get_project(project_id=b54ee3887c794301936349de95d2df39)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:55.255124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247716d2-2353-4d7b-879c-13629d293e89 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:55.256737 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 429/848] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:33:55 2022] GET /identity/v3/projects/b54ee3887c794301936349de95d2df39 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:55.262757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872be29-1ebc-4e5d-b99d-ff5399e6383e None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:55.271932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:55.359554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872be29-1ebc-4e5d-b99d-ff5399e6383e None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:55.391655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638982f2-9f5a-4e2b-8620-344dcb9c33da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:55.392274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638982f2-9f5a-4e2b-8620-344dcb9c33da None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:55.392542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638982f2-9f5a-4e2b-8620-344dcb9c33da None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:55.392802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638982f2-9f5a-4e2b-8620-344dcb9c33da None admin] PATH_INFO: `/v3/projects/b54ee3887c794301936349de95d2df39/users/1849b5501fba4649899ab6606c5e3b61/roles/25b2c5582f194703afe45b5d8cc79171` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:55.395492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638982f2-9f5a-4e2b-8620-344dcb9c33da None admin] RBAC: Authorizing `identity:create_grant(project_id=b54ee3887c794301936349de95d2df39, user_id=1849b5501fba4649899ab6606c5e3b61, role_id=25b2c5582f194703afe45b5d8cc79171)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:55.396905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638982f2-9f5a-4e2b-8620-344dcb9c33da None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:55.409848 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 420/849] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:33:55 2022] PUT /identity/v3/projects/b54ee3887c794301936349de95d2df39/users/1849b5501fba4649899ab6606c5e3b61/roles/25b2c5582f194703afe45b5d8cc79171 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:33:57.787022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a770cf-05f7-4c84-b3fe-e52b518136d1 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:57.787652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a770cf-05f7-4c84-b3fe-e52b518136d1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:57.787652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a770cf-05f7-4c84-b3fe-e52b518136d1 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:57.788840 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 430/850] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:33:57.794115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a8c08-27ce-428c-8f2c-b70e5ceb61df None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:57.794518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a8c08-27ce-428c-8f2c-b70e5ceb61df None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:57.794644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a8c08-27ce-428c-8f2c-b70e5ceb61df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:57.812895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-525a8c08-27ce-428c-8f2c-b70e5ceb61df None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:33:57.814082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-525a8c08-27ce-428c-8f2c-b70e5ceb61df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:33:57.873247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:57.882072 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 421/851] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:33:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:33:58.205903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a770cf-05f7-4c84-b3fe-e52b518136d1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:58.303649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a770cf-05f7-4c84-b3fe-e52b518136d1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:58.336132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4515bd1-3352-492a-8654-26fbfb7cd8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:58.336726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515bd1-3352-492a-8654-26fbfb7cd8ee None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:58.336996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515bd1-3352-492a-8654-26fbfb7cd8ee None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:58.337287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515bd1-3352-492a-8654-26fbfb7cd8ee None admin] PATH_INFO: `/v3/roles/25b2c5582f194703afe45b5d8cc79171` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:58.338640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4515bd1-3352-492a-8654-26fbfb7cd8ee None admin] RBAC: Authorizing `identity:get_role(role_id=25b2c5582f194703afe45b5d8cc79171)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:58.340002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4515bd1-3352-492a-8654-26fbfb7cd8ee None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:58.341342 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 431/852] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:33:58 2022] GET /identity/v3/roles/25b2c5582f194703afe45b5d8cc79171 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:58.347008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a8c08-27ce-428c-8f2c-b70e5ceb61df None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:58.442935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a8c08-27ce-428c-8f2c-b70e5ceb61df None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:58.475807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788dc7f-653f-4234-942a-63caa8b2e860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:58.476349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788dc7f-653f-4234-942a-63caa8b2e860 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:58.476616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788dc7f-653f-4234-942a-63caa8b2e860 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:58.476874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788dc7f-653f-4234-942a-63caa8b2e860 None admin] PATH_INFO: `/v3/users/1849b5501fba4649899ab6606c5e3b61` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:58.478000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788dc7f-653f-4234-942a-63caa8b2e860 None admin] RBAC: Authorizing `identity:get_user(user_id=1849b5501fba4649899ab6606c5e3b61)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:58.479409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788dc7f-653f-4234-942a-63caa8b2e860 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:58.480946 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 422/853] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:33:58 2022] GET /identity/v3/users/1849b5501fba4649899ab6606c5e3b61 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:58.486645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4515bd1-3352-492a-8654-26fbfb7cd8ee None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:58.529462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:58.584863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4515bd1-3352-492a-8654-26fbfb7cd8ee None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:58.618529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d0318-7175-419f-a1f7-c730c515186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:58.619115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d0318-7175-419f-a1f7-c730c515186f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:58.619389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d0318-7175-419f-a1f7-c730c515186f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:58.619658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d0318-7175-419f-a1f7-c730c515186f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:58.684419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d0318-7175-419f-a1f7-c730c515186f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:58.685865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d0318-7175-419f-a1f7-c730c515186f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:58.726548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:58.782652 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 432/854] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:33:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 20:33:58.789469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788dc7f-653f-4234-942a-63caa8b2e860 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:58.849521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:33:58.886660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788dc7f-653f-4234-942a-63caa8b2e860 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:58.920933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc08d-e673-46e5-914d-e72b9a6615f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:58.921564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc08d-e673-46e5-914d-e72b9a6615f3 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:58.921829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc08d-e673-46e5-914d-e72b9a6615f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:58.922092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc08d-e673-46e5-914d-e72b9a6615f3 None admin] PATH_INFO: `/v3/projects/b54ee3887c794301936349de95d2df39` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:58.923178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dc08d-e673-46e5-914d-e72b9a6615f3 None admin] RBAC: Authorizing `identity:get_project(project_id=b54ee3887c794301936349de95d2df39)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:58.924703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dc08d-e673-46e5-914d-e72b9a6615f3 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:58.926406 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 423/855] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:33:58 2022] GET /identity/v3/projects/b54ee3887c794301936349de95d2df39 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:33:58.932685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d0318-7175-419f-a1f7-c730c515186f None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:33:59.029111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d0318-7175-419f-a1f7-c730c515186f None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:33:59.062195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa743d-96d1-40c7-a4b0-926c03f88fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:33:59.062773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa743d-96d1-40c7-a4b0-926c03f88fbb None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:33:59.063045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa743d-96d1-40c7-a4b0-926c03f88fbb None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:33:59.063308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa743d-96d1-40c7-a4b0-926c03f88fbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:33:59.064014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa743d-96d1-40c7-a4b0-926c03f88fbb None admin] RBAC: Adding query filter params (role.id=25b2c5582f194703afe45b5d8cc79171, scope.project.id=b54ee3887c794301936349de95d2df39, user.id=1849b5501fba4649899ab6606c5e3b61) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:33:59.064311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa743d-96d1-40c7-a4b0-926c03f88fbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:33:59.066184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa743d-96d1-40c7-a4b0-926c03f88fbb None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:33:59.071667 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 433/856] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:33:58 2022] GET /identity/v3/role_assignments?user.id=1849b5501fba4649899ab6606c5e3b61&scope.project.id=b54ee3887c794301936349de95d2df39&role.id=25b2c5582f194703afe45b5d8cc79171 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:01.454615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7479a6-bf81-48b7-8b8e-29fed9a3def2 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:01.455127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7479a6-bf81-48b7-8b8e-29fed9a3def2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:01.455127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7479a6-bf81-48b7-8b8e-29fed9a3def2 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:01.456653 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 424/857] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:34:01.461800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d3adf-2bb6-47fd-9d54-c63576500e8d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:01.462176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d3adf-2bb6-47fd-9d54-c63576500e8d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:01.462334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d3adf-2bb6-47fd-9d54-c63576500e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:01.475617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:01.480913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-767d3adf-2bb6-47fd-9d54-c63576500e8d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:34:01.482126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-767d3adf-2bb6-47fd-9d54-c63576500e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:34:01.546312 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 434/858] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:34:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:34:01.869849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7479a6-bf81-48b7-8b8e-29fed9a3def2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:01.972356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7479a6-bf81-48b7-8b8e-29fed9a3def2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:02.000817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:02.006623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f41c8a-a827-4290-beaf-8edbff4ecf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:02.007228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f41c8a-a827-4290-beaf-8edbff4ecf98 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:02.007521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f41c8a-a827-4290-beaf-8edbff4ecf98 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:02.007782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f41c8a-a827-4290-beaf-8edbff4ecf98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:02.008913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f41c8a-a827-4290-beaf-8edbff4ecf98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:02.010374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f41c8a-a827-4290-beaf-8edbff4ecf98 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:02.011928 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 425/859] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:34:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:02.017871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d3adf-2bb6-47fd-9d54-c63576500e8d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:02.113881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d3adf-2bb6-47fd-9d54-c63576500e8d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:02.146852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c45a9-2e64-4d74-8e94-9f588171d77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:02.147442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c45a9-2e64-4d74-8e94-9f588171d77d None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:02.147715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c45a9-2e64-4d74-8e94-9f588171d77d None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:02.147981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c45a9-2e64-4d74-8e94-9f588171d77d None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:02.148620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c45a9-2e64-4d74-8e94-9f588171d77d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:02.150057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c45a9-2e64-4d74-8e94-9f588171d77d None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:02.155228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:02.176225 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 435/860] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:34:02 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:34:04.578922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68a7a6-dc69-42a5-b5cc-8cf247f6c28f None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:04.579441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68a7a6-dc69-42a5-b5cc-8cf247f6c28f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:04.579441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68a7a6-dc69-42a5-b5cc-8cf247f6c28f None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:04.581264 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 426/861] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:34:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:34:04.586518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d684a8-fd52-40d1-b8b2-87bb4f8876df None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:04.586942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d684a8-fd52-40d1-b8b2-87bb4f8876df None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:04.587041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d684a8-fd52-40d1-b8b2-87bb4f8876df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:04.605168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b4d684a8-fd52-40d1-b8b2-87bb4f8876df None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:34:04.606325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b4d684a8-fd52-40d1-b8b2-87bb4f8876df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:34:04.670135 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 436/862] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:34:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:34:04.997283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68a7a6-dc69-42a5-b5cc-8cf247f6c28f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:05.095364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68a7a6-dc69-42a5-b5cc-8cf247f6c28f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:05.128525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a3fb5-114b-4acf-a933-2f995899994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:05.129172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3fb5-114b-4acf-a933-2f995899994a None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:05.129448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3fb5-114b-4acf-a933-2f995899994a None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:05.129714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3fb5-114b-4acf-a933-2f995899994a None admin] PATH_INFO: `/v3/domains/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:05.130806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a3fb5-114b-4acf-a933-2f995899994a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:05.132240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a3fb5-114b-4acf-a933-2f995899994a None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:05.133815 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 427/863] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:34:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:05.139514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d684a8-fd52-40d1-b8b2-87bb4f8876df None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:05.238226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d684a8-fd52-40d1-b8b2-87bb4f8876df None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:05.255659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:05.272940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfc54e-a54c-4e52-9764-06e971fd432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:05.273629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfc54e-a54c-4e52-9764-06e971fd432f None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:05.273901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfc54e-a54c-4e52-9764-06e971fd432f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:05.274162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfc54e-a54c-4e52-9764-06e971fd432f None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:05.274797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfc54e-a54c-4e52-9764-06e971fd432f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:05.276196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfc54e-a54c-4e52-9764-06e971fd432f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:05.299005 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 437/864] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:34:05 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:34:07.681681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d22f9-f145-401f-92f6-a76af83eab23 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:07.682198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d22f9-f145-401f-92f6-a76af83eab23 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:07.682198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d22f9-f145-401f-92f6-a76af83eab23 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:07.683540 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 428/865] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:34:07.688764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18111e70-2813-4d71-b5d1-a52236a98108 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:07.689199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18111e70-2813-4d71-b5d1-a52236a98108 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:07.689433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18111e70-2813-4d71-b5d1-a52236a98108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:07.708269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-18111e70-2813-4d71-b5d1-a52236a98108 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:34:07.709469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-18111e70-2813-4d71-b5d1-a52236a98108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:34:07.772728 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 438/866] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:34:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:34:08.102250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d22f9-f145-401f-92f6-a76af83eab23 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:08.169656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:08.207596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d22f9-f145-401f-92f6-a76af83eab23 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:08.242517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef1d0b-db54-4991-a7a1-e6c94b1e5710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:08.243136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef1d0b-db54-4991-a7a1-e6c94b1e5710 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:08.243409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef1d0b-db54-4991-a7a1-e6c94b1e5710 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:08.243672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef1d0b-db54-4991-a7a1-e6c94b1e5710 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:08.254072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef1d0b-db54-4991-a7a1-e6c94b1e5710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-ddef1d0b-db54-4991-a7a1-e6c94b1e5710 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:34:08.258739 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:34:08.259827 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:34:08.261023 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 429/867] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:34:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:34:08.265950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18111e70-2813-4d71-b5d1-a52236a98108 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:08.360372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18111e70-2813-4d71-b5d1-a52236a98108 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:08.363154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:08.394112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24a9a5-ba16-4a7b-8053-93c4813d01c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:08.394697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24a9a5-ba16-4a7b-8053-93c4813d01c5 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:08.394968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24a9a5-ba16-4a7b-8053-93c4813d01c5 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:08.395235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24a9a5-ba16-4a7b-8053-93c4813d01c5 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:08.395881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24a9a5-ba16-4a7b-8053-93c4813d01c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:34:08.396179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24a9a5-ba16-4a7b-8053-93c4813d01c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:08.397583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24a9a5-ba16-4a7b-8053-93c4813d01c5 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:08.405319 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 439/868] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:34:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:08.411663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef1d0b-db54-4991-a7a1-e6c94b1e5710 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:08.508064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef1d0b-db54-4991-a7a1-e6c94b1e5710 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:08.534068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:08.542304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8a01d-31f7-418b-a58f-a136d4a4b9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:08.542913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8a01d-31f7-418b-a58f-a136d4a4b9c1 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:08.543187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8a01d-31f7-418b-a58f-a136d4a4b9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:08.543457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8a01d-31f7-418b-a58f-a136d4a4b9c1 None admin] PATH_INFO: `/v3/users/1b8c0e53102f4c698f075f69eb098624` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:08.557417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8a01d-31f7-418b-a58f-a136d4a4b9c1 None admin] RBAC: Authorizing `identity:get_user(user_id=1b8c0e53102f4c698f075f69eb098624)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:08.558855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8a01d-31f7-418b-a58f-a136d4a4b9c1 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:08.560424 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 430/869] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:34:08 2022] GET /identity/v3/users/1b8c0e53102f4c698f075f69eb098624 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:08.566388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24a9a5-ba16-4a7b-8053-93c4813d01c5 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:08.662912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24a9a5-ba16-4a7b-8053-93c4813d01c5 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:08.694116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94893895-66da-468e-9078-e753b53488db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:08.694716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94893895-66da-468e-9078-e753b53488db None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:08.694983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94893895-66da-468e-9078-e753b53488db None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:08.695240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94893895-66da-468e-9078-e753b53488db None admin] PATH_INFO: `/v3/projects/875c30a03b8f435ba6cd0985c4323ec7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:08.696314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94893895-66da-468e-9078-e753b53488db None admin] RBAC: Authorizing `identity:get_project(project_id=875c30a03b8f435ba6cd0985c4323ec7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:08.697729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94893895-66da-468e-9078-e753b53488db None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:08.699306 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 440/870] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:34:08 2022] GET /identity/v3/projects/875c30a03b8f435ba6cd0985c4323ec7 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:08.705677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8a01d-31f7-418b-a58f-a136d4a4b9c1 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:08.800968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8a01d-31f7-418b-a58f-a136d4a4b9c1 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:08.833624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff14ce4-795b-43c6-b1c5-94d0c086a780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:08.834246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff14ce4-795b-43c6-b1c5-94d0c086a780 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:08.834513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff14ce4-795b-43c6-b1c5-94d0c086a780 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:08.834774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff14ce4-795b-43c6-b1c5-94d0c086a780 None admin] PATH_INFO: `/v3/projects/875c30a03b8f435ba6cd0985c4323ec7/users/1b8c0e53102f4c698f075f69eb098624/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:08.836611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff14ce4-795b-43c6-b1c5-94d0c086a780 None admin] RBAC: Authorizing `identity:create_grant(project_id=875c30a03b8f435ba6cd0985c4323ec7, user_id=1b8c0e53102f4c698f075f69eb098624, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:08.838054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff14ce4-795b-43c6-b1c5-94d0c086a780 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:08.850573 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 431/871] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:34:08 2022] PUT /identity/v3/projects/875c30a03b8f435ba6cd0985c4323ec7/users/1b8c0e53102f4c698f075f69eb098624/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:34:11.214929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8217c9-396b-4b5b-b4c8-9f80f110ec0b None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:11.215468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8217c9-396b-4b5b-b4c8-9f80f110ec0b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:11.215468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8217c9-396b-4b5b-b4c8-9f80f110ec0b None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:11.216764 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 441/872] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:34:11.222710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c855-85f8-4733-8477-d1a408f766aa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:11.223111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c855-85f8-4733-8477-d1a408f766aa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:11.223237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c855-85f8-4733-8477-d1a408f766aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:11.241333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d010c855-85f8-4733-8477-d1a408f766aa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:34:11.242495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d010c855-85f8-4733-8477-d1a408f766aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:34:11.283205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:11.307160 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 432/873] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:34:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:34:11.633867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8217c9-396b-4b5b-b4c8-9f80f110ec0b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:11.668454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:11.733536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8217c9-396b-4b5b-b4c8-9f80f110ec0b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:11.767462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02da493-592e-4af1-b118-f5fbbc233404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:11.768254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02da493-592e-4af1-b118-f5fbbc233404 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:11.768529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02da493-592e-4af1-b118-f5fbbc233404 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:11.768795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02da493-592e-4af1-b118-f5fbbc233404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:11.778628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02da493-592e-4af1-b118-f5fbbc233404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-b02da493-592e-4af1-b118-f5fbbc233404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:34:11.783454 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:34:11.784580 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:34:11.785602 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 442/874] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:34:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:34:11.790605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c855-85f8-4733-8477-d1a408f766aa None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:11.886101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c855-85f8-4733-8477-d1a408f766aa None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:11.919057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9fceb-5ff3-43ce-a53d-2a1f645506be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:11.919634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9fceb-5ff3-43ce-a53d-2a1f645506be None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:11.919902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9fceb-5ff3-43ce-a53d-2a1f645506be None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:11.920165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9fceb-5ff3-43ce-a53d-2a1f645506be None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:11.920809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9fceb-5ff3-43ce-a53d-2a1f645506be None admin] RBAC: Adding query filter params (name=admin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:34:11.921141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9fceb-5ff3-43ce-a53d-2a1f645506be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:11.922497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9fceb-5ff3-43ce-a53d-2a1f645506be None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:11.930193 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 433/875] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:34:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:11.936220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02da493-592e-4af1-b118-f5fbbc233404 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:12.026273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:12.033190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02da493-592e-4af1-b118-f5fbbc233404 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:12.066415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a129d-de4e-44d6-b8f0-d93b99f5e49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:12.067018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a129d-de4e-44d6-b8f0-d93b99f5e49e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:12.067288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a129d-de4e-44d6-b8f0-d93b99f5e49e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:12.067552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a129d-de4e-44d6-b8f0-d93b99f5e49e None admin] PATH_INFO: `/v3/users/1b8c0e53102f4c698f075f69eb098624` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:12.068638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a129d-de4e-44d6-b8f0-d93b99f5e49e None admin] RBAC: Authorizing `identity:get_user(user_id=1b8c0e53102f4c698f075f69eb098624)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:12.070082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a129d-de4e-44d6-b8f0-d93b99f5e49e None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:12.071642 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 443/876] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:34:11 2022] GET /identity/v3/users/1b8c0e53102f4c698f075f69eb098624 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:12.077609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9fceb-5ff3-43ce-a53d-2a1f645506be None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:12.116638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:12.175091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9fceb-5ff3-43ce-a53d-2a1f645506be None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:12.209297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc183a-687d-4a6d-b6ac-1b7b201f1654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:12.209881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc183a-687d-4a6d-b6ac-1b7b201f1654 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:12.210151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc183a-687d-4a6d-b6ac-1b7b201f1654 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:12.210412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc183a-687d-4a6d-b6ac-1b7b201f1654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:12.277417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc183a-687d-4a6d-b6ac-1b7b201f1654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:12.278900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc183a-687d-4a6d-b6ac-1b7b201f1654 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:12.316242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:12.375241 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 434/877] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:34:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 20:34:12.381310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a129d-de4e-44d6-b8f0-d93b99f5e49e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:12.478640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a129d-de4e-44d6-b8f0-d93b99f5e49e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:12.510755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b110e-ca18-4e55-85fa-3c6fadbf7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:12.511351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b110e-ca18-4e55-85fa-3c6fadbf7d7f None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:12.511623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b110e-ca18-4e55-85fa-3c6fadbf7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:12.511887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b110e-ca18-4e55-85fa-3c6fadbf7d7f None admin] PATH_INFO: `/v3/projects/875c30a03b8f435ba6cd0985c4323ec7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:12.512968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b110e-ca18-4e55-85fa-3c6fadbf7d7f None admin] RBAC: Authorizing `identity:get_project(project_id=875c30a03b8f435ba6cd0985c4323ec7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:12.514397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b110e-ca18-4e55-85fa-3c6fadbf7d7f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:12.515972 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 444/878] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:34:12 2022] GET /identity/v3/projects/875c30a03b8f435ba6cd0985c4323ec7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:12.522451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc183a-687d-4a6d-b6ac-1b7b201f1654 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:12.619088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc183a-687d-4a6d-b6ac-1b7b201f1654 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:12.651781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d733e-7763-4e7f-bc81-884286f29573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:12.652357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d733e-7763-4e7f-bc81-884286f29573 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:12.652622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d733e-7763-4e7f-bc81-884286f29573 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:12.652882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d733e-7763-4e7f-bc81-884286f29573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:12.653611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d733e-7763-4e7f-bc81-884286f29573 None admin] RBAC: Adding query filter params (role.id=fab66a1a6ba64c9583182c78978b218e, scope.project.id=875c30a03b8f435ba6cd0985c4323ec7, user.id=1b8c0e53102f4c698f075f69eb098624) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:34:12.653906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d733e-7763-4e7f-bc81-884286f29573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:12.655309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d733e-7763-4e7f-bc81-884286f29573 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:12.661518 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 435/879] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:34:12 2022] GET /identity/v3/role_assignments?user.id=1b8c0e53102f4c698f075f69eb098624&scope.project.id=875c30a03b8f435ba6cd0985c4323ec7&role.id=fab66a1a6ba64c9583182c78978b218e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:15.042647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab2e6-c314-46a1-8de8-e4c21f91a7f5 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:15.043210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab2e6-c314-46a1-8de8-e4c21f91a7f5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:15.043210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab2e6-c314-46a1-8de8-e4c21f91a7f5 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:15.044487 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 445/880] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:34:15.050168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961188-8b51-4584-bea6-4e56edbf7089 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:15.050550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961188-8b51-4584-bea6-4e56edbf7089 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:15.050691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961188-8b51-4584-bea6-4e56edbf7089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:15.056827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:15.069605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-99961188-8b51-4584-bea6-4e56edbf7089 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:34:15.070791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-99961188-8b51-4584-bea6-4e56edbf7089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:34:15.135732 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 436/881] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:34:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:34:15.467584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ab2e6-c314-46a1-8de8-e4c21f91a7f5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:15.498590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:15.570929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ab2e6-c314-46a1-8de8-e4c21f91a7f5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:15.605006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653615e8-8e93-4d01-9877-f5cee6e7108f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:15.605613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653615e8-8e93-4d01-9877-f5cee6e7108f None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:15.605885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653615e8-8e93-4d01-9877-f5cee6e7108f None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:15.606148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653615e8-8e93-4d01-9877-f5cee6e7108f None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:15.606804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653615e8-8e93-4d01-9877-f5cee6e7108f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:15.608135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653615e8-8e93-4d01-9877-f5cee6e7108f None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:15.623807 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 446/882] 10.222.0.16 () {60 vars in 1138 bytes} [Sat Oct 15 20:34:15 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:34:18.183564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9b7f6-b75a-4d78-9e31-21a508edb75a None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:18.184146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9b7f6-b75a-4d78-9e31-21a508edb75a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:18.184146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9b7f6-b75a-4d78-9e31-21a508edb75a None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:18.185642 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 437/883] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:34:18.192431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801b17-ae08-4b14-b184-7bfa51b15469 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:18.192978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801b17-ae08-4b14-b184-7bfa51b15469 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:18.192978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801b17-ae08-4b14-b184-7bfa51b15469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:18.214251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1d801b17-ae08-4b14-b184-7bfa51b15469 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:34:18.215765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1d801b17-ae08-4b14-b184-7bfa51b15469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:34:18.299530 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 447/884] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:34:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:34:18.709780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9b7f6-b75a-4d78-9e31-21a508edb75a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:18.808519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9b7f6-b75a-4d78-9e31-21a508edb75a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:18.837231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:18.842386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca77fc-4d89-4f85-aed2-88022929dc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:18.842996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca77fc-4d89-4f85-aed2-88022929dc0f None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:18.843268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca77fc-4d89-4f85-aed2-88022929dc0f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:18.843530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca77fc-4d89-4f85-aed2-88022929dc0f None admin] PATH_INFO: `/v3/domains/d58361e9f0fe47fd83e07a82a42fef7b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:18.854752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca77fc-4d89-4f85-aed2-88022929dc0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=d58361e9f0fe47fd83e07a82a42fef7b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:18.856181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca77fc-4d89-4f85-aed2-88022929dc0f None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:18.857800 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 438/885] 10.222.0.16 () {56 vars in 1183 bytes} [Sat Oct 15 20:34:18 2022] GET /identity/v3/domains/d58361e9f0fe47fd83e07a82a42fef7b => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:18.864332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d801b17-ae08-4b14-b184-7bfa51b15469 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:18.884962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:18.960346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d801b17-ae08-4b14-b184-7bfa51b15469 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:18.993587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574393b-ea64-4111-9b50-a8b05ccf235c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:18.994165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574393b-ea64-4111-9b50-a8b05ccf235c None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:18.994440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574393b-ea64-4111-9b50-a8b05ccf235c None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:18.994709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574393b-ea64-4111-9b50-a8b05ccf235c None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:18.995341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574393b-ea64-4111-9b50-a8b05ccf235c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:18.996746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574393b-ea64-4111-9b50-a8b05ccf235c None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:19.022427 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 448/886] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:34:18 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:34:21.441389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e54d6-ce31-4515-bebb-9cc5cd1e4eaa None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:21.441933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e54d6-ce31-4515-bebb-9cc5cd1e4eaa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:21.441933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e54d6-ce31-4515-bebb-9cc5cd1e4eaa None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:21.443255 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 439/887] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:34:21.448552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dc6b6-0a7b-4868-9171-01b430a1e213 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:21.448923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dc6b6-0a7b-4868-9171-01b430a1e213 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:21.449020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dc6b6-0a7b-4868-9171-01b430a1e213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:21.467649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cd6dc6b6-0a7b-4868-9171-01b430a1e213 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:34:21.468819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cd6dc6b6-0a7b-4868-9171-01b430a1e213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:34:21.534667 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 449/888] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:34:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:34:21.859536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e54d6-ce31-4515-bebb-9cc5cd1e4eaa None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:21.969403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e54d6-ce31-4515-bebb-9cc5cd1e4eaa None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:22.003925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df848e50-8dee-4f5e-b669-7f25e4d1d76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:22.004535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df848e50-8dee-4f5e-b669-7f25e4d1d76d None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:22.004806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df848e50-8dee-4f5e-b669-7f25e4d1d76d None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:22.005092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df848e50-8dee-4f5e-b669-7f25e4d1d76d None admin] PATH_INFO: `/v3/domains/d58361e9f0fe47fd83e07a82a42fef7b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:22.006203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df848e50-8dee-4f5e-b669-7f25e4d1d76d None admin] RBAC: Authorizing `identity:get_domain(domain_id=d58361e9f0fe47fd83e07a82a42fef7b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:22.007631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df848e50-8dee-4f5e-b669-7f25e4d1d76d None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:22.009204 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 440/889] 10.222.0.16 () {56 vars in 1183 bytes} [Sat Oct 15 20:34:21 2022] GET /identity/v3/domains/d58361e9f0fe47fd83e07a82a42fef7b => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:22.015269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6dc6b6-0a7b-4868-9171-01b430a1e213 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:22.018004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:22.116630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6dc6b6-0a7b-4868-9171-01b430a1e213 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:22.150115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d263664-dc05-4a78-8c34-277abd12d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:22.150702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d263664-dc05-4a78-8c34-277abd12d638 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:22.150975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d263664-dc05-4a78-8c34-277abd12d638 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:22.151241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d263664-dc05-4a78-8c34-277abd12d638 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:22.151885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d263664-dc05-4a78-8c34-277abd12d638 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:22.153331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d263664-dc05-4a78-8c34-277abd12d638 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:22.174082 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 450/890] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:34:22 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:34:24.569431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4553229-32e6-4d7f-811e-98943db33658 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:24.569936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4553229-32e6-4d7f-811e-98943db33658 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:24.569936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4553229-32e6-4d7f-811e-98943db33658 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:24.571222 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 441/891] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:34:24.576252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99080014-68fc-40c5-9cfd-ed51420894e1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:24.576667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99080014-68fc-40c5-9cfd-ed51420894e1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:24.576820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99080014-68fc-40c5-9cfd-ed51420894e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:24.596140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-99080014-68fc-40c5-9cfd-ed51420894e1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:34:24.597339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-99080014-68fc-40c5-9cfd-ed51420894e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:34:24.648309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:24.663439 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 451/892] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:34:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:34:24.988621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4553229-32e6-4d7f-811e-98943db33658 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:25.048260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:25.090521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4553229-32e6-4d7f-811e-98943db33658 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:25.122434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269933dc-447f-420f-8f97-9a3e6b3900df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:25.123046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269933dc-447f-420f-8f97-9a3e6b3900df None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:25.123319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269933dc-447f-420f-8f97-9a3e6b3900df None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:25.123585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269933dc-447f-420f-8f97-9a3e6b3900df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:25.134434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269933dc-447f-420f-8f97-9a3e6b3900df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-269933dc-447f-420f-8f97-9a3e6b3900df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:34:25.139114 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:34:25.140181 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:34:25.141386 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 442/893] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:34:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:34:25.146807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99080014-68fc-40c5-9cfd-ed51420894e1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:25.243769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99080014-68fc-40c5-9cfd-ed51420894e1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:25.277238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172cf94-be61-42a1-85a7-64e5959a1891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:25.277821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172cf94-be61-42a1-85a7-64e5959a1891 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:25.278090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172cf94-be61-42a1-85a7-64e5959a1891 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:25.278347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172cf94-be61-42a1-85a7-64e5959a1891 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:25.278987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172cf94-be61-42a1-85a7-64e5959a1891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:34:25.279276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172cf94-be61-42a1-85a7-64e5959a1891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:25.280610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172cf94-be61-42a1-85a7-64e5959a1891 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:25.288470 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 452/894] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:34:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:25.294449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269933dc-447f-420f-8f97-9a3e6b3900df None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:25.397338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269933dc-447f-420f-8f97-9a3e6b3900df None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:25.416395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:25.430991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65ee44-ce2a-45f0-9bb4-859c23b36838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:25.431592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65ee44-ce2a-45f0-9bb4-859c23b36838 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:25.431865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65ee44-ce2a-45f0-9bb4-859c23b36838 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:25.432130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65ee44-ce2a-45f0-9bb4-859c23b36838 None admin] PATH_INFO: `/v3/users/64385921e9df4ed3ba058b48785bf0de` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:25.445894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65ee44-ce2a-45f0-9bb4-859c23b36838 None admin] RBAC: Authorizing `identity:get_user(user_id=64385921e9df4ed3ba058b48785bf0de)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:25.447324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65ee44-ce2a-45f0-9bb4-859c23b36838 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:25.448892 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 443/895] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:34:25 2022] GET /identity/v3/users/64385921e9df4ed3ba058b48785bf0de => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:34:25.454888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172cf94-be61-42a1-85a7-64e5959a1891 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:25.504381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:25.552667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172cf94-be61-42a1-85a7-64e5959a1891 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:25.583938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267bc0e-33d8-4ef0-a2cd-e5e108534e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:25.584517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267bc0e-33d8-4ef0-a2cd-e5e108534e25 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:25.584790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267bc0e-33d8-4ef0-a2cd-e5e108534e25 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:25.585076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267bc0e-33d8-4ef0-a2cd-e5e108534e25 None admin] PATH_INFO: `/v3/projects/8fda9bb25fa1439e9c9a7776cffd4d09` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:25.586156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267bc0e-33d8-4ef0-a2cd-e5e108534e25 None admin] RBAC: Authorizing `identity:get_project(project_id=8fda9bb25fa1439e9c9a7776cffd4d09)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:25.587547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267bc0e-33d8-4ef0-a2cd-e5e108534e25 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:25.589139 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 453/896] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:34:25 2022] GET /identity/v3/projects/8fda9bb25fa1439e9c9a7776cffd4d09 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:25.595827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65ee44-ce2a-45f0-9bb4-859c23b36838 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:25.692677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65ee44-ce2a-45f0-9bb4-859c23b36838 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:25.726237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96becf20-f084-4a00-b8ce-55016f032853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:25.726866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becf20-f084-4a00-b8ce-55016f032853 None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:25.727136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becf20-f084-4a00-b8ce-55016f032853 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:25.727399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becf20-f084-4a00-b8ce-55016f032853 None admin] PATH_INFO: `/v3/projects/8fda9bb25fa1439e9c9a7776cffd4d09/users/64385921e9df4ed3ba058b48785bf0de/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:25.729307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96becf20-f084-4a00-b8ce-55016f032853 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fda9bb25fa1439e9c9a7776cffd4d09, user_id=64385921e9df4ed3ba058b48785bf0de, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:25.730720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96becf20-f084-4a00-b8ce-55016f032853 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:25.743962 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 444/897] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:34:25 2022] PUT /identity/v3/projects/8fda9bb25fa1439e9c9a7776cffd4d09/users/64385921e9df4ed3ba058b48785bf0de/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:34:28.081490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a838ec-30e2-4ccd-a314-f4cef6b7f264 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:28.082123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a838ec-30e2-4ccd-a314-f4cef6b7f264 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:28.082123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a838ec-30e2-4ccd-a314-f4cef6b7f264 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:28.083333 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 454/898] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:34:28.088766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c649fe5-e801-4810-9084-a9fa81100a50 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:28.089222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c649fe5-e801-4810-9084-a9fa81100a50 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:28.089317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c649fe5-e801-4810-9084-a9fa81100a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:28.107652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6c649fe5-e801-4810-9084-a9fa81100a50 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:34:28.108826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6c649fe5-e801-4810-9084-a9fa81100a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:34:28.138290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:28.176434 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 445/899] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:34:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:34:28.498440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a838ec-30e2-4ccd-a314-f4cef6b7f264 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:28.600804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a838ec-30e2-4ccd-a314-f4cef6b7f264 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:28.617680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:28.634811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5931e-672b-4583-9620-ec7c8c99fc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:28.635418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5931e-672b-4583-9620-ec7c8c99fc6b None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:28.635692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5931e-672b-4583-9620-ec7c8c99fc6b None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:28.635960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5931e-672b-4583-9620-ec7c8c99fc6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:28.646384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5931e-672b-4583-9620-ec7c8c99fc6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-97f5931e-672b-4583-9620-ec7c8c99fc6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:34:28.651090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:34:28.652206 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:34:28.653485 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 455/900] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:34:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:34:28.658724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c649fe5-e801-4810-9084-a9fa81100a50 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:28.757404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c649fe5-e801-4810-9084-a9fa81100a50 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:28.791596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733717ca-9787-49c7-b717-9d8382284ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:28.792193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733717ca-9787-49c7-b717-9d8382284ba6 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:28.792469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733717ca-9787-49c7-b717-9d8382284ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:28.792772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733717ca-9787-49c7-b717-9d8382284ba6 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:28.793498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733717ca-9787-49c7-b717-9d8382284ba6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:34:28.793819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733717ca-9787-49c7-b717-9d8382284ba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:28.795275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733717ca-9787-49c7-b717-9d8382284ba6 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:28.803500 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 446/901] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:28.809426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5931e-672b-4583-9620-ec7c8c99fc6b None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:28.906284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5931e-672b-4583-9620-ec7c8c99fc6b None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:28.939333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ee504-970f-4292-86be-a70f806eb1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:28.939933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ee504-970f-4292-86be-a70f806eb1ae None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:28.940206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ee504-970f-4292-86be-a70f806eb1ae None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:28.940471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ee504-970f-4292-86be-a70f806eb1ae None admin] PATH_INFO: `/v3/users/64385921e9df4ed3ba058b48785bf0de` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:28.941591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ee504-970f-4292-86be-a70f806eb1ae None admin] RBAC: Authorizing `identity:get_user(user_id=64385921e9df4ed3ba058b48785bf0de)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:28.943005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ee504-970f-4292-86be-a70f806eb1ae None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:28.944546 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 456/902] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Oct 15 20:34:28 2022] GET /identity/v3/users/64385921e9df4ed3ba058b48785bf0de => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:28.950298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733717ca-9787-49c7-b717-9d8382284ba6 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:28.979690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:29.047038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733717ca-9787-49c7-b717-9d8382284ba6 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:29.079857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a36444-8a87-40e7-8ccb-3ea0994a0c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:29.080436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a36444-8a87-40e7-8ccb-3ea0994a0c29 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:29.080704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a36444-8a87-40e7-8ccb-3ea0994a0c29 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:29.080966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a36444-8a87-40e7-8ccb-3ea0994a0c29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:29.146454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a36444-8a87-40e7-8ccb-3ea0994a0c29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:29.147888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a36444-8a87-40e7-8ccb-3ea0994a0c29 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:29.175920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:29.246191 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 447/903] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:34:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 20:34:29.252949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ee504-970f-4292-86be-a70f806eb1ae None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:29.299621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:29.353984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ee504-970f-4292-86be-a70f806eb1ae None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:29.391549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1d407-5d11-4dbf-ba93-9a23632d6dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:29.392265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1d407-5d11-4dbf-ba93-9a23632d6dfd None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:29.392538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1d407-5d11-4dbf-ba93-9a23632d6dfd None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:29.392803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1d407-5d11-4dbf-ba93-9a23632d6dfd None admin] PATH_INFO: `/v3/projects/8fda9bb25fa1439e9c9a7776cffd4d09` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:29.393928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea1d407-5d11-4dbf-ba93-9a23632d6dfd None admin] RBAC: Authorizing `identity:get_project(project_id=8fda9bb25fa1439e9c9a7776cffd4d09)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:29.395339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea1d407-5d11-4dbf-ba93-9a23632d6dfd None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:29.396952 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 457/904] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Oct 15 20:34:29 2022] GET /identity/v3/projects/8fda9bb25fa1439e9c9a7776cffd4d09 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:29.403184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a36444-8a87-40e7-8ccb-3ea0994a0c29 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:34:29.501185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a36444-8a87-40e7-8ccb-3ea0994a0c29 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:34:29.528790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:29.535537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645532f-5fdc-4ec7-83b9-e2c5c80ea7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:34:29.536121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645532f-5fdc-4ec7-83b9-e2c5c80ea7f3 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:34:29.536391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645532f-5fdc-4ec7-83b9-e2c5c80ea7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:34:29.536653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645532f-5fdc-4ec7-83b9-e2c5c80ea7f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:34:29.537405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645532f-5fdc-4ec7-83b9-e2c5c80ea7f3 None admin] RBAC: Adding query filter params (role.id=fab66a1a6ba64c9583182c78978b218e, scope.project.id=8fda9bb25fa1439e9c9a7776cffd4d09, user.id=64385921e9df4ed3ba058b48785bf0de) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:34:29.537706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645532f-5fdc-4ec7-83b9-e2c5c80ea7f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:34:29.539154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645532f-5fdc-4ec7-83b9-e2c5c80ea7f3 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:34:29.544835 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 448/905] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:34:29 2022] GET /identity/v3/role_assignments?user.id=64385921e9df4ed3ba058b48785bf0de&scope.project.id=8fda9bb25fa1439e9c9a7776cffd4d09&role.id=fab66a1a6ba64c9583182c78978b218e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:34:39.387333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=43 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:34:39.542355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:33.255166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2821de-2af3-4907-97f3-d127d4caf9ac None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:33.255777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2821de-2af3-4907-97f3-d127d4caf9ac None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:33.255777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2821de-2af3-4907-97f3-d127d4caf9ac None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:33.257014 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 458/906] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:35:33.263310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594de2b6-acd3-44e4-93f6-8cc506720a41 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:33.263839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594de2b6-acd3-44e4-93f6-8cc506720a41 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:33.264053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594de2b6-acd3-44e4-93f6-8cc506720a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:33.303336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-594de2b6-acd3-44e4-93f6-8cc506720a41 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:35:33.304620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-594de2b6-acd3-44e4-93f6-8cc506720a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:35:33.401150 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 449/907] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:35:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:35:33.744108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2821de-2af3-4907-97f3-d127d4caf9ac None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:33.882814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2821de-2af3-4907-97f3-d127d4caf9ac None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:33.929625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d6dd7-c3f0-4ab9-b1e6-ac46359f0942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:33.930633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d6dd7-c3f0-4ab9-b1e6-ac46359f0942 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:33.931284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d6dd7-c3f0-4ab9-b1e6-ac46359f0942 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:33.931903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d6dd7-c3f0-4ab9-b1e6-ac46359f0942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:33.948523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d6dd7-c3f0-4ab9-b1e6-ac46359f0942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:33.950804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d6dd7-c3f0-4ab9-b1e6-ac46359f0942 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-4e3d6dd7-c3f0-4ab9-b1e6-ac46359f0942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:35:33.973938 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:33.975016 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:35:33.975969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:35:33.975969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:33.975969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:33.977958 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 459/908] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:33.986716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594de2b6-acd3-44e4-93f6-8cc506720a41 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:34.152120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594de2b6-acd3-44e4-93f6-8cc506720a41 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:34.202329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:34.212311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4148e70-243d-4b24-8ecd-c833ae0eed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:34.213572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148e70-243d-4b24-8ecd-c833ae0eed64 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:34.214260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148e70-243d-4b24-8ecd-c833ae0eed64 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:34.214734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148e70-243d-4b24-8ecd-c833ae0eed64 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:34.215667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4148e70-243d-4b24-8ecd-c833ae0eed64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:34.216409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4148e70-243d-4b24-8ecd-c833ae0eed64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:34.218164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4148e70-243d-4b24-8ecd-c833ae0eed64 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:34.235932 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 450/909] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:34.243732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d6dd7-c3f0-4ab9-b1e6-ac46359f0942 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:34.328320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:34.387572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d6dd7-c3f0-4ab9-b1e6-ac46359f0942 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:34.431339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64ad2c-0502-4ab3-9376-04d3c8d68dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:34.431879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64ad2c-0502-4ab3-9376-04d3c8d68dba None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:34.432151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64ad2c-0502-4ab3-9376-04d3c8d68dba None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:34.432419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64ad2c-0502-4ab3-9376-04d3c8d68dba None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:34.433841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64ad2c-0502-4ab3-9376-04d3c8d68dba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:34.435118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64ad2c-0502-4ab3-9376-04d3c8d68dba None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:34.462675 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 460/910] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Oct 15 20:35:34 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:35:37.389002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd249ca-931d-4776-a7bb-0f1b3fc0a44d None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:37.389656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd249ca-931d-4776-a7bb-0f1b3fc0a44d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:37.389748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd249ca-931d-4776-a7bb-0f1b3fc0a44d None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:37.391271 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 451/911] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:35:37.396363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee4ed7-9562-49c6-9229-8edb64c2919b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:37.396771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee4ed7-9562-49c6-9229-8edb64c2919b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:37.396965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee4ed7-9562-49c6-9229-8edb64c2919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:37.417303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b2ee4ed7-9562-49c6-9229-8edb64c2919b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:35:37.418635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b2ee4ed7-9562-49c6-9229-8edb64c2919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:35:37.503250 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 461/912] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:35:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:35:37.866725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd249ca-931d-4776-a7bb-0f1b3fc0a44d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:37.996561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd249ca-931d-4776-a7bb-0f1b3fc0a44d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:38.037555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88809298-5988-4a35-a969-ce08c71e80de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:38.038073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88809298-5988-4a35-a969-ce08c71e80de None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:38.038283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88809298-5988-4a35-a969-ce08c71e80de None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:38.038589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88809298-5988-4a35-a969-ce08c71e80de None admin] PATH_INFO: `/v3/roles/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:38.051101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88809298-5988-4a35-a969-ce08c71e80de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-88809298-5988-4a35-a969-ce08c71e80de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:35:38.056293 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:38.057222 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:38.058782 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 452/913] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:38.063696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee4ed7-9562-49c6-9229-8edb64c2919b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:38.119767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:38.192575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee4ed7-9562-49c6-9229-8edb64c2919b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:38.237132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae8237-a1f4-4750-8c5f-f83475b0cb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:38.237725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae8237-a1f4-4750-8c5f-f83475b0cb98 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:38.237996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae8237-a1f4-4750-8c5f-f83475b0cb98 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:38.238257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae8237-a1f4-4750-8c5f-f83475b0cb98 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:38.238940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae8237-a1f4-4750-8c5f-f83475b0cb98 None admin] RBAC: Adding query filter params (name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:38.239451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae8237-a1f4-4750-8c5f-f83475b0cb98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:38.241319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae8237-a1f4-4750-8c5f-f83475b0cb98 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:38.251092 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 462/914] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:35:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:38.257197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88809298-5988-4a35-a969-ce08c71e80de None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:38.320488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:38.385518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88809298-5988-4a35-a969-ce08c71e80de None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:38.427843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e880cff-2956-4982-8e8a-ae94a104107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:38.428537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e880cff-2956-4982-8e8a-ae94a104107c None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:38.428783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e880cff-2956-4982-8e8a-ae94a104107c None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:38.429058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e880cff-2956-4982-8e8a-ae94a104107c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:38.444732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e880cff-2956-4982-8e8a-ae94a104107c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:38.446575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e880cff-2956-4982-8e8a-ae94a104107c None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-5e880cff-2956-4982-8e8a-ae94a104107c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:35:38.468921 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:38.470498 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:35:38.471640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:35:38.471640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:38.471640 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:38.471640 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 453/915] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:38.476960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae8237-a1f4-4750-8c5f-f83475b0cb98 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:38.602338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:38.608782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae8237-a1f4-4750-8c5f-f83475b0cb98 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:38.652209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865a34b-4599-422f-a0a0-e9c64b726b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:38.652826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865a34b-4599-422f-a0a0-e9c64b726b50 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:38.653194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865a34b-4599-422f-a0a0-e9c64b726b50 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:38.653521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865a34b-4599-422f-a0a0-e9c64b726b50 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:38.654300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865a34b-4599-422f-a0a0-e9c64b726b50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:38.654563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865a34b-4599-422f-a0a0-e9c64b726b50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:38.656049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865a34b-4599-422f-a0a0-e9c64b726b50 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:38.670462 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 463/916] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:35:38.677512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e880cff-2956-4982-8e8a-ae94a104107c None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:38.796215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e880cff-2956-4982-8e8a-ae94a104107c None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:38.797919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:38.840295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32130e67-dac8-45a1-ad42-fab0b92fd916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:38.840944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32130e67-dac8-45a1-ad42-fab0b92fd916 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:38.841290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32130e67-dac8-45a1-ad42-fab0b92fd916 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:38.841560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32130e67-dac8-45a1-ad42-fab0b92fd916 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:38.850952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32130e67-dac8-45a1-ad42-fab0b92fd916 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:38.854313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32130e67-dac8-45a1-ad42-fab0b92fd916 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-32130e67-dac8-45a1-ad42-fab0b92fd916 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:35:38.868708 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:38.870333 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:38.871260 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 454/917] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:38.876796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865a34b-4599-422f-a0a0-e9c64b726b50 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:38.998013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865a34b-4599-422f-a0a0-e9c64b726b50 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:39.037577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235513-e087-4d28-90a5-7956188b8021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:39.038142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235513-e087-4d28-90a5-7956188b8021 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:39.038446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235513-e087-4d28-90a5-7956188b8021 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:39.038838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235513-e087-4d28-90a5-7956188b8021 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:39.039562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70235513-e087-4d28-90a5-7956188b8021 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:39.039890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70235513-e087-4d28-90a5-7956188b8021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:39.041640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70235513-e087-4d28-90a5-7956188b8021 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:39.061726 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 464/918] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Oct 15 20:35:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:35:39.067822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32130e67-dac8-45a1-ad42-fab0b92fd916 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:39.189285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32130e67-dac8-45a1-ad42-fab0b92fd916 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:39.227747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd6cde-7cf3-423e-8bb0-85c1ace0b2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:39.228450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd6cde-7cf3-423e-8bb0-85c1ace0b2a0 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:39.228792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd6cde-7cf3-423e-8bb0-85c1ace0b2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:39.229155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd6cde-7cf3-423e-8bb0-85c1ace0b2a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:39.239985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd6cde-7cf3-423e-8bb0-85c1ace0b2a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:39.241577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd6cde-7cf3-423e-8bb0-85c1ace0b2a0 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-a7fd6cde-7cf3-423e-8bb0-85c1ace0b2a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:35:39.256718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:39.262788 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:35:39.264142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:35:39.264142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:39.264142 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:39.264142 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 455/919] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:39.264393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235513-e087-4d28-90a5-7956188b8021 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:39.283170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:39.380846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235513-e087-4d28-90a5-7956188b8021 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:39.426148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa3d35-ab30-4a96-a9eb-082e7db1dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:39.426872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa3d35-ab30-4a96-a9eb-082e7db1dc35 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:39.427082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa3d35-ab30-4a96-a9eb-082e7db1dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:39.427392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa3d35-ab30-4a96-a9eb-082e7db1dc35 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:39.428028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa3d35-ab30-4a96-a9eb-082e7db1dc35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:39.428325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa3d35-ab30-4a96-a9eb-082e7db1dc35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:39.429711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa3d35-ab30-4a96-a9eb-082e7db1dc35 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:39.441870 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 465/920] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:39.451256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd6cde-7cf3-423e-8bb0-85c1ace0b2a0 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:39.469789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:39.563033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd6cde-7cf3-423e-8bb0-85c1ace0b2a0 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:39.601801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788eff8-0e77-4963-95ca-a1141c7afc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:39.602544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788eff8-0e77-4963-95ca-a1141c7afc5d None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:39.602921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788eff8-0e77-4963-95ca-a1141c7afc5d None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:39.603249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788eff8-0e77-4963-95ca-a1141c7afc5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:39.612425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788eff8-0e77-4963-95ca-a1141c7afc5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:39.614440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788eff8-0e77-4963-95ca-a1141c7afc5d None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-6788eff8-0e77-4963-95ca-a1141c7afc5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:35:39.627367 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:39.628365 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:39.629738 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:35:39.629738 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:35:39.629738 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:35:39.629738 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:35:39.629738 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:35:39.629738 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:35:39.629738 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:35:39.629738 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:39.633559 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 456/921] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:39.635039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa3d35-ab30-4a96-a9eb-082e7db1dc35 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:39.731269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:39.762112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa3d35-ab30-4a96-a9eb-082e7db1dc35 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:39.806250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06270cbd-50d9-4135-81e7-e540d2894f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:39.806850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06270cbd-50d9-4135-81e7-e540d2894f40 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:39.807123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06270cbd-50d9-4135-81e7-e540d2894f40 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:39.807387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06270cbd-50d9-4135-81e7-e540d2894f40 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:39.808084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06270cbd-50d9-4135-81e7-e540d2894f40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:39.808369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06270cbd-50d9-4135-81e7-e540d2894f40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:39.810266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06270cbd-50d9-4135-81e7-e540d2894f40 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:39.822822 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 466/922] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:35:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:39.829164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788eff8-0e77-4963-95ca-a1141c7afc5d None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:39.931974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:39.962447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788eff8-0e77-4963-95ca-a1141c7afc5d None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:40.000891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3d6a3-c2db-46eb-823e-3c0d91cc6ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:40.001606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3d6a3-c2db-46eb-823e-3c0d91cc6ace None admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:40.001876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3d6a3-c2db-46eb-823e-3c0d91cc6ace None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:40.002137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3d6a3-c2db-46eb-823e-3c0d91cc6ace None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/1ca62592177d43b5b07106003c9efa3b/roles/492ce6c3fedb4109ac31de9467e8ef3b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:40.019990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3d6a3-c2db-46eb-823e-3c0d91cc6ace None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=1ca62592177d43b5b07106003c9efa3b, role_id=492ce6c3fedb4109ac31de9467e8ef3b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:40.021742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3d6a3-c2db-46eb-823e-3c0d91cc6ace None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:40.038435 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 457/923] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:35:39 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/1ca62592177d43b5b07106003c9efa3b/roles/492ce6c3fedb4109ac31de9467e8ef3b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:35:42.580177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f32d5-1420-49b3-b7ee-cbdea4a43224 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:42.580787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f32d5-1420-49b3-b7ee-cbdea4a43224 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:42.580787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f32d5-1420-49b3-b7ee-cbdea4a43224 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:42.582343 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 467/924] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:35:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:35:42.587757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274b470-6c8e-442b-88d5-1fc59b7f267e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:42.588175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274b470-6c8e-442b-88d5-1fc59b7f267e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:42.588443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274b470-6c8e-442b-88d5-1fc59b7f267e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:42.608844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4274b470-6c8e-442b-88d5-1fc59b7f267e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:35:42.610326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4274b470-6c8e-442b-88d5-1fc59b7f267e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:35:42.691360 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 458/925] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:35:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:35:43.046258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f32d5-1420-49b3-b7ee-cbdea4a43224 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:43.166796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f32d5-1420-49b3-b7ee-cbdea4a43224 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:43.208810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca4864-3220-46c9-b89f-efe19403818e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:43.209589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca4864-3220-46c9-b89f-efe19403818e None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:43.209987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca4864-3220-46c9-b89f-efe19403818e None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:43.210384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca4864-3220-46c9-b89f-efe19403818e None admin] PATH_INFO: `/v3/roles/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:43.217970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:43.222211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca4864-3220-46c9-b89f-efe19403818e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-28ca4864-3220-46c9-b89f-efe19403818e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:35:43.227142 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:43.228272 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:43.229648 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 468/926] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Oct 15 20:35:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:43.236071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274b470-6c8e-442b-88d5-1fc59b7f267e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:43.293438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:43.344217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274b470-6c8e-442b-88d5-1fc59b7f267e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:43.380239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e73212-a5c0-4cd1-af49-2e1f2be0de74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:43.380827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e73212-a5c0-4cd1-af49-2e1f2be0de74 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:43.381128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e73212-a5c0-4cd1-af49-2e1f2be0de74 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:43.381396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e73212-a5c0-4cd1-af49-2e1f2be0de74 None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:43.382048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e73212-a5c0-4cd1-af49-2e1f2be0de74 None admin] RBAC: Adding query filter params (name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:43.382342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e73212-a5c0-4cd1-af49-2e1f2be0de74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:43.383819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e73212-a5c0-4cd1-af49-2e1f2be0de74 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:43.392104 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 459/927] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Oct 15 20:35:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:43.398242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca4864-3220-46c9-b89f-efe19403818e None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:43.512430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca4864-3220-46c9-b89f-efe19403818e None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:43.557856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510c4d9-0176-47e3-9888-51e2f0742bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:43.558580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510c4d9-0176-47e3-9888-51e2f0742bce None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:43.558859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510c4d9-0176-47e3-9888-51e2f0742bce None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:43.559254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510c4d9-0176-47e3-9888-51e2f0742bce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:43.572901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510c4d9-0176-47e3-9888-51e2f0742bce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:43.574747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510c4d9-0176-47e3-9888-51e2f0742bce None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-b510c4d9-0176-47e3-9888-51e2f0742bce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:35:43.591349 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:43.593069 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:35:43.594309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:35:43.594309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:43.594309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:43.594309 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 469/928] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 20:35:43.599014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e73212-a5c0-4cd1-af49-2e1f2be0de74 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:43.716863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e73212-a5c0-4cd1-af49-2e1f2be0de74 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:43.734988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:43.754924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3bd77-7330-483f-a8fd-6e705940ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:43.755632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3bd77-7330-483f-a8fd-6e705940ec16 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:43.755948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3bd77-7330-483f-a8fd-6e705940ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:43.756212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3bd77-7330-483f-a8fd-6e705940ec16 None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:43.756867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3bd77-7330-483f-a8fd-6e705940ec16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:43.757197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3bd77-7330-483f-a8fd-6e705940ec16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:43.758878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3bd77-7330-483f-a8fd-6e705940ec16 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:43.772407 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 460/929] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:43.778282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510c4d9-0176-47e3-9888-51e2f0742bce None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:43.835629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:43.890939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510c4d9-0176-47e3-9888-51e2f0742bce None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:43.932688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d49f7-41fb-44fe-a33e-76c6c4c70987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:43.933338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d49f7-41fb-44fe-a33e-76c6c4c70987 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:43.933610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d49f7-41fb-44fe-a33e-76c6c4c70987 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:43.933874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d49f7-41fb-44fe-a33e-76c6c4c70987 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:43.943271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d49f7-41fb-44fe-a33e-76c6c4c70987 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:43.945066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d49f7-41fb-44fe-a33e-76c6c4c70987 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-3d5d49f7-41fb-44fe-a33e-76c6c4c70987 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:35:43.958087 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:43.959233 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:43.960458 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 470/930] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:43.966775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3bd77-7330-483f-a8fd-6e705940ec16 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:44.085455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3bd77-7330-483f-a8fd-6e705940ec16 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:44.126162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22feda99-f74e-42d9-8599-d3e1d486aba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:44.126857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feda99-f74e-42d9-8599-d3e1d486aba2 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:44.127194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feda99-f74e-42d9-8599-d3e1d486aba2 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:44.127450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feda99-f74e-42d9-8599-d3e1d486aba2 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:44.128195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22feda99-f74e-42d9-8599-d3e1d486aba2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:44.128558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22feda99-f74e-42d9-8599-d3e1d486aba2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:44.130087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22feda99-f74e-42d9-8599-d3e1d486aba2 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:44.148863 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 461/931] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Oct 15 20:35:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:44.155478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d49f7-41fb-44fe-a33e-76c6c4c70987 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:44.272547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d49f7-41fb-44fe-a33e-76c6c4c70987 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:44.285508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:44.310130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71524ce9-e727-4062-ace3-cd85188cbe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:44.310800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71524ce9-e727-4062-ace3-cd85188cbe33 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:44.311072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71524ce9-e727-4062-ace3-cd85188cbe33 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:44.311333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71524ce9-e727-4062-ace3-cd85188cbe33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:44.379458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71524ce9-e727-4062-ace3-cd85188cbe33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:44.381126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71524ce9-e727-4062-ace3-cd85188cbe33 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:44.490931 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 471/932] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:35:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 20:35:44.498389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22feda99-f74e-42d9-8599-d3e1d486aba2 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:44.537465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:44.604438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22feda99-f74e-42d9-8599-d3e1d486aba2 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:44.643762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9e45a-b391-4829-8a3d-b66e895856fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:44.644374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9e45a-b391-4829-8a3d-b66e895856fd None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:44.644640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9e45a-b391-4829-8a3d-b66e895856fd None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:44.644900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9e45a-b391-4829-8a3d-b66e895856fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:44.654881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9e45a-b391-4829-8a3d-b66e895856fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:44.656870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9e45a-b391-4829-8a3d-b66e895856fd None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-8dc9e45a-b391-4829-8a3d-b66e895856fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:35:44.671184 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:44.672322 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:35:44.673227 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:35:44.673227 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:44.673227 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:44.673227 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 462/933] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:44.678780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71524ce9-e727-4062-ace3-cd85188cbe33 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:44.690987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:44.786141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71524ce9-e727-4062-ace3-cd85188cbe33 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:44.822750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6f4a3-afef-405e-8a68-02bdd4234fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:44.823341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6f4a3-afef-405e-8a68-02bdd4234fb3 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:44.823609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6f4a3-afef-405e-8a68-02bdd4234fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:44.823869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6f4a3-afef-405e-8a68-02bdd4234fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:44.824494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6f4a3-afef-405e-8a68-02bdd4234fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:44.824786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6f4a3-afef-405e-8a68-02bdd4234fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:44.826741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6f4a3-afef-405e-8a68-02bdd4234fb3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:44.839037 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 472/934] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:44.845348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9e45a-b391-4829-8a3d-b66e895856fd None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:44.949638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:44.968587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9e45a-b391-4829-8a3d-b66e895856fd None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:45.006985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c611c4d-0326-4f49-9d30-0c04d5dbc1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:45.007605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c611c4d-0326-4f49-9d30-0c04d5dbc1c4 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:45.007875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c611c4d-0326-4f49-9d30-0c04d5dbc1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:45.008138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c611c4d-0326-4f49-9d30-0c04d5dbc1c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:45.016402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c611c4d-0326-4f49-9d30-0c04d5dbc1c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:45.018186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c611c4d-0326-4f49-9d30-0c04d5dbc1c4 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-5c611c4d-0326-4f49-9d30-0c04d5dbc1c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:35:45.030352 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:45.031514 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:45.032405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:35:45.032405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:35:45.032405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:35:45.032405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:35:45.032405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:35:45.032405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:35:45.032405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:35:45.032405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:45.032405 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 463/935] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:45.037660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6f4a3-afef-405e-8a68-02bdd4234fb3 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:45.122468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:45.156813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6f4a3-afef-405e-8a68-02bdd4234fb3 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:45.203473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d23ba8-4984-46f8-a704-2ba5d1caf809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:45.204082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d23ba8-4984-46f8-a704-2ba5d1caf809 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:45.204357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d23ba8-4984-46f8-a704-2ba5d1caf809 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:45.204624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d23ba8-4984-46f8-a704-2ba5d1caf809 None admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:45.205350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d23ba8-4984-46f8-a704-2ba5d1caf809 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:45.205649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d23ba8-4984-46f8-a704-2ba5d1caf809 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:45.207054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d23ba8-4984-46f8-a704-2ba5d1caf809 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:45.220190 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 473/936] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:35:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:45.226829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c611c4d-0326-4f49-9d30-0c04d5dbc1c4 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:45.342860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c611c4d-0326-4f49-9d30-0c04d5dbc1c4 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:45.376897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6b26a-bae7-482b-84e7-af471d0eba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:45.377563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6b26a-bae7-482b-84e7-af471d0eba23 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:45.377828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6b26a-bae7-482b-84e7-af471d0eba23 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:45.378097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6b26a-bae7-482b-84e7-af471d0eba23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:45.378828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6b26a-bae7-482b-84e7-af471d0eba23 None admin] RBAC: Adding query filter params (role.id=492ce6c3fedb4109ac31de9467e8ef3b, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=1ca62592177d43b5b07106003c9efa3b) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:45.379126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6b26a-bae7-482b-84e7-af471d0eba23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:45.380621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6b26a-bae7-482b-84e7-af471d0eba23 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:45.386633 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 464/937] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:35:45 2022] GET /identity/v3/role_assignments?user.id=1ca62592177d43b5b07106003c9efa3b&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=492ce6c3fedb4109ac31de9467e8ef3b => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:47.894201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f3982-b241-4125-a8e0-e4291105a626 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:47.894769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f3982-b241-4125-a8e0-e4291105a626 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:47.894769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f3982-b241-4125-a8e0-e4291105a626 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:47.896460 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 474/938] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:35:47.901732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a7eb0-d221-4ddc-9a7c-fa2aa55d46e2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:47.902184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a7eb0-d221-4ddc-9a7c-fa2aa55d46e2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:47.902275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a7eb0-d221-4ddc-9a7c-fa2aa55d46e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:47.921547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e05a7eb0-d221-4ddc-9a7c-fa2aa55d46e2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:35:47.922750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e05a7eb0-d221-4ddc-9a7c-fa2aa55d46e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:35:47.927117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:47.997840 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 465/939] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:35:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:35:48.376001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f3982-b241-4125-a8e0-e4291105a626 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:48.494772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f3982-b241-4125-a8e0-e4291105a626 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:48.528767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:48.536566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d43332-9921-46a8-b29a-13a8c2c30d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:48.537433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d43332-9921-46a8-b29a-13a8c2c30d3c None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:48.537738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d43332-9921-46a8-b29a-13a8c2c30d3c None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:48.538058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d43332-9921-46a8-b29a-13a8c2c30d3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:48.550868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d43332-9921-46a8-b29a-13a8c2c30d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-94d43332-9921-46a8-b29a-13a8c2c30d3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:35:48.555947 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:48.557090 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:48.557926 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 475/940] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:35:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:48.564461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a7eb0-d221-4ddc-9a7c-fa2aa55d46e2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:48.683966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a7eb0-d221-4ddc-9a7c-fa2aa55d46e2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:48.724080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442bd3c-096d-4dae-8ce6-beb4ce7b908b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:48.725398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442bd3c-096d-4dae-8ce6-beb4ce7b908b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:48.725679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442bd3c-096d-4dae-8ce6-beb4ce7b908b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:48.725945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442bd3c-096d-4dae-8ce6-beb4ce7b908b None admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:48.726611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442bd3c-096d-4dae-8ce6-beb4ce7b908b None admin] RBAC: Adding query filter params (name=admin) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:48.726908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442bd3c-096d-4dae-8ce6-beb4ce7b908b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:48.728307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442bd3c-096d-4dae-8ce6-beb4ce7b908b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:48.737620 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 466/941] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:35:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:48.744665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d43332-9921-46a8-b29a-13a8c2c30d3c None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:48.860673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d43332-9921-46a8-b29a-13a8c2c30d3c None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:48.901876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283abc2-3d63-407c-975b-56a7445137c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:48.903147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283abc2-3d63-407c-975b-56a7445137c4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:48.903147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283abc2-3d63-407c-975b-56a7445137c4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:48.903147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283abc2-3d63-407c-975b-56a7445137c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:48.913930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283abc2-3d63-407c-975b-56a7445137c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:48.915461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283abc2-3d63-407c-975b-56a7445137c4 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-2283abc2-3d63-407c-975b-56a7445137c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:35:48.931725 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:48.932879 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:35:48.933825 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:35:48.933825 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:48.933825 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:48.933825 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 476/942] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:48.939923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442bd3c-096d-4dae-8ce6-beb4ce7b908b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:48.943127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:49.055995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442bd3c-096d-4dae-8ce6-beb4ce7b908b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:49.093381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6c1d8-3809-4737-9132-bbffae75f42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:49.093978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6c1d8-3809-4737-9132-bbffae75f42b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:49.094250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6c1d8-3809-4737-9132-bbffae75f42b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:49.094511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6c1d8-3809-4737-9132-bbffae75f42b None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:49.095136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6c1d8-3809-4737-9132-bbffae75f42b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:49.095428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6c1d8-3809-4737-9132-bbffae75f42b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:49.096787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6c1d8-3809-4737-9132-bbffae75f42b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:49.109874 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 467/943] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:49.116338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283abc2-3d63-407c-975b-56a7445137c4 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:49.165126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:49.232184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283abc2-3d63-407c-975b-56a7445137c4 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:49.270555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7238b6-1c6a-4802-8e5b-358c469d9e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:49.271189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7238b6-1c6a-4802-8e5b-358c469d9e95 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:49.271470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7238b6-1c6a-4802-8e5b-358c469d9e95 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:49.271741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7238b6-1c6a-4802-8e5b-358c469d9e95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:49.281068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7238b6-1c6a-4802-8e5b-358c469d9e95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:49.282666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7238b6-1c6a-4802-8e5b-358c469d9e95 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-5f7238b6-1c6a-4802-8e5b-358c469d9e95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:35:49.296330 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:49.297592 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:49.298706 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 477/944] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:49.304255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6c1d8-3809-4737-9132-bbffae75f42b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:49.376128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:49.421540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6c1d8-3809-4737-9132-bbffae75f42b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:49.461498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87d83a-be2b-4076-93aa-b4dba0be1a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:49.462162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87d83a-be2b-4076-93aa-b4dba0be1a42 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:49.462399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87d83a-be2b-4076-93aa-b4dba0be1a42 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:49.462625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87d83a-be2b-4076-93aa-b4dba0be1a42 None admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:49.463341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87d83a-be2b-4076-93aa-b4dba0be1a42 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:49.463636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87d83a-be2b-4076-93aa-b4dba0be1a42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:49.465121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87d83a-be2b-4076-93aa-b4dba0be1a42 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:49.483591 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 468/945] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Oct 15 20:35:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:49.489568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7238b6-1c6a-4802-8e5b-358c469d9e95 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:49.607330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7238b6-1c6a-4802-8e5b-358c469d9e95 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:49.609068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:49.649203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56395dd9-8257-426f-a216-67b364eec577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:49.649939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56395dd9-8257-426f-a216-67b364eec577 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:49.650256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56395dd9-8257-426f-a216-67b364eec577 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:49.650662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56395dd9-8257-426f-a216-67b364eec577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:49.663988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56395dd9-8257-426f-a216-67b364eec577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:49.665453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56395dd9-8257-426f-a216-67b364eec577 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-56395dd9-8257-426f-a216-67b364eec577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:35:49.682239 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:49.683404 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:35:49.684369 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:35:49.684369 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:49.684369 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:49.684369 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 478/946] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:49.690052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87d83a-be2b-4076-93aa-b4dba0be1a42 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:49.809967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87d83a-be2b-4076-93aa-b4dba0be1a42 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:49.834168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:49.852120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4128b39-60ed-45b6-baa5-858b0420f45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:49.852709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4128b39-60ed-45b6-baa5-858b0420f45b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:49.852981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4128b39-60ed-45b6-baa5-858b0420f45b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:49.853271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4128b39-60ed-45b6-baa5-858b0420f45b None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:49.853934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4128b39-60ed-45b6-baa5-858b0420f45b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:49.854200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4128b39-60ed-45b6-baa5-858b0420f45b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:49.855569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4128b39-60ed-45b6-baa5-858b0420f45b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:49.870185 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 469/947] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:49.876286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56395dd9-8257-426f-a216-67b364eec577 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:50.000212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56395dd9-8257-426f-a216-67b364eec577 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:50.046227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dc653-f680-4abe-89d2-76cde6fd6178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:50.046942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dc653-f680-4abe-89d2-76cde6fd6178 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:50.047287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dc653-f680-4abe-89d2-76cde6fd6178 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:50.047610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dc653-f680-4abe-89d2-76cde6fd6178 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:50.056324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039dc653-f680-4abe-89d2-76cde6fd6178 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:50.057780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039dc653-f680-4abe-89d2-76cde6fd6178 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-039dc653-f680-4abe-89d2-76cde6fd6178 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:35:50.069112 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:50.070952 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:50.072712 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:35:50.072712 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:35:50.072712 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:35:50.072712 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:35:50.072712 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:35:50.072712 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:35:50.072712 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:35:50.072712 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:50.072712 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 479/948] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:50.076911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4128b39-60ed-45b6-baa5-858b0420f45b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:50.193706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4128b39-60ed-45b6-baa5-858b0420f45b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:50.237796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a189c-c7ed-424a-bf16-67cfe9af1911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:50.238393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a189c-c7ed-424a-bf16-67cfe9af1911 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:50.238663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a189c-c7ed-424a-bf16-67cfe9af1911 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:50.238925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a189c-c7ed-424a-bf16-67cfe9af1911 None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:50.239863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a189c-c7ed-424a-bf16-67cfe9af1911 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:50.240158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a189c-c7ed-424a-bf16-67cfe9af1911 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:50.241682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a189c-c7ed-424a-bf16-67cfe9af1911 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:50.255180 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 470/949] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:50.261017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dc653-f680-4abe-89d2-76cde6fd6178 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:50.279815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:50.376530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dc653-f680-4abe-89d2-76cde6fd6178 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:50.421703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004d423-12aa-47a5-a08d-b494a1fb4ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:50.422359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d423-12aa-47a5-a08d-b494a1fb4ba3 None admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:50.422642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d423-12aa-47a5-a08d-b494a1fb4ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:50.422928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d423-12aa-47a5-a08d-b494a1fb4ba3 None admin] PATH_INFO: `/v3/projects/178e554a27104762a2d5b21906024be0/users/1ca62592177d43b5b07106003c9efa3b/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:50.424799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004d423-12aa-47a5-a08d-b494a1fb4ba3 None admin] RBAC: Authorizing `identity:create_grant(project_id=178e554a27104762a2d5b21906024be0, user_id=1ca62592177d43b5b07106003c9efa3b, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:50.426306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004d423-12aa-47a5-a08d-b494a1fb4ba3 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:50.440464 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 480/950] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Oct 15 20:35:50 2022] PUT /identity/v3/projects/178e554a27104762a2d5b21906024be0/users/1ca62592177d43b5b07106003c9efa3b/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:35:53.080316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ab827-3d00-4622-a1c8-f39623eaf7dc None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:53.080844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ab827-3d00-4622-a1c8-f39623eaf7dc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:53.080844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ab827-3d00-4622-a1c8-f39623eaf7dc None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:53.082191 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 471/951] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:35:53.087254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34efcd-3d26-4e2d-a373-4aaf02c24201 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:53.087680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34efcd-3d26-4e2d-a373-4aaf02c24201 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:53.087771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34efcd-3d26-4e2d-a373-4aaf02c24201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:53.105627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6e34efcd-3d26-4e2d-a373-4aaf02c24201 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:35:53.107391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e34efcd-3d26-4e2d-a373-4aaf02c24201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:35:53.171752 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 481/952] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:35:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:35:53.500151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ab827-3d00-4622-a1c8-f39623eaf7dc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:53.542761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:53.600983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ab827-3d00-4622-a1c8-f39623eaf7dc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:53.633707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c1d11-4648-4e5e-af4d-f7abd4c07b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:53.634305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c1d11-4648-4e5e-af4d-f7abd4c07b88 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:53.634572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c1d11-4648-4e5e-af4d-f7abd4c07b88 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:53.634832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c1d11-4648-4e5e-af4d-f7abd4c07b88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:53.645169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c1d11-4648-4e5e-af4d-f7abd4c07b88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-b50c1d11-4648-4e5e-af4d-f7abd4c07b88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 20:35:53.650265 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:53.651313 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:53.652459 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 472/953] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Oct 15 20:35:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:53.657403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34efcd-3d26-4e2d-a373-4aaf02c24201 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:53.660428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:53.755776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34efcd-3d26-4e2d-a373-4aaf02c24201 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:53.789250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77823479-251a-462d-be92-767f536d30c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:53.789833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77823479-251a-462d-be92-767f536d30c9 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:53.790100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77823479-251a-462d-be92-767f536d30c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:53.790362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77823479-251a-462d-be92-767f536d30c9 None admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:53.791010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77823479-251a-462d-be92-767f536d30c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:53.791305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77823479-251a-462d-be92-767f536d30c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:53.792646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77823479-251a-462d-be92-767f536d30c9 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:53.802307 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 482/954] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Oct 15 20:35:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:53.808106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c1d11-4648-4e5e-af4d-f7abd4c07b88 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:53.911576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:53.915004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c1d11-4648-4e5e-af4d-f7abd4c07b88 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:53.951078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cae8f9-ffae-40be-8101-79e98af6adc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:53.951692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cae8f9-ffae-40be-8101-79e98af6adc8 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:53.951962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cae8f9-ffae-40be-8101-79e98af6adc8 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:53.952226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cae8f9-ffae-40be-8101-79e98af6adc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:53.962360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cae8f9-ffae-40be-8101-79e98af6adc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:53.963800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cae8f9-ffae-40be-8101-79e98af6adc8 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-78cae8f9-ffae-40be-8101-79e98af6adc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:35:53.977426 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:53.978415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:35:53.979362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:35:53.979362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:53.979362 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:53.979362 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 473/955] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:53.984995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77823479-251a-462d-be92-767f536d30c9 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:54.051463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:54.085959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77823479-251a-462d-be92-767f536d30c9 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:54.119395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1ab17-52bb-4580-abbd-7ba139ed0589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:54.119978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1ab17-52bb-4580-abbd-7ba139ed0589 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:54.120247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1ab17-52bb-4580-abbd-7ba139ed0589 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:54.120510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1ab17-52bb-4580-abbd-7ba139ed0589 None admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:54.121163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1ab17-52bb-4580-abbd-7ba139ed0589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:54.121458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1ab17-52bb-4580-abbd-7ba139ed0589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:54.123753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1ab17-52bb-4580-abbd-7ba139ed0589 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:54.134953 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 483/956] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:54.141171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cae8f9-ffae-40be-8101-79e98af6adc8 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:54.242775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cae8f9-ffae-40be-8101-79e98af6adc8 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:54.274936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd722b-0126-45da-b332-d7d22b3d4dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:54.275545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd722b-0126-45da-b332-d7d22b3d4dcd None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:54.275816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd722b-0126-45da-b332-d7d22b3d4dcd None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:54.276078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd722b-0126-45da-b332-d7d22b3d4dcd None admin] PATH_INFO: `/v3/users/placement` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:54.283889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd722b-0126-45da-b332-d7d22b3d4dcd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:54.285355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd722b-0126-45da-b332-d7d22b3d4dcd None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-0ddd722b-0126-45da-b332-d7d22b3d4dcd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 20:35:54.296323 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:54.297393 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:54.298418 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 474/957] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:54.303265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1ab17-52bb-4580-abbd-7ba139ed0589 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:54.402070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1ab17-52bb-4580-abbd-7ba139ed0589 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:54.427712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:54.437762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9e74d-851a-4c23-b0bd-bac3b5439998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:54.438349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9e74d-851a-4c23-b0bd-bac3b5439998 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:54.438618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9e74d-851a-4c23-b0bd-bac3b5439998 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:54.438887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9e74d-851a-4c23-b0bd-bac3b5439998 None admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:54.439599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9e74d-851a-4c23-b0bd-bac3b5439998 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:54.439899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9e74d-851a-4c23-b0bd-bac3b5439998 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:54.441339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9e74d-851a-4c23-b0bd-bac3b5439998 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:54.457887 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 484/958] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Oct 15 20:35:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:54.463561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd722b-0126-45da-b332-d7d22b3d4dcd None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:54.478387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:54.566345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd722b-0126-45da-b332-d7d22b3d4dcd None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:54.598946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19babf3d-c7da-4e16-9faa-ed7f88df0900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:54.599527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19babf3d-c7da-4e16-9faa-ed7f88df0900 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:54.599798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19babf3d-c7da-4e16-9faa-ed7f88df0900 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:54.600060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19babf3d-c7da-4e16-9faa-ed7f88df0900 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:54.668179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19babf3d-c7da-4e16-9faa-ed7f88df0900 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:54.670311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19babf3d-c7da-4e16-9faa-ed7f88df0900 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:54.683267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:54.765963 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 475/959] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Oct 15 20:35:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 20:35:54.771914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9e74d-851a-4c23-b0bd-bac3b5439998 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:54.869412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9e74d-851a-4c23-b0bd-bac3b5439998 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:54.904928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49ca27-5c87-4aec-bfeb-8649780908dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:54.905584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49ca27-5c87-4aec-bfeb-8649780908dc None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:54.905873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49ca27-5c87-4aec-bfeb-8649780908dc None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:54.906136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49ca27-5c87-4aec-bfeb-8649780908dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:54.916065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49ca27-5c87-4aec-bfeb-8649780908dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:54.917515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49ca27-5c87-4aec-bfeb-8649780908dc None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-9b49ca27-5c87-4aec-bfeb-8649780908dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 20:35:54.932048 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:54.933214 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 20:35:54.934603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 20:35:54.934603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 20:35:54.934603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:54.934603 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 485/960] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:54.939090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19babf3d-c7da-4e16-9faa-ed7f88df0900 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:55.035555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19babf3d-c7da-4e16-9faa-ed7f88df0900 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:55.049767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:55.070881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7acaf-f5f0-4c68-99ca-b0cc2f09627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:55.071472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7acaf-f5f0-4c68-99ca-b0cc2f09627b None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:55.071746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7acaf-f5f0-4c68-99ca-b0cc2f09627b None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:55.072012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7acaf-f5f0-4c68-99ca-b0cc2f09627b None admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:55.072639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7acaf-f5f0-4c68-99ca-b0cc2f09627b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:55.072935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7acaf-f5f0-4c68-99ca-b0cc2f09627b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:55.074370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7acaf-f5f0-4c68-99ca-b0cc2f09627b None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:55.085583 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 476/961] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Oct 15 20:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:55.091619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49ca27-5c87-4aec-bfeb-8649780908dc None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:55.125858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:55.188295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49ca27-5c87-4aec-bfeb-8649780908dc None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:55.223033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25660317-45c8-4caf-a619-4d98c6550973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:55.223643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25660317-45c8-4caf-a619-4d98c6550973 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:55.223909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25660317-45c8-4caf-a619-4d98c6550973 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:55.224167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25660317-45c8-4caf-a619-4d98c6550973 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:55.230468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25660317-45c8-4caf-a619-4d98c6550973 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:55.231910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25660317-45c8-4caf-a619-4d98c6550973 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-25660317-45c8-4caf-a619-4d98c6550973 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 20:35:55.242769 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:55.243783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:55.245138 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 20:35:55.245138 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 20:35:55.245138 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 20:35:55.245138 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 20:35:55.245138 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 20:35:55.245138 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 20:35:55.245138 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 20:35:55.245138 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 20:35:55.245138 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 486/962] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Oct 15 20:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:55.249734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7acaf-f5f0-4c68-99ca-b0cc2f09627b None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:55.350385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7acaf-f5f0-4c68-99ca-b0cc2f09627b None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:55.382948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fb3f2-eab7-485a-95f4-a3ecd3dc9888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:55.383535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fb3f2-eab7-485a-95f4-a3ecd3dc9888 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:55.383808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fb3f2-eab7-485a-95f4-a3ecd3dc9888 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:55.384075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fb3f2-eab7-485a-95f4-a3ecd3dc9888 None admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:55.384742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fb3f2-eab7-485a-95f4-a3ecd3dc9888 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:55.385072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fb3f2-eab7-485a-95f4-a3ecd3dc9888 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:55.386543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fb3f2-eab7-485a-95f4-a3ecd3dc9888 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:55.398910 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 477/963] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Oct 15 20:35:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:55.405655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25660317-45c8-4caf-a619-4d98c6550973 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:55.495722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:55.502548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25660317-45c8-4caf-a619-4d98c6550973 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:55.534788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9771938-38d0-458c-b09f-a56bbdf9edc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:55.535365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9771938-38d0-458c-b09f-a56bbdf9edc1 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:55.535636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9771938-38d0-458c-b09f-a56bbdf9edc1 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:55.535901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9771938-38d0-458c-b09f-a56bbdf9edc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:55.536605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9771938-38d0-458c-b09f-a56bbdf9edc1 None admin] RBAC: Adding query filter params (role.id=fab66a1a6ba64c9583182c78978b218e, scope.project.id=178e554a27104762a2d5b21906024be0, user.id=1ca62592177d43b5b07106003c9efa3b) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:55.536903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9771938-38d0-458c-b09f-a56bbdf9edc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:55.538346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9771938-38d0-458c-b09f-a56bbdf9edc1 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:55.543841 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 487/964] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Oct 15 20:35:55 2022] GET /identity/v3/role_assignments?user.id=1ca62592177d43b5b07106003c9efa3b&scope.project.id=178e554a27104762a2d5b21906024be0&role.id=fab66a1a6ba64c9583182c78978b218e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:57.923604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4e97a-2a42-4ad2-a26d-e57392994045 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:57.924129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4e97a-2a42-4ad2-a26d-e57392994045 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:57.924129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4e97a-2a42-4ad2-a26d-e57392994045 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:57.925506 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 478/965] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:35:57.930625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dc6eb-315c-44ba-bb2a-741fa9775a8d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:57.931004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dc6eb-315c-44ba-bb2a-741fa9775a8d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:57.931147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dc6eb-315c-44ba-bb2a-741fa9775a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:57.949426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b56dc6eb-315c-44ba-bb2a-741fa9775a8d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:35:57.951146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b56dc6eb-315c-44ba-bb2a-741fa9775a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:35:58.016552 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 488/966] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:35:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:35:58.345536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4e97a-2a42-4ad2-a26d-e57392994045 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:58.376232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:58.444967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4e97a-2a42-4ad2-a26d-e57392994045 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:58.477984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdddea-9c9c-487d-b871-6001164619f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:58.478592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdddea-9c9c-487d-b871-6001164619f6 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:58.478861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdddea-9c9c-487d-b871-6001164619f6 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:58.479122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdddea-9c9c-487d-b871-6001164619f6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:58.479680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdddea-9c9c-487d-b871-6001164619f6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:58.481074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdddea-9c9c-487d-b871-6001164619f6 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-85fdddea-9c9c-487d-b871-6001164619f6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:35:58.489757 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:35:58.490917 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:35:58.491907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 479/967] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Oct 15 20:35:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:35:58.497384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dc6eb-315c-44ba-bb2a-741fa9775a8d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:58.585583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:58.595491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dc6eb-315c-44ba-bb2a-741fa9775a8d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:58.628475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c955e2-ab56-40d7-b3fa-c6e0588c87d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:58.629078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c955e2-ab56-40d7-b3fa-c6e0588c87d4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:58.629356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c955e2-ab56-40d7-b3fa-c6e0588c87d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:58.629623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c955e2-ab56-40d7-b3fa-c6e0588c87d4 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:58.630244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c955e2-ab56-40d7-b3fa-c6e0588c87d4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:58.630544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c955e2-ab56-40d7-b3fa-c6e0588c87d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:58.631892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c955e2-ab56-40d7-b3fa-c6e0588c87d4 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:58.636534 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 489/968] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Oct 15 20:35:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:35:58.643133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdddea-9c9c-487d-b871-6001164619f6 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:35:58.736034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:35:58.742553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdddea-9c9c-487d-b871-6001164619f6 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:35:58.775833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e348e-6990-43b7-9721-887c57b3ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:35:58.776410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e348e-6990-43b7-9721-887c57b3ff49 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:35:58.776686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e348e-6990-43b7-9721-887c57b3ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:35:58.776953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e348e-6990-43b7-9721-887c57b3ff49 None admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:35:58.777598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e348e-6990-43b7-9721-887c57b3ff49 None admin] RBAC: Adding query filter params (type=placement) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:35:58.777932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e348e-6990-43b7-9721-887c57b3ff49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:35:58.779295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e348e-6990-43b7-9721-887c57b3ff49 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:35:58.783555 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 480/969] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Oct 15 20:35:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:36:01.149857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36398a0c-a211-4574-b136-df373419d5e3 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:01.150434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36398a0c-a211-4574-b136-df373419d5e3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:01.150434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36398a0c-a211-4574-b136-df373419d5e3 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:01.151711 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 490/970] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:36:01.156846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce315-2be4-4a7a-8863-9f3dacf90275 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:01.157288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce315-2be4-4a7a-8863-9f3dacf90275 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:01.157432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce315-2be4-4a7a-8863-9f3dacf90275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:01.176128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2ebce315-2be4-4a7a-8863-9f3dacf90275 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:36:01.177348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2ebce315-2be4-4a7a-8863-9f3dacf90275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:36:01.242749 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 481/971] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:36:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 20:36:01.571777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36398a0c-a211-4574-b136-df373419d5e3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:36:01.672597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36398a0c-a211-4574-b136-df373419d5e3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:36:01.705315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600388cf-7115-4a99-912a-7164c86de199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:36:01.705893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600388cf-7115-4a99-912a-7164c86de199 None admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:01.706164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600388cf-7115-4a99-912a-7164c86de199 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:01.706429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600388cf-7115-4a99-912a-7164c86de199 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:01.707074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600388cf-7115-4a99-912a-7164c86de199 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:36:01.708390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600388cf-7115-4a99-912a-7164c86de199 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:36:01.722994 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 491/972] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Oct 15 20:36:01 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:36:04.130865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912539c-ff77-4b6f-924d-3e89d49c2d40 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:04.131413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912539c-ff77-4b6f-924d-3e89d49c2d40 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:04.131413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912539c-ff77-4b6f-924d-3e89d49c2d40 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:04.132725 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 482/973] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:36:04.138475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fc948-7713-4b0e-b929-cdc3d9d1ea62 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:04.138921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fc948-7713-4b0e-b929-cdc3d9d1ea62 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:04.139022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fc948-7713-4b0e-b929-cdc3d9d1ea62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:04.157205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-529fc948-7713-4b0e-b929-cdc3d9d1ea62 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:36:04.158377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-529fc948-7713-4b0e-b929-cdc3d9d1ea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:36:04.167264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:36:04.232509 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 492/974] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:36:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:36:04.565289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912539c-ff77-4b6f-924d-3e89d49c2d40 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:36:04.655394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:36:04.666263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912539c-ff77-4b6f-924d-3e89d49c2d40 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:36:04.699204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae62b63-c15a-4bf1-9554-9c0964fba9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:36:04.700464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae62b63-c15a-4bf1-9554-9c0964fba9db None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:04.700745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae62b63-c15a-4bf1-9554-9c0964fba9db None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:04.701010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae62b63-c15a-4bf1-9554-9c0964fba9db None admin] PATH_INFO: `/v3/services/placement` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:04.701610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae62b63-c15a-4bf1-9554-9c0964fba9db None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:36:04.702980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae62b63-c15a-4bf1-9554-9c0964fba9db None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-dae62b63-c15a-4bf1-9554-9c0964fba9db None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:36:04.711511 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:36:04.712815 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:36:04.714042 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 483/975] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Oct 15 20:36:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:36:04.718890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fc948-7713-4b0e-b929-cdc3d9d1ea62 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:36:04.814988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fc948-7713-4b0e-b929-cdc3d9d1ea62 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:36:04.847391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d6318-d04c-4fe9-a107-414eb1df1b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:36:04.847908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d6318-d04c-4fe9-a107-414eb1df1b87 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:04.848174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d6318-d04c-4fe9-a107-414eb1df1b87 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:04.848518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d6318-d04c-4fe9-a107-414eb1df1b87 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:04.849208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d6318-d04c-4fe9-a107-414eb1df1b87 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:36:04.849603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d6318-d04c-4fe9-a107-414eb1df1b87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:36:04.851084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d6318-d04c-4fe9-a107-414eb1df1b87 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:36:04.857264 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 493/976] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Oct 15 20:36:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:36:04.862946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae62b63-c15a-4bf1-9554-9c0964fba9db None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:36:04.973834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae62b63-c15a-4bf1-9554-9c0964fba9db None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:36:05.007098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb0d49-2176-4d82-9b0b-de4f76fc0ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:36:05.007684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb0d49-2176-4d82-9b0b-de4f76fc0ce0 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:05.007955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb0d49-2176-4d82-9b0b-de4f76fc0ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:05.008219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb0d49-2176-4d82-9b0b-de4f76fc0ce0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:05.008903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb0d49-2176-4d82-9b0b-de4f76fc0ce0 None admin] RBAC: Adding query filter params (interface=public, service_id=1d8702c1c01140d8abd591926969a285, region_id=RegionOne) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:36:05.009235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb0d49-2176-4d82-9b0b-de4f76fc0ce0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:36:05.010603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb0d49-2176-4d82-9b0b-de4f76fc0ce0 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:36:05.015584 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 484/977] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Oct 15 20:36:04 2022] GET /identity/v3/endpoints?service_id=1d8702c1c01140d8abd591926969a285&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:36:05.021414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d6318-d04c-4fe9-a107-414eb1df1b87 None admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:36:05.025058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:36:05.118722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d6318-d04c-4fe9-a107-414eb1df1b87 None admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:36:05.150907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6861b-1b5d-462d-b152-e4d36015ed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:36:05.151484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6861b-1b5d-462d-b152-e4d36015ed14 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:05.151757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6861b-1b5d-462d-b152-e4d36015ed14 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:05.152023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6861b-1b5d-462d-b152-e4d36015ed14 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:05.152974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6861b-1b5d-462d-b152-e4d36015ed14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:36:05.154359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6861b-1b5d-462d-b152-e4d36015ed14 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:36:05.160920 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 494/978] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Oct 15 20:36:05 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:36:07.523408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fc8ff-6aea-4d8c-890e-1d96d92fdf66 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:07.523926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fc8ff-6aea-4d8c-890e-1d96d92fdf66 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:07.523926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fc8ff-6aea-4d8c-890e-1d96d92fdf66 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:07.525214 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 485/979] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:36:07.530900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba62d7-23d4-4f7a-8476-164af0460f2b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:07.531279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba62d7-23d4-4f7a-8476-164af0460f2b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:07.531425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba62d7-23d4-4f7a-8476-164af0460f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:07.549764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-05ba62d7-23d4-4f7a-8476-164af0460f2b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:36:07.550922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-05ba62d7-23d4-4f7a-8476-164af0460f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:36:07.610013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:36:07.616074 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 495/980] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:36:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 20:36:07.946300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fc8ff-6aea-4d8c-890e-1d96d92fdf66 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:36:07.966468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:36:08.047450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fc8ff-6aea-4d8c-890e-1d96d92fdf66 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:36:08.079636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26860ea7-9d1b-4209-8def-083e15d3b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:36:08.080238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860ea7-9d1b-4209-8def-083e15d3b311 None admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:08.080507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860ea7-9d1b-4209-8def-083e15d3b311 None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:08.080767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860ea7-9d1b-4209-8def-083e15d3b311 None admin] PATH_INFO: `/v3/services/placement` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:08.081353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26860ea7-9d1b-4209-8def-083e15d3b311 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:36:08.082707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26860ea7-9d1b-4209-8def-083e15d3b311 None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-26860ea7-9d1b-4209-8def-083e15d3b311 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 20:36:08.091649 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 20:36:08.092664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 20:36:08.093590 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 486/981] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Oct 15 20:36:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 20:36:08.098686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba62d7-23d4-4f7a-8476-164af0460f2b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:36:08.195925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba62d7-23d4-4f7a-8476-164af0460f2b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:36:08.229804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258791c7-f32b-4233-b57e-af88852de3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:36:08.230330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258791c7-f32b-4233-b57e-af88852de3b4 None admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:08.230667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258791c7-f32b-4233-b57e-af88852de3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:08.231029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258791c7-f32b-4233-b57e-af88852de3b4 None admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:08.231653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258791c7-f32b-4233-b57e-af88852de3b4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:36:08.231955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258791c7-f32b-4233-b57e-af88852de3b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:36:08.233338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258791c7-f32b-4233-b57e-af88852de3b4 None admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:36:08.238320 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 496/982] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Oct 15 20:36:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:36:08.244305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26860ea7-9d1b-4209-8def-083e15d3b311 None admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:36:08.326400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:36:08.344926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26860ea7-9d1b-4209-8def-083e15d3b311 None admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:36:08.384837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b667fdd-c6b7-4151-9c41-e8055e20116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:36:08.385467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b667fdd-c6b7-4151-9c41-e8055e20116f None admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:36:08.385741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b667fdd-c6b7-4151-9c41-e8055e20116f None admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:36:08.386006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b667fdd-c6b7-4151-9c41-e8055e20116f None admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:36:08.386671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b667fdd-c6b7-4151-9c41-e8055e20116f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:36:08.388604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b667fdd-c6b7-4151-9c41-e8055e20116f None admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:36:08.408388 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 487/983] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:36:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:36:18.235872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=69 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:36:18.404480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:21.174160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0704d1b-38f3-4134-8d0d-5b7ab5de7511 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:21.175238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0704d1b-38f3-4134-8d0d-5b7ab5de7511 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:21.175938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0704d1b-38f3-4134-8d0d-5b7ab5de7511 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:21.178968 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 497/984] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:21.186606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb69048-9a5c-4d83-a035-fa85423f9acf None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:21.187748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb69048-9a5c-4d83-a035-fa85423f9acf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:21.188461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb69048-9a5c-4d83-a035-fa85423f9acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:21.320452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fbb69048-9a5c-4d83-a035-fa85423f9acf None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:21.323312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fbb69048-9a5c-4d83-a035-fa85423f9acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:21.492136 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 488/985] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:21.843474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a030272-744c-4fb0-ad06-26356809a23b None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:21.843750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a030272-744c-4fb0-ad06-26356809a23b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:21.844265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a030272-744c-4fb0-ad06-26356809a23b None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:21.845836 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 498/986] 10.222.0.16 () {54 vars in 964 bytes} [Sat Oct 15 20:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:21.851775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b795-078f-4211-b368-d0c72b17b1d9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:21.852117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b795-078f-4211-b368-d0c72b17b1d9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:21.852973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b795-078f-4211-b368-d0c72b17b1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:21.934539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ec25b795-078f-4211-b368-d0c72b17b1d9 None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:21.935726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ec25b795-078f-4211-b368-d0c72b17b1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:21.957576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:22.023058 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 489/987] 10.222.0.16 () {58 vars in 1061 bytes} [Sat Oct 15 20:37:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:22.032837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a030272-744c-4fb0-ad06-26356809a23b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:37:22.034743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5a030272-744c-4fb0-ad06-26356809a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:22.158075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a030272-744c-4fb0-ad06-26356809a23b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:37:22.260146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff068c05-8fcf-4738-8c78-6599742094af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:37:22.260918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff068c05-8fcf-4738-8c78-6599742094af service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:22.265267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff068c05-8fcf-4738-8c78-6599742094af service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:22.265746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff068c05-8fcf-4738-8c78-6599742094af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:22.268112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ff068c05-8fcf-4738-8c78-6599742094af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:22.338917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:22.407244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff068c05-8fcf-4738-8c78-6599742094af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:37:22.409277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff068c05-8fcf-4738-8c78-6599742094af service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:37:22.433237 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 499/988] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:37:25.353389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653bff5-16b2-4c43-a444-eed1302566ac None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:25.353933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653bff5-16b2-4c43-a444-eed1302566ac None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:25.354209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653bff5-16b2-4c43-a444-eed1302566ac None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:25.357423 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 490/989] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:25.360929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db18bf9-b884-4b1f-8e3c-59ee0ead1e9f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:25.361388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db18bf9-b884-4b1f-8e3c-59ee0ead1e9f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:25.361575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db18bf9-b884-4b1f-8e3c-59ee0ead1e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:25.450227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5db18bf9-b884-4b1f-8e3c-59ee0ead1e9f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:25.451472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5db18bf9-b884-4b1f-8e3c-59ee0ead1e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:25.557053 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 500/990] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:25.897049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653bff5-16b2-4c43-a444-eed1302566ac None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:37:25.898601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a653bff5-16b2-4c43-a444-eed1302566ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:26.005685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653bff5-16b2-4c43-a444-eed1302566ac None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:37:26.080339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:26.082777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29be803-ebc9-470b-aa12-bb42b48c54f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:37:26.083396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29be803-ebc9-470b-aa12-bb42b48c54f7 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:26.083857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29be803-ebc9-470b-aa12-bb42b48c54f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:26.084312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29be803-ebc9-470b-aa12-bb42b48c54f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:26.086176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e29be803-ebc9-470b-aa12-bb42b48c54f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:26.198349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29be803-ebc9-470b-aa12-bb42b48c54f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:37:26.199801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29be803-ebc9-470b-aa12-bb42b48c54f7 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:37:26.220460 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 491/991] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:37:28.971357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3663-39f0-4a8a-9b7f-b05d5a569039 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:28.971894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3663-39f0-4a8a-9b7f-b05d5a569039 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:28.971894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3663-39f0-4a8a-9b7f-b05d5a569039 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:28.973286 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 501/992] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:28.979427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c793a-074a-4917-9e5c-91e2bbfc58a5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:28.979844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c793a-074a-4917-9e5c-91e2bbfc58a5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:28.979963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c793a-074a-4917-9e5c-91e2bbfc58a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:29.047537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c43c793a-074a-4917-9e5c-91e2bbfc58a5 None None] MFA Rules not processed for user `22b1d8706e1e41dbb9984cbc4c9d7be5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:29.048729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c43c793a-074a-4917-9e5c-91e2bbfc58a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:29.098839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:29.142525 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 492/993] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:29.495291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c3663-39f0-4a8a-9b7f-b05d5a569039 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:37:29.496810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-485c3663-39f0-4a8a-9b7f-b05d5a569039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:29.507288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:29.600659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c3663-39f0-4a8a-9b7f-b05d5a569039 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:37:29.675377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5035d-2228-4c88-982e-a9d1627fb803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:37:29.675969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5035d-2228-4c88-982e-a9d1627fb803 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:29.676240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5035d-2228-4c88-982e-a9d1627fb803 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:29.676504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5035d-2228-4c88-982e-a9d1627fb803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:29.678149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-89f5035d-2228-4c88-982e-a9d1627fb803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:29.738662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:29.786391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5035d-2228-4c88-982e-a9d1627fb803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:37:29.787736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5035d-2228-4c88-982e-a9d1627fb803 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:37:29.805846 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 502/994] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:37:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:37:32.779624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffccdb-1796-49d8-9a24-fbd72eafb5fe None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:32.780252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffccdb-1796-49d8-9a24-fbd72eafb5fe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:32.780252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffccdb-1796-49d8-9a24-fbd72eafb5fe None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:32.781537 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 493/995] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:32.786646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44056088-cddd-41ea-81f5-0c68c1db060b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:32.787078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44056088-cddd-41ea-81f5-0c68c1db060b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:32.787182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44056088-cddd-41ea-81f5-0c68c1db060b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:32.853863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-44056088-cddd-41ea-81f5-0c68c1db060b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:32.855055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-44056088-cddd-41ea-81f5-0c68c1db060b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:32.951886 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 503/996] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:33.290583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffccdb-1796-49d8-9a24-fbd72eafb5fe None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:37:33.292118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cdffccdb-1796-49d8-9a24-fbd72eafb5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:33.394564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffccdb-1796-49d8-9a24-fbd72eafb5fe None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:37:33.467867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af386a6b-4b1b-459e-b6ae-baa18693a849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:37:33.468380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af386a6b-4b1b-459e-b6ae-baa18693a849 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:33.468651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af386a6b-4b1b-459e-b6ae-baa18693a849 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:33.468916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af386a6b-4b1b-459e-b6ae-baa18693a849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:33.470514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af386a6b-4b1b-459e-b6ae-baa18693a849 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:33.477574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:33.579662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af386a6b-4b1b-459e-b6ae-baa18693a849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:37:33.581291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af386a6b-4b1b-459e-b6ae-baa18693a849 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:37:33.599325 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 494/997] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:37:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:37:36.644011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dea2c2-fd9d-466f-b306-6badc2c36b6d None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:36.644545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dea2c2-fd9d-466f-b306-6badc2c36b6d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:36.644545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dea2c2-fd9d-466f-b306-6badc2c36b6d None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:36.645898 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 504/998] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:36.651217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd304319-dcb5-448e-88b8-747b0e074440 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:36.651640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd304319-dcb5-448e-88b8-747b0e074440 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:36.651762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd304319-dcb5-448e-88b8-747b0e074440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:36.718259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cd304319-dcb5-448e-88b8-747b0e074440 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:36.719456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cd304319-dcb5-448e-88b8-747b0e074440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:36.777236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:36.816775 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 495/999] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:37.178980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f6aa8-d8d9-43d6-b2f7-fc3dd8c6d631 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:37.179255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f6aa8-d8d9-43d6-b2f7-fc3dd8c6d631 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:37.179523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f6aa8-d8d9-43d6-b2f7-fc3dd8c6d631 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:37.181458 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 505/1000] 10.222.0.16 () {54 vars in 964 bytes} [Sat Oct 15 20:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:37.187289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e872a-cd0b-4610-b544-8c40e3e5abae None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:37.187572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e872a-cd0b-4610-b544-8c40e3e5abae None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:37.187844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e872a-cd0b-4610-b544-8c40e3e5abae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:37.266164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9b6e872a-cd0b-4610-b544-8c40e3e5abae None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:37.267392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9b6e872a-cd0b-4610-b544-8c40e3e5abae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:37.343234 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 496/1001] 10.222.0.16 () {58 vars in 1061 bytes} [Sat Oct 15 20:37:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:37.352844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f6aa8-d8d9-43d6-b2f7-fc3dd8c6d631 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:37:37.354446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ae7f6aa8-d8d9-43d6-b2f7-fc3dd8c6d631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:37.374588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:37.464555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f6aa8-d8d9-43d6-b2f7-fc3dd8c6d631 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:37:37.539010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51054af4-788a-418b-8021-a7c601f37f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:37:37.540329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054af4-788a-418b-8021-a7c601f37f99 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:37.540606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054af4-788a-418b-8021-a7c601f37f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:37.540870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054af4-788a-418b-8021-a7c601f37f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:37.542483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-51054af4-788a-418b-8021-a7c601f37f99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:37.624474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:37.667040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51054af4-788a-418b-8021-a7c601f37f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:37:37.668515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51054af4-788a-418b-8021-a7c601f37f99 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:37:37.688346 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 506/1002] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:37:40.397171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aea999-2491-47ab-8a3f-d2cf1ffb71e6 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:40.397740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aea999-2491-47ab-8a3f-d2cf1ffb71e6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:40.397740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aea999-2491-47ab-8a3f-d2cf1ffb71e6 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:40.399192 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 497/1003] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:40.405393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afcc41-d353-464f-a05a-391a3785e2ef None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:40.405942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afcc41-d353-464f-a05a-391a3785e2ef None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:40.405942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afcc41-d353-464f-a05a-391a3785e2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:40.471518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-95afcc41-d353-464f-a05a-391a3785e2ef None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:40.472624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-95afcc41-d353-464f-a05a-391a3785e2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:40.569229 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 507/1004] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:40.909454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aea999-2491-47ab-8a3f-d2cf1ffb71e6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:37:40.911022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-97aea999-2491-47ab-8a3f-d2cf1ffb71e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:40.977801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:41.026712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aea999-2491-47ab-8a3f-d2cf1ffb71e6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:37:41.107243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2e801-bbe6-4b8b-ae28-6dce3d246175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:37:41.107838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e801-bbe6-4b8b-ae28-6dce3d246175 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:41.108107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e801-bbe6-4b8b-ae28-6dce3d246175 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:41.108371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e801-bbe6-4b8b-ae28-6dce3d246175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:41.109961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-90d2e801-bbe6-4b8b-ae28-6dce3d246175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:41.224172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:41.226732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2e801-bbe6-4b8b-ae28-6dce3d246175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:37:41.228850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2e801-bbe6-4b8b-ae28-6dce3d246175 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:37:41.248358 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 498/1005] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:37:43.707726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126046c-c522-41db-9167-7561b1111acf None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:43.708290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126046c-c522-41db-9167-7561b1111acf None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:43.708290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126046c-c522-41db-9167-7561b1111acf None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:43.709627 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 508/1006] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:43.715601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d914-dea5-405e-95fd-f62df6dbe275 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:43.716048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d914-dea5-405e-95fd-f62df6dbe275 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:43.716157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d914-dea5-405e-95fd-f62df6dbe275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:43.783511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-36b1d914-dea5-405e-95fd-f62df6dbe275 None None] MFA Rules not processed for user `22b1d8706e1e41dbb9984cbc4c9d7be5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:43.784715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-36b1d914-dea5-405e-95fd-f62df6dbe275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:43.882538 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 499/1007] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:44.219887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126046c-c522-41db-9167-7561b1111acf None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:37:44.221642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1126046c-c522-41db-9167-7561b1111acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:44.256338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:44.337287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126046c-c522-41db-9167-7561b1111acf None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:37:44.417473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814048e-9387-4ff0-a4a3-bce6b5602c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:37:44.418087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814048e-9387-4ff0-a4a3-bce6b5602c67 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:44.418358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814048e-9387-4ff0-a4a3-bce6b5602c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:44.418617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814048e-9387-4ff0-a4a3-bce6b5602c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:44.420199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a814048e-9387-4ff0-a4a3-bce6b5602c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:44.499949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:44.533608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814048e-9387-4ff0-a4a3-bce6b5602c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:37:44.535072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814048e-9387-4ff0-a4a3-bce6b5602c67 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:37:44.552941 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 509/1008] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:37:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:37:47.404750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b836cee-eac6-4d3b-ac52-c674b8aabfc8 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:47.405249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b836cee-eac6-4d3b-ac52-c674b8aabfc8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:47.405352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b836cee-eac6-4d3b-ac52-c674b8aabfc8 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:47.406671 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 500/1009] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:47.411905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682b3e0-07dc-4377-a6b2-fec0f77265da None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:47.412314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682b3e0-07dc-4377-a6b2-fec0f77265da None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:47.412449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682b3e0-07dc-4377-a6b2-fec0f77265da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:47.483557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0682b3e0-07dc-4377-a6b2-fec0f77265da None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:47.484783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0682b3e0-07dc-4377-a6b2-fec0f77265da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:47.581785 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 510/1010] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:47.919465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b836cee-eac6-4d3b-ac52-c674b8aabfc8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:37:47.921088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9b836cee-eac6-4d3b-ac52-c674b8aabfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:47.995164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:48.032601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b836cee-eac6-4d3b-ac52-c674b8aabfc8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:37:48.109954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231567e-cac7-4629-9806-e6febe2d4539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:37:48.110496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231567e-cac7-4629-9806-e6febe2d4539 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:48.110773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231567e-cac7-4629-9806-e6febe2d4539 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:48.111043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231567e-cac7-4629-9806-e6febe2d4539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:48.112678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e231567e-cac7-4629-9806-e6febe2d4539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:48.232584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231567e-cac7-4629-9806-e6febe2d4539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:37:48.234055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231567e-cac7-4629-9806-e6febe2d4539 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:37:48.245136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:48.253993 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 501/1011] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:37:50.696108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fd548-bfe6-4a3e-bf58-35b504b0f1c4 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:50.696660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fd548-bfe6-4a3e-bf58-35b504b0f1c4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:50.696660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fd548-bfe6-4a3e-bf58-35b504b0f1c4 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:50.697986 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 511/1012] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:50.703230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeed15-6957-43d3-ba4d-ca593325940e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:50.703635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeed15-6957-43d3-ba4d-ca593325940e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:50.703788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeed15-6957-43d3-ba4d-ca593325940e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:50.769866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-edeeed15-6957-43d3-ba4d-ca593325940e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:50.771046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-edeeed15-6957-43d3-ba4d-ca593325940e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:50.866016 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 502/1013] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:51.211622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fd548-bfe6-4a3e-bf58-35b504b0f1c4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:37:51.213160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5c3fd548-bfe6-4a3e-bf58-35b504b0f1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:51.252454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:51.316111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fd548-bfe6-4a3e-bf58-35b504b0f1c4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:37:51.388588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72061df5-5eb4-4928-9495-268542aba8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:37:51.389216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72061df5-5eb4-4928-9495-268542aba8ce service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:51.389491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72061df5-5eb4-4928-9495-268542aba8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:51.389755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72061df5-5eb4-4928-9495-268542aba8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:51.391323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-72061df5-5eb4-4928-9495-268542aba8ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:51.477124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:51.503981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72061df5-5eb4-4928-9495-268542aba8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:37:51.505481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72061df5-5eb4-4928-9495-268542aba8ce service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:37:51.524794 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 512/1014] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:37:54.198593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d011c-6e61-46ef-88c1-262f9914d3d1 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:54.198963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d011c-6e61-46ef-88c1-262f9914d3d1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:54.199120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d011c-6e61-46ef-88c1-262f9914d3d1 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:54.200465 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 503/1015] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:54.205750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dafbc-be5c-4ad0-9b7e-14fed3dce12b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:54.206135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dafbc-be5c-4ad0-9b7e-14fed3dce12b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:54.206324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dafbc-be5c-4ad0-9b7e-14fed3dce12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:54.273750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9e7dafbc-be5c-4ad0-9b7e-14fed3dce12b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:54.274954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e7dafbc-be5c-4ad0-9b7e-14fed3dce12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:54.369954 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 513/1016] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:37:54.707983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d011c-6e61-46ef-88c1-262f9914d3d1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:37:54.709533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aa4d011c-6e61-46ef-88c1-262f9914d3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:54.784464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:54.813910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d011c-6e61-46ef-88c1-262f9914d3d1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:37:54.888206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebaa5a-13ee-46ba-a246-0e1aff0244f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:37:54.888870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebaa5a-13ee-46ba-a246-0e1aff0244f5 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:54.889186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebaa5a-13ee-46ba-a246-0e1aff0244f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:54.889600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebaa5a-13ee-46ba-a246-0e1aff0244f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:54.891215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a9ebaa5a-13ee-46ba-a246-0e1aff0244f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:55.007000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebaa5a-13ee-46ba-a246-0e1aff0244f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:37:55.008994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebaa5a-13ee-46ba-a246-0e1aff0244f5 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:37:55.027653 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 504/1017] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:37:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:37:59.540745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a1f6e-2652-4abc-a1a3-24e6fe17a24f None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:59.541325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a1f6e-2652-4abc-a1a3-24e6fe17a24f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:59.541325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a1f6e-2652-4abc-a1a3-24e6fe17a24f None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:59.542617 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 514/1018] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:37:59.547894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95149d36-fbbe-4502-9385-56b862d6097b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:37:59.548304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95149d36-fbbe-4502-9385-56b862d6097b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:37:59.548425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95149d36-fbbe-4502-9385-56b862d6097b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:37:59.554486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:37:59.615995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-95149d36-fbbe-4502-9385-56b862d6097b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:37:59.617973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-95149d36-fbbe-4502-9385-56b862d6097b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:37:59.716089 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 505/1019] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:00.057983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a1f6e-2652-4abc-a1a3-24e6fe17a24f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:00.059509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-815a1f6e-2652-4abc-a1a3-24e6fe17a24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:00.109273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:00.167157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a1f6e-2652-4abc-a1a3-24e6fe17a24f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:00.241374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53edab-9163-475e-9c92-e408edb913e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:00.242478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53edab-9163-475e-9c92-e408edb913e9 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:00.243284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53edab-9163-475e-9c92-e408edb913e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:00.243555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53edab-9163-475e-9c92-e408edb913e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:00.245141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2d53edab-9163-475e-9c92-e408edb913e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:00.339583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:00.355935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53edab-9163-475e-9c92-e408edb913e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:00.357405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53edab-9163-475e-9c92-e408edb913e9 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:00.377165 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 515/1020] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:03.406884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f08b60-4b92-40e1-9e51-a4d1f0b48c1a None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:03.407467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f08b60-4b92-40e1-9e51-a4d1f0b48c1a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:03.407467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f08b60-4b92-40e1-9e51-a4d1f0b48c1a None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:03.408842 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 506/1021] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:03.414665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeecd15-7652-4957-a811-b099563e78d3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:03.415096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeecd15-7652-4957-a811-b099563e78d3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:03.415194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeecd15-7652-4957-a811-b099563e78d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:03.480622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-feeecd15-7652-4957-a811-b099563e78d3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:03.481852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-feeecd15-7652-4957-a811-b099563e78d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:03.577070 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 516/1022] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:03.924856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f08b60-4b92-40e1-9e51-a4d1f0b48c1a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:03.926525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-64f08b60-4b92-40e1-9e51-a4d1f0b48c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:04.015678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:04.033217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f08b60-4b92-40e1-9e51-a4d1f0b48c1a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:04.109740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade95db-12c8-430c-8f5c-55e0684b4dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:04.110420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade95db-12c8-430c-8f5c-55e0684b4dfb service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:04.110717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade95db-12c8-430c-8f5c-55e0684b4dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:04.110987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade95db-12c8-430c-8f5c-55e0684b4dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:04.112590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9ade95db-12c8-430c-8f5c-55e0684b4dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:04.232099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade95db-12c8-430c-8f5c-55e0684b4dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:04.233619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade95db-12c8-430c-8f5c-55e0684b4dfb service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:04.252604 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 507/1023] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:08.602352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eec61d-4423-404e-85e6-f82979fda1c7 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:08.602907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eec61d-4423-404e-85e6-f82979fda1c7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:08.602907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eec61d-4423-404e-85e6-f82979fda1c7 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:08.604195 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 517/1024] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:08.609578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d21d-be2c-4222-a8e5-5a14e7fcb31c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:08.610004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d21d-be2c-4222-a8e5-5a14e7fcb31c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:08.610179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d21d-be2c-4222-a8e5-5a14e7fcb31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:08.626857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:08.679447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9961d21d-be2c-4222-a8e5-5a14e7fcb31c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:08.680664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9961d21d-be2c-4222-a8e5-5a14e7fcb31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:08.779195 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 508/1025] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:09.120664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eec61d-4423-404e-85e6-f82979fda1c7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:09.122226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-40eec61d-4423-404e-85e6-f82979fda1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:09.196536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:09.243570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eec61d-4423-404e-85e6-f82979fda1c7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:09.321406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bac968-51cb-48c7-a93d-14825d142f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:09.322006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bac968-51cb-48c7-a93d-14825d142f23 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:09.322282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bac968-51cb-48c7-a93d-14825d142f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:09.322551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bac968-51cb-48c7-a93d-14825d142f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:09.324141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8bac968-51cb-48c7-a93d-14825d142f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:09.433907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:09.438445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bac968-51cb-48c7-a93d-14825d142f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:09.439871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bac968-51cb-48c7-a93d-14825d142f23 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:09.458051 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 518/1026] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:14.917610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e74e5-5cb4-4312-82c6-fd09efe206fd None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:14.918133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e74e5-5cb4-4312-82c6-fd09efe206fd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:14.918133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e74e5-5cb4-4312-82c6-fd09efe206fd None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:14.919549 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 509/1027] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:14.925373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c66ae-0bf5-48af-a37f-9ce3b6271f65 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:14.926173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c66ae-0bf5-48af-a37f-9ce3b6271f65 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:14.926670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c66ae-0bf5-48af-a37f-9ce3b6271f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:15.008445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-229c66ae-0bf5-48af-a37f-9ce3b6271f65 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:15.009969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-229c66ae-0bf5-48af-a37f-9ce3b6271f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:15.149274 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 519/1028] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:38:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:18.777616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=59 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:19.612582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd864e-68bc-40b8-9a4a-6a2ac14dd8fd None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:19.612852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd864e-68bc-40b8-9a4a-6a2ac14dd8fd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:19.613328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd864e-68bc-40b8-9a4a-6a2ac14dd8fd None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:19.614928 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 510/1029] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:19.620927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535deff7-b167-4e06-91a7-eb04bd4be587 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:19.621410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535deff7-b167-4e06-91a7-eb04bd4be587 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:19.621699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535deff7-b167-4e06-91a7-eb04bd4be587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:19.686075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:19.711689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-535deff7-b167-4e06-91a7-eb04bd4be587 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:19.712978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-535deff7-b167-4e06-91a7-eb04bd4be587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:19.837853 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 520/1030] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:38:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:21.416140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6f02b-41e0-498a-b521-3a6b08b49d76 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:21.416648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6f02b-41e0-498a-b521-3a6b08b49d76 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:21.416648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6f02b-41e0-498a-b521-3a6b08b49d76 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:21.418179 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 511/1031] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:21.422978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87e1c5-2b05-43eb-b182-38bed6aa6d51 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:21.423372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87e1c5-2b05-43eb-b182-38bed6aa6d51 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:21.423671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87e1c5-2b05-43eb-b182-38bed6aa6d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:21.518355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6c87e1c5-2b05-43eb-b182-38bed6aa6d51 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:21.519598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6c87e1c5-2b05-43eb-b182-38bed6aa6d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:21.563837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:21.630735 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 521/1032] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:38:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:25.514968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b38d59-91fb-4be3-a546-d763dbcb2eb5 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:25.515491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b38d59-91fb-4be3-a546-d763dbcb2eb5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:25.515491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b38d59-91fb-4be3-a546-d763dbcb2eb5 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:25.516966 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 512/1033] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:25.522599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aedaa8-6621-4abf-b0e5-c77c8d278505 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:25.523015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aedaa8-6621-4abf-b0e5-c77c8d278505 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:25.523128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aedaa8-6621-4abf-b0e5-c77c8d278505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:25.592922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-50aedaa8-6621-4abf-b0e5-c77c8d278505 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:25.594116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-50aedaa8-6621-4abf-b0e5-c77c8d278505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:25.696631 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 522/1034] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:26.054289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd42f9-388f-4912-b37c-0be6e94b7c97 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:26.055096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd42f9-388f-4912-b37c-0be6e94b7c97 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:26.055574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd42f9-388f-4912-b37c-0be6e94b7c97 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:26.057437 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 513/1035] 10.222.0.16 () {54 vars in 961 bytes} [Sat Oct 15 20:38:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:26.066082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b9bdf-2c40-451f-9f27-8371044ded08 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:26.066900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b9bdf-2c40-451f-9f27-8371044ded08 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:26.067372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b9bdf-2c40-451f-9f27-8371044ded08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:26.090679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:26.161381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3b0b9bdf-2c40-451f-9f27-8371044ded08 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:26.162983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3b0b9bdf-2c40-451f-9f27-8371044ded08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:26.300593 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 523/1036] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Oct 15 20:38:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:26.308602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd42f9-388f-4912-b37c-0be6e94b7c97 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:26.310295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-07cd42f9-388f-4912-b37c-0be6e94b7c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:26.465396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd42f9-388f-4912-b37c-0be6e94b7c97 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:26.544813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:26.587346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874228c-c330-4976-b346-c98941f835fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:26.588005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874228c-c330-4976-b346-c98941f835fb service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:26.588378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874228c-c330-4976-b346-c98941f835fb service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:26.588771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874228c-c330-4976-b346-c98941f835fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:26.590458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0874228c-c330-4976-b346-c98941f835fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:26.685585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874228c-c330-4976-b346-c98941f835fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:26.687303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874228c-c330-4976-b346-c98941f835fb service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:26.705827 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 514/1037] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:29.381995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79b27a-ab34-4e53-a82f-b6545af6e962 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:29.382514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79b27a-ab34-4e53-a82f-b6545af6e962 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:29.382514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79b27a-ab34-4e53-a82f-b6545af6e962 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:29.383851 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 524/1038] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:29.389654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639997-e4d7-457d-82ec-8a821bae6473 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:29.390193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639997-e4d7-457d-82ec-8a821bae6473 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:29.390193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639997-e4d7-457d-82ec-8a821bae6473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:29.477222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9d639997-e4d7-457d-82ec-8a821bae6473 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:29.478608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9d639997-e4d7-457d-82ec-8a821bae6473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:29.519993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:29.582864 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 515/1039] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:29.931240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79b27a-ab34-4e53-a82f-b6545af6e962 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:29.932758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1b79b27a-ab34-4e53-a82f-b6545af6e962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:30.002831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:30.098484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79b27a-ab34-4e53-a82f-b6545af6e962 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:30.220170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae2fb-20ba-43d3-a8d8-d4c91c387e57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:30.220773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae2fb-20ba-43d3-a8d8-d4c91c387e57 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:30.221070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae2fb-20ba-43d3-a8d8-d4c91c387e57 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:30.221331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae2fb-20ba-43d3-a8d8-d4c91c387e57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:30.222898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-770ae2fb-20ba-43d3-a8d8-d4c91c387e57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:30.249403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:30.322758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ae2fb-20ba-43d3-a8d8-d4c91c387e57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:30.324355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ae2fb-20ba-43d3-a8d8-d4c91c387e57 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:30.343660 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 525/1040] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:33.119391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e29ac-26f3-4079-ab30-466d300eb094 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:33.119902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e29ac-26f3-4079-ab30-466d300eb094 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:33.120012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e29ac-26f3-4079-ab30-466d300eb094 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:33.121306 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 516/1041] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:33.126754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585a896-b485-44d7-b441-ae8f9c68f39b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:33.127147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585a896-b485-44d7-b441-ae8f9c68f39b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:33.127327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585a896-b485-44d7-b441-ae8f9c68f39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:33.204156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6585a896-b485-44d7-b441-ae8f9c68f39b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:33.205412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6585a896-b485-44d7-b441-ae8f9c68f39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:33.350303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:33.361071 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 526/1042] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:33.774690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9293e6f-4df1-48bd-9178-873d3b22d5a4 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:33.774972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9293e6f-4df1-48bd-9178-873d3b22d5a4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:33.775411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9293e6f-4df1-48bd-9178-873d3b22d5a4 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:33.777223 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 517/1043] 10.222.0.16 () {54 vars in 961 bytes} [Sat Oct 15 20:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:33.784494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff7f70-6e93-4750-9339-18ada24bacd0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:33.784902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff7f70-6e93-4750-9339-18ada24bacd0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:33.790536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff7f70-6e93-4750-9339-18ada24bacd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:33.892308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0bff7f70-6e93-4750-9339-18ada24bacd0 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:33.893550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0bff7f70-6e93-4750-9339-18ada24bacd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:34.063821 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 527/1044] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Oct 15 20:38:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:34.076111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9293e6f-4df1-48bd-9178-873d3b22d5a4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:34.078471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c9293e6f-4df1-48bd-9178-873d3b22d5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:34.312342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9293e6f-4df1-48bd-9178-873d3b22d5a4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:34.344964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:34.560912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a77959-db51-457d-84b1-fe9de1b9e11a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:34.565386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a77959-db51-457d-84b1-fe9de1b9e11a service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:34.565386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a77959-db51-457d-84b1-fe9de1b9e11a service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:34.565386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a77959-db51-457d-84b1-fe9de1b9e11a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:34.565386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-96a77959-db51-457d-84b1-fe9de1b9e11a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:34.666982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a77959-db51-457d-84b1-fe9de1b9e11a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:34.669155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a77959-db51-457d-84b1-fe9de1b9e11a service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:34.689832 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 518/1045] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:37.493473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d54afb-37bd-4338-b2c9-5b152a52714b None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:37.494403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d54afb-37bd-4338-b2c9-5b152a52714b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:37.494848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d54afb-37bd-4338-b2c9-5b152a52714b None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:37.496611 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 528/1046] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:37.502997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e76b4b-bb86-4f39-a443-0e17ad9e58e5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:37.503970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e76b4b-bb86-4f39-a443-0e17ad9e58e5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:37.504452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e76b4b-bb86-4f39-a443-0e17ad9e58e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:37.541214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:37.597601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-67e76b4b-bb86-4f39-a443-0e17ad9e58e5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:37.597601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-67e76b4b-bb86-4f39-a443-0e17ad9e58e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:37.775910 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 519/1047] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:38.201766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d54afb-37bd-4338-b2c9-5b152a52714b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:38.201766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-65d54afb-37bd-4338-b2c9-5b152a52714b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:38.275318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:38.401403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d54afb-37bd-4338-b2c9-5b152a52714b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:38.417257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e50967-7137-4c65-b446-288d946cddbf None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:38.417587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e50967-7137-4c65-b446-288d946cddbf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:38.417811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e50967-7137-4c65-b446-288d946cddbf None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:38.419293 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 520/1048] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:38.424470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a698c-c010-419b-9b05-f342384bbfda None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:38.424750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a698c-c010-419b-9b05-f342384bbfda None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:38.425075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a698c-c010-419b-9b05-f342384bbfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:38.501299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f46a698c-c010-419b-9b05-f342384bbfda None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:38.502514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f46a698c-c010-419b-9b05-f342384bbfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:38.545198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6809ba-c24c-4fcd-8779-9b85aaa521af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:38.545788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6809ba-c24c-4fcd-8779-9b85aaa521af service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:38.545954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6809ba-c24c-4fcd-8779-9b85aaa521af service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:38.546214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6809ba-c24c-4fcd-8779-9b85aaa521af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:38.547801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6d6809ba-c24c-4fcd-8779-9b85aaa521af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:38.558928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:38.563141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:38.615738 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 521/1049] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:38.646888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6809ba-c24c-4fcd-8779-9b85aaa521af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:38.648481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6809ba-c24c-4fcd-8779-9b85aaa521af service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:38.667229 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 529/1050] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:39.142535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dedee-4516-44e0-939e-b9ec9d21bf54 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:39.143676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dedee-4516-44e0-939e-b9ec9d21bf54 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:39.143676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dedee-4516-44e0-939e-b9ec9d21bf54 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:39.144548 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 522/1051] 10.222.0.16 () {54 vars in 963 bytes} [Sat Oct 15 20:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:39.150191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fbd4b-cffa-4a6c-a362-ffaa4be740ab None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:39.150513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fbd4b-cffa-4a6c-a362-ffaa4be740ab None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:39.150701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fbd4b-cffa-4a6c-a362-ffaa4be740ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:39.248015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fd0fbd4b-cffa-4a6c-a362-ffaa4be740ab None None] MFA Rules not processed for user `aa85b647a4be4d81b966bb50691bd3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:39.249252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd0fbd4b-cffa-4a6c-a362-ffaa4be740ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:39.335516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:39.337425 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 530/1052] 10.222.0.16 () {58 vars in 1060 bytes} [Sat Oct 15 20:38:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:39.346002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dedee-4516-44e0-939e-b9ec9d21bf54 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:39.347542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c48dedee-4516-44e0-939e-b9ec9d21bf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:39.471051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dedee-4516-44e0-939e-b9ec9d21bf54 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:39.554396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e53d81-a14a-4831-8e34-c0ca31d4f40e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:39.554989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e53d81-a14a-4831-8e34-c0ca31d4f40e service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:39.555255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e53d81-a14a-4831-8e34-c0ca31d4f40e service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:39.555514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e53d81-a14a-4831-8e34-c0ca31d4f40e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:39.557100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-41e53d81-a14a-4831-8e34-c0ca31d4f40e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:39.565969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:39.676718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e53d81-a14a-4831-8e34-c0ca31d4f40e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:39.679044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e53d81-a14a-4831-8e34-c0ca31d4f40e service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:39.713324 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 523/1053] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:41.383417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b958b9a-dfc1-4ff6-9c92-36a94e0516aa None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:41.384005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b958b9a-dfc1-4ff6-9c92-36a94e0516aa None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:41.384005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b958b9a-dfc1-4ff6-9c92-36a94e0516aa None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:41.389143 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 531/1054] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:41.390860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126d2e9-f624-445e-8cd9-c5b7cfb445d2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:41.391326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126d2e9-f624-445e-8cd9-c5b7cfb445d2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:41.391437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126d2e9-f624-445e-8cd9-c5b7cfb445d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:41.467763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7126d2e9-f624-445e-8cd9-c5b7cfb445d2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:41.469007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7126d2e9-f624-445e-8cd9-c5b7cfb445d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:41.538014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:41.581631 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 524/1055] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:41.931701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b958b9a-dfc1-4ff6-9c92-36a94e0516aa None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:41.932879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0b958b9a-dfc1-4ff6-9c92-36a94e0516aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:42.087679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b958b9a-dfc1-4ff6-9c92-36a94e0516aa None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:42.166884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:42.214922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bda15-5c89-421b-88bf-845338d0419d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:42.215529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bda15-5c89-421b-88bf-845338d0419d service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:42.215806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bda15-5c89-421b-88bf-845338d0419d service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:42.216074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bda15-5c89-421b-88bf-845338d0419d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:42.217714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6c1bda15-5c89-421b-88bf-845338d0419d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:42.317919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bda15-5c89-421b-88bf-845338d0419d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:42.319518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bda15-5c89-421b-88bf-845338d0419d service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:42.338563 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 532/1056] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:44.811407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74922f2d-c959-4dec-a33c-50eeaef70c14 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:44.812007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74922f2d-c959-4dec-a33c-50eeaef70c14 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:44.812007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74922f2d-c959-4dec-a33c-50eeaef70c14 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:44.813570 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 525/1057] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:44.819533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc856b-35b5-40a1-a8b1-d9251c032657 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:44.820063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc856b-35b5-40a1-a8b1-d9251c032657 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:44.820063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc856b-35b5-40a1-a8b1-d9251c032657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:44.890813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a2cc856b-35b5-40a1-a8b1-d9251c032657 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:44.892227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a2cc856b-35b5-40a1-a8b1-d9251c032657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:44.942798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:45.007910 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 533/1058] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:45.363756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74922f2d-c959-4dec-a33c-50eeaef70c14 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:45.365424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-74922f2d-c959-4dec-a33c-50eeaef70c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:45.547831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74922f2d-c959-4dec-a33c-50eeaef70c14 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:45.603103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:45.690436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7df0ee-2745-4e3a-b60f-5957041732a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:45.691028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df0ee-2745-4e3a-b60f-5957041732a0 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:45.691298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df0ee-2745-4e3a-b60f-5957041732a0 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:45.691563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df0ee-2745-4e3a-b60f-5957041732a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:45.694063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6b7df0ee-2745-4e3a-b60f-5957041732a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:45.802601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7df0ee-2745-4e3a-b60f-5957041732a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:45.804339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7df0ee-2745-4e3a-b60f-5957041732a0 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:45.823337 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 526/1059] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:48.338508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05f179-a57b-4575-ad3d-22bd6d54d9cd None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:48.339252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05f179-a57b-4575-ad3d-22bd6d54d9cd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:48.339252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05f179-a57b-4575-ad3d-22bd6d54d9cd None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:48.340503 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 534/1060] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:48.347017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e0715-b646-4961-a680-872d193ca6d0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:48.347481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e0715-b646-4961-a680-872d193ca6d0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:48.347577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e0715-b646-4961-a680-872d193ca6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:48.416003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2b9e0715-b646-4961-a680-872d193ca6d0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:48.417247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2b9e0715-b646-4961-a680-872d193ca6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:48.419253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:48.521493 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 527/1061] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:48.994679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05f179-a57b-4575-ad3d-22bd6d54d9cd None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:48.996186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eb05f179-a57b-4575-ad3d-22bd6d54d9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:49.148641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05f179-a57b-4575-ad3d-22bd6d54d9cd None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:49.175725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:49.295323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa94635-2adc-4ae7-a571-74eca40211a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:49.295846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa94635-2adc-4ae7-a571-74eca40211a0 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:49.296115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa94635-2adc-4ae7-a571-74eca40211a0 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:49.296377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa94635-2adc-4ae7-a571-74eca40211a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:49.298060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2fa94635-2adc-4ae7-a571-74eca40211a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:49.391456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa94635-2adc-4ae7-a571-74eca40211a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:49.399874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa94635-2adc-4ae7-a571-74eca40211a0 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:49.418781 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 535/1062] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:38:51.969493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54064e67-2b65-4edf-8890-63e889bbba5d None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:51.970093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54064e67-2b65-4edf-8890-63e889bbba5d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:51.970093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54064e67-2b65-4edf-8890-63e889bbba5d None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:51.971682 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 528/1063] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:51.977854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955de4d-1cbe-44fc-adff-34c67a1338cc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:51.978287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955de4d-1cbe-44fc-adff-34c67a1338cc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:51.978425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955de4d-1cbe-44fc-adff-34c67a1338cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:52.011448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:52.048160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1955de4d-1cbe-44fc-adff-34c67a1338cc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:52.049412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1955de4d-1cbe-44fc-adff-34c67a1338cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:52.153659 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 536/1064] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:38:52.494952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54064e67-2b65-4edf-8890-63e889bbba5d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:52.496795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-54064e67-2b65-4edf-8890-63e889bbba5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:52.647945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:52.670178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54064e67-2b65-4edf-8890-63e889bbba5d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:52.788832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a8e7b-60d2-455f-8d16-32539b641bef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:52.789598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8e7b-60d2-455f-8d16-32539b641bef service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:52.789873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8e7b-60d2-455f-8d16-32539b641bef service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:52.790248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8e7b-60d2-455f-8d16-32539b641bef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:52.791854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-914a8e7b-60d2-455f-8d16-32539b641bef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:52.888883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a8e7b-60d2-455f-8d16-32539b641bef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:52.891204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a8e7b-60d2-455f-8d16-32539b641bef service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:52.899296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:52.917669 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 529/1065] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:55.530268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c87be-92d5-4e51-9597-790ff337bda2 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:55.530906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c87be-92d5-4e51-9597-790ff337bda2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:55.530906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c87be-92d5-4e51-9597-790ff337bda2 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:55.532297 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 537/1066] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:55.538040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a61b4f-7c4b-414a-b659-2a7522a5561e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:55.538528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a61b4f-7c4b-414a-b659-2a7522a5561e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:55.538633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a61b4f-7c4b-414a-b659-2a7522a5561e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:55.609302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-60a61b4f-7c4b-414a-b659-2a7522a5561e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:55.618199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-60a61b4f-7c4b-414a-b659-2a7522a5561e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:55.721462 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 530/1067] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:56.067267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c87be-92d5-4e51-9597-790ff337bda2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:56.068780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4d8c87be-92d5-4e51-9597-790ff337bda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:56.162236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafecfd-bec3-4bc3-a022-580ba995a316 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:56.162507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafecfd-bec3-4bc3-a022-580ba995a316 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:56.162771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafecfd-bec3-4bc3-a022-580ba995a316 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:56.164144 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 531/1068] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:38:56.169057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:56.169214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25912c9-a54d-47d8-b690-1678665ca583 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:56.169451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25912c9-a54d-47d8-b690-1678665ca583 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:56.169750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25912c9-a54d-47d8-b690-1678665ca583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:56.221878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c87be-92d5-4e51-9597-790ff337bda2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:56.238747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e25912c9-a54d-47d8-b690-1678665ca583 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:56.239941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e25912c9-a54d-47d8-b690-1678665ca583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:56.260494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:56.339042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725e53d-8d62-496a-b12a-6910ee60f7d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:56.339679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725e53d-8d62-496a-b12a-6910ee60f7d8 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:56.339830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725e53d-8d62-496a-b12a-6910ee60f7d8 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:56.340070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725e53d-8d62-496a-b12a-6910ee60f7d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:56.341682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a725e53d-8d62-496a-b12a-6910ee60f7d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:56.342424 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 532/1069] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:38:56.411697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:56.433792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725e53d-8d62-496a-b12a-6910ee60f7d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:56.435369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725e53d-8d62-496a-b12a-6910ee60f7d8 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:56.453716 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 538/1070] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:56.715951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014bac0-c2c5-4601-9be4-1c9668f10025 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:56.716541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014bac0-c2c5-4601-9be4-1c9668f10025 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:56.716541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014bac0-c2c5-4601-9be4-1c9668f10025 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:56.717976 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 533/1071] 10.222.0.16 () {54 vars in 962 bytes} [Sat Oct 15 20:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:56.723604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2257e-8979-4db8-8c12-f6c9c9526817 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:56.724073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2257e-8979-4db8-8c12-f6c9c9526817 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:56.724186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2257e-8979-4db8-8c12-f6c9c9526817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:56.800815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6dd2257e-8979-4db8-8c12-f6c9c9526817 None None] MFA Rules not processed for user `af8591ebfa174c91853b673360326a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:56.803078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6dd2257e-8979-4db8-8c12-f6c9c9526817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:56.879979 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 539/1072] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Oct 15 20:38:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:56.887773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014bac0-c2c5-4601-9be4-1c9668f10025 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:56.889302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1014bac0-c2c5-4601-9be4-1c9668f10025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:56.998593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014bac0-c2c5-4601-9be4-1c9668f10025 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:57.053096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:57.077746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e21c1-5e25-46be-9ddd-0d220ae0abd2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:57.078456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e21c1-5e25-46be-9ddd-0d220ae0abd2 service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:57.078735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e21c1-5e25-46be-9ddd-0d220ae0abd2 service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:57.079001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e21c1-5e25-46be-9ddd-0d220ae0abd2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:57.080573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2f5e21c1-5e25-46be-9ddd-0d220ae0abd2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:57.193147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e21c1-5e25-46be-9ddd-0d220ae0abd2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:57.194685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e21c1-5e25-46be-9ddd-0d220ae0abd2 service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:57.214518 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 534/1073] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:38:57.806528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e24e-4a4a-4681-96eb-d950d4adfc09 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:57.807037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e24e-4a4a-4681-96eb-d950d4adfc09 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:57.807037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e24e-4a4a-4681-96eb-d950d4adfc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:57.876557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5c10e24e-4a4a-4681-96eb-d950d4adfc09 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:57.877801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5c10e24e-4a4a-4681-96eb-d950d4adfc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:57.885534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:57.971495 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 540/1074] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:38:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:58.019628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c442a7-f11e-4be9-ad76-542dea2223b1 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:58.020127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c442a7-f11e-4be9-ad76-542dea2223b1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:58.020404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c442a7-f11e-4be9-ad76-542dea2223b1 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:58.021856 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 535/1075] 10.222.0.16 () {54 vars in 943 bytes} [Sat Oct 15 20:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:58.028303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0e3af-631c-4f34-9f5a-840b8ba5762a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:58.028578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0e3af-631c-4f34-9f5a-840b8ba5762a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:58.028841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0e3af-631c-4f34-9f5a-840b8ba5762a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:58.105014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-44d0e3af-631c-4f34-9f5a-840b8ba5762a None None] MFA Rules not processed for user `abfea6b853134fed8ea5e32e5406bd5f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:58.106245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-44d0e3af-631c-4f34-9f5a-840b8ba5762a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:58.189939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:58.226934 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 541/1076] 10.222.0.16 () {58 vars in 1040 bytes} [Sat Oct 15 20:38:58 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:38:58.235937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c442a7-f11e-4be9-ad76-542dea2223b1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:58.237511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-51c442a7-f11e-4be9-ad76-542dea2223b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:58.323141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:58.382682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c442a7-f11e-4be9-ad76-542dea2223b1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:58.499755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838f186-007b-4a0a-a065-7b384c2f629a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:58.499755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838f186-007b-4a0a-a065-7b384c2f629a service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:58.499755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838f186-007b-4a0a-a065-7b384c2f629a service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:58.499755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838f186-007b-4a0a-a065-7b384c2f629a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:58.500318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f838f186-007b-4a0a-a065-7b384c2f629a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:58.600524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:58.657831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838f186-007b-4a0a-a065-7b384c2f629a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:58.659469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838f186-007b-4a0a-a065-7b384c2f629a service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:38:58.668981 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 536/1077] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:38:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:38:58.995257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f226ae-3557-45cb-958c-6f7d8b6dd82b None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:58.995860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f226ae-3557-45cb-958c-6f7d8b6dd82b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:58.996033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f226ae-3557-45cb-958c-6f7d8b6dd82b None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:58.997608 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 542/1078] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:38:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:38:59.002821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6dd1a-aa8f-4b8e-a35c-c37c7701208d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:59.003213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6dd1a-aa8f-4b8e-a35c-c37c7701208d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:59.003628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6dd1a-aa8f-4b8e-a35c-c37c7701208d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:59.081571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8ee6dd1a-aa8f-4b8e-a35c-c37c7701208d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:38:59.083151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8ee6dd1a-aa8f-4b8e-a35c-c37c7701208d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:59.192217 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 537/1079] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:38:59.544655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f226ae-3557-45cb-958c-6f7d8b6dd82b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:38:59.545671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a0f226ae-3557-45cb-958c-6f7d8b6dd82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:59.704471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f226ae-3557-45cb-958c-6f7d8b6dd82b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:38:59.765439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:59.829024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05fefe-f0e1-4035-a1ee-9286ed4ca498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:38:59.829548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05fefe-f0e1-4035-a1ee-9286ed4ca498 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:38:59.829809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05fefe-f0e1-4035-a1ee-9286ed4ca498 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:38:59.830074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05fefe-f0e1-4035-a1ee-9286ed4ca498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:38:59.831633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cc05fefe-f0e1-4035-a1ee-9286ed4ca498 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:38:59.989624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05fefe-f0e1-4035-a1ee-9286ed4ca498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:38:59.991421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05fefe-f0e1-4035-a1ee-9286ed4ca498 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:00.013430 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 543/1080] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:38:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:02.515811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838285a-c3e2-4fe9-b18b-0f2e7f22e901 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:02.516404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838285a-c3e2-4fe9-b18b-0f2e7f22e901 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:02.516404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838285a-c3e2-4fe9-b18b-0f2e7f22e901 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:02.517652 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 538/1081] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:02.524430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10359ff-8551-4034-9a74-3e0b1d1462af None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:02.524430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10359ff-8551-4034-9a74-3e0b1d1462af None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:02.524430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10359ff-8551-4034-9a74-3e0b1d1462af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:02.595274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c10359ff-8551-4034-9a74-3e0b1d1462af None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:02.596592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c10359ff-8551-4034-9a74-3e0b1d1462af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:02.615172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:02.705059 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 544/1082] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:03.076202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838285a-c3e2-4fe9-b18b-0f2e7f22e901 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:03.078113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8838285a-c3e2-4fe9-b18b-0f2e7f22e901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:03.150321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:03.267288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838285a-c3e2-4fe9-b18b-0f2e7f22e901 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:03.396755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0eb253-381a-41af-b4dd-8eb8f9658e1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:03.397388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eb253-381a-41af-b4dd-8eb8f9658e1c service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:03.397659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eb253-381a-41af-b4dd-8eb8f9658e1c service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:03.397922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eb253-381a-41af-b4dd-8eb8f9658e1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:03.399509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5c0eb253-381a-41af-b4dd-8eb8f9658e1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:03.418972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:03.505662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0eb253-381a-41af-b4dd-8eb8f9658e1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:03.507549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0eb253-381a-41af-b4dd-8eb8f9658e1c service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:03.528041 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 539/1083] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:05.996391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7544-95df-43a4-a80a-a937ec22299a None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:05.996996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7544-95df-43a4-a80a-a937ec22299a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:05.996996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7544-95df-43a4-a80a-a937ec22299a None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:05.998260 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 545/1084] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:06.005907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af638f-f0cb-4b21-8b4e-5813e8327f2e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:06.006471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af638f-f0cb-4b21-8b4e-5813e8327f2e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:06.006471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af638f-f0cb-4b21-8b4e-5813e8327f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:06.077017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-43af638f-f0cb-4b21-8b4e-5813e8327f2e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:06.078248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-43af638f-f0cb-4b21-8b4e-5813e8327f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:06.183363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:06.194046 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 540/1085] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:06.546526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e7544-95df-43a4-a80a-a937ec22299a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:06.548152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-448e7544-95df-43a4-a80a-a937ec22299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:06.712825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:06.714212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e7544-95df-43a4-a80a-a937ec22299a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:06.856766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc078af-2667-471d-972d-3a13da95e9e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:06.857303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc078af-2667-471d-972d-3a13da95e9e8 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:06.857569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc078af-2667-471d-972d-3a13da95e9e8 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:06.857831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc078af-2667-471d-972d-3a13da95e9e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:06.859417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9cc078af-2667-471d-972d-3a13da95e9e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:06.961161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc078af-2667-471d-972d-3a13da95e9e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:06.962772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc078af-2667-471d-972d-3a13da95e9e8 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:06.982829 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 546/1086] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:16.188572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:16.978847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:22.787243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e9eff-4f9f-4a63-8744-f69f713ad8e1 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:22.787802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e9eff-4f9f-4a63-8744-f69f713ad8e1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:22.787802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e9eff-4f9f-4a63-8744-f69f713ad8e1 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:22.789237 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 541/1087] 10.222.0.16 () {54 vars in 918 bytes} [Sat Oct 15 20:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:22.795606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c233e-0d8b-4722-97ec-fe2cb69e90ab None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:22.795993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c233e-0d8b-4722-97ec-fe2cb69e90ab None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:22.796152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c233e-0d8b-4722-97ec-fe2cb69e90ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:22.872348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a26c233e-0d8b-4722-97ec-fe2cb69e90ab None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:22.873560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a26c233e-0d8b-4722-97ec-fe2cb69e90ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:22.977413 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 547/1088] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Oct 15 20:39:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:22.992557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa927f-8f13-4cde-9438-40262b3947b2 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:22.992831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa927f-8f13-4cde-9438-40262b3947b2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:22.993120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa927f-8f13-4cde-9438-40262b3947b2 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:22.994433 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 542/1089] 10.222.0.16 () {54 vars in 918 bytes} [Sat Oct 15 20:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:22.999196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3773e-6386-4108-939c-287daa6fb909 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:22.999481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3773e-6386-4108-939c-287daa6fb909 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:22.999753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3773e-6386-4108-939c-287daa6fb909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:23.073447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:23.076247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bff3773e-6386-4108-939c-287daa6fb909 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:23.078110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bff3773e-6386-4108-939c-287daa6fb909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:23.195071 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 548/1090] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Oct 15 20:39:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:26.030236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029edec3-cd5b-406d-b8a4-824844cd57d7 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:26.030794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029edec3-cd5b-406d-b8a4-824844cd57d7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:26.030794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029edec3-cd5b-406d-b8a4-824844cd57d7 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:26.032083 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 543/1091] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:26.037416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d9505-f8f2-46e6-8282-d81f13c6b000 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:26.037915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d9505-f8f2-46e6-8282-d81f13c6b000 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:26.038026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d9505-f8f2-46e6-8282-d81f13c6b000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:26.103756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4b5d9505-f8f2-46e6-8282-d81f13c6b000 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:26.104932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4b5d9505-f8f2-46e6-8282-d81f13c6b000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:26.159376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:26.203123 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 549/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:39:26.552444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029edec3-cd5b-406d-b8a4-824844cd57d7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:26.554032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-029edec3-cd5b-406d-b8a4-824844cd57d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:26.707922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029edec3-cd5b-406d-b8a4-824844cd57d7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:26.783002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:26.830058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91e6df-eaac-4d35-bc95-fb4f1a4b0189 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:26.830058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91e6df-eaac-4d35-bc95-fb4f1a4b0189 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:26.830431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91e6df-eaac-4d35-bc95-fb4f1a4b0189 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:26.830533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91e6df-eaac-4d35-bc95-fb4f1a4b0189 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:26.832428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ef91e6df-eaac-4d35-bc95-fb4f1a4b0189 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:26.941554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91e6df-eaac-4d35-bc95-fb4f1a4b0189 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:26.943160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91e6df-eaac-4d35-bc95-fb4f1a4b0189 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:26.963358 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 544/1093] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:28.722582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03ffd6-e934-4e41-98d3-941e723e5914 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:28.723125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03ffd6-e934-4e41-98d3-941e723e5914 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:28.723125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03ffd6-e934-4e41-98d3-941e723e5914 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:28.724440 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 550/1094] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 20:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:28.729290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa9c52-1261-4222-8036-6afa20a1ab15 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:28.729726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa9c52-1261-4222-8036-6afa20a1ab15 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:28.729836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa9c52-1261-4222-8036-6afa20a1ab15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:28.806176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b8fa9c52-1261-4222-8036-6afa20a1ab15 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:28.807358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b8fa9c52-1261-4222-8036-6afa20a1ab15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:28.834858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:28.905214 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 545/1095] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 20:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:29.139232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03ffd6-e934-4e41-98d3-941e723e5914 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:29.140790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd03ffd6-e934-4e41-98d3-941e723e5914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:29.287473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03ffd6-e934-4e41-98d3-941e723e5914 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:29.327057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:29.399113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eb26b-12a2-45c0-9848-6852efc3d8a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:29.399770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb26b-12a2-45c0-9848-6852efc3d8a4 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:29.400050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb26b-12a2-45c0-9848-6852efc3d8a4 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:29.400344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb26b-12a2-45c0-9848-6852efc3d8a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:29.401969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-861eb26b-12a2-45c0-9848-6852efc3d8a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:29.492128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861eb26b-12a2-45c0-9848-6852efc3d8a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:29.493742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861eb26b-12a2-45c0-9848-6852efc3d8a4 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:29.511958 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 551/1096] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:31.248925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770710a-4a5e-4903-b775-c31148fbfe64 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:31.249523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770710a-4a5e-4903-b775-c31148fbfe64 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:31.249523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770710a-4a5e-4903-b775-c31148fbfe64 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:31.250854 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 546/1097] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 20:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:31.256523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1b34b-52ea-4285-8644-eaea3323273e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:31.256936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1b34b-52ea-4285-8644-eaea3323273e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:31.257069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1b34b-52ea-4285-8644-eaea3323273e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:31.310388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:31.332909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-add1b34b-52ea-4285-8644-eaea3323273e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:31.334135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-add1b34b-52ea-4285-8644-eaea3323273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:31.429905 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 552/1098] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 20:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:31.590158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770710a-4a5e-4903-b775-c31148fbfe64 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:31.591687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6770710a-4a5e-4903-b775-c31148fbfe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:31.743546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770710a-4a5e-4903-b775-c31148fbfe64 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:31.756022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:31.855015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ae449-6691-4c92-a0a2-3073147c2403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:31.855597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ae449-6691-4c92-a0a2-3073147c2403 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:31.855871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ae449-6691-4c92-a0a2-3073147c2403 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:31.856133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ae449-6691-4c92-a0a2-3073147c2403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:31.857712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4a1ae449-6691-4c92-a0a2-3073147c2403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:31.946327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ae449-6691-4c92-a0a2-3073147c2403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:31.947901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ae449-6691-4c92-a0a2-3073147c2403 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:31.965769 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 547/1099] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:33.628415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b931faa-58e3-4eb7-813c-b1465a36afb1 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:33.628977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b931faa-58e3-4eb7-813c-b1465a36afb1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:33.628977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b931faa-58e3-4eb7-813c-b1465a36afb1 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:33.630314 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 553/1100] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 20:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:33.635289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74652249-f00a-4966-9ff7-96aa71a11d83 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:33.635715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74652249-f00a-4966-9ff7-96aa71a11d83 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:33.635831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74652249-f00a-4966-9ff7-96aa71a11d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:33.655846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:33.712291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-74652249-f00a-4966-9ff7-96aa71a11d83 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:33.713508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-74652249-f00a-4966-9ff7-96aa71a11d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:33.808652 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 548/1101] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 20:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:39:33.968358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b931faa-58e3-4eb7-813c-b1465a36afb1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:33.969873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8b931faa-58e3-4eb7-813c-b1465a36afb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:34.079132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:34.115245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b931faa-58e3-4eb7-813c-b1465a36afb1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:34.228575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b64da-900f-49c2-9a8e-0f77157499f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:34.229128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b64da-900f-49c2-9a8e-0f77157499f5 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:34.229399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b64da-900f-49c2-9a8e-0f77157499f5 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:34.229702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b64da-900f-49c2-9a8e-0f77157499f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:34.231243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c21b64da-900f-49c2-9a8e-0f77157499f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:34.307289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:34.321273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b64da-900f-49c2-9a8e-0f77157499f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:34.322855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b64da-900f-49c2-9a8e-0f77157499f5 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:34.341164 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 554/1102] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:36.020612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496cb3-af96-4dd4-a191-e922dc49aaf3 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:36.021333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496cb3-af96-4dd4-a191-e922dc49aaf3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:36.021333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496cb3-af96-4dd4-a191-e922dc49aaf3 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:36.022554 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 549/1103] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 20:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:36.027495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f6812-c376-49b3-8e2c-bec8244612f8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:36.027939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f6812-c376-49b3-8e2c-bec8244612f8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:36.028036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f6812-c376-49b3-8e2c-bec8244612f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:36.104559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7f6f6812-c376-49b3-8e2c-bec8244612f8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:36.105780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7f6f6812-c376-49b3-8e2c-bec8244612f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:36.208311 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 555/1104] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 20:39:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:36.368231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d496cb3-af96-4dd4-a191-e922dc49aaf3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:36.369771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3d496cb3-af96-4dd4-a191-e922dc49aaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:36.454860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:36.516403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d496cb3-af96-4dd4-a191-e922dc49aaf3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:36.628565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e929504-fdc5-4fee-bff7-b5add5f2635c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:36.629097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e929504-fdc5-4fee-bff7-b5add5f2635c service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:36.629474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e929504-fdc5-4fee-bff7-b5add5f2635c service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:36.629738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e929504-fdc5-4fee-bff7-b5add5f2635c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:36.631291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2e929504-fdc5-4fee-bff7-b5add5f2635c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:36.678487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:36.721927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e929504-fdc5-4fee-bff7-b5add5f2635c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:36.723490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e929504-fdc5-4fee-bff7-b5add5f2635c service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:36.741648 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 550/1105] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:38.361958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1baf4-7b6b-4b42-b319-49244fbf9007 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:38.362794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1baf4-7b6b-4b42-b319-49244fbf9007 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:38.362930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1baf4-7b6b-4b42-b319-49244fbf9007 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:38.364201 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 556/1106] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 20:39:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:38.369267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d347f10-ce9d-43c8-bf29-a9be16267d7a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:38.369664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d347f10-ce9d-43c8-bf29-a9be16267d7a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:38.369806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d347f10-ce9d-43c8-bf29-a9be16267d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:38.445249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1d347f10-ce9d-43c8-bf29-a9be16267d7a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:38.446484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1d347f10-ce9d-43c8-bf29-a9be16267d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:38.541508 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 551/1107] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 20:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:38.701485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1baf4-7b6b-4b42-b319-49244fbf9007 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:38.702983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fcf1baf4-7b6b-4b42-b319-49244fbf9007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:38.752554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:38.849674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1baf4-7b6b-4b42-b319-49244fbf9007 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:38.962768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d9443-d0eb-48b1-bfaf-9c6508e8f788 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:38.963285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d9443-d0eb-48b1-bfaf-9c6508e8f788 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:38.963555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d9443-d0eb-48b1-bfaf-9c6508e8f788 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:38.963820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d9443-d0eb-48b1-bfaf-9c6508e8f788 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:38.965432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e27d9443-d0eb-48b1-bfaf-9c6508e8f788 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:38.972892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:39.056177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d9443-d0eb-48b1-bfaf-9c6508e8f788 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:39.057785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d9443-d0eb-48b1-bfaf-9c6508e8f788 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:39.075726 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 557/1108] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:40.715717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9090059-1a64-4db9-866a-39acff5a7634 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:40.716290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9090059-1a64-4db9-866a-39acff5a7634 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:40.716290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9090059-1a64-4db9-866a-39acff5a7634 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:40.717616 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 552/1109] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 20:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:40.723322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fdc39-e828-4b30-abc0-bf5106f2577e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:40.723749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fdc39-e828-4b30-abc0-bf5106f2577e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:40.723857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fdc39-e828-4b30-abc0-bf5106f2577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:40.800842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0c9fdc39-e828-4b30-abc0-bf5106f2577e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:40.802054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0c9fdc39-e828-4b30-abc0-bf5106f2577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:40.872048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:40.901904 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 558/1110] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 20:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:41.061700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9090059-1a64-4db9-866a-39acff5a7634 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:41.063113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c9090059-1a64-4db9-866a-39acff5a7634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:41.081333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:41.209689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9090059-1a64-4db9-866a-39acff5a7634 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:41.298339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:41.324107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086c070-6db0-41e2-b686-6de524101180 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:41.324631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086c070-6db0-41e2-b686-6de524101180 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:41.324908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086c070-6db0-41e2-b686-6de524101180 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:41.325205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086c070-6db0-41e2-b686-6de524101180 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:41.326782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7086c070-6db0-41e2-b686-6de524101180 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:41.415336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086c070-6db0-41e2-b686-6de524101180 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:41.416913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086c070-6db0-41e2-b686-6de524101180 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:41.434913 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 553/1111] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:43.069655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ed8fe-3add-4033-8c73-9b76c37181ef None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:43.070223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ed8fe-3add-4033-8c73-9b76c37181ef None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:43.070223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ed8fe-3add-4033-8c73-9b76c37181ef None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:43.071552 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 559/1112] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 20:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:43.076538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbad5f-3c40-49c6-bcf8-80621db34008 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:43.076937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbad5f-3c40-49c6-bcf8-80621db34008 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:43.077106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbad5f-3c40-49c6-bcf8-80621db34008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:43.154132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-facbad5f-3c40-49c6-bcf8-80621db34008 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:43.155329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-facbad5f-3c40-49c6-bcf8-80621db34008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:43.200143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:43.255680 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 554/1113] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 20:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:43.416071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ed8fe-3add-4033-8c73-9b76c37181ef None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:43.417619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-318ed8fe-3add-4033-8c73-9b76c37181ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:43.562834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ed8fe-3add-4033-8c73-9b76c37181ef None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:43.618241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:43.678455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030ba57-8bbe-422f-872a-7e7c06e3628a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:43.678974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030ba57-8bbe-422f-872a-7e7c06e3628a service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:43.679243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030ba57-8bbe-422f-872a-7e7c06e3628a service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:43.679504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030ba57-8bbe-422f-872a-7e7c06e3628a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:43.681127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1030ba57-8bbe-422f-872a-7e7c06e3628a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:43.771074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030ba57-8bbe-422f-872a-7e7c06e3628a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:43.772651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030ba57-8bbe-422f-872a-7e7c06e3628a service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:43.791048 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 560/1114] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:45.416223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2616f5-4daa-4dd9-92af-d053a400fc15 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:45.417027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2616f5-4daa-4dd9-92af-d053a400fc15 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:45.417027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2616f5-4daa-4dd9-92af-d053a400fc15 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:45.418187 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 555/1115] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 20:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:45.424319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01284c3-ec3f-4d64-a304-d52bd8c52eb0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:45.424770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01284c3-ec3f-4d64-a304-d52bd8c52eb0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:45.424886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01284c3-ec3f-4d64-a304-d52bd8c52eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:45.503172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e01284c3-ec3f-4d64-a304-d52bd8c52eb0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:45.504351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e01284c3-ec3f-4d64-a304-d52bd8c52eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:45.506348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:45.600089 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 561/1116] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 20:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:45.761342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2616f5-4daa-4dd9-92af-d053a400fc15 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:45.762848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a2616f5-4daa-4dd9-92af-d053a400fc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:45.912350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2616f5-4daa-4dd9-92af-d053a400fc15 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:45.939962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:46.026079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0af376-82f5-4d24-a421-81ae749289fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:46.026597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0af376-82f5-4d24-a421-81ae749289fe service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:46.026869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0af376-82f5-4d24-a421-81ae749289fe service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:46.027133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0af376-82f5-4d24-a421-81ae749289fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:46.028720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2b0af376-82f5-4d24-a421-81ae749289fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:46.118667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0af376-82f5-4d24-a421-81ae749289fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:46.120241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0af376-82f5-4d24-a421-81ae749289fe service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:46.137623 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 556/1117] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:48.469465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb5b14-9d7b-4b7a-978c-4e5b4b642a4a None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:48.469977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb5b14-9d7b-4b7a-978c-4e5b4b642a4a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:48.469977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb5b14-9d7b-4b7a-978c-4e5b4b642a4a None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:48.471383 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 562/1118] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:39:48.476783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc99fcb-42fd-4b72-bc17-3a8ed0cca110 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:48.477235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc99fcb-42fd-4b72-bc17-3a8ed0cca110 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:48.477335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc99fcb-42fd-4b72-bc17-3a8ed0cca110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:48.518958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:48.544199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bdc99fcb-42fd-4b72-bc17-3a8ed0cca110 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:48.545437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bdc99fcb-42fd-4b72-bc17-3a8ed0cca110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:48.642155 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 557/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:48.993370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb5b14-9d7b-4b7a-978c-4e5b4b642a4a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:48.994924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e6bb5b14-9d7b-4b7a-978c-4e5b4b642a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:49.097590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb5b14-9d7b-4b7a-978c-4e5b4b642a4a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:49.130938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:49.173648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820ae58-cc1e-4643-ae36-2a214488f0e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:49.174233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820ae58-cc1e-4643-ae36-2a214488f0e1 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:49.174501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820ae58-cc1e-4643-ae36-2a214488f0e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:49.174761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820ae58-cc1e-4643-ae36-2a214488f0e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:49.176313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4820ae58-cc1e-4643-ae36-2a214488f0e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:49.288392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820ae58-cc1e-4643-ae36-2a214488f0e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:49.289802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820ae58-cc1e-4643-ae36-2a214488f0e1 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:49.307546 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 563/1120] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:51.940157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb93103-251c-4d06-84df-528453cf9965 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:51.940692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb93103-251c-4d06-84df-528453cf9965 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:51.940692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb93103-251c-4d06-84df-528453cf9965 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:51.942046 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 558/1121] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:51.947209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e946-d3e8-4f23-9688-19071eca8e87 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:51.947705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e946-d3e8-4f23-9688-19071eca8e87 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:51.947803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e946-d3e8-4f23-9688-19071eca8e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:52.014051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ebf5e946-d3e8-4f23-9688-19071eca8e87 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:52.015231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ebf5e946-d3e8-4f23-9688-19071eca8e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:52.027640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:52.110207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 564/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:52.449352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb93103-251c-4d06-84df-528453cf9965 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:52.450846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3fb93103-251c-4d06-84df-528453cf9965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:52.552760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb93103-251c-4d06-84df-528453cf9965 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:52.565771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:52.629664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486938b5-e7be-42a8-9392-a8ef96322eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:52.630277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486938b5-e7be-42a8-9392-a8ef96322eea service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:52.630544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486938b5-e7be-42a8-9392-a8ef96322eea service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:52.630804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486938b5-e7be-42a8-9392-a8ef96322eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:52.632374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-486938b5-e7be-42a8-9392-a8ef96322eea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:52.739764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486938b5-e7be-42a8-9392-a8ef96322eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:52.741128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486938b5-e7be-42a8-9392-a8ef96322eea service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:52.760791 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 559/1123] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:55.469430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6ac4a-bd22-4f6f-b2f2-d88313cd524c None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:55.470073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6ac4a-bd22-4f6f-b2f2-d88313cd524c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:55.470073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6ac4a-bd22-4f6f-b2f2-d88313cd524c None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:55.471308 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 565/1124] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:55.476630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52157e88-226d-44cf-a62a-dadf2811bf7f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:55.477146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52157e88-226d-44cf-a62a-dadf2811bf7f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:55.477263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52157e88-226d-44cf-a62a-dadf2811bf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:55.526371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:55.545063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-52157e88-226d-44cf-a62a-dadf2811bf7f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:55.546252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-52157e88-226d-44cf-a62a-dadf2811bf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:55.642162 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 560/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:55.982600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6ac4a-bd22-4f6f-b2f2-d88313cd524c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:55.984197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-36e6ac4a-bd22-4f6f-b2f2-d88313cd524c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:56.087365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6ac4a-bd22-4f6f-b2f2-d88313cd524c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:56.128236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:56.164304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4844df-f080-4e79-ac00-8675f3bc001c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:56.164899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4844df-f080-4e79-ac00-8675f3bc001c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:56.165204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4844df-f080-4e79-ac00-8675f3bc001c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:56.165471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4844df-f080-4e79-ac00-8675f3bc001c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:56.167038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fc4844df-f080-4e79-ac00-8675f3bc001c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:56.275349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4844df-f080-4e79-ac00-8675f3bc001c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:56.276690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4844df-f080-4e79-ac00-8675f3bc001c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:56.295408 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 566/1126] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:58.842151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e58169-48fe-4a45-9db4-a41a66047c30 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:58.842797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e58169-48fe-4a45-9db4-a41a66047c30 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:58.842797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e58169-48fe-4a45-9db4-a41a66047c30 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:58.844019 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 561/1127] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:39:58.849797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c2f96-7886-4727-b040-3b6d63e5a527 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:58.850244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c2f96-7886-4727-b040-3b6d63e5a527 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:58.850355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c2f96-7886-4727-b040-3b6d63e5a527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:58.918397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d31c2f96-7886-4727-b040-3b6d63e5a527 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:39:58.919622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d31c2f96-7886-4727-b040-3b6d63e5a527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:58.940323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:59.017770 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 567/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:39:59.365943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e58169-48fe-4a45-9db4-a41a66047c30 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:39:59.367488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a5e58169-48fe-4a45-9db4-a41a66047c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:59.470099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e58169-48fe-4a45-9db4-a41a66047c30 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:39:59.490513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:59.543542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2db3fe-1000-447e-a9dc-98f82755f7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:39:59.544140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2db3fe-1000-447e-a9dc-98f82755f7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:39:59.544412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2db3fe-1000-447e-a9dc-98f82755f7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:39:59.544678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2db3fe-1000-447e-a9dc-98f82755f7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:39:59.546281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a2db3fe-1000-447e-a9dc-98f82755f7b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:39:59.655360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2db3fe-1000-447e-a9dc-98f82755f7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:39:59.656786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2db3fe-1000-447e-a9dc-98f82755f7b4 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:39:59.675055 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 562/1129] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:40:02.168416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594846f-7f34-48bf-a492-90385f24f7d4 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:02.169014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594846f-7f34-48bf-a492-90385f24f7d4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:02.169014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594846f-7f34-48bf-a492-90385f24f7d4 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:02.170340 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 568/1130] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:02.176447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175b2a-4fd7-4f53-bff8-47a55f7b2ef8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:02.176878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175b2a-4fd7-4f53-bff8-47a55f7b2ef8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:02.176984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175b2a-4fd7-4f53-bff8-47a55f7b2ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:02.244000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:02.246674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5f175b2a-4fd7-4f53-bff8-47a55f7b2ef8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:02.248382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5f175b2a-4fd7-4f53-bff8-47a55f7b2ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:02.345205 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 563/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:40:02.686278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594846f-7f34-48bf-a492-90385f24f7d4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:02.687829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b594846f-7f34-48bf-a492-90385f24f7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:02.791366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594846f-7f34-48bf-a492-90385f24f7d4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:02.841171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:02.866456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8fa60-82d6-41a0-99f4-f114b1e4047a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:02.867049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8fa60-82d6-41a0-99f4-f114b1e4047a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:02.867318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8fa60-82d6-41a0-99f4-f114b1e4047a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:02.867579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8fa60-82d6-41a0-99f4-f114b1e4047a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:02.869156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a7b8fa60-82d6-41a0-99f4-f114b1e4047a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:02.978873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8fa60-82d6-41a0-99f4-f114b1e4047a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:02.980212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8fa60-82d6-41a0-99f4-f114b1e4047a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:02.998102 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 569/1132] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:40:05.499504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b82ca-d398-4c10-a502-b3ddbfe8ae59 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:05.500048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b82ca-d398-4c10-a502-b3ddbfe8ae59 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:05.500164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b82ca-d398-4c10-a502-b3ddbfe8ae59 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:05.502018 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 564/1133] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:40:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:40:05.507447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce4055-06de-4a97-b015-79d137e3929a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:05.507875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce4055-06de-4a97-b015-79d137e3929a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:05.507999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce4055-06de-4a97-b015-79d137e3929a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:05.572282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-dcce4055-06de-4a97-b015-79d137e3929a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:05.573505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dcce4055-06de-4a97-b015-79d137e3929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:05.601698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:05.669764 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 570/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:40:06.013188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b82ca-d398-4c10-a502-b3ddbfe8ae59 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:06.014784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0d0b82ca-d398-4c10-a502-b3ddbfe8ae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:06.118107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b82ca-d398-4c10-a502-b3ddbfe8ae59 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:06.146790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:06.192017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88211e5-f42e-4c4d-b5a3-208b8b8d4530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:06.192614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88211e5-f42e-4c4d-b5a3-208b8b8d4530 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:06.192881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88211e5-f42e-4c4d-b5a3-208b8b8d4530 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:06.193172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88211e5-f42e-4c4d-b5a3-208b8b8d4530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:06.194762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f88211e5-f42e-4c4d-b5a3-208b8b8d4530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:06.306176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88211e5-f42e-4c4d-b5a3-208b8b8d4530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:06.307535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88211e5-f42e-4c4d-b5a3-208b8b8d4530 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:06.325552 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 565/1135] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:40:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:40:08.784728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18e7af-30ee-496c-91b7-3ba2c521a69d None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:08.785327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18e7af-30ee-496c-91b7-3ba2c521a69d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:08.785327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18e7af-30ee-496c-91b7-3ba2c521a69d None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:08.786641 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 571/1136] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:08.792312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b8184-9888-4c5f-bc26-f721553f97fd None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:08.792753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b8184-9888-4c5f-bc26-f721553f97fd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:08.792872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b8184-9888-4c5f-bc26-f721553f97fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:08.860165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3f3b8184-9888-4c5f-bc26-f721553f97fd None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:08.863904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3f3b8184-9888-4c5f-bc26-f721553f97fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:08.870009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:08.959550 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 566/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:40:09.300215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18e7af-30ee-496c-91b7-3ba2c521a69d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:09.301785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ed18e7af-30ee-496c-91b7-3ba2c521a69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:09.405118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18e7af-30ee-496c-91b7-3ba2c521a69d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:09.459675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:09.477838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de91b4b-ac3b-4f1a-adf3-1e41151b5674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:09.478429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de91b4b-ac3b-4f1a-adf3-1e41151b5674 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:09.478710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de91b4b-ac3b-4f1a-adf3-1e41151b5674 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:09.479054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de91b4b-ac3b-4f1a-adf3-1e41151b5674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:09.480778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6de91b4b-ac3b-4f1a-adf3-1e41151b5674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:09.596223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de91b4b-ac3b-4f1a-adf3-1e41151b5674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:09.597687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de91b4b-ac3b-4f1a-adf3-1e41151b5674 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:09.616009 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 572/1138] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:40:12.136331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233b6fa-a52f-4848-85e6-98760fcdf98a None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:12.136895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233b6fa-a52f-4848-85e6-98760fcdf98a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:12.136895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233b6fa-a52f-4848-85e6-98760fcdf98a None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:12.138214 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 567/1139] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:12.143987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0f5e9-59a5-4cb6-9dc3-a9bcfb9d23c0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:12.144443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0f5e9-59a5-4cb6-9dc3-a9bcfb9d23c0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:12.144549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0f5e9-59a5-4cb6-9dc3-a9bcfb9d23c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:12.210717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-adf0f5e9-59a5-4cb6-9dc3-a9bcfb9d23c0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:12.211929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-adf0f5e9-59a5-4cb6-9dc3-a9bcfb9d23c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:12.252089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:12.310215 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 573/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:40:12.649632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233b6fa-a52f-4848-85e6-98760fcdf98a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:12.651157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f233b6fa-a52f-4848-85e6-98760fcdf98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:12.755174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233b6fa-a52f-4848-85e6-98760fcdf98a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:12.792512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:12.829594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21171e3f-b220-4250-9af5-68f31cbd0294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:12.830176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21171e3f-b220-4250-9af5-68f31cbd0294 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:12.830446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21171e3f-b220-4250-9af5-68f31cbd0294 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:12.830704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21171e3f-b220-4250-9af5-68f31cbd0294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:12.832257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-21171e3f-b220-4250-9af5-68f31cbd0294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:12.943117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21171e3f-b220-4250-9af5-68f31cbd0294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:12.944468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21171e3f-b220-4250-9af5-68f31cbd0294 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:12.963015 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 568/1141] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:40:15.428492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd5b1d-0baa-4f54-8de5-1c9712a2c8e6 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:15.428997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd5b1d-0baa-4f54-8de5-1c9712a2c8e6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:15.429144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd5b1d-0baa-4f54-8de5-1c9712a2c8e6 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:15.430380 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 574/1142] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:15.435765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624e3c3-f1b1-408f-8033-6402840c8c84 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:15.436201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624e3c3-f1b1-408f-8033-6402840c8c84 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:15.436308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624e3c3-f1b1-408f-8033-6402840c8c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:15.503859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7624e3c3-f1b1-408f-8033-6402840c8c84 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:15.505113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7624e3c3-f1b1-408f-8033-6402840c8c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:15.523105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:15.601574 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 569/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:40:15.942547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd5b1d-0baa-4f54-8de5-1c9712a2c8e6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:15.944097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-55cd5b1d-0baa-4f54-8de5-1c9712a2c8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:16.047740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd5b1d-0baa-4f54-8de5-1c9712a2c8e6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:16.114242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:16.121239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb86188-c5a2-488a-acf9-33c079d36fcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:16.121864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb86188-c5a2-488a-acf9-33c079d36fcf service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:16.122135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb86188-c5a2-488a-acf9-33c079d36fcf service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:16.122400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb86188-c5a2-488a-acf9-33c079d36fcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:16.123962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8bb86188-c5a2-488a-acf9-33c079d36fcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:16.238139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb86188-c5a2-488a-acf9-33c079d36fcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:16.239667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb86188-c5a2-488a-acf9-33c079d36fcf service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:16.258647 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 575/1144] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:40:18.670585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32f951-2dbe-4c01-8541-a72cbd87ccea None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:18.671146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32f951-2dbe-4c01-8541-a72cbd87ccea None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:18.671146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32f951-2dbe-4c01-8541-a72cbd87ccea None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:18.672973 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 570/1145] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:18.678216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671006a0-8c41-464a-a70a-24a03e3cf7e5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:18.678663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671006a0-8c41-464a-a70a-24a03e3cf7e5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:18.678806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671006a0-8c41-464a-a70a-24a03e3cf7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:18.744818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-671006a0-8c41-464a-a70a-24a03e3cf7e5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:18.746044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-671006a0-8c41-464a-a70a-24a03e3cf7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:18.794490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:18.844134 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 576/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:40:19.186891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32f951-2dbe-4c01-8541-a72cbd87ccea None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:19.189189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2c32f951-2dbe-4c01-8541-a72cbd87ccea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:19.293519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32f951-2dbe-4c01-8541-a72cbd87ccea None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:19.341703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:19.368417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac6d59-5e42-460b-9eac-2d6bcb3d22a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:19.369010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac6d59-5e42-460b-9eac-2d6bcb3d22a5 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:19.369316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac6d59-5e42-460b-9eac-2d6bcb3d22a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:19.369582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac6d59-5e42-460b-9eac-2d6bcb3d22a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:19.371155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-02ac6d59-5e42-460b-9eac-2d6bcb3d22a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:19.480631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac6d59-5e42-460b-9eac-2d6bcb3d22a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:19.482033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac6d59-5e42-460b-9eac-2d6bcb3d22a5 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:19.501106 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 571/1147] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:40:28.842090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=22 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:29.498362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=66 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:44.126920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d519e-d013-49b2-a686-15b5736d491f None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:44.127542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d519e-d013-49b2-a686-15b5736d491f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:44.127542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d519e-d013-49b2-a686-15b5736d491f None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:44.128771 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 577/1148] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:44.133954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c991b-bf86-4911-96e9-2129fe515331 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:44.134403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c991b-bf86-4911-96e9-2129fe515331 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:44.134509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c991b-bf86-4911-96e9-2129fe515331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:44.200856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e17c991b-bf86-4911-96e9-2129fe515331 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:44.202688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e17c991b-bf86-4911-96e9-2129fe515331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:44.298345 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 572/1149] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:40:44.651141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989533c-5400-43f4-b9ad-f7fbe66281b5 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:44.651419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989533c-5400-43f4-b9ad-f7fbe66281b5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:44.651686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989533c-5400-43f4-b9ad-f7fbe66281b5 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:44.652996 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 578/1150] 10.222.0.16 () {54 vars in 962 bytes} [Sat Oct 15 20:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:44.658520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d2189-a011-4cf9-9ddd-6381a6bb2eef None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:44.658802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d2189-a011-4cf9-9ddd-6381a6bb2eef None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:44.659070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d2189-a011-4cf9-9ddd-6381a6bb2eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:44.734274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6c5d2189-a011-4cf9-9ddd-6381a6bb2eef None None] MFA Rules not processed for user `af8591ebfa174c91853b673360326a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:44.735477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6c5d2189-a011-4cf9-9ddd-6381a6bb2eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:44.755916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:44.810943 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 573/1151] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Oct 15 20:40:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:40:44.818860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989533c-5400-43f4-b9ad-f7fbe66281b5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:44.820545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9989533c-5400-43f4-b9ad-f7fbe66281b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:44.924382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989533c-5400-43f4-b9ad-f7fbe66281b5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:44.996521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f22d0-84bc-4982-88d4-13cd27c8b94f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:44.997148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f22d0-84bc-4982-88d4-13cd27c8b94f service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:44.997427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f22d0-84bc-4982-88d4-13cd27c8b94f service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:44.997690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f22d0-84bc-4982-88d4-13cd27c8b94f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:44.999256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-487f22d0-84bc-4982-88d4-13cd27c8b94f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:45.054466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:45.112204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f22d0-84bc-4982-88d4-13cd27c8b94f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:45.113648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f22d0-84bc-4982-88d4-13cd27c8b94f service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:45.132479 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 579/1152] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:40:47.641569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f3b86-26a9-49e3-8e29-023cafea42b4 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:47.642090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f3b86-26a9-49e3-8e29-023cafea42b4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:47.642090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f3b86-26a9-49e3-8e29-023cafea42b4 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:47.643458 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 574/1153] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:47.648698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130186ec-4273-4dff-a19f-a59e29ecd2b6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:47.649141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130186ec-4273-4dff-a19f-a59e29ecd2b6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:47.649246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130186ec-4273-4dff-a19f-a59e29ecd2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:47.715061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-130186ec-4273-4dff-a19f-a59e29ecd2b6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:47.716262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-130186ec-4273-4dff-a19f-a59e29ecd2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:47.812854 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 580/1154] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:40:48.144364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f3b86-26a9-49e3-8e29-023cafea42b4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:48.145972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-832f3b86-26a9-49e3-8e29-023cafea42b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:48.273868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f3b86-26a9-49e3-8e29-023cafea42b4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:48.327722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:48.372202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45ac6f-c3e0-40f2-b17e-cbe47af0c028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:48.372786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45ac6f-c3e0-40f2-b17e-cbe47af0c028 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:48.373317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45ac6f-c3e0-40f2-b17e-cbe47af0c028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:48.373508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45ac6f-c3e0-40f2-b17e-cbe47af0c028 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:48.373993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45ac6f-c3e0-40f2-b17e-cbe47af0c028 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:48.375083 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-1c45ac6f-c3e0-40f2-b17e-cbe47af0c028 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:40:48.375083 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 20:40:48.375083 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 20:40:48.376211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45ac6f-c3e0-40f2-b17e-cbe47af0c028 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:48.395406 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 575/1155] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Oct 15 20:40:48 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:40:50.821784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d94cd3-318b-419d-a0e6-06fde4e46b3a None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:50.822316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d94cd3-318b-419d-a0e6-06fde4e46b3a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:50.822412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d94cd3-318b-419d-a0e6-06fde4e46b3a None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:50.823695 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 581/1156] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:50.828958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324ddf-57e3-4050-85af-bbf673216b55 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:50.829420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324ddf-57e3-4050-85af-bbf673216b55 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:50.829538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324ddf-57e3-4050-85af-bbf673216b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:50.894542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d1324ddf-57e3-4050-85af-bbf673216b55 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:50.895727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d1324ddf-57e3-4050-85af-bbf673216b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:50.990976 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 576/1157] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:40:51.338206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d94cd3-318b-419d-a0e6-06fde4e46b3a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:51.339864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b9d94cd3-318b-419d-a0e6-06fde4e46b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:51.350875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:51.496464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d94cd3-318b-419d-a0e6-06fde4e46b3a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:51.576786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:51.609681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e053fa-a054-4355-99a1-e11f294020ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:51.610280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e053fa-a054-4355-99a1-e11f294020ef service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:51.610547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e053fa-a054-4355-99a1-e11f294020ef service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:51.610834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e053fa-a054-4355-99a1-e11f294020ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:51.612656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e1e053fa-a054-4355-99a1-e11f294020ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:51.702002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e053fa-a054-4355-99a1-e11f294020ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:51.703592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e053fa-a054-4355-99a1-e11f294020ef service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:51.721407 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 582/1158] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:40:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:40:54.290893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52426b81-ef32-4218-afca-aa9aa9d53106 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:54.291478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52426b81-ef32-4218-afca-aa9aa9d53106 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:54.291478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52426b81-ef32-4218-afca-aa9aa9d53106 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:54.292783 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 577/1159] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:54.298091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5089f0-fc9e-408d-bff8-af59bb4af197 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:54.298488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5089f0-fc9e-408d-bff8-af59bb4af197 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:54.298627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5089f0-fc9e-408d-bff8-af59bb4af197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:54.364965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-be5089f0-fc9e-408d-bff8-af59bb4af197 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:54.366190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-be5089f0-fc9e-408d-bff8-af59bb4af197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:54.412098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:54.462331 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 583/1160] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:40:54.800349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52426b81-ef32-4218-afca-aa9aa9d53106 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:54.801904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-52426b81-ef32-4218-afca-aa9aa9d53106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:54.826497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:54.906906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52426b81-ef32-4218-afca-aa9aa9d53106 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:54.981394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f7a53-e0df-4bc3-84ea-eef2ac770d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:54.982536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f7a53-e0df-4bc3-84ea-eef2ac770d8b service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:54.982816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f7a53-e0df-4bc3-84ea-eef2ac770d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:54.983085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f7a53-e0df-4bc3-84ea-eef2ac770d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:54.984699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-481f7a53-e0df-4bc3-84ea-eef2ac770d8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:55.061066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:55.096684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f7a53-e0df-4bc3-84ea-eef2ac770d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:55.098182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f7a53-e0df-4bc3-84ea-eef2ac770d8b service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:55.116250 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 578/1161] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:40:57.551588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75830c-a02d-4ef8-8772-d7e375047039 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:57.552145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75830c-a02d-4ef8-8772-d7e375047039 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:57.552145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75830c-a02d-4ef8-8772-d7e375047039 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:57.553464 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 584/1162] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:40:57.558753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec0ffe-bf36-4831-a147-1d136036de62 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:57.559193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec0ffe-bf36-4831-a147-1d136036de62 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:57.559288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec0ffe-bf36-4831-a147-1d136036de62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:57.625365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8aec0ffe-bf36-4831-a147-1d136036de62 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:40:57.626556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8aec0ffe-bf36-4831-a147-1d136036de62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:57.721675 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 579/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:40:58.070469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75830c-a02d-4ef8-8772-d7e375047039 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:40:58.071979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5a75830c-a02d-4ef8-8772-d7e375047039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:58.174284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75830c-a02d-4ef8-8772-d7e375047039 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:40:58.247169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:58.249862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e75c92-1ec3-43c8-95b4-59ab48de3abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:40:58.250529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e75c92-1ec3-43c8-95b4-59ab48de3abe service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:40:58.250998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e75c92-1ec3-43c8-95b4-59ab48de3abe service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:40:58.251428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e75c92-1ec3-43c8-95b4-59ab48de3abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:40:58.253271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-24e75c92-1ec3-43c8-95b4-59ab48de3abe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:40:58.363591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e75c92-1ec3-43c8-95b4-59ab48de3abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:40:58.364910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e75c92-1ec3-43c8-95b4-59ab48de3abe service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:40:58.382950 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 585/1164] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:41:00.967703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4778a4-ba09-4e33-99ec-7cc67591473a None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:41:00.968269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4778a4-ba09-4e33-99ec-7cc67591473a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:41:00.968269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4778a4-ba09-4e33-99ec-7cc67591473a None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:41:00.969634 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 580/1165] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:41:00.974757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc61e66-2c92-43ea-ae5a-19356fe30c01 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:41:00.975180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc61e66-2c92-43ea-ae5a-19356fe30c01 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:41:00.975296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc61e66-2c92-43ea-ae5a-19356fe30c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:41:01.041275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fdc61e66-2c92-43ea-ae5a-19356fe30c01 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:41:01.042470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fdc61e66-2c92-43ea-ae5a-19356fe30c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:41:01.095276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:41:01.140068 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 586/1166] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:41:01.485279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4778a4-ba09-4e33-99ec-7cc67591473a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:41:01.487119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0e4778a4-ba09-4e33-99ec-7cc67591473a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:41:01.522817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:41:01.595763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4778a4-ba09-4e33-99ec-7cc67591473a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:41:01.671384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f32d7-7a46-4490-bb7d-98875db071b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:41:01.671976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f32d7-7a46-4490-bb7d-98875db071b3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:41:01.672250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f32d7-7a46-4490-bb7d-98875db071b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:41:01.672516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f32d7-7a46-4490-bb7d-98875db071b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:41:01.674143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6b6f32d7-7a46-4490-bb7d-98875db071b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:41:01.756224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:41:01.785260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f32d7-7a46-4490-bb7d-98875db071b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:41:01.786689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f32d7-7a46-4490-bb7d-98875db071b3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:41:01.804744 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 581/1167] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:41:04.557564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901c9d-73f3-413b-8383-6a8def4d36d3 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:41:04.558131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901c9d-73f3-413b-8383-6a8def4d36d3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:41:04.558131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901c9d-73f3-413b-8383-6a8def4d36d3 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:41:04.559362 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 587/1168] 10.222.0.16 () {54 vars in 927 bytes} [Sat Oct 15 20:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:41:04.565078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65486f12-126e-45a4-b874-9095ae1bf3ad None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:41:04.565518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65486f12-126e-45a4-b874-9095ae1bf3ad None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:41:04.565623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65486f12-126e-45a4-b874-9095ae1bf3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:41:04.633482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-65486f12-126e-45a4-b874-9095ae1bf3ad None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:41:04.634691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-65486f12-126e-45a4-b874-9095ae1bf3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:41:04.732690 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 582/1169] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Oct 15 20:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:41:05.073948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7901c9d-73f3-413b-8383-6a8def4d36d3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:41:05.075524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a7901c9d-73f3-413b-8383-6a8def4d36d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:41:05.215981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7901c9d-73f3-413b-8383-6a8def4d36d3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:41:05.310800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d992902-7487-4432-bf85-b4fcebd37ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:41:05.311410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d992902-7487-4432-bf85-b4fcebd37ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:41:05.311685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d992902-7487-4432-bf85-b4fcebd37ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:41:05.311953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d992902-7487-4432-bf85-b4fcebd37ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:41:05.313597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6d992902-7487-4432-bf85-b4fcebd37ee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:41:05.322646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:41:05.439648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d992902-7487-4432-bf85-b4fcebd37ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:41:05.441121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d992902-7487-4432-bf85-b4fcebd37ee7 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:41:05.460304 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 588/1170] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:41:14.730963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=82 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:41:15.457997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=50 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:42:20.782203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3a2d2-e776-4ba7-b5bd-1ae209b4fb47 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:20.782803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3a2d2-e776-4ba7-b5bd-1ae209b4fb47 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:20.782803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3a2d2-e776-4ba7-b5bd-1ae209b4fb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:20.858430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-98c3a2d2-e776-4ba7-b5bd-1ae209b4fb47 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:42:20.859891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-98c3a2d2-e776-4ba7-b5bd-1ae209b4fb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:20.955970 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 583/1171] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:42:20.964094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd6c9a-a523-4c52-8824-db5a6cdc1e9c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:20.964572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd6c9a-a523-4c52-8824-db5a6cdc1e9c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:20.964674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd6c9a-a523-4c52-8824-db5a6cdc1e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:21.042666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-befd6c9a-a523-4c52-8824-db5a6cdc1e9c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:42:21.043804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-befd6c9a-a523-4c52-8824-db5a6cdc1e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:21.138585 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 589/1172] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:42:21.143364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3a2d2-e776-4ba7-b5bd-1ae209b4fb47 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:42:21.144559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-98c3a2d2-e776-4ba7-b5bd-1ae209b4fb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:21.156727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3a2d2-e776-4ba7-b5bd-1ae209b4fb47 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:42:21.219755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:42:21.259972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:42:21.260565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:21.260834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:21.261137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:21.261770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:42:21.262064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:42:21.263066 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:42:21.263066 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 20:42:21.263066 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 20:42:21.263938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:42:21.277460 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 584/1173] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:42:21.286863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd6c9a-a523-4c52-8824-db5a6cdc1e9c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:42:21.296847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd6c9a-a523-4c52-8824-db5a6cdc1e9c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:42:21.357027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:42:21.390262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711e3c0-ab36-4884-956e-72ffd1739629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:42:21.390848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711e3c0-ab36-4884-956e-72ffd1739629 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:21.391115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711e3c0-ab36-4884-956e-72ffd1739629 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:21.391376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711e3c0-ab36-4884-956e-72ffd1739629 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:21.391995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711e3c0-ab36-4884-956e-72ffd1739629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:42:21.392984 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-f711e3c0-ab36-4884-956e-72ffd1739629 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:42:21.392984 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 20:42:21.392984 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 20:42:21.394213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711e3c0-ab36-4884-956e-72ffd1739629 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:42:21.448911 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 590/1174] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:42:21 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:42:21.453059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:42:21.455007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:21.492985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33f631-f1b7-4973-b5d2-cdc2e9e7a3a0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:42:21.591994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467278c4-3f57-4de0-b9ef-21d925217237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:42:21.592501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467278c4-3f57-4de0-b9ef-21d925217237 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:21.592772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467278c4-3f57-4de0-b9ef-21d925217237 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:21.593059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467278c4-3f57-4de0-b9ef-21d925217237 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:21.593684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467278c4-3f57-4de0-b9ef-21d925217237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:42:21.594688 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-467278c4-3f57-4de0-b9ef-21d925217237 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:42:21.594688 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 20:42:21.594688 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 20:42:21.595884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467278c4-3f57-4de0-b9ef-21d925217237 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:42:21.639888 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 585/1175] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:42:21 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:42:21.643953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711e3c0-ab36-4884-956e-72ffd1739629 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:42:21.645517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f711e3c0-ab36-4884-956e-72ffd1739629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:21.707586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711e3c0-ab36-4884-956e-72ffd1739629 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:42:21.792241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:42:21.803111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a86193-d921-448e-8742-f95405d17049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:42:21.803699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a86193-d921-448e-8742-f95405d17049 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:21.803967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a86193-d921-448e-8742-f95405d17049 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:21.804228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a86193-d921-448e-8742-f95405d17049 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:21.804881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a86193-d921-448e-8742-f95405d17049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:42:21.806547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a86193-d921-448e-8742-f95405d17049 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:42:21.818799 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 591/1176] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:42:21.822917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467278c4-3f57-4de0-b9ef-21d925217237 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:42:21.832651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467278c4-3f57-4de0-b9ef-21d925217237 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:42:21.837767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e9411-b5bd-4d19-afc8-6a7a01df9bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:42:21.838341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e9411-b5bd-4d19-afc8-6a7a01df9bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:21.838612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e9411-b5bd-4d19-afc8-6a7a01df9bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:21.838874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e9411-b5bd-4d19-afc8-6a7a01df9bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:21.839496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e9411-b5bd-4d19-afc8-6a7a01df9bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:42:21.841097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e9411-b5bd-4d19-afc8-6a7a01df9bf0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:42:21.854504 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 586/1177] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:42:21.858484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a86193-d921-448e-8742-f95405d17049 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:42:21.860261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-49a86193-d921-448e-8742-f95405d17049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:21.894486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a86193-d921-448e-8742-f95405d17049 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:42:21.989657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34301c2-2cd1-41d4-b3ea-b9d5c30bd457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:42:21.990348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34301c2-2cd1-41d4-b3ea-b9d5c30bd457 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:21.990572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34301c2-2cd1-41d4-b3ea-b9d5c30bd457 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:21.990828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34301c2-2cd1-41d4-b3ea-b9d5c30bd457 admin admin] PATH_INFO: `/v3/projects/278f8764f08c4d00a2d2386140a82960/users/a80fd2e8d63d452a8b28ab52ee97e011/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:22.013082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34301c2-2cd1-41d4-b3ea-b9d5c30bd457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=278f8764f08c4d00a2d2386140a82960, user_id=a80fd2e8d63d452a8b28ab52ee97e011, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:42:22.015720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34301c2-2cd1-41d4-b3ea-b9d5c30bd457 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:42:22.042004 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 592/1178] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:42:21 2022] PUT /identity/v3/projects/278f8764f08c4d00a2d2386140a82960/users/a80fd2e8d63d452a8b28ab52ee97e011/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:42:22.052742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d1d3c-90c3-4211-b04a-ded0ef48f05b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:22.053116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d1d3c-90c3-4211-b04a-ded0ef48f05b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:22.053388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d1d3c-90c3-4211-b04a-ded0ef48f05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:22.077643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:42:22.104120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cb3d1d3c-90c3-4211-b04a-ded0ef48f05b None None] MFA Rules not processed for user `a80fd2e8d63d452a8b28ab52ee97e011`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:42:22.105383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cb3d1d3c-90c3-4211-b04a-ded0ef48f05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:22.169569 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 587/1179] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:42:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:42:22.182510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953bef-0864-450f-8524-be42ff1928cd None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:22.182787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953bef-0864-450f-8524-be42ff1928cd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:22.183051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953bef-0864-450f-8524-be42ff1928cd None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:22.184385 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 593/1180] 10.222.0.16 () {54 vars in 963 bytes} [Sat Oct 15 20:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:42:22.189143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950ca41-d126-4755-b6a2-e9cfcbd568ae None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:22.189434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950ca41-d126-4755-b6a2-e9cfcbd568ae None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:22.189710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950ca41-d126-4755-b6a2-e9cfcbd568ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:22.261816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5950ca41-d126-4755-b6a2-e9cfcbd568ae None None] MFA Rules not processed for user `aa85b647a4be4d81b966bb50691bd3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:42:22.263026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5950ca41-d126-4755-b6a2-e9cfcbd568ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:22.338036 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 588/1181] 10.222.0.16 () {58 vars in 1060 bytes} [Sat Oct 15 20:42:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:42:22.345701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953bef-0864-450f-8524-be42ff1928cd None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:42:22.357112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953bef-0864-450f-8524-be42ff1928cd None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:42:22.360900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e8624-8978-4096-993e-5f0ef8a5feda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:42:22.361571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e8624-8978-4096-993e-5f0ef8a5feda service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:22.361838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e8624-8978-4096-993e-5f0ef8a5feda service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:22.362097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e8624-8978-4096-993e-5f0ef8a5feda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:22.370294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e8624-8978-4096-993e-5f0ef8a5feda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:42:22.371729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e8624-8978-4096-993e-5f0ef8a5feda service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:42:22.380267 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 594/1182] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:42:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:42:22.426594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc374e7-0527-424b-ae7e-45548f7e0cfa None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:22.426882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc374e7-0527-424b-ae7e-45548f7e0cfa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:22.427152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc374e7-0527-424b-ae7e-45548f7e0cfa None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:22.428437 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 589/1183] 10.222.0.16 () {52 vars in 830 bytes} [Sat Oct 15 20:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:42:22.432046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e8624-8978-4096-993e-5f0ef8a5feda service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:42:22.442591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e8624-8978-4096-993e-5f0ef8a5feda service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:42:22.447997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:42:22.448588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:22.448857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:22.449185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] PATH_INFO: `/v3/users/a80fd2e8d63d452a8b28ab52ee97e011` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:22.450286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a80fd2e8d63d452a8b28ab52ee97e011)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:42:22.451291 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:42:22.451291 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 20:42:22.451291 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 20:42:22.452457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:42:22.494309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:42:22.521030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a80fd2e8d63d452a8b28ab52ee97e011'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:42:22.535972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a80fd2e8d63d452a8b28ab52ee97e011'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:42:22.559547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a80fd2e8d63d452a8b28ab52ee97e011'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:42:22.568904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a80fd2e8d63d452a8b28ab52ee97e011'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:42:22.569570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a80fd2e8d63d452a8b28ab52ee97e011'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:42:22.576226 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 595/1184] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:42:22 2022] DELETE /identity/v3/users/a80fd2e8d63d452a8b28ab52ee97e011 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:42:22.580770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407a1c2-af72-4f25-bc11-f0665b508be9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:22.581069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407a1c2-af72-4f25-bc11-f0665b508be9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:22.581369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407a1c2-af72-4f25-bc11-f0665b508be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:22.608538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8407a1c2-af72-4f25-bc11-f0665b508be9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:42:22.609984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8407a1c2-af72-4f25-bc11-f0665b508be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:22.628792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:42:22.670632 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 590/1185] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:42:22.681280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:42:22.682694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:22.761682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821a7c-382f-4b3d-b8b5-e12c6fd7dda3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:42:22.765339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cee92-5ffe-4b50-9008-df92008c7edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:42:22.765933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cee92-5ffe-4b50-9008-df92008c7edc service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:22.766202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cee92-5ffe-4b50-9008-df92008c7edc service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:22.766465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cee92-5ffe-4b50-9008-df92008c7edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:22.775740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cee92-5ffe-4b50-9008-df92008c7edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:42:22.777179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cee92-5ffe-4b50-9008-df92008c7edc service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:42:22.786381 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 596/1186] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:42:23.140559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407a1c2-af72-4f25-bc11-f0665b508be9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:42:23.141998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8407a1c2-af72-4f25-bc11-f0665b508be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:42:23.154043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407a1c2-af72-4f25-bc11-f0665b508be9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:42:23.159726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:42:23.160564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:42:23.160904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:42:23.161272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] PATH_INFO: `/v3/projects/278f8764f08c4d00a2d2386140a82960` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:42:23.162580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=278f8764f08c4d00a2d2386140a82960)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:42:23.163671 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:42:23.163671 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 20:42:23.163671 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 20:42:23.168196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:42:23.232139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278f8764f08c4d00a2d2386140a82960', 'request_id': 'req-3150fa35-98b4-49c2-b1a6-df156ff87420'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:42:23.241629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278f8764f08c4d00a2d2386140a82960', 'request_id': 'req-3150fa35-98b4-49c2-b1a6-df156ff87420'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:42:23.257653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278f8764f08c4d00a2d2386140a82960', 'request_id': 'req-3150fa35-98b4-49c2-b1a6-df156ff87420'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:42:23.268501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] The token cache is being invalidate because project 278f8764f08c4d00a2d2386140a82960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:42:23.268776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:42:23.269432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3150fa35-98b4-49c2-b1a6-df156ff87420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:42:23.271003 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 591/1187] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:42:23 2022] DELETE /identity/v3/projects/278f8764f08c4d00a2d2386140a82960 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:42:32.782291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=54,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:42:33.265501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:45:36.356142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fded454-6bc3-4d36-80a3-5a6f8b0b5d9e None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:45:36.356733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fded454-6bc3-4d36-80a3-5a6f8b0b5d9e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:45:36.356733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fded454-6bc3-4d36-80a3-5a6f8b0b5d9e None None] PATH_INFO: `/v3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:45:36.358030 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 597/1188] 10.222.0.17 () {54 vars in 925 bytes} [Sat Oct 15 20:45:36 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:45:36.364095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a72e2-0d50-46d2-8567-8bc3bd4d6430 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:45:36.364520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a72e2-0d50-46d2-8567-8bc3bd4d6430 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:45:36.364672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a72e2-0d50-46d2-8567-8bc3bd4d6430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:45:36.413393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fb9a72e2-0d50-46d2-8567-8bc3bd4d6430 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:45:36.415161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fb9a72e2-0d50-46d2-8567-8bc3bd4d6430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:45:36.487305 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 592/1189] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Oct 15 20:45:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:45:40.149363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cb173-fe07-4e5f-9544-c722e0df8a16 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:45:40.149839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cb173-fe07-4e5f-9544-c722e0df8a16 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:45:40.149947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cb173-fe07-4e5f-9544-c722e0df8a16 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:45:40.151848 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 598/1190] 10.222.0.16 () {54 vars in 961 bytes} [Sat Oct 15 20:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:45:40.157324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b93858-9a8b-4ca8-8c89-abb0ab7aa4ab None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:45:40.157782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b93858-9a8b-4ca8-8c89-abb0ab7aa4ab None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:45:40.157887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b93858-9a8b-4ca8-8c89-abb0ab7aa4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:45:40.177969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-28b93858-9a8b-4ca8-8c89-abb0ab7aa4ab None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:45:40.179147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-28b93858-9a8b-4ca8-8c89-abb0ab7aa4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:45:40.187958 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 593/1191] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Oct 15 20:45:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:45:40.195694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cb173-fe07-4e5f-9544-c722e0df8a16 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:45:40.214446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cb173-fe07-4e5f-9544-c722e0df8a16 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:45:40.219493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fe1eb-b14a-49be-90a5-f8230ff7fcf5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:45:40.220075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fe1eb-b14a-49be-90a5-f8230ff7fcf5 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:45:40.220348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fe1eb-b14a-49be-90a5-f8230ff7fcf5 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:45:40.220614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fe1eb-b14a-49be-90a5-f8230ff7fcf5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:45:40.228966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fe1eb-b14a-49be-90a5-f8230ff7fcf5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:45:40.231685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fe1eb-b14a-49be-90a5-f8230ff7fcf5 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:45:40.241182 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 599/1192] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:45:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:45:40.689418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5035b4-83af-43ae-bf10-d9a6eb8d4496 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:45:40.689701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5035b4-83af-43ae-bf10-d9a6eb8d4496 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:45:40.689969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5035b4-83af-43ae-bf10-d9a6eb8d4496 None None] PATH_INFO: `/v3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:45:40.691271 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 594/1193] 10.222.0.25 () {54 vars in 925 bytes} [Sat Oct 15 20:45:40 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:45:40.697623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b2e7-3e84-4554-b7e3-f7cad974d592 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:45:40.698559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b2e7-3e84-4554-b7e3-f7cad974d592 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:45:40.698804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b2e7-3e84-4554-b7e3-f7cad974d592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:45:40.720251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-aab2b2e7-3e84-4554-b7e3-f7cad974d592 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:45:40.721491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aab2b2e7-3e84-4554-b7e3-f7cad974d592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:45:40.730202 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 600/1194] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Oct 15 20:45:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:45:44.377116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baa164-0acc-4ad9-aea9-760effb795ac None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:45:44.377543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baa164-0acc-4ad9-aea9-760effb795ac None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:45:44.377642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baa164-0acc-4ad9-aea9-760effb795ac None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:45:44.378949 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 595/1195] 10.222.0.16 () {54 vars in 961 bytes} [Sat Oct 15 20:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:45:44.383822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb4d16-668d-4d40-91b6-5c1903fee7f6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:45:44.384226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb4d16-668d-4d40-91b6-5c1903fee7f6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:45:44.384358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb4d16-668d-4d40-91b6-5c1903fee7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:45:44.405418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-acbb4d16-668d-4d40-91b6-5c1903fee7f6 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:45:44.406596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-acbb4d16-668d-4d40-91b6-5c1903fee7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:45:44.414973 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 601/1196] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Oct 15 20:45:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:45:44.422877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baa164-0acc-4ad9-aea9-760effb795ac None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:45:44.436789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baa164-0acc-4ad9-aea9-760effb795ac None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:45:44.442004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27177beb-f555-4bb4-ba8d-2767546a8c0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:45:44.442595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27177beb-f555-4bb4-ba8d-2767546a8c0d service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:45:44.442865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27177beb-f555-4bb4-ba8d-2767546a8c0d service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:45:44.443155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27177beb-f555-4bb4-ba8d-2767546a8c0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:45:44.451754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27177beb-f555-4bb4-ba8d-2767546a8c0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:45:44.453379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27177beb-f555-4bb4-ba8d-2767546a8c0d service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:45:44.463717 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 596/1197] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:45:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:45:54.400410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=12 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:45:54.459514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=53 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:46:18.184132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bf1cc-6b3a-4172-b8c5-2412f18c9c69 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:18.184727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bf1cc-6b3a-4172-b8c5-2412f18c9c69 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:18.184727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bf1cc-6b3a-4172-b8c5-2412f18c9c69 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:18.186065 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 602/1198] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 20:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:46:18.191186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b84c24-903c-4318-8571-195b03ad758b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:18.191647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b84c24-903c-4318-8571-195b03ad758b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:18.191751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b84c24-903c-4318-8571-195b03ad758b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:18.232308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-71b84c24-903c-4318-8571-195b03ad758b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:46:18.233522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-71b84c24-903c-4318-8571-195b03ad758b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:46:18.291514 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 597/1199] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 20:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:46:18.460157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bf1cc-6b3a-4172-b8c5-2412f18c9c69 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:46:18.461950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b72bf1cc-6b3a-4172-b8c5-2412f18c9c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:46:18.563315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bf1cc-6b3a-4172-b8c5-2412f18c9c69 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:46:18.569503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d560da-03c1-4f8a-be85-c078e92ee44b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:46:18.570098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d560da-03c1-4f8a-be85-c078e92ee44b service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:18.570373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d560da-03c1-4f8a-be85-c078e92ee44b service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:18.570639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d560da-03c1-4f8a-be85-c078e92ee44b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:18.578798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d560da-03c1-4f8a-be85-c078e92ee44b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:46:18.580384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d560da-03c1-4f8a-be85-c078e92ee44b service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:46:18.590287 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 603/1200] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:46:20.215352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ab37-44bd-45ed-b348-b417facf5459 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:20.215804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ab37-44bd-45ed-b348-b417facf5459 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:20.215912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ab37-44bd-45ed-b348-b417facf5459 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:20.217283 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 598/1201] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 20:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:46:20.222065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fffb5c-b18e-449c-844e-1f496ba96212 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:20.222509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fffb5c-b18e-449c-844e-1f496ba96212 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:20.222613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fffb5c-b18e-449c-844e-1f496ba96212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:20.247716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-96fffb5c-b18e-449c-844e-1f496ba96212 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:46:20.249082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-96fffb5c-b18e-449c-844e-1f496ba96212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:46:20.257292 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 604/1202] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 20:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:46:20.422995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7ab37-44bd-45ed-b348-b417facf5459 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:46:20.437431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7ab37-44bd-45ed-b348-b417facf5459 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:46:20.443003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896eaaa5-1103-4017-a23e-fd0827bc5830 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:46:20.443582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896eaaa5-1103-4017-a23e-fd0827bc5830 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:20.443855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896eaaa5-1103-4017-a23e-fd0827bc5830 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:20.444122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896eaaa5-1103-4017-a23e-fd0827bc5830 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:20.452291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896eaaa5-1103-4017-a23e-fd0827bc5830 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:46:20.453888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896eaaa5-1103-4017-a23e-fd0827bc5830 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:46:20.463912 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 599/1203] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:46:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:46:24.565804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41874f91-a07c-4e37-92d7-b9e87614afbe None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:24.566386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41874f91-a07c-4e37-92d7-b9e87614afbe None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:24.566386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41874f91-a07c-4e37-92d7-b9e87614afbe None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:24.567631 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 605/1204] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:46:24.573379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be35ee9-c6fa-4c21-a4b7-3abe61070630 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:24.573785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be35ee9-c6fa-4c21-a4b7-3abe61070630 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:24.573922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be35ee9-c6fa-4c21-a4b7-3abe61070630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:24.593505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4be35ee9-c6fa-4c21-a4b7-3abe61070630 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:46:24.594671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4be35ee9-c6fa-4c21-a4b7-3abe61070630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:46:24.602369 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 600/1205] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:46:24.948589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41874f91-a07c-4e37-92d7-b9e87614afbe None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:46:24.951079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-41874f91-a07c-4e37-92d7-b9e87614afbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:46:25.008741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41874f91-a07c-4e37-92d7-b9e87614afbe None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:46:25.012193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb4236-7d2e-4d0f-b62a-f7fde5083f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:46:25.012780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb4236-7d2e-4d0f-b62a-f7fde5083f82 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:25.013058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb4236-7d2e-4d0f-b62a-f7fde5083f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:25.013337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb4236-7d2e-4d0f-b62a-f7fde5083f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:25.021984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb4236-7d2e-4d0f-b62a-f7fde5083f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:46:25.023535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb4236-7d2e-4d0f-b62a-f7fde5083f82 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:46:25.032657 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 606/1206] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:46:27.403647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa94d1-af65-4c2e-98b2-8840ccffb6d7 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:27.404176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa94d1-af65-4c2e-98b2-8840ccffb6d7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:27.404176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa94d1-af65-4c2e-98b2-8840ccffb6d7 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:27.405567 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 601/1207] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 20:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:46:27.411042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426519-7def-4df6-8e0a-6e4a156951ba None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:27.411482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426519-7def-4df6-8e0a-6e4a156951ba None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:27.411588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426519-7def-4df6-8e0a-6e4a156951ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:27.431138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cc426519-7def-4df6-8e0a-6e4a156951ba None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:46:27.432297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cc426519-7def-4df6-8e0a-6e4a156951ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:46:27.439915 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 607/1208] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 20:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:46:27.776919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa94d1-af65-4c2e-98b2-8840ccffb6d7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:46:27.788306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa94d1-af65-4c2e-98b2-8840ccffb6d7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:46:27.791798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab28d83-c4e2-47e1-9543-f15ccd14876b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:46:27.792374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28d83-c4e2-47e1-9543-f15ccd14876b service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:46:27.792643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28d83-c4e2-47e1-9543-f15ccd14876b service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:46:27.792904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28d83-c4e2-47e1-9543-f15ccd14876b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:46:27.801755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab28d83-c4e2-47e1-9543-f15ccd14876b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:46:27.803905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab28d83-c4e2-47e1-9543-f15ccd14876b service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:46:27.813415 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 602/1209] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:46:37.425927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=78 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:46:37.809197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=49 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:17.290460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53d0f-5960-4707-956a-97b469213fdd None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:17.291055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53d0f-5960-4707-956a-97b469213fdd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:17.291055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53d0f-5960-4707-956a-97b469213fdd None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:17.292592 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 608/1210] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:47:17.297902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58f6c5-630d-4db5-9d0b-fae4a3029472 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:17.298372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58f6c5-630d-4db5-9d0b-fae4a3029472 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:17.298574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58f6c5-630d-4db5-9d0b-fae4a3029472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:17.318833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9f58f6c5-630d-4db5-9d0b-fae4a3029472 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:17.320016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9f58f6c5-630d-4db5-9d0b-fae4a3029472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:17.328948 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 603/1211] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:47:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:27.313884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:50.791664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22844b-9fe8-4dc1-a362-c66ae16a8e46 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:50.792342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22844b-9fe8-4dc1-a362-c66ae16a8e46 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:50.792459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22844b-9fe8-4dc1-a362-c66ae16a8e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:50.814778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1c22844b-9fe8-4dc1-a362-c66ae16a8e46 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:50.816069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1c22844b-9fe8-4dc1-a362-c66ae16a8e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:50.829812 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 609/1212] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:47:50.833482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc50dd-6add-420d-84bb-c7374b993f09 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:50.833986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc50dd-6add-420d-84bb-c7374b993f09 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:50.834101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc50dd-6add-420d-84bb-c7374b993f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:50.866193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-14fc50dd-6add-420d-84bb-c7374b993f09 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:50.867400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-14fc50dd-6add-420d-84bb-c7374b993f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:50.877670 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 604/1213] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:50.882593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5460d-f400-475d-8d1f-fa4fc49891b4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:50.882875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5460d-f400-475d-8d1f-fa4fc49891b4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:50.883138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5460d-f400-475d-8d1f-fa4fc49891b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:50.895070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e87868-b3d6-471c-812e-9f6d2a759c6b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:50.895423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e87868-b3d6-471c-812e-9f6d2a759c6b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:50.895618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e87868-b3d6-471c-812e-9f6d2a759c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:50.915591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0ea5460d-f400-475d-8d1f-fa4fc49891b4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:50.916797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0ea5460d-f400-475d-8d1f-fa4fc49891b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:50.930032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-47e87868-b3d6-471c-812e-9f6d2a759c6b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:50.932015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-47e87868-b3d6-471c-812e-9f6d2a759c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:50.937744 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 610/1214] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:47:50.943527 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 605/1215] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:50.948379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5460d-f400-475d-8d1f-fa4fc49891b4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:50.948567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e87868-b3d6-471c-812e-9f6d2a759c6b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:50.964075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e87868-b3d6-471c-812e-9f6d2a759c6b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:50.965939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5460d-f400-475d-8d1f-fa4fc49891b4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:50.973305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3a979-8305-48a0-9a6f-5721d65edef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:50.973913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a979-8305-48a0-9a6f-5721d65edef9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:50.974224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a979-8305-48a0-9a6f-5721d65edef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:50.974486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a979-8305-48a0-9a6f-5721d65edef9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:50.975261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3a979-8305-48a0-9a6f-5721d65edef9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:47:50.975836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3a979-8305-48a0-9a6f-5721d65edef9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:50.979208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35951f0f-6538-47fa-a532-5daa06d0222e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:50.979826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35951f0f-6538-47fa-a532-5daa06d0222e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:50.980105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35951f0f-6538-47fa-a532-5daa06d0222e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:50.980621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35951f0f-6538-47fa-a532-5daa06d0222e admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:50.981303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35951f0f-6538-47fa-a532-5daa06d0222e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:47:50.981602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3a979-8305-48a0-9a6f-5721d65edef9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:50.981722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35951f0f-6538-47fa-a532-5daa06d0222e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:50.982605 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-35951f0f-6538-47fa-a532-5daa06d0222e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:47:50.982605 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 20:47:50.982605 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 20:47:50.983478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35951f0f-6538-47fa-a532-5daa06d0222e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:50.999388 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 606/1216] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:47:51.005615 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 611/1217] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:47:51.010991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3a979-8305-48a0-9a6f-5721d65edef9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.011529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feba5d0-69f5-4664-a384-49093bde2f22 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.011923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feba5d0-69f5-4664-a384-49093bde2f22 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.012209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feba5d0-69f5-4664-a384-49093bde2f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.026250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3a979-8305-48a0-9a6f-5721d65edef9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.037275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22691c-3dbd-435a-98ad-a200bad03a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.038352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22691c-3dbd-435a-98ad-a200bad03a82 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.038987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22691c-3dbd-435a-98ad-a200bad03a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.039258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22691c-3dbd-435a-98ad-a200bad03a82 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.040499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22691c-3dbd-435a-98ad-a200bad03a82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.042424 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-1c22691c-3dbd-435a-98ad-a200bad03a82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:47:51.042424 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 20:47:51.042424 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 20:47:51.043699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22691c-3dbd-435a-98ad-a200bad03a82 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.047872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0feba5d0-69f5-4664-a384-49093bde2f22 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:51.049076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0feba5d0-69f5-4664-a384-49093bde2f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:51.068590 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 612/1218] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:47:51.069451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feba5d0-69f5-4664-a384-49093bde2f22 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.083621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feba5d0-69f5-4664-a384-49093bde2f22 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.094179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7263bd-7422-4871-aaa4-099483202f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.094826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7263bd-7422-4871-aaa4-099483202f6a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.095109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7263bd-7422-4871-aaa4-099483202f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.095426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7263bd-7422-4871-aaa4-099483202f6a admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.096065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7263bd-7422-4871-aaa4-099483202f6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.098332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7263bd-7422-4871-aaa4-099483202f6a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.111685 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 607/1219] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:51.113073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c496f-4e6f-48d3-a0a5-4f05ab480bf8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.113356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c496f-4e6f-48d3-a0a5-4f05ab480bf8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.114178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c496f-4e6f-48d3-a0a5-4f05ab480bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.151233 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 613/1220] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:51.151920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-451c496f-4e6f-48d3-a0a5-4f05ab480bf8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:51.152122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7263bd-7422-4871-aaa4-099483202f6a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.153746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-451c496f-4e6f-48d3-a0a5-4f05ab480bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:51.291236 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 608/1221] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:51.292763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c496f-4e6f-48d3-a0a5-4f05ab480bf8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.312845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7263bd-7422-4871-aaa4-099483202f6a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.317008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c496f-4e6f-48d3-a0a5-4f05ab480bf8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.322298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ca40e-2a20-4355-9153-23be4037f76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.322945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ca40e-2a20-4355-9153-23be4037f76f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.323165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ca40e-2a20-4355-9153-23be4037f76f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.323427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ca40e-2a20-4355-9153-23be4037f76f admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.324100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ca40e-2a20-4355-9153-23be4037f76f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.325208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da6a72-a4c8-48d4-8f84-3c2b1771a7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.326067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da6a72-a4c8-48d4-8f84-3c2b1771a7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.326558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da6a72-a4c8-48d4-8f84-3c2b1771a7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.327032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da6a72-a4c8-48d4-8f84-3c2b1771a7a4 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.327890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da6a72-a4c8-48d4-8f84-3c2b1771a7a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.329399 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-27da6a72-a4c8-48d4-8f84-3c2b1771a7a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:47:51.329399 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 20:47:51.329399 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 20:47:51.330273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ca40e-2a20-4355-9153-23be4037f76f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.331390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da6a72-a4c8-48d4-8f84-3c2b1771a7a4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.376467 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 614/1222] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:51.376954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da6a72-a4c8-48d4-8f84-3c2b1771a7a4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.382149 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 609/1223] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:51.384539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ca40e-2a20-4355-9153-23be4037f76f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.399621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da6a72-a4c8-48d4-8f84-3c2b1771a7a4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.405778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ca40e-2a20-4355-9153-23be4037f76f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.411072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a090c-5237-49e5-ac7e-322926d5021f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.411750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a090c-5237-49e5-ac7e-322926d5021f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.412017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a090c-5237-49e5-ac7e-322926d5021f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.412285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a090c-5237-49e5-ac7e-322926d5021f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.412995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a090c-5237-49e5-ac7e-322926d5021f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.414181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3006cd1-313d-43f7-8d7c-4d499fd34abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.415550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3006cd1-313d-43f7-8d7c-4d499fd34abf admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.415828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3006cd1-313d-43f7-8d7c-4d499fd34abf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.416226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3006cd1-313d-43f7-8d7c-4d499fd34abf admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.416908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3006cd1-313d-43f7-8d7c-4d499fd34abf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:47:51.417277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3006cd1-313d-43f7-8d7c-4d499fd34abf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.418869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a090c-5237-49e5-ac7e-322926d5021f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.419000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3006cd1-313d-43f7-8d7c-4d499fd34abf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.443318 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 615/1224] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:47:51.444301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3006cd1-313d-43f7-8d7c-4d499fd34abf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.444529 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 610/1225] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:51.450745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a090c-5237-49e5-ac7e-322926d5021f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.470384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3006cd1-313d-43f7-8d7c-4d499fd34abf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.472569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a090c-5237-49e5-ac7e-322926d5021f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.481686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026dd2d-175f-42e3-b6b9-e1251f55d19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.482593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026dd2d-175f-42e3-b6b9-e1251f55d19b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.483081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026dd2d-175f-42e3-b6b9-e1251f55d19b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.483552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026dd2d-175f-42e3-b6b9-e1251f55d19b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.484381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026dd2d-175f-42e3-b6b9-e1251f55d19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.485737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59093f1f-d782-4023-b32e-ca6d48deaf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.486350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59093f1f-d782-4023-b32e-ca6d48deaf33 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.486628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59093f1f-d782-4023-b32e-ca6d48deaf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.486896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59093f1f-d782-4023-b32e-ca6d48deaf33 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.487512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026dd2d-175f-42e3-b6b9-e1251f55d19b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.487630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59093f1f-d782-4023-b32e-ca6d48deaf33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.489142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59093f1f-d782-4023-b32e-ca6d48deaf33 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.504553 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 616/1226] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:51.509980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59093f1f-d782-4023-b32e-ca6d48deaf33 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.535494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59093f1f-d782-4023-b32e-ca6d48deaf33 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.536947 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 611/1227] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:51.541417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026dd2d-175f-42e3-b6b9-e1251f55d19b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.547707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fdd12-7c28-4075-8f29-1d73b194df89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.548388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fdd12-7c28-4075-8f29-1d73b194df89 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.548668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fdd12-7c28-4075-8f29-1d73b194df89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.548934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fdd12-7c28-4075-8f29-1d73b194df89 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.549606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fdd12-7c28-4075-8f29-1d73b194df89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.551613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fdd12-7c28-4075-8f29-1d73b194df89 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.565025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026dd2d-175f-42e3-b6b9-e1251f55d19b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.574453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac087fda-634c-4a9e-9690-4a2007f75bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.575375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac087fda-634c-4a9e-9690-4a2007f75bbf admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.575895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac087fda-634c-4a9e-9690-4a2007f75bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.576370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac087fda-634c-4a9e-9690-4a2007f75bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.577243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac087fda-634c-4a9e-9690-4a2007f75bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.579559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac087fda-634c-4a9e-9690-4a2007f75bbf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.592647 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 612/1228] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:51.593804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac087fda-634c-4a9e-9690-4a2007f75bbf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.613838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac087fda-634c-4a9e-9690-4a2007f75bbf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.626821 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 617/1229] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:51.628703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa547ddb-6889-487c-aa39-f8839481ed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.629177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f32947-0abb-4c31-b6e4-9ae74ef9f738 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.629459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f32947-0abb-4c31-b6e4-9ae74ef9f738 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.629727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f32947-0abb-4c31-b6e4-9ae74ef9f738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.631283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa547ddb-6889-487c-aa39-f8839481ed1a admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.631562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa547ddb-6889-487c-aa39-f8839481ed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.631831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa547ddb-6889-487c-aa39-f8839481ed1a admin admin] PATH_INFO: `/v3/projects/528b4193b251489e91659997e5723d08/users/2c81e608ec934eb0ade4927aa7347dc4/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.660755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa547ddb-6889-487c-aa39-f8839481ed1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=528b4193b251489e91659997e5723d08, user_id=2c81e608ec934eb0ade4927aa7347dc4, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.663527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa547ddb-6889-487c-aa39-f8839481ed1a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.669322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-36f32947-0abb-4c31-b6e4-9ae74ef9f738 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:51.670532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-36f32947-0abb-4c31-b6e4-9ae74ef9f738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:51.683436 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 613/1230] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:51 2022] PUT /identity/v3/projects/528b4193b251489e91659997e5723d08/users/2c81e608ec934eb0ade4927aa7347dc4/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:51.684676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db925c-0ff1-43d2-977e-b1fda95d57ad None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.684951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db925c-0ff1-43d2-977e-b1fda95d57ad None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.686498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db925c-0ff1-43d2-977e-b1fda95d57ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.721865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-24db925c-0ff1-43d2-977e-b1fda95d57ad None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:51.723203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-24db925c-0ff1-43d2-977e-b1fda95d57ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:51.731134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:51.749230 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 618/1231] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:51.750329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f32947-0abb-4c31-b6e4-9ae74ef9f738 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.756469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:51.760791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f32947-0abb-4c31-b6e4-9ae74ef9f738 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.765899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e20a4-df49-4ddb-8dfc-b5867d795bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.766542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e20a4-df49-4ddb-8dfc-b5867d795bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.766810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e20a4-df49-4ddb-8dfc-b5867d795bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.767071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e20a4-df49-4ddb-8dfc-b5867d795bd9 admin admin] PATH_INFO: `/v3/projects/e9404d6e1de94dabb849df1ecd92e083/users/55dd0c9b11ed40ebb292b03ad57078bb/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.782322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e20a4-df49-4ddb-8dfc-b5867d795bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9404d6e1de94dabb849df1ecd92e083, user_id=55dd0c9b11ed40ebb292b03ad57078bb, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.785384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e20a4-df49-4ddb-8dfc-b5867d795bd9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.791150 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 614/1232] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:51.792725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b7331-f23d-45df-9774-707fda17e634 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.793027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b7331-f23d-45df-9774-707fda17e634 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.793318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b7331-f23d-45df-9774-707fda17e634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.810820 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 619/1233] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:51 2022] PUT /identity/v3/projects/e9404d6e1de94dabb849df1ecd92e083/users/55dd0c9b11ed40ebb292b03ad57078bb/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:51.811608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e20a4-df49-4ddb-8dfc-b5867d795bd9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:51.813666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-476b7331-f23d-45df-9774-707fda17e634 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:51.814850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-476b7331-f23d-45df-9774-707fda17e634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:51.876931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e20a4-df49-4ddb-8dfc-b5867d795bd9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:51.878402 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 615/1234] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:51.879720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3891b2-1d80-4e05-a067-c57425282062 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.880079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3891b2-1d80-4e05-a067-c57425282062 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.880467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3891b2-1d80-4e05-a067-c57425282062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.882473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934fa60-29e7-449b-aa16-16f140a4e5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:51.883157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934fa60-29e7-449b-aa16-16f140a4e5bd admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.883518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934fa60-29e7-449b-aa16-16f140a4e5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.883874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934fa60-29e7-449b-aa16-16f140a4e5bd admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.891595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934fa60-29e7-449b-aa16-16f140a4e5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:51.891595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934fa60-29e7-449b-aa16-16f140a4e5bd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:51.909214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 620/1235] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:51.909441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3c3891b2-1d80-4e05-a067-c57425282062 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:51.910590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c22ba-67c7-44f0-84c9-2f23c81f320d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.910707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3c3891b2-1d80-4e05-a067-c57425282062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:51.910981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c22ba-67c7-44f0-84c9-2f23c81f320d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.911316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c22ba-67c7-44f0-84c9-2f23c81f320d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.920379 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 616/1236] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:51.921666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b3f50-ede7-4e5e-88bc-154f0cd903d9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.921952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b3f50-ede7-4e5e-88bc-154f0cd903d9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.922222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b3f50-ede7-4e5e-88bc-154f0cd903d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.941770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f71c22ba-67c7-44f0-84c9-2f23c81f320d None None] MFA Rules not processed for user `2c81e608ec934eb0ade4927aa7347dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:51.942983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f71c22ba-67c7-44f0-84c9-2f23c81f320d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:51.943095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e85b3f50-ede7-4e5e-88bc-154f0cd903d9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:51.944147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e85b3f50-ede7-4e5e-88bc-154f0cd903d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:51.952273 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 617/1237] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:51.953555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e659600-e260-4d65-abb8-e9171adb5d17 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.953837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e659600-e260-4d65-abb8-e9171adb5d17 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.954105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e659600-e260-4d65-abb8-e9171adb5d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:51.974612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5e659600-e260-4d65-abb8-e9171adb5d17 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:51.976048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5e659600-e260-4d65-abb8-e9171adb5d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:51.984816 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 618/1238] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:51.986019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf5e75-7ef8-474c-9422-6f1bcf05a5de None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:51.986290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf5e75-7ef8-474c-9422-6f1bcf05a5de None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:51.986551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf5e75-7ef8-474c-9422-6f1bcf05a5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.005058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b6cf5e75-7ef8-474c-9422-6f1bcf05a5de None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:52.006248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b6cf5e75-7ef8-474c-9422-6f1bcf05a5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:52.015219 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 619/1239] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:52.016257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a0bb-1436-4c10-9e92-994628ec8f5c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.016529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a0bb-1436-4c10-9e92-994628ec8f5c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.016790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a0bb-1436-4c10-9e92-994628ec8f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.020801 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 621/1240] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:52.022136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53addf06-59dd-4101-a7ed-3cb23654ae7d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.022426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53addf06-59dd-4101-a7ed-3cb23654ae7d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.022769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53addf06-59dd-4101-a7ed-3cb23654ae7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.042418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-53addf06-59dd-4101-a7ed-3cb23654ae7d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:52.043702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-53addf06-59dd-4101-a7ed-3cb23654ae7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:52.045155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b9b1a0bb-1436-4c10-9e92-994628ec8f5c None None] MFA Rules not processed for user `55dd0c9b11ed40ebb292b03ad57078bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:52.046317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9b1a0bb-1436-4c10-9e92-994628ec8f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:52.051730 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 622/1241] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:52.052511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53addf06-59dd-4101-a7ed-3cb23654ae7d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.061989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53addf06-59dd-4101-a7ed-3cb23654ae7d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.066399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051b0b3-31c2-4471-a194-77d11434a5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.066980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051b0b3-31c2-4471-a194-77d11434a5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.067247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051b0b3-31c2-4471-a194-77d11434a5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.067509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051b0b3-31c2-4471-a194-77d11434a5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.068130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051b0b3-31c2-4471-a194-77d11434a5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.069762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051b0b3-31c2-4471-a194-77d11434a5e9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.082871 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 623/1242] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:52.082871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb804-b498-4b87-bea1-2e801b96e7d2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.082871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb804-b498-4b87-bea1-2e801b96e7d2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.082871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb804-b498-4b87-bea1-2e801b96e7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.102272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-52cdb804-b498-4b87-bea1-2e801b96e7d2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:52.103529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-52cdb804-b498-4b87-bea1-2e801b96e7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:52.111311 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 624/1243] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:52.112468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db4f73-47f3-4f71-8b6a-203eeefaa328 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.112748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db4f73-47f3-4f71-8b6a-203eeefaa328 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.112992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db4f73-47f3-4f71-8b6a-203eeefaa328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.115319 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 620/1244] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:52.116143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1a0bb-1436-4c10-9e92-994628ec8f5c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.127960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1a0bb-1436-4c10-9e92-994628ec8f5c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.132443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974b354-0515-46b4-aad4-133b75c4de49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.133019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974b354-0515-46b4-aad4-133b75c4de49 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.133841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974b354-0515-46b4-aad4-133b75c4de49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.134112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974b354-0515-46b4-aad4-133b75c4de49 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.134880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974b354-0515-46b4-aad4-133b75c4de49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:47:52.135177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974b354-0515-46b4-aad4-133b75c4de49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.135790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-33db4f73-47f3-4f71-8b6a-203eeefaa328 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:52.136762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974b354-0515-46b4-aad4-133b75c4de49 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.136981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-33db4f73-47f3-4f71-8b6a-203eeefaa328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:52.145002 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 625/1245] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:52.145793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db4f73-47f3-4f71-8b6a-203eeefaa328 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.148675 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 621/1246] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:47:52.149549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974b354-0515-46b4-aad4-133b75c4de49 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.159559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974b354-0515-46b4-aad4-133b75c4de49 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.160393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db4f73-47f3-4f71-8b6a-203eeefaa328 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.164131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315b9cb-ffc2-4175-b803-071896ecda0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.165265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b9cb-ffc2-4175-b803-071896ecda0a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.165542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b9cb-ffc2-4175-b803-071896ecda0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.165811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b9cb-ffc2-4175-b803-071896ecda0a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.165934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396edc3f-869c-45ad-bb0c-c65bd757325d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.166443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396edc3f-869c-45ad-bb0c-c65bd757325d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.166554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315b9cb-ffc2-4175-b803-071896ecda0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.166719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396edc3f-869c-45ad-bb0c-c65bd757325d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.166987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396edc3f-869c-45ad-bb0c-c65bd757325d admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.167652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396edc3f-869c-45ad-bb0c-c65bd757325d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:47:52.167955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396edc3f-869c-45ad-bb0c-c65bd757325d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.168330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315b9cb-ffc2-4175-b803-071896ecda0a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.169547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396edc3f-869c-45ad-bb0c-c65bd757325d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.178603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:52.183422 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 626/1247] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:47:52.185290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396edc3f-869c-45ad-bb0c-c65bd757325d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.196776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396edc3f-869c-45ad-bb0c-c65bd757325d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.197711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:52.203291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c563f7a-7714-4e7e-a6ef-307b6da24ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.203912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c563f7a-7714-4e7e-a6ef-307b6da24ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.204184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c563f7a-7714-4e7e-a6ef-307b6da24ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.204445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c563f7a-7714-4e7e-a6ef-307b6da24ee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.205182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c563f7a-7714-4e7e-a6ef-307b6da24ee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:47:52.205565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c563f7a-7714-4e7e-a6ef-307b6da24ee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.207216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c563f7a-7714-4e7e-a6ef-307b6da24ee5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.209802 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 622/1248] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.210732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315b9cb-ffc2-4175-b803-071896ecda0a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.220314 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 627/1249] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:47:52.221150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c563f7a-7714-4e7e-a6ef-307b6da24ee5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.231506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c563f7a-7714-4e7e-a6ef-307b6da24ee5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.278258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315b9cb-ffc2-4175-b803-071896ecda0a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.283230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186a303-8d96-4d8a-9410-7699267fc5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.283823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186a303-8d96-4d8a-9410-7699267fc5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.284096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186a303-8d96-4d8a-9410-7699267fc5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.284361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186a303-8d96-4d8a-9410-7699267fc5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.284990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186a303-8d96-4d8a-9410-7699267fc5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.286627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186a303-8d96-4d8a-9410-7699267fc5d0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.288942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ecae8-d7b3-4317-90d6-8d9cdb968be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.290125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ecae8-d7b3-4317-90d6-8d9cdb968be5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.290388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ecae8-d7b3-4317-90d6-8d9cdb968be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.290649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ecae8-d7b3-4317-90d6-8d9cdb968be5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.291278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ecae8-d7b3-4317-90d6-8d9cdb968be5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:47:52.291568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ecae8-d7b3-4317-90d6-8d9cdb968be5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.293204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ecae8-d7b3-4317-90d6-8d9cdb968be5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.297149 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 623/1250] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:52.297971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186a303-8d96-4d8a-9410-7699267fc5d0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.312430 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 628/1251] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:47:52.312430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ecae8-d7b3-4317-90d6-8d9cdb968be5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.315065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186a303-8d96-4d8a-9410-7699267fc5d0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.315506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ecae8-d7b3-4317-90d6-8d9cdb968be5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.319822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b7783-ba99-49a5-8351-b508bbd90af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.320333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b99216-375a-45ca-8b5b-f5ae17773141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.320569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b7783-ba99-49a5-8351-b508bbd90af9 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.321439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b99216-375a-45ca-8b5b-f5ae17773141 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.321569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b7783-ba99-49a5-8351-b508bbd90af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.321690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b99216-375a-45ca-8b5b-f5ae17773141 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.321925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b7783-ba99-49a5-8351-b508bbd90af9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.322032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b99216-375a-45ca-8b5b-f5ae17773141 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.322670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b99216-375a-45ca-8b5b-f5ae17773141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:47:52.322774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b7783-ba99-49a5-8351-b508bbd90af9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.322965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b99216-375a-45ca-8b5b-f5ae17773141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.324517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b99216-375a-45ca-8b5b-f5ae17773141 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.324964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b7783-ba99-49a5-8351-b508bbd90af9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.338207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 629/1252] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:47:52.338995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b99216-375a-45ca-8b5b-f5ae17773141 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.351303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b99216-375a-45ca-8b5b-f5ae17773141 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.355892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f00ea2-057a-4b14-8001-b8d974d8a661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.356224 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 624/1253] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.356480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f00ea2-057a-4b14-8001-b8d974d8a661 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.356736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f00ea2-057a-4b14-8001-b8d974d8a661 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.356996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f00ea2-057a-4b14-8001-b8d974d8a661 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.357249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b7783-ba99-49a5-8351-b508bbd90af9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.357637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f00ea2-057a-4b14-8001-b8d974d8a661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.359779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f00ea2-057a-4b14-8001-b8d974d8a661 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.387564 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 630/1254] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.388299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f00ea2-057a-4b14-8001-b8d974d8a661 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.422599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b7783-ba99-49a5-8351-b508bbd90af9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.451109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f00ea2-057a-4b14-8001-b8d974d8a661 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.456301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5669289-9ca1-47b3-8022-d96a34479fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.456891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5669289-9ca1-47b3-8022-d96a34479fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.457201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5669289-9ca1-47b3-8022-d96a34479fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.457498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5669289-9ca1-47b3-8022-d96a34479fa6 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.458138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5669289-9ca1-47b3-8022-d96a34479fa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.460037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5669289-9ca1-47b3-8022-d96a34479fa6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.484285 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 631/1255] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.484285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5669289-9ca1-47b3-8022-d96a34479fa6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.484587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60df1a-e1a8-43ef-a2e7-61b7594596eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.484587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60df1a-e1a8-43ef-a2e7-61b7594596eb admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.484587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60df1a-e1a8-43ef-a2e7-61b7594596eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.484867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60df1a-e1a8-43ef-a2e7-61b7594596eb admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.485275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60df1a-e1a8-43ef-a2e7-61b7594596eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.487186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60df1a-e1a8-43ef-a2e7-61b7594596eb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.493403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5669289-9ca1-47b3-8022-d96a34479fa6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.498184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a2059-4209-48d7-8ff1-018ebfbf7b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.498880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a2059-4209-48d7-8ff1-018ebfbf7b27 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.499160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a2059-4209-48d7-8ff1-018ebfbf7b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.499433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a2059-4209-48d7-8ff1-018ebfbf7b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.500049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a2059-4209-48d7-8ff1-018ebfbf7b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.501952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a2059-4209-48d7-8ff1-018ebfbf7b27 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.516732 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 625/1256] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.517455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60df1a-e1a8-43ef-a2e7-61b7594596eb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.522969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:52.536085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 632/1257] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.536085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a2059-4209-48d7-8ff1-018ebfbf7b27 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.547217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:52.583732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60df1a-e1a8-43ef-a2e7-61b7594596eb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.598583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a2059-4209-48d7-8ff1-018ebfbf7b27 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.603875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9595d-d248-4479-9ec5-b707e8b279b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.604756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9595d-d248-4479-9ec5-b707e8b279b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.605027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9595d-d248-4479-9ec5-b707e8b279b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.605320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9595d-d248-4479-9ec5-b707e8b279b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.605931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9595d-d248-4479-9ec5-b707e8b279b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.607832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9595d-d248-4479-9ec5-b707e8b279b1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.637204 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 633/1258] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.638402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9595d-d248-4479-9ec5-b707e8b279b1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.645246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07260d-01a2-4630-b2ec-ec69a9d1699a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.646344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07260d-01a2-4630-b2ec-ec69a9d1699a admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.646344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07260d-01a2-4630-b2ec-ec69a9d1699a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.646553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07260d-01a2-4630-b2ec-ec69a9d1699a admin admin] PATH_INFO: `/v3/projects/b002a399b040452ba66910f5961ad6de/users/5dd7ec5d407f4a0fbfd5593140e0bb0e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.661562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07260d-01a2-4630-b2ec-ec69a9d1699a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b002a399b040452ba66910f5961ad6de, user_id=5dd7ec5d407f4a0fbfd5593140e0bb0e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.664237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07260d-01a2-4630-b2ec-ec69a9d1699a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.679778 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 626/1259] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:52 2022] PUT /identity/v3/projects/b002a399b040452ba66910f5961ad6de/users/5dd7ec5d407f4a0fbfd5593140e0bb0e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:52.680693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07260d-01a2-4630-b2ec-ec69a9d1699a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.707412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9595d-d248-4479-9ec5-b707e8b279b1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.744635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07260d-01a2-4630-b2ec-ec69a9d1699a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.749273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21943d54-33e3-4e1c-80ef-ddf4ef0aafed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.749859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21943d54-33e3-4e1c-80ef-ddf4ef0aafed admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.750129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21943d54-33e3-4e1c-80ef-ddf4ef0aafed admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.750389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21943d54-33e3-4e1c-80ef-ddf4ef0aafed admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.750993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21943d54-33e3-4e1c-80ef-ddf4ef0aafed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.752904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21943d54-33e3-4e1c-80ef-ddf4ef0aafed admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.764941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2379873-203a-4dbb-8eb6-9ef97742035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.765705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2379873-203a-4dbb-8eb6-9ef97742035e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.765974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2379873-203a-4dbb-8eb6-9ef97742035e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.766292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2379873-203a-4dbb-8eb6-9ef97742035e admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.766961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2379873-203a-4dbb-8eb6-9ef97742035e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.768969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2379873-203a-4dbb-8eb6-9ef97742035e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.778774 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 627/1260] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.778774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21943d54-33e3-4e1c-80ef-ddf4ef0aafed admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.789379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21943d54-33e3-4e1c-80ef-ddf4ef0aafed admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.794945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268db457-cc41-41dc-9d8c-905b90299fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.795817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268db457-cc41-41dc-9d8c-905b90299fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.796096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268db457-cc41-41dc-9d8c-905b90299fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.796355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268db457-cc41-41dc-9d8c-905b90299fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.796968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268db457-cc41-41dc-9d8c-905b90299fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.801576 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 634/1261] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/projects => generated 397 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.801708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268db457-cc41-41dc-9d8c-905b90299fc1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.801891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2379873-203a-4dbb-8eb6-9ef97742035e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.825881 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 628/1262] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/users => generated 289 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.826780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268db457-cc41-41dc-9d8c-905b90299fc1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.838359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268db457-cc41-41dc-9d8c-905b90299fc1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.843599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:52.846329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:52.879380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2379873-203a-4dbb-8eb6-9ef97742035e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.884519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b346cd-7f3b-48ea-b1a4-f704fed79c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.885160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b346cd-7f3b-48ea-b1a4-f704fed79c87 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.885439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b346cd-7f3b-48ea-b1a4-f704fed79c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.885707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b346cd-7f3b-48ea-b1a4-f704fed79c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.886342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b346cd-7f3b-48ea-b1a4-f704fed79c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.887917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b346cd-7f3b-48ea-b1a4-f704fed79c87 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.898172 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 635/1263] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:52.899763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b346cd-7f3b-48ea-b1a4-f704fed79c87 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.902357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1b8cc-75dc-4a8f-a983-f62ec884bf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.902982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1b8cc-75dc-4a8f-a983-f62ec884bf12 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.903249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1b8cc-75dc-4a8f-a983-f62ec884bf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.903511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1b8cc-75dc-4a8f-a983-f62ec884bf12 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.904148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1b8cc-75dc-4a8f-a983-f62ec884bf12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.906595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1b8cc-75dc-4a8f-a983-f62ec884bf12 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.909746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b346cd-7f3b-48ea-b1a4-f704fed79c87 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.914801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5729c89-8286-4f8c-bd26-07e1e959cf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.915375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5729c89-8286-4f8c-bd26-07e1e959cf6a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.915658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5729c89-8286-4f8c-bd26-07e1e959cf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.915922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5729c89-8286-4f8c-bd26-07e1e959cf6a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.916536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5729c89-8286-4f8c-bd26-07e1e959cf6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.918763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5729c89-8286-4f8c-bd26-07e1e959cf6a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.929477 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 629/1264] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.930323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1b8cc-75dc-4a8f-a983-f62ec884bf12 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.941457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1b8cc-75dc-4a8f-a983-f62ec884bf12 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.945053 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 636/1265] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/users => generated 302 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.945053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9eae9-4362-4aaa-93a0-ec2b42e247c7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.945053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9eae9-4362-4aaa-93a0-ec2b42e247c7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.945053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9eae9-4362-4aaa-93a0-ec2b42e247c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.947071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff249e-2712-49cb-b759-a89ec9cf9739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.947701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff249e-2712-49cb-b759-a89ec9cf9739 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.947977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff249e-2712-49cb-b759-a89ec9cf9739 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.948241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff249e-2712-49cb-b759-a89ec9cf9739 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.948850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff249e-2712-49cb-b759-a89ec9cf9739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.950814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff249e-2712-49cb-b759-a89ec9cf9739 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:52.972195 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 630/1266] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:52.972566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-dfe9eae9-4362-4aaa-93a0-ec2b42e247c7 None None] MFA Rules not processed for user `5dd7ec5d407f4a0fbfd5593140e0bb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:52.973022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff249e-2712-49cb-b759-a89ec9cf9739 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:52.973876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dfe9eae9-4362-4aaa-93a0-ec2b42e247c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:52.984139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff249e-2712-49cb-b759-a89ec9cf9739 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:52.988880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c49195-4768-43b1-a27c-725bef0718a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:52.989514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c49195-4768-43b1-a27c-725bef0718a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:52.989790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c49195-4768-43b1-a27c-725bef0718a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:52.990058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c49195-4768-43b1-a27c-725bef0718a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:52.990693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c49195-4768-43b1-a27c-725bef0718a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:52.992271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c49195-4768-43b1-a27c-725bef0718a7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.006526 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 631/1267] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.006526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c49195-4768-43b1-a27c-725bef0718a7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.014103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c49195-4768-43b1-a27c-725bef0718a7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.018922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f48f8-3c9b-40b9-8f88-6f6979edfc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.019494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f48f8-3c9b-40b9-8f88-6f6979edfc90 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.019771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f48f8-3c9b-40b9-8f88-6f6979edfc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.020037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f48f8-3c9b-40b9-8f88-6f6979edfc90 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.020647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f48f8-3c9b-40b9-8f88-6f6979edfc90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.022582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f48f8-3c9b-40b9-8f88-6f6979edfc90 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.044640 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 632/1268] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:53 2022] POST /identity/v3/users => generated 299 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:53.044980 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 637/1269] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:53.045311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f48f8-3c9b-40b9-8f88-6f6979edfc90 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.045823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9eae9-4362-4aaa-93a0-ec2b42e247c7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.055004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f48f8-3c9b-40b9-8f88-6f6979edfc90 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.060243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd0247-50a5-4277-adbe-d0e7cc916d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.060791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9eae9-4362-4aaa-93a0-ec2b42e247c7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.060915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd0247-50a5-4277-adbe-d0e7cc916d4e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.061113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd0247-50a5-4277-adbe-d0e7cc916d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.061422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd0247-50a5-4277-adbe-d0e7cc916d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.062000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd0247-50a5-4277-adbe-d0e7cc916d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.063592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd0247-50a5-4277-adbe-d0e7cc916d4e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.065530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dce4e-ffd3-4756-aadb-d31080d726a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.066196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dce4e-ffd3-4756-aadb-d31080d726a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.066499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dce4e-ffd3-4756-aadb-d31080d726a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.066763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dce4e-ffd3-4756-aadb-d31080d726a5 admin admin] PATH_INFO: `/v3/projects/3c51574deb904dce89e171e4f52ab665/users/110fc0e5be3141caa029822a601ff2ed/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.073522 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 633/1270] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.074344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd0247-50a5-4277-adbe-d0e7cc916d4e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.081484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1dce4e-ffd3-4756-aadb-d31080d726a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c51574deb904dce89e171e4f52ab665, user_id=110fc0e5be3141caa029822a601ff2ed, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.084142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1dce4e-ffd3-4756-aadb-d31080d726a5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.084502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd0247-50a5-4277-adbe-d0e7cc916d4e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.089661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a07597-5ff0-46a7-bb43-c6742e7a5605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.090384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a07597-5ff0-46a7-bb43-c6742e7a5605 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.090719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a07597-5ff0-46a7-bb43-c6742e7a5605 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.091050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a07597-5ff0-46a7-bb43-c6742e7a5605 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.091760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a07597-5ff0-46a7-bb43-c6742e7a5605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.093835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a07597-5ff0-46a7-bb43-c6742e7a5605 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.098103 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 638/1271] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:53 2022] PUT /identity/v3/projects/3c51574deb904dce89e171e4f52ab665/users/110fc0e5be3141caa029822a601ff2ed/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:53.101473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dce4e-ffd3-4756-aadb-d31080d726a5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.110578 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 634/1272] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.110578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a07597-5ff0-46a7-bb43-c6742e7a5605 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.118333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a07597-5ff0-46a7-bb43-c6742e7a5605 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.163823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dce4e-ffd3-4756-aadb-d31080d726a5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.169391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2a0ad-fc5b-4c29-b910-11e2b56def6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.169985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2a0ad-fc5b-4c29-b910-11e2b56def6e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.170257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2a0ad-fc5b-4c29-b910-11e2b56def6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.170527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2a0ad-fc5b-4c29-b910-11e2b56def6e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.171148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2a0ad-fc5b-4c29-b910-11e2b56def6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.172729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2a0ad-fc5b-4c29-b910-11e2b56def6e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.174694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac4433-a3c8-4b1a-8cbf-99a897f4de8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.175275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac4433-a3c8-4b1a-8cbf-99a897f4de8e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.175596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac4433-a3c8-4b1a-8cbf-99a897f4de8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.175866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac4433-a3c8-4b1a-8cbf-99a897f4de8e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.176492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac4433-a3c8-4b1a-8cbf-99a897f4de8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.178106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac4433-a3c8-4b1a-8cbf-99a897f4de8e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.183388 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 639/1273] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.184376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2a0ad-fc5b-4c29-b910-11e2b56def6e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.190141 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 635/1274] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.191550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac4433-a3c8-4b1a-8cbf-99a897f4de8e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.194859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2a0ad-fc5b-4c29-b910-11e2b56def6e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.202702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50a4ed-3acd-4801-bf27-772e7ff86f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.203148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac4433-a3c8-4b1a-8cbf-99a897f4de8e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.203350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a4ed-3acd-4801-bf27-772e7ff86f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.203624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a4ed-3acd-4801-bf27-772e7ff86f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.203888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a4ed-3acd-4801-bf27-772e7ff86f6c admin admin] PATH_INFO: `/v3/projects/bfde48d2eac54169b872ab50c329039e/users/5dc2be35c23442d29f8cae35e6728810/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.208997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86930692-7cd4-4c4d-840a-898ffb766fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.209603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86930692-7cd4-4c4d-840a-898ffb766fdb admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.209977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86930692-7cd4-4c4d-840a-898ffb766fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.210374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86930692-7cd4-4c4d-840a-898ffb766fdb admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.211099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86930692-7cd4-4c4d-840a-898ffb766fdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.213866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86930692-7cd4-4c4d-840a-898ffb766fdb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.214221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:53.221742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50a4ed-3acd-4801-bf27-772e7ff86f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfde48d2eac54169b872ab50c329039e, user_id=5dc2be35c23442d29f8cae35e6728810, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.224811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50a4ed-3acd-4801-bf27-772e7ff86f6c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.240320 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 640/1275] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:53 2022] PUT /identity/v3/projects/bfde48d2eac54169b872ab50c329039e/users/5dc2be35c23442d29f8cae35e6728810/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:53.243481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50a4ed-3acd-4801-bf27-772e7ff86f6c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.246904 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 636/1276] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:53 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:53.246904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86930692-7cd4-4c4d-840a-898ffb766fdb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.250222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:53.306653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50a4ed-3acd-4801-bf27-772e7ff86f6c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.310898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86930692-7cd4-4c4d-840a-898ffb766fdb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.311542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d1a42-be9a-4018-a510-277233c7caf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.312291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d1a42-be9a-4018-a510-277233c7caf8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.312561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d1a42-be9a-4018-a510-277233c7caf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.312847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d1a42-be9a-4018-a510-277233c7caf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.313528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d1a42-be9a-4018-a510-277233c7caf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.315204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d1a42-be9a-4018-a510-277233c7caf8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.315616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d77aec-acf9-42b1-a701-6a1e2109e9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.316208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d77aec-acf9-42b1-a701-6a1e2109e9bb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.316483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d77aec-acf9-42b1-a701-6a1e2109e9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.316752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d77aec-acf9-42b1-a701-6a1e2109e9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.317407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d77aec-acf9-42b1-a701-6a1e2109e9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.318998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d77aec-acf9-42b1-a701-6a1e2109e9bb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.326993 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 641/1277] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.327643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d1a42-be9a-4018-a510-277233c7caf8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.330426 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 637/1278] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.333812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d77aec-acf9-42b1-a701-6a1e2109e9bb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.337517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d1a42-be9a-4018-a510-277233c7caf8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.342364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92218c-258c-44e4-bd31-43b26db3519e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.343000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92218c-258c-44e4-bd31-43b26db3519e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.343411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92218c-258c-44e4-bd31-43b26db3519e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.343678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92218c-258c-44e4-bd31-43b26db3519e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.343776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d77aec-acf9-42b1-a701-6a1e2109e9bb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.344309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92218c-258c-44e4-bd31-43b26db3519e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.346066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92218c-258c-44e4-bd31-43b26db3519e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.348254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba7f6-2d6e-485e-934c-acc22cdf4b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.348821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba7f6-2d6e-485e-934c-acc22cdf4b83 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.349121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba7f6-2d6e-485e-934c-acc22cdf4b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.349388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba7f6-2d6e-485e-934c-acc22cdf4b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.350007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ba7f6-2d6e-485e-934c-acc22cdf4b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.351831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ba7f6-2d6e-485e-934c-acc22cdf4b83 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.357231 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 642/1279] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.358094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92218c-258c-44e4-bd31-43b26db3519e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.365020 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 638/1280] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.365770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba7f6-2d6e-485e-934c-acc22cdf4b83 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.371315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92218c-258c-44e4-bd31-43b26db3519e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.375869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359b950-4939-423a-85b6-bdbc8fc43c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.376484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359b950-4939-423a-85b6-bdbc8fc43c04 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.376759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359b950-4939-423a-85b6-bdbc8fc43c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.377022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359b950-4939-423a-85b6-bdbc8fc43c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.377303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba7f6-2d6e-485e-934c-acc22cdf4b83 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.377723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359b950-4939-423a-85b6-bdbc8fc43c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.379334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359b950-4939-423a-85b6-bdbc8fc43c04 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.382157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a07cc-e2e9-46e5-90f0-2600ec13b5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.382808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a07cc-e2e9-46e5-90f0-2600ec13b5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.383230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a07cc-e2e9-46e5-90f0-2600ec13b5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.384070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a07cc-e2e9-46e5-90f0-2600ec13b5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.384932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a07cc-e2e9-46e5-90f0-2600ec13b5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.386662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a07cc-e2e9-46e5-90f0-2600ec13b5c3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.390963 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 643/1281] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.393967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359b950-4939-423a-85b6-bdbc8fc43c04 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.397504 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 639/1282] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.398324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a07cc-e2e9-46e5-90f0-2600ec13b5c3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.404561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359b950-4939-423a-85b6-bdbc8fc43c04 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.407737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a07cc-e2e9-46e5-90f0-2600ec13b5c3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.409494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803c2da-9146-4966-b85f-da39df18ea22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.410073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803c2da-9146-4966-b85f-da39df18ea22 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.410351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803c2da-9146-4966-b85f-da39df18ea22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.410666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803c2da-9146-4966-b85f-da39df18ea22 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.411294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803c2da-9146-4966-b85f-da39df18ea22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.412412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcfee2-3517-4823-afc8-559bf98f330b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.412851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803c2da-9146-4966-b85f-da39df18ea22 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.412969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcfee2-3517-4823-afc8-559bf98f330b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.413255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcfee2-3517-4823-afc8-559bf98f330b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.413515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcfee2-3517-4823-afc8-559bf98f330b admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.414111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fcfee2-3517-4823-afc8-559bf98f330b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.417089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fcfee2-3517-4823-afc8-559bf98f330b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.423339 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 644/1283] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.424166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803c2da-9146-4966-b85f-da39df18ea22 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.438887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803c2da-9146-4966-b85f-da39df18ea22 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.440204 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 640/1284] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:53 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:53.440995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcfee2-3517-4823-afc8-559bf98f330b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.446869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea788f78-dcf7-4faf-a782-7b329c056c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.447460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea788f78-dcf7-4faf-a782-7b329c056c73 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.447731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea788f78-dcf7-4faf-a782-7b329c056c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.447996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea788f78-dcf7-4faf-a782-7b329c056c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.448707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea788f78-dcf7-4faf-a782-7b329c056c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.450247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea788f78-dcf7-4faf-a782-7b329c056c73 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.450969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcfee2-3517-4823-afc8-559bf98f330b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.455581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b8d90-a066-434b-82fb-0897271e96fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.456251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b8d90-a066-434b-82fb-0897271e96fa admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.456517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b8d90-a066-434b-82fb-0897271e96fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.456786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b8d90-a066-434b-82fb-0897271e96fa admin admin] PATH_INFO: `/v3/projects/732d361e9c7041cea306ecc4805ffdaf/users/6b5a19d928dc47e48486a58359778635/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.460703 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 645/1285] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.461527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea788f78-dcf7-4faf-a782-7b329c056c73 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.470899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea788f78-dcf7-4faf-a782-7b329c056c73 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.471225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b8d90-a066-434b-82fb-0897271e96fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=732d361e9c7041cea306ecc4805ffdaf, user_id=6b5a19d928dc47e48486a58359778635, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.473945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b8d90-a066-434b-82fb-0897271e96fa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.475837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319b306-b8d1-4dce-a0c3-2e59b63dd925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.476461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319b306-b8d1-4dce-a0c3-2e59b63dd925 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.476732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319b306-b8d1-4dce-a0c3-2e59b63dd925 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.477082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319b306-b8d1-4dce-a0c3-2e59b63dd925 admin admin] PATH_INFO: `/v3/projects/3c51574deb904dce89e171e4f52ab665/users/110fc0e5be3141caa029822a601ff2ed/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.479039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319b306-b8d1-4dce-a0c3-2e59b63dd925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c51574deb904dce89e171e4f52ab665, user_id=110fc0e5be3141caa029822a601ff2ed, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.482214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319b306-b8d1-4dce-a0c3-2e59b63dd925 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.487841 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 641/1286] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:53 2022] PUT /identity/v3/projects/732d361e9c7041cea306ecc4805ffdaf/users/6b5a19d928dc47e48486a58359778635/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:53.489012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b8d90-a066-434b-82fb-0897271e96fa admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.501538 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 646/1287] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:53 2022] PUT /identity/v3/projects/3c51574deb904dce89e171e4f52ab665/users/110fc0e5be3141caa029822a601ff2ed/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:53.506551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319b306-b8d1-4dce-a0c3-2e59b63dd925 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.553738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b8d90-a066-434b-82fb-0897271e96fa admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.569049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319b306-b8d1-4dce-a0c3-2e59b63dd925 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.574162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbad48-9920-4c63-92c8-3f4a7c481972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.574803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbad48-9920-4c63-92c8-3f4a7c481972 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.575073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbad48-9920-4c63-92c8-3f4a7c481972 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.575335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbad48-9920-4c63-92c8-3f4a7c481972 admin admin] PATH_INFO: `/v3/projects/f856d315b3b84bdcac94b6d300ef8b2b/users/14bd717b05ce4e40a1cd54c2db8474a6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.589990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbad48-9920-4c63-92c8-3f4a7c481972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f856d315b3b84bdcac94b6d300ef8b2b, user_id=14bd717b05ce4e40a1cd54c2db8474a6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.592673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbad48-9920-4c63-92c8-3f4a7c481972 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.596389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:53.607748 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 647/1288] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:53 2022] PUT /identity/v3/projects/f856d315b3b84bdcac94b6d300ef8b2b/users/14bd717b05ce4e40a1cd54c2db8474a6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:53.608315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbad48-9920-4c63-92c8-3f4a7c481972 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.618457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:53.620964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7096131-41f2-4cf4-ba39-d96247073251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.621684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7096131-41f2-4cf4-ba39-d96247073251 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.621959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7096131-41f2-4cf4-ba39-d96247073251 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.622221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7096131-41f2-4cf4-ba39-d96247073251 admin admin] PATH_INFO: `/v3/projects/e442345a419943b3809728546377b3d1/users/5c46bbe8072c44beb2ea45c0a3fe845e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.636775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7096131-41f2-4cf4-ba39-d96247073251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e442345a419943b3809728546377b3d1, user_id=5c46bbe8072c44beb2ea45c0a3fe845e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.639762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7096131-41f2-4cf4-ba39-d96247073251 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.652670 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 642/1289] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:53 2022] PUT /identity/v3/projects/e442345a419943b3809728546377b3d1/users/5c46bbe8072c44beb2ea45c0a3fe845e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:53.653481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7096131-41f2-4cf4-ba39-d96247073251 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.678716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbad48-9920-4c63-92c8-3f4a7c481972 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.721192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7096131-41f2-4cf4-ba39-d96247073251 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.727509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059be63a-b4a7-488c-a87c-c6a4ab146355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.728452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059be63a-b4a7-488c-a87c-c6a4ab146355 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.728714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059be63a-b4a7-488c-a87c-c6a4ab146355 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.728985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059be63a-b4a7-488c-a87c-c6a4ab146355 admin admin] PATH_INFO: `/v3/projects/bfde48d2eac54169b872ab50c329039e/users/5dc2be35c23442d29f8cae35e6728810/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.731154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059be63a-b4a7-488c-a87c-c6a4ab146355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfde48d2eac54169b872ab50c329039e, user_id=5dc2be35c23442d29f8cae35e6728810, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.733864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059be63a-b4a7-488c-a87c-c6a4ab146355 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.735737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4155-f1eb-4e6c-97ef-1a46ca215f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.736540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4155-f1eb-4e6c-97ef-1a46ca215f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.736909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4155-f1eb-4e6c-97ef-1a46ca215f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.737271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4155-f1eb-4e6c-97ef-1a46ca215f95 admin admin] PATH_INFO: `/v3/projects/474c2d620e224f06b409adf776004cac/users/4b16bd0809f3411a91fae6c5e2f3acee/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.745195 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 643/1290] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:53 2022] PUT /identity/v3/projects/bfde48d2eac54169b872ab50c329039e/users/5dc2be35c23442d29f8cae35e6728810/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:53.746010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059be63a-b4a7-488c-a87c-c6a4ab146355 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.752509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac4155-f1eb-4e6c-97ef-1a46ca215f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474c2d620e224f06b409adf776004cac, user_id=4b16bd0809f3411a91fae6c5e2f3acee, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.755308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac4155-f1eb-4e6c-97ef-1a46ca215f95 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.768669 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 648/1291] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:53 2022] PUT /identity/v3/projects/474c2d620e224f06b409adf776004cac/users/4b16bd0809f3411a91fae6c5e2f3acee/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:53.769489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4155-f1eb-4e6c-97ef-1a46ca215f95 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:53.812165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059be63a-b4a7-488c-a87c-c6a4ab146355 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.832604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4155-f1eb-4e6c-97ef-1a46ca215f95 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:53.838353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961ca60-ba01-44f4-bf50-23377ed8adb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.838840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961ca60-ba01-44f4-bf50-23377ed8adb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.839121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961ca60-ba01-44f4-bf50-23377ed8adb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.839417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961ca60-ba01-44f4-bf50-23377ed8adb5 admin admin] PATH_INFO: `/v3/projects/41d79b5940a246eabe32fc5c5309a3d6/users/b13dec0172eb4a7b9c28d89456d2a6cb/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.855880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961ca60-ba01-44f4-bf50-23377ed8adb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41d79b5940a246eabe32fc5c5309a3d6, user_id=b13dec0172eb4a7b9c28d89456d2a6cb, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.858452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961ca60-ba01-44f4-bf50-23377ed8adb5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.870436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f1035-6403-429c-b373-a6e7ec168c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:53.871037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f1035-6403-429c-b373-a6e7ec168c24 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.871311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f1035-6403-429c-b373-a6e7ec168c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.871582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f1035-6403-429c-b373-a6e7ec168c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.872213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f1035-6403-429c-b373-a6e7ec168c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:53.872741 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 649/1292] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:53 2022] PUT /identity/v3/projects/41d79b5940a246eabe32fc5c5309a3d6/users/b13dec0172eb4a7b9c28d89456d2a6cb/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:53.873850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f1035-6403-429c-b373-a6e7ec168c24 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:53.874111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0856e-63d2-4e66-83ea-4b50efdad513 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.874513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0856e-63d2-4e66-83ea-4b50efdad513 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.874786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0856e-63d2-4e66-83ea-4b50efdad513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.888238 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 644/1293] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:53.888238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869997b4-333b-4cc9-8853-8bd2dae6d3f6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.888238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869997b4-333b-4cc9-8853-8bd2dae6d3f6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.888238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869997b4-333b-4cc9-8853-8bd2dae6d3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.894089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:53.906796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-67d0856e-63d2-4e66-83ea-4b50efdad513 None None] MFA Rules not processed for user `6b5a19d928dc47e48486a58359778635`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:53.908152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-67d0856e-63d2-4e66-83ea-4b50efdad513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:53.918600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-869997b4-333b-4cc9-8853-8bd2dae6d3f6 None None] MFA Rules not processed for user `110fc0e5be3141caa029822a601ff2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:53.919267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:53.919889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-869997b4-333b-4cc9-8853-8bd2dae6d3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:53.976590 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 650/1294] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:53.978147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986d557-69db-43eb-82f7-2a38fb410002 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.978424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986d557-69db-43eb-82f7-2a38fb410002 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.978683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986d557-69db-43eb-82f7-2a38fb410002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:53.993133 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 645/1295] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:53.994343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1a871-97e4-4ab4-a319-cc31f0efbee9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:53.994687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1a871-97e4-4ab4-a319-cc31f0efbee9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:53.995023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1a871-97e4-4ab4-a319-cc31f0efbee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.001082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0986d557-69db-43eb-82f7-2a38fb410002 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:54.002283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0986d557-69db-43eb-82f7-2a38fb410002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:54.024048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4dc1a871-97e4-4ab4-a319-cc31f0efbee9 None None] MFA Rules not processed for user `5c46bbe8072c44beb2ea45c0a3fe845e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:54.025277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4dc1a871-97e4-4ab4-a319-cc31f0efbee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:54.065182 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 651/1296] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:54.066426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f2a3-afad-433a-9785-ea0c587a116e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.066702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f2a3-afad-433a-9785-ea0c587a116e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.066964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f2a3-afad-433a-9785-ea0c587a116e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.092809 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 646/1297] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:54.094003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d79d0-b170-4573-9070-8758b2ab1945 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.094279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d79d0-b170-4573-9070-8758b2ab1945 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.094552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d79d0-b170-4573-9070-8758b2ab1945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.096372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4508f2a3-afad-433a-9785-ea0c587a116e None None] MFA Rules not processed for user `5dc2be35c23442d29f8cae35e6728810`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:54.102520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4508f2a3-afad-433a-9785-ea0c587a116e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:54.123084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ad1d79d0-b170-4573-9070-8758b2ab1945 None None] MFA Rules not processed for user `4b16bd0809f3411a91fae6c5e2f3acee`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:54.124482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ad1d79d0-b170-4573-9070-8758b2ab1945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:54.174444 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 652/1298] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:54.175822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acbbc2-fd55-41d4-adca-5557999b8658 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.176259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acbbc2-fd55-41d4-adca-5557999b8658 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.176532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acbbc2-fd55-41d4-adca-5557999b8658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.205659 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 647/1299] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:54.206496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d79d0-b170-4573-9070-8758b2ab1945 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.215357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c1acbbc2-fd55-41d4-adca-5557999b8658 None None] MFA Rules not processed for user `b13dec0172eb4a7b9c28d89456d2a6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:54.216801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c1acbbc2-fd55-41d4-adca-5557999b8658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:54.218705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d79d0-b170-4573-9070-8758b2ab1945 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.224877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553c1a1-ff1c-4847-8d40-dbe3f1b49d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.225823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553c1a1-ff1c-4847-8d40-dbe3f1b49d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.226193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553c1a1-ff1c-4847-8d40-dbe3f1b49d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.226375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553c1a1-ff1c-4847-8d40-dbe3f1b49d77 admin admin] PATH_INFO: `/v3/projects/3ea4f0b754b1409b8a39c6b335b33bae/users/0bcee660c15d444d9c502256c10f5ce8/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.228877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.244562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553c1a1-ff1c-4847-8d40-dbe3f1b49d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea4f0b754b1409b8a39c6b335b33bae, user_id=0bcee660c15d444d9c502256c10f5ce8, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.247251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553c1a1-ff1c-4847-8d40-dbe3f1b49d77 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.248639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.261641 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 648/1300] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:54 2022] PUT /identity/v3/projects/3ea4f0b754b1409b8a39c6b335b33bae/users/0bcee660c15d444d9c502256c10f5ce8/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:54.262437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553c1a1-ff1c-4847-8d40-dbe3f1b49d77 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.287911 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 653/1301] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:54.289254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f30e9-0f3d-42a5-b497-d0c7ef215d7e None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.290163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f30e9-0f3d-42a5-b497-d0c7ef215d7e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.290434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f30e9-0f3d-42a5-b497-d0c7ef215d7e None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.291640 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 654/1302] 10.222.0.16 () {54 vars in 961 bytes} [Sat Oct 15 20:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:47:54.293809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f30e9-0f3d-42a5-b497-d0c7ef215d7e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.328053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553c1a1-ff1c-4847-8d40-dbe3f1b49d77 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.332736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db02a6f-cd51-450d-897c-61f5fff4148e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.333362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db02a6f-cd51-450d-897c-61f5fff4148e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.333636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db02a6f-cd51-450d-897c-61f5fff4148e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.333900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db02a6f-cd51-450d-897c-61f5fff4148e admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.334520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db02a6f-cd51-450d-897c-61f5fff4148e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.336463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db02a6f-cd51-450d-897c-61f5fff4148e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.350883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f30e9-0f3d-42a5-b497-d0c7ef215d7e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.355188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c40d4-d1c4-4910-bfb9-e930cf67c8ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.355782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c40d4-d1c4-4910-bfb9-e930cf67c8ea service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.356051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c40d4-d1c4-4910-bfb9-e930cf67c8ea service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.356330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c40d4-d1c4-4910-bfb9-e930cf67c8ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.367267 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 649/1303] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:54.367267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db02a6f-cd51-450d-897c-61f5fff4148e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.369467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c40d4-d1c4-4910-bfb9-e930cf67c8ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.370906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c40d4-d1c4-4910-bfb9-e930cf67c8ea service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.380633 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 655/1304] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:54.381315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c40d4-d1c4-4910-bfb9-e930cf67c8ea service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.390636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c40d4-d1c4-4910-bfb9-e930cf67c8ea service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.423285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db02a6f-cd51-450d-897c-61f5fff4148e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.427725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb41c2-c85f-48dd-86cb-d4d1c9345b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.428082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb41c2-c85f-48dd-86cb-d4d1c9345b3c service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.428296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb41c2-c85f-48dd-86cb-d4d1c9345b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.428562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb41c2-c85f-48dd-86cb-d4d1c9345b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.452077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77d902-e18c-44c9-9e80-f8f3cfe209dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.452760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77d902-e18c-44c9-9e80-f8f3cfe209dc admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.453062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77d902-e18c-44c9-9e80-f8f3cfe209dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.453327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77d902-e18c-44c9-9e80-f8f3cfe209dc admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.453946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77d902-e18c-44c9-9e80-f8f3cfe209dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.455846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77d902-e18c-44c9-9e80-f8f3cfe209dc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.488995 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 656/1305] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:54.488995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b30d83-847f-44ef-aaa8-d3b3af2e7b8d None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.488995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b30d83-847f-44ef-aaa8-d3b3af2e7b8d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.488995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b30d83-847f-44ef-aaa8-d3b3af2e7b8d None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.490269 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 657/1306] 10.222.0.16 () {54 vars in 961 bytes} [Sat Oct 15 20:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:47:54.491189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b30d83-847f-44ef-aaa8-d3b3af2e7b8d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.499978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb41c2-c85f-48dd-86cb-d4d1c9345b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.501459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb41c2-c85f-48dd-86cb-d4d1c9345b3c service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.514255 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 650/1307] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:54.515588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb41c2-c85f-48dd-86cb-d4d1c9345b3c service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.528821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb41c2-c85f-48dd-86cb-d4d1c9345b3c service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.553315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.561090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.576190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b30d83-847f-44ef-aaa8-d3b3af2e7b8d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.582103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d4ee7-c113-4925-9232-f025bb9410d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.582807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d4ee7-c113-4925-9232-f025bb9410d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.583156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d4ee7-c113-4925-9232-f025bb9410d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.583497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d4ee7-c113-4925-9232-f025bb9410d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.584237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d4ee7-c113-4925-9232-f025bb9410d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.586741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d4ee7-c113-4925-9232-f025bb9410d8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.600202 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 658/1308] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:54.605680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40b4f-b1ab-4695-a90d-9f2bf0fe1635 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.605992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40b4f-b1ab-4695-a90d-9f2bf0fe1635 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.606363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40b4f-b1ab-4695-a90d-9f2bf0fe1635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.615893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778189f-cbb7-4c18-8a91-4a4063c8ae05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.616811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778189f-cbb7-4c18-8a91-4a4063c8ae05 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.617314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778189f-cbb7-4c18-8a91-4a4063c8ae05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.618587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778189f-cbb7-4c18-8a91-4a4063c8ae05 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.619457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778189f-cbb7-4c18-8a91-4a4063c8ae05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.621980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778189f-cbb7-4c18-8a91-4a4063c8ae05 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.655417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-76a40b4f-b1ab-4695-a90d-9f2bf0fe1635 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:54.656769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76a40b4f-b1ab-4695-a90d-9f2bf0fe1635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:54.659381 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 651/1309] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/projects => generated 399 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:54.660848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778189f-cbb7-4c18-8a91-4a4063c8ae05 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.745885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778189f-cbb7-4c18-8a91-4a4063c8ae05 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.754144 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 659/1310] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:54.754901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb87d7-259b-4bb1-81a9-db3ce5aa7309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.755614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb87d7-259b-4bb1-81a9-db3ce5aa7309 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.756444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb87d7-259b-4bb1-81a9-db3ce5aa7309 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.756444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb87d7-259b-4bb1-81a9-db3ce5aa7309 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.757069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb87d7-259b-4bb1-81a9-db3ce5aa7309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.760146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb87d7-259b-4bb1-81a9-db3ce5aa7309 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.761215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a40b4f-b1ab-4695-a90d-9f2bf0fe1635 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.771744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a40b4f-b1ab-4695-a90d-9f2bf0fe1635 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.777932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bff9ad-fa30-4309-9d42-8b570e1f0998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.778649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bff9ad-fa30-4309-9d42-8b570e1f0998 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.779211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bff9ad-fa30-4309-9d42-8b570e1f0998 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.779567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bff9ad-fa30-4309-9d42-8b570e1f0998 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.780534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bff9ad-fa30-4309-9d42-8b570e1f0998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.783055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bff9ad-fa30-4309-9d42-8b570e1f0998 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.796280 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 652/1311] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:54.798000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427bbb-9630-4712-a3fc-22a4199b4fd7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.798651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427bbb-9630-4712-a3fc-22a4199b4fd7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.799591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427bbb-9630-4712-a3fc-22a4199b4fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.812726 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 660/1312] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/users => generated 298 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:54.813679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bff9ad-fa30-4309-9d42-8b570e1f0998 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.827791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bff9ad-fa30-4309-9d42-8b570e1f0998 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.834386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b9427bbb-9630-4712-a3fc-22a4199b4fd7 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:54.835447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9ed10-909e-4eb9-9b26-ab35e330c600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.836464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9ed10-909e-4eb9-9b26-ab35e330c600 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.836715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9ed10-909e-4eb9-9b26-ab35e330c600 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.837115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9ed10-909e-4eb9-9b26-ab35e330c600 admin admin] PATH_INFO: `/v3/projects/3ea4f0b754b1409b8a39c6b335b33bae/users/0bcee660c15d444d9c502256c10f5ce8/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.839498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9ed10-909e-4eb9-9b26-ab35e330c600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea4f0b754b1409b8a39c6b335b33bae, user_id=0bcee660c15d444d9c502256c10f5ce8, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.844325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9ed10-909e-4eb9-9b26-ab35e330c600 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.861573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9427bbb-9630-4712-a3fc-22a4199b4fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:54.879712 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 661/1313] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:54 2022] PUT /identity/v3/projects/3ea4f0b754b1409b8a39c6b335b33bae/users/0bcee660c15d444d9c502256c10f5ce8/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:54.880926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9ed10-909e-4eb9-9b26-ab35e330c600 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:54.956653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9ed10-909e-4eb9-9b26-ab35e330c600 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:54.963964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecf411-ee2d-4177-91ee-a3886782173e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:54.964715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecf411-ee2d-4177-91ee-a3886782173e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:54.965121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecf411-ee2d-4177-91ee-a3886782173e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:54.965952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecf411-ee2d-4177-91ee-a3886782173e admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:54.966748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecf411-ee2d-4177-91ee-a3886782173e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:54.969466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecf411-ee2d-4177-91ee-a3886782173e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:54.993850 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 653/1314] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:54.995283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9427bbb-9630-4712-a3fc-22a4199b4fd7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.002399 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 662/1315] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:54 2022] POST /identity/v3/users => generated 306 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:55.005021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecf411-ee2d-4177-91ee-a3886782173e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.009574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9427bbb-9630-4712-a3fc-22a4199b4fd7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.015546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc026d-acf2-4e00-bf78-6ccf9ae83121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.016101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc026d-acf2-4e00-bf78-6ccf9ae83121 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.016384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc026d-acf2-4e00-bf78-6ccf9ae83121 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.016655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc026d-acf2-4e00-bf78-6ccf9ae83121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.017357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecf411-ee2d-4177-91ee-a3886782173e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.023370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9741738-6784-4bc7-9703-ba44b1c1f714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.024657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9741738-6784-4bc7-9703-ba44b1c1f714 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.024996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9741738-6784-4bc7-9703-ba44b1c1f714 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.025352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9741738-6784-4bc7-9703-ba44b1c1f714 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.026056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9741738-6784-4bc7-9703-ba44b1c1f714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.027624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9741738-6784-4bc7-9703-ba44b1c1f714 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.041496 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 663/1316] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:55.041496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9741738-6784-4bc7-9703-ba44b1c1f714 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.049287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9741738-6784-4bc7-9703-ba44b1c1f714 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.053085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:55.054820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6aa46-1990-45cc-8333-5f979dc10024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.056015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6aa46-1990-45cc-8333-5f979dc10024 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.056283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6aa46-1990-45cc-8333-5f979dc10024 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.056606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6aa46-1990-45cc-8333-5f979dc10024 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.057334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6aa46-1990-45cc-8333-5f979dc10024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.058967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6aa46-1990-45cc-8333-5f979dc10024 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.073835 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 664/1317] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:55.075225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed5abb-74b2-48e3-8968-af24603d1589 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.075603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed5abb-74b2-48e3-8968-af24603d1589 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.075971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed5abb-74b2-48e3-8968-af24603d1589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.091342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:55.095657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc026d-acf2-4e00-bf78-6ccf9ae83121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.097607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc026d-acf2-4e00-bf78-6ccf9ae83121 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.109815 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 654/1318] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:55.110739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc026d-acf2-4e00-bf78-6ccf9ae83121 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.110985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-75ed5abb-74b2-48e3-8968-af24603d1589 None None] MFA Rules not processed for user `0bcee660c15d444d9c502256c10f5ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:55.113074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-75ed5abb-74b2-48e3-8968-af24603d1589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:55.123931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc026d-acf2-4e00-bf78-6ccf9ae83121 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.129852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.131091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.131741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.132021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] PATH_INFO: `/v3/users/5c46bbe8072c44beb2ea45c0a3fe845e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.133945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c46bbe8072c44beb2ea45c0a3fe845e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.135102 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:47:55.135102 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 20:47:55.135102 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 20:47:55.136303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.213207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 665/1319] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:55.214137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed5abb-74b2-48e3-8968-af24603d1589 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.229849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c46bbe8072c44beb2ea45c0a3fe845e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:55.239829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed5abb-74b2-48e3-8968-af24603d1589 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.252631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab73a4-5c1d-4053-8bae-d8e8ec8ac4e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.254068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab73a4-5c1d-4053-8bae-d8e8ec8ac4e5 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.255166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab73a4-5c1d-4053-8bae-d8e8ec8ac4e5 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.255603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab73a4-5c1d-4053-8bae-d8e8ec8ac4e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.265410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c46bbe8072c44beb2ea45c0a3fe845e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:55.280265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c46bbe8072c44beb2ea45c0a3fe845e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:55.299291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c46bbe8072c44beb2ea45c0a3fe845e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:55.300100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c46bbe8072c44beb2ea45c0a3fe845e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:55.308710 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 655/1320] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:47:55 2022] DELETE /identity/v3/users/5c46bbe8072c44beb2ea45c0a3fe845e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:55.311811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.311811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:55.390382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12a716-52e3-4d5c-b3e6-22ff86304cb0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.395417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c17ae-a581-4611-b6f8-fc999d0d2da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.396033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c17ae-a581-4611-b6f8-fc999d0d2da6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.396486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c17ae-a581-4611-b6f8-fc999d0d2da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.397066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c17ae-a581-4611-b6f8-fc999d0d2da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.397610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c17ae-a581-4611-b6f8-fc999d0d2da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.399438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c17ae-a581-4611-b6f8-fc999d0d2da6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.415164 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 656/1321] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:55.415984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab73a4-5c1d-4053-8bae-d8e8ec8ac4e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.416502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c17ae-a581-4611-b6f8-fc999d0d2da6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.418711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-518c17ae-a581-4611-b6f8-fc999d0d2da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:55.420837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab73a4-5c1d-4053-8bae-d8e8ec8ac4e5 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.436244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c17ae-a581-4611-b6f8-fc999d0d2da6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.443123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adf6f7-6072-4c42-8e2b-c11d61157530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.444142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adf6f7-6072-4c42-8e2b-c11d61157530 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.444612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adf6f7-6072-4c42-8e2b-c11d61157530 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.445093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adf6f7-6072-4c42-8e2b-c11d61157530 admin admin] PATH_INFO: `/v3/projects/56bd982260c64bf4986964be986b12fa/users/fe181325988b46d6a8316868736310c4/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.463101 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 666/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:55.471051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab73a4-5c1d-4053-8bae-d8e8ec8ac4e5 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.493700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55adf6f7-6072-4c42-8e2b-c11d61157530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56bd982260c64bf4986964be986b12fa, user_id=fe181325988b46d6a8316868736310c4, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.495869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55adf6f7-6072-4c42-8e2b-c11d61157530 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.500054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab73a4-5c1d-4053-8bae-d8e8ec8ac4e5 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.518671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce24178-4f61-4f15-b989-9ab81b279c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.519337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce24178-4f61-4f15-b989-9ab81b279c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.519750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce24178-4f61-4f15-b989-9ab81b279c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.519991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce24178-4f61-4f15-b989-9ab81b279c54 admin admin] PATH_INFO: `/v3/projects/80a8775b9bd74eed81b89c8eb16ed258/users/3e09319070164e118879498437350f7f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.525751 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 657/1323] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:55 2022] PUT /identity/v3/projects/56bd982260c64bf4986964be986b12fa/users/fe181325988b46d6a8316868736310c4/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:55.526965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16575c44-dcd2-42ca-99b8-fa352013389e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.527349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16575c44-dcd2-42ca-99b8-fa352013389e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.527718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16575c44-dcd2-42ca-99b8-fa352013389e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.541068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce24178-4f61-4f15-b989-9ab81b279c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a8775b9bd74eed81b89c8eb16ed258, user_id=3e09319070164e118879498437350f7f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.543852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce24178-4f61-4f15-b989-9ab81b279c54 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.552550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:55.566230 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 667/1324] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:55 2022] PUT /identity/v3/projects/80a8775b9bd74eed81b89c8eb16ed258/users/3e09319070164e118879498437350f7f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:55.569768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-16575c44-dcd2-42ca-99b8-fa352013389e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:55.571345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce24178-4f61-4f15-b989-9ab81b279c54 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.572943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-16575c44-dcd2-42ca-99b8-fa352013389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:55.671641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce24178-4f61-4f15-b989-9ab81b279c54 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.676347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa094a3-d181-4ce5-bcdb-f969d8e4f084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.677663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa094a3-d181-4ce5-bcdb-f969d8e4f084 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.678030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa094a3-d181-4ce5-bcdb-f969d8e4f084 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.678339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa094a3-d181-4ce5-bcdb-f969d8e4f084 admin admin] PATH_INFO: `/v3/projects/1b4645ec7c584e549cdc3cbf9965a11d/users/a1e9c90d8c1549aa83893ed2476c3a6d/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.683527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:55.707683 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 658/1325] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:55.708398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa094a3-d181-4ce5-bcdb-f969d8e4f084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b4645ec7c584e549cdc3cbf9965a11d, user_id=a1e9c90d8c1549aa83893ed2476c3a6d, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.708654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16575c44-dcd2-42ca-99b8-fa352013389e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.710029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-16575c44-dcd2-42ca-99b8-fa352013389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:55.712458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa094a3-d181-4ce5-bcdb-f969d8e4f084 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.736783 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 668/1326] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:55 2022] PUT /identity/v3/projects/1b4645ec7c584e549cdc3cbf9965a11d/users/a1e9c90d8c1549aa83893ed2476c3a6d/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:55.738083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa094a3-d181-4ce5-bcdb-f969d8e4f084 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.828752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa094a3-d181-4ce5-bcdb-f969d8e4f084 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.840514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6797d-9ef3-493f-8d34-b779773e9773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.841740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6797d-9ef3-493f-8d34-b779773e9773 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.841928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6797d-9ef3-493f-8d34-b779773e9773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.843788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6797d-9ef3-493f-8d34-b779773e9773 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.843788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6797d-9ef3-493f-8d34-b779773e9773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.844981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6797d-9ef3-493f-8d34-b779773e9773 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.862955 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 669/1327] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:55.864093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6797d-9ef3-493f-8d34-b779773e9773 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.866443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-18a6797d-9ef3-493f-8d34-b779773e9773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:55.881985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16575c44-dcd2-42ca-99b8-fa352013389e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.885257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6797d-9ef3-493f-8d34-b779773e9773 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:55.893886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ccdd6d-527d-45f9-8e86-7d819f32dc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.895060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccdd6d-527d-45f9-8e86-7d819f32dc47 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.895288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccdd6d-527d-45f9-8e86-7d819f32dc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.895553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccdd6d-527d-45f9-8e86-7d819f32dc47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.897577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ccdd6d-527d-45f9-8e86-7d819f32dc47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:55.899413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ccdd6d-527d-45f9-8e86-7d819f32dc47 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:55.916058 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 670/1328] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:55.917638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ccdd6d-527d-45f9-8e86-7d819f32dc47 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:55.920215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-88ccdd6d-527d-45f9-8e86-7d819f32dc47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:55.974507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec6549-a3be-470c-85b1-26755a5caa09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:55.976166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6549-a3be-470c-85b1-26755a5caa09 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:55.976444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6549-a3be-470c-85b1-26755a5caa09 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:55.977708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6549-a3be-470c-85b1-26755a5caa09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:55.980236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6eec6549-a3be-470c-85b1-26755a5caa09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:56.008036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ccdd6d-527d-45f9-8e86-7d819f32dc47 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:56.012166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee8ec1-1d71-41ad-b1fc-bb21d687d94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:56.013511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee8ec1-1d71-41ad-b1fc-bb21d687d94c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:56.013943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee8ec1-1d71-41ad-b1fc-bb21d687d94c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:56.014378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee8ec1-1d71-41ad-b1fc-bb21d687d94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:56.028727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee8ec1-1d71-41ad-b1fc-bb21d687d94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:56.031298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee8ec1-1d71-41ad-b1fc-bb21d687d94c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:56.049828 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 671/1329] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:56.052052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee8ec1-1d71-41ad-b1fc-bb21d687d94c service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:56.076397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:56.079593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee8ec1-1d71-41ad-b1fc-bb21d687d94c service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:56.094350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec69b4-b497-4685-86f7-353dbb33af2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:56.094630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec69b4-b497-4685-86f7-353dbb33af2b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:56.094995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec69b4-b497-4685-86f7-353dbb33af2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:56.095321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec69b4-b497-4685-86f7-353dbb33af2b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:56.096196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec69b4-b497-4685-86f7-353dbb33af2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:56.097941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec69b4-b497-4685-86f7-353dbb33af2b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:56.126007 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 672/1330] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:56.127819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec69b4-b497-4685-86f7-353dbb33af2b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:56.128206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec6549-a3be-470c-85b1-26755a5caa09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:56.130258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec6549-a3be-470c-85b1-26755a5caa09 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:56.155808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec69b4-b497-4685-86f7-353dbb33af2b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:56.159899 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 659/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:56.163207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec6549-a3be-470c-85b1-26755a5caa09 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:56.175278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51b273-25eb-4f33-8b9d-16687e077bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:56.176251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b273-25eb-4f33-8b9d-16687e077bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:56.176534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b273-25eb-4f33-8b9d-16687e077bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:56.176807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b273-25eb-4f33-8b9d-16687e077bd0 admin admin] PATH_INFO: `/v3/projects/56bd982260c64bf4986964be986b12fa/users/fe181325988b46d6a8316868736310c4/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:56.183473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51b273-25eb-4f33-8b9d-16687e077bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56bd982260c64bf4986964be986b12fa, user_id=fe181325988b46d6a8316868736310c4, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:56.187232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51b273-25eb-4f33-8b9d-16687e077bd0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:56.188447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec6549-a3be-470c-85b1-26755a5caa09 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:56.203325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee378d3-a114-41d6-b87b-450ce58f4b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:56.206938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee378d3-a114-41d6-b87b-450ce58f4b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:56.207376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee378d3-a114-41d6-b87b-450ce58f4b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:56.208333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee378d3-a114-41d6-b87b-450ce58f4b02 admin admin] PATH_INFO: `/v3/projects/80a8775b9bd74eed81b89c8eb16ed258/users/3e09319070164e118879498437350f7f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:56.214124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee378d3-a114-41d6-b87b-450ce58f4b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a8775b9bd74eed81b89c8eb16ed258, user_id=3e09319070164e118879498437350f7f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:56.221845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee378d3-a114-41d6-b87b-450ce58f4b02 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:56.228688 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 673/1332] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:56 2022] PUT /identity/v3/projects/56bd982260c64bf4986964be986b12fa/users/fe181325988b46d6a8316868736310c4/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:56.231031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51b273-25eb-4f33-8b9d-16687e077bd0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:56.275419 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 660/1333] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:56 2022] PUT /identity/v3/projects/80a8775b9bd74eed81b89c8eb16ed258/users/3e09319070164e118879498437350f7f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:56.276750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86324a9-0a7f-4f5d-ac23-022e2f9d9219 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:56.277035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86324a9-0a7f-4f5d-ac23-022e2f9d9219 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:56.277408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86324a9-0a7f-4f5d-ac23-022e2f9d9219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:56.370048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b86324a9-0a7f-4f5d-ac23-022e2f9d9219 None None] MFA Rules not processed for user `fe181325988b46d6a8316868736310c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:56.371955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b86324a9-0a7f-4f5d-ac23-022e2f9d9219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:56.373087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:56.401392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51b273-25eb-4f33-8b9d-16687e077bd0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:56.534217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d52e3-208f-48f7-8507-b38f02dd7519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:56.534217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d52e3-208f-48f7-8507-b38f02dd7519 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:56.534515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d52e3-208f-48f7-8507-b38f02dd7519 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:56.535360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d52e3-208f-48f7-8507-b38f02dd7519 admin admin] PATH_INFO: `/v3/projects/1b4645ec7c584e549cdc3cbf9965a11d/users/a1e9c90d8c1549aa83893ed2476c3a6d/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:56.539808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d52e3-208f-48f7-8507-b38f02dd7519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b4645ec7c584e549cdc3cbf9965a11d, user_id=a1e9c90d8c1549aa83893ed2476c3a6d, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:56.551804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d52e3-208f-48f7-8507-b38f02dd7519 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:56.552932 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 661/1334] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:56.580806 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 674/1335] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:56 2022] PUT /identity/v3/projects/1b4645ec7c584e549cdc3cbf9965a11d/users/a1e9c90d8c1549aa83893ed2476c3a6d/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:56.581686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d52e3-208f-48f7-8507-b38f02dd7519 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:56.583610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1c4d52e3-208f-48f7-8507-b38f02dd7519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:56.592486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975d260-982b-4555-ab9e-4f47b7162bd5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:56.592882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975d260-982b-4555-ab9e-4f47b7162bd5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:56.593288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975d260-982b-4555-ab9e-4f47b7162bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:56.645253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e975d260-982b-4555-ab9e-4f47b7162bd5 None None] MFA Rules not processed for user `3e09319070164e118879498437350f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:56.647399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e975d260-982b-4555-ab9e-4f47b7162bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:56.704005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d52e3-208f-48f7-8507-b38f02dd7519 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:56.712522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b639b8a-da5d-43d5-8456-05eb987523e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:56.736556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b639b8a-da5d-43d5-8456-05eb987523e9 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:56.736556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b639b8a-da5d-43d5-8456-05eb987523e9 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:56.736556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b639b8a-da5d-43d5-8456-05eb987523e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:56.736556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6b639b8a-da5d-43d5-8456-05eb987523e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:56.760442 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 662/1336] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:56.761709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975d260-982b-4555-ab9e-4f47b7162bd5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:56.763260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e975d260-982b-4555-ab9e-4f47b7162bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:56.808344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b639b8a-da5d-43d5-8456-05eb987523e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:56.809985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b639b8a-da5d-43d5-8456-05eb987523e9 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:56.855865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:56.857511 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 675/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:56.859029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab822d8b-7ab6-42da-9772-5ca9deb6b38f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:56.859323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab822d8b-7ab6-42da-9772-5ca9deb6b38f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:56.859595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab822d8b-7ab6-42da-9772-5ca9deb6b38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:56.906429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ab822d8b-7ab6-42da-9772-5ca9deb6b38f None None] MFA Rules not processed for user `a1e9c90d8c1549aa83893ed2476c3a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:56.907804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ab822d8b-7ab6-42da-9772-5ca9deb6b38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:56.910693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975d260-982b-4555-ab9e-4f47b7162bd5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:56.925314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:56.925964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:56.926238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:56.926503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] PATH_INFO: `/v3/projects/e442345a419943b3809728546377b3d1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:56.928353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:56.928481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e442345a419943b3809728546377b3d1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:56.931247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:57.005676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e442345a419943b3809728546377b3d1', 'request_id': 'req-83eeadc0-7f21-43f6-be59-b339b8452126'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:57.018579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e442345a419943b3809728546377b3d1', 'request_id': 'req-83eeadc0-7f21-43f6-be59-b339b8452126'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:57.030066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e442345a419943b3809728546377b3d1', 'request_id': 'req-83eeadc0-7f21-43f6-be59-b339b8452126'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:57.045369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] The token cache is being invalidate because project e442345a419943b3809728546377b3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:47:57.045659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:57.046468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:57.048417 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 663/1338] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:47:56 2022] DELETE /identity/v3/projects/e442345a419943b3809728546377b3d1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:57.049268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:57.051512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.092445 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 676/1339] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:57.094018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a68fbf-e01a-4778-9a8a-298d6553a492 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.094246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a68fbf-e01a-4778-9a8a-298d6553a492 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.095688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a68fbf-e01a-4778-9a8a-298d6553a492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.116821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d7a68fbf-e01a-4778-9a8a-298d6553a492 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:57.118608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d7a68fbf-e01a-4778-9a8a-298d6553a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.145444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeadc0-7f21-43f6-be59-b339b8452126 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:57.153856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081874db-cfb5-4cac-9699-1d9897cd8f25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:57.154659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081874db-cfb5-4cac-9699-1d9897cd8f25 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.154988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081874db-cfb5-4cac-9699-1d9897cd8f25 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.155316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081874db-cfb5-4cac-9699-1d9897cd8f25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.157456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-081874db-cfb5-4cac-9699-1d9897cd8f25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.189016 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 677/1340] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:57.189994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a68fbf-e01a-4778-9a8a-298d6553a492 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:57.191741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d7a68fbf-e01a-4778-9a8a-298d6553a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.209002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a68fbf-e01a-4778-9a8a-298d6553a492 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:57.215275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:57.215916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.216192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.216462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] PATH_INFO: `/v3/users/55dd0c9b11ed40ebb292b03ad57078bb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.217624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dd0c9b11ed40ebb292b03ad57078bb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:57.220050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:57.255154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081874db-cfb5-4cac-9699-1d9897cd8f25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:57.257004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081874db-cfb5-4cac-9699-1d9897cd8f25 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:57.262296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:57.281722 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 664/1341] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:57.283246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081874db-cfb5-4cac-9699-1d9897cd8f25 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:57.286359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-081874db-cfb5-4cac-9699-1d9897cd8f25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.288558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dd0c9b11ed40ebb292b03ad57078bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:57.298678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dd0c9b11ed40ebb292b03ad57078bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:57.308751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081874db-cfb5-4cac-9699-1d9897cd8f25 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:57.315951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b96334-2fd5-4723-9f61-768700ca29cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:57.316962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b96334-2fd5-4723-9f61-768700ca29cb service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.317684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b96334-2fd5-4723-9f61-768700ca29cb service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.317963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b96334-2fd5-4723-9f61-768700ca29cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.320109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a8b96334-2fd5-4723-9f61-768700ca29cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.326977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dd0c9b11ed40ebb292b03ad57078bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:57.337876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dd0c9b11ed40ebb292b03ad57078bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:57.338660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-029e94e8-840e-48b7-bf2f-7a68f04a9d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dd0c9b11ed40ebb292b03ad57078bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:57.352652 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 678/1342] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:47:57 2022] DELETE /identity/v3/users/55dd0c9b11ed40ebb292b03ad57078bb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:57.354285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd159802-122e-4894-a6ea-42f802e560cb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.354729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd159802-122e-4894-a6ea-42f802e560cb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.355004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd159802-122e-4894-a6ea-42f802e560cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.387956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bd159802-122e-4894-a6ea-42f802e560cb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:57.389297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bd159802-122e-4894-a6ea-42f802e560cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.447020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b96334-2fd5-4723-9f61-768700ca29cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:57.448849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b96334-2fd5-4723-9f61-768700ca29cb service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:57.458985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:57.478300 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 665/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:57.490468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b905e-5eb1-4d7a-b140-58bb8b763e98 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.490911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b905e-5eb1-4d7a-b140-58bb8b763e98 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.491262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b905e-5eb1-4d7a-b140-58bb8b763e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.538874 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 679/1344] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:57.541516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c26594-b126-42bc-9c91-826cf5f81f6f None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.541798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c26594-b126-42bc-9c91-826cf5f81f6f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.542082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c26594-b126-42bc-9c91-826cf5f81f6f None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.543276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-392b905e-5eb1-4d7a-b140-58bb8b763e98 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:57.543529 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 680/1345] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:47:57.544310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c26594-b126-42bc-9c91-826cf5f81f6f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:57.544516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-392b905e-5eb1-4d7a-b140-58bb8b763e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.559895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c26594-b126-42bc-9c91-826cf5f81f6f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:57.565901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727b1ae-2400-4253-931f-4b923e6ca856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:57.566665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727b1ae-2400-4253-931f-4b923e6ca856 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.567003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727b1ae-2400-4253-931f-4b923e6ca856 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.567337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727b1ae-2400-4253-931f-4b923e6ca856 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.568051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727b1ae-2400-4253-931f-4b923e6ca856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:47:57.568353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727b1ae-2400-4253-931f-4b923e6ca856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:57.570345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727b1ae-2400-4253-931f-4b923e6ca856 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:57.573328 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 666/1346] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:57.574639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dc638-3d5e-43b4-8fb8-021bdfb392a1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.574921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dc638-3d5e-43b4-8fb8-021bdfb392a1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.575219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dc638-3d5e-43b4-8fb8-021bdfb392a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.592288 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 681/1347] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:47:57.593204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727b1ae-2400-4253-931f-4b923e6ca856 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:57.597281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a727b1ae-2400-4253-931f-4b923e6ca856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.642003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4b8dc638-3d5e-43b4-8fb8-021bdfb392a1 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:57.643686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4b8dc638-3d5e-43b4-8fb8-021bdfb392a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.691023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727b1ae-2400-4253-931f-4b923e6ca856 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:57.695175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab3e35-a30d-40ce-8bf9-669f05622edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:57.695839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab3e35-a30d-40ce-8bf9-669f05622edb service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.696116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab3e35-a30d-40ce-8bf9-669f05622edb service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.696384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab3e35-a30d-40ce-8bf9-669f05622edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.707584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab3e35-a30d-40ce-8bf9-669f05622edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:57.708610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab3e35-a30d-40ce-8bf9-669f05622edb service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:57.735408 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 682/1348] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:57.736326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab3e35-a30d-40ce-8bf9-669f05622edb service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:57.750938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 667/1349] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:47:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:57.752326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8dc638-3d5e-43b4-8fb8-021bdfb392a1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:57.753871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4b8dc638-3d5e-43b4-8fb8-021bdfb392a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.754588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab3e35-a30d-40ce-8bf9-669f05622edb service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:57.763546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1de4b-6978-4ae6-9241-c95bcc339af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:57.764232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1de4b-6978-4ae6-9241-c95bcc339af0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.764514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1de4b-6978-4ae6-9241-c95bcc339af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.764781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1de4b-6978-4ae6-9241-c95bcc339af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.765447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1de4b-6978-4ae6-9241-c95bcc339af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:57.767369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1de4b-6978-4ae6-9241-c95bcc339af0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:57.817375 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 683/1350] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:57 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:57.817375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1de4b-6978-4ae6-9241-c95bcc339af0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:57.819291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-21d1de4b-6978-4ae6-9241-c95bcc339af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:57.900940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8dc638-3d5e-43b4-8fb8-021bdfb392a1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:57.939085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:57.967124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1de4b-6978-4ae6-9241-c95bcc339af0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:57.978602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79358bb-8575-49dd-b865-6ad2e4a22241 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:57.979377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79358bb-8575-49dd-b865-6ad2e4a22241 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:57.979754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79358bb-8575-49dd-b865-6ad2e4a22241 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:57.980131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79358bb-8575-49dd-b865-6ad2e4a22241 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:57.993888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79358bb-8575-49dd-b865-6ad2e4a22241 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:57.995272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79358bb-8575-49dd-b865-6ad2e4a22241 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:58.008499 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 684/1351] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:58.009666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79358bb-8575-49dd-b865-6ad2e4a22241 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:58.023190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7844ed-58bf-4c25-b4a2-fbf657ce880e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:58.024468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7844ed-58bf-4c25-b4a2-fbf657ce880e service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.024920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7844ed-58bf-4c25-b4a2-fbf657ce880e service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.025541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7844ed-58bf-4c25-b4a2-fbf657ce880e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.027744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4a7844ed-58bf-4c25-b4a2-fbf657ce880e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:58.045067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:58.113727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7844ed-58bf-4c25-b4a2-fbf657ce880e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:58.115615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7844ed-58bf-4c25-b4a2-fbf657ce880e service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:58.138221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79358bb-8575-49dd-b865-6ad2e4a22241 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:58.144682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45380b5c-c7fd-4620-8e05-fc522288f297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:58.145632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45380b5c-c7fd-4620-8e05-fc522288f297 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.146129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45380b5c-c7fd-4620-8e05-fc522288f297 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.147480 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 668/1352] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:57 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:47:58.148109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45380b5c-c7fd-4620-8e05-fc522288f297 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.148109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45380b5c-c7fd-4620-8e05-fc522288f297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:58.150942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45380b5c-c7fd-4620-8e05-fc522288f297 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:58.183938 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 685/1353] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:58 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:58.188775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7844ed-58bf-4c25-b4a2-fbf657ce880e service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:58.206961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7844ed-58bf-4c25-b4a2-fbf657ce880e service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:58.228527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbf197-6edf-4715-96ce-87d36ac01696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:58.231270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbf197-6edf-4715-96ce-87d36ac01696 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.231610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbf197-6edf-4715-96ce-87d36ac01696 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.231943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbf197-6edf-4715-96ce-87d36ac01696 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.233236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbf197-6edf-4715-96ce-87d36ac01696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:58.242354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbf197-6edf-4715-96ce-87d36ac01696 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:58.271664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45380b5c-c7fd-4620-8e05-fc522288f297 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:58.278206 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 669/1354] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:58.288447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45380b5c-c7fd-4620-8e05-fc522288f297 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:58.296730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474dee-646c-4108-992a-837a3ace6a5f None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.297681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474dee-646c-4108-992a-837a3ace6a5f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.298051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474dee-646c-4108-992a-837a3ace6a5f None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.299725 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 670/1355] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:47:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:47:58.304522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e973a1b-e1d5-4a33-a5f8-3a36851d0bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:58.306227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e973a1b-e1d5-4a33-a5f8-3a36851d0bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.306590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e973a1b-e1d5-4a33-a5f8-3a36851d0bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.306913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e973a1b-e1d5-4a33-a5f8-3a36851d0bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.307692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e973a1b-e1d5-4a33-a5f8-3a36851d0bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:58.311004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f6798-3448-4cf3-a7f6-1b059fd2ffcc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.311483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f6798-3448-4cf3-a7f6-1b059fd2ffcc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.311857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f6798-3448-4cf3-a7f6-1b059fd2ffcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.312451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e973a1b-e1d5-4a33-a5f8-3a36851d0bd3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:58.342443 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 686/1356] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:47:58.345577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-017f6798-3448-4cf3-a7f6-1b059fd2ffcc None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:58.347006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-017f6798-3448-4cf3-a7f6-1b059fd2ffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:58.347364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e973a1b-e1d5-4a33-a5f8-3a36851d0bd3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:58.380278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e973a1b-e1d5-4a33-a5f8-3a36851d0bd3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:58.380879 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 671/1357] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:47:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:58.385265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc5139-b0a3-43d8-aabe-f8c6a62aff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:58.385932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc5139-b0a3-43d8-aabe-f8c6a62aff5e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.388445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc5139-b0a3-43d8-aabe-f8c6a62aff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.388743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc5139-b0a3-43d8-aabe-f8c6a62aff5e admin admin] PATH_INFO: `/v3/projects/2da218e9269f428fa9a89771f41aa244/users/7ec562be10db4c2fb57476fda5a18867/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.407349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc5139-b0a3-43d8-aabe-f8c6a62aff5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da218e9269f428fa9a89771f41aa244, user_id=7ec562be10db4c2fb57476fda5a18867, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:58.410688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc5139-b0a3-43d8-aabe-f8c6a62aff5e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:58.420872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f6798-3448-4cf3-a7f6-1b059fd2ffcc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:58.439779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f6798-3448-4cf3-a7f6-1b059fd2ffcc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:58.460511 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 687/1358] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:47:58 2022] PUT /identity/v3/projects/2da218e9269f428fa9a89771f41aa244/users/7ec562be10db4c2fb57476fda5a18867/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:58.464978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de0077-5bb0-47f1-9581-521ec3dcc122 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.465285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de0077-5bb0-47f1-9581-521ec3dcc122 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.465544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de0077-5bb0-47f1-9581-521ec3dcc122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.517409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d4de0077-5bb0-47f1-9581-521ec3dcc122 None None] MFA Rules not processed for user `7ec562be10db4c2fb57476fda5a18867`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:58.519770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d4de0077-5bb0-47f1-9581-521ec3dcc122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:58.596421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90575359-e2db-44fb-a6a3-949b6d8d3200 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:58.597063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90575359-e2db-44fb-a6a3-949b6d8d3200 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.597336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90575359-e2db-44fb-a6a3-949b6d8d3200 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.597599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90575359-e2db-44fb-a6a3-949b6d8d3200 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.608332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90575359-e2db-44fb-a6a3-949b6d8d3200 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:58.610103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90575359-e2db-44fb-a6a3-949b6d8d3200 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:58.633700 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 672/1359] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:47:58.636969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90575359-e2db-44fb-a6a3-949b6d8d3200 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:58.638593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-90575359-e2db-44fb-a6a3-949b6d8d3200 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:58.684451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:58.697681 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 688/1360] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:58.699494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de0077-5bb0-47f1-9581-521ec3dcc122 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:58.703396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d4de0077-5bb0-47f1-9581-521ec3dcc122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:58.760964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90575359-e2db-44fb-a6a3-949b6d8d3200 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:58.771844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:58.772980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.774266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.774765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] PATH_INFO: `/v3/projects/e9404d6e1de94dabb849df1ecd92e083` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.776315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9404d6e1de94dabb849df1ecd92e083)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:58.778942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:58.863876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9404d6e1de94dabb849df1ecd92e083', 'request_id': 'req-ede84966-cef7-4abe-8c76-bd90bda30b47'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:58.865429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de0077-5bb0-47f1-9581-521ec3dcc122 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:58.877486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:58.883995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9404d6e1de94dabb849df1ecd92e083', 'request_id': 'req-ede84966-cef7-4abe-8c76-bd90bda30b47'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:58.899259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9404d6e1de94dabb849df1ecd92e083', 'request_id': 'req-ede84966-cef7-4abe-8c76-bd90bda30b47'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:58.926112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] The token cache is being invalidate because project e9404d6e1de94dabb849df1ecd92e083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:47:58.926647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:58.927562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:58.930219 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 673/1361] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:47:58 2022] DELETE /identity/v3/projects/e9404d6e1de94dabb849df1ecd92e083 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:58.931586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:58.933305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:58.978114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:58.979045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:58.979045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:58.979288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] PATH_INFO: `/v3/users/5dd7ec5d407f4a0fbfd5593140e0bb0e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:58.980448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd7ec5d407f4a0fbfd5593140e0bb0e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:58.982752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:59.079922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede84966-cef7-4abe-8c76-bd90bda30b47 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:59.092358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd7ec5d407f4a0fbfd5593140e0bb0e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.094025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541b0a8-9848-4767-93f5-f2fcd71e34cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:59.094637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541b0a8-9848-4767-93f5-f2fcd71e34cb admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:59.094912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541b0a8-9848-4767-93f5-f2fcd71e34cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:59.095178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541b0a8-9848-4767-93f5-f2fcd71e34cb admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:59.095813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541b0a8-9848-4767-93f5-f2fcd71e34cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:59.097773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541b0a8-9848-4767-93f5-f2fcd71e34cb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:59.108629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd7ec5d407f4a0fbfd5593140e0bb0e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.153229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd7ec5d407f4a0fbfd5593140e0bb0e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.164411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd7ec5d407f4a0fbfd5593140e0bb0e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.165398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd7ec5d407f4a0fbfd5593140e0bb0e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.175987 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 689/1362] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:47:58 2022] DELETE /identity/v3/users/5dd7ec5d407f4a0fbfd5593140e0bb0e => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:59.178302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:59.180769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:59.181532 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 674/1363] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:47:58 2022] POST /identity/v3/projects => generated 381 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:59.182491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541b0a8-9848-4767-93f5-f2fcd71e34cb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:59.187283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f541b0a8-9848-4767-93f5-f2fcd71e34cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:59.286537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:59.341926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f732202-9266-4c59-a2e4-ce463bb33ac7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:59.353568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:59.354216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:59.355446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:59.355745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] PATH_INFO: `/v3/users/2c81e608ec934eb0ade4927aa7347dc4` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:59.373071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c81e608ec934eb0ade4927aa7347dc4)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:59.373355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541b0a8-9848-4767-93f5-f2fcd71e34cb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:59.373425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:59.390406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:59.393449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:59.393959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:59.395256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] PATH_INFO: `/v3/users/5dc2be35c23442d29f8cae35e6728810` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:59.398187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc2be35c23442d29f8cae35e6728810)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:59.406139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:59.444037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:59.464632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c81e608ec934eb0ade4927aa7347dc4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.498384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c81e608ec934eb0ade4927aa7347dc4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.522834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc2be35c23442d29f8cae35e6728810'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.528050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c81e608ec934eb0ade4927aa7347dc4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.541235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc2be35c23442d29f8cae35e6728810'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.550590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c81e608ec934eb0ade4927aa7347dc4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.553815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8fd26b3-3f98-4f89-a880-7404a217f9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c81e608ec934eb0ade4927aa7347dc4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.555606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc2be35c23442d29f8cae35e6728810'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.563338 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 690/1364] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:47:59 2022] DELETE /identity/v3/users/2c81e608ec934eb0ade4927aa7347dc4 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:59.566715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b4430-b64e-43a0-b07a-080447ab417e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:59.567165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b4430-b64e-43a0-b07a-080447ab417e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:59.567561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b4430-b64e-43a0-b07a-080447ab417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:59.574117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc2be35c23442d29f8cae35e6728810'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.576550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc2be35c23442d29f8cae35e6728810'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:47:59.590793 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 675/1365] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:47:59 2022] DELETE /identity/v3/users/5dc2be35c23442d29f8cae35e6728810 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:47:59.591871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:59.596820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:59.597567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9f4b4430-b64e-43a0-b07a-080447ab417e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:59.599950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9f4b4430-b64e-43a0-b07a-080447ab417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:59.694400 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 691/1366] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:59.698554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f288-bc90-484d-b9a1-72557a021073 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:59.699210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f288-bc90-484d-b9a1-72557a021073 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:59.699530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f288-bc90-484d-b9a1-72557a021073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:59.732004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c606f288-bc90-484d-b9a1-72557a021073 None None] MFA Rules not processed for user `4b16bd0809f3411a91fae6c5e2f3acee`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:59.733226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c606f288-bc90-484d-b9a1-72557a021073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:59.736652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926bccc3-7b41-494e-a245-6b015d60a26a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:59.755096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f14c2-da8d-4bee-b8cd-954622add196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:59.756078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f14c2-da8d-4bee-b8cd-954622add196 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:59.756379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f14c2-da8d-4bee-b8cd-954622add196 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:59.756805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f14c2-da8d-4bee-b8cd-954622add196 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:59.757736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f14c2-da8d-4bee-b8cd-954622add196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:47:59.763340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f14c2-da8d-4bee-b8cd-954622add196 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:47:59.812740 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 692/1367] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:59.814120 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 676/1368] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:47:59 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:47:59.815168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606f288-bc90-484d-b9a1-72557a021073 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:59.815783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761bd3e-86f4-4f94-bfa3-02d450a91643 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:59.816068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761bd3e-86f4-4f94-bfa3-02d450a91643 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:59.816337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761bd3e-86f4-4f94-bfa3-02d450a91643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:47:59.818619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c606f288-bc90-484d-b9a1-72557a021073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:59.853835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4761bd3e-86f4-4f94-bfa3-02d450a91643 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:47:59.855135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4761bd3e-86f4-4f94-bfa3-02d450a91643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:59.865863 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 677/1369] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:47:59.867002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761bd3e-86f4-4f94-bfa3-02d450a91643 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:47:59.869234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4761bd3e-86f4-4f94-bfa3-02d450a91643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:47:59.952567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:59.975851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761bd3e-86f4-4f94-bfa3-02d450a91643 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:47:59.986443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:59.986994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869858-2be8-45b2-ab0e-7a3bdfce2653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:47:59.988108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869858-2be8-45b2-ab0e-7a3bdfce2653 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:47:59.988824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869858-2be8-45b2-ab0e-7a3bdfce2653 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:47:59.989793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869858-2be8-45b2-ab0e-7a3bdfce2653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.005647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e869858-2be8-45b2-ab0e-7a3bdfce2653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:00.006927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e869858-2be8-45b2-ab0e-7a3bdfce2653 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:00.021162 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 678/1370] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:00.022407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869858-2be8-45b2-ab0e-7a3bdfce2653 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:00.031820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606f288-bc90-484d-b9a1-72557a021073 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:00.035290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869858-2be8-45b2-ab0e-7a3bdfce2653 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:00.041632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74eb43-3a8a-4480-87a2-f537ba57626f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:00.042387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74eb43-3a8a-4480-87a2-f537ba57626f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.042735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74eb43-3a8a-4480-87a2-f537ba57626f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.043279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74eb43-3a8a-4480-87a2-f537ba57626f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.045234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74eb43-3a8a-4480-87a2-f537ba57626f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:00.046138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74eb43-3a8a-4480-87a2-f537ba57626f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:00.046651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfc6a6-babe-4eab-a3b9-d09049e8c9b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:00.047286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfc6a6-babe-4eab-a3b9-d09049e8c9b9 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.047560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfc6a6-babe-4eab-a3b9-d09049e8c9b9 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.050462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfc6a6-babe-4eab-a3b9-d09049e8c9b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.055357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c5cfc6a6-babe-4eab-a3b9-d09049e8c9b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:00.063171 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 679/1371] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:00.068628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74eb43-3a8a-4480-87a2-f537ba57626f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:00.076686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-db74eb43-3a8a-4480-87a2-f537ba57626f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:00.122513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74eb43-3a8a-4480-87a2-f537ba57626f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:00.141205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864009b-fbe9-436e-8a7e-364e5d29f120 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:00.142292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864009b-fbe9-436e-8a7e-364e5d29f120 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.144115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864009b-fbe9-436e-8a7e-364e5d29f120 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.144908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864009b-fbe9-436e-8a7e-364e5d29f120 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.150641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8864009b-fbe9-436e-8a7e-364e5d29f120 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:00.176207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864009b-fbe9-436e-8a7e-364e5d29f120 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:00.181999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864009b-fbe9-436e-8a7e-364e5d29f120 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:00.204973 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 680/1372] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:00.205875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864009b-fbe9-436e-8a7e-364e5d29f120 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:00.209564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfc6a6-babe-4eab-a3b9-d09049e8c9b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:00.213728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfc6a6-babe-4eab-a3b9-d09049e8c9b9 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:00.235600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864009b-fbe9-436e-8a7e-364e5d29f120 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:00.236992 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 693/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:47:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:00.239110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfc6a6-babe-4eab-a3b9-d09049e8c9b9 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:00.245343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b64277-ec7b-4dac-976c-7d3b9b71f170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:00.245983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b64277-ec7b-4dac-976c-7d3b9b71f170 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.246260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b64277-ec7b-4dac-976c-7d3b9b71f170 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.246524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b64277-ec7b-4dac-976c-7d3b9b71f170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.266026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfc6a6-babe-4eab-a3b9-d09049e8c9b9 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:00.272162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b64277-ec7b-4dac-976c-7d3b9b71f170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:00.275282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b64277-ec7b-4dac-976c-7d3b9b71f170 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:00.281808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997a271-fec2-4b0e-9da7-fdcbf250d170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:00.282765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997a271-fec2-4b0e-9da7-fdcbf250d170 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.282878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997a271-fec2-4b0e-9da7-fdcbf250d170 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.283196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997a271-fec2-4b0e-9da7-fdcbf250d170 admin admin] PATH_INFO: `/v3/projects/99711b076aba40b39ffccc514b6542c9/users/5649d8e2f3e54fbc8aa86d743240b41b/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.301493 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 681/1374] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:00.302459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b64277-ec7b-4dac-976c-7d3b9b71f170 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:00.306682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-29b64277-ec7b-4dac-976c-7d3b9b71f170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:00.337422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997a271-fec2-4b0e-9da7-fdcbf250d170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99711b076aba40b39ffccc514b6542c9, user_id=5649d8e2f3e54fbc8aa86d743240b41b, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:00.344281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997a271-fec2-4b0e-9da7-fdcbf250d170 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:00.355496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b64277-ec7b-4dac-976c-7d3b9b71f170 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:00.370064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:00.375945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.377250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.377680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] PATH_INFO: `/v3/projects/bfde48d2eac54169b872ab50c329039e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.378134 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 694/1375] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:00 2022] PUT /identity/v3/projects/99711b076aba40b39ffccc514b6542c9/users/5649d8e2f3e54fbc8aa86d743240b41b/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:00.379495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0949706-9eca-4a53-ba1d-13a402f6e403 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.379776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0949706-9eca-4a53-ba1d-13a402f6e403 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.380049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0949706-9eca-4a53-ba1d-13a402f6e403 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.384507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfde48d2eac54169b872ab50c329039e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:00.385382 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 695/1376] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:48:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:00.386296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0949706-9eca-4a53-ba1d-13a402f6e403 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:00.389920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:00.493207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfde48d2eac54169b872ab50c329039e', 'request_id': 'req-b2d1b336-cef3-4fb1-bbba-871b403d26c9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:00.521696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfde48d2eac54169b872ab50c329039e', 'request_id': 'req-b2d1b336-cef3-4fb1-bbba-871b403d26c9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:00.540578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfde48d2eac54169b872ab50c329039e', 'request_id': 'req-b2d1b336-cef3-4fb1-bbba-871b403d26c9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:00.547233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0949706-9eca-4a53-ba1d-13a402f6e403 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:00.562288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] The token cache is being invalidate because project bfde48d2eac54169b872ab50c329039e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:00.562646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:00.563354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2d1b336-cef3-4fb1-bbba-871b403d26c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:00.566608 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 682/1377] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:00 2022] DELETE /identity/v3/projects/bfde48d2eac54169b872ab50c329039e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:00.568240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066623c7-fc7f-4f84-8d57-57ea0543dd40 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.570060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066623c7-fc7f-4f84-8d57-57ea0543dd40 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.570712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066623c7-fc7f-4f84-8d57-57ea0543dd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.626749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-066623c7-fc7f-4f84-8d57-57ea0543dd40 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:00.628644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-066623c7-fc7f-4f84-8d57-57ea0543dd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:00.704590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6bd3c-5c92-4fe7-96c4-b7e70e7b6c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:00.705605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6bd3c-5c92-4fe7-96c4-b7e70e7b6c79 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.705880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6bd3c-5c92-4fe7-96c4-b7e70e7b6c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.706148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6bd3c-5c92-4fe7-96c4-b7e70e7b6c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.707124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6bd3c-5c92-4fe7-96c4-b7e70e7b6c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:00.709062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6bd3c-5c92-4fe7-96c4-b7e70e7b6c79 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:00.741285 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 683/1378] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:48:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:00.748243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772e5b2-f3c3-45c3-8f6f-e8bc3430646a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.748243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772e5b2-f3c3-45c3-8f6f-e8bc3430646a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.748415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772e5b2-f3c3-45c3-8f6f-e8bc3430646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.777226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4772e5b2-f3c3-45c3-8f6f-e8bc3430646a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:00.777471 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 696/1379] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:00.777707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6bd3c-5c92-4fe7-96c4-b7e70e7b6c79 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:00.777828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4772e5b2-f3c3-45c3-8f6f-e8bc3430646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:00.779654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c4d6bd3c-5c92-4fe7-96c4-b7e70e7b6c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:00.794938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 684/1380] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:00.796055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772e5b2-f3c3-45c3-8f6f-e8bc3430646a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:00.797780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4772e5b2-f3c3-45c3-8f6f-e8bc3430646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:00.809715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6bd3c-5c92-4fe7-96c4-b7e70e7b6c79 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:00.834846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:00.838723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:00.838723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:00.838723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] PATH_INFO: `/v3/projects/b002a399b040452ba66910f5961ad6de` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:00.839827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b002a399b040452ba66910f5961ad6de)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:00.840745 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 20:48:00.840745 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 20:48:00.840745 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 20:48:00.841960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:00.857418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:00.875446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:00.965469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b002a399b040452ba66910f5961ad6de', 'request_id': 'req-77c2a589-090a-4f79-b832-ba509159d179'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.002816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b002a399b040452ba66910f5961ad6de', 'request_id': 'req-77c2a589-090a-4f79-b832-ba509159d179'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.045993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772e5b2-f3c3-45c3-8f6f-e8bc3430646a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:01.046386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b002a399b040452ba66910f5961ad6de', 'request_id': 'req-77c2a589-090a-4f79-b832-ba509159d179'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.084392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] The token cache is being invalidate because project b002a399b040452ba66910f5961ad6de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:01.086727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.089931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.095613 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 697/1381] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:00 2022] DELETE /identity/v3/projects/b002a399b040452ba66910f5961ad6de => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:01.107259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:01.112139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:01.160971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4840142-52d7-4f42-a669-6c0984058d22 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:01.162001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4840142-52d7-4f42-a669-6c0984058d22 service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:01.162408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4840142-52d7-4f42-a669-6c0984058d22 service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:01.162993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4840142-52d7-4f42-a669-6c0984058d22 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:01.166030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4840142-52d7-4f42-a669-6c0984058d22 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:01.288472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2a589-090a-4f79-b832-ba509159d179 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:01.301946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec627e16-e78c-4de6-b7f4-7fa9af61d588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:01.302640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec627e16-e78c-4de6-b7f4-7fa9af61d588 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:01.303415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec627e16-e78c-4de6-b7f4-7fa9af61d588 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:01.303694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec627e16-e78c-4de6-b7f4-7fa9af61d588 admin admin] PATH_INFO: `/v3/projects/99711b076aba40b39ffccc514b6542c9/users/5649d8e2f3e54fbc8aa86d743240b41b/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:01.306782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec627e16-e78c-4de6-b7f4-7fa9af61d588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99711b076aba40b39ffccc514b6542c9, user_id=5649d8e2f3e54fbc8aa86d743240b41b, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:01.312177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec627e16-e78c-4de6-b7f4-7fa9af61d588 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:01.339815 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 698/1382] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:01 2022] PUT /identity/v3/projects/99711b076aba40b39ffccc514b6542c9/users/5649d8e2f3e54fbc8aa86d743240b41b/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:01.340178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3e869-166e-49fd-829a-5a7650a7a906 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:01.340877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3e869-166e-49fd-829a-5a7650a7a906 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:01.347583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3e869-166e-49fd-829a-5a7650a7a906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:01.376018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4840142-52d7-4f42-a669-6c0984058d22 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:01.379900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4840142-52d7-4f42-a669-6c0984058d22 service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:01.393303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-06c3e869-166e-49fd-829a-5a7650a7a906 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:01.394538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-06c3e869-166e-49fd-829a-5a7650a7a906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:01.420017 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 685/1383] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:01.422919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4840142-52d7-4f42-a669-6c0984058d22 service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:01.425900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4840142-52d7-4f42-a669-6c0984058d22 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:01.446492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4840142-52d7-4f42-a669-6c0984058d22 service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:01.507363 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 699/1384] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:01.508994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3e869-166e-49fd-829a-5a7650a7a906 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:01.511788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-06c3e869-166e-49fd-829a-5a7650a7a906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:01.527937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3e869-166e-49fd-829a-5a7650a7a906 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:01.534878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:01.544707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:01.549658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:01.564631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:01.565414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:01.565692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:01.565964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] PATH_INFO: `/v3/users/0bcee660c15d444d9c502256c10f5ce8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:01.567125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bcee660c15d444d9c502256c10f5ce8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:01.569320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:01.572087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] PATH_INFO: `/v3/projects/528b4193b251489e91659997e5723d08` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:01.574262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:01.574981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=528b4193b251489e91659997e5723d08)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:01.578223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:01.658720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bcee660c15d444d9c502256c10f5ce8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.660397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:01.689436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bcee660c15d444d9c502256c10f5ce8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.704663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528b4193b251489e91659997e5723d08', 'request_id': 'req-b860652f-e391-4461-939f-d0a7cc3bacfe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.713529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bcee660c15d444d9c502256c10f5ce8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.719217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528b4193b251489e91659997e5723d08', 'request_id': 'req-b860652f-e391-4461-939f-d0a7cc3bacfe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.729321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bcee660c15d444d9c502256c10f5ce8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.730348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c49396d0-c721-4ac4-af01-b193b14f767b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bcee660c15d444d9c502256c10f5ce8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.737461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528b4193b251489e91659997e5723d08', 'request_id': 'req-b860652f-e391-4461-939f-d0a7cc3bacfe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.746536 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 686/1385] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:01 2022] DELETE /identity/v3/users/0bcee660c15d444d9c502256c10f5ce8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:01.746536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51880dba-cf5f-41b8-9ef4-e9d9d9bd3b4a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:01.746769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51880dba-cf5f-41b8-9ef4-e9d9d9bd3b4a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:01.748241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51880dba-cf5f-41b8-9ef4-e9d9d9bd3b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:01.764246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] The token cache is being invalidate because project 528b4193b251489e91659997e5723d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:01.764740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.765971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:01.768122 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 700/1386] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:01 2022] DELETE /identity/v3/projects/528b4193b251489e91659997e5723d08 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:01.769555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:01.771806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:01.809016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-51880dba-cf5f-41b8-9ef4-e9d9d9bd3b4a None None] MFA Rules not processed for user `5649d8e2f3e54fbc8aa86d743240b41b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:01.810566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-51880dba-cf5f-41b8-9ef4-e9d9d9bd3b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:01.934998 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 687/1387] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:01.936103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51880dba-cf5f-41b8-9ef4-e9d9d9bd3b4a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:01.938275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-51880dba-cf5f-41b8-9ef4-e9d9d9bd3b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:01.968144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860652f-e391-4461-939f-d0a7cc3bacfe admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:02.066071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:02.069269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51880dba-cf5f-41b8-9ef4-e9d9d9bd3b4a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:02.083551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:02.084328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:02.084812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:02.085347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] PATH_INFO: `/v3/users/110fc0e5be3141caa029822a601ff2ed` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:02.087546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=110fc0e5be3141caa029822a601ff2ed)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:02.092257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:02.149019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c526628-f9a2-431b-ac3c-32042234ba01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:02.149739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c526628-f9a2-431b-ac3c-32042234ba01 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:02.150073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c526628-f9a2-431b-ac3c-32042234ba01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:02.150378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c526628-f9a2-431b-ac3c-32042234ba01 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:02.151156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c526628-f9a2-431b-ac3c-32042234ba01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:02.151457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c526628-f9a2-431b-ac3c-32042234ba01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:02.153376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c526628-f9a2-431b-ac3c-32042234ba01 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:02.178770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110fc0e5be3141caa029822a601ff2ed'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:02.182625 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 701/1388] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:02.183501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c526628-f9a2-431b-ac3c-32042234ba01 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:02.202129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3c526628-f9a2-431b-ac3c-32042234ba01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:02.220954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110fc0e5be3141caa029822a601ff2ed'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:02.234576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110fc0e5be3141caa029822a601ff2ed'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:02.247581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110fc0e5be3141caa029822a601ff2ed'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:02.250196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110fc0e5be3141caa029822a601ff2ed'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:02.261085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 688/1389] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:01 2022] DELETE /identity/v3/users/110fc0e5be3141caa029822a601ff2ed => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:02.262807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:02.265606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:02.312473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:02.362776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c526628-f9a2-431b-ac3c-32042234ba01 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:02.507076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea923f67-06db-43d0-bba5-4116cf348d2d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:02.512722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd5285-dc46-49e9-ace5-58915ee577e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:02.515474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd5285-dc46-49e9-ace5-58915ee577e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:02.515901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd5285-dc46-49e9-ace5-58915ee577e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:02.516374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd5285-dc46-49e9-ace5-58915ee577e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:02.517185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd5285-dc46-49e9-ace5-58915ee577e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:02.520153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd5285-dc46-49e9-ace5-58915ee577e5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:02.544790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a5bf7-7893-4a28-a7b9-eb02fa3b300c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:02.545747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a5bf7-7893-4a28-a7b9-eb02fa3b300c service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:02.546376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a5bf7-7893-4a28-a7b9-eb02fa3b300c service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:02.546862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a5bf7-7893-4a28-a7b9-eb02fa3b300c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:02.549596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-149a5bf7-7893-4a28-a7b9-eb02fa3b300c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:02.593491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 689/1390] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:02 2022] POST /identity/v3/projects => generated 385 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:02.594857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2567e97-2568-4ded-b3c1-21fe31904f65 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:02.595132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2567e97-2568-4ded-b3c1-21fe31904f65 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:02.595394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2567e97-2568-4ded-b3c1-21fe31904f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:02.674042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b2567e97-2568-4ded-b3c1-21fe31904f65 None None] MFA Rules not processed for user `14bd717b05ce4e40a1cd54c2db8474a6`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:02.675394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b2567e97-2568-4ded-b3c1-21fe31904f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:02.786313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:02.841146 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 690/1391] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:02 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:02.843539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2567e97-2568-4ded-b3c1-21fe31904f65 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:02.877627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a5bf7-7893-4a28-a7b9-eb02fa3b300c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:02.879385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a5bf7-7893-4a28-a7b9-eb02fa3b300c service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:02.909122 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 702/1392] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:02.910686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a5bf7-7893-4a28-a7b9-eb02fa3b300c service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:02.912416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-149a5bf7-7893-4a28-a7b9-eb02fa3b300c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:02.997200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2567e97-2568-4ded-b3c1-21fe31904f65 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.003658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa488e-d88e-4826-898b-331d5abcb773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.004397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa488e-d88e-4826-898b-331d5abcb773 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.004670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa488e-d88e-4826-898b-331d5abcb773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.005138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa488e-d88e-4826-898b-331d5abcb773 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.005917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa488e-d88e-4826-898b-331d5abcb773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.007833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa488e-d88e-4826-898b-331d5abcb773 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.022579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a5bf7-7893-4a28-a7b9-eb02fa3b300c service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.033166 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 691/1393] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:02 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:03.033166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa488e-d88e-4826-898b-331d5abcb773 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.034551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5bfa488e-d88e-4826-898b-331d5abcb773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.047774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa488e-d88e-4826-898b-331d5abcb773 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.052458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f06651-7258-4afa-ba44-8bb682eb36f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.053022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f06651-7258-4afa-ba44-8bb682eb36f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.053326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f06651-7258-4afa-ba44-8bb682eb36f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.053591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f06651-7258-4afa-ba44-8bb682eb36f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.054255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f06651-7258-4afa-ba44-8bb682eb36f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.055462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:03.056303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f06651-7258-4afa-ba44-8bb682eb36f6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.083829 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 692/1394] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:03 2022] POST /identity/v3/projects => generated 401 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:03.084707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f06651-7258-4afa-ba44-8bb682eb36f6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.089354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-22f06651-7258-4afa-ba44-8bb682eb36f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.090920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258abc-a234-485e-93f5-71949f90d290 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.091644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258abc-a234-485e-93f5-71949f90d290 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.091964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258abc-a234-485e-93f5-71949f90d290 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.092320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258abc-a234-485e-93f5-71949f90d290 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.094156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-32258abc-a234-485e-93f5-71949f90d290 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.106679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258abc-a234-485e-93f5-71949f90d290 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.108246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258abc-a234-485e-93f5-71949f90d290 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.118173 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 703/1395] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:03.118901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258abc-a234-485e-93f5-71949f90d290 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.120289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-32258abc-a234-485e-93f5-71949f90d290 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.133673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258abc-a234-485e-93f5-71949f90d290 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.156803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f06651-7258-4afa-ba44-8bb682eb36f6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.162457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.163831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.165915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.166861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] PATH_INFO: `/v3/projects/3c51574deb904dce89e171e4f52ab665` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.169151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c51574deb904dce89e171e4f52ab665)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.172181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.215839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.216700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.217113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.217452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] PATH_INFO: `/v3/users/7ec562be10db4c2fb57476fda5a18867` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.218653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ec562be10db4c2fb57476fda5a18867)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.220649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.232458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c51574deb904dce89e171e4f52ab665', 'request_id': 'req-eb6a6b29-8e90-4b51-b4c6-86885f878e70'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.240851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c51574deb904dce89e171e4f52ab665', 'request_id': 'req-eb6a6b29-8e90-4b51-b4c6-86885f878e70'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.249821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c51574deb904dce89e171e4f52ab665', 'request_id': 'req-eb6a6b29-8e90-4b51-b4c6-86885f878e70'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.251315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:03.266499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] The token cache is being invalidate because project 3c51574deb904dce89e171e4f52ab665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:03.266854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.267677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.272717 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 693/1396] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:03 2022] DELETE /identity/v3/projects/3c51574deb904dce89e171e4f52ab665 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:03.272717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.274995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.290934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ec562be10db4c2fb57476fda5a18867'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.299563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ec562be10db4c2fb57476fda5a18867'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.314390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ec562be10db4c2fb57476fda5a18867'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.323135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ec562be10db4c2fb57476fda5a18867'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.323828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ec562be10db4c2fb57476fda5a18867'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.331110 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 704/1397] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:03 2022] DELETE /identity/v3/users/7ec562be10db4c2fb57476fda5a18867 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:03.332177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.333991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.353270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a6b29-8e90-4b51-b4c6-86885f878e70 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.403830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:03.442537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.443173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.443506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.443770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] PATH_INFO: `/v3/projects/3ea4f0b754b1409b8a39c6b335b33bae` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.446801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ea4f0b754b1409b8a39c6b335b33bae)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.449770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.454322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e3679-788a-44e5-bc09-c41412e54d37 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.464974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.466801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.467207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.467607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] PATH_INFO: `/v3/users/6b5a19d928dc47e48486a58359778635` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.470002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5a19d928dc47e48486a58359778635)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.479634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.514413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ea4f0b754b1409b8a39c6b335b33bae', 'request_id': 'req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.526435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ea4f0b754b1409b8a39c6b335b33bae', 'request_id': 'req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.536469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ea4f0b754b1409b8a39c6b335b33bae', 'request_id': 'req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.552060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5a19d928dc47e48486a58359778635'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.553729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] The token cache is being invalidate because project 3ea4f0b754b1409b8a39c6b335b33bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:03.554164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.555793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.557746 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 694/1398] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:03 2022] DELETE /identity/v3/projects/3ea4f0b754b1409b8a39c6b335b33bae => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:03.558819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.560174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.561292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5a19d928dc47e48486a58359778635'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.584685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5a19d928dc47e48486a58359778635'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.603094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5a19d928dc47e48486a58359778635'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.603769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5a19d928dc47e48486a58359778635'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:03.612967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:03.615591 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 705/1399] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:03 2022] DELETE /identity/v3/users/6b5a19d928dc47e48486a58359778635 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:03.616581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.619371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.665361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a62e5-896d-45fd-8c39-0c5b45e5fc67 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.712060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edf3b4-c21e-41ff-ba3c-92ddb7cb1e78 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.716877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdc297-f563-49db-a31c-d6526c76c124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.717539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdc297-f563-49db-a31c-d6526c76c124 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.717818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdc297-f563-49db-a31c-d6526c76c124 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.718086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdc297-f563-49db-a31c-d6526c76c124 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.718714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdc297-f563-49db-a31c-d6526c76c124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.721072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdc297-f563-49db-a31c-d6526c76c124 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.745005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de9d5f-45a1-437d-b6cb-090360a77081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.745560 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 706/1400] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:03 2022] POST /identity/v3/users => generated 307 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:03.747067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61521be-0924-45d4-b7f4-e65647ddacf2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.747437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61521be-0924-45d4-b7f4-e65647ddacf2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.747727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61521be-0924-45d4-b7f4-e65647ddacf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.749317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de9d5f-45a1-437d-b6cb-090360a77081 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.749617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de9d5f-45a1-437d-b6cb-090360a77081 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.749876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de9d5f-45a1-437d-b6cb-090360a77081 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.750557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de9d5f-45a1-437d-b6cb-090360a77081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.752806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de9d5f-45a1-437d-b6cb-090360a77081 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.765909 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 695/1401] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:03.767517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f675-9d6a-4224-b410-eda5225dac03 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.767975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f675-9d6a-4224-b410-eda5225dac03 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.768484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f675-9d6a-4224-b410-eda5225dac03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.771420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b61521be-0924-45d4-b7f4-e65647ddacf2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:03.772632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b61521be-0924-45d4-b7f4-e65647ddacf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.781799 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 707/1402] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:03.782909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797c43e-cc54-46e4-bf4f-43eacf626559 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.783183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797c43e-cc54-46e4-bf4f-43eacf626559 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.783439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797c43e-cc54-46e4-bf4f-43eacf626559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.793284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ac97f675-9d6a-4224-b410-eda5225dac03 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:03.794019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac97f675-9d6a-4224-b410-eda5225dac03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.805103 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 696/1403] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:03.806933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e63ff-0818-4e11-a3f0-2f8a71ff299a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.807286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e63ff-0818-4e11-a3f0-2f8a71ff299a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.807623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e63ff-0818-4e11-a3f0-2f8a71ff299a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.813010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f797c43e-cc54-46e4-bf4f-43eacf626559 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:03.815431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f797c43e-cc54-46e4-bf4f-43eacf626559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.826723 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 708/1404] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:03.827561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797c43e-cc54-46e4-bf4f-43eacf626559 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.834811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-646e63ff-0818-4e11-a3f0-2f8a71ff299a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:03.837215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-646e63ff-0818-4e11-a3f0-2f8a71ff299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.846783 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 697/1405] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:03.849976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e63ff-0818-4e11-a3f0-2f8a71ff299a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.850317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797c43e-cc54-46e4-bf4f-43eacf626559 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.850391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-646e63ff-0818-4e11-a3f0-2f8a71ff299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.856004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862daf75-9386-43d4-88e1-0c15383203c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.858021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862daf75-9386-43d4-88e1-0c15383203c2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.858021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862daf75-9386-43d4-88e1-0c15383203c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.858021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862daf75-9386-43d4-88e1-0c15383203c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.858191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862daf75-9386-43d4-88e1-0c15383203c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.859889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862daf75-9386-43d4-88e1-0c15383203c2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.867723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e63ff-0818-4e11-a3f0-2f8a71ff299a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.873105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad94cba-b259-47e5-b041-b6a9639a58e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.873105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad94cba-b259-47e5-b041-b6a9639a58e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.873105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad94cba-b259-47e5-b041-b6a9639a58e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.873402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad94cba-b259-47e5-b041-b6a9639a58e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.874056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad94cba-b259-47e5-b041-b6a9639a58e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:03.874422 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 709/1406] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:03.876082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a37e59-25a6-4404-9563-d6712f31c440 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.876603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a37e59-25a6-4404-9563-d6712f31c440 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.876765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a37e59-25a6-4404-9563-d6712f31c440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.879239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad94cba-b259-47e5-b041-b6a9639a58e5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:03.894056 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 698/1407] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:03.895747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad94cba-b259-47e5-b041-b6a9639a58e5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.898189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1ad94cba-b259-47e5-b041-b6a9639a58e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.902547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-52a37e59-25a6-4404-9563-d6712f31c440 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:03.904101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-52a37e59-25a6-4404-9563-d6712f31c440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.915212 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 710/1408] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:03.917825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426f7bc-7c7f-4c20-9880-24e21f89c403 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.918304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426f7bc-7c7f-4c20-9880-24e21f89c403 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.919119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426f7bc-7c7f-4c20-9880-24e21f89c403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:03.944780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a426f7bc-7c7f-4c20-9880-24e21f89c403 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:03.946678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a426f7bc-7c7f-4c20-9880-24e21f89c403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.956188 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 711/1409] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:03.958208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426f7bc-7c7f-4c20-9880-24e21f89c403 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:03.960229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a426f7bc-7c7f-4c20-9880-24e21f89c403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:03.978020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:03.981797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad94cba-b259-47e5-b041-b6a9639a58e5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:03.987984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab15eb-415d-47fb-963e-039d790d3d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:03.990097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab15eb-415d-47fb-963e-039d790d3d96 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:03.990228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab15eb-415d-47fb-963e-039d790d3d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:03.990398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab15eb-415d-47fb-963e-039d790d3d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.003075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab15eb-415d-47fb-963e-039d790d3d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.005549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab15eb-415d-47fb-963e-039d790d3d96 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.019501 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 699/1410] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:04.021420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab15eb-415d-47fb-963e-039d790d3d96 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.034271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab15eb-415d-47fb-963e-039d790d3d96 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.038159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426f7bc-7c7f-4c20-9880-24e21f89c403 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.040566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54373bbe-e567-4c53-86f3-a74a1a5f9474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.041271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54373bbe-e567-4c53-86f3-a74a1a5f9474 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.041750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54373bbe-e567-4c53-86f3-a74a1a5f9474 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.042014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54373bbe-e567-4c53-86f3-a74a1a5f9474 admin admin] PATH_INFO: `/v3/projects/e43279179c3b432cbf49db0581d3b8cc/users/b8343eb55cd148ef94141af734ef6abf/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.048715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2591527-26d7-43fe-992b-634508e4367a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.049573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2591527-26d7-43fe-992b-634508e4367a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.049851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2591527-26d7-43fe-992b-634508e4367a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.050182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2591527-26d7-43fe-992b-634508e4367a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.062510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54373bbe-e567-4c53-86f3-a74a1a5f9474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43279179c3b432cbf49db0581d3b8cc, user_id=b8343eb55cd148ef94141af734ef6abf, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.066325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2591527-26d7-43fe-992b-634508e4367a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.066820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54373bbe-e567-4c53-86f3-a74a1a5f9474 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.067858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2591527-26d7-43fe-992b-634508e4367a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.087683 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 712/1411] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:04.088751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2591527-26d7-43fe-992b-634508e4367a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.090014 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 700/1412] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:04 2022] PUT /identity/v3/projects/e43279179c3b432cbf49db0581d3b8cc/users/b8343eb55cd148ef94141af734ef6abf/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:04.094001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54373bbe-e567-4c53-86f3-a74a1a5f9474 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.101418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2591527-26d7-43fe-992b-634508e4367a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.193681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:04.221853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc82fb0-7475-4d46-9baa-a01ecb48665c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.222620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc82fb0-7475-4d46-9baa-a01ecb48665c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.222987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc82fb0-7475-4d46-9baa-a01ecb48665c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.223325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc82fb0-7475-4d46-9baa-a01ecb48665c admin admin] PATH_INFO: `/v3/projects/b6cefe34fe37400d98f8ef15da8df500/users/c3266dc808c84798a7156fe670ea5cad/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.242587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc82fb0-7475-4d46-9baa-a01ecb48665c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6cefe34fe37400d98f8ef15da8df500, user_id=c3266dc808c84798a7156fe670ea5cad, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.245902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc82fb0-7475-4d46-9baa-a01ecb48665c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.268892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 713/1413] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:04 2022] PUT /identity/v3/projects/b6cefe34fe37400d98f8ef15da8df500/users/c3266dc808c84798a7156fe670ea5cad/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:04.279166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc82fb0-7475-4d46-9baa-a01ecb48665c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.290624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54373bbe-e567-4c53-86f3-a74a1a5f9474 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.400516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4296d-fc27-4f4a-a6d0-d776136a7b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.402061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4296d-fc27-4f4a-a6d0-d776136a7b7d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.402394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc82fb0-7475-4d46-9baa-a01ecb48665c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.402955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4296d-fc27-4f4a-a6d0-d776136a7b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.403673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4296d-fc27-4f4a-a6d0-d776136a7b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.404638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4296d-fc27-4f4a-a6d0-d776136a7b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:04.405178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4296d-fc27-4f4a-a6d0-d776136a7b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.408346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4296d-fc27-4f4a-a6d0-d776136a7b7d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.419861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459e5b2-5c78-4955-8bc8-5b56f3e2a55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.420307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459e5b2-5c78-4955-8bc8-5b56f3e2a55d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.420580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459e5b2-5c78-4955-8bc8-5b56f3e2a55d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.420912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459e5b2-5c78-4955-8bc8-5b56f3e2a55d admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.425072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459e5b2-5c78-4955-8bc8-5b56f3e2a55d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:04.426418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459e5b2-5c78-4955-8bc8-5b56f3e2a55d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.428869 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 701/1414] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:04.432389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4296d-fc27-4f4a-a6d0-d776136a7b7d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.448495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459e5b2-5c78-4955-8bc8-5b56f3e2a55d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.465683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4296d-fc27-4f4a-a6d0-d776136a7b7d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.471525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2599bb-978d-403d-bc9a-34fd0d9ea22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.472410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2599bb-978d-403d-bc9a-34fd0d9ea22e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.472695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2599bb-978d-403d-bc9a-34fd0d9ea22e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.472963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2599bb-978d-403d-bc9a-34fd0d9ea22e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.473647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2599bb-978d-403d-bc9a-34fd0d9ea22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.475847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2599bb-978d-403d-bc9a-34fd0d9ea22e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.487617 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 714/1415] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:04.490251 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 702/1416] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:04.491520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2599bb-978d-403d-bc9a-34fd0d9ea22e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.492093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f260483-6549-4b9d-bd2f-46035fe6762f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.492438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f260483-6549-4b9d-bd2f-46035fe6762f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.492792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f260483-6549-4b9d-bd2f-46035fe6762f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.519702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2599bb-978d-403d-bc9a-34fd0d9ea22e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.530353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46856270-3831-464d-b674-8d050df25e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.543418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46856270-3831-464d-b674-8d050df25e27 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.543880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46856270-3831-464d-b674-8d050df25e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.546580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46856270-3831-464d-b674-8d050df25e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.547829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46856270-3831-464d-b674-8d050df25e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.552802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46856270-3831-464d-b674-8d050df25e27 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.553195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9f260483-6549-4b9d-bd2f-46035fe6762f None None] MFA Rules not processed for user `c3266dc808c84798a7156fe670ea5cad`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:04.555051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9f260483-6549-4b9d-bd2f-46035fe6762f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:04.581524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:04.609991 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 703/1417] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:04 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:04.609991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46856270-3831-464d-b674-8d050df25e27 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.609991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-46856270-3831-464d-b674-8d050df25e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:04.666509 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 715/1418] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:04.667345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f260483-6549-4b9d-bd2f-46035fe6762f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.683613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f260483-6549-4b9d-bd2f-46035fe6762f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.715732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46856270-3831-464d-b674-8d050df25e27 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.722363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.723147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.723531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.723857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] PATH_INFO: `/v3/projects/2da218e9269f428fa9a89771f41aa244` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.725389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da218e9269f428fa9a89771f41aa244)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.728174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.770763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8415a-35dc-49b9-9493-e4bf9f8ad00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.771556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8415a-35dc-49b9-9493-e4bf9f8ad00d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.771934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8415a-35dc-49b9-9493-e4bf9f8ad00d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.772270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8415a-35dc-49b9-9493-e4bf9f8ad00d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.772956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8415a-35dc-49b9-9493-e4bf9f8ad00d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.775905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8415a-35dc-49b9-9493-e4bf9f8ad00d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.776598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:04.791711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da218e9269f428fa9a89771f41aa244', 'request_id': 'req-f67f6029-6378-4584-8f08-4abff3d1864f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:04.802982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da218e9269f428fa9a89771f41aa244', 'request_id': 'req-f67f6029-6378-4584-8f08-4abff3d1864f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:04.810685 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 716/1419] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:04 2022] POST /identity/v3/projects => generated 391 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:04.810849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da218e9269f428fa9a89771f41aa244', 'request_id': 'req-f67f6029-6378-4584-8f08-4abff3d1864f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:04.811505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8415a-35dc-49b9-9493-e4bf9f8ad00d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.824704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] The token cache is being invalidate because project 2da218e9269f428fa9a89771f41aa244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:04.824982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:04.825858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:04.829130 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 704/1420] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:04 2022] DELETE /identity/v3/projects/2da218e9269f428fa9a89771f41aa244 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:04.830941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.834648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:04.892411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8415a-35dc-49b9-9493-e4bf9f8ad00d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.897392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf14fe-7197-4380-803e-2be321afe181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.898146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf14fe-7197-4380-803e-2be321afe181 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.899077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf14fe-7197-4380-803e-2be321afe181 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.899358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf14fe-7197-4380-803e-2be321afe181 admin admin] PATH_INFO: `/v3/projects/e43279179c3b432cbf49db0581d3b8cc/users/b8343eb55cd148ef94141af734ef6abf/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.901309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf14fe-7197-4380-803e-2be321afe181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43279179c3b432cbf49db0581d3b8cc, user_id=b8343eb55cd148ef94141af734ef6abf, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.904004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf14fe-7197-4380-803e-2be321afe181 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.906303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f6029-6378-4584-8f08-4abff3d1864f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:04.911269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d95a7-85f4-49b0-ad45-8c30fe72d036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:04.912231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d95a7-85f4-49b0-ad45-8c30fe72d036 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:04.912504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d95a7-85f4-49b0-ad45-8c30fe72d036 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:04.912799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d95a7-85f4-49b0-ad45-8c30fe72d036 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:04.913457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d95a7-85f4-49b0-ad45-8c30fe72d036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:04.915413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d95a7-85f4-49b0-ad45-8c30fe72d036 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:04.916157 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 717/1421] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:04 2022] PUT /identity/v3/projects/e43279179c3b432cbf49db0581d3b8cc/users/b8343eb55cd148ef94141af734ef6abf/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:04.916771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf14fe-7197-4380-803e-2be321afe181 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.922540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f0bf14fe-7197-4380-803e-2be321afe181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:04.944751 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 705/1422] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:04 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:04.944751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d95a7-85f4-49b0-ad45-8c30fe72d036 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:04.948132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4c0d95a7-85f4-49b0-ad45-8c30fe72d036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:04.965267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d95a7-85f4-49b0-ad45-8c30fe72d036 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:05.030413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:05.042812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429e159-3e3e-4e53-a38b-65cb70ad8321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:05.043637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429e159-3e3e-4e53-a38b-65cb70ad8321 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.043931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429e159-3e3e-4e53-a38b-65cb70ad8321 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.044395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429e159-3e3e-4e53-a38b-65cb70ad8321 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.045992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429e159-3e3e-4e53-a38b-65cb70ad8321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:05.049024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429e159-3e3e-4e53-a38b-65cb70ad8321 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:05.083575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf14fe-7197-4380-803e-2be321afe181 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:05.094352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:05.095110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.095459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.095599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] PATH_INFO: `/v3/projects/732d361e9c7041cea306ecc4805ffdaf` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.102425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] RBAC: Authorizing `identity:delete_project(project_id=732d361e9c7041cea306ecc4805ffdaf)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:05.104748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:05.111400 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 706/1423] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:04 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:05.112550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429e159-3e3e-4e53-a38b-65cb70ad8321 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:05.120343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9429e159-3e3e-4e53-a38b-65cb70ad8321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:05.235984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:05.284039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '732d361e9c7041cea306ecc4805ffdaf', 'request_id': 'req-2e26302e-aac1-4076-8902-2882cc00448b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.302036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '732d361e9c7041cea306ecc4805ffdaf', 'request_id': 'req-2e26302e-aac1-4076-8902-2882cc00448b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.304117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429e159-3e3e-4e53-a38b-65cb70ad8321 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:05.311642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '732d361e9c7041cea306ecc4805ffdaf', 'request_id': 'req-2e26302e-aac1-4076-8902-2882cc00448b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.328012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] The token cache is being invalidate because project 732d361e9c7041cea306ecc4805ffdaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:05.328340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.329379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.333880 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 718/1424] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:04 2022] DELETE /identity/v3/projects/732d361e9c7041cea306ecc4805ffdaf => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:05.335283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:05.337479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:05.438766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b99af-bce7-419c-9f09-1b322ec105bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:05.439611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b99af-bce7-419c-9f09-1b322ec105bf admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.440435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b99af-bce7-419c-9f09-1b322ec105bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.440779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b99af-bce7-419c-9f09-1b322ec105bf admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.441967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b99af-bce7-419c-9f09-1b322ec105bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:05.444804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b99af-bce7-419c-9f09-1b322ec105bf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:05.487070 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 707/1425] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:05 2022] POST /identity/v3/users => generated 296 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:05.488066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b99af-bce7-419c-9f09-1b322ec105bf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:05.488870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26302e-aac1-4076-8902-2882cc00448b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:05.489908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9f3b99af-bce7-419c-9f09-1b322ec105bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:05.494510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:05.495374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.495653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.495920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] PATH_INFO: `/v3/users/5649d8e2f3e54fbc8aa86d743240b41b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.497513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5649d8e2f3e54fbc8aa86d743240b41b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:05.499563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:05.505602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b99af-bce7-419c-9f09-1b322ec105bf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:05.515291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427fae9-9b81-4c28-b885-df4593500779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:05.516243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427fae9-9b81-4c28-b885-df4593500779 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.517260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427fae9-9b81-4c28-b885-df4593500779 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.517545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427fae9-9b81-4c28-b885-df4593500779 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.518660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427fae9-9b81-4c28-b885-df4593500779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:05.520296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427fae9-9b81-4c28-b885-df4593500779 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:05.542037 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 708/1426] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:05.543529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427fae9-9b81-4c28-b885-df4593500779 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:05.546552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8427fae9-9b81-4c28-b885-df4593500779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:05.563039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427fae9-9b81-4c28-b885-df4593500779 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:05.568814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f8c7f-9ecd-42d8-89a7-95c16f36c1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:05.569787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f8c7f-9ecd-42d8-89a7-95c16f36c1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.570589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f8c7f-9ecd-42d8-89a7-95c16f36c1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.570866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f8c7f-9ecd-42d8-89a7-95c16f36c1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.571680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f8c7f-9ecd-42d8-89a7-95c16f36c1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:05.574260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f8c7f-9ecd-42d8-89a7-95c16f36c1f2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:05.589680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5649d8e2f3e54fbc8aa86d743240b41b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.608220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5649d8e2f3e54fbc8aa86d743240b41b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.630407 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 709/1427] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:05 2022] POST /identity/v3/users => generated 299 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:05.631293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f8c7f-9ecd-42d8-89a7-95c16f36c1f2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:05.639568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-202f8c7f-9ecd-42d8-89a7-95c16f36c1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:05.644868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5649d8e2f3e54fbc8aa86d743240b41b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.649013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:05.660859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5649d8e2f3e54fbc8aa86d743240b41b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.661821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e993536f-3a4d-4246-ae70-a0d85731c9f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5649d8e2f3e54fbc8aa86d743240b41b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.667354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f8c7f-9ecd-42d8-89a7-95c16f36c1f2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:05.672968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:05.673717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.673991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.674414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] PATH_INFO: `/v3/users/fe181325988b46d6a8316868736310c4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.675662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe181325988b46d6a8316868736310c4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:05.678263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:05.679015 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 719/1428] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:05 2022] DELETE /identity/v3/users/5649d8e2f3e54fbc8aa86d743240b41b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:05.681077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed285a-a1a3-4ca9-9fe8-f2d3d02320be None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.684949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed285a-a1a3-4ca9-9fe8-f2d3d02320be None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.685347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed285a-a1a3-4ca9-9fe8-f2d3d02320be None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.687182 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 720/1429] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:48:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:05.687965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed285a-a1a3-4ca9-9fe8-f2d3d02320be None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:05.694213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f1ed285a-a1a3-4ca9-9fe8-f2d3d02320be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:05.750999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe181325988b46d6a8316868736310c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.769903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe181325988b46d6a8316868736310c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.787777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe181325988b46d6a8316868736310c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.801265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe181325988b46d6a8316868736310c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.803398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe181325988b46d6a8316868736310c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:05.814450 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 710/1430] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:05 2022] DELETE /identity/v3/users/fe181325988b46d6a8316868736310c4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:05.815759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:05.817270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:05.843886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed285a-a1a3-4ca9-9fe8-f2d3d02320be None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:05.853290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:05.932440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc268f-12c7-4453-9486-2308f486036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:05.933163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc268f-12c7-4453-9486-2308f486036f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.933611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc268f-12c7-4453-9486-2308f486036f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.934059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc268f-12c7-4453-9486-2308f486036f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.934920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc268f-12c7-4453-9486-2308f486036f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:05.936731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc268f-12c7-4453-9486-2308f486036f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:05.948870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce447c-bbcc-4104-b290-585febf3941d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:05.950009 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 721/1431] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:05.951174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc268f-12c7-4453-9486-2308f486036f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:05.954120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1bcc268f-12c7-4453-9486-2308f486036f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:05.963388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb53c37-1249-46f1-88a6-39f6e4c35070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:05.965493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb53c37-1249-46f1-88a6-39f6e4c35070 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.965986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb53c37-1249-46f1-88a6-39f6e4c35070 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.967423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb53c37-1249-46f1-88a6-39f6e4c35070 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.968174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb53c37-1249-46f1-88a6-39f6e4c35070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:05.970064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb53c37-1249-46f1-88a6-39f6e4c35070 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:05.978327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc268f-12c7-4453-9486-2308f486036f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:05.983791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c303e-b641-4eee-b51d-ab20255f3036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:05.984630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c303e-b641-4eee-b51d-ab20255f3036 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.985116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c303e-b641-4eee-b51d-ab20255f3036 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.985489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c303e-b641-4eee-b51d-ab20255f3036 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:05.986331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c303e-b641-4eee-b51d-ab20255f3036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:05.988184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c303e-b641-4eee-b51d-ab20255f3036 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:05.991519 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 711/1432] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:05.993319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a875b3e-0597-4f1c-a41a-8d9b0e00bc9c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:05.993436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a875b3e-0597-4f1c-a41a-8d9b0e00bc9c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:05.993725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a875b3e-0597-4f1c-a41a-8d9b0e00bc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:06.013568 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 722/1433] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:06.015039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f66a06-0591-4062-819f-8bc3c06de75c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:06.015335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f66a06-0591-4062-819f-8bc3c06de75c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:06.015625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f66a06-0591-4062-819f-8bc3c06de75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:06.041015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-47f66a06-0591-4062-819f-8bc3c06de75c None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:06.042263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-47f66a06-0591-4062-819f-8bc3c06de75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:06.066802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1a875b3e-0597-4f1c-a41a-8d9b0e00bc9c None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:06.069063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1a875b3e-0597-4f1c-a41a-8d9b0e00bc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:06.150811 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 712/1434] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:48:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:06.152409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a875b3e-0597-4f1c-a41a-8d9b0e00bc9c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:06.154133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1a875b3e-0597-4f1c-a41a-8d9b0e00bc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:06.156636 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 723/1435] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:48:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:06.157877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f66a06-0591-4062-819f-8bc3c06de75c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:06.175924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f66a06-0591-4062-819f-8bc3c06de75c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:06.185303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df0ea-5ec9-43dc-a270-113191a9f687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:06.186153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df0ea-5ec9-43dc-a270-113191a9f687 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:06.186954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df0ea-5ec9-43dc-a270-113191a9f687 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:06.187352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df0ea-5ec9-43dc-a270-113191a9f687 admin admin] PATH_INFO: `/v3/projects/c126854c21954928af16db403a0b8919/users/c5a178b6a74140d4910f77d811febc08/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:06.202637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a875b3e-0597-4f1c-a41a-8d9b0e00bc9c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:06.208973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:06.213546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068df0ea-5ec9-43dc-a270-113191a9f687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c126854c21954928af16db403a0b8919, user_id=c5a178b6a74140d4910f77d811febc08, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:06.216528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068df0ea-5ec9-43dc-a270-113191a9f687 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:06.232291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb621f4-6a4c-42b4-9036-1788d184ef6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:06.233256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb621f4-6a4c-42b4-9036-1788d184ef6a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:06.233256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb621f4-6a4c-42b4-9036-1788d184ef6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:06.233479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb621f4-6a4c-42b4-9036-1788d184ef6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:06.234630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb621f4-6a4c-42b4-9036-1788d184ef6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:06.235413 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 724/1436] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:06 2022] PUT /identity/v3/projects/c126854c21954928af16db403a0b8919/users/c5a178b6a74140d4910f77d811febc08/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:06.236432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df0ea-5ec9-43dc-a270-113191a9f687 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:06.236645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb621f4-6a4c-42b4-9036-1788d184ef6a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:06.254649 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 713/1437] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:06.260747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb621f4-6a4c-42b4-9036-1788d184ef6a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:06.263426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cfb621f4-6a4c-42b4-9036-1788d184ef6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:06.370045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df0ea-5ec9-43dc-a270-113191a9f687 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:06.373462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb621f4-6a4c-42b4-9036-1788d184ef6a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:06.375573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0bc62a-083e-4fca-a40a-77a651aaf8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:06.376392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bc62a-083e-4fca-a40a-77a651aaf8ca admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:06.376816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bc62a-083e-4fca-a40a-77a651aaf8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:06.377196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bc62a-083e-4fca-a40a-77a651aaf8ca admin admin] PATH_INFO: `/v3/projects/0a478079144c42ffbcf918e78463b600/users/4a500f865677457a83b99e172936d65e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:06.396909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0bc62a-083e-4fca-a40a-77a651aaf8ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a478079144c42ffbcf918e78463b600, user_id=4a500f865677457a83b99e172936d65e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:06.400523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0bc62a-083e-4fca-a40a-77a651aaf8ca admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:06.413120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:06.427894 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 725/1438] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:06 2022] PUT /identity/v3/projects/0a478079144c42ffbcf918e78463b600/users/4a500f865677457a83b99e172936d65e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:06.430300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b93e-7329-4e70-91a0-4f42cbfcc11d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:06.430518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b93e-7329-4e70-91a0-4f42cbfcc11d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:06.430762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b93e-7329-4e70-91a0-4f42cbfcc11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:06.461148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27f8dc-0425-4a13-8de6-b9ab6a845dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:06.462688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27f8dc-0425-4a13-8de6-b9ab6a845dab service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:06.463043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27f8dc-0425-4a13-8de6-b9ab6a845dab service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:06.463739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27f8dc-0425-4a13-8de6-b9ab6a845dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:06.498004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e5e6b93e-7329-4e70-91a0-4f42cbfcc11d None None] MFA Rules not processed for user `c5a178b6a74140d4910f77d811febc08`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:06.499739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e5e6b93e-7329-4e70-91a0-4f42cbfcc11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:06.620272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27f8dc-0425-4a13-8de6-b9ab6a845dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:06.622032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27f8dc-0425-4a13-8de6-b9ab6a845dab service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:06.652369 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 726/1439] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:06.654452 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 714/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:06.655193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b93e-7329-4e70-91a0-4f42cbfcc11d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:06.655520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27f8dc-0425-4a13-8de6-b9ab6a845dab service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:06.657030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9d27f8dc-0425-4a13-8de6-b9ab6a845dab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:06.796182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27f8dc-0425-4a13-8de6-b9ab6a845dab service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:06.798116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b93e-7329-4e70-91a0-4f42cbfcc11d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:06.814682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22cee-e311-4dcc-aca5-2e27e818e7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:06.815069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22cee-e311-4dcc-aca5-2e27e818e7f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:06.815342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22cee-e311-4dcc-aca5-2e27e818e7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:06.815609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22cee-e311-4dcc-aca5-2e27e818e7f4 admin admin] PATH_INFO: `/v3/projects/62f55575373c47e3879ccdc7f3d4a0a6/users/eb728e2da397471ebadcb5154316d69f/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:06.831916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:06.846811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c22cee-e311-4dcc-aca5-2e27e818e7f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62f55575373c47e3879ccdc7f3d4a0a6, user_id=eb728e2da397471ebadcb5154316d69f, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:06.852761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c22cee-e311-4dcc-aca5-2e27e818e7f4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:06.887732 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 727/1441] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:06 2022] PUT /identity/v3/projects/62f55575373c47e3879ccdc7f3d4a0a6/users/eb728e2da397471ebadcb5154316d69f/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:06.889717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22cee-e311-4dcc-aca5-2e27e818e7f4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:06.931494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:06.961887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06b1bb-be0d-4df0-b918-34dd40d80468 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:06.962568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06b1bb-be0d-4df0-b918-34dd40d80468 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:06.962849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06b1bb-be0d-4df0-b918-34dd40d80468 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:06.963191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06b1bb-be0d-4df0-b918-34dd40d80468 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:07.055412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22cee-e311-4dcc-aca5-2e27e818e7f4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:07.062517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e0580-e09d-4616-ae37-8a24f7cabf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:07.063682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e0580-e09d-4616-ae37-8a24f7cabf1f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:07.064111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e0580-e09d-4616-ae37-8a24f7cabf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:07.065243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e0580-e09d-4616-ae37-8a24f7cabf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:07.066063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e0580-e09d-4616-ae37-8a24f7cabf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:07.068220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e0580-e09d-4616-ae37-8a24f7cabf1f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:07.082259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06b1bb-be0d-4df0-b918-34dd40d80468 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:07.084007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06b1bb-be0d-4df0-b918-34dd40d80468 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:07.091035 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 728/1442] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:07.092669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e0580-e09d-4616-ae37-8a24f7cabf1f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:07.102418 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 715/1443] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:48:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:07.112062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e1376-98ed-44b5-8c7b-a0203135294a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:07.112565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e1376-98ed-44b5-8c7b-a0203135294a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:07.113004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e1376-98ed-44b5-8c7b-a0203135294a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:07.120379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e0580-e09d-4616-ae37-8a24f7cabf1f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:07.136367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae0eca-7b5e-4359-9778-52295bc7109e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:07.137198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae0eca-7b5e-4359-9778-52295bc7109e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:07.137473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae0eca-7b5e-4359-9778-52295bc7109e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:07.137740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae0eca-7b5e-4359-9778-52295bc7109e admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:07.138774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae0eca-7b5e-4359-9778-52295bc7109e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:07.140661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae0eca-7b5e-4359-9778-52295bc7109e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:07.166933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3f8e1376-98ed-44b5-8c7b-a0203135294a None None] MFA Rules not processed for user `eb728e2da397471ebadcb5154316d69f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:07.169996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3f8e1376-98ed-44b5-8c7b-a0203135294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:07.220906 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 729/1444] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:07 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:07.226853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae0eca-7b5e-4359-9778-52295bc7109e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:07.233265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-06ae0eca-7b5e-4359-9778-52295bc7109e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:07.365166 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 716/1445] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:07.366150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e1376-98ed-44b5-8c7b-a0203135294a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:07.370176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3f8e1376-98ed-44b5-8c7b-a0203135294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:07.388965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae0eca-7b5e-4359-9778-52295bc7109e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:07.419219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e1376-98ed-44b5-8c7b-a0203135294a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:07.425719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:07.432332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:07.433576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:07.437056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] PATH_INFO: `/v3/projects/99711b076aba40b39ffccc514b6542c9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:07.442442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:07.443103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:07.443526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:07.443698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] PATH_INFO: `/v3/projects/56bd982260c64bf4986964be986b12fa` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:07.446319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56bd982260c64bf4986964be986b12fa)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:07.448278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:07.450007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] RBAC: Authorizing `identity:delete_project(project_id=99711b076aba40b39ffccc514b6542c9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:07.458110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:07.614058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56bd982260c64bf4986964be986b12fa', 'request_id': 'req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:07.614990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99711b076aba40b39ffccc514b6542c9', 'request_id': 'req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:07.631006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56bd982260c64bf4986964be986b12fa', 'request_id': 'req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:07.638155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99711b076aba40b39ffccc514b6542c9', 'request_id': 'req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:07.658616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99711b076aba40b39ffccc514b6542c9', 'request_id': 'req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:07.658831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56bd982260c64bf4986964be986b12fa', 'request_id': 'req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:07.670362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:07.694653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] The token cache is being invalidate because project 56bd982260c64bf4986964be986b12fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:07.695722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:07.699499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:07.701639 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 717/1446] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:07 2022] DELETE /identity/v3/projects/56bd982260c64bf4986964be986b12fa => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:07.702610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:07.705539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] The token cache is being invalidate because project 99711b076aba40b39ffccc514b6542c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:07.706660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:07.707568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:07.709071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:07.716418 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 730/1447] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:07 2022] DELETE /identity/v3/projects/99711b076aba40b39ffccc514b6542c9 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:07.724363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:07.731431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:07.731759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:07.835624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad51f6-64bd-4b9b-a0b8-ee98bbeafc56 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:07.938052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc0db1-282b-4cff-86d4-1e2a43d35efc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:07.955741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082592d-ace5-447b-981e-982221227681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:07.956359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082592d-ace5-447b-981e-982221227681 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:07.956639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082592d-ace5-447b-981e-982221227681 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:07.956911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082592d-ace5-447b-981e-982221227681 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:07.957587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082592d-ace5-447b-981e-982221227681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:07.959594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082592d-ace5-447b-981e-982221227681 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:08.007109 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 731/1448] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:07 2022] POST /identity/v3/users => generated 300 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:08.007109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082592d-ace5-447b-981e-982221227681 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:08.009285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c082592d-ace5-447b-981e-982221227681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:08.014625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb007d-9818-4b57-828d-a24100074690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:08.015315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb007d-9818-4b57-828d-a24100074690 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.015596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb007d-9818-4b57-828d-a24100074690 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.015866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb007d-9818-4b57-828d-a24100074690 admin admin] PATH_INFO: `/v3/projects/0a478079144c42ffbcf918e78463b600/users/4a500f865677457a83b99e172936d65e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:08.023105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb007d-9818-4b57-828d-a24100074690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a478079144c42ffbcf918e78463b600, user_id=4a500f865677457a83b99e172936d65e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:08.037096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb007d-9818-4b57-828d-a24100074690 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:08.037289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082592d-ace5-447b-981e-982221227681 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:08.044953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b6bed-3edd-48b1-a82f-844a5745ac05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:08.045654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b6bed-3edd-48b1-a82f-844a5745ac05 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.045947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b6bed-3edd-48b1-a82f-844a5745ac05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.046218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b6bed-3edd-48b1-a82f-844a5745ac05 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:08.046852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b6bed-3edd-48b1-a82f-844a5745ac05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:08.058084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b6bed-3edd-48b1-a82f-844a5745ac05 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:08.080494 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 718/1449] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:07 2022] PUT /identity/v3/projects/0a478079144c42ffbcf918e78463b600/users/4a500f865677457a83b99e172936d65e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:08.084563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea52b93-f8e9-4fbf-9cb2-043d0bebdb08 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.084847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea52b93-f8e9-4fbf-9cb2-043d0bebdb08 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.087910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea52b93-f8e9-4fbf-9cb2-043d0bebdb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:08.131168 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 732/1450] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:08 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:08.132969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6fd90-a062-48f6-a1d1-30db911f5ae0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.133197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6fd90-a062-48f6-a1d1-30db911f5ae0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.133469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6fd90-a062-48f6-a1d1-30db911f5ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:08.149364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cea52b93-f8e9-4fbf-9cb2-043d0bebdb08 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:08.150705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cea52b93-f8e9-4fbf-9cb2-043d0bebdb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:08.218680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-12a6fd90-a062-48f6-a1d1-30db911f5ae0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:08.219989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-12a6fd90-a062-48f6-a1d1-30db911f5ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:08.396567 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 719/1451] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:08.398379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea52b93-f8e9-4fbf-9cb2-043d0bebdb08 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:08.432246 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 733/1452] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:08.433740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9d749-522b-4565-899a-a10fe8496fee None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.434003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9d749-522b-4565-899a-a10fe8496fee None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.434281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9d749-522b-4565-899a-a10fe8496fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:08.455279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea52b93-f8e9-4fbf-9cb2-043d0bebdb08 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:08.478034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d9289-c355-488a-aa05-ab4c2d585bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:08.479282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d9289-c355-488a-aa05-ab4c2d585bda admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.479719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d9289-c355-488a-aa05-ab4c2d585bda admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.480868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d9289-c355-488a-aa05-ab4c2d585bda admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:08.482659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d9289-c355-488a-aa05-ab4c2d585bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:08.484564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d9289-c355-488a-aa05-ab4c2d585bda admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:08.508450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2db9d749-522b-4565-899a-a10fe8496fee None None] MFA Rules not processed for user `4a500f865677457a83b99e172936d65e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:08.509769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2db9d749-522b-4565-899a-a10fe8496fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:08.517274 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 720/1453] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:08.518525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d9289-c355-488a-aa05-ab4c2d585bda admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:08.549643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d9289-c355-488a-aa05-ab4c2d585bda admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:08.565516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf21a28-8c33-4160-a035-42ff053c1157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:08.566501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf21a28-8c33-4160-a035-42ff053c1157 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.567363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf21a28-8c33-4160-a035-42ff053c1157 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.567931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf21a28-8c33-4160-a035-42ff053c1157 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:08.568775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf21a28-8c33-4160-a035-42ff053c1157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:08.575878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf21a28-8c33-4160-a035-42ff053c1157 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:08.645930 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 721/1454] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:08 2022] POST /identity/v3/users => generated 302 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:08.652700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51f90d-afca-450e-b4fb-e147df4c5243 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.655922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51f90d-afca-450e-b4fb-e147df4c5243 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.656326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51f90d-afca-450e-b4fb-e147df4c5243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:08.657945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:08.689665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bc51f90d-afca-450e-b4fb-e147df4c5243 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:08.691016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bc51f90d-afca-450e-b4fb-e147df4c5243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:08.710310 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 722/1455] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:08.714005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23461f-8406-41dc-972b-ac47037b15c2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.716395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23461f-8406-41dc-972b-ac47037b15c2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.717505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23461f-8406-41dc-972b-ac47037b15c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:08.730192 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 734/1456] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:08.732240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de62d9-19a7-4e2b-8b31-273225a7cead None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.732968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de62d9-19a7-4e2b-8b31-273225a7cead None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.733909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de62d9-19a7-4e2b-8b31-273225a7cead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:08.790075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-82de62d9-19a7-4e2b-8b31-273225a7cead None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:08.794904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-82de62d9-19a7-4e2b-8b31-273225a7cead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:08.812545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:08.853234 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 735/1457] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:08.859577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de62d9-19a7-4e2b-8b31-273225a7cead None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:08.896772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8e23461f-8406-41dc-972b-ac47037b15c2 None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:08.909306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8e23461f-8406-41dc-972b-ac47037b15c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:08.930142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de62d9-19a7-4e2b-8b31-273225a7cead None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:08.960120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45507889-6381-45d2-9b06-80c3428113f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:08.967779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45507889-6381-45d2-9b06-80c3428113f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:08.968439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45507889-6381-45d2-9b06-80c3428113f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:08.969947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45507889-6381-45d2-9b06-80c3428113f8 admin admin] PATH_INFO: `/v3/projects/08e758aebf084d609d8183f2644b54fe/users/e25fb2f175bd434b824ec70f41abe8ec/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:09.036040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45507889-6381-45d2-9b06-80c3428113f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e758aebf084d609d8183f2644b54fe, user_id=e25fb2f175bd434b824ec70f41abe8ec, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:09.041972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45507889-6381-45d2-9b06-80c3428113f8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:09.075672 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 736/1458] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:08 2022] PUT /identity/v3/projects/08e758aebf084d609d8183f2644b54fe/users/e25fb2f175bd434b824ec70f41abe8ec/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:09.075672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45507889-6381-45d2-9b06-80c3428113f8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:09.112116 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 723/1459] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Oct 15 20:48:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:09.113197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23461f-8406-41dc-972b-ac47037b15c2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:09.149073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23461f-8406-41dc-972b-ac47037b15c2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:09.227247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45507889-6381-45d2-9b06-80c3428113f8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:09.242991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e9217-de52-468f-99f4-0f1bbeaaec0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:09.246017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e9217-de52-468f-99f4-0f1bbeaaec0c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:09.246512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e9217-de52-468f-99f4-0f1bbeaaec0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:09.246962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e9217-de52-468f-99f4-0f1bbeaaec0c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:09.247807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e9217-de52-468f-99f4-0f1bbeaaec0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:09.252552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e9217-de52-468f-99f4-0f1bbeaaec0c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:09.277697 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 737/1460] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:09.278642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e9217-de52-468f-99f4-0f1bbeaaec0c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:09.280754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e2e9217-de52-468f-99f4-0f1bbeaaec0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:09.340437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae469a-55d2-4db5-a2ea-f016fceff5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:09.345233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae469a-55d2-4db5-a2ea-f016fceff5de admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:09.345912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae469a-55d2-4db5-a2ea-f016fceff5de admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:09.347016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae469a-55d2-4db5-a2ea-f016fceff5de admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:09.348753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae469a-55d2-4db5-a2ea-f016fceff5de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:09.349110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae469a-55d2-4db5-a2ea-f016fceff5de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:09.366517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae469a-55d2-4db5-a2ea-f016fceff5de admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:09.394983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e9217-de52-468f-99f4-0f1bbeaaec0c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:09.405382 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 724/1461] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:09.406626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae469a-55d2-4db5-a2ea-f016fceff5de admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:09.424276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a06a1-e649-4a03-87b2-c687ee9e0541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:09.425148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a06a1-e649-4a03-87b2-c687ee9e0541 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:09.425437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a06a1-e649-4a03-87b2-c687ee9e0541 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:09.425684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a06a1-e649-4a03-87b2-c687ee9e0541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:09.434631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae469a-55d2-4db5-a2ea-f016fceff5de admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:09.442279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a15839-95ad-44b9-8463-a69d90c0d30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:09.456997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a15839-95ad-44b9-8463-a69d90c0d30f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:09.458880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a15839-95ad-44b9-8463-a69d90c0d30f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:09.459172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a15839-95ad-44b9-8463-a69d90c0d30f admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:09.462509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a15839-95ad-44b9-8463-a69d90c0d30f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:09.464033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a15839-95ad-44b9-8463-a69d90c0d30f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:09.468833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a15839-95ad-44b9-8463-a69d90c0d30f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:09.498315 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 725/1462] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:09.498315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a15839-95ad-44b9-8463-a69d90c0d30f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:09.530924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:09.532555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a15839-95ad-44b9-8463-a69d90c0d30f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:09.546401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56335e-843f-4315-82fe-f614a4a21977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:09.547051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56335e-843f-4315-82fe-f614a4a21977 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:09.550167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56335e-843f-4315-82fe-f614a4a21977 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:09.550502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56335e-843f-4315-82fe-f614a4a21977 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:09.551963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56335e-843f-4315-82fe-f614a4a21977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:09.553679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56335e-843f-4315-82fe-f614a4a21977 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:09.567106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a06a1-e649-4a03-87b2-c687ee9e0541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:09.569134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a06a1-e649-4a03-87b2-c687ee9e0541 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:09.593232 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 726/1463] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:09.594142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56335e-843f-4315-82fe-f614a4a21977 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:09.605592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e56335e-843f-4315-82fe-f614a4a21977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:09.630728 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 738/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:09.633947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a06a1-e649-4a03-87b2-c687ee9e0541 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:09.677656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a06a1-e649-4a03-87b2-c687ee9e0541 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:09.700169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca8451-3597-4bde-833e-31eec811c8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:09.703481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca8451-3597-4bde-833e-31eec811c8ce admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:09.705309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca8451-3597-4bde-833e-31eec811c8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:09.706150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca8451-3597-4bde-833e-31eec811c8ce admin admin] PATH_INFO: `/v3/projects/017d5d6222204d759c027f3503532604/users/06ccada894e74ba4a08f9928e8036987/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:09.737069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca8451-3597-4bde-833e-31eec811c8ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=017d5d6222204d759c027f3503532604, user_id=06ccada894e74ba4a08f9928e8036987, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:09.739822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca8451-3597-4bde-833e-31eec811c8ce admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:09.777853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56335e-843f-4315-82fe-f614a4a21977 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:09.794578 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 739/1465] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:09 2022] PUT /identity/v3/projects/017d5d6222204d759c027f3503532604/users/06ccada894e74ba4a08f9928e8036987/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:09.799456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca8451-3597-4bde-833e-31eec811c8ce admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:09.863213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:09.889764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e347f9-c989-4954-a0a8-6eaf1b55a163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:09.890838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e347f9-c989-4954-a0a8-6eaf1b55a163 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:09.891182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e347f9-c989-4954-a0a8-6eaf1b55a163 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:09.891659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e347f9-c989-4954-a0a8-6eaf1b55a163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:09.903079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e347f9-c989-4954-a0a8-6eaf1b55a163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:09.904776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e347f9-c989-4954-a0a8-6eaf1b55a163 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:09.916295 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 727/1466] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:09.917270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e347f9-c989-4954-a0a8-6eaf1b55a163 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:09.941757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca8451-3597-4bde-833e-31eec811c8ce admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:09.949867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e5707-0c88-4a7d-a20f-1190e26ba5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:09.951367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e5707-0c88-4a7d-a20f-1190e26ba5cb admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:09.951788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e5707-0c88-4a7d-a20f-1190e26ba5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:09.952252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e5707-0c88-4a7d-a20f-1190e26ba5cb admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:09.953029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e5707-0c88-4a7d-a20f-1190e26ba5cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:09.955848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e5707-0c88-4a7d-a20f-1190e26ba5cb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:10.037316 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 740/1467] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:09 2022] POST /identity/v3/projects => generated 375 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:10.037316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e5707-0c88-4a7d-a20f-1190e26ba5cb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:10.093401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e347f9-c989-4954-a0a8-6eaf1b55a163 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:10.239585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e5707-0c88-4a7d-a20f-1190e26ba5cb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:10.252758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29146e-f4a3-46f0-9cd3-515447459775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:10.253464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29146e-f4a3-46f0-9cd3-515447459775 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:10.253746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29146e-f4a3-46f0-9cd3-515447459775 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:10.254020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29146e-f4a3-46f0-9cd3-515447459775 admin admin] PATH_INFO: `/v3/projects/08e758aebf084d609d8183f2644b54fe/users/e25fb2f175bd434b824ec70f41abe8ec/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:10.260726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29146e-f4a3-46f0-9cd3-515447459775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e758aebf084d609d8183f2644b54fe, user_id=e25fb2f175bd434b824ec70f41abe8ec, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:10.262686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfae6e1-f787-4161-a38b-658dcc43d59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:10.263485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfae6e1-f787-4161-a38b-658dcc43d59b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:10.263824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29146e-f4a3-46f0-9cd3-515447459775 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:10.264592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfae6e1-f787-4161-a38b-658dcc43d59b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:10.265048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfae6e1-f787-4161-a38b-658dcc43d59b admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:10.266295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfae6e1-f787-4161-a38b-658dcc43d59b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:10.276509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfae6e1-f787-4161-a38b-658dcc43d59b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:10.303342 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 741/1468] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:10 2022] PUT /identity/v3/projects/08e758aebf084d609d8183f2644b54fe/users/e25fb2f175bd434b824ec70f41abe8ec/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:10.304433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29146e-f4a3-46f0-9cd3-515447459775 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:10.364549 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 728/1469] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:09 2022] POST /identity/v3/projects => generated 379 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:10.365518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfae6e1-f787-4161-a38b-658dcc43d59b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:10.447087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:10.500241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29146e-f4a3-46f0-9cd3-515447459775 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:10.534647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfae6e1-f787-4161-a38b-658dcc43d59b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:10.542730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85239d35-a846-4fe9-8185-04b5ca2ff07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:10.543678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85239d35-a846-4fe9-8185-04b5ca2ff07e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:10.544208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85239d35-a846-4fe9-8185-04b5ca2ff07e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:10.544677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85239d35-a846-4fe9-8185-04b5ca2ff07e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:10.545637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85239d35-a846-4fe9-8185-04b5ca2ff07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:10.548260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85239d35-a846-4fe9-8185-04b5ca2ff07e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:10.584749 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 729/1470] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:10 2022] POST /identity/v3/users => generated 288 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:10.586729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3593f64-4292-49f0-8d47-07265b8d198b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:10.587234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3593f64-4292-49f0-8d47-07265b8d198b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:10.588405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3593f64-4292-49f0-8d47-07265b8d198b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:10.597565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:10.663125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d57f74-9bb3-46a1-a380-456af4305fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:10.672734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d57f74-9bb3-46a1-a380-456af4305fbf admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:10.673244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f3593f64-4292-49f0-8d47-07265b8d198b None None] MFA Rules not processed for user `e25fb2f175bd434b824ec70f41abe8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:10.673526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d57f74-9bb3-46a1-a380-456af4305fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:10.674399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d57f74-9bb3-46a1-a380-456af4305fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:10.675422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f3593f64-4292-49f0-8d47-07265b8d198b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:10.675738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d57f74-9bb3-46a1-a380-456af4305fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:10.679673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d57f74-9bb3-46a1-a380-456af4305fbf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:10.720980 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 742/1471] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:10.720980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d57f74-9bb3-46a1-a380-456af4305fbf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:10.751303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d57f74-9bb3-46a1-a380-456af4305fbf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:10.769390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee68dd8-01f7-4d79-a86e-6c7f692896a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:10.771170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee68dd8-01f7-4d79-a86e-6c7f692896a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:10.772096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee68dd8-01f7-4d79-a86e-6c7f692896a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:10.774534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee68dd8-01f7-4d79-a86e-6c7f692896a9 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:10.777088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee68dd8-01f7-4d79-a86e-6c7f692896a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:10.779883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee68dd8-01f7-4d79-a86e-6c7f692896a9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:10.839410 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 743/1472] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:10 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:10.841528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee68dd8-01f7-4d79-a86e-6c7f692896a9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:10.846080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0ee68dd8-01f7-4d79-a86e-6c7f692896a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:10.888766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee68dd8-01f7-4d79-a86e-6c7f692896a9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:10.913979 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 730/1473] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:10.914558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:10.914678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3593f64-4292-49f0-8d47-07265b8d198b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:10.914764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:10.914764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:10.914764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] PATH_INFO: `/v3/users/c3266dc808c84798a7156fe670ea5cad` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:10.914764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3266dc808c84798a7156fe670ea5cad)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:10.914764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:10.944127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3593f64-4292-49f0-8d47-07265b8d198b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:10.961349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004493b-1f64-4732-addb-dbe96d209cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:10.962047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004493b-1f64-4732-addb-dbe96d209cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:10.962351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004493b-1f64-4732-addb-dbe96d209cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:10.962597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004493b-1f64-4732-addb-dbe96d209cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:10.963249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004493b-1f64-4732-addb-dbe96d209cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:10.975092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004493b-1f64-4732-addb-dbe96d209cb9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:11.014068 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 731/1474] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:11.015018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004493b-1f64-4732-addb-dbe96d209cb9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:11.040306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004493b-1f64-4732-addb-dbe96d209cb9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:11.041715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3266dc808c84798a7156fe670ea5cad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:11.059792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3266dc808c84798a7156fe670ea5cad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:11.067912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534c04-8ba2-4a43-a778-2a823c6bd898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:11.068769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534c04-8ba2-4a43-a778-2a823c6bd898 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:11.069074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534c04-8ba2-4a43-a778-2a823c6bd898 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:11.069989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534c04-8ba2-4a43-a778-2a823c6bd898 admin admin] PATH_INFO: `/v3/projects/017d5d6222204d759c027f3503532604/users/06ccada894e74ba4a08f9928e8036987/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:11.072917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1534c04-8ba2-4a43-a778-2a823c6bd898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017d5d6222204d759c027f3503532604, user_id=06ccada894e74ba4a08f9928e8036987, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:11.075739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1534c04-8ba2-4a43-a778-2a823c6bd898 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:11.090736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3266dc808c84798a7156fe670ea5cad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:11.101855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3266dc808c84798a7156fe670ea5cad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:11.102686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3266dc808c84798a7156fe670ea5cad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:11.114389 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 744/1475] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:10 2022] DELETE /identity/v3/users/c3266dc808c84798a7156fe670ea5cad => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:11.115390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:11.118479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:11.149721 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 732/1476] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:11 2022] PUT /identity/v3/projects/017d5d6222204d759c027f3503532604/users/06ccada894e74ba4a08f9928e8036987/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:11.149721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0489fcb-dd78-4952-a991-6d3ebb6262cb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:11.149721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0489fcb-dd78-4952-a991-6d3ebb6262cb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:11.149721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0489fcb-dd78-4952-a991-6d3ebb6262cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:11.209262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a0489fcb-dd78-4952-a991-6d3ebb6262cb None None] MFA Rules not processed for user `c5a178b6a74140d4910f77d811febc08`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:11.211060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a0489fcb-dd78-4952-a991-6d3ebb6262cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:11.211836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:11.291341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cb024-9465-42fc-ad8e-14df42b9487e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:11.300942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965ead5-8419-4dd6-b61c-c286314a33b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:11.304653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965ead5-8419-4dd6-b61c-c286314a33b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:11.306185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965ead5-8419-4dd6-b61c-c286314a33b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:11.307938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965ead5-8419-4dd6-b61c-c286314a33b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:11.309921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965ead5-8419-4dd6-b61c-c286314a33b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:11.313008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965ead5-8419-4dd6-b61c-c286314a33b6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:11.335900 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 745/1477] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:11.338684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965ead5-8419-4dd6-b61c-c286314a33b6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:11.343500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3965ead5-8419-4dd6-b61c-c286314a33b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:11.374295 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 733/1478] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:11.376235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67cb84-0427-439e-b56d-b7c52de0145e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:11.376679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67cb84-0427-439e-b56d-b7c52de0145e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:11.377152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67cb84-0427-439e-b56d-b7c52de0145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:11.379486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965ead5-8419-4dd6-b61c-c286314a33b6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:11.394211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6d108-8be2-4cfc-a214-f61f8e60450b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:11.395956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6d108-8be2-4cfc-a214-f61f8e60450b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:11.397651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6d108-8be2-4cfc-a214-f61f8e60450b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:11.399046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6d108-8be2-4cfc-a214-f61f8e60450b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:11.399922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6d108-8be2-4cfc-a214-f61f8e60450b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:11.403416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6d108-8be2-4cfc-a214-f61f8e60450b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:11.404096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-af67cb84-0427-439e-b56d-b7c52de0145e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:11.405710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af67cb84-0427-439e-b56d-b7c52de0145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:11.439467 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 734/1479] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:11.442519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11206c21-6429-4269-a108-f8937043d770 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:11.442808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11206c21-6429-4269-a108-f8937043d770 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:11.446088 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 746/1480] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:11.447739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6d108-8be2-4cfc-a214-f61f8e60450b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:11.448380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11206c21-6429-4269-a108-f8937043d770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:11.482927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6d108-8be2-4cfc-a214-f61f8e60450b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:11.503270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcb852-a1af-4616-bf79-498b738fca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:11.505431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcb852-a1af-4616-bf79-498b738fca0b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:11.507964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcb852-a1af-4616-bf79-498b738fca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:11.508581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcb852-a1af-4616-bf79-498b738fca0b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:11.510143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcb852-a1af-4616-bf79-498b738fca0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:11.513881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcb852-a1af-4616-bf79-498b738fca0b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:11.536460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-11206c21-6429-4269-a108-f8937043d770 None None] MFA Rules not processed for user `06ccada894e74ba4a08f9928e8036987`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:11.540039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-11206c21-6429-4269-a108-f8937043d770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:11.561431 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 747/1481] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:11.561431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcb852-a1af-4616-bf79-498b738fca0b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:11.564841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-82fcb852-a1af-4616-bf79-498b738fca0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:11.739488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:11.810811 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 735/1482] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:11 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:11.812368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11206c21-6429-4269-a108-f8937043d770 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:11.844744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcb852-a1af-4616-bf79-498b738fca0b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:11.875056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11206c21-6429-4269-a108-f8937043d770 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:11.881375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce2f8a-566e-42de-84ec-c3894f41b61c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:11.882180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce2f8a-566e-42de-84ec-c3894f41b61c service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:11.884829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce2f8a-566e-42de-84ec-c3894f41b61c service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:11.885248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce2f8a-566e-42de-84ec-c3894f41b61c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:11.890911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e33b4-ea98-4714-938a-e8e90bdc6fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:11.893196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e33b4-ea98-4714-938a-e8e90bdc6fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:11.893597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e33b4-ea98-4714-938a-e8e90bdc6fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:11.896654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-edce2f8a-566e-42de-84ec-c3894f41b61c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:11.897349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e33b4-ea98-4714-938a-e8e90bdc6fb0 admin admin] PATH_INFO: `/v3/projects/788a7292d90d4a069e17d14bfcfcdb1d/users/e6f40eab39e740f0b9b646444f00d0df/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:11.941323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e33b4-ea98-4714-938a-e8e90bdc6fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=788a7292d90d4a069e17d14bfcfcdb1d, user_id=e6f40eab39e740f0b9b646444f00d0df, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:11.945629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e33b4-ea98-4714-938a-e8e90bdc6fb0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:11.954756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce2f8a-566e-42de-84ec-c3894f41b61c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:11.956506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce2f8a-566e-42de-84ec-c3894f41b61c service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:11.991002 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 736/1483] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:11 2022] PUT /identity/v3/projects/788a7292d90d4a069e17d14bfcfcdb1d/users/e6f40eab39e740f0b9b646444f00d0df/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:11.992423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e33b4-ea98-4714-938a-e8e90bdc6fb0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:11.994422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1f8e33b4-ea98-4714-938a-e8e90bdc6fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:12.013643 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 748/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:12.016259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce2f8a-566e-42de-84ec-c3894f41b61c service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:12.099108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce2f8a-566e-42de-84ec-c3894f41b61c service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:12.107315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e33b4-ea98-4714-938a-e8e90bdc6fb0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:12.121553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534e1de-bb79-4258-9b47-a8743d0f4bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:12.122442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e1de-bb79-4258-9b47-a8743d0f4bce service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:12.124299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e1de-bb79-4258-9b47-a8743d0f4bce service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:12.124722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e1de-bb79-4258-9b47-a8743d0f4bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:12.340556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1aee01-c7bd-41c9-b634-14862521959f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:12.341318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1aee01-c7bd-41c9-b634-14862521959f admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:12.341598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1aee01-c7bd-41c9-b634-14862521959f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:12.341868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1aee01-c7bd-41c9-b634-14862521959f admin admin] PATH_INFO: `/v3/projects/54f631cb375e4c4a951d60f3d9f7402e/users/1c8110368826488c8c24b1ca7831e139/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:12.367712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534e1de-bb79-4258-9b47-a8743d0f4bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:12.373792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534e1de-bb79-4258-9b47-a8743d0f4bce service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:12.387198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1aee01-c7bd-41c9-b634-14862521959f admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f631cb375e4c4a951d60f3d9f7402e, user_id=1c8110368826488c8c24b1ca7831e139, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:12.389950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1aee01-c7bd-41c9-b634-14862521959f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:12.418121 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 749/1485] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:12 2022] PUT /identity/v3/projects/54f631cb375e4c4a951d60f3d9f7402e/users/1c8110368826488c8c24b1ca7831e139/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:12.418771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1aee01-c7bd-41c9-b634-14862521959f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:12.445520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ce1aee01-c7bd-41c9-b634-14862521959f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:12.447090 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 737/1486] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:12.450335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b43a9-e1f4-475c-9e73-e8e0fc01ded9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:12.450622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b43a9-e1f4-475c-9e73-e8e0fc01ded9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:12.450895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b43a9-e1f4-475c-9e73-e8e0fc01ded9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:12.514879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:12.565734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ce8b43a9-e1f4-475c-9e73-e8e0fc01ded9 None None] MFA Rules not processed for user `e6f40eab39e740f0b9b646444f00d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:12.566782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ce8b43a9-e1f4-475c-9e73-e8e0fc01ded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:12.639077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1aee01-c7bd-41c9-b634-14862521959f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:12.650743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:12.651340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:12.651615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:12.652027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] PATH_INFO: `/v3/users/eb728e2da397471ebadcb5154316d69f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:12.653896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb728e2da397471ebadcb5154316d69f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:12.656139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:12.768507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb728e2da397471ebadcb5154316d69f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:12.805765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb728e2da397471ebadcb5154316d69f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:12.840512 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 738/1487] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:12.843247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38faef8-68d0-43ae-b87f-bc91e523a66b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:12.843537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38faef8-68d0-43ae-b87f-bc91e523a66b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:12.843809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38faef8-68d0-43ae-b87f-bc91e523a66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:12.846571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb728e2da397471ebadcb5154316d69f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:12.862707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:12.867313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb728e2da397471ebadcb5154316d69f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:12.868054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb728e2da397471ebadcb5154316d69f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:12.902371 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 750/1488] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:12 2022] DELETE /identity/v3/users/eb728e2da397471ebadcb5154316d69f => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:12.904304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:12.910147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:12.932613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d38faef8-68d0-43ae-b87f-bc91e523a66b None None] MFA Rules not processed for user `1c8110368826488c8c24b1ca7831e139`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:12.933876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d38faef8-68d0-43ae-b87f-bc91e523a66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:13.090208 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 739/1489] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:13.092191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8e5b5-a1a3-4091-805f-c3608a1cddd2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.092588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8e5b5-a1a3-4091-805f-c3608a1cddd2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.092867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8e5b5-a1a3-4091-805f-c3608a1cddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.101800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f825c-2115-4dd2-bbb2-83b052d0aa1b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:13.119706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b6393-8d88-4db5-af25-3b4b32dc629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:13.120656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b6393-8d88-4db5-af25-3b4b32dc629f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.121377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b6393-8d88-4db5-af25-3b4b32dc629f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.121652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b6393-8d88-4db5-af25-3b4b32dc629f admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.122557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b6393-8d88-4db5-af25-3b4b32dc629f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:13.127155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b6393-8d88-4db5-af25-3b4b32dc629f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:13.133422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-eff8e5b5-a1a3-4091-805f-c3608a1cddd2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:13.136579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eff8e5b5-a1a3-4091-805f-c3608a1cddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:13.164071 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 740/1490] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:13.166992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0e8e3-4954-42d2-9ae1-80840e27c516 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.167275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0e8e3-4954-42d2-9ae1-80840e27c516 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.167561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0e8e3-4954-42d2-9ae1-80840e27c516 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.169778 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 741/1491] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:48:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:13.170569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0e8e3-4954-42d2-9ae1-80840e27c516 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:13.173058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ede0e8e3-4954-42d2-9ae1-80840e27c516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:13.205588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0e8e3-4954-42d2-9ae1-80840e27c516 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:13.211371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea9d93-d24e-4a46-8f17-4196a827f284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:13.212131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea9d93-d24e-4a46-8f17-4196a827f284 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.212414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea9d93-d24e-4a46-8f17-4196a827f284 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.212686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea9d93-d24e-4a46-8f17-4196a827f284 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.213481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea9d93-d24e-4a46-8f17-4196a827f284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:13.215523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea9d93-d24e-4a46-8f17-4196a827f284 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:13.230935 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 751/1492] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:12 2022] POST /identity/v3/projects => generated 377 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:13.232301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b6393-8d88-4db5-af25-3b4b32dc629f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:13.234397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-547b6393-8d88-4db5-af25-3b4b32dc629f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:13.280490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:13.280796 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 742/1493] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:13 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:13.282493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea9d93-d24e-4a46-8f17-4196a827f284 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:13.369587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b6393-8d88-4db5-af25-3b4b32dc629f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:13.518177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea9d93-d24e-4a46-8f17-4196a827f284 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:13.529972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd3e5a-4e32-4e1b-9f87-3ff0b39fa5fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:13.534868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50add0c0-0709-40bd-aa52-00c5af52dd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:13.535503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50add0c0-0709-40bd-aa52-00c5af52dd79 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.535825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50add0c0-0709-40bd-aa52-00c5af52dd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.536092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50add0c0-0709-40bd-aa52-00c5af52dd79 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.537187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3e5a-4e32-4e1b-9f87-3ff0b39fa5fe service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.537579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3e5a-4e32-4e1b-9f87-3ff0b39fa5fe service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.537873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50add0c0-0709-40bd-aa52-00c5af52dd79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:13.538040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3e5a-4e32-4e1b-9f87-3ff0b39fa5fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.544169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50add0c0-0709-40bd-aa52-00c5af52dd79 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:13.560160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd3e5a-4e32-4e1b-9f87-3ff0b39fa5fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:13.564805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd3e5a-4e32-4e1b-9f87-3ff0b39fa5fe service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:13.579307 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 743/1494] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:13 2022] POST /identity/v3/users => generated 295 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:13.579837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50add0c0-0709-40bd-aa52-00c5af52dd79 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:13.587465 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 752/1495] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:13.591113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa617-23c9-4544-bfdb-0a33250c1464 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.591934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50add0c0-0709-40bd-aa52-00c5af52dd79 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:13.592350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa617-23c9-4544-bfdb-0a33250c1464 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.596614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa617-23c9-4544-bfdb-0a33250c1464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.624807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685d8e2-1238-4222-8e63-0434768b2b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:13.625605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685d8e2-1238-4222-8e63-0434768b2b8e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.625906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685d8e2-1238-4222-8e63-0434768b2b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.627340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685d8e2-1238-4222-8e63-0434768b2b8e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.627993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685d8e2-1238-4222-8e63-0434768b2b8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:13.634899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685d8e2-1238-4222-8e63-0434768b2b8e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:13.647849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e84aa617-23c9-4544-bfdb-0a33250c1464 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:13.649840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e84aa617-23c9-4544-bfdb-0a33250c1464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:13.683985 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 744/1496] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:13 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:13.686281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685d8e2-1238-4222-8e63-0434768b2b8e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:13.703021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685d8e2-1238-4222-8e63-0434768b2b8e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:13.724708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1232f7d-aa6f-491a-b7f7-3ab1d651cecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:13.725801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1232f7d-aa6f-491a-b7f7-3ab1d651cecf admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.727137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1232f7d-aa6f-491a-b7f7-3ab1d651cecf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.728066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1232f7d-aa6f-491a-b7f7-3ab1d651cecf admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.731394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1232f7d-aa6f-491a-b7f7-3ab1d651cecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:13.736270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1232f7d-aa6f-491a-b7f7-3ab1d651cecf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:13.794914 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 745/1497] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:13.795987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1232f7d-aa6f-491a-b7f7-3ab1d651cecf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:13.848791 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 753/1498] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:48:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:13.850498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84aa617-23c9-4544-bfdb-0a33250c1464 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:13.857703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1232f7d-aa6f-491a-b7f7-3ab1d651cecf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:13.882312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5031a6f-41b3-4169-8e5a-1a1e1c767961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:13.885104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5031a6f-41b3-4169-8e5a-1a1e1c767961 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.886780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5031a6f-41b3-4169-8e5a-1a1e1c767961 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.888688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5031a6f-41b3-4169-8e5a-1a1e1c767961 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.892103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5031a6f-41b3-4169-8e5a-1a1e1c767961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:13.893359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84aa617-23c9-4544-bfdb-0a33250c1464 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:13.899624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5031a6f-41b3-4169-8e5a-1a1e1c767961 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:13.915203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc72bb-f92b-4a6d-925e-a3e1ad4cabba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:13.918477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc72bb-f92b-4a6d-925e-a3e1ad4cabba admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:13.918926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc72bb-f92b-4a6d-925e-a3e1ad4cabba admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:13.919302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc72bb-f92b-4a6d-925e-a3e1ad4cabba admin admin] PATH_INFO: `/v3/projects/b96fc1fd9d574d609b8375160a01ea28/users/64ce45a906224d74a2aaf34aac5175fe/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:13.986251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc72bb-f92b-4a6d-925e-a3e1ad4cabba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96fc1fd9d574d609b8375160a01ea28, user_id=64ce45a906224d74a2aaf34aac5175fe, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:13.994377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc72bb-f92b-4a6d-925e-a3e1ad4cabba admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:14.002449 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 746/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:14.010458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5031a6f-41b3-4169-8e5a-1a1e1c767961 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:14.025891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a5031a6f-41b3-4169-8e5a-1a1e1c767961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:14.030840 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 754/1500] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:13 2022] PUT /identity/v3/projects/b96fc1fd9d574d609b8375160a01ea28/users/64ce45a906224d74a2aaf34aac5175fe/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:14.030840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc72bb-f92b-4a6d-925e-a3e1ad4cabba admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:14.092831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5031a6f-41b3-4169-8e5a-1a1e1c767961 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:14.129424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:14.175655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:14.213913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc72bb-f92b-4a6d-925e-a3e1ad4cabba admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:14.229388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4057c52-871f-4f48-8420-c1ed65da9f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:14.230073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057c52-871f-4f48-8420-c1ed65da9f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:14.230347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057c52-871f-4f48-8420-c1ed65da9f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:14.230617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057c52-871f-4f48-8420-c1ed65da9f8b admin admin] PATH_INFO: `/v3/projects/50a7fc61b0984db7b9a07f25e7738c0b/users/6d248fb973f84e6b848b7f2678a884f4/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:14.276258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4057c52-871f-4f48-8420-c1ed65da9f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a7fc61b0984db7b9a07f25e7738c0b, user_id=6d248fb973f84e6b848b7f2678a884f4, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:14.285574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4057c52-871f-4f48-8420-c1ed65da9f8b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:14.306717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5d5d3-c7da-4c93-86f8-3c5cd6ed35e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:14.309982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5d5d3-c7da-4c93-86f8-3c5cd6ed35e1 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:14.311170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5d5d3-c7da-4c93-86f8-3c5cd6ed35e1 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:14.311794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5d5d3-c7da-4c93-86f8-3c5cd6ed35e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:14.314903 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 755/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:14 2022] PUT /identity/v3/projects/50a7fc61b0984db7b9a07f25e7738c0b/users/6d248fb973f84e6b848b7f2678a884f4/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:14.314903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4057c52-871f-4f48-8420-c1ed65da9f8b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:14.336844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5d5d3-c7da-4c93-86f8-3c5cd6ed35e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:14.339725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5d5d3-c7da-4c93-86f8-3c5cd6ed35e1 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:14.381126 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 747/1502] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:14.384653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5d5d3-c7da-4c93-86f8-3c5cd6ed35e1 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:14.412008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5d5d3-c7da-4c93-86f8-3c5cd6ed35e1 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:14.518943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4057c52-871f-4f48-8420-c1ed65da9f8b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:14.537176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29936b09-3d23-4083-865d-decf1432da37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:14.542112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec2e82-8292-4bb3-aad9-2080d3d43721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:14.542408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29936b09-3d23-4083-865d-decf1432da37 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:14.542724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec2e82-8292-4bb3-aad9-2080d3d43721 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:14.543000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec2e82-8292-4bb3-aad9-2080d3d43721 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:14.543267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec2e82-8292-4bb3-aad9-2080d3d43721 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:14.543486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29936b09-3d23-4083-865d-decf1432da37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:14.545108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec2e82-8292-4bb3-aad9-2080d3d43721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:14.545552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29936b09-3d23-4083-865d-decf1432da37 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:14.547138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29936b09-3d23-4083-865d-decf1432da37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:14.549068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec2e82-8292-4bb3-aad9-2080d3d43721 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:14.550266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29936b09-3d23-4083-865d-decf1432da37 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:14.581656 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 748/1503] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:14.583693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec2e82-8292-4bb3-aad9-2080d3d43721 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:14.615525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec2e82-8292-4bb3-aad9-2080d3d43721 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:14.620823 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 756/1504] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:14.622323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29936b09-3d23-4083-865d-decf1432da37 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:14.630723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9643a-e7aa-4329-87ec-0c67d8479127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:14.632523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9643a-e7aa-4329-87ec-0c67d8479127 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:14.632789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9643a-e7aa-4329-87ec-0c67d8479127 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:14.642207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9643a-e7aa-4329-87ec-0c67d8479127 admin admin] PATH_INFO: `/v3/projects/50a7fc61b0984db7b9a07f25e7738c0b/users/6d248fb973f84e6b848b7f2678a884f4/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:14.644574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9643a-e7aa-4329-87ec-0c67d8479127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a7fc61b0984db7b9a07f25e7738c0b, user_id=6d248fb973f84e6b848b7f2678a884f4, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:14.647255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9643a-e7aa-4329-87ec-0c67d8479127 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:14.672980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29936b09-3d23-4083-865d-decf1432da37 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:14.696608 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 749/1505] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:14 2022] PUT /identity/v3/projects/50a7fc61b0984db7b9a07f25e7738c0b/users/6d248fb973f84e6b848b7f2678a884f4/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:14.697414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660d190-9768-4475-83a9-0a4a786d3643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:14.698887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660d190-9768-4475-83a9-0a4a786d3643 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:14.700703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660d190-9768-4475-83a9-0a4a786d3643 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:14.701171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660d190-9768-4475-83a9-0a4a786d3643 admin admin] PATH_INFO: `/v3/projects/b96fc1fd9d574d609b8375160a01ea28/users/64ce45a906224d74a2aaf34aac5175fe/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:14.707656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25919531-5aae-4b5e-b5cc-d43aa750ee25 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:14.708343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25919531-5aae-4b5e-b5cc-d43aa750ee25 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:14.708754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25919531-5aae-4b5e-b5cc-d43aa750ee25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:14.730116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660d190-9768-4475-83a9-0a4a786d3643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96fc1fd9d574d609b8375160a01ea28, user_id=64ce45a906224d74a2aaf34aac5175fe, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:14.739043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660d190-9768-4475-83a9-0a4a786d3643 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:14.822958 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 757/1506] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:14 2022] PUT /identity/v3/projects/b96fc1fd9d574d609b8375160a01ea28/users/64ce45a906224d74a2aaf34aac5175fe/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:14.831047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-25919531-5aae-4b5e-b5cc-d43aa750ee25 None None] MFA Rules not processed for user `6d248fb973f84e6b848b7f2678a884f4`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:14.832305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-25919531-5aae-4b5e-b5cc-d43aa750ee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:14.840940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feffad-42fb-46d0-b401-ab7fdaab0ec5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:14.843381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feffad-42fb-46d0-b401-ab7fdaab0ec5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:14.847425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feffad-42fb-46d0-b401-ab7fdaab0ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:15.024098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b2feffad-42fb-46d0-b401-ab7fdaab0ec5 None None] MFA Rules not processed for user `64ce45a906224d74a2aaf34aac5175fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:15.025765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b2feffad-42fb-46d0-b401-ab7fdaab0ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:15.143827 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 750/1507] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:15.152150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25919531-5aae-4b5e-b5cc-d43aa750ee25 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:15.164521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:15.390033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25919531-5aae-4b5e-b5cc-d43aa750ee25 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:15.396188 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 758/1508] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 562 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:48:15.406714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2feffad-42fb-46d0-b401-ab7fdaab0ec5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:15.413788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d34a0-8aa9-438c-9f89-f249385ead46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:15.414433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d34a0-8aa9-438c-9f89-f249385ead46 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:15.414873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d34a0-8aa9-438c-9f89-f249385ead46 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:15.414984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b2feffad-42fb-46d0-b401-ab7fdaab0ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:15.415140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d34a0-8aa9-438c-9f89-f249385ead46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:15.437324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d34a0-8aa9-438c-9f89-f249385ead46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:15.439131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d34a0-8aa9-438c-9f89-f249385ead46 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:15.462407 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 751/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:15.463291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d34a0-8aa9-438c-9f89-f249385ead46 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:15.466332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dc1d34a0-8aa9-438c-9f89-f249385ead46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:15.578916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d34a0-8aa9-438c-9f89-f249385ead46 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:15.603610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:15.604246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:15.604522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:15.604786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] PATH_INFO: `/v3/projects/b6cefe34fe37400d98f8ef15da8df500` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:15.606981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:15.610046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6cefe34fe37400d98f8ef15da8df500)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:15.641301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:15.705320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2feffad-42fb-46d0-b401-ab7fdaab0ec5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:15.718049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:15.720339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:15.720725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:15.721943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] PATH_INFO: `/v3/projects/62f55575373c47e3879ccdc7f3d4a0a6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:15.725114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62f55575373c47e3879ccdc7f3d4a0a6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:15.730738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:15.801323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6cefe34fe37400d98f8ef15da8df500', 'request_id': 'req-23d3b6de-928f-4553-a46f-7b38780d0956'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:15.812646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6cefe34fe37400d98f8ef15da8df500', 'request_id': 'req-23d3b6de-928f-4553-a46f-7b38780d0956'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:15.824524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6cefe34fe37400d98f8ef15da8df500', 'request_id': 'req-23d3b6de-928f-4553-a46f-7b38780d0956'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:15.828138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62f55575373c47e3879ccdc7f3d4a0a6', 'request_id': 'req-5da1a493-8242-41b8-b91e-b5143dd35e65'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:15.846447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] The token cache is being invalidate because project b6cefe34fe37400d98f8ef15da8df500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:15.846796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62f55575373c47e3879ccdc7f3d4a0a6', 'request_id': 'req-5da1a493-8242-41b8-b91e-b5143dd35e65'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:15.847880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:15.849451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:15.856049 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 752/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:15 2022] DELETE /identity/v3/projects/b6cefe34fe37400d98f8ef15da8df500 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:15.857648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:15.860829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:15.863282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62f55575373c47e3879ccdc7f3d4a0a6', 'request_id': 'req-5da1a493-8242-41b8-b91e-b5143dd35e65'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:15.885139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] The token cache is being invalidate because project 62f55575373c47e3879ccdc7f3d4a0a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:15.885652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:15.886958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:15.891207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 759/1511] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:15 2022] DELETE /identity/v3/projects/62f55575373c47e3879ccdc7f3d4a0a6 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:15.895440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:15.898746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:15.957952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:16.085620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3b6de-928f-4553-a46f-7b38780d0956 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:16.087624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1a493-8242-41b8-b91e-b5143dd35e65 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:16.112024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:16.112687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:16.112965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:16.113531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] PATH_INFO: `/v3/users/4a500f865677457a83b99e172936d65e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:16.119175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a500f865677457a83b99e172936d65e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:16.124865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:16.296378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a500f865677457a83b99e172936d65e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:16.310704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6aa53-122a-4c97-ac67-e60fe27fd3fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:16.313274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6aa53-122a-4c97-ac67-e60fe27fd3fd service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:16.314260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6aa53-122a-4c97-ac67-e60fe27fd3fd service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:16.315310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6aa53-122a-4c97-ac67-e60fe27fd3fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:16.316301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a500f865677457a83b99e172936d65e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:16.320936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-35d6aa53-122a-4c97-ac67-e60fe27fd3fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:16.354161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:16.357149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a500f865677457a83b99e172936d65e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:16.405979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a500f865677457a83b99e172936d65e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:16.407792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a500f865677457a83b99e172936d65e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:16.426485 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 760/1512] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:15 2022] DELETE /identity/v3/users/4a500f865677457a83b99e172936d65e => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:16.434984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:16.439015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:16.579045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6aa53-122a-4c97-ac67-e60fe27fd3fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:16.590251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6aa53-122a-4c97-ac67-e60fe27fd3fd service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:16.635903 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 753/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:16.667692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a984bb-2078-4c66-9d3f-83ea66657d11 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:16.680324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:16.680980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:16.683093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:16.684003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] PATH_INFO: `/v3/users/06ccada894e74ba4a08f9928e8036987` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:16.687062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ccada894e74ba4a08f9928e8036987)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:16.690562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:16.846742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ccada894e74ba4a08f9928e8036987'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:16.866701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ccada894e74ba4a08f9928e8036987'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:16.898707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ccada894e74ba4a08f9928e8036987'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:16.911302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ccada894e74ba4a08f9928e8036987'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:16.912406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ccada894e74ba4a08f9928e8036987'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:16.929335 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 761/1514] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:16 2022] DELETE /identity/v3/users/06ccada894e74ba4a08f9928e8036987 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:17.232107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f498-dad0-4b01-b573-933fb0bce610 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:17.232462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f498-dad0-4b01-b573-933fb0bce610 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:17.232749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f498-dad0-4b01-b573-933fb0bce610 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:17.235112 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 754/1515] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:48:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:17.242674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:48:17.244764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:17.435282 ncdv-master-1 devstack@keystone.service[99099]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Oct 15 20:48:17.435642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:17.437011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:17.445334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:17.706095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1be8d-850a-4989-99dc-035feba536fe admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:17.728710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12e35d-0aaf-4d24-957c-4889f5e10d46 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ce45a906224d74a2aaf34aac5175fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b96fc1fd9d574d609b8375160a01ea28', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:17.729428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12e35d-0aaf-4d24-957c-4889f5e10d46 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:17.729761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12e35d-0aaf-4d24-957c-4889f5e10d46 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:17.730031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12e35d-0aaf-4d24-957c-4889f5e10d46 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] PATH_INFO: `/v3/projects/788a7292d90d4a069e17d14bfcfcdb1d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:17.731482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12e35d-0aaf-4d24-957c-4889f5e10d46 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] RBAC: Authorizing `identity:get_project(project_id=788a7292d90d4a069e17d14bfcfcdb1d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:17.734729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12e35d-0aaf-4d24-957c-4889f5e10d46 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:17.767962 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 762/1516] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:48:17 2022] GET /identity/v3/projects/788a7292d90d4a069e17d14bfcfcdb1d => generated 375 bytes in 524 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:17.855745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f498-dad0-4b01-b573-933fb0bce610 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:17.857829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7216f498-dad0-4b01-b573-933fb0bce610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:17.883327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12e35d-0aaf-4d24-957c-4889f5e10d46 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:17.919428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12e35d-0aaf-4d24-957c-4889f5e10d46 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:17.947184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b9c4e-dfa7-47d7-83cd-a3fdda2c2e7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:17.949209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b9c4e-dfa7-47d7-83cd-a3fdda2c2e7c service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:17.951088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b9c4e-dfa7-47d7-83cd-a3fdda2c2e7c service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:17.952043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b9c4e-dfa7-47d7-83cd-a3fdda2c2e7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:17.957886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-461b9c4e-dfa7-47d7-83cd-a3fdda2c2e7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:18.006780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:18.029382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f498-dad0-4b01-b573-933fb0bce610 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:18.034934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:18.036005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:18.036486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:18.036947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] PATH_INFO: `/v3/users/1c8110368826488c8c24b1ca7831e139` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:18.039328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8110368826488c8c24b1ca7831e139)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:18.042123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:18.102560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b9c4e-dfa7-47d7-83cd-a3fdda2c2e7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:18.105064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b9c4e-dfa7-47d7-83cd-a3fdda2c2e7c service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:18.136955 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 763/1517] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:18.138137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8110368826488c8c24b1ca7831e139'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:18.158185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8110368826488c8c24b1ca7831e139'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:18.170691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8110368826488c8c24b1ca7831e139'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:18.219294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8110368826488c8c24b1ca7831e139'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:18.220410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8110368826488c8c24b1ca7831e139'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:18.233469 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 755/1518] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:17 2022] DELETE /identity/v3/users/1c8110368826488c8c24b1ca7831e139 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:18.235582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d081e5-4e3f-42e5-8d74-2b736b6e3b49 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:18.236353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d081e5-4e3f-42e5-8d74-2b736b6e3b49 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:18.236821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d081e5-4e3f-42e5-8d74-2b736b6e3b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:18.265259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:18.267081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:18.269260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b6d081e5-4e3f-42e5-8d74-2b736b6e3b49 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:18.270687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b6d081e5-4e3f-42e5-8d74-2b736b6e3b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:18.492301 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 764/1519] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:18.496065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c973cb-8069-4ea5-97c3-eb5029930e7f None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:18.496666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c973cb-8069-4ea5-97c3-eb5029930e7f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:18.497534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c973cb-8069-4ea5-97c3-eb5029930e7f None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:18.504548 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 765/1520] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:48:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:18.506772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e084471-8e1e-496c-9d63-80efd5f3ccf1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:18.507631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e084471-8e1e-496c-9d63-80efd5f3ccf1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:18.508186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e084471-8e1e-496c-9d63-80efd5f3ccf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:18.511960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cc568-16f3-4f26-b8aa-108552f13a89 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:18.525907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:18.527326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:18.527629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:18.527953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] PATH_INFO: `/v3/projects/0a478079144c42ffbcf918e78463b600` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:18.530667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a478079144c42ffbcf918e78463b600)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:18.539045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9e084471-8e1e-496c-9d63-80efd5f3ccf1 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:18.543434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e084471-8e1e-496c-9d63-80efd5f3ccf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:18.545977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:18.667364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a478079144c42ffbcf918e78463b600', 'request_id': 'req-acafbfd1-1321-4ac4-9333-460129b60341'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:18.680489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a478079144c42ffbcf918e78463b600', 'request_id': 'req-acafbfd1-1321-4ac4-9333-460129b60341'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:18.686742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:18.692102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a478079144c42ffbcf918e78463b600', 'request_id': 'req-acafbfd1-1321-4ac4-9333-460129b60341'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:18.743659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] The token cache is being invalidate because project 0a478079144c42ffbcf918e78463b600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:18.744118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:18.744931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:18.748556 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 756/1521] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:18 2022] DELETE /identity/v3/projects/0a478079144c42ffbcf918e78463b600 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:18.749397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:18.760152 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 766/1522] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:48:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:18.769723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:18.770908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e084471-8e1e-496c-9d63-80efd5f3ccf1 None None] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:48:18.775013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e084471-8e1e-496c-9d63-80efd5f3ccf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:18.788893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:18.899604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafbfd1-1321-4ac4-9333-460129b60341 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:18.906482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f2926-5689-4e25-9f1b-e8c015a9475b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:18.907268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f2926-5689-4e25-9f1b-e8c015a9475b service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:18.907560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f2926-5689-4e25-9f1b-e8c015a9475b service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:18.907828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f2926-5689-4e25-9f1b-e8c015a9475b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:18.915234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a97f2926-5689-4e25-9f1b-e8c015a9475b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:19.015115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e084471-8e1e-496c-9d63-80efd5f3ccf1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:19.019124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e084471-8e1e-496c-9d63-80efd5f3ccf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:19.023834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f2926-5689-4e25-9f1b-e8c015a9475b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:19.025957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f2926-5689-4e25-9f1b-e8c015a9475b service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:19.040630 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 757/1523] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:19.042517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461fe0f-e7aa-40fe-9b8a-3236a0a90cd8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:19.043055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461fe0f-e7aa-40fe-9b8a-3236a0a90cd8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:19.043699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461fe0f-e7aa-40fe-9b8a-3236a0a90cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:19.083856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a461fe0f-e7aa-40fe-9b8a-3236a0a90cd8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:19.085289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a461fe0f-e7aa-40fe-9b8a-3236a0a90cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:19.114795 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 758/1524] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:19.116576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b52a2-dea0-4148-a406-2f88345b9ca0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:19.116941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b52a2-dea0-4148-a406-2f88345b9ca0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:19.117241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b52a2-dea0-4148-a406-2f88345b9ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:19.190494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e084471-8e1e-496c-9d63-80efd5f3ccf1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:19.194805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-314b52a2-dea0-4148-a406-2f88345b9ca0 None None] MFA Rules not processed for user `b8343eb55cd148ef94141af734ef6abf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:19.196301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-314b52a2-dea0-4148-a406-2f88345b9ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:19.344711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12341f95-c4a4-4249-b2db-e22d090f6564 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ce45a906224d74a2aaf34aac5175fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b96fc1fd9d574d609b8375160a01ea28', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:19.345498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12341f95-c4a4-4249-b2db-e22d090f6564 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:19.345861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12341f95-c4a4-4249-b2db-e22d090f6564 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:19.349279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12341f95-c4a4-4249-b2db-e22d090f6564 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] PATH_INFO: `/v3/projects/788a7292d90d4a069e17d14bfcfcdb1d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:19.352696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12341f95-c4a4-4249-b2db-e22d090f6564 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] RBAC: Authorizing `identity:get_project(project_id=788a7292d90d4a069e17d14bfcfcdb1d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:19.355171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12341f95-c4a4-4249-b2db-e22d090f6564 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:19.362817 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 767/1525] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:48:18 2022] GET /identity/v3/projects/788a7292d90d4a069e17d14bfcfcdb1d => generated 375 bytes in 602 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:19.364278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12341f95-c4a4-4249-b2db-e22d090f6564 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:19.370553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-12341f95-c4a4-4249-b2db-e22d090f6564 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:19.389633 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 759/1526] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:19 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:19.397004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12726a9f-af4e-4a7a-bddf-5ad5b87a39d2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:19.397336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12726a9f-af4e-4a7a-bddf-5ad5b87a39d2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:19.397641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12726a9f-af4e-4a7a-bddf-5ad5b87a39d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:19.459671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:19.470958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-12726a9f-af4e-4a7a-bddf-5ad5b87a39d2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:19.476566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-12726a9f-af4e-4a7a-bddf-5ad5b87a39d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:19.520885 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 760/1527] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:19.523995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12726a9f-af4e-4a7a-bddf-5ad5b87a39d2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:19.529254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-12726a9f-af4e-4a7a-bddf-5ad5b87a39d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:19.552821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12341f95-c4a4-4249-b2db-e22d090f6564 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:19.567480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7242c7-5237-47da-8d96-bde156fde9cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:19.570362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7242c7-5237-47da-8d96-bde156fde9cc service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:19.577284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7242c7-5237-47da-8d96-bde156fde9cc service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:19.578905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7242c7-5237-47da-8d96-bde156fde9cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:19.584652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12726a9f-af4e-4a7a-bddf-5ad5b87a39d2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:19.584894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1f7242c7-5237-47da-8d96-bde156fde9cc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:19.606275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:19.606934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:19.607254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:19.607529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] PATH_INFO: `/v3/projects/017d5d6222204d759c027f3503532604` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:19.609369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=017d5d6222204d759c027f3503532604)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:19.611278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:19.711118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:19.782113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017d5d6222204d759c027f3503532604', 'request_id': 'req-894fc8ba-4ae6-456d-96ec-d1b4736537f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:19.786958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7242c7-5237-47da-8d96-bde156fde9cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:19.793298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7242c7-5237-47da-8d96-bde156fde9cc service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:19.805761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017d5d6222204d759c027f3503532604', 'request_id': 'req-894fc8ba-4ae6-456d-96ec-d1b4736537f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:19.815765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017d5d6222204d759c027f3503532604', 'request_id': 'req-894fc8ba-4ae6-456d-96ec-d1b4736537f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:19.829882 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 768/1528] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:48:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:19.831020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7242c7-5237-47da-8d96-bde156fde9cc service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:19.838793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1f7242c7-5237-47da-8d96-bde156fde9cc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:19.846468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] The token cache is being invalidate because project 017d5d6222204d759c027f3503532604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:19.848153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:19.850956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:19.854684 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 761/1529] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:19 2022] DELETE /identity/v3/projects/017d5d6222204d759c027f3503532604 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:19.860923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:19.864319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:20.131413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fc8ba-4ae6-456d-96ec-d1b4736537f0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:20.140431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7242c7-5237-47da-8d96-bde156fde9cc service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:20.141968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34885144-1f16-44b7-b8d3-4490fd3aeb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:20.144431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34885144-1f16-44b7-b8d3-4490fd3aeb08 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:20.145375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34885144-1f16-44b7-b8d3-4490fd3aeb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:20.145375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34885144-1f16-44b7-b8d3-4490fd3aeb08 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:20.147387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34885144-1f16-44b7-b8d3-4490fd3aeb08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:20.147739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34885144-1f16-44b7-b8d3-4490fd3aeb08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:20.150459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34885144-1f16-44b7-b8d3-4490fd3aeb08 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:20.260950 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 762/1530] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:20.291083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34885144-1f16-44b7-b8d3-4490fd3aeb08 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:20.302708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-34885144-1f16-44b7-b8d3-4490fd3aeb08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:20.308510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1db9d-2c3f-4638-936a-46cbfdee9133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:20.309291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1db9d-2c3f-4638-936a-46cbfdee9133 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:20.310010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1db9d-2c3f-4638-936a-46cbfdee9133 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:20.311380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1db9d-2c3f-4638-936a-46cbfdee9133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:20.314468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ffd1db9d-2c3f-4638-936a-46cbfdee9133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:20.338324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:20.415289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34885144-1f16-44b7-b8d3-4490fd3aeb08 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:20.453789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:20.453789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:20.454096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:20.454350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] PATH_INFO: `/v3/projects/54f631cb375e4c4a951d60f3d9f7402e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:20.459883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f631cb375e4c4a951d60f3d9f7402e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:20.469475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:20.529864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd1db9d-2c3f-4638-936a-46cbfdee9133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:20.539038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd1db9d-2c3f-4638-936a-46cbfdee9133 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:20.587230 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 769/1531] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:19 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 756 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:20.588744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ec55-39e0-4a97-9959-4650991f8386 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:20.594435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ec55-39e0-4a97-9959-4650991f8386 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:20.594674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ec55-39e0-4a97-9959-4650991f8386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:20.667708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-00f4ec55-39e0-4a97-9959-4650991f8386 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:20.669404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-00f4ec55-39e0-4a97-9959-4650991f8386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:20.673883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f631cb375e4c4a951d60f3d9f7402e', 'request_id': 'req-df39cbea-1419-4316-b00a-433400589872'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:20.694776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f631cb375e4c4a951d60f3d9f7402e', 'request_id': 'req-df39cbea-1419-4316-b00a-433400589872'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:20.719780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f631cb375e4c4a951d60f3d9f7402e', 'request_id': 'req-df39cbea-1419-4316-b00a-433400589872'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:20.756121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] The token cache is being invalidate because project 54f631cb375e4c4a951d60f3d9f7402e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:20.761432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:20.767512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:20.770696 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 763/1532] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:20 2022] DELETE /identity/v3/projects/54f631cb375e4c4a951d60f3d9f7402e => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:20.771748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:20.778081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:20.868672 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 770/1533] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:20.870355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4ec55-39e0-4a97-9959-4650991f8386 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:20.886088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-00f4ec55-39e0-4a97-9959-4650991f8386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:20.891315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:20.958317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4ec55-39e0-4a97-9959-4650991f8386 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:20.969421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:20.974218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:20.975132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:20.977965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] PATH_INFO: `/v3/users/e6f40eab39e740f0b9b646444f00d0df` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:20.981556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f40eab39e740f0b9b646444f00d0df)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:20.986083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:20.992031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39cbea-1419-4316-b00a-433400589872 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:21.001807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6159f-8aca-4ea8-89b3-e3ad69d32cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:21.002444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6159f-8aca-4ea8-89b3-e3ad69d32cce admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:21.002890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6159f-8aca-4ea8-89b3-e3ad69d32cce admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:21.003232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6159f-8aca-4ea8-89b3-e3ad69d32cce admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:21.003876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6159f-8aca-4ea8-89b3-e3ad69d32cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:21.006604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6159f-8aca-4ea8-89b3-e3ad69d32cce admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:21.063910 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 764/1534] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:20 2022] POST /identity/v3/projects => generated 375 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:21.064993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6159f-8aca-4ea8-89b3-e3ad69d32cce admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:21.071857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c0e6159f-8aca-4ea8-89b3-e3ad69d32cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:21.185621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f40eab39e740f0b9b646444f00d0df'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:21.207153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f40eab39e740f0b9b646444f00d0df'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:21.233016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f40eab39e740f0b9b646444f00d0df'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:21.243594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f40eab39e740f0b9b646444f00d0df'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:21.244620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f40eab39e740f0b9b646444f00d0df'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:21.258334 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 771/1535] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:20 2022] DELETE /identity/v3/users/e6f40eab39e740f0b9b646444f00d0df => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:21.258334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:21.268228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:21.280267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6159f-8aca-4ea8-89b3-e3ad69d32cce admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:21.338720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:21.452706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda84b9b-6182-43a2-afe5-b30956fc2f9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:21.453873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda84b9b-6182-43a2-afe5-b30956fc2f9c service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:21.454443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda84b9b-6182-43a2-afe5-b30956fc2f9c service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:21.454561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda84b9b-6182-43a2-afe5-b30956fc2f9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:21.462266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cda84b9b-6182-43a2-afe5-b30956fc2f9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:21.463311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bc743-558d-458e-8af1-ade81fde124a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:21.475491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:21.476189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:21.476618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:21.476731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] PATH_INFO: `/v3/users/6d248fb973f84e6b848b7f2678a884f4` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:21.480244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d248fb973f84e6b848b7f2678a884f4)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:21.488952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:21.590560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d248fb973f84e6b848b7f2678a884f4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:21.601662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:21.609185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d248fb973f84e6b848b7f2678a884f4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:21.627753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda84b9b-6182-43a2-afe5-b30956fc2f9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:21.629893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda84b9b-6182-43a2-afe5-b30956fc2f9c service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:21.633466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d248fb973f84e6b848b7f2678a884f4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:21.644400 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 765/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:21 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:21.645404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d248fb973f84e6b848b7f2678a884f4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:21.646156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000ae52-3333-46b2-a634-c86c8590c534 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:21.646423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d248fb973f84e6b848b7f2678a884f4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:21.646603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000ae52-3333-46b2-a634-c86c8590c534 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:21.647588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000ae52-3333-46b2-a634-c86c8590c534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:21.663674 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 772/1537] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:21 2022] DELETE /identity/v3/users/6d248fb973f84e6b848b7f2678a884f4 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:21.665008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:21.688518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6000ae52-3333-46b2-a634-c86c8590c534 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:21.690023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6000ae52-3333-46b2-a634-c86c8590c534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:21.704483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:21.862294 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 766/1538] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:21.863857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34e23a-f7e4-436d-a5fb-6fb1701aa58f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:21.865493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34e23a-f7e4-436d-a5fb-6fb1701aa58f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:21.867778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34e23a-f7e4-436d-a5fb-6fb1701aa58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:21.917447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9a34e23a-f7e4-436d-a5fb-6fb1701aa58f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:21.923809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9a34e23a-f7e4-436d-a5fb-6fb1701aa58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:21.926350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc4abf-49dc-4f8b-9249-1c48a054b5ab admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:21.939883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a94d6-236d-4bab-a7ec-7d8fdcd48123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:21.947070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a94d6-236d-4bab-a7ec-7d8fdcd48123 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:21.948609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a94d6-236d-4bab-a7ec-7d8fdcd48123 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:21.949301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a94d6-236d-4bab-a7ec-7d8fdcd48123 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:21.950223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a94d6-236d-4bab-a7ec-7d8fdcd48123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:21.954895 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 767/1539] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:21.956340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5b7b7-a3da-409a-b76f-71e7117473dc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:21.956472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a94d6-236d-4bab-a7ec-7d8fdcd48123 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:21.956805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5b7b7-a3da-409a-b76f-71e7117473dc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:21.957137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5b7b7-a3da-409a-b76f-71e7117473dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.003073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1ad5b7b7-a3da-409a-b76f-71e7117473dc None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:22.004834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1ad5b7b7-a3da-409a-b76f-71e7117473dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:22.015532 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 773/1540] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:21 2022] POST /identity/v3/users => generated 288 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:22.018200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a94d6-236d-4bab-a7ec-7d8fdcd48123 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:22.040682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a94d6-236d-4bab-a7ec-7d8fdcd48123 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:22.055935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb076a-8558-446c-9146-011ab4d65ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:22.057228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb076a-8558-446c-9146-011ab4d65ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:22.057675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb076a-8558-446c-9146-011ab4d65ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:22.057947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb076a-8558-446c-9146-011ab4d65ea1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.059219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb076a-8558-446c-9146-011ab4d65ea1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:22.059956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb076a-8558-446c-9146-011ab4d65ea1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:22.062170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb076a-8558-446c-9146-011ab4d65ea1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:22.093434 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 774/1541] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:22.094518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb076a-8558-446c-9146-011ab4d65ea1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:22.097142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a3eb076a-8558-446c-9146-011ab4d65ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:22.172050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:22.187550 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 768/1542] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Oct 15 20:48:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:22.188512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5b7b7-a3da-409a-b76f-71e7117473dc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:22.217070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5b7b7-a3da-409a-b76f-71e7117473dc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:22.233860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee368cca-7658-4c4c-b3c8-a91a0a2a311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:22.234492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee368cca-7658-4c4c-b3c8-a91a0a2a311e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:22.234766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee368cca-7658-4c4c-b3c8-a91a0a2a311e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:22.235030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee368cca-7658-4c4c-b3c8-a91a0a2a311e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.235681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee368cca-7658-4c4c-b3c8-a91a0a2a311e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:22.239020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee368cca-7658-4c4c-b3c8-a91a0a2a311e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:22.266597 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 769/1543] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:22.267679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee368cca-7658-4c4c-b3c8-a91a0a2a311e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:22.286842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb076a-8558-446c-9146-011ab4d65ea1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:22.293395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee368cca-7658-4c4c-b3c8-a91a0a2a311e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:22.299119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626eadba-ff8f-43b0-be4a-0fa8853e3bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:22.300131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eadba-ff8f-43b0-be4a-0fa8853e3bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:22.300617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eadba-ff8f-43b0-be4a-0fa8853e3bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:22.303664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eadba-ff8f-43b0-be4a-0fa8853e3bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.310304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34d984-340f-4f45-a7e4-421afbac9787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:22.310936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34d984-340f-4f45-a7e4-421afbac9787 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:22.311211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34d984-340f-4f45-a7e4-421afbac9787 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:22.311480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34d984-340f-4f45-a7e4-421afbac9787 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.312209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34d984-340f-4f45-a7e4-421afbac9787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:22.314179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34d984-340f-4f45-a7e4-421afbac9787 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:22.329309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626eadba-ff8f-43b0-be4a-0fa8853e3bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:22.331127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626eadba-ff8f-43b0-be4a-0fa8853e3bd0 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:22.365343 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 775/1544] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:22.368797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626eadba-ff8f-43b0-be4a-0fa8853e3bd0 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:22.377816 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 770/1545] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:22 2022] POST /identity/v3/projects => generated 395 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:22.378716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34d984-340f-4f45-a7e4-421afbac9787 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:22.391138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626eadba-ff8f-43b0-be4a-0fa8853e3bd0 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:22.475216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd17bd4-1e8f-4375-a3f2-920ad9101a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:22.476486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd17bd4-1e8f-4375-a3f2-920ad9101a86 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:22.476994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd17bd4-1e8f-4375-a3f2-920ad9101a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:22.477516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd17bd4-1e8f-4375-a3f2-920ad9101a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.495076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:22.504364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd17bd4-1e8f-4375-a3f2-920ad9101a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:22.508065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd17bd4-1e8f-4375-a3f2-920ad9101a86 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:22.522272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34d984-340f-4f45-a7e4-421afbac9787 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:22.524487 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 776/1546] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:22.525731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd17bd4-1e8f-4375-a3f2-920ad9101a86 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:22.533623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebfaa1-240a-43fd-b5e3-df63d7d6fde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:22.534256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebfaa1-240a-43fd-b5e3-df63d7d6fde6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:22.534531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebfaa1-240a-43fd-b5e3-df63d7d6fde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:22.534800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebfaa1-240a-43fd-b5e3-df63d7d6fde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.535447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebfaa1-240a-43fd-b5e3-df63d7d6fde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:22.537080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebfaa1-240a-43fd-b5e3-df63d7d6fde6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:22.602645 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 771/1547] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:22.606148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd17bd4-1e8f-4375-a3f2-920ad9101a86 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:22.607834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebfaa1-240a-43fd-b5e3-df63d7d6fde6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:22.621203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988ee30-d045-45c2-81be-a70042aa67e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:22.622946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988ee30-d045-45c2-81be-a70042aa67e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:22.624815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988ee30-d045-45c2-81be-a70042aa67e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:22.624815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988ee30-d045-45c2-81be-a70042aa67e3 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.626485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988ee30-d045-45c2-81be-a70042aa67e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:22.628432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988ee30-d045-45c2-81be-a70042aa67e3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:22.640206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebfaa1-240a-43fd-b5e3-df63d7d6fde6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:22.652451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa1f1f-e842-4eb4-9926-28caa947c6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:22.653209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa1f1f-e842-4eb4-9926-28caa947c6ba admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:22.653486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa1f1f-e842-4eb4-9926-28caa947c6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:22.653754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa1f1f-e842-4eb4-9926-28caa947c6ba admin admin] PATH_INFO: `/v3/projects/1f0b466076164a309233726db31b0efd/users/c84b11d2d54b43cbac87115e227ef50e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.689205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa1f1f-e842-4eb4-9926-28caa947c6ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0b466076164a309233726db31b0efd, user_id=c84b11d2d54b43cbac87115e227ef50e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:22.690437 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 777/1548] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:22 2022] POST /identity/v3/users => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:22.691381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988ee30-d045-45c2-81be-a70042aa67e3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:22.693354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa1f1f-e842-4eb4-9926-28caa947c6ba admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:22.713780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988ee30-d045-45c2-81be-a70042aa67e3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:22.733734 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 772/1549] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:22 2022] PUT /identity/v3/projects/1f0b466076164a309233726db31b0efd/users/c84b11d2d54b43cbac87115e227ef50e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:22.738375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c42eb0-5843-4e6d-a18b-19e54a9c1f21 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:22.738846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c42eb0-5843-4e6d-a18b-19e54a9c1f21 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:22.739100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c42eb0-5843-4e6d-a18b-19e54a9c1f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.814483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f6c42eb0-5843-4e6d-a18b-19e54a9c1f21 None None] MFA Rules not processed for user `c84b11d2d54b43cbac87115e227ef50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:22.818524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f6c42eb0-5843-4e6d-a18b-19e54a9c1f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:22.919931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd025fb-75d4-480e-b4ef-d036cf2d218c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:22.922392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd025fb-75d4-480e-b4ef-d036cf2d218c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:22.923426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd025fb-75d4-480e-b4ef-d036cf2d218c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:22.923750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd025fb-75d4-480e-b4ef-d036cf2d218c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:22.924482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd025fb-75d4-480e-b4ef-d036cf2d218c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:22.928978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd025fb-75d4-480e-b4ef-d036cf2d218c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:22.960873 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 778/1550] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:22.966632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd025fb-75d4-480e-b4ef-d036cf2d218c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:22.991966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd025fb-75d4-480e-b4ef-d036cf2d218c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:23.003346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5723295-e60f-4c64-8e34-7e2ca2a6d587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:23.004416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723295-e60f-4c64-8e34-7e2ca2a6d587 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:23.007536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723295-e60f-4c64-8e34-7e2ca2a6d587 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:23.008340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723295-e60f-4c64-8e34-7e2ca2a6d587 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:23.009276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5723295-e60f-4c64-8e34-7e2ca2a6d587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:23.011594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5723295-e60f-4c64-8e34-7e2ca2a6d587 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:23.028610 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 779/1551] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:23.033303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5723295-e60f-4c64-8e34-7e2ca2a6d587 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:23.042021 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 773/1552] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:23.046791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c42eb0-5843-4e6d-a18b-19e54a9c1f21 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:23.058551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5723295-e60f-4c64-8e34-7e2ca2a6d587 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:23.068114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c42eb0-5843-4e6d-a18b-19e54a9c1f21 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:23.070402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c3181-aac2-4ccf-affd-fc474a2dd41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:23.073133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c3181-aac2-4ccf-affd-fc474a2dd41a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:23.074404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c3181-aac2-4ccf-affd-fc474a2dd41a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:23.074691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c3181-aac2-4ccf-affd-fc474a2dd41a admin admin] PATH_INFO: `/v3/projects/826dd1e9524246118d199ef440c8e03f/users/006cf4ffc75243e1823e3ad33fd32d87/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:23.077222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f453e88-2cbf-4f02-8bb7-3a29e4020550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:23.078085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f453e88-2cbf-4f02-8bb7-3a29e4020550 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:23.078431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f453e88-2cbf-4f02-8bb7-3a29e4020550 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:23.078772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f453e88-2cbf-4f02-8bb7-3a29e4020550 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:23.079488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f453e88-2cbf-4f02-8bb7-3a29e4020550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:23.081970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f453e88-2cbf-4f02-8bb7-3a29e4020550 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:23.099030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c3181-aac2-4ccf-affd-fc474a2dd41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=826dd1e9524246118d199ef440c8e03f, user_id=006cf4ffc75243e1823e3ad33fd32d87, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:23.103393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c3181-aac2-4ccf-affd-fc474a2dd41a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:23.124330 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 774/1553] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:23 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:23.125061 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 780/1554] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:23 2022] PUT /identity/v3/projects/826dd1e9524246118d199ef440c8e03f/users/006cf4ffc75243e1823e3ad33fd32d87/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:23.143209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f453e88-2cbf-4f02-8bb7-3a29e4020550 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:23.146037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94301f-ae10-4df6-8fe8-958d513887c6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:23.146323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94301f-ae10-4df6-8fe8-958d513887c6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:23.149379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94301f-ae10-4df6-8fe8-958d513887c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:23.202404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:23.247263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-eb94301f-ae10-4df6-8fe8-958d513887c6 None None] MFA Rules not processed for user `006cf4ffc75243e1823e3ad33fd32d87`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:23.251208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eb94301f-ae10-4df6-8fe8-958d513887c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:23.285848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f453e88-2cbf-4f02-8bb7-3a29e4020550 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:23.313103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19247d-a76c-43ca-ba06-6d6a0d54cc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:23.315885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19247d-a76c-43ca-ba06-6d6a0d54cc3c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:23.316337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19247d-a76c-43ca-ba06-6d6a0d54cc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:23.316623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19247d-a76c-43ca-ba06-6d6a0d54cc3c admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:23.318968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19247d-a76c-43ca-ba06-6d6a0d54cc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:23.322022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19247d-a76c-43ca-ba06-6d6a0d54cc3c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:23.399412 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 775/1555] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:23 2022] POST /identity/v3/users => generated 293 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:23.403253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19247d-a76c-43ca-ba06-6d6a0d54cc3c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:23.440144 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 781/1556] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:23.445598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94301f-ae10-4df6-8fe8-958d513887c6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:23.454764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19247d-a76c-43ca-ba06-6d6a0d54cc3c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:23.466604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94301f-ae10-4df6-8fe8-958d513887c6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:23.473390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cea8b-d0fc-40a6-9729-7d3689b62c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:23.474735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cea8b-d0fc-40a6-9729-7d3689b62c0f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:23.475985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cea8b-d0fc-40a6-9729-7d3689b62c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:23.476671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cea8b-d0fc-40a6-9729-7d3689b62c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:23.477644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0cea8b-d0fc-40a6-9729-7d3689b62c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:23.479829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0cea8b-d0fc-40a6-9729-7d3689b62c0f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:23.488367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181632ea-a686-4d45-a270-4bc00993ff90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:23.491004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181632ea-a686-4d45-a270-4bc00993ff90 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:23.491329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181632ea-a686-4d45-a270-4bc00993ff90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:23.491564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181632ea-a686-4d45-a270-4bc00993ff90 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:23.492223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181632ea-a686-4d45-a270-4bc00993ff90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:23.507604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181632ea-a686-4d45-a270-4bc00993ff90 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:23.545690 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 782/1557] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:23 2022] POST /identity/v3/projects => generated 395 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:23.547052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cea8b-d0fc-40a6-9729-7d3689b62c0f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:23.555561 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 776/1558] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:23.563496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181632ea-a686-4d45-a270-4bc00993ff90 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:23.603132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181632ea-a686-4d45-a270-4bc00993ff90 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:23.621183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:23.808627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cea8b-d0fc-40a6-9729-7d3689b62c0f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:23.824814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb855af-c8c1-4af3-ab46-5a9d3174f143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:23.825997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb855af-c8c1-4af3-ab46-5a9d3174f143 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:23.826211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb855af-c8c1-4af3-ab46-5a9d3174f143 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:23.826470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb855af-c8c1-4af3-ab46-5a9d3174f143 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:23.827108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb855af-c8c1-4af3-ab46-5a9d3174f143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:23.831388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb855af-c8c1-4af3-ab46-5a9d3174f143 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:23.831851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b5eb0-7070-4069-8cf9-9d214aaf3b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:23.833723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b5eb0-7070-4069-8cf9-9d214aaf3b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:23.835018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b5eb0-7070-4069-8cf9-9d214aaf3b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:23.836102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b5eb0-7070-4069-8cf9-9d214aaf3b3d admin admin] PATH_INFO: `/v3/projects/5c7b5e1d939a4b4592b1a81e4a30bcab/users/15b61f670adb4431b2445f6c89e21b18/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:23.881020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b5eb0-7070-4069-8cf9-9d214aaf3b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c7b5e1d939a4b4592b1a81e4a30bcab, user_id=15b61f670adb4431b2445f6c89e21b18, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:23.888393 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 783/1559] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:23 2022] POST /identity/v3/users => generated 304 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:23.890690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb855af-c8c1-4af3-ab46-5a9d3174f143 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:23.891192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b5eb0-7070-4069-8cf9-9d214aaf3b3d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:23.921814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb855af-c8c1-4af3-ab46-5a9d3174f143 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:23.928646 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 777/1560] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:23 2022] PUT /identity/v3/projects/5c7b5e1d939a4b4592b1a81e4a30bcab/users/15b61f670adb4431b2445f6c89e21b18/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:23.933256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b5eb0-7070-4069-8cf9-9d214aaf3b3d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:24.111672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcfb16-1556-41d0-b435-1e8d5cf9de36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:24.112299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcfb16-1556-41d0-b435-1e8d5cf9de36 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:24.112576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcfb16-1556-41d0-b435-1e8d5cf9de36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:24.112844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcfb16-1556-41d0-b435-1e8d5cf9de36 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:24.113595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcfb16-1556-41d0-b435-1e8d5cf9de36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:24.115385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcfb16-1556-41d0-b435-1e8d5cf9de36 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:24.123799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b5eb0-7070-4069-8cf9-9d214aaf3b3d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:24.133816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:24.140713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03746e25-1b7e-4f02-9ec8-18d625596528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:24.141348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03746e25-1b7e-4f02-9ec8-18d625596528 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:24.141865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03746e25-1b7e-4f02-9ec8-18d625596528 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:24.143040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03746e25-1b7e-4f02-9ec8-18d625596528 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:24.145282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03746e25-1b7e-4f02-9ec8-18d625596528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:24.150538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03746e25-1b7e-4f02-9ec8-18d625596528 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:24.150947 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 784/1561] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:24.151866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcfb16-1556-41d0-b435-1e8d5cf9de36 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:24.166448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcfb16-1556-41d0-b435-1e8d5cf9de36 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:24.182114 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 778/1562] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:24.184972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03746e25-1b7e-4f02-9ec8-18d625596528 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:24.214249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf517fd-d2a5-4541-9081-f6651dd3dea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:24.214491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03746e25-1b7e-4f02-9ec8-18d625596528 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:24.215716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf517fd-d2a5-4541-9081-f6651dd3dea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:24.216804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf517fd-d2a5-4541-9081-f6651dd3dea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:24.217324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf517fd-d2a5-4541-9081-f6651dd3dea1 admin admin] PATH_INFO: `/v3/projects/8109cb9c5b8348d38601bf41ffb9fd7d/users/fe9fa273f51640469491f85de54d4bb4/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:24.229410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4288ffe-78fa-4f11-8885-70cf04c452a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:24.230569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4288ffe-78fa-4f11-8885-70cf04c452a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:24.230569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4288ffe-78fa-4f11-8885-70cf04c452a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:24.230963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4288ffe-78fa-4f11-8885-70cf04c452a9 admin admin] PATH_INFO: `/v3/projects/5c7b5e1d939a4b4592b1a81e4a30bcab/users/15b61f670adb4431b2445f6c89e21b18/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:24.234623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4288ffe-78fa-4f11-8885-70cf04c452a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c7b5e1d939a4b4592b1a81e4a30bcab, user_id=15b61f670adb4431b2445f6c89e21b18, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:24.244038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4288ffe-78fa-4f11-8885-70cf04c452a9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:24.261982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf517fd-d2a5-4541-9081-f6651dd3dea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8109cb9c5b8348d38601bf41ffb9fd7d, user_id=fe9fa273f51640469491f85de54d4bb4, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:24.266472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf517fd-d2a5-4541-9081-f6651dd3dea1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:24.284463 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 779/1563] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:24 2022] PUT /identity/v3/projects/5c7b5e1d939a4b4592b1a81e4a30bcab/users/15b61f670adb4431b2445f6c89e21b18/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:24.286394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd54173-e349-44d9-8532-1a8a6bd86539 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:24.286881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd54173-e349-44d9-8532-1a8a6bd86539 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:24.287041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd54173-e349-44d9-8532-1a8a6bd86539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:24.318459 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 785/1564] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:24 2022] PUT /identity/v3/projects/8109cb9c5b8348d38601bf41ffb9fd7d/users/fe9fa273f51640469491f85de54d4bb4/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:24.322547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf517fd-d2a5-4541-9081-f6651dd3dea1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:24.364851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4cd54173-e349-44d9-8532-1a8a6bd86539 None None] MFA Rules not processed for user `15b61f670adb4431b2445f6c89e21b18`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:24.375407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4cd54173-e349-44d9-8532-1a8a6bd86539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:24.513101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf517fd-d2a5-4541-9081-f6651dd3dea1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:24.550065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ee27c-88a9-4453-be44-b58ecdeb2be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:24.551293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ee27c-88a9-4453-be44-b58ecdeb2be0 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:24.552564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ee27c-88a9-4453-be44-b58ecdeb2be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:24.553421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ee27c-88a9-4453-be44-b58ecdeb2be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:24.554480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ee27c-88a9-4453-be44-b58ecdeb2be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:24.557152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ee27c-88a9-4453-be44-b58ecdeb2be0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:24.579866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:24.591362 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 780/1565] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:24.597377 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 786/1566] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:24.597859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd54173-e349-44d9-8532-1a8a6bd86539 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:24.603281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1b875-4aa3-4eb0-85d1-818c6eab7e72 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:24.603777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1b875-4aa3-4eb0-85d1-818c6eab7e72 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:24.604286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1b875-4aa3-4eb0-85d1-818c6eab7e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:24.621607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd54173-e349-44d9-8532-1a8a6bd86539 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:24.638236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7e3c8-064b-44e3-90fa-7181fceb5b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:24.638921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7e3c8-064b-44e3-90fa-7181fceb5b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:24.639198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7e3c8-064b-44e3-90fa-7181fceb5b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:24.639467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7e3c8-064b-44e3-90fa-7181fceb5b37 admin admin] PATH_INFO: `/v3/projects/8109cb9c5b8348d38601bf41ffb9fd7d/users/fe9fa273f51640469491f85de54d4bb4/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:24.643399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7e3c8-064b-44e3-90fa-7181fceb5b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8109cb9c5b8348d38601bf41ffb9fd7d, user_id=fe9fa273f51640469491f85de54d4bb4, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:24.650422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7e3c8-064b-44e3-90fa-7181fceb5b37 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:24.665576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3bb1b875-4aa3-4eb0-85d1-818c6eab7e72 None None] MFA Rules not processed for user `c84b11d2d54b43cbac87115e227ef50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:24.669725 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 781/1567] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:24 2022] PUT /identity/v3/projects/8109cb9c5b8348d38601bf41ffb9fd7d/users/fe9fa273f51640469491f85de54d4bb4/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:24.674770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3bb1b875-4aa3-4eb0-85d1-818c6eab7e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:24.685580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1f357-74c4-4246-aa0d-7b7593f344fc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:24.686158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1f357-74c4-4246-aa0d-7b7593f344fc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:24.686566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1f357-74c4-4246-aa0d-7b7593f344fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:24.785619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-23d1f357-74c4-4246-aa0d-7b7593f344fc None None] MFA Rules not processed for user `fe9fa273f51640469491f85de54d4bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:24.790920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-23d1f357-74c4-4246-aa0d-7b7593f344fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:24.812361 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 787/1568] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:24.814027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1b875-4aa3-4eb0-85d1-818c6eab7e72 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:24.816000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3bb1b875-4aa3-4eb0-85d1-818c6eab7e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:24.948334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1b875-4aa3-4eb0-85d1-818c6eab7e72 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:24.956085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:24.956859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:24.957161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:24.957468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] PATH_INFO: `/v3/projects/50a7fc61b0984db7b9a07f25e7738c0b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:24.958702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a7fc61b0984db7b9a07f25e7738c0b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:24.963577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:25.008672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:25.031280 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 782/1569] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:25.032642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1f357-74c4-4246-aa0d-7b7593f344fc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:25.047240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-23d1f357-74c4-4246-aa0d-7b7593f344fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:25.121075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a7fc61b0984db7b9a07f25e7738c0b', 'request_id': 'req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:25.142579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a7fc61b0984db7b9a07f25e7738c0b', 'request_id': 'req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:25.163944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a7fc61b0984db7b9a07f25e7738c0b', 'request_id': 'req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:25.193771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] The token cache is being invalidate because project 50a7fc61b0984db7b9a07f25e7738c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:25.194934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:25.196868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:25.200847 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 788/1570] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:24 2022] DELETE /identity/v3/projects/50a7fc61b0984db7b9a07f25e7738c0b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:25.204171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:25.206020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:25.237894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1f357-74c4-4246-aa0d-7b7593f344fc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:25.395012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b82c0-e8b4-4e96-8f96-ae27a588fd83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:25.395735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b82c0-e8b4-4e96-8f96-ae27a588fd83 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:25.395908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b82c0-e8b4-4e96-8f96-ae27a588fd83 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:25.396379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b82c0-e8b4-4e96-8f96-ae27a588fd83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:25.399585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ff2b82c0-e8b4-4e96-8f96-ae27a588fd83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:25.411993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74513e3-e342-4a76-86a6-bcd3c4fc6eae admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:25.417801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:25.444989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043d5e3-56f3-4aaf-b61e-85ac7fb38d62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:25.445732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043d5e3-56f3-4aaf-b61e-85ac7fb38d62 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:25.446176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043d5e3-56f3-4aaf-b61e-85ac7fb38d62 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:25.446460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043d5e3-56f3-4aaf-b61e-85ac7fb38d62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:25.450069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0043d5e3-56f3-4aaf-b61e-85ac7fb38d62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:25.570502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b82c0-e8b4-4e96-8f96-ae27a588fd83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:25.572178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b82c0-e8b4-4e96-8f96-ae27a588fd83 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:25.606053 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 783/1571] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:25.610417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab380-4c69-41a2-9255-cc1cf7e32ae0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:25.610678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab380-4c69-41a2-9255-cc1cf7e32ae0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:25.610954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab380-4c69-41a2-9255-cc1cf7e32ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:25.614148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043d5e3-56f3-4aaf-b61e-85ac7fb38d62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:25.619022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043d5e3-56f3-4aaf-b61e-85ac7fb38d62 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:25.671319 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 789/1572] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:25 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:25.672441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043d5e3-56f3-4aaf-b61e-85ac7fb38d62 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:25.675489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0043d5e3-56f3-4aaf-b61e-85ac7fb38d62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:25.698451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ce6ab380-4c69-41a2-9255-cc1cf7e32ae0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:25.699723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ce6ab380-4c69-41a2-9255-cc1cf7e32ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:25.848811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:25.973131 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 784/1573] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:25.975441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50261202-4abd-4ab0-aacf-cdaf7e756d20 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:25.976033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50261202-4abd-4ab0-aacf-cdaf7e756d20 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:25.976452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50261202-4abd-4ab0-aacf-cdaf7e756d20 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:25.978475 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 785/1574] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:48:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:25.982733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837abd3-0318-4e41-8316-6ee62d4da3e0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:25.983076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837abd3-0318-4e41-8316-6ee62d4da3e0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:25.983411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837abd3-0318-4e41-8316-6ee62d4da3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:26.010208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043d5e3-56f3-4aaf-b61e-85ac7fb38d62 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:26.025431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f837abd3-0318-4e41-8316-6ee62d4da3e0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:26.027654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f837abd3-0318-4e41-8316-6ee62d4da3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:26.036288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:26.037112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:26.038056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:26.038439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] PATH_INFO: `/v3/projects/788a7292d90d4a069e17d14bfcfcdb1d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:26.045949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=788a7292d90d4a069e17d14bfcfcdb1d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:26.048077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:26.059888 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 786/1575] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:26.064370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837abd3-0318-4e41-8316-6ee62d4da3e0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:26.092107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837abd3-0318-4e41-8316-6ee62d4da3e0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:26.118519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed5951-5e5a-46e5-a886-ff8f8e9603cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:26.119144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed5951-5e5a-46e5-a886-ff8f8e9603cb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:26.119451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed5951-5e5a-46e5-a886-ff8f8e9603cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:26.119714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed5951-5e5a-46e5-a886-ff8f8e9603cb admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:26.120393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed5951-5e5a-46e5-a886-ff8f8e9603cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:26.120694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed5951-5e5a-46e5-a886-ff8f8e9603cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:26.125576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed5951-5e5a-46e5-a886-ff8f8e9603cb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:26.148379 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 787/1576] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:26.148379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659ce2b-60dd-42de-9128-2a1680163d3f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:26.148379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659ce2b-60dd-42de-9128-2a1680163d3f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:26.148379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659ce2b-60dd-42de-9128-2a1680163d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:26.169176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4659ce2b-60dd-42de-9128-2a1680163d3f None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:26.170403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '788a7292d90d4a069e17d14bfcfcdb1d', 'request_id': 'req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:26.170527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4659ce2b-60dd-42de-9128-2a1680163d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:26.190421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '788a7292d90d4a069e17d14bfcfcdb1d', 'request_id': 'req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:26.206888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '788a7292d90d4a069e17d14bfcfcdb1d', 'request_id': 'req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:26.226051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] The token cache is being invalidate because project 788a7292d90d4a069e17d14bfcfcdb1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:26.227089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:26.228887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:26.230633 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 790/1577] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:25 2022] DELETE /identity/v3/projects/788a7292d90d4a069e17d14bfcfcdb1d => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:26.232310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:26.233935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:26.326321 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 788/1578] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:48:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:26.341642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659ce2b-60dd-42de-9128-2a1680163d3f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:26.345544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4659ce2b-60dd-42de-9128-2a1680163d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:26.358902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da63a6-ffcb-4d45-a96a-2eb20983d6c1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:26.367767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659ce2b-60dd-42de-9128-2a1680163d3f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:26.374678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:26.374962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfc9d1-1bea-4554-97a1-39e2ec74ceea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:26.375576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:26.376041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:26.376359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfc9d1-1bea-4554-97a1-39e2ec74ceea admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:26.376479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] PATH_INFO: `/v3/users/64ce45a906224d74a2aaf34aac5175fe` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:26.376856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfc9d1-1bea-4554-97a1-39e2ec74ceea admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:26.378262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfc9d1-1bea-4554-97a1-39e2ec74ceea admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:26.378489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ce45a906224d74a2aaf34aac5175fe)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:26.380800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:26.381180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfc9d1-1bea-4554-97a1-39e2ec74ceea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:26.384404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfc9d1-1bea-4554-97a1-39e2ec74ceea admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:26.416600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:26.464861 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 791/1579] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:26 2022] POST /identity/v3/projects => generated 407 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:26.466090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfc9d1-1bea-4554-97a1-39e2ec74ceea admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:26.560489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ce45a906224d74a2aaf34aac5175fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:26.623121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ce45a906224d74a2aaf34aac5175fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:26.625857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:26.642692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfc9d1-1bea-4554-97a1-39e2ec74ceea admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:26.643866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ce45a906224d74a2aaf34aac5175fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:26.669342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be813e8c-2107-43b2-b5bc-4b33becb8282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:26.669949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be813e8c-2107-43b2-b5bc-4b33becb8282 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:26.670251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be813e8c-2107-43b2-b5bc-4b33becb8282 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:26.670565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be813e8c-2107-43b2-b5bc-4b33becb8282 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:26.671215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be813e8c-2107-43b2-b5bc-4b33becb8282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:26.673269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be813e8c-2107-43b2-b5bc-4b33becb8282 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:26.676142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ce45a906224d74a2aaf34aac5175fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:26.678503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ce45a906224d74a2aaf34aac5175fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:26.703476 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 789/1580] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:26 2022] DELETE /identity/v3/users/64ce45a906224d74a2aaf34aac5175fe => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:26.734122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:26.736439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:26.740376 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 792/1581] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:26 2022] POST /identity/v3/users => generated 304 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:26.902393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c32bca-38b0-4154-9d27-08ab29f4fa41 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:26.912694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fd698-e1ed-4eb4-b824-6d6d8db5cf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:26.914268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fd698-e1ed-4eb4-b824-6d6d8db5cf16 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:26.914856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fd698-e1ed-4eb4-b824-6d6d8db5cf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:26.915238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fd698-e1ed-4eb4-b824-6d6d8db5cf16 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:26.916668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fd698-e1ed-4eb4-b824-6d6d8db5cf16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:26.921218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fd698-e1ed-4eb4-b824-6d6d8db5cf16 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:26.945452 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 790/1582] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:26.947967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be813e8c-2107-43b2-b5bc-4b33becb8282 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:26.948824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fd698-e1ed-4eb4-b824-6d6d8db5cf16 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:26.950064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-be813e8c-2107-43b2-b5bc-4b33becb8282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:26.995107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be813e8c-2107-43b2-b5bc-4b33becb8282 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:26.999058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fd698-e1ed-4eb4-b824-6d6d8db5cf16 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:27.017507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7827-4014-48d9-be66-994e25d6d716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:27.018263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7827-4014-48d9-be66-994e25d6d716 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:27.019147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7827-4014-48d9-be66-994e25d6d716 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:27.019522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7827-4014-48d9-be66-994e25d6d716 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:27.020276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c7827-4014-48d9-be66-994e25d6d716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:27.022106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c7827-4014-48d9-be66-994e25d6d716 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:27.027336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:27.028004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:27.028265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:27.028523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] PATH_INFO: `/v3/users/006cf4ffc75243e1823e3ad33fd32d87` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:27.043620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=006cf4ffc75243e1823e3ad33fd32d87)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:27.055001 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 791/1583] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:48:27.056140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7827-4014-48d9-be66-994e25d6d716 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:27.058294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ae8c7827-4014-48d9-be66-994e25d6d716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:27.066033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:27.242137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006cf4ffc75243e1823e3ad33fd32d87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:27.251880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006cf4ffc75243e1823e3ad33fd32d87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:27.253932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7827-4014-48d9-be66-994e25d6d716 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:27.261686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdb4ad-6335-4efc-acca-109b8c7e6dda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:27.262687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb4ad-6335-4efc-acca-109b8c7e6dda service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:27.263044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb4ad-6335-4efc-acca-109b8c7e6dda service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:27.263397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb4ad-6335-4efc-acca-109b8c7e6dda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:27.265677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e2bdb4ad-6335-4efc-acca-109b8c7e6dda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:27.282664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdb4ad-6335-4efc-acca-109b8c7e6dda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:27.284619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdb4ad-6335-4efc-acca-109b8c7e6dda service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:27.290776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006cf4ffc75243e1823e3ad33fd32d87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:27.308581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006cf4ffc75243e1823e3ad33fd32d87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:27.310109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006cf4ffc75243e1823e3ad33fd32d87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:27.319806 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 792/1584] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:27.326290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e7070-5ea2-4977-9a9f-270d4ad60087 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:27.326893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e7070-5ea2-4977-9a9f-270d4ad60087 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:27.327854 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 793/1585] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:26 2022] DELETE /identity/v3/users/006cf4ffc75243e1823e3ad33fd32d87 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:27.330848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e7070-5ea2-4977-9a9f-270d4ad60087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:27.332053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:27.340361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:27.348846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:27.393713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-551e7070-5ea2-4977-9a9f-270d4ad60087 None None] MFA Rules not processed for user `c84b11d2d54b43cbac87115e227ef50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:27.398129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-551e7070-5ea2-4977-9a9f-270d4ad60087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:27.556337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29b124-2413-45d6-a192-47ab773a30f1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:27.559039 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 793/1586] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:27.574652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b10f1-4b0f-4fbd-98f0-d75e09f65a76 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:27.579001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b10f1-4b0f-4fbd-98f0-d75e09f65a76 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:27.582273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b10f1-4b0f-4fbd-98f0-d75e09f65a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:27.603093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67548dd-a2fa-494e-9b59-26e4cc65bc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:27.603794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67548dd-a2fa-494e-9b59-26e4cc65bc22 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:27.617674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67548dd-a2fa-494e-9b59-26e4cc65bc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:27.617953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67548dd-a2fa-494e-9b59-26e4cc65bc22 admin admin] PATH_INFO: `/v3/projects/3fcb4e2a06884be29a06f7910ff2635a/users/f9e0957b26c14313867167480a60d849/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:27.648875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b34b10f1-4b0f-4fbd-98f0-d75e09f65a76 None None] MFA Rules not processed for user `14bd717b05ce4e40a1cd54c2db8474a6`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:27.652404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b34b10f1-4b0f-4fbd-98f0-d75e09f65a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:27.707091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67548dd-a2fa-494e-9b59-26e4cc65bc22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fcb4e2a06884be29a06f7910ff2635a, user_id=f9e0957b26c14313867167480a60d849, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:27.717457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67548dd-a2fa-494e-9b59-26e4cc65bc22 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:27.767164 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 794/1587] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:27 2022] PUT /identity/v3/projects/3fcb4e2a06884be29a06f7910ff2635a/users/f9e0957b26c14313867167480a60d849/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:27.767164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36311317-a189-4e4d-bb0c-03fadc97bd2a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:27.767164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36311317-a189-4e4d-bb0c-03fadc97bd2a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:27.767164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36311317-a189-4e4d-bb0c-03fadc97bd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:27.823715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-36311317-a189-4e4d-bb0c-03fadc97bd2a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:27.828025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-36311317-a189-4e4d-bb0c-03fadc97bd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:27.833178 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 794/1588] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:27 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:27.835927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11650-06e1-47b8-a33e-5328bab35d59 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:27.839685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11650-06e1-47b8-a33e-5328bab35d59 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:27.840100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11650-06e1-47b8-a33e-5328bab35d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:27.959435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:27.994088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1ec11650-06e1-47b8-a33e-5328bab35d59 None None] MFA Rules not processed for user `f9e0957b26c14313867167480a60d849`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:27.995368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1ec11650-06e1-47b8-a33e-5328bab35d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:28.018461 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 795/1589] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:28.092574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36311317-a189-4e4d-bb0c-03fadc97bd2a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:28.094513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-36311317-a189-4e4d-bb0c-03fadc97bd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:28.238488 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 795/1590] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:27 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:28.239448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec11650-06e1-47b8-a33e-5328bab35d59 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:28.241391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1ec11650-06e1-47b8-a33e-5328bab35d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:28.250054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36311317-a189-4e4d-bb0c-03fadc97bd2a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:28.256359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c740708-c470-4c2a-8939-9bf2cb15842b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:28.259184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c740708-c470-4c2a-8939-9bf2cb15842b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:28.259654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c740708-c470-4c2a-8939-9bf2cb15842b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:28.262352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c740708-c470-4c2a-8939-9bf2cb15842b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:28.284063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec11650-06e1-47b8-a33e-5328bab35d59 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:28.297121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:28.299748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:28.300107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:28.300408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] PATH_INFO: `/v3/projects/b96fc1fd9d574d609b8375160a01ea28` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:28.309517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96fc1fd9d574d609b8375160a01ea28)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:28.309737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c740708-c470-4c2a-8939-9bf2cb15842b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:28.311369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c740708-c470-4c2a-8939-9bf2cb15842b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:28.321615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:28.324781 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 796/1591] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:28.326007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c740708-c470-4c2a-8939-9bf2cb15842b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:28.339981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c740708-c470-4c2a-8939-9bf2cb15842b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:28.348554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018e343-aac8-4c2c-85fa-0846ec0efd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:28.350830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018e343-aac8-4c2c-85fa-0846ec0efd8d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:28.351386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018e343-aac8-4c2c-85fa-0846ec0efd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:28.351793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018e343-aac8-4c2c-85fa-0846ec0efd8d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:28.352933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018e343-aac8-4c2c-85fa-0846ec0efd8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:28.355393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018e343-aac8-4c2c-85fa-0846ec0efd8d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:28.402307 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 797/1592] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:28 2022] POST /identity/v3/projects => generated 409 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:28.406060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018e343-aac8-4c2c-85fa-0846ec0efd8d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:28.429269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:28.457435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96fc1fd9d574d609b8375160a01ea28', 'request_id': 'req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:28.473929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96fc1fd9d574d609b8375160a01ea28', 'request_id': 'req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:28.484473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96fc1fd9d574d609b8375160a01ea28', 'request_id': 'req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:28.511610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] The token cache is being invalidate because project b96fc1fd9d574d609b8375160a01ea28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:28.511965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:28.515038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c3fb92b0-8d86-4fc7-9739-be221e3bb8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:28.527224 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 796/1593] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:28 2022] DELETE /identity/v3/projects/b96fc1fd9d574d609b8375160a01ea28 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:28.528936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17aa6-4173-47c5-a1e9-a07fa8ee57f4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:28.529273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17aa6-4173-47c5-a1e9-a07fa8ee57f4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:28.529566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17aa6-4173-47c5-a1e9-a07fa8ee57f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:28.592142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-43d17aa6-4173-47c5-a1e9-a07fa8ee57f4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:28.593429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-43d17aa6-4173-47c5-a1e9-a07fa8ee57f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:28.644076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018e343-aac8-4c2c-85fa-0846ec0efd8d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:28.769256 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 797/1594] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:28.774623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bb05d-e046-412f-b258-e7bea08d0e77 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:28.775205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bb05d-e046-412f-b258-e7bea08d0e77 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:28.775750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bb05d-e046-412f-b258-e7bea08d0e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:28.831317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3b7bb05d-e046-412f-b258-e7bea08d0e77 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:28.833779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3b7bb05d-e046-412f-b258-e7bea08d0e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:28.839683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcc768-4ca2-4cb4-a491-29cfa39b69e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:28.840344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcc768-4ca2-4cb4-a491-29cfa39b69e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:28.840649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcc768-4ca2-4cb4-a491-29cfa39b69e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:28.840898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcc768-4ca2-4cb4-a491-29cfa39b69e1 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:28.841891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcc768-4ca2-4cb4-a491-29cfa39b69e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:28.844564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcc768-4ca2-4cb4-a491-29cfa39b69e1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:28.864298 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 798/1595] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:28.866457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae9b40-9515-4d75-aa85-4a8b7e9c5504 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:28.866849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae9b40-9515-4d75-aa85-4a8b7e9c5504 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:28.867146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae9b40-9515-4d75-aa85-4a8b7e9c5504 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:28.868440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:28.874619 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 799/1596] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:48:28 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:28.875597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae9b40-9515-4d75-aa85-4a8b7e9c5504 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:28.887310 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 798/1597] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:28 2022] POST /identity/v3/users => generated 311 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:28.891336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dffa8-a9a2-4de2-b6c4-f5455387f2d5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:28.893158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dffa8-a9a2-4de2-b6c4-f5455387f2d5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:28.893848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dffa8-a9a2-4de2-b6c4-f5455387f2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:28.915460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae9b40-9515-4d75-aa85-4a8b7e9c5504 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:28.930837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64e026-4134-4f51-b7fc-d3dad45c1513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:28.934483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64e026-4134-4f51-b7fc-d3dad45c1513 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:28.935369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64e026-4134-4f51-b7fc-d3dad45c1513 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:28.936016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64e026-4134-4f51-b7fc-d3dad45c1513 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:28.939000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64e026-4134-4f51-b7fc-d3dad45c1513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:28.940464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64e026-4134-4f51-b7fc-d3dad45c1513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:28.947788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64e026-4134-4f51-b7fc-d3dad45c1513 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:28.968040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d07dffa8-a9a2-4de2-b6c4-f5455387f2d5 None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:28.972335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d07dffa8-a9a2-4de2-b6c4-f5455387f2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:28.994841 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 800/1598] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:28.998246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64e026-4134-4f51-b7fc-d3dad45c1513 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:29.002732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dc64e026-4134-4f51-b7fc-d3dad45c1513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:29.043474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64e026-4134-4f51-b7fc-d3dad45c1513 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:29.058940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c321c51-e7d2-4a81-8c8d-045dfae5fa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:29.060868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c321c51-e7d2-4a81-8c8d-045dfae5fa17 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.062122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c321c51-e7d2-4a81-8c8d-045dfae5fa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.062503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c321c51-e7d2-4a81-8c8d-045dfae5fa17 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.064709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c321c51-e7d2-4a81-8c8d-045dfae5fa17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:29.069434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c321c51-e7d2-4a81-8c8d-045dfae5fa17 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:29.100632 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 801/1599] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:29.104304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c321c51-e7d2-4a81-8c8d-045dfae5fa17 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:29.134888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c321c51-e7d2-4a81-8c8d-045dfae5fa17 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:29.137113 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 799/1600] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:48:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:29.139705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07dffa8-a9a2-4de2-b6c4-f5455387f2d5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:29.145723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba89dc8-918f-49d6-9cfc-0f0a18d4484e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:29.147611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba89dc8-918f-49d6-9cfc-0f0a18d4484e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.149060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba89dc8-918f-49d6-9cfc-0f0a18d4484e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.149825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba89dc8-918f-49d6-9cfc-0f0a18d4484e admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.152374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba89dc8-918f-49d6-9cfc-0f0a18d4484e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:29.156555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba89dc8-918f-49d6-9cfc-0f0a18d4484e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:29.165191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07dffa8-a9a2-4de2-b6c4-f5455387f2d5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:29.180981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4f972-2885-4b1d-92ca-2b7eec08baf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:29.183623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4f972-2885-4b1d-92ca-2b7eec08baf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.183965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4f972-2885-4b1d-92ca-2b7eec08baf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.186492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4f972-2885-4b1d-92ca-2b7eec08baf2 admin admin] PATH_INFO: `/v3/projects/341f31852cf343caa3a4e6abf2b6a8c2/users/f10c851fb47541728f7b044fbcff158a/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.224945 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 802/1601] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:29 2022] POST /identity/v3/projects => generated 403 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:29.226886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602df3fa-5a75-4394-a057-edd5473905cf None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.227203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602df3fa-5a75-4394-a057-edd5473905cf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.229468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602df3fa-5a75-4394-a057-edd5473905cf None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.233780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4f972-2885-4b1d-92ca-2b7eec08baf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=341f31852cf343caa3a4e6abf2b6a8c2, user_id=f10c851fb47541728f7b044fbcff158a, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:29.234207 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 803/1602] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:48:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:29.235028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602df3fa-5a75-4394-a057-edd5473905cf None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:29.239570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4f972-2885-4b1d-92ca-2b7eec08baf2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:29.242897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-602df3fa-5a75-4394-a057-edd5473905cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:29.278437 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 800/1603] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:29 2022] PUT /identity/v3/projects/341f31852cf343caa3a4e6abf2b6a8c2/users/f10c851fb47541728f7b044fbcff158a/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:29.278437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4f972-2885-4b1d-92ca-2b7eec08baf2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:29.416594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602df3fa-5a75-4394-a057-edd5473905cf None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:29.454966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4f972-2885-4b1d-92ca-2b7eec08baf2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:29.470028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89760f-0a29-45a2-9c24-76bc703b9800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:29.470630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89760f-0a29-45a2-9c24-76bc703b9800 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.470899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89760f-0a29-45a2-9c24-76bc703b9800 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.471159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89760f-0a29-45a2-9c24-76bc703b9800 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.471883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89760f-0a29-45a2-9c24-76bc703b9800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:29.487175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89760f-0a29-45a2-9c24-76bc703b9800 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:29.507034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:29.555877 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 801/1604] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:29 2022] POST /identity/v3/users => generated 302 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:29.560992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad299f7-ae8c-493e-a225-11afff450153 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.561301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad299f7-ae8c-493e-a225-11afff450153 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.561564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad299f7-ae8c-493e-a225-11afff450153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.578391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfa490-f32d-4b1a-91ed-e3e91c4047d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:29.581430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfa490-f32d-4b1a-91ed-e3e91c4047d4 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.581933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfa490-f32d-4b1a-91ed-e3e91c4047d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.582379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfa490-f32d-4b1a-91ed-e3e91c4047d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.602165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfa490-f32d-4b1a-91ed-e3e91c4047d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:29.603609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfa490-f32d-4b1a-91ed-e3e91c4047d4 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:29.617621 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 804/1605] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:29.621228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfa490-f32d-4b1a-91ed-e3e91c4047d4 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:29.641912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1ad299f7-ae8c-493e-a225-11afff450153 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:29.641912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1ad299f7-ae8c-493e-a225-11afff450153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:29.669488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfa490-f32d-4b1a-91ed-e3e91c4047d4 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:29.680296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ab839-941f-4ba1-a8a7-2d74bb21c8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:29.681103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ab839-941f-4ba1-a8a7-2d74bb21c8ad admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.682722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ab839-941f-4ba1-a8a7-2d74bb21c8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.683152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ab839-941f-4ba1-a8a7-2d74bb21c8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.684521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ab839-941f-4ba1-a8a7-2d74bb21c8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:29.687092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ab839-941f-4ba1-a8a7-2d74bb21c8ad admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:29.720829 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 805/1606] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:29.722757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ab839-941f-4ba1-a8a7-2d74bb21c8ad admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:29.738072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ab839-941f-4ba1-a8a7-2d74bb21c8ad admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:29.774186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189dfb1-a126-4854-8ef7-45224d00f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:29.774965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189dfb1-a126-4854-8ef7-45224d00f299 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.775348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189dfb1-a126-4854-8ef7-45224d00f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.775718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189dfb1-a126-4854-8ef7-45224d00f299 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.776469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189dfb1-a126-4854-8ef7-45224d00f299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:29.778227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189dfb1-a126-4854-8ef7-45224d00f299 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:29.796607 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 806/1607] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:29.797620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189dfb1-a126-4854-8ef7-45224d00f299 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:29.812596 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 802/1608] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:48:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:29.813510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad299f7-ae8c-493e-a225-11afff450153 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:29.827761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189dfb1-a126-4854-8ef7-45224d00f299 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:29.835366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:29.838674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c8b25-6107-4476-8233-dee0643f6f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:29.839364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8b25-6107-4476-8233-dee0643f6f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.839889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8b25-6107-4476-8233-dee0643f6f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.843950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad299f7-ae8c-493e-a225-11afff450153 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:29.846105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8b25-6107-4476-8233-dee0643f6f17 admin admin] PATH_INFO: `/v3/projects/341f31852cf343caa3a4e6abf2b6a8c2/users/f10c851fb47541728f7b044fbcff158a/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.851263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c8b25-6107-4476-8233-dee0643f6f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=341f31852cf343caa3a4e6abf2b6a8c2, user_id=f10c851fb47541728f7b044fbcff158a, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:29.856849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614d4e0-e95f-467a-bffc-256bf66924ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:29.857499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614d4e0-e95f-467a-bffc-256bf66924ee admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.857770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614d4e0-e95f-467a-bffc-256bf66924ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.857874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c8b25-6107-4476-8233-dee0643f6f17 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:29.858036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614d4e0-e95f-467a-bffc-256bf66924ee admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.858747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614d4e0-e95f-467a-bffc-256bf66924ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:29.863199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614d4e0-e95f-467a-bffc-256bf66924ee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:29.898113 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 803/1610] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:29.898113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614d4e0-e95f-467a-bffc-256bf66924ee admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:29.898862 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 807/1610] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:29 2022] PUT /identity/v3/projects/341f31852cf343caa3a4e6abf2b6a8c2/users/f10c851fb47541728f7b044fbcff158a/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:29.901366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07692777-29cd-47d2-b921-a7fbf7c1b506 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:29.901874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07692777-29cd-47d2-b921-a7fbf7c1b506 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:29.902873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07692777-29cd-47d2-b921-a7fbf7c1b506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:29.921291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614d4e0-e95f-467a-bffc-256bf66924ee admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:30.008568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-07692777-29cd-47d2-b921-a7fbf7c1b506 None None] MFA Rules not processed for user `f10c851fb47541728f7b044fbcff158a`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:30.013009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-07692777-29cd-47d2-b921-a7fbf7c1b506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:30.039047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe092a-0c31-4eb4-97c1-8f1b43dd3651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:30.039872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe092a-0c31-4eb4-97c1-8f1b43dd3651 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:30.040233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe092a-0c31-4eb4-97c1-8f1b43dd3651 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:30.040500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe092a-0c31-4eb4-97c1-8f1b43dd3651 admin admin] PATH_INFO: `/v3/projects/180fa0d40e2546448720f9c60a0d581f/users/3a5dc54f7703403fa7e19332d5ff81e8/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:30.061632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe092a-0c31-4eb4-97c1-8f1b43dd3651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=180fa0d40e2546448720f9c60a0d581f, user_id=3a5dc54f7703403fa7e19332d5ff81e8, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:30.065050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe092a-0c31-4eb4-97c1-8f1b43dd3651 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:30.116955 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 804/1611] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:29 2022] PUT /identity/v3/projects/180fa0d40e2546448720f9c60a0d581f/users/3a5dc54f7703403fa7e19332d5ff81e8/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:30.120656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe092a-0c31-4eb4-97c1-8f1b43dd3651 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:30.125572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-61fe092a-0c31-4eb4-97c1-8f1b43dd3651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:30.246786 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 808/1612] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:29 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:30.248193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39462a-734d-495a-9052-c967b27c9b22 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:30.248478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39462a-734d-495a-9052-c967b27c9b22 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:30.248907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39462a-734d-495a-9052-c967b27c9b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:30.288949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe092a-0c31-4eb4-97c1-8f1b43dd3651 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:30.294407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:30.295042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:30.295332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:30.295568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] PATH_INFO: `/v3/projects/826dd1e9524246118d199ef440c8e03f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:30.296701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=826dd1e9524246118d199ef440c8e03f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:30.298717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:30.311599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5c39462a-734d-495a-9052-c967b27c9b22 None None] MFA Rules not processed for user `3a5dc54f7703403fa7e19332d5ff81e8`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:30.312842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5c39462a-734d-495a-9052-c967b27c9b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:30.371713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '826dd1e9524246118d199ef440c8e03f', 'request_id': 'req-054ff2f0-2a61-4071-a1a7-b26b56540e82'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:30.380817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '826dd1e9524246118d199ef440c8e03f', 'request_id': 'req-054ff2f0-2a61-4071-a1a7-b26b56540e82'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:30.389877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '826dd1e9524246118d199ef440c8e03f', 'request_id': 'req-054ff2f0-2a61-4071-a1a7-b26b56540e82'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:30.404621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] The token cache is being invalidate because project 826dd1e9524246118d199ef440c8e03f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:30.405390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:30.407870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:30.415052 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 805/1613] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:30 2022] DELETE /identity/v3/projects/826dd1e9524246118d199ef440c8e03f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:30.416204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:30.419867 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 809/1614] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:30.420861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39462a-734d-495a-9052-c967b27c9b22 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:30.426942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:30.440600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:30.445477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39462a-734d-495a-9052-c967b27c9b22 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:30.457196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:30.535348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:30.536166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:30.536603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:30.536785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ff2f0-2a61-4071-a1a7-b26b56540e82 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:30.537916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] PATH_INFO: `/v3/users/fe9fa273f51640469491f85de54d4bb4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:30.539163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9fa273f51640469491f85de54d4bb4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:30.541413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:30.543183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8358d05-1225-4c57-9b5e-f3ed7e1ab618 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:30.543730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8358d05-1225-4c57-9b5e-f3ed7e1ab618 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:30.544006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8358d05-1225-4c57-9b5e-f3ed7e1ab618 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:30.544276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8358d05-1225-4c57-9b5e-f3ed7e1ab618 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:30.546059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a8358d05-1225-4c57-9b5e-f3ed7e1ab618 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:30.559497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8358d05-1225-4c57-9b5e-f3ed7e1ab618 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:30.561123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8358d05-1225-4c57-9b5e-f3ed7e1ab618 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:30.573977 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 806/1615] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:30.575119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8358d05-1225-4c57-9b5e-f3ed7e1ab618 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:30.576991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a8358d05-1225-4c57-9b5e-f3ed7e1ab618 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:30.591085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8358d05-1225-4c57-9b5e-f3ed7e1ab618 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:30.601868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1c1a5-fef9-4c5b-9762-4af606f8b0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:30.602154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1c1a5-fef9-4c5b-9762-4af606f8b0af admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:30.602552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1c1a5-fef9-4c5b-9762-4af606f8b0af admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:30.602893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1c1a5-fef9-4c5b-9762-4af606f8b0af admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:30.603649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1c1a5-fef9-4c5b-9762-4af606f8b0af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:30.605784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1c1a5-fef9-4c5b-9762-4af606f8b0af admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:30.638511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9fa273f51640469491f85de54d4bb4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:30.648712 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 807/1616] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:30 2022] POST /identity/v3/projects => generated 403 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:30.648712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1c1a5-fef9-4c5b-9762-4af606f8b0af admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:30.653910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1da1c1a5-fef9-4c5b-9762-4af606f8b0af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:30.694362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9fa273f51640469491f85de54d4bb4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:30.708136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9fa273f51640469491f85de54d4bb4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:30.729387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9fa273f51640469491f85de54d4bb4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:30.731179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9fa273f51640469491f85de54d4bb4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:30.748946 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 810/1617] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:30 2022] DELETE /identity/v3/users/fe9fa273f51640469491f85de54d4bb4 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:30.749887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:30.751519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:30.869314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1c1a5-fef9-4c5b-9762-4af606f8b0af admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:30.883351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd3486-d1df-4232-bce3-83132ab7e66b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:30.889076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdff926-3c04-41fc-ab41-8d90ccd24cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:30.889701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdff926-3c04-41fc-ab41-8d90ccd24cad admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:30.889974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdff926-3c04-41fc-ab41-8d90ccd24cad admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:30.890366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdff926-3c04-41fc-ab41-8d90ccd24cad admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:30.891591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdff926-3c04-41fc-ab41-8d90ccd24cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:30.894999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdff926-3c04-41fc-ab41-8d90ccd24cad admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:30.937251 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 811/1618] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:30 2022] POST /identity/v3/users => generated 308 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:30.947916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdff926-3c04-41fc-ab41-8d90ccd24cad admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:30.966832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdff926-3c04-41fc-ab41-8d90ccd24cad admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:30.974707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f0778-f1f6-496c-837b-ee0c1b1582fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:30.975318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f0778-f1f6-496c-837b-ee0c1b1582fe admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:30.975590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f0778-f1f6-496c-837b-ee0c1b1582fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:30.975842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f0778-f1f6-496c-837b-ee0c1b1582fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:30.976748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f0778-f1f6-496c-837b-ee0c1b1582fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:30.979372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f0778-f1f6-496c-837b-ee0c1b1582fe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:30.996437 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 812/1619] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:48:31.000652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f0778-f1f6-496c-837b-ee0c1b1582fe admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:31.002625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a49ab-0d6a-4ef7-8ab6-d333f42c9ac0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:31.003370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a49ab-0d6a-4ef7-8ab6-d333f42c9ac0 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:31.003650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a49ab-0d6a-4ef7-8ab6-d333f42c9ac0 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:31.003971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a49ab-0d6a-4ef7-8ab6-d333f42c9ac0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:31.007522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c55a49ab-0d6a-4ef7-8ab6-d333f42c9ac0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:31.015223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f0778-f1f6-496c-837b-ee0c1b1582fe admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:31.016022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:31.033112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad6288-e3f1-446f-a89b-a62d87232db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:31.034322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad6288-e3f1-446f-a89b-a62d87232db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:31.036342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad6288-e3f1-446f-a89b-a62d87232db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:31.049825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad6288-e3f1-446f-a89b-a62d87232db5 admin admin] PATH_INFO: `/v3/projects/eb4fbfc6501a4ff4bb849f86cd3fbc04/users/2c6909b25e674917abe5692130bb05fc/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:31.059774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:31.113579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad6288-e3f1-446f-a89b-a62d87232db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4fbfc6501a4ff4bb849f86cd3fbc04, user_id=2c6909b25e674917abe5692130bb05fc, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:31.123721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad6288-e3f1-446f-a89b-a62d87232db5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:31.145151 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 813/1620] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:30 2022] PUT /identity/v3/projects/eb4fbfc6501a4ff4bb849f86cd3fbc04/users/2c6909b25e674917abe5692130bb05fc/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:31.149290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad6288-e3f1-446f-a89b-a62d87232db5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:31.168328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a49ab-0d6a-4ef7-8ab6-d333f42c9ac0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:31.174269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a49ab-0d6a-4ef7-8ab6-d333f42c9ac0 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:31.217130 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 808/1621] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:30 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:31.300975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad6288-e3f1-446f-a89b-a62d87232db5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:31.313408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ae76b-f23e-4048-8f63-f3b6e83dc6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:31.314251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ae76b-f23e-4048-8f63-f3b6e83dc6ec admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:31.314513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ae76b-f23e-4048-8f63-f3b6e83dc6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:31.314978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ae76b-f23e-4048-8f63-f3b6e83dc6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:31.316383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ae76b-f23e-4048-8f63-f3b6e83dc6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:31.319194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ae76b-f23e-4048-8f63-f3b6e83dc6ec admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:31.337320 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 814/1622] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:31.341766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a49ab-0d6a-4ef7-8ab6-d333f42c9ac0 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:31.367906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a49ab-0d6a-4ef7-8ab6-d333f42c9ac0 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:31.379042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0fd95-b670-41f5-b5e4-e82b166e9eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:31.381923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0fd95-b670-41f5-b5e4-e82b166e9eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:31.382213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0fd95-b670-41f5-b5e4-e82b166e9eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:31.384913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0fd95-b670-41f5-b5e4-e82b166e9eaa admin admin] PATH_INFO: `/v3/projects/eb4fbfc6501a4ff4bb849f86cd3fbc04/users/2c6909b25e674917abe5692130bb05fc/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:31.389512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0fd95-b670-41f5-b5e4-e82b166e9eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4fbfc6501a4ff4bb849f86cd3fbc04, user_id=2c6909b25e674917abe5692130bb05fc, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:31.401924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0fd95-b670-41f5-b5e4-e82b166e9eaa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:31.405435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ae76b-f23e-4048-8f63-f3b6e83dc6ec admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:31.407633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-711ae76b-f23e-4048-8f63-f3b6e83dc6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:31.464889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ae76b-f23e-4048-8f63-f3b6e83dc6ec admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:31.470298 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 809/1623] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:31 2022] PUT /identity/v3/projects/eb4fbfc6501a4ff4bb849f86cd3fbc04/users/2c6909b25e674917abe5692130bb05fc/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:31.471192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0fd95-b670-41f5-b5e4-e82b166e9eaa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:31.472835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aec0fd95-b670-41f5-b5e4-e82b166e9eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:31.577868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:31.578587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:31.578880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:31.579176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] PATH_INFO: `/v3/projects/8109cb9c5b8348d38601bf41ffb9fd7d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:31.580376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8109cb9c5b8348d38601bf41ffb9fd7d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:31.583080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:31.617464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0fd95-b670-41f5-b5e4-e82b166e9eaa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:31.634756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efeeb3-421d-47a5-acaf-d80f3d1190fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:31.635704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efeeb3-421d-47a5-acaf-d80f3d1190fe service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:31.635986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efeeb3-421d-47a5-acaf-d80f3d1190fe service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:31.636248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efeeb3-421d-47a5-acaf-d80f3d1190fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:31.643048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d7efeeb3-421d-47a5-acaf-d80f3d1190fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:31.710722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efeeb3-421d-47a5-acaf-d80f3d1190fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:31.712659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efeeb3-421d-47a5-acaf-d80f3d1190fe service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:31.723137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 810/1624] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:31 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:31.726075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646de11a-bac2-40be-b399-1b50ab115608 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:31.726345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646de11a-bac2-40be-b399-1b50ab115608 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:31.726604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646de11a-bac2-40be-b399-1b50ab115608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:31.764452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-646de11a-bac2-40be-b399-1b50ab115608 None None] MFA Rules not processed for user `2c6909b25e674917abe5692130bb05fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:31.765896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-646de11a-bac2-40be-b399-1b50ab115608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:31.779675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8109cb9c5b8348d38601bf41ffb9fd7d', 'request_id': 'req-00beb42b-2310-4bab-8043-8fbeb66f8063'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:31.786862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:31.790078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8109cb9c5b8348d38601bf41ffb9fd7d', 'request_id': 'req-00beb42b-2310-4bab-8043-8fbeb66f8063'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:31.799806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8109cb9c5b8348d38601bf41ffb9fd7d', 'request_id': 'req-00beb42b-2310-4bab-8043-8fbeb66f8063'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:31.816127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] The token cache is being invalidate because project 8109cb9c5b8348d38601bf41ffb9fd7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:31.816333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:31.817008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-00beb42b-2310-4bab-8043-8fbeb66f8063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:31.819937 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 815/1625] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:31 2022] DELETE /identity/v3/projects/8109cb9c5b8348d38601bf41ffb9fd7d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:31.825676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32481b9-23f7-4114-83f1-cb1d3822e8e5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:31.825963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32481b9-23f7-4114-83f1-cb1d3822e8e5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:31.826231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32481b9-23f7-4114-83f1-cb1d3822e8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:31.851839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d32481b9-23f7-4114-83f1-cb1d3822e8e5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:31.854026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d32481b9-23f7-4114-83f1-cb1d3822e8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:31.861940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:31.871746 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 811/1626] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:31 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:31.891440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646de11a-bac2-40be-b399-1b50ab115608 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:31.893820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-646de11a-bac2-40be-b399-1b50ab115608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:31.985246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646de11a-bac2-40be-b399-1b50ab115608 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:31.992493 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 816/1627] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:31.993273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b527b-01f8-4349-bc90-9a8a7b595672 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:31.994472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b527b-01f8-4349-bc90-9a8a7b595672 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:31.994472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b527b-01f8-4349-bc90-9a8a7b595672 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:31.994472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b527b-01f8-4349-bc90-9a8a7b595672 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:31.995642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32481b9-23f7-4114-83f1-cb1d3822e8e5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:31.998273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d32481b9-23f7-4114-83f1-cb1d3822e8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:32.014862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b527b-01f8-4349-bc90-9a8a7b595672 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.016458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b527b-01f8-4349-bc90-9a8a7b595672 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.017784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32481b9-23f7-4114-83f1-cb1d3822e8e5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:32.024783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:32.025516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.025794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.026060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] PATH_INFO: `/v3/users/f9e0957b26c14313867167480a60d849` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.027272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9e0957b26c14313867167480a60d849)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.029652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.029834 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 812/1628] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:31 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:32.030954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc142479-7213-4522-ba0e-59d97034821e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.031239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc142479-7213-4522-ba0e-59d97034821e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.031511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc142479-7213-4522-ba0e-59d97034821e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.076880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-dc142479-7213-4522-ba0e-59d97034821e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:32.085514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dc142479-7213-4522-ba0e-59d97034821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:32.098390 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 813/1629] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:32.100285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc142479-7213-4522-ba0e-59d97034821e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:32.106112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9e0957b26c14313867167480a60d849'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:32.123053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc142479-7213-4522-ba0e-59d97034821e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:32.123617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9e0957b26c14313867167480a60d849'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:32.133812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd91ec-33a7-48e9-819a-acccbe114de5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:32.135197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd91ec-33a7-48e9-819a-acccbe114de5 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.135655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd91ec-33a7-48e9-819a-acccbe114de5 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.136568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd91ec-33a7-48e9-819a-acccbe114de5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.137397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9e0957b26c14313867167480a60d849'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:32.139052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-22fd91ec-33a7-48e9-819a-acccbe114de5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:32.155844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9e0957b26c14313867167480a60d849'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:32.157075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9e0957b26c14313867167480a60d849'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:32.165779 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 817/1630] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:31 2022] DELETE /identity/v3/users/f9e0957b26c14313867167480a60d849 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:32.167440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:32.169543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:32.226658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd91ec-33a7-48e9-819a-acccbe114de5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.231249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd91ec-33a7-48e9-819a-acccbe114de5 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.247039 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 814/1631] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:32 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:32.248394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed3f6-0bb3-4d3a-aa0e-2675a0008d95 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.248717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed3f6-0bb3-4d3a-aa0e-2675a0008d95 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.248985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed3f6-0bb3-4d3a-aa0e-2675a0008d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.301095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-683ed3f6-0bb3-4d3a-aa0e-2675a0008d95 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:32.302394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-683ed3f6-0bb3-4d3a-aa0e-2675a0008d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:32.347579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd945477-2908-4765-b4d3-90735d210907 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:32.355910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e44b65-acd3-4611-bd42-2f87e8147552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:32.356887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44b65-acd3-4611-bd42-2f87e8147552 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.357382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44b65-acd3-4611-bd42-2f87e8147552 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.357835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44b65-acd3-4611-bd42-2f87e8147552 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.358825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:32.359681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e44b65-acd3-4611-bd42-2f87e8147552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:32.360152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e44b65-acd3-4611-bd42-2f87e8147552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.362560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e44b65-acd3-4611-bd42-2f87e8147552 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.386883 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 818/1632] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:32.388197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e44b65-acd3-4611-bd42-2f87e8147552 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:32.400908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e44b65-acd3-4611-bd42-2f87e8147552 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:32.415443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eb5d1-84bb-4023-bfca-ee2259b329c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:32.416061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eb5d1-84bb-4023-bfca-ee2259b329c6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.416407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eb5d1-84bb-4023-bfca-ee2259b329c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.416742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eb5d1-84bb-4023-bfca-ee2259b329c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.418404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338eb5d1-84bb-4023-bfca-ee2259b329c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.420322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338eb5d1-84bb-4023-bfca-ee2259b329c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.421872 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 815/1633] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:32.432031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ed3f6-0bb3-4d3a-aa0e-2675a0008d95 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:32.433963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-683ed3f6-0bb3-4d3a-aa0e-2675a0008d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:32.441912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:32.473093 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 819/1634] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:32 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:32.477778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eb5d1-84bb-4023-bfca-ee2259b329c6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:32.533692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ed3f6-0bb3-4d3a-aa0e-2675a0008d95 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:32.566619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eb5d1-84bb-4023-bfca-ee2259b329c6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:32.574693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69473356-1690-4a43-a6dc-b592bb5943f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:32.575237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69473356-1690-4a43-a6dc-b592bb5943f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.575513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69473356-1690-4a43-a6dc-b592bb5943f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.575781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69473356-1690-4a43-a6dc-b592bb5943f1 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.576411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69473356-1690-4a43-a6dc-b592bb5943f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.579766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69473356-1690-4a43-a6dc-b592bb5943f1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.617086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 820/1635] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:32 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:32.619246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69473356-1690-4a43-a6dc-b592bb5943f1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:32.628655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdf259-b112-4b90-9d9f-3405b673c3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:32.629985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdf259-b112-4b90-9d9f-3405b673c3bc service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.630460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdf259-b112-4b90-9d9f-3405b673c3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.631438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69473356-1690-4a43-a6dc-b592bb5943f1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:32.631927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdf259-b112-4b90-9d9f-3405b673c3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.637617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784133b-b3d2-4c99-83ef-9ea75ac6abcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:32.638334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784133b-b3d2-4c99-83ef-9ea75ac6abcb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.638710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784133b-b3d2-4c99-83ef-9ea75ac6abcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.639081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784133b-b3d2-4c99-83ef-9ea75ac6abcb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.639834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784133b-b3d2-4c99-83ef-9ea75ac6abcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.641677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784133b-b3d2-4c99-83ef-9ea75ac6abcb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.647369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdf259-b112-4b90-9d9f-3405b673c3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.650083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdf259-b112-4b90-9d9f-3405b673c3bc service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.665255 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 821/1636] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:48:32.666384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784133b-b3d2-4c99-83ef-9ea75ac6abcb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:32.669914 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 816/1637] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:32.697782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784133b-b3d2-4c99-83ef-9ea75ac6abcb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:32.706896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850d1b-519a-4b7b-b28b-80519bef4989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:32.707752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850d1b-519a-4b7b-b28b-80519bef4989 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.708069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850d1b-519a-4b7b-b28b-80519bef4989 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.708404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850d1b-519a-4b7b-b28b-80519bef4989 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.709799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e850d1b-519a-4b7b-b28b-80519bef4989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.718532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e850d1b-519a-4b7b-b28b-80519bef4989 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.738622 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 822/1638] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:32.743740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdf259-b112-4b90-9d9f-3405b673c3bc service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:32.765918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdf259-b112-4b90-9d9f-3405b673c3bc service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:32.766848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850d1b-519a-4b7b-b28b-80519bef4989 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:32.768830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e850d1b-519a-4b7b-b28b-80519bef4989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:32.774011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57aeb4-1119-4a74-b426-a7497537559f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:32.775298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57aeb4-1119-4a74-b426-a7497537559f admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.775666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57aeb4-1119-4a74-b426-a7497537559f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.775991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57aeb4-1119-4a74-b426-a7497537559f admin admin] PATH_INFO: `/v3/projects/95c8e192e5e8482f82d9604cb3a12e7e/users/2f0ab1dabe374b2ab1d3e7323bb1e357/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.789913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850d1b-519a-4b7b-b28b-80519bef4989 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:32.795739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:32.796393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.796667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.796937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] PATH_INFO: `/v3/users/3a5dc54f7703403fa7e19332d5ff81e8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.799469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a5dc54f7703403fa7e19332d5ff81e8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.802454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.807727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57aeb4-1119-4a74-b426-a7497537559f admin admin] RBAC: Authorizing `identity:create_grant(project_id=95c8e192e5e8482f82d9604cb3a12e7e, user_id=2f0ab1dabe374b2ab1d3e7323bb1e357, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:32.810387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57aeb4-1119-4a74-b426-a7497537559f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:32.849992 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 817/1639] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:32 2022] PUT /identity/v3/projects/95c8e192e5e8482f82d9604cb3a12e7e/users/2f0ab1dabe374b2ab1d3e7323bb1e357/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:32.856548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca66d9-245f-43eb-8a90-470d6a6313af None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.856917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca66d9-245f-43eb-8a90-470d6a6313af None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.857284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca66d9-245f-43eb-8a90-470d6a6313af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:32.901209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a5dc54f7703403fa7e19332d5ff81e8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:32.919815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a5dc54f7703403fa7e19332d5ff81e8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:32.930862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a5dc54f7703403fa7e19332d5ff81e8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:32.937443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4eca66d9-245f-43eb-8a90-470d6a6313af None None] MFA Rules not processed for user `2f0ab1dabe374b2ab1d3e7323bb1e357`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:32.939273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4eca66d9-245f-43eb-8a90-470d6a6313af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:32.946297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a5dc54f7703403fa7e19332d5ff81e8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:32.947760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f940308-22b4-42c1-a332-ca9e230b5851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a5dc54f7703403fa7e19332d5ff81e8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:32.961096 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 823/1640] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:32 2022] DELETE /identity/v3/users/3a5dc54f7703403fa7e19332d5ff81e8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:32.967328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb6931-f638-4e80-aa88-2b699a8a589c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:32.967899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb6931-f638-4e80-aa88-2b699a8a589c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:32.968786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb6931-f638-4e80-aa88-2b699a8a589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:33.002759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b0eb6931-f638-4e80-aa88-2b699a8a589c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:33.004374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b0eb6931-f638-4e80-aa88-2b699a8a589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:33.031820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:33.056917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:33.068953 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 818/1641] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:48:33.072770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca66d9-245f-43eb-8a90-470d6a6313af None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:33.075062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4eca66d9-245f-43eb-8a90-470d6a6313af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:33.108233 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 824/1642] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:48:33.170042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb6931-f638-4e80-aa88-2b699a8a589c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:33.171531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b0eb6931-f638-4e80-aa88-2b699a8a589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:33.195444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca66d9-245f-43eb-8a90-470d6a6313af None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:33.222464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3724ee7-4eef-4f71-9ef3-99cf9b81ff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:33.223107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3724ee7-4eef-4f71-9ef3-99cf9b81ff62 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:33.223440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3724ee7-4eef-4f71-9ef3-99cf9b81ff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:33.223709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3724ee7-4eef-4f71-9ef3-99cf9b81ff62 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:33.224798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3724ee7-4eef-4f71-9ef3-99cf9b81ff62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:33.227898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3724ee7-4eef-4f71-9ef3-99cf9b81ff62 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:33.267906 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 819/1643] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:33 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:33.270240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb6931-f638-4e80-aa88-2b699a8a589c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:33.272144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3724ee7-4eef-4f71-9ef3-99cf9b81ff62 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:33.372642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa642c13-69bc-4f6d-94ab-19541410ccef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:33.373897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa642c13-69bc-4f6d-94ab-19541410ccef service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:33.374180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa642c13-69bc-4f6d-94ab-19541410ccef service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:33.374451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa642c13-69bc-4f6d-94ab-19541410ccef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:33.405023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3724ee7-4eef-4f71-9ef3-99cf9b81ff62 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:33.411242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b12e27-5af1-4283-8dae-854dcc2758ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:33.413413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b12e27-5af1-4283-8dae-854dcc2758ee admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:33.413698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b12e27-5af1-4283-8dae-854dcc2758ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:33.413969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b12e27-5af1-4283-8dae-854dcc2758ee admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:33.415651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b12e27-5af1-4283-8dae-854dcc2758ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:33.418082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b12e27-5af1-4283-8dae-854dcc2758ee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:33.465101 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 820/1644] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:33 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:33.467065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b12e27-5af1-4283-8dae-854dcc2758ee admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:33.483768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b12e27-5af1-4283-8dae-854dcc2758ee admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:33.496837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9a6c-631a-45c8-90b0-92934ebe27af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:33.497173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9a6c-631a-45c8-90b0-92934ebe27af admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:33.497455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9a6c-631a-45c8-90b0-92934ebe27af admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:33.497722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9a6c-631a-45c8-90b0-92934ebe27af admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:33.498368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f9a6c-631a-45c8-90b0-92934ebe27af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:33.500565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f9a6c-631a-45c8-90b0-92934ebe27af admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:33.523625 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 821/1645] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:33.529999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9a6c-631a-45c8-90b0-92934ebe27af admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:33.540853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1c0f9a6c-631a-45c8-90b0-92934ebe27af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:33.542079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa642c13-69bc-4f6d-94ab-19541410ccef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:33.544489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa642c13-69bc-4f6d-94ab-19541410ccef service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:33.565101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9a6c-631a-45c8-90b0-92934ebe27af admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:33.579737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:33.580386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:33.580663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:33.580929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] PATH_INFO: `/v3/projects/3fcb4e2a06884be29a06f7910ff2635a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:33.582127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fcb4e2a06884be29a06f7910ff2635a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:33.584218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:33.590217 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 825/1646] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:33.593438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa642c13-69bc-4f6d-94ab-19541410ccef service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:33.632600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa642c13-69bc-4f6d-94ab-19541410ccef service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:33.680787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fcb4e2a06884be29a06f7910ff2635a', 'request_id': 'req-6e1ed928-2972-462d-b3a3-84f5bbdb127a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:33.682870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:33.701920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fcb4e2a06884be29a06f7910ff2635a', 'request_id': 'req-6e1ed928-2972-462d-b3a3-84f5bbdb127a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:33.716422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fcb4e2a06884be29a06f7910ff2635a', 'request_id': 'req-6e1ed928-2972-462d-b3a3-84f5bbdb127a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:33.745464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] The token cache is being invalidate because project 3fcb4e2a06884be29a06f7910ff2635a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:33.747185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:33.748679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:33.755232 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 822/1647] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:33 2022] DELETE /identity/v3/projects/3fcb4e2a06884be29a06f7910ff2635a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:33.756777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:33.766241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:33.770133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:33.818666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e1c6c-9781-47aa-b72e-1a5af439dd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:33.820233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e1c6c-9781-47aa-b72e-1a5af439dd61 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:33.820233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e1c6c-9781-47aa-b72e-1a5af439dd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:33.820233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e1c6c-9781-47aa-b72e-1a5af439dd61 admin admin] PATH_INFO: `/v3/projects/dc24ff4f02c848cca3fa4c91366d7909/users/5f8db9ba76f84384b2f20bbffe100b67/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:33.855874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e1c6c-9781-47aa-b72e-1a5af439dd61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc24ff4f02c848cca3fa4c91366d7909, user_id=5f8db9ba76f84384b2f20bbffe100b67, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:33.858591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e1c6c-9781-47aa-b72e-1a5af439dd61 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:33.892715 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 826/1648] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:33 2022] PUT /identity/v3/projects/dc24ff4f02c848cca3fa4c91366d7909/users/5f8db9ba76f84384b2f20bbffe100b67/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:33.897724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e1c6c-9781-47aa-b72e-1a5af439dd61 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:33.902075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-042e1c6c-9781-47aa-b72e-1a5af439dd61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:33.909581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ed928-2972-462d-b3a3-84f5bbdb127a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:33.991561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:33.992355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:33.993496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:33.993877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] PATH_INFO: `/v3/users/f10c851fb47541728f7b044fbcff158a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:33.995890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f10c851fb47541728f7b044fbcff158a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:33.998189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:34.026893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e1c6c-9781-47aa-b72e-1a5af439dd61 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:34.036378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912b1bb-4787-4eee-82c4-2636e11b89be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:34.038519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912b1bb-4787-4eee-82c4-2636e11b89be admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:34.039177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912b1bb-4787-4eee-82c4-2636e11b89be admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:34.039681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912b1bb-4787-4eee-82c4-2636e11b89be admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:34.040853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912b1bb-4787-4eee-82c4-2636e11b89be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:34.042882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912b1bb-4787-4eee-82c4-2636e11b89be admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:34.067274 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 827/1649] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:34.071040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912b1bb-4787-4eee-82c4-2636e11b89be admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:34.090332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f10c851fb47541728f7b044fbcff158a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:34.091165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912b1bb-4787-4eee-82c4-2636e11b89be admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:34.100467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb61001-7f26-470d-a0f3-3e9e03e3e982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:34.100721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f10c851fb47541728f7b044fbcff158a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:34.101584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb61001-7f26-470d-a0f3-3e9e03e3e982 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:34.101953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb61001-7f26-470d-a0f3-3e9e03e3e982 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:34.102318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb61001-7f26-470d-a0f3-3e9e03e3e982 admin admin] PATH_INFO: `/v3/projects/dc24ff4f02c848cca3fa4c91366d7909/users/5f8db9ba76f84384b2f20bbffe100b67/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:34.106688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb61001-7f26-470d-a0f3-3e9e03e3e982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc24ff4f02c848cca3fa4c91366d7909, user_id=5f8db9ba76f84384b2f20bbffe100b67, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:34.112452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb61001-7f26-470d-a0f3-3e9e03e3e982 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:34.123287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f10c851fb47541728f7b044fbcff158a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:34.133895 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 828/1650] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:34 2022] PUT /identity/v3/projects/dc24ff4f02c848cca3fa4c91366d7909/users/5f8db9ba76f84384b2f20bbffe100b67/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:34.137182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f10c851fb47541728f7b044fbcff158a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:34.137823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f10c851fb47541728f7b044fbcff158a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:34.145718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81317912-823e-4041-b9bb-42d344457499 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:34.145976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81317912-823e-4041-b9bb-42d344457499 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:34.146400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81317912-823e-4041-b9bb-42d344457499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:34.152081 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 823/1651] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:33 2022] DELETE /identity/v3/users/f10c851fb47541728f7b044fbcff158a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:34.210459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-81317912-823e-4041-b9bb-42d344457499 None None] MFA Rules not processed for user `5f8db9ba76f84384b2f20bbffe100b67`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:34.211668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-81317912-823e-4041-b9bb-42d344457499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:34.333424 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 829/1652] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:34.343350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:34.352186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:34.357243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efeac7-560e-4721-a214-a32a29a9f509 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:34.364635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42ea71-83f9-4440-aedb-36c49cb7afe7 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:34.366126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ea71-83f9-4440-aedb-36c49cb7afe7 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:34.367289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ea71-83f9-4440-aedb-36c49cb7afe7 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:34.367652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ea71-83f9-4440-aedb-36c49cb7afe7 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:34.368721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42ea71-83f9-4440-aedb-36c49cb7afe7 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:34.368721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42ea71-83f9-4440-aedb-36c49cb7afe7 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:34.371139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42ea71-83f9-4440-aedb-36c49cb7afe7 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:34.385091 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 824/1653] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:34.388574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81317912-823e-4041-b9bb-42d344457499 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:34.400459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81317912-823e-4041-b9bb-42d344457499 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:34.405828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6288af2-8d5d-4d96-90b5-79afdcab67bd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:34.406452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6288af2-8d5d-4d96-90b5-79afdcab67bd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:34.406745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6288af2-8d5d-4d96-90b5-79afdcab67bd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:34.407014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6288af2-8d5d-4d96-90b5-79afdcab67bd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:34.407654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6288af2-8d5d-4d96-90b5-79afdcab67bd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:34.409836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6288af2-8d5d-4d96-90b5-79afdcab67bd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:34.443360 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 830/1654] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:34 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:34.447797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42ea71-83f9-4440-aedb-36c49cb7afe7 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:34.460704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42ea71-83f9-4440-aedb-36c49cb7afe7 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:34.532087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1405b50-ff43-4039-bfd4-70aecf6fc49e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:34.533606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1405b50-ff43-4039-bfd4-70aecf6fc49e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:34.533893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1405b50-ff43-4039-bfd4-70aecf6fc49e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:34.534175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1405b50-ff43-4039-bfd4-70aecf6fc49e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:34.535024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1405b50-ff43-4039-bfd4-70aecf6fc49e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:34.535338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1405b50-ff43-4039-bfd4-70aecf6fc49e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:34.537762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1405b50-ff43-4039-bfd4-70aecf6fc49e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:34.555957 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 825/1655] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:34.567751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6288af2-8d5d-4d96-90b5-79afdcab67bd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:34.570343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c6288af2-8d5d-4d96-90b5-79afdcab67bd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:34.591203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:34.684638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6288af2-8d5d-4d96-90b5-79afdcab67bd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:34.688793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1405b50-ff43-4039-bfd4-70aecf6fc49e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:34.691193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b1405b50-ff43-4039-bfd4-70aecf6fc49e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:34.718560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33363-a883-40dc-98c9-c7c20c7e3e71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:34.720749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33363-a883-40dc-98c9-c7c20c7e3e71 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:34.721010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33363-a883-40dc-98c9-c7c20c7e3e71 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:34.721302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33363-a883-40dc-98c9-c7c20c7e3e71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:34.762538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33363-a883-40dc-98c9-c7c20c7e3e71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:34.764191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33363-a883-40dc-98c9-c7c20c7e3e71 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:34.783792 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 831/1656] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:34.818085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5fba5-47a5-4db6-9627-c14f52cf9bd1 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:34.818491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5fba5-47a5-4db6-9627-c14f52cf9bd1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:34.818857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5fba5-47a5-4db6-9627-c14f52cf9bd1 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:34.821110 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 832/1657] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:34.828639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5fba5-47a5-4db6-9627-c14f52cf9bd1 None None] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:48:34.854198 ncdv-master-1 devstack@keystone.service[99100]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-22a5fba5-47a5-4db6-9627-c14f52cf9bd1 None None] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Oct 15 20:48:34.854502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5fba5-47a5-4db6-9627-c14f52cf9bd1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:34.877765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5fba5-47a5-4db6-9627-c14f52cf9bd1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:34.882714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c54bdf-2c85-4322-ba1d-ed9134e3b123 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:34.883046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1405b50-ff43-4039-bfd4-70aecf6fc49e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:34.883347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c54bdf-2c85-4322-ba1d-ed9134e3b123 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:34.883742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c54bdf-2c85-4322-ba1d-ed9134e3b123 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:34.884188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c54bdf-2c85-4322-ba1d-ed9134e3b123 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/projects/3ec5392df39b4786b3e8819ec8cf6104` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:34.885413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c54bdf-2c85-4322-ba1d-ed9134e3b123 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:get_project(project_id=3ec5392df39b4786b3e8819ec8cf6104)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:34.888024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:34.888168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c54bdf-2c85-4322-ba1d-ed9134e3b123 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:34.888647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:34.888917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:34.889240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] PATH_INFO: `/v3/projects/180fa0d40e2546448720f9c60a0d581f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:34.890233 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 833/1658] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:48:34 2022] GET /identity/v3/projects/3ec5392df39b4786b3e8819ec8cf6104 => generated 371 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:48:34.898493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=180fa0d40e2546448720f9c60a0d581f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:34.900400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:35.026018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '180fa0d40e2546448720f9c60a0d581f', 'request_id': 'req-2d560096-4985-424c-b7b6-cc241524d7ac'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.043257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '180fa0d40e2546448720f9c60a0d581f', 'request_id': 'req-2d560096-4985-424c-b7b6-cc241524d7ac'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.054287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '180fa0d40e2546448720f9c60a0d581f', 'request_id': 'req-2d560096-4985-424c-b7b6-cc241524d7ac'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.070242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] The token cache is being invalidate because project 180fa0d40e2546448720f9c60a0d581f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:35.070674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.072208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d560096-4985-424c-b7b6-cc241524d7ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.074153 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 826/1659] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:34 2022] DELETE /identity/v3/projects/180fa0d40e2546448720f9c60a0d581f => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:35.077831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c54bdf-2c85-4322-ba1d-ed9134e3b123 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:35.080085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-04c54bdf-2c85-4322-ba1d-ed9134e3b123 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:35.101784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c54bdf-2c85-4322-ba1d-ed9134e3b123 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:35.213195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:35.213938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:35.214217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:35.214485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] PATH_INFO: `/v3/users/2c6909b25e674917abe5692130bb05fc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:35.216247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6909b25e674917abe5692130bb05fc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:35.218665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:35.301200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6909b25e674917abe5692130bb05fc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.317366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6909b25e674917abe5692130bb05fc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.327154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6909b25e674917abe5692130bb05fc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.336352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6909b25e674917abe5692130bb05fc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.337252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6909b25e674917abe5692130bb05fc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.344162 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 834/1660] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:35 2022] DELETE /identity/v3/users/2c6909b25e674917abe5692130bb05fc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:35.387465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3a55-5e1b-47d1-bbd8-e16541f51fe9 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:35.387976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3a55-5e1b-47d1-bbd8-e16541f51fe9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:35.387976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3a55-5e1b-47d1-bbd8-e16541f51fe9 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:35.389605 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 827/1661] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:35.394758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:48:35.397500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:35.418186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:35.527079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:35.528640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:35.529573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb3a55-5e1b-47d1-bbd8-e16541f51fe9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:35.533222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6aeb3a55-5e1b-47d1-bbd8-e16541f51fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:35.615787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:35.634926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dd916-69a5-4ad4-a1cf-b92aafe6dd85 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:35.641627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33746b-ceb0-4469-86ec-683fbb6c1819 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:35.642366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33746b-ceb0-4469-86ec-683fbb6c1819 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:35.642659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33746b-ceb0-4469-86ec-683fbb6c1819 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:35.642917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33746b-ceb0-4469-86ec-683fbb6c1819 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/projects/3ec5392df39b4786b3e8819ec8cf6104` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:35.644185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33746b-ceb0-4469-86ec-683fbb6c1819 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:get_project(project_id=3ec5392df39b4786b3e8819ec8cf6104)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:35.647742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33746b-ceb0-4469-86ec-683fbb6c1819 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:35.651625 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 835/1662] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:48:35 2022] GET /identity/v3/projects/3ec5392df39b4786b3e8819ec8cf6104 => generated 371 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:35.709780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb3a55-5e1b-47d1-bbd8-e16541f51fe9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:35.726132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:35.726903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:35.727184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:35.727685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] PATH_INFO: `/v3/projects/341f31852cf343caa3a4e6abf2b6a8c2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:35.727892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33746b-ceb0-4469-86ec-683fbb6c1819 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:35.741932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=341f31852cf343caa3a4e6abf2b6a8c2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:35.742990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33746b-ceb0-4469-86ec-683fbb6c1819 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:35.743861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:35.749518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8361079-a49c-414a-9384-1695a536661d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:35.750134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8361079-a49c-414a-9384-1695a536661d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:35.750419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8361079-a49c-414a-9384-1695a536661d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:35.751026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8361079-a49c-414a-9384-1695a536661d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:35.751492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8361079-a49c-414a-9384-1695a536661d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:35.751905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8361079-a49c-414a-9384-1695a536661d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:35.753736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8361079-a49c-414a-9384-1695a536661d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:35.790673 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 836/1663] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:35.795604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8361079-a49c-414a-9384-1695a536661d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:35.840654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '341f31852cf343caa3a4e6abf2b6a8c2', 'request_id': 'req-90b75cc1-d659-448b-a151-7c93f65fff93'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.842478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8361079-a49c-414a-9384-1695a536661d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:35.855099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '341f31852cf343caa3a4e6abf2b6a8c2', 'request_id': 'req-90b75cc1-d659-448b-a151-7c93f65fff93'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.870469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '341f31852cf343caa3a4e6abf2b6a8c2', 'request_id': 'req-90b75cc1-d659-448b-a151-7c93f65fff93'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.895197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] The token cache is being invalidate because project 341f31852cf343caa3a4e6abf2b6a8c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:35.897178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.899123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90b75cc1-d659-448b-a151-7c93f65fff93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:35.907512 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 828/1664] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:35 2022] DELETE /identity/v3/projects/341f31852cf343caa3a4e6abf2b6a8c2 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:35.910789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82334181-2404-45a2-a5bf-124e0397efc8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:35.911316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82334181-2404-45a2-a5bf-124e0397efc8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:35.912361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82334181-2404-45a2-a5bf-124e0397efc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:35.970658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-82334181-2404-45a2-a5bf-124e0397efc8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:35.975274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-82334181-2404-45a2-a5bf-124e0397efc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:35.984539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91edfe-2e32-4507-af5d-3e1e75b24ba4 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:35.985255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91edfe-2e32-4507-af5d-3e1e75b24ba4 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:35.985545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91edfe-2e32-4507-af5d-3e1e75b24ba4 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:35.985845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91edfe-2e32-4507-af5d-3e1e75b24ba4 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:35.991922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91edfe-2e32-4507-af5d-3e1e75b24ba4 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:35.994472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91edfe-2e32-4507-af5d-3e1e75b24ba4 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.055404 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 837/1665] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:35 2022] POST /identity/v3/users => generated 383 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:36.057569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91edfe-2e32-4507-af5d-3e1e75b24ba4 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.060700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fb91edfe-2e32-4507-af5d-3e1e75b24ba4 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:36.089199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91edfe-2e32-4507-af5d-3e1e75b24ba4 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.105218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887ee7c-bec5-4898-9372-0cc2b5b911fd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.105870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887ee7c-bec5-4898-9372-0cc2b5b911fd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.106146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887ee7c-bec5-4898-9372-0cc2b5b911fd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.106420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887ee7c-bec5-4898-9372-0cc2b5b911fd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.107079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887ee7c-bec5-4898-9372-0cc2b5b911fd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:36.107388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887ee7c-bec5-4898-9372-0cc2b5b911fd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.112509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887ee7c-bec5-4898-9372-0cc2b5b911fd tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.134929 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 829/1666] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:36.136256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82334181-2404-45a2-a5bf-124e0397efc8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.141390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-82334181-2404-45a2-a5bf-124e0397efc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:36.162602 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 838/1667] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:36.162602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f0908-1fc1-4e61-88e6-baa62cc04ef3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.162602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f0908-1fc1-4e61-88e6-baa62cc04ef3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.162602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f0908-1fc1-4e61-88e6-baa62cc04ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.163035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82334181-2404-45a2-a5bf-124e0397efc8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.168237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.169244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.178892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.178892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] PATH_INFO: `/v3/projects/eb4fbfc6501a4ff4bb849f86cd3fbc04` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.181507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb4fbfc6501a4ff4bb849f86cd3fbc04)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.184416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.218701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:36.227162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e77f0908-1fc1-4e61-88e6-baa62cc04ef3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:36.228785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e77f0908-1fc1-4e61-88e6-baa62cc04ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:36.246693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:36.267019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb4fbfc6501a4ff4bb849f86cd3fbc04', 'request_id': 'req-e6a27a81-80ca-443f-8cee-035e44b4b525'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:36.284541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb4fbfc6501a4ff4bb849f86cd3fbc04', 'request_id': 'req-e6a27a81-80ca-443f-8cee-035e44b4b525'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:36.300679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb4fbfc6501a4ff4bb849f86cd3fbc04', 'request_id': 'req-e6a27a81-80ca-443f-8cee-035e44b4b525'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:36.330809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] The token cache is being invalidate because project eb4fbfc6501a4ff4bb849f86cd3fbc04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:36.331156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:36.331851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6a27a81-80ca-443f-8cee-035e44b4b525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:36.337578 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 830/1668] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:36 2022] DELETE /identity/v3/projects/eb4fbfc6501a4ff4bb849f86cd3fbc04 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:36.340694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ae950-5902-47a2-837b-7ac764ac85f4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.340983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ae950-5902-47a2-837b-7ac764ac85f4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.344118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ae950-5902-47a2-837b-7ac764ac85f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.377398 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 839/1669] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:36.382221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f0908-1fc1-4e61-88e6-baa62cc04ef3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.388374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b42ae950-5902-47a2-837b-7ac764ac85f4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:36.389698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b42ae950-5902-47a2-837b-7ac764ac85f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:36.397385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f0908-1fc1-4e61-88e6-baa62cc04ef3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.411208 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 831/1670] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:36.412105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63203e-b331-458c-99e9-9cf0e91a140a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.412105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63203e-b331-458c-99e9-9cf0e91a140a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.412105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63203e-b331-458c-99e9-9cf0e91a140a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.412105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63203e-b331-458c-99e9-9cf0e91a140a admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.412416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63203e-b331-458c-99e9-9cf0e91a140a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:36.412416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63203e-b331-458c-99e9-9cf0e91a140a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.413365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1794f4-ea05-4b49-a23a-2353e33d0563 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.413675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1794f4-ea05-4b49-a23a-2353e33d0563 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.413913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1794f4-ea05-4b49-a23a-2353e33d0563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.417851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63203e-b331-458c-99e9-9cf0e91a140a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.436681 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 840/1671] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:48:36.441320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63203e-b331-458c-99e9-9cf0e91a140a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.457940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cd1794f4-ea05-4b49-a23a-2353e33d0563 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:36.459424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cd1794f4-ea05-4b49-a23a-2353e33d0563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:36.469658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63203e-b331-458c-99e9-9cf0e91a140a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.477541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8f5eb-9dcb-4c65-a09c-c4328b410fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.478607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f5eb-9dcb-4c65-a09c-c4328b410fcd admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.479116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f5eb-9dcb-4c65-a09c-c4328b410fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.479640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f5eb-9dcb-4c65-a09c-c4328b410fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.480478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8f5eb-9dcb-4c65-a09c-c4328b410fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.483634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8f5eb-9dcb-4c65-a09c-c4328b410fcd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.485974 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 832/1672] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:36.490685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1794f4-ea05-4b49-a23a-2353e33d0563 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.500059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1794f4-ea05-4b49-a23a-2353e33d0563 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.504962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17df5b2-3574-4acd-ae80-7a055e575833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.505699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17df5b2-3574-4acd-ae80-7a055e575833 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.505971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17df5b2-3574-4acd-ae80-7a055e575833 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.506239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17df5b2-3574-4acd-ae80-7a055e575833 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.506881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17df5b2-3574-4acd-ae80-7a055e575833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:36.507222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17df5b2-3574-4acd-ae80-7a055e575833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.508942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17df5b2-3574-4acd-ae80-7a055e575833 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.521908 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 841/1673] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:36 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:36.522621 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 833/1674] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:36.523543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fd6f2-fac8-4507-8741-866b21a1fb81 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.523891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fd6f2-fac8-4507-8741-866b21a1fb81 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.524228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fd6f2-fac8-4507-8741-866b21a1fb81 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.525714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17df5b2-3574-4acd-ae80-7a055e575833 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.526022 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 842/1675] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:36.529523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fd6f2-fac8-4507-8741-866b21a1fb81 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.538252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17df5b2-3574-4acd-ae80-7a055e575833 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.612083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fd6f2-fac8-4507-8741-866b21a1fb81 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.615847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.616889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.617248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.617553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.618357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.624642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd5757-72f1-4938-ada0-a61ed22e28da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.626914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.627015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5757-72f1-4938-ada0-a61ed22e28da admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.627015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5757-72f1-4938-ada0-a61ed22e28da admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.627015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5757-72f1-4938-ada0-a61ed22e28da admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.627015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd5757-72f1-4938-ada0-a61ed22e28da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.627015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd5757-72f1-4938-ada0-a61ed22e28da admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.649834 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 834/1676] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:36 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:36.651743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:48:36.663151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:36.683899 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 843/1677] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:36 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:36.685113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd5757-72f1-4938-ada0-a61ed22e28da admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.790426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.792313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:36.799748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:36.805139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd5757-72f1-4938-ada0-a61ed22e28da admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.813965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c483264-68fd-4398-8a68-58e829fd6946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.814705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c483264-68fd-4398-8a68-58e829fd6946 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.815043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c483264-68fd-4398-8a68-58e829fd6946 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.815323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c483264-68fd-4398-8a68-58e829fd6946 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.816290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c483264-68fd-4398-8a68-58e829fd6946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.818351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c483264-68fd-4398-8a68-58e829fd6946 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.832065 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 844/1678] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:36.837439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c483264-68fd-4398-8a68-58e829fd6946 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.840417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:36.854355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c483264-68fd-4398-8a68-58e829fd6946 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.863176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5168f0-9096-461b-9a2a-f6a7bd932ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.863810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5168f0-9096-461b-9a2a-f6a7bd932ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.864089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5168f0-9096-461b-9a2a-f6a7bd932ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.864357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5168f0-9096-461b-9a2a-f6a7bd932ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.865515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5168f0-9096-461b-9a2a-f6a7bd932ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.868784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5168f0-9096-461b-9a2a-f6a7bd932ee7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.897454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d14f2-84ee-4a77-92fb-1b11c2ff061c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.905105 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 845/1679] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:36 2022] POST /identity/v3/users => generated 287 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:36.906549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5168f0-9096-461b-9a2a-f6a7bd932ee7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.924947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5168f0-9096-461b-9a2a-f6a7bd932ee7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.931467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a123d8-b403-4d4f-a197-3b6e0b71a940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.932523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a123d8-b403-4d4f-a197-3b6e0b71a940 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.932804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a123d8-b403-4d4f-a197-3b6e0b71a940 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.933299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a123d8-b403-4d4f-a197-3b6e0b71a940 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.933956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a123d8-b403-4d4f-a197-3b6e0b71a940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.936136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a123d8-b403-4d4f-a197-3b6e0b71a940 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.957270 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 846/1680] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:36.957270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a123d8-b403-4d4f-a197-3b6e0b71a940 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:36.968541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a123d8-b403-4d4f-a197-3b6e0b71a940 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:36.975995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3f75f-476b-411d-8b63-1f02e82b9a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.976208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3f75f-476b-411d-8b63-1f02e82b9a6a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.976823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3f75f-476b-411d-8b63-1f02e82b9a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.977126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3f75f-476b-411d-8b63-1f02e82b9a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.978215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3f75f-476b-411d-8b63-1f02e82b9a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.979488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3f75f-476b-411d-8b63-1f02e82b9a6a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.980964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b3b4c-e37b-427a-85f2-a9a5a419aa0d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:36.981821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b3b4c-e37b-427a-85f2-a9a5a419aa0d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:36.983037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b3b4c-e37b-427a-85f2-a9a5a419aa0d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:36.983037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b3b4c-e37b-427a-85f2-a9a5a419aa0d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/projects/3ec5392df39b4786b3e8819ec8cf6104` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:36.984392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b3b4c-e37b-427a-85f2-a9a5a419aa0d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:get_project(project_id=3ec5392df39b4786b3e8819ec8cf6104)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:36.986769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b3b4c-e37b-427a-85f2-a9a5a419aa0d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:36.993360 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 847/1681] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:36.994645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3f75f-476b-411d-8b63-1f02e82b9a6a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.006835 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 835/1682] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:48:36 2022] GET /identity/v3/projects/3ec5392df39b4786b3e8819ec8cf6104 => generated 371 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:37.006835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b3b4c-e37b-427a-85f2-a9a5a419aa0d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.020661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3f75f-476b-411d-8b63-1f02e82b9a6a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.029262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fc209-760a-468f-a090-29a279f5eb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.030248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fc209-760a-468f-a090-29a279f5eb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.030626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fc209-760a-468f-a090-29a279f5eb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.031089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fc209-760a-468f-a090-29a279f5eb94 admin admin] PATH_INFO: `/v3/projects/5b8143192a324b3782a0650d3efb060c/users/2d907d81af2a4c5486fc464320bb84e4/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.034054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b3b4c-e37b-427a-85f2-a9a5a419aa0d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.040700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c912b9-e4b1-43e0-95b8-6a6377fb0873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.041649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c912b9-e4b1-43e0-95b8-6a6377fb0873 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.042472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c912b9-e4b1-43e0-95b8-6a6377fb0873 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.042931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c912b9-e4b1-43e0-95b8-6a6377fb0873 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.043806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c912b9-e4b1-43e0-95b8-6a6377fb0873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.046686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c912b9-e4b1-43e0-95b8-6a6377fb0873 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:37.057000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fc209-760a-468f-a090-29a279f5eb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8143192a324b3782a0650d3efb060c, user_id=2d907d81af2a4c5486fc464320bb84e4, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.065515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fc209-760a-468f-a090-29a279f5eb94 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:37.066011 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 836/1683] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:37.072035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c912b9-e4b1-43e0-95b8-6a6377fb0873 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.091482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c912b9-e4b1-43e0-95b8-6a6377fb0873 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.112925 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 848/1684] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:36 2022] PUT /identity/v3/projects/5b8143192a324b3782a0650d3efb060c/users/2d907d81af2a4c5486fc464320bb84e4/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:37.116338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d7b5e-3e0c-403f-a435-e6f38f4f9a3c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.117292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d7b5e-3e0c-403f-a435-e6f38f4f9a3c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.117573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d7b5e-3e0c-403f-a435-e6f38f4f9a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.180181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-226d7b5e-3e0c-403f-a435-e6f38f4f9a3c None None] MFA Rules not processed for user `2d907d81af2a4c5486fc464320bb84e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:37.181655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-226d7b5e-3e0c-403f-a435-e6f38f4f9a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:37.227739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7764020-4aba-4a44-964e-f47f8473e2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.228684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7764020-4aba-4a44-964e-f47f8473e2cc admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.230758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7764020-4aba-4a44-964e-f47f8473e2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.230758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7764020-4aba-4a44-964e-f47f8473e2cc admin admin] PATH_INFO: `/v3/projects/12f5d58f9d7f4ab69cb207e25b87a9a8/users/a5beae9aac914403b1ff1b226f59813b/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.274832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7764020-4aba-4a44-964e-f47f8473e2cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f5d58f9d7f4ab69cb207e25b87a9a8, user_id=a5beae9aac914403b1ff1b226f59813b, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.277552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7764020-4aba-4a44-964e-f47f8473e2cc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:37.322912 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 837/1685] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:37 2022] PUT /identity/v3/projects/12f5d58f9d7f4ab69cb207e25b87a9a8/users/a5beae9aac914403b1ff1b226f59813b/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:37.326884 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 849/1686] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:37.332163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7764020-4aba-4a44-964e-f47f8473e2cc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.336880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4942ff-611c-43ce-a9a6-81ae8bb61a96 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.337410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4942ff-611c-43ce-a9a6-81ae8bb61a96 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.339136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4942ff-611c-43ce-a9a6-81ae8bb61a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.382237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5d4942ff-611c-43ce-a9a6-81ae8bb61a96 None None] MFA Rules not processed for user `a5beae9aac914403b1ff1b226f59813b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:37.383557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5d4942ff-611c-43ce-a9a6-81ae8bb61a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:37.494509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:37.534507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7764020-4aba-4a44-964e-f47f8473e2cc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.541503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ca6ca-9cc1-4620-a8df-d30f4a3c73e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.542312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ca6ca-9cc1-4620-a8df-d30f4a3c73e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.542586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ca6ca-9cc1-4620-a8df-d30f4a3c73e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.542851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ca6ca-9cc1-4620-a8df-d30f4a3c73e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.543556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ca6ca-9cc1-4620-a8df-d30f4a3c73e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.545618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ca6ca-9cc1-4620-a8df-d30f4a3c73e0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:37.556449 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 850/1687] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:48:37.558015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1418a-bdc6-4a13-8f3d-a386a05ea65f None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.558442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1418a-bdc6-4a13-8f3d-a386a05ea65f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.558845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1418a-bdc6-4a13-8f3d-a386a05ea65f None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.564896 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 851/1688] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:48:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:48:37.565838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1418a-bdc6-4a13-8f3d-a386a05ea65f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.588694 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 838/1689] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:37 2022] POST /identity/v3/projects => generated 375 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:37.590883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ca6ca-9cc1-4620-a8df-d30f4a3c73e0 admin admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:48:37.592810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa9ca6ca-9cc1-4620-a8df-d30f4a3c73e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:37.609218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1418a-bdc6-4a13-8f3d-a386a05ea65f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.615290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:37.686484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6446f-cb97-47ac-a9a3-7d1710da72ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.687938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6446f-cb97-47ac-a9a3-7d1710da72ec admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.689145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6446f-cb97-47ac-a9a3-7d1710da72ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.689425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6446f-cb97-47ac-a9a3-7d1710da72ec admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.690067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6446f-cb97-47ac-a9a3-7d1710da72ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.692407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6446f-cb97-47ac-a9a3-7d1710da72ec admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:37.700685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ca6ca-9cc1-4620-a8df-d30f4a3c73e0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.733546 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 852/1690] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:37 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:37.734643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6446f-cb97-47ac-a9a3-7d1710da72ec admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.801483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ca6ca-9cc1-4620-a8df-d30f4a3c73e0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.825779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6446f-cb97-47ac-a9a3-7d1710da72ec admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.831608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bd426-1b36-4316-a65a-f08b65e4557d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.832479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bd426-1b36-4316-a65a-f08b65e4557d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.832815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bd426-1b36-4316-a65a-f08b65e4557d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.833315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bd426-1b36-4316-a65a-f08b65e4557d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.834627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bd426-1b36-4316-a65a-f08b65e4557d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.837063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bd426-1b36-4316-a65a-f08b65e4557d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:37.868235 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 853/1691] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:37 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:37.871095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bd426-1b36-4316-a65a-f08b65e4557d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.878575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917405b4-acd5-40ed-babf-49a5bf279e8e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.879263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917405b4-acd5-40ed-babf-49a5bf279e8e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.879547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917405b4-acd5-40ed-babf-49a5bf279e8e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.879960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917405b4-acd5-40ed-babf-49a5bf279e8e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/projects/3ec5392df39b4786b3e8819ec8cf6104` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.881932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917405b4-acd5-40ed-babf-49a5bf279e8e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:get_project(project_id=3ec5392df39b4786b3e8819ec8cf6104)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.884028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917405b4-acd5-40ed-babf-49a5bf279e8e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:37.886548 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 839/1692] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:48:37 2022] GET /identity/v3/projects/3ec5392df39b4786b3e8819ec8cf6104 => generated 371 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:37.887411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917405b4-acd5-40ed-babf-49a5bf279e8e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.888874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bd426-1b36-4316-a65a-f08b65e4557d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.899187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d1a7a-f4e9-46b5-b372-255bd3792b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.899884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d1a7a-f4e9-46b5-b372-255bd3792b14 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.900332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d1a7a-f4e9-46b5-b372-255bd3792b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.900822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d1a7a-f4e9-46b5-b372-255bd3792b14 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.901479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d1a7a-f4e9-46b5-b372-255bd3792b14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.904791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d1a7a-f4e9-46b5-b372-255bd3792b14 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:37.911372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917405b4-acd5-40ed-babf-49a5bf279e8e tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.919396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c75a8-8a4f-476d-9ef0-e762807ff3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.920006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c75a8-8a4f-476d-9ef0-e762807ff3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.920300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c75a8-8a4f-476d-9ef0-e762807ff3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.920556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c75a8-8a4f-476d-9ef0-e762807ff3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.921688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c75a8-8a4f-476d-9ef0-e762807ff3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.924119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c75a8-8a4f-476d-9ef0-e762807ff3b0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:37.938235 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 840/1693] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:37.939254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c75a8-8a4f-476d-9ef0-e762807ff3b0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.954893 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 854/1694] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:37 2022] POST /identity/v3/users => generated 292 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:37.955144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c75a8-8a4f-476d-9ef0-e762807ff3b0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.956058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d1a7a-f4e9-46b5-b372-255bd3792b14 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:37.967968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d1a7a-f4e9-46b5-b372-255bd3792b14 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:37.974651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.975502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.975830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.976165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/users/3505aa8d89534b48a9f50907e1a6bbe6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.976983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847254e-13b6-423e-a2ee-09fc1b061aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:37.977762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847254e-13b6-423e-a2ee-09fc1b061aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:37.978051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847254e-13b6-423e-a2ee-09fc1b061aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:37.978301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847254e-13b6-423e-a2ee-09fc1b061aa1 admin admin] PATH_INFO: `/v3/projects/0325b17a072841fbb0c06c45040e912e/users/2fe64fcd24244811b4e6506f1b212bff/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:37.978955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:37.995058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:delete_user(user_id=3505aa8d89534b48a9f50907e1a6bbe6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.996262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847254e-13b6-423e-a2ee-09fc1b061aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0325b17a072841fbb0c06c45040e912e, user_id=2fe64fcd24244811b4e6506f1b212bff, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:37.997438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:37.998520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847254e-13b6-423e-a2ee-09fc1b061aa1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:38.015315 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 855/1695] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:37 2022] PUT /identity/v3/projects/0325b17a072841fbb0c06c45040e912e/users/2fe64fcd24244811b4e6506f1b212bff/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:38.019157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847254e-13b6-423e-a2ee-09fc1b061aa1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:38.073819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3505aa8d89534b48a9f50907e1a6bbe6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.079719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:38.084476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3505aa8d89534b48a9f50907e1a6bbe6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.098975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3505aa8d89534b48a9f50907e1a6bbe6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.109065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3505aa8d89534b48a9f50907e1a6bbe6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.109364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847254e-13b6-423e-a2ee-09fc1b061aa1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:38.110067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3505aa8d89534b48a9f50907e1a6bbe6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.115585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe91577-f40f-4ff1-95a9-ecaf47848777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:38.116153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe91577-f40f-4ff1-95a9-ecaf47848777 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:38.116523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe91577-f40f-4ff1-95a9-ecaf47848777 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:38.116853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe91577-f40f-4ff1-95a9-ecaf47848777 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:38.117616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe91577-f40f-4ff1-95a9-ecaf47848777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:38.121095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 841/1696] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:37 2022] DELETE /identity/v3/users/3505aa8d89534b48a9f50907e1a6bbe6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:38.121095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:38.122312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:38.122892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe91577-f40f-4ff1-95a9-ecaf47848777 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:38.134471 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 856/1697] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:38.135285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe91577-f40f-4ff1-95a9-ecaf47848777 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:38.137005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dbe91577-f40f-4ff1-95a9-ecaf47848777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:38.223289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe91577-f40f-4ff1-95a9-ecaf47848777 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:38.244729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120b6d5-aa67-4b53-9e6c-4a50020e3cd9 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:38.251078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af70d6f-feb6-4a1f-bbdb-ad2910f1fc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:38.251981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af70d6f-feb6-4a1f-bbdb-ad2910f1fc9a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:38.252788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af70d6f-feb6-4a1f-bbdb-ad2910f1fc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:38.253276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af70d6f-feb6-4a1f-bbdb-ad2910f1fc9a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:38.254159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af70d6f-feb6-4a1f-bbdb-ad2910f1fc9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:38.256550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af70d6f-feb6-4a1f-bbdb-ad2910f1fc9a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:38.269016 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 842/1698] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:38.270249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af70d6f-feb6-4a1f-bbdb-ad2910f1fc9a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:38.272759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9af70d6f-feb6-4a1f-bbdb-ad2910f1fc9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:38.291211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af70d6f-feb6-4a1f-bbdb-ad2910f1fc9a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:38.298112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fe28d-cf81-46e3-9407-cd08b3f49c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:38.298850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fe28d-cf81-46e3-9407-cd08b3f49c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:38.299277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fe28d-cf81-46e3-9407-cd08b3f49c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:38.299557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fe28d-cf81-46e3-9407-cd08b3f49c81 admin admin] PATH_INFO: `/v3/projects/31caa50628804cd597330477886dc146/users/7ea831d2d7ab4ccf898c5236b932db85/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:38.324259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fe28d-cf81-46e3-9407-cd08b3f49c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31caa50628804cd597330477886dc146, user_id=7ea831d2d7ab4ccf898c5236b932db85, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:38.328281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fe28d-cf81-46e3-9407-cd08b3f49c81 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:38.328651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f8db9ba76f84384b2f20bbffe100b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc24ff4f02c848cca3fa4c91366d7909', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:38.329688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:38.330136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:38.330551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] PATH_INFO: `/v3/projects/3ec5392df39b4786b3e8819ec8cf6104` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:38.332278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3ec5392df39b4786b3e8819ec8cf6104)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:38.339674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:38.354400 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 843/1699] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:38 2022] PUT /identity/v3/projects/31caa50628804cd597330477886dc146/users/7ea831d2d7ab4ccf898c5236b932db85/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:38.355232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fe28d-cf81-46e3-9407-cd08b3f49c81 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:38.396708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec5392df39b4786b3e8819ec8cf6104', 'request_id': 'req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.404526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:38.406039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec5392df39b4786b3e8819ec8cf6104', 'request_id': 'req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.415778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec5392df39b4786b3e8819ec8cf6104', 'request_id': 'req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.430553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] The token cache is being invalidate because project 3ec5392df39b4786b3e8819ec8cf6104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:38.430885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.431509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.433020 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 857/1700] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:38 2022] DELETE /identity/v3/projects/3ec5392df39b4786b3e8819ec8cf6104 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:38.433873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:38.435154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:38.443394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fe28d-cf81-46e3-9407-cd08b3f49c81 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:38.471251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:38.519339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fb772-8a4a-461b-a9c6-01b8b921a45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:38.520203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fb772-8a4a-461b-a9c6-01b8b921a45f admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:38.520544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fb772-8a4a-461b-a9c6-01b8b921a45f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:38.520854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fb772-8a4a-461b-a9c6-01b8b921a45f admin admin] PATH_INFO: `/v3/projects/0325b17a072841fbb0c06c45040e912e/users/2fe64fcd24244811b4e6506f1b212bff/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:38.523397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fb772-8a4a-461b-a9c6-01b8b921a45f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0325b17a072841fbb0c06c45040e912e, user_id=2fe64fcd24244811b4e6506f1b212bff, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:38.527318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fb772-8a4a-461b-a9c6-01b8b921a45f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:38.541253 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 844/1701] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:38 2022] PUT /identity/v3/projects/0325b17a072841fbb0c06c45040e912e/users/2fe64fcd24244811b4e6506f1b212bff/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:38.542253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fb772-8a4a-461b-a9c6-01b8b921a45f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:38.551189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-310fb772-8a4a-461b-a9c6-01b8b921a45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:38.592986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2ca5a-9982-4230-b1aa-88cd03a1dd79 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:38.700994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fb772-8a4a-461b-a9c6-01b8b921a45f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:38.712490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:38.713646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:38.714070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:38.715691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] PATH_INFO: `/v3/users/2f0ab1dabe374b2ab1d3e7323bb1e357` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:38.717391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f0ab1dabe374b2ab1d3e7323bb1e357)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:38.721462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:38.749407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c5d42-e4ed-47d3-8ad2-5d2fdc8fddb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:38.751430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5d42-e4ed-47d3-8ad2-5d2fdc8fddb9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:38.767992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5d42-e4ed-47d3-8ad2-5d2fdc8fddb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:38.768817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5d42-e4ed-47d3-8ad2-5d2fdc8fddb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:38.775239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5d42-e4ed-47d3-8ad2-5d2fdc8fddb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:38.777423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5d42-e4ed-47d3-8ad2-5d2fdc8fddb9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:38.798069 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 858/1702] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:38.799660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28e2d-b325-436c-9b00-448c22a0bed1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:38.800126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28e2d-b325-436c-9b00-448c22a0bed1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:38.800603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28e2d-b325-436c-9b00-448c22a0bed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:38.852639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9bc28e2d-b325-436c-9b00-448c22a0bed1 None None] MFA Rules not processed for user `2fe64fcd24244811b4e6506f1b212bff`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:38.853412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f0ab1dabe374b2ab1d3e7323bb1e357'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.853923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9bc28e2d-b325-436c-9b00-448c22a0bed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:38.866390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f0ab1dabe374b2ab1d3e7323bb1e357'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.890846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f0ab1dabe374b2ab1d3e7323bb1e357'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.902109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f0ab1dabe374b2ab1d3e7323bb1e357'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.903626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f0ab1dabe374b2ab1d3e7323bb1e357'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:38.906713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:38.914111 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 845/1703] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:38 2022] DELETE /identity/v3/users/2f0ab1dabe374b2ab1d3e7323bb1e357 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:38.915492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:38.917311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:38.957463 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 859/1704] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:38.959520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b452f42-843b-4f50-a7c0-6f74ba805a90 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:38.960159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b452f42-843b-4f50-a7c0-6f74ba805a90 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:38.960605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b452f42-843b-4f50-a7c0-6f74ba805a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:38.984821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7b452f42-843b-4f50-a7c0-6f74ba805a90 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:38.986538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7b452f42-843b-4f50-a7c0-6f74ba805a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:39.004312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c59e-4c69-469c-bccb-16d94d86d905 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:39.011506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b192d9f-b794-45ed-9c6e-ed3f07908924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:39.012443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b192d9f-b794-45ed-9c6e-ed3f07908924 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:39.012925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b192d9f-b794-45ed-9c6e-ed3f07908924 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:39.013440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b192d9f-b794-45ed-9c6e-ed3f07908924 admin admin] PATH_INFO: `/v3/projects/31caa50628804cd597330477886dc146/users/7ea831d2d7ab4ccf898c5236b932db85/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:39.016139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b192d9f-b794-45ed-9c6e-ed3f07908924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31caa50628804cd597330477886dc146, user_id=7ea831d2d7ab4ccf898c5236b932db85, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:39.018908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b192d9f-b794-45ed-9c6e-ed3f07908924 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:39.031240 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 846/1705] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:38 2022] PUT /identity/v3/projects/31caa50628804cd597330477886dc146/users/7ea831d2d7ab4ccf898c5236b932db85/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:39.031872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:39.032092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b192d9f-b794-45ed-9c6e-ed3f07908924 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:39.033813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7b192d9f-b794-45ed-9c6e-ed3f07908924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:39.064523 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 860/1706] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:39.073161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39d8c7-7374-4da9-9bb1-348b53829dcf None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:39.073940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39d8c7-7374-4da9-9bb1-348b53829dcf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:39.074448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39d8c7-7374-4da9-9bb1-348b53829dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:39.113486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ec39d8c7-7374-4da9-9bb1-348b53829dcf None None] MFA Rules not processed for user `7ea831d2d7ab4ccf898c5236b932db85`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:39.115583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ec39d8c7-7374-4da9-9bb1-348b53829dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:39.149298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b192d9f-b794-45ed-9c6e-ed3f07908924 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:39.156729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d09f8b-8e2e-4db1-860b-1a8518c8533f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:39.157365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d09f8b-8e2e-4db1-860b-1a8518c8533f service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:39.157637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d09f8b-8e2e-4db1-860b-1a8518c8533f service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:39.157902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d09f8b-8e2e-4db1-860b-1a8518c8533f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:39.220095 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 861/1707] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:39.222107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39d8c7-7374-4da9-9bb1-348b53829dcf None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:39.224730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ec39d8c7-7374-4da9-9bb1-348b53829dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:39.266556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d09f8b-8e2e-4db1-860b-1a8518c8533f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:39.268739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d09f8b-8e2e-4db1-860b-1a8518c8533f service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:39.280005 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 847/1708] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:39.301551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39d8c7-7374-4da9-9bb1-348b53829dcf None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:39.312320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d885f-0527-4144-b767-835f477513b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:39.312984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d885f-0527-4144-b767-835f477513b6 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:39.313289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d885f-0527-4144-b767-835f477513b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:39.313564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d885f-0527-4144-b767-835f477513b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:39.390046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d09f8b-8e2e-4db1-860b-1a8518c8533f service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:39.406515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d09f8b-8e2e-4db1-860b-1a8518c8533f service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:39.412941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4d259-cbe5-4b7e-9e5d-9fe118ae36b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:39.415143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d259-cbe5-4b7e-9e5d-9fe118ae36b7 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:39.415523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d259-cbe5-4b7e-9e5d-9fe118ae36b7 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:39.415883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d259-cbe5-4b7e-9e5d-9fe118ae36b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:39.417624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d885f-0527-4144-b767-835f477513b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:39.419236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d885f-0527-4144-b767-835f477513b6 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:39.428858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4d259-cbe5-4b7e-9e5d-9fe118ae36b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:39.430499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4d259-cbe5-4b7e-9e5d-9fe118ae36b7 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:39.435834 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 862/1709] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:39.466016 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 848/1710] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:40.039357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d885f-0527-4144-b767-835f477513b6 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:40.041587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-be0d885f-0527-4144-b767-835f477513b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:40.070150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d885f-0527-4144-b767-835f477513b6 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:40.088047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:40.091865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:40.092278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:40.092657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] PATH_INFO: `/v3/projects/95c8e192e5e8482f82d9604cb3a12e7e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:40.094118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95c8e192e5e8482f82d9604cb3a12e7e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:40.096682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:40.110271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4d259-cbe5-4b7e-9e5d-9fe118ae36b7 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:40.111956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-57a4d259-cbe5-4b7e-9e5d-9fe118ae36b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:40.130912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4d259-cbe5-4b7e-9e5d-9fe118ae36b7 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:40.136279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:40.136967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:40.137364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:40.137640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] PATH_INFO: `/v3/users/2d907d81af2a4c5486fc464320bb84e4` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:40.138952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d907d81af2a4c5486fc464320bb84e4)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:40.140901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:40.141458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:40.147463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:40.219863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95c8e192e5e8482f82d9604cb3a12e7e', 'request_id': 'req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.229291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d907d81af2a4c5486fc464320bb84e4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.240795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95c8e192e5e8482f82d9604cb3a12e7e', 'request_id': 'req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.246889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d907d81af2a4c5486fc464320bb84e4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.252534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95c8e192e5e8482f82d9604cb3a12e7e', 'request_id': 'req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.265736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d907d81af2a4c5486fc464320bb84e4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.278015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] The token cache is being invalidate because project 95c8e192e5e8482f82d9604cb3a12e7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:40.278415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.278670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d907d81af2a4c5486fc464320bb84e4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.279244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.279353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9bea4a12-9917-45e8-bed2-7b2bb21f98e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d907d81af2a4c5486fc464320bb84e4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.280904 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 863/1711] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:40 2022] DELETE /identity/v3/projects/95c8e192e5e8482f82d9604cb3a12e7e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:40.288033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:40.290884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:40.298589 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 849/1712] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:40 2022] DELETE /identity/v3/users/2d907d81af2a4c5486fc464320bb84e4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:40.301407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782dbe2-8701-4fb3-b979-544cd478df0e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:40.301687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782dbe2-8701-4fb3-b979-544cd478df0e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:40.302078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782dbe2-8701-4fb3-b979-544cd478df0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:40.339045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6782dbe2-8701-4fb3-b979-544cd478df0e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:40.341848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6782dbe2-8701-4fb3-b979-544cd478df0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:40.402914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb1c49-d265-4ab2-bcff-0c6a9d040a71 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:40.429497 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 850/1713] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:40.485544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:40.486188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:40.486458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:40.486722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] PATH_INFO: `/v3/users/5f8db9ba76f84384b2f20bbffe100b67` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:40.488146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8db9ba76f84384b2f20bbffe100b67)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:40.490661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:40.579164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8db9ba76f84384b2f20bbffe100b67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.592083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:40.619064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8db9ba76f84384b2f20bbffe100b67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.639127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8db9ba76f84384b2f20bbffe100b67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.651065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8db9ba76f84384b2f20bbffe100b67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.655681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8db9ba76f84384b2f20bbffe100b67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:40.677538 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 864/1714] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:40 2022] DELETE /identity/v3/users/5f8db9ba76f84384b2f20bbffe100b67 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:40.684364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782dbe2-8701-4fb3-b979-544cd478df0e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:40.687708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6782dbe2-8701-4fb3-b979-544cd478df0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:40.836379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782dbe2-8701-4fb3-b979-544cd478df0e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:40.914730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:40.952212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96eed70-047e-4a3d-989f-f0c285a63d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:40.954232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eed70-047e-4a3d-989f-f0c285a63d43 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:40.954232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eed70-047e-4a3d-989f-f0c285a63d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:40.954232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eed70-047e-4a3d-989f-f0c285a63d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:40.956430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f96eed70-047e-4a3d-989f-f0c285a63d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:41.075949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96eed70-047e-4a3d-989f-f0c285a63d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:41.078230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96eed70-047e-4a3d-989f-f0c285a63d43 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:41.093452 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 851/1715] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:41.462423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:41.464004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:41.487366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbeff9-0403-4a42-8c8b-bbc717b108d3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:41.494006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:41.495030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:41.495502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:41.495768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] PATH_INFO: `/v3/projects/dc24ff4f02c848cca3fa4c91366d7909` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:41.497479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc24ff4f02c848cca3fa4c91366d7909)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:41.499594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:41.523142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96eed70-047e-4a3d-989f-f0c285a63d43 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:41.524897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f96eed70-047e-4a3d-989f-f0c285a63d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:41.538937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96eed70-047e-4a3d-989f-f0c285a63d43 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:41.544389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:41.545110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:41.545360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:41.545628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] PATH_INFO: `/v3/projects/5b8143192a324b3782a0650d3efb060c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:41.546771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8143192a324b3782a0650d3efb060c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:41.549081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:41.591858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc24ff4f02c848cca3fa4c91366d7909', 'request_id': 'req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.601393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc24ff4f02c848cca3fa4c91366d7909', 'request_id': 'req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.609386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc24ff4f02c848cca3fa4c91366d7909', 'request_id': 'req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.628333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8143192a324b3782a0650d3efb060c', 'request_id': 'req-7921d3a0-44c7-4e73-a29d-b904b29d6729'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.631942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] The token cache is being invalidate because project dc24ff4f02c848cca3fa4c91366d7909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:41.632408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.633789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bd33dcd-5a7c-493d-bf6d-5af93417d90f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.641090 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 865/1716] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:41 2022] DELETE /identity/v3/projects/dc24ff4f02c848cca3fa4c91366d7909 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:41.642433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8143192a324b3782a0650d3efb060c', 'request_id': 'req-7921d3a0-44c7-4e73-a29d-b904b29d6729'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.643925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0280a4-f062-45b5-ad34-79f7a1104663 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:41.644385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0280a4-f062-45b5-ad34-79f7a1104663 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:41.644860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0280a4-f062-45b5-ad34-79f7a1104663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:41.650960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8143192a324b3782a0650d3efb060c', 'request_id': 'req-7921d3a0-44c7-4e73-a29d-b904b29d6729'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.666044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] The token cache is being invalidate because project 5b8143192a324b3782a0650d3efb060c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:41.666044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.666706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.671322 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 852/1717] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:41 2022] DELETE /identity/v3/projects/5b8143192a324b3782a0650d3efb060c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:41.673150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:41.675101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:41.680667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bd0280a4-f062-45b5-ad34-79f7a1104663 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:41.681920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bd0280a4-f062-45b5-ad34-79f7a1104663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:41.747316 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 866/1718] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:41.747664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921d3a0-44c7-4e73-a29d-b904b29d6729 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:41.752461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:41.753117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:41.753396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:41.753990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] PATH_INFO: `/v3/users/2fe64fcd24244811b4e6506f1b212bff` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:41.754879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fe64fcd24244811b4e6506f1b212bff)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:41.755865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5f297-8987-472b-aadf-eed06e826481 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:41.756149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5f297-8987-472b-aadf-eed06e826481 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:41.756417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5f297-8987-472b-aadf-eed06e826481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:41.757491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:41.775771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:41.775974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-76a5f297-8987-472b-aadf-eed06e826481 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:41.777153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-76a5f297-8987-472b-aadf-eed06e826481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:41.785725 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 867/1719] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:48:41.790325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5f297-8987-472b-aadf-eed06e826481 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:41.800306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5f297-8987-472b-aadf-eed06e826481 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:41.804928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c04f0d-166c-4929-ac1c-81bcfbc61d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:41.805537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c04f0d-166c-4929-ac1c-81bcfbc61d43 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:41.805899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c04f0d-166c-4929-ac1c-81bcfbc61d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:41.806070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c04f0d-166c-4929-ac1c-81bcfbc61d43 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:41.806705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c04f0d-166c-4929-ac1c-81bcfbc61d43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:41.806999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c04f0d-166c-4929-ac1c-81bcfbc61d43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:41.808304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fe64fcd24244811b4e6506f1b212bff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.808588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c04f0d-166c-4929-ac1c-81bcfbc61d43 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:41.815706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fe64fcd24244811b4e6506f1b212bff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.821006 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 868/1720] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:48:41.825357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c04f0d-166c-4929-ac1c-81bcfbc61d43 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:41.829325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fe64fcd24244811b4e6506f1b212bff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.837423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c04f0d-166c-4929-ac1c-81bcfbc61d43 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:41.839192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fe64fcd24244811b4e6506f1b212bff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.839835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fe64fcd24244811b4e6506f1b212bff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:41.843913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafbbc7-71de-4dc3-8e6a-b65e92257fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:41.844577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafbbc7-71de-4dc3-8e6a-b65e92257fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:41.844918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafbbc7-71de-4dc3-8e6a-b65e92257fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:41.845222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafbbc7-71de-4dc3-8e6a-b65e92257fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:41.845951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafbbc7-71de-4dc3-8e6a-b65e92257fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:41.846536 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 853/1721] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:41 2022] DELETE /identity/v3/users/2fe64fcd24244811b4e6506f1b212bff => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:41.851624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafbbc7-71de-4dc3-8e6a-b65e92257fd4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:41.886352 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 869/1722] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:41 2022] POST /identity/v3/projects => generated 393 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:41.890530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:41.891963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:41.979721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2784b69-8c7a-4b28-992d-2803c0a40655 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.079997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cc683-57b7-4cb6-b66f-3ed7718cf394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.080691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cc683-57b7-4cb6-b66f-3ed7718cf394 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.080977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cc683-57b7-4cb6-b66f-3ed7718cf394 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.081446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cc683-57b7-4cb6-b66f-3ed7718cf394 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.082094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cc683-57b7-4cb6-b66f-3ed7718cf394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.085637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cc683-57b7-4cb6-b66f-3ed7718cf394 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.123829 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 854/1723] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:41 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:42.125072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafbbc7-71de-4dc3-8e6a-b65e92257fd4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.142639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafbbc7-71de-4dc3-8e6a-b65e92257fd4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.148129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2528a07-2c62-4946-b13b-604da2575d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.148739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2528a07-2c62-4946-b13b-604da2575d9f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.149012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2528a07-2c62-4946-b13b-604da2575d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.149480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2528a07-2c62-4946-b13b-604da2575d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.149972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2528a07-2c62-4946-b13b-604da2575d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.151568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2528a07-2c62-4946-b13b-604da2575d9f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.160802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:42.171279 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 870/1724] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:42.172340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cc683-57b7-4cb6-b66f-3ed7718cf394 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.186917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cc683-57b7-4cb6-b66f-3ed7718cf394 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.200389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c6f43-34f1-40da-b76e-fba405d29d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.201695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c6f43-34f1-40da-b76e-fba405d29d84 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.202354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c6f43-34f1-40da-b76e-fba405d29d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.202824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c6f43-34f1-40da-b76e-fba405d29d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.204229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c6f43-34f1-40da-b76e-fba405d29d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.206761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c6f43-34f1-40da-b76e-fba405d29d84 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.221467 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 855/1725] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:42.226508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2528a07-2c62-4946-b13b-604da2575d9f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.251592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2528a07-2c62-4946-b13b-604da2575d9f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.264003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c6f43-34f1-40da-b76e-fba405d29d84 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.265770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cd5c6f43-34f1-40da-b76e-fba405d29d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:42.271929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e5d75-f17b-436f-82ab-8b4632f75947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.272682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e5d75-f17b-436f-82ab-8b4632f75947 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.273055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e5d75-f17b-436f-82ab-8b4632f75947 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.273948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e5d75-f17b-436f-82ab-8b4632f75947 admin admin] PATH_INFO: `/v3/projects/a2d555ea5162478db2ba424f8351b2a3/users/9ef1f3504aac48f6a241c0d546271d06/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.280745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c6f43-34f1-40da-b76e-fba405d29d84 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.287903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.289057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.289057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.289304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] PATH_INFO: `/v3/projects/0325b17a072841fbb0c06c45040e912e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.291110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0325b17a072841fbb0c06c45040e912e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.294328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.296945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e5d75-f17b-436f-82ab-8b4632f75947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d555ea5162478db2ba424f8351b2a3, user_id=9ef1f3504aac48f6a241c0d546271d06, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.299549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e5d75-f17b-436f-82ab-8b4632f75947 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.316306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:42.327163 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 871/1726] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:42 2022] PUT /identity/v3/projects/a2d555ea5162478db2ba424f8351b2a3/users/9ef1f3504aac48f6a241c0d546271d06/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:42.339229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f9726-d03b-4427-9287-a508c623b7b8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.339676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f9726-d03b-4427-9287-a508c623b7b8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.340693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f9726-d03b-4427-9287-a508c623b7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.369907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0325b17a072841fbb0c06c45040e912e', 'request_id': 'req-506b3fb4-46ab-4c2e-aad6-3fc97206134c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:42.381610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0325b17a072841fbb0c06c45040e912e', 'request_id': 'req-506b3fb4-46ab-4c2e-aad6-3fc97206134c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:42.386683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a93f9726-d03b-4427-9287-a508c623b7b8 None None] MFA Rules not processed for user `9ef1f3504aac48f6a241c0d546271d06`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:42.388189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a93f9726-d03b-4427-9287-a508c623b7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:42.392967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0325b17a072841fbb0c06c45040e912e', 'request_id': 'req-506b3fb4-46ab-4c2e-aad6-3fc97206134c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:42.405618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] The token cache is being invalidate because project 0325b17a072841fbb0c06c45040e912e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:42.405907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:42.406574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-506b3fb4-46ab-4c2e-aad6-3fc97206134c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:42.413151 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 856/1727] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:42 2022] DELETE /identity/v3/projects/0325b17a072841fbb0c06c45040e912e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:42.415781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8953a-f428-4ec1-b1a6-d4e0a0eb2612 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.416071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8953a-f428-4ec1-b1a6-d4e0a0eb2612 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.416348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8953a-f428-4ec1-b1a6-d4e0a0eb2612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.462737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-10f8953a-f428-4ec1-b1a6-d4e0a0eb2612 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:42.464282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-10f8953a-f428-4ec1-b1a6-d4e0a0eb2612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:42.497700 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 872/1728] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:42.497700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f9726-d03b-4427-9287-a508c623b7b8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.497700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a93f9726-d03b-4427-9287-a508c623b7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:42.569725 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 857/1729] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:42.572208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f9726-d03b-4427-9287-a508c623b7b8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.577869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747991b8-a461-44db-9082-3635f205e515 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.578359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747991b8-a461-44db-9082-3635f205e515 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.578737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747991b8-a461-44db-9082-3635f205e515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.580253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb35da-10de-4308-bfe5-f63d08961bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.580938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb35da-10de-4308-bfe5-f63d08961bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.581320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb35da-10de-4308-bfe5-f63d08961bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.581712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb35da-10de-4308-bfe5-f63d08961bf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.582415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb35da-10de-4308-bfe5-f63d08961bf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.585184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb35da-10de-4308-bfe5-f63d08961bf5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.612965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-747991b8-a461-44db-9082-3635f205e515 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:42.616211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-747991b8-a461-44db-9082-3635f205e515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:42.623255 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 873/1730] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:42 2022] POST /identity/v3/projects => generated 395 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:42.624782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb35da-10de-4308-bfe5-f63d08961bf5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.686740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:42.703584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb35da-10de-4308-bfe5-f63d08961bf5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.710565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dc0ac-987d-4df1-bab7-bc383ae2ed52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.711180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dc0ac-987d-4df1-bab7-bc383ae2ed52 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.711453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dc0ac-987d-4df1-bab7-bc383ae2ed52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.711719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dc0ac-987d-4df1-bab7-bc383ae2ed52 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.712481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4dc0ac-987d-4df1-bab7-bc383ae2ed52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.714616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4dc0ac-987d-4df1-bab7-bc383ae2ed52 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.716161 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 858/1731] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:42.722110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747991b8-a461-44db-9082-3635f205e515 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.732564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747991b8-a461-44db-9082-3635f205e515 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.737930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1492e3-300c-4c44-b298-885fd706670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.738088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1492e3-300c-4c44-b298-885fd706670b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.738354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1492e3-300c-4c44-b298-885fd706670b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.738711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1492e3-300c-4c44-b298-885fd706670b admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.740058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1492e3-300c-4c44-b298-885fd706670b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:42.740563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1492e3-300c-4c44-b298-885fd706670b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.742048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1492e3-300c-4c44-b298-885fd706670b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.766324 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 874/1732] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:42 2022] POST /identity/v3/users => generated 304 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:42.767414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dc0ac-987d-4df1-bab7-bc383ae2ed52 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.772455 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 859/1733] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:42.778122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1492e3-300c-4c44-b298-885fd706670b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.789156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dc0ac-987d-4df1-bab7-bc383ae2ed52 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.795129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098ed97-3a61-44fc-a5fa-a8de1ec66796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.796010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098ed97-3a61-44fc-a5fa-a8de1ec66796 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.796489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098ed97-3a61-44fc-a5fa-a8de1ec66796 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.796959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098ed97-3a61-44fc-a5fa-a8de1ec66796 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.797893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098ed97-3a61-44fc-a5fa-a8de1ec66796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.799660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098ed97-3a61-44fc-a5fa-a8de1ec66796 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.800691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1492e3-300c-4c44-b298-885fd706670b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.809487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a37532-8f6e-46fd-9a82-9482c5ad7d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.810777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a37532-8f6e-46fd-9a82-9482c5ad7d67 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.811058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a37532-8f6e-46fd-9a82-9482c5ad7d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.811329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a37532-8f6e-46fd-9a82-9482c5ad7d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.812218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a37532-8f6e-46fd-9a82-9482c5ad7d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.816420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a37532-8f6e-46fd-9a82-9482c5ad7d67 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.817623 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 875/1734] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:48:42.818708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098ed97-3a61-44fc-a5fa-a8de1ec66796 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.839175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098ed97-3a61-44fc-a5fa-a8de1ec66796 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:42.847234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c445c4-b702-4651-922f-5a04b0016c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:42.848718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c445c4-b702-4651-922f-5a04b0016c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:42.849355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c445c4-b702-4651-922f-5a04b0016c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:42.849877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c445c4-b702-4651-922f-5a04b0016c2e admin admin] PATH_INFO: `/v3/projects/b6069eff7ef84788972155fabf617d31/users/3721e6caed9b42509c7a9d2522a1b9ce/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:42.869263 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 860/1735] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:42 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:42.871187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c445c4-b702-4651-922f-5a04b0016c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6069eff7ef84788972155fabf617d31, user_id=3721e6caed9b42509c7a9d2522a1b9ce, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:42.872963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a37532-8f6e-46fd-9a82-9482c5ad7d67 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:42.874042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c445c4-b702-4651-922f-5a04b0016c2e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:42.888348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:42.910550 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 876/1736] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:42 2022] PUT /identity/v3/projects/b6069eff7ef84788972155fabf617d31/users/3721e6caed9b42509c7a9d2522a1b9ce/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:42.912431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c445c4-b702-4651-922f-5a04b0016c2e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:43.009897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a37532-8f6e-46fd-9a82-9482c5ad7d67 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:43.049107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c445c4-b702-4651-922f-5a04b0016c2e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:43.062238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c31f32-1eab-4ce3-8ad8-97f9a56d75ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:43.064354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c31f32-1eab-4ce3-8ad8-97f9a56d75ae admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.064643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c31f32-1eab-4ce3-8ad8-97f9a56d75ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.064912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c31f32-1eab-4ce3-8ad8-97f9a56d75ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.066066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c31f32-1eab-4ce3-8ad8-97f9a56d75ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:43.068861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c31f32-1eab-4ce3-8ad8-97f9a56d75ae admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:43.087671 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 877/1737] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:43.091756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c31f32-1eab-4ce3-8ad8-97f9a56d75ae admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:43.121588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c31f32-1eab-4ce3-8ad8-97f9a56d75ae admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:43.128708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502492da-f279-4d75-a51e-b39679d01596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:43.129772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502492da-f279-4d75-a51e-b39679d01596 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.130433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502492da-f279-4d75-a51e-b39679d01596 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.131138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502492da-f279-4d75-a51e-b39679d01596 admin admin] PATH_INFO: `/v3/projects/b6069eff7ef84788972155fabf617d31/users/3721e6caed9b42509c7a9d2522a1b9ce/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.133591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502492da-f279-4d75-a51e-b39679d01596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6069eff7ef84788972155fabf617d31, user_id=3721e6caed9b42509c7a9d2522a1b9ce, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:43.136469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ffac4-bfbb-439d-a557-0a1285297592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:43.137430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ffac4-bfbb-439d-a557-0a1285297592 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.137803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502492da-f279-4d75-a51e-b39679d01596 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:43.138064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ffac4-bfbb-439d-a557-0a1285297592 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.138546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ffac4-bfbb-439d-a557-0a1285297592 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.139614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ffac4-bfbb-439d-a557-0a1285297592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:43.142790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ffac4-bfbb-439d-a557-0a1285297592 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:43.153120 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 878/1738] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:43 2022] PUT /identity/v3/projects/b6069eff7ef84788972155fabf617d31/users/3721e6caed9b42509c7a9d2522a1b9ce/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:43.166667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79b432-3699-4936-9853-36156dea6fa5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.166942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79b432-3699-4936-9853-36156dea6fa5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.167210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79b432-3699-4936-9853-36156dea6fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.181246 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 861/1739] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:42 2022] POST /identity/v3/users => generated 287 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:43.182601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ffac4-bfbb-439d-a557-0a1285297592 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:43.203896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ffac4-bfbb-439d-a557-0a1285297592 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:43.222825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6b79b432-3699-4936-9853-36156dea6fa5 None None] MFA Rules not processed for user `3721e6caed9b42509c7a9d2522a1b9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:43.226125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6b79b432-3699-4936-9853-36156dea6fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:43.290652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171ec45-06de-422a-a059-8f21fce1dce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:43.291298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171ec45-06de-422a-a059-8f21fce1dce5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.293835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171ec45-06de-422a-a059-8f21fce1dce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.294117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171ec45-06de-422a-a059-8f21fce1dce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.296260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171ec45-06de-422a-a059-8f21fce1dce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:43.298686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171ec45-06de-422a-a059-8f21fce1dce5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:43.315774 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 862/1740] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:43.321214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171ec45-06de-422a-a059-8f21fce1dce5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:43.354265 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 879/1741] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:48:43.354452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171ec45-06de-422a-a059-8f21fce1dce5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:43.375800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53f5f4-4f7e-4a27-8a79-4ded8cd3deab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:43.376391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53f5f4-4f7e-4a27-8a79-4ded8cd3deab admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.376681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53f5f4-4f7e-4a27-8a79-4ded8cd3deab admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.376948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53f5f4-4f7e-4a27-8a79-4ded8cd3deab admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.377860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53f5f4-4f7e-4a27-8a79-4ded8cd3deab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:43.382030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53f5f4-4f7e-4a27-8a79-4ded8cd3deab admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:43.410435 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 863/1742] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:43.416396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79b432-3699-4936-9853-36156dea6fa5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:43.433225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79b432-3699-4936-9853-36156dea6fa5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:43.441513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40260ac3-92f9-4bcb-aff4-1d101281446d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:43.442193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40260ac3-92f9-4bcb-aff4-1d101281446d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.442531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40260ac3-92f9-4bcb-aff4-1d101281446d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.443469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40260ac3-92f9-4bcb-aff4-1d101281446d admin admin] PATH_INFO: `/v3/projects/94e961feee714587bb2ffe545d286c5c/users/e32a1c53b4be44188b07c768405961ed/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.481010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:43.492824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40260ac3-92f9-4bcb-aff4-1d101281446d admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e961feee714587bb2ffe545d286c5c, user_id=e32a1c53b4be44188b07c768405961ed, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:43.495783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40260ac3-92f9-4bcb-aff4-1d101281446d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:43.534820 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 880/1743] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:43 2022] PUT /identity/v3/projects/94e961feee714587bb2ffe545d286c5c/users/e32a1c53b4be44188b07c768405961ed/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:43.545825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebae4dd-dfa3-412a-bf43-902ed78bfa50 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.546573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebae4dd-dfa3-412a-bf43-902ed78bfa50 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.546858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebae4dd-dfa3-412a-bf43-902ed78bfa50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.585681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1ebae4dd-dfa3-412a-bf43-902ed78bfa50 None None] MFA Rules not processed for user `e32a1c53b4be44188b07c768405961ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:43.586910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1ebae4dd-dfa3-412a-bf43-902ed78bfa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:43.617014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:43.629175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40260ac3-92f9-4bcb-aff4-1d101281446d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:43.632211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-40260ac3-92f9-4bcb-aff4-1d101281446d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:43.687615 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 864/1744] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:43.691850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebae4dd-dfa3-412a-bf43-902ed78bfa50 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:43.738955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40260ac3-92f9-4bcb-aff4-1d101281446d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:43.751354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800e323-19c6-448b-81f4-d14f865da4ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:43.751964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800e323-19c6-448b-81f4-d14f865da4ad service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.752235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800e323-19c6-448b-81f4-d14f865da4ad service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.752500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800e323-19c6-448b-81f4-d14f865da4ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.780289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebae4dd-dfa3-412a-bf43-902ed78bfa50 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:43.788475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed64864-a882-4a78-a81a-00973ad81c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:43.788670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed64864-a882-4a78-a81a-00973ad81c5a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.788851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed64864-a882-4a78-a81a-00973ad81c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.789131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed64864-a882-4a78-a81a-00973ad81c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.789764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed64864-a882-4a78-a81a-00973ad81c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:43.792826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed64864-a882-4a78-a81a-00973ad81c5a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:43.820618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800e323-19c6-448b-81f4-d14f865da4ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:43.822893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800e323-19c6-448b-81f4-d14f865da4ad service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:43.833749 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 865/1745] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:43 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:43.834213 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 881/1746] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:43 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:43.837401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed64864-a882-4a78-a81a-00973ad81c5a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:43.916378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed64864-a882-4a78-a81a-00973ad81c5a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:43.928878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad099fe8-9ecf-4003-a7b7-529ce8212083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:43.929721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad099fe8-9ecf-4003-a7b7-529ce8212083 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:43.929994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad099fe8-9ecf-4003-a7b7-529ce8212083 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:43.930256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad099fe8-9ecf-4003-a7b7-529ce8212083 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:43.930909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad099fe8-9ecf-4003-a7b7-529ce8212083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:43.933711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad099fe8-9ecf-4003-a7b7-529ce8212083 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:43.970469 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 866/1747] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:43 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:43.975115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800e323-19c6-448b-81f4-d14f865da4ad service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:43.988906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800e323-19c6-448b-81f4-d14f865da4ad service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:44.000714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf806618-b164-44ed-b20a-9bef9d726405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:44.001861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf806618-b164-44ed-b20a-9bef9d726405 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.003026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf806618-b164-44ed-b20a-9bef9d726405 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.003396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf806618-b164-44ed-b20a-9bef9d726405 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.004541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf806618-b164-44ed-b20a-9bef9d726405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:44.006421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf806618-b164-44ed-b20a-9bef9d726405 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:44.021982 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 882/1748] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:44.023142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad099fe8-9ecf-4003-a7b7-529ce8212083 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:44.039811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad099fe8-9ecf-4003-a7b7-529ce8212083 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:44.045146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef257a-cc74-4aba-9a69-9a006b73667d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:44.045811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef257a-cc74-4aba-9a69-9a006b73667d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.046198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef257a-cc74-4aba-9a69-9a006b73667d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.046663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef257a-cc74-4aba-9a69-9a006b73667d admin admin] PATH_INFO: `/v3/projects/08e2c30e5266478893afa293c836a3ed/users/81fdbd60e924457692039abcb456cb89/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.067406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef257a-cc74-4aba-9a69-9a006b73667d admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e2c30e5266478893afa293c836a3ed, user_id=81fdbd60e924457692039abcb456cb89, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:44.070994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef257a-cc74-4aba-9a69-9a006b73667d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:44.094188 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 867/1749] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:44 2022] PUT /identity/v3/projects/08e2c30e5266478893afa293c836a3ed/users/81fdbd60e924457692039abcb456cb89/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:44.099544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf806618-b164-44ed-b20a-9bef9d726405 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:44.118044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf806618-b164-44ed-b20a-9bef9d726405 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:44.205448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef257a-cc74-4aba-9a69-9a006b73667d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:44.234560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:44.268901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d937285-a540-4280-92fc-ffb0f519f57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:44.269533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d937285-a540-4280-92fc-ffb0f519f57b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.269808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d937285-a540-4280-92fc-ffb0f519f57b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.270149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d937285-a540-4280-92fc-ffb0f519f57b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.271269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d937285-a540-4280-92fc-ffb0f519f57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:44.273172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d937285-a540-4280-92fc-ffb0f519f57b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:44.291940 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 883/1750] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:48:44.296090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d937285-a540-4280-92fc-ffb0f519f57b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:44.307890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d937285-a540-4280-92fc-ffb0f519f57b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:44.313029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242c63e-4a1b-4958-affb-70297d8b516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:44.313477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef257a-cc74-4aba-9a69-9a006b73667d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:44.313707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242c63e-4a1b-4958-affb-70297d8b516c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.313978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242c63e-4a1b-4958-affb-70297d8b516c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.314241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242c63e-4a1b-4958-affb-70297d8b516c admin admin] PATH_INFO: `/v3/projects/08e2c30e5266478893afa293c836a3ed/users/81fdbd60e924457692039abcb456cb89/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.316141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242c63e-4a1b-4958-affb-70297d8b516c admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e2c30e5266478893afa293c836a3ed, user_id=81fdbd60e924457692039abcb456cb89, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:44.319578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242c63e-4a1b-4958-affb-70297d8b516c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:44.320484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c999a0-0173-41e6-ad4b-52c780db12d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:44.321113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c999a0-0173-41e6-ad4b-52c780db12d8 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.321392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c999a0-0173-41e6-ad4b-52c780db12d8 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.321670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c999a0-0173-41e6-ad4b-52c780db12d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.322427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:44.338827 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 884/1751] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:44 2022] PUT /identity/v3/projects/08e2c30e5266478893afa293c836a3ed/users/81fdbd60e924457692039abcb456cb89/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:44.351329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78caea6-f6e5-44a0-b487-c78126c637ca None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.351636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78caea6-f6e5-44a0-b487-c78126c637ca None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.352128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78caea6-f6e5-44a0-b487-c78126c637ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.390648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a78caea6-f6e5-44a0-b487-c78126c637ca None None] MFA Rules not processed for user `81fdbd60e924457692039abcb456cb89`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:44.392956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a78caea6-f6e5-44a0-b487-c78126c637ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:44.444149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c999a0-0173-41e6-ad4b-52c780db12d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:44.446297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c999a0-0173-41e6-ad4b-52c780db12d8 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:44.481429 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 868/1752] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:44.494564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c999a0-0173-41e6-ad4b-52c780db12d8 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:44.515297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c999a0-0173-41e6-ad4b-52c780db12d8 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:44.545019 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 885/1753] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:44.556762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710de534-81e0-4215-bcb4-5aa78ce54555 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.557211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710de534-81e0-4215-bcb4-5aa78ce54555 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.564740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710de534-81e0-4215-bcb4-5aa78ce54555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.590477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-710de534-81e0-4215-bcb4-5aa78ce54555 None None] MFA Rules not processed for user `e32a1c53b4be44188b07c768405961ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:44.592084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-710de534-81e0-4215-bcb4-5aa78ce54555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:44.642428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:44.643165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.643836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.644178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] PATH_INFO: `/v3/users/9ef1f3504aac48f6a241c0d546271d06` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.647005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ef1f3504aac48f6a241c0d546271d06)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:44.649411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:44.683624 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 886/1754] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:44.721368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ef1f3504aac48f6a241c0d546271d06'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:44.725410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:44.732887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ef1f3504aac48f6a241c0d546271d06'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:44.748637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ef1f3504aac48f6a241c0d546271d06'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:44.762154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ef1f3504aac48f6a241c0d546271d06'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:44.763423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ef1f3504aac48f6a241c0d546271d06'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:44.772421 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 869/1755] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:44 2022] DELETE /identity/v3/users/9ef1f3504aac48f6a241c0d546271d06 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:44.776938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24615-99b6-4a48-a9d9-2d26fafbded3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.777477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24615-99b6-4a48-a9d9-2d26fafbded3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.777816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24615-99b6-4a48-a9d9-2d26fafbded3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.798202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b2a24615-99b6-4a48-a9d9-2d26fafbded3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:44.799512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b2a24615-99b6-4a48-a9d9-2d26fafbded3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:44.878408 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 887/1756] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:44.882709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:44.884122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:44.889924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a24615-99b6-4a48-a9d9-2d26fafbded3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:44.891377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b2a24615-99b6-4a48-a9d9-2d26fafbded3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:44.921726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:44.966651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a24615-99b6-4a48-a9d9-2d26fafbded3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:44.970918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887def9-b4fc-4de2-a9a2-0ff902f20618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:44.971617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887def9-b4fc-4de2-a9a2-0ff902f20618 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.972007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887def9-b4fc-4de2-a9a2-0ff902f20618 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.972370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887def9-b4fc-4de2-a9a2-0ff902f20618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.981353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6823d7-e2f4-4ef8-8b96-306856f96287 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:44.984427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887def9-b4fc-4de2-a9a2-0ff902f20618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:44.987329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887def9-b4fc-4de2-a9a2-0ff902f20618 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:44.994987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689c395-efda-4cce-8cb0-86884539e01b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:44.995733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689c395-efda-4cce-8cb0-86884539e01b service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:44.995961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689c395-efda-4cce-8cb0-86884539e01b service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:44.996229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689c395-efda-4cce-8cb0-86884539e01b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:44.997981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0689c395-efda-4cce-8cb0-86884539e01b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:44.999435 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 888/1757] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:45.071073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689c395-efda-4cce-8cb0-86884539e01b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:45.073366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689c395-efda-4cce-8cb0-86884539e01b service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:45.084621 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 870/1758] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:45.416092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887def9-b4fc-4de2-a9a2-0ff902f20618 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:45.417785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3887def9-b4fc-4de2-a9a2-0ff902f20618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:45.444878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887def9-b4fc-4de2-a9a2-0ff902f20618 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:45.450448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:45.451433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:45.451974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:45.452419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] PATH_INFO: `/v3/projects/a2d555ea5162478db2ba424f8351b2a3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:45.453766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d555ea5162478db2ba424f8351b2a3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:45.456346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:45.525860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689c395-efda-4cce-8cb0-86884539e01b service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:45.539501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689c395-efda-4cce-8cb0-86884539e01b service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:45.552798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d555ea5162478db2ba424f8351b2a3', 'request_id': 'req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:45.579545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d555ea5162478db2ba424f8351b2a3', 'request_id': 'req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:45.591521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d555ea5162478db2ba424f8351b2a3', 'request_id': 'req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:45.617586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] The token cache is being invalidate because project a2d555ea5162478db2ba424f8351b2a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:45.617879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:45.619371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:45.623331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:45.624836 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 889/1759] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:45 2022] DELETE /identity/v3/projects/a2d555ea5162478db2ba424f8351b2a3 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:45.630120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:45.633730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:45.637276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b8fe8-8bbd-49a4-87b7-884a837f7141 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:45.638500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b8fe8-8bbd-49a4-87b7-884a837f7141 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:45.639669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b8fe8-8bbd-49a4-87b7-884a837f7141 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:45.640821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b8fe8-8bbd-49a4-87b7-884a837f7141 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:45.644688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c19b8fe8-8bbd-49a4-87b7-884a837f7141 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:45.743844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b8fe8-8bbd-49a4-87b7-884a837f7141 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:45.749234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b8fe8-8bbd-49a4-87b7-884a837f7141 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:45.761799 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 871/1760] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:45.834130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f6d6c-1c7e-4ecc-b05b-b5af4469ebbd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:45.841848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:45.842784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:45.843139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:45.843501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] PATH_INFO: `/v3/users/3721e6caed9b42509c7a9d2522a1b9ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:45.844834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3721e6caed9b42509c7a9d2522a1b9ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:45.847551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:45.922624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3721e6caed9b42509c7a9d2522a1b9ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:45.929670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:45.954732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3721e6caed9b42509c7a9d2522a1b9ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:45.968498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3721e6caed9b42509c7a9d2522a1b9ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:45.984501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3721e6caed9b42509c7a9d2522a1b9ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:45.985930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-456bc7ef-89d2-4911-a35e-1da2ad4f9cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3721e6caed9b42509c7a9d2522a1b9ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:45.993371 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 890/1761] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:45 2022] DELETE /identity/v3/users/3721e6caed9b42509c7a9d2522a1b9ce => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:46.465488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b8fe8-8bbd-49a4-87b7-884a837f7141 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:46.492307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b8fe8-8bbd-49a4-87b7-884a837f7141 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:46.581200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:46.581716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:46.582065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:46.582404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] PATH_INFO: `/v3/projects/b6069eff7ef84788972155fabf617d31` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:46.584187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6069eff7ef84788972155fabf617d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:46.586767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:46.644693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6069eff7ef84788972155fabf617d31', 'request_id': 'req-413c5de3-6eeb-4702-a836-04b450661984'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:46.653837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6069eff7ef84788972155fabf617d31', 'request_id': 'req-413c5de3-6eeb-4702-a836-04b450661984'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:46.661280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6069eff7ef84788972155fabf617d31', 'request_id': 'req-413c5de3-6eeb-4702-a836-04b450661984'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:46.682127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] The token cache is being invalidate because project b6069eff7ef84788972155fabf617d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:46.682705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:46.683068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-413c5de3-6eeb-4702-a836-04b450661984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:46.684761 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 872/1762] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:46 2022] DELETE /identity/v3/projects/b6069eff7ef84788972155fabf617d31 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:46.690842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5aaec2-3178-4149-bc34-2c8013323997 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:46.690842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5aaec2-3178-4149-bc34-2c8013323997 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:46.690842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5aaec2-3178-4149-bc34-2c8013323997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:46.717843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8a5aaec2-3178-4149-bc34-2c8013323997 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:46.719259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a5aaec2-3178-4149-bc34-2c8013323997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:46.808253 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 891/1763] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:46.822763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6776b-4db1-414d-ac89-c7872bfc10a7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:46.824643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6776b-4db1-414d-ac89-c7872bfc10a7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:46.824643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6776b-4db1-414d-ac89-c7872bfc10a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:46.847726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-24c6776b-4db1-414d-ac89-c7872bfc10a7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:46.853652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-24c6776b-4db1-414d-ac89-c7872bfc10a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:46.876708 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 873/1764] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:46.882154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5aaec2-3178-4149-bc34-2c8013323997 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:46.897596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5aaec2-3178-4149-bc34-2c8013323997 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:46.908261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6776b-4db1-414d-ac89-c7872bfc10a7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:46.910102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-24c6776b-4db1-414d-ac89-c7872bfc10a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:46.912317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f455c9d-61a1-448d-babd-7cfc385154a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:46.913978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f455c9d-61a1-448d-babd-7cfc385154a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:46.916896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f455c9d-61a1-448d-babd-7cfc385154a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:46.917298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f455c9d-61a1-448d-babd-7cfc385154a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:46.921058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f455c9d-61a1-448d-babd-7cfc385154a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:46.921443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f455c9d-61a1-448d-babd-7cfc385154a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:46.924710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f455c9d-61a1-448d-babd-7cfc385154a7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:46.959825 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 892/1765] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:46.960027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:46.961074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f455c9d-61a1-448d-babd-7cfc385154a7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:46.999003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f455c9d-61a1-448d-babd-7cfc385154a7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:47.006359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38e8f5-a254-41cc-986a-b92110e47e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:47.006981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38e8f5-a254-41cc-986a-b92110e47e93 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:47.007252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38e8f5-a254-41cc-986a-b92110e47e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:47.008381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38e8f5-a254-41cc-986a-b92110e47e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:47.009580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38e8f5-a254-41cc-986a-b92110e47e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:47.012614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38e8f5-a254-41cc-986a-b92110e47e93 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:47.056735 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 893/1766] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:46 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:47.061462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38e8f5-a254-41cc-986a-b92110e47e93 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:47.111670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6776b-4db1-414d-ac89-c7872bfc10a7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:47.143941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38e8f5-a254-41cc-986a-b92110e47e93 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:47.151889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20a697-881a-4df1-b371-11767265dc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:47.152500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20a697-881a-4df1-b371-11767265dc92 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:47.152775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20a697-881a-4df1-b371-11767265dc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:47.153235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20a697-881a-4df1-b371-11767265dc92 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:47.153875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20a697-881a-4df1-b371-11767265dc92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:47.155888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20a697-881a-4df1-b371-11767265dc92 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:47.213222 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 894/1767] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:47 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:47.214617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20a697-881a-4df1-b371-11767265dc92 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:47.234552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20a697-881a-4df1-b371-11767265dc92 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:47.240428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651cdfe-00c6-4722-9c14-e05305b3209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:47.241438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651cdfe-00c6-4722-9c14-e05305b3209e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:47.241708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651cdfe-00c6-4722-9c14-e05305b3209e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:47.242063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651cdfe-00c6-4722-9c14-e05305b3209e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:47.242708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651cdfe-00c6-4722-9c14-e05305b3209e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:47.243270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba8c12-6ede-49ca-97c0-563f2fa34791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:47.243947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba8c12-6ede-49ca-97c0-563f2fa34791 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:47.244318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba8c12-6ede-49ca-97c0-563f2fa34791 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:47.244468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651cdfe-00c6-4722-9c14-e05305b3209e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:47.244721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba8c12-6ede-49ca-97c0-563f2fa34791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:47.247195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4aba8c12-6ede-49ca-97c0-563f2fa34791 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:47.255953 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 895/1768] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:47.261558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651cdfe-00c6-4722-9c14-e05305b3209e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:47.272864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651cdfe-00c6-4722-9c14-e05305b3209e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:47.279041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab231b0a-d71e-4fa9-8d8b-a6b1a3366a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:47.279721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231b0a-d71e-4fa9-8d8b-a6b1a3366a3d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:47.280123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231b0a-d71e-4fa9-8d8b-a6b1a3366a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:47.280397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231b0a-d71e-4fa9-8d8b-a6b1a3366a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:47.281084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab231b0a-d71e-4fa9-8d8b-a6b1a3366a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:47.282698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab231b0a-d71e-4fa9-8d8b-a6b1a3366a3d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:47.299926 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 896/1769] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:48:47.300931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab231b0a-d71e-4fa9-8d8b-a6b1a3366a3d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:47.307104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:47.315473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab231b0a-d71e-4fa9-8d8b-a6b1a3366a3d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:47.325190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba8c12-6ede-49ca-97c0-563f2fa34791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:47.326851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff500cdd-a163-453f-a883-0f58fbcc6a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:47.327018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba8c12-6ede-49ca-97c0-563f2fa34791 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:47.333632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff500cdd-a163-453f-a883-0f58fbcc6a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:47.334957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff500cdd-a163-453f-a883-0f58fbcc6a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:47.335559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff500cdd-a163-453f-a883-0f58fbcc6a61 admin admin] PATH_INFO: `/v3/projects/86408ec8ca9f4e33a16bdf9a23aa9faa/users/507faa1518b34884b3e095bfae431bc2/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:47.338524 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 874/1770] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:47.357305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff500cdd-a163-453f-a883-0f58fbcc6a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86408ec8ca9f4e33a16bdf9a23aa9faa, user_id=507faa1518b34884b3e095bfae431bc2, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:47.359100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff500cdd-a163-453f-a883-0f58fbcc6a61 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:47.380565 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 897/1771] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:47 2022] PUT /identity/v3/projects/86408ec8ca9f4e33a16bdf9a23aa9faa/users/507faa1518b34884b3e095bfae431bc2/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:47.385488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befde7f9-f973-49e1-953e-d317a03b897e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:47.385770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befde7f9-f973-49e1-953e-d317a03b897e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:47.386039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befde7f9-f973-49e1-953e-d317a03b897e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:47.409668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-befde7f9-f973-49e1-953e-d317a03b897e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:47.411078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-befde7f9-f973-49e1-953e-d317a03b897e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:47.449637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff500cdd-a163-453f-a883-0f58fbcc6a61 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:47.451697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ff500cdd-a163-453f-a883-0f58fbcc6a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:47.484673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:47.499091 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 875/1772] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:47.511322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befde7f9-f973-49e1-953e-d317a03b897e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:47.512897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-befde7f9-f973-49e1-953e-d317a03b897e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:47.546224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff500cdd-a163-453f-a883-0f58fbcc6a61 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:47.552816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:47.553940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:47.554065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:47.554358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] PATH_INFO: `/v3/users/a5beae9aac914403b1ff1b226f59813b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:47.555602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5beae9aac914403b1ff1b226f59813b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:47.571572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:47.609358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befde7f9-f973-49e1-953e-d317a03b897e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:47.609358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e86945-67e8-42eb-afa4-0d5ffd035c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:47.609358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e86945-67e8-42eb-afa4-0d5ffd035c50 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:47.609358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e86945-67e8-42eb-afa4-0d5ffd035c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:47.609754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e86945-67e8-42eb-afa4-0d5ffd035c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:47.619822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e86945-67e8-42eb-afa4-0d5ffd035c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:47.621383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e86945-67e8-42eb-afa4-0d5ffd035c50 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:47.633136 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 876/1773] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:47.671750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5beae9aac914403b1ff1b226f59813b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:47.691168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5beae9aac914403b1ff1b226f59813b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:47.704752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5beae9aac914403b1ff1b226f59813b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:47.717524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5beae9aac914403b1ff1b226f59813b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:47.718559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5beae9aac914403b1ff1b226f59813b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:47.730641 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 898/1774] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:47 2022] DELETE /identity/v3/users/a5beae9aac914403b1ff1b226f59813b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:47.733062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296c855-cfb9-4a7f-bc4e-895ac3442342 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:47.733480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296c855-cfb9-4a7f-bc4e-895ac3442342 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:47.734319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296c855-cfb9-4a7f-bc4e-895ac3442342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:47.773572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3296c855-cfb9-4a7f-bc4e-895ac3442342 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:47.774992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3296c855-cfb9-4a7f-bc4e-895ac3442342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:47.899455 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 877/1775] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:47.949127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:47.950463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:48.084662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39e157-12da-49eb-b379-0787705f22ee admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:48.088634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2dc70-4919-4564-96f6-feecb47d862e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:48.090753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2dc70-4919-4564-96f6-feecb47d862e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:48.091252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2dc70-4919-4564-96f6-feecb47d862e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:48.091617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2dc70-4919-4564-96f6-feecb47d862e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:48.106412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2dc70-4919-4564-96f6-feecb47d862e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:48.109906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2dc70-4919-4564-96f6-feecb47d862e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:48.125649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:48.150042 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 899/1776] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:48.484106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296c855-cfb9-4a7f-bc4e-895ac3442342 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:48.484106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3296c855-cfb9-4a7f-bc4e-895ac3442342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:48.499984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296c855-cfb9-4a7f-bc4e-895ac3442342 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:48.510906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:48.511876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:48.512249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:48.512634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] PATH_INFO: `/v3/users/4b16bd0809f3411a91fae6c5e2f3acee` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:48.515490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b16bd0809f3411a91fae6c5e2f3acee)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:48.519452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:48.613421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b16bd0809f3411a91fae6c5e2f3acee'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:48.630782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b16bd0809f3411a91fae6c5e2f3acee'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:48.657230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b16bd0809f3411a91fae6c5e2f3acee'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:48.669251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b16bd0809f3411a91fae6c5e2f3acee'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:48.670439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b16bd0809f3411a91fae6c5e2f3acee'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:48.684481 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 878/1777] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:48 2022] DELETE /identity/v3/users/4b16bd0809f3411a91fae6c5e2f3acee => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:48.689360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edf2d9-316e-4f11-b449-7a4e6a79d3b5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:48.689822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edf2d9-316e-4f11-b449-7a4e6a79d3b5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:48.690781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edf2d9-316e-4f11-b449-7a4e6a79d3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:48.754366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b4edf2d9-316e-4f11-b449-7a4e6a79d3b5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:48.757681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b4edf2d9-316e-4f11-b449-7a4e6a79d3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:48.979181 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 900/1778] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:49.051270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:49.057412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:49.088963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:49.222733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0ff20-c357-459d-b309-a83159aa2db3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:49.229889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d091fbf-3311-4f0d-a1a0-c26b7c473ddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:49.230762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d091fbf-3311-4f0d-a1a0-c26b7c473ddd service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:49.231169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d091fbf-3311-4f0d-a1a0-c26b7c473ddd service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:49.231531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d091fbf-3311-4f0d-a1a0-c26b7c473ddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:49.243731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d091fbf-3311-4f0d-a1a0-c26b7c473ddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:49.245412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d091fbf-3311-4f0d-a1a0-c26b7c473ddd service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:49.259582 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 879/1779] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:49.809632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edf2d9-316e-4f11-b449-7a4e6a79d3b5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:49.811080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b4edf2d9-316e-4f11-b449-7a4e6a79d3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:49.823968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edf2d9-316e-4f11-b449-7a4e6a79d3b5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:49.833016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:49.833667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:49.836247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:49.836719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] PATH_INFO: `/v3/projects/474c2d620e224f06b409adf776004cac` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:49.839825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=474c2d620e224f06b409adf776004cac)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:49.842594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:49.950254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474c2d620e224f06b409adf776004cac', 'request_id': 'req-60221b29-d9cc-413a-934c-4bc81b9f4a5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:49.962152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474c2d620e224f06b409adf776004cac', 'request_id': 'req-60221b29-d9cc-413a-934c-4bc81b9f4a5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:49.975373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474c2d620e224f06b409adf776004cac', 'request_id': 'req-60221b29-d9cc-413a-934c-4bc81b9f4a5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:49.987748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d091fbf-3311-4f0d-a1a0-c26b7c473ddd service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:49.988025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] The token cache is being invalidate because project 474c2d620e224f06b409adf776004cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:49.988388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:49.990196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:49.991439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4d091fbf-3311-4f0d-a1a0-c26b7c473ddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:49.992161 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 901/1780] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:49 2022] DELETE /identity/v3/projects/474c2d620e224f06b409adf776004cac => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:49.995951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:49.997359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:50.010991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d091fbf-3311-4f0d-a1a0-c26b7c473ddd service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:50.128438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60221b29-d9cc-413a-934c-4bc81b9f4a5f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:50.138054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:50.138916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:50.139239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:50.139578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] PATH_INFO: `/v3/users/3e09319070164e118879498437350f7f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:50.140839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e09319070164e118879498437350f7f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:50.143015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:50.150475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:50.152249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:50.152542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:50.152803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] PATH_INFO: `/v3/users/b13dec0172eb4a7b9c28d89456d2a6cb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:50.159242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b13dec0172eb4a7b9c28d89456d2a6cb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:50.162334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:50.186958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:50.233409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e09319070164e118879498437350f7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.261087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e09319070164e118879498437350f7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.275525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e09319070164e118879498437350f7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.294581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e09319070164e118879498437350f7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.295819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e09319070164e118879498437350f7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.306128 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 902/1781] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:49 2022] DELETE /identity/v3/users/3e09319070164e118879498437350f7f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:50.328030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b13dec0172eb4a7b9c28d89456d2a6cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.341647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:50.344840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:50.350946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b13dec0172eb4a7b9c28d89456d2a6cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.417520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b13dec0172eb4a7b9c28d89456d2a6cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.448165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b13dec0172eb4a7b9c28d89456d2a6cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.448757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b755092a-3f2e-42ed-bd28-63522e8b9f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b13dec0172eb4a7b9c28d89456d2a6cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.469660 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 880/1782] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:49 2022] DELETE /identity/v3/users/b13dec0172eb4a7b9c28d89456d2a6cb => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:50.475592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cf5c7-ca9c-4547-b2bd-ea1dafe5500b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:50.476144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cf5c7-ca9c-4547-b2bd-ea1dafe5500b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:50.476628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cf5c7-ca9c-4547-b2bd-ea1dafe5500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:50.529401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276beddd-3888-4afe-95e4-ef0f7c6ad743 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:50.566348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c26cf5c7-ca9c-4547-b2bd-ea1dafe5500b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:50.571161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c26cf5c7-ca9c-4547-b2bd-ea1dafe5500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:50.612669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:50.614527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:50.615606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:50.616093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:50.616563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] PATH_INFO: `/v3/projects/12f5d58f9d7f4ab69cb207e25b87a9a8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:50.618247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12f5d58f9d7f4ab69cb207e25b87a9a8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:50.621654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:50.669769 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 881/1783] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:50.688759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cf5c7-ca9c-4547-b2bd-ea1dafe5500b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:50.691510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c26cf5c7-ca9c-4547-b2bd-ea1dafe5500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:50.718834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12f5d58f9d7f4ab69cb207e25b87a9a8', 'request_id': 'req-db4bb5b2-83d1-4180-9df1-9a8912587f83'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.725662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:50.734372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12f5d58f9d7f4ab69cb207e25b87a9a8', 'request_id': 'req-db4bb5b2-83d1-4180-9df1-9a8912587f83'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.756166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12f5d58f9d7f4ab69cb207e25b87a9a8', 'request_id': 'req-db4bb5b2-83d1-4180-9df1-9a8912587f83'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.783757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] The token cache is being invalidate because project 12f5d58f9d7f4ab69cb207e25b87a9a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:50.784411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.787134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:50.790453 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 903/1784] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:50 2022] DELETE /identity/v3/projects/12f5d58f9d7f4ab69cb207e25b87a9a8 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:50.794673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:50.796437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:50.816554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cf5c7-ca9c-4547-b2bd-ea1dafe5500b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:50.968620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f798fa4-c6ad-4a70-8748-bd54ec962634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:50.969421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f798fa4-c6ad-4a70-8748-bd54ec962634 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:50.969690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f798fa4-c6ad-4a70-8748-bd54ec962634 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:50.969958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f798fa4-c6ad-4a70-8748-bd54ec962634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:50.972927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6f798fa4-c6ad-4a70-8748-bd54ec962634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:50.978912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bb5b2-83d1-4180-9df1-9a8912587f83 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:50.985521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:50.986388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:50.986913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:50.987842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] PATH_INFO: `/v3/users/7ea831d2d7ab4ccf898c5236b932db85` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:50.989381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea831d2d7ab4ccf898c5236b932db85)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:50.991842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:51.009147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f798fa4-c6ad-4a70-8748-bd54ec962634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:51.010761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f798fa4-c6ad-4a70-8748-bd54ec962634 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:51.039186 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 882/1785] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:51.076198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea831d2d7ab4ccf898c5236b932db85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:51.102642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea831d2d7ab4ccf898c5236b932db85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:51.112344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea831d2d7ab4ccf898c5236b932db85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:51.123734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea831d2d7ab4ccf898c5236b932db85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:51.124545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4b841e06-986a-47bb-ba0d-5c2269802e65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea831d2d7ab4ccf898c5236b932db85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:51.142098 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 904/1786] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:50 2022] DELETE /identity/v3/users/7ea831d2d7ab4ccf898c5236b932db85 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:51.280300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f798fa4-c6ad-4a70-8748-bd54ec962634 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:51.282283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6f798fa4-c6ad-4a70-8748-bd54ec962634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:51.307998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f798fa4-c6ad-4a70-8748-bd54ec962634 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:51.493676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:51.494346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:51.494671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:51.495136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] PATH_INFO: `/v3/projects/80a8775b9bd74eed81b89c8eb16ed258` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:51.496751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a8775b9bd74eed81b89c8eb16ed258)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:51.500323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:51.603768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182268e-7067-4481-9a50-c0dfd6cf1380 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:51.604047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182268e-7067-4481-9a50-c0dfd6cf1380 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:51.604322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182268e-7067-4481-9a50-c0dfd6cf1380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:51.611389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:51.647798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b182268e-7067-4481-9a50-c0dfd6cf1380 None None] MFA Rules not processed for user `c84b11d2d54b43cbac87115e227ef50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:51.648863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a8775b9bd74eed81b89c8eb16ed258', 'request_id': 'req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:51.649951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b182268e-7067-4481-9a50-c0dfd6cf1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:51.725348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a8775b9bd74eed81b89c8eb16ed258', 'request_id': 'req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:51.760633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a8775b9bd74eed81b89c8eb16ed258', 'request_id': 'req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:51.792725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] The token cache is being invalidate because project 80a8775b9bd74eed81b89c8eb16ed258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:51.793581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:51.794534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d04f8bf1-e8d7-4d1d-bacc-09fb7d225525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:51.795275 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 905/1787] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:48:51.797054 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 883/1788] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:51 2022] DELETE /identity/v3/projects/80a8775b9bd74eed81b89c8eb16ed258 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:51.807523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62b9e8-470d-4ccc-89b8-151750664db1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:51.807523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62b9e8-470d-4ccc-89b8-151750664db1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:51.807969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62b9e8-470d-4ccc-89b8-151750664db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:51.819098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810dafc3-4b86-4551-aeae-b679a571e1e5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:51.819370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810dafc3-4b86-4551-aeae-b679a571e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:51.819636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810dafc3-4b86-4551-aeae-b679a571e1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:51.825419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:51.854546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dc62b9e8-470d-4ccc-89b8-151750664db1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:51.858705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dc62b9e8-470d-4ccc-89b8-151750664db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:51.878229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-810dafc3-4b86-4551-aeae-b679a571e1e5 None None] MFA Rules not processed for user `c5a178b6a74140d4910f77d811febc08`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:51.879563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-810dafc3-4b86-4551-aeae-b679a571e1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:52.008535 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 906/1789] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:52.010670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6daade-b3a6-4bd2-ae99-210ee595d23b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:52.010969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6daade-b3a6-4bd2-ae99-210ee595d23b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:52.011244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6daade-b3a6-4bd2-ae99-210ee595d23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:52.044214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 884/1790] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:52.046645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-be6daade-b3a6-4bd2-ae99-210ee595d23b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:52.047970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-be6daade-b3a6-4bd2-ae99-210ee595d23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:52.062733 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 907/1791] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:52.066522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810dafc3-4b86-4551-aeae-b679a571e1e5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:52.083985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810dafc3-4b86-4551-aeae-b679a571e1e5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:52.101456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd02364-9021-441c-bf63-33821af7a387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:52.102086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd02364-9021-441c-bf63-33821af7a387 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:52.102366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd02364-9021-441c-bf63-33821af7a387 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:52.102637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd02364-9021-441c-bf63-33821af7a387 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:52.103291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd02364-9021-441c-bf63-33821af7a387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:52.103593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd02364-9021-441c-bf63-33821af7a387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:52.105226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd02364-9021-441c-bf63-33821af7a387 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:52.123356 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 885/1792] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:52.124896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6daade-b3a6-4bd2-ae99-210ee595d23b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:52.134823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6daade-b3a6-4bd2-ae99-210ee595d23b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:52.149957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b93f68-6785-4ea1-9313-e25c6cc5509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:52.150669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b93f68-6785-4ea1-9313-e25c6cc5509b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:52.150861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b93f68-6785-4ea1-9313-e25c6cc5509b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:52.151128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b93f68-6785-4ea1-9313-e25c6cc5509b admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:52.151758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b93f68-6785-4ea1-9313-e25c6cc5509b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:52.156437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b93f68-6785-4ea1-9313-e25c6cc5509b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:52.235197 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 908/1793] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:52 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:52.236741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd02364-9021-441c-bf63-33821af7a387 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:52.252296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd02364-9021-441c-bf63-33821af7a387 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:52.384544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2ea7d-e2e3-4ce9-88a8-2ac564cb70ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:52.387523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2ea7d-e2e3-4ce9-88a8-2ac564cb70ba admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:52.388150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2ea7d-e2e3-4ce9-88a8-2ac564cb70ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:52.388603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2ea7d-e2e3-4ce9-88a8-2ac564cb70ba admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:52.389576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2ea7d-e2e3-4ce9-88a8-2ac564cb70ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:52.391928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2ea7d-e2e3-4ce9-88a8-2ac564cb70ba admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:52.437676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b93f68-6785-4ea1-9313-e25c6cc5509b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:52.459086 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 886/1794] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:52 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:52.464579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b93f68-6785-4ea1-9313-e25c6cc5509b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:52.476650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac31d84-c87f-4018-a3dc-c701793e8efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:52.477699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac31d84-c87f-4018-a3dc-c701793e8efe admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:52.478020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac31d84-c87f-4018-a3dc-c701793e8efe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:52.478324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac31d84-c87f-4018-a3dc-c701793e8efe admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:52.479473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac31d84-c87f-4018-a3dc-c701793e8efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:52.481273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac31d84-c87f-4018-a3dc-c701793e8efe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:52.545659 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 909/1795] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:52.549410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2ea7d-e2e3-4ce9-88a8-2ac564cb70ba admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:52.563729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2ea7d-e2e3-4ce9-88a8-2ac564cb70ba admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:52.575735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2278723-e77c-41d1-9431-302b016b1c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:52.576575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2278723-e77c-41d1-9431-302b016b1c4f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:52.577002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2278723-e77c-41d1-9431-302b016b1c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:52.577440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2278723-e77c-41d1-9431-302b016b1c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:52.578270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2278723-e77c-41d1-9431-302b016b1c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:52.582570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2278723-e77c-41d1-9431-302b016b1c4f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:52.602869 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 887/1796] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:48:52.606673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac31d84-c87f-4018-a3dc-c701793e8efe admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:52.621624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac31d84-c87f-4018-a3dc-c701793e8efe admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:52.639115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc23c52-1d0e-45d7-b692-669037ad5e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:52.639954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc23c52-1d0e-45d7-b692-669037ad5e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:52.640306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc23c52-1d0e-45d7-b692-669037ad5e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:52.640648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc23c52-1d0e-45d7-b692-669037ad5e00 admin admin] PATH_INFO: `/v3/projects/4a94a4756e9f452f82d7a91ae2a887d2/users/e7c171cfd1f442289a3e59c54cbd1421/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:52.677279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc23c52-1d0e-45d7-b692-669037ad5e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a94a4756e9f452f82d7a91ae2a887d2, user_id=e7c171cfd1f442289a3e59c54cbd1421, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:52.680078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc23c52-1d0e-45d7-b692-669037ad5e00 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:52.709093 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 910/1797] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:52 2022] PUT /identity/v3/projects/4a94a4756e9f452f82d7a91ae2a887d2/users/e7c171cfd1f442289a3e59c54cbd1421/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:52.716830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf2c46-1794-4b33-96a2-f8f9d70b69dd None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:52.717226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf2c46-1794-4b33-96a2-f8f9d70b69dd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:52.717570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf2c46-1794-4b33-96a2-f8f9d70b69dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:52.749934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc23c52-1d0e-45d7-b692-669037ad5e00 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:52.751839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ccc23c52-1d0e-45d7-b692-669037ad5e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:52.803805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-decf2c46-1794-4b33-96a2-f8f9d70b69dd None None] MFA Rules not processed for user `e7c171cfd1f442289a3e59c54cbd1421`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:52.805124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-decf2c46-1794-4b33-96a2-f8f9d70b69dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:52.840891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:52.904067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc23c52-1d0e-45d7-b692-669037ad5e00 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:52.923661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:52.924365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:52.924754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:52.925935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] PATH_INFO: `/v3/projects/41d79b5940a246eabe32fc5c5309a3d6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:52.928144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=41d79b5940a246eabe32fc5c5309a3d6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:52.930326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:52.955235 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 888/1798] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:52.956778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf2c46-1794-4b33-96a2-f8f9d70b69dd None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:52.959984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-decf2c46-1794-4b33-96a2-f8f9d70b69dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:53.044844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf2c46-1794-4b33-96a2-f8f9d70b69dd None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:53.063776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:53.072553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41d79b5940a246eabe32fc5c5309a3d6', 'request_id': 'req-9d241e37-e977-43fb-9d1c-b402b8504b9c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.103070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41d79b5940a246eabe32fc5c5309a3d6', 'request_id': 'req-9d241e37-e977-43fb-9d1c-b402b8504b9c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.125172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41d79b5940a246eabe32fc5c5309a3d6', 'request_id': 'req-9d241e37-e977-43fb-9d1c-b402b8504b9c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.164822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] The token cache is being invalidate because project 41d79b5940a246eabe32fc5c5309a3d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:53.165070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.173906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.185448 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 911/1799] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:52 2022] DELETE /identity/v3/projects/41d79b5940a246eabe32fc5c5309a3d6 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:48:53.188102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:53.198125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:53.228881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:53.229555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:53.229828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:53.230093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] PATH_INFO: `/v3/projects/31caa50628804cd597330477886dc146` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:53.234465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31caa50628804cd597330477886dc146)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:53.236432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:53.364484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d241e37-e977-43fb-9d1c-b402b8504b9c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:53.407426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31caa50628804cd597330477886dc146', 'request_id': 'req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.428660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31caa50628804cd597330477886dc146', 'request_id': 'req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.450699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31caa50628804cd597330477886dc146', 'request_id': 'req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.496455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] The token cache is being invalidate because project 31caa50628804cd597330477886dc146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:53.496984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.501219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.503989 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 889/1800] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:52 2022] DELETE /identity/v3/projects/31caa50628804cd597330477886dc146 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:53.508325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:53.513349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:53.530980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9889f7e-c8ad-4c5a-8309-e145b0ecc868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:53.540301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb004d-89bb-4baa-88ed-0f2e2acd0ef5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:53.541703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9889f7e-c8ad-4c5a-8309-e145b0ecc868 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:53.542106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9889f7e-c8ad-4c5a-8309-e145b0ecc868 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:53.544087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9889f7e-c8ad-4c5a-8309-e145b0ecc868 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:53.544087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9889f7e-c8ad-4c5a-8309-e145b0ecc868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:53.552129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:53.553146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:53.553565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:53.554377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] PATH_INFO: `/v3/users/a1e9c90d8c1549aa83893ed2476c3a6d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:53.558265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1e9c90d8c1549aa83893ed2476c3a6d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:53.558669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9889f7e-c8ad-4c5a-8309-e145b0ecc868 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:53.570940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:53.655259 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 912/1801] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:53 2022] POST /identity/v3/projects => generated 369 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:53.656816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926697c-a853-4d5b-8a83-5b4fef86955d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:53.657136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926697c-a853-4d5b-8a83-5b4fef86955d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:53.657402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926697c-a853-4d5b-8a83-5b4fef86955d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:53.707634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d926697c-a853-4d5b-8a83-5b4fef86955d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:53.709694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d926697c-a853-4d5b-8a83-5b4fef86955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:53.733360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1e9c90d8c1549aa83893ed2476c3a6d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.737303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:53.753965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1e9c90d8c1549aa83893ed2476c3a6d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.784478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1e9c90d8c1549aa83893ed2476c3a6d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.813759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1e9c90d8c1549aa83893ed2476c3a6d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.814378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1e9c90d8c1549aa83893ed2476c3a6d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:53.831490 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 890/1802] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:53 2022] DELETE /identity/v3/users/a1e9c90d8c1549aa83893ed2476c3a6d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:53.831490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:53.832600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:53.882782 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 913/1803] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:53.884434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fc091-6535-4fbd-aafb-b625aacde3d0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:53.884711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fc091-6535-4fbd-aafb-b625aacde3d0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:53.884977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fc091-6535-4fbd-aafb-b625aacde3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:53.900604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:53.919450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f77fc091-6535-4fbd-aafb-b625aacde3d0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:53.923986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f77fc091-6535-4fbd-aafb-b625aacde3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:54.039716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ab86d-b0dd-47b1-8604-d3d1a732429e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:54.064748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b97e6-cefc-4351-9825-45842120f8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:54.065730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b97e6-cefc-4351-9825-45842120f8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:54.066162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b97e6-cefc-4351-9825-45842120f8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:54.066613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b97e6-cefc-4351-9825-45842120f8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:54.068009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b97e6-cefc-4351-9825-45842120f8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:54.072677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b97e6-cefc-4351-9825-45842120f8d5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:54.112919 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 914/1804] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:48:54.118663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fc091-6535-4fbd-aafb-b625aacde3d0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:54.143291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fc091-6535-4fbd-aafb-b625aacde3d0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:54.148982 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 891/1805] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:53 2022] POST /identity/v3/users => generated 291 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:54.153846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b97e6-cefc-4351-9825-45842120f8d5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:54.159291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea7862-ad64-4ed0-bdf9-85aaebbd1999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:54.159968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7862-ad64-4ed0-bdf9-85aaebbd1999 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:54.162870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7862-ad64-4ed0-bdf9-85aaebbd1999 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:54.166328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7862-ad64-4ed0-bdf9-85aaebbd1999 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:54.167756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea7862-ad64-4ed0-bdf9-85aaebbd1999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:54.168068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea7862-ad64-4ed0-bdf9-85aaebbd1999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:54.172773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea7862-ad64-4ed0-bdf9-85aaebbd1999 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:54.187441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b97e6-cefc-4351-9825-45842120f8d5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:54.198645 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 915/1806] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:54.199723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea7862-ad64-4ed0-bdf9-85aaebbd1999 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:54.200555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142009f-3a81-49b7-9166-29503fd9ae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:54.202125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142009f-3a81-49b7-9166-29503fd9ae96 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:54.202672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142009f-3a81-49b7-9166-29503fd9ae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:54.203279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142009f-3a81-49b7-9166-29503fd9ae96 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:54.204163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142009f-3a81-49b7-9166-29503fd9ae96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:54.212531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142009f-3a81-49b7-9166-29503fd9ae96 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:54.240151 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 892/1807] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:54.244421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea7862-ad64-4ed0-bdf9-85aaebbd1999 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:54.245629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142009f-3a81-49b7-9166-29503fd9ae96 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:54.262368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67acc0-be5a-4f11-83ea-539d754e3c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:54.263010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67acc0-be5a-4f11-83ea-539d754e3c67 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:54.263266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67acc0-be5a-4f11-83ea-539d754e3c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:54.263533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67acc0-be5a-4f11-83ea-539d754e3c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:54.264226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67acc0-be5a-4f11-83ea-539d754e3c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:54.265889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142009f-3a81-49b7-9166-29503fd9ae96 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:54.267084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67acc0-be5a-4f11-83ea-539d754e3c67 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:54.282444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768d88b-a926-4e1e-858c-998718620662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:54.283687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768d88b-a926-4e1e-858c-998718620662 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:54.284199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768d88b-a926-4e1e-858c-998718620662 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:54.284678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768d88b-a926-4e1e-858c-998718620662 admin admin] PATH_INFO: `/v3/projects/c33bb86097044d40b41078392b87160c/users/d3094a373cdd414ca66485d33b8e106e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:54.309614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768d88b-a926-4e1e-858c-998718620662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c33bb86097044d40b41078392b87160c, user_id=d3094a373cdd414ca66485d33b8e106e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:54.312877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768d88b-a926-4e1e-858c-998718620662 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:54.321547 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 916/1808] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:54 2022] POST /identity/v3/projects => generated 399 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:54.322407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67acc0-be5a-4f11-83ea-539d754e3c67 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:54.354117 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 893/1809] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:54 2022] PUT /identity/v3/projects/c33bb86097044d40b41078392b87160c/users/d3094a373cdd414ca66485d33b8e106e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:54.358194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768d88b-a926-4e1e-858c-998718620662 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:54.483836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67acc0-be5a-4f11-83ea-539d754e3c67 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:54.526081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768d88b-a926-4e1e-858c-998718620662 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:54.532899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c5617-28b7-4009-8bdf-04e7dbb01dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:54.534176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c5617-28b7-4009-8bdf-04e7dbb01dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:54.534635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c5617-28b7-4009-8bdf-04e7dbb01dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:54.535440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c5617-28b7-4009-8bdf-04e7dbb01dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:54.536709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c5617-28b7-4009-8bdf-04e7dbb01dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:54.539588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c5617-28b7-4009-8bdf-04e7dbb01dc8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:54.557270 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 894/1810] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:54.560933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c5617-28b7-4009-8bdf-04e7dbb01dc8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:54.574593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c5617-28b7-4009-8bdf-04e7dbb01dc8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:54.580880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd752d3-6a49-485f-8b51-0f524aed4d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:54.581546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd752d3-6a49-485f-8b51-0f524aed4d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:54.581890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd752d3-6a49-485f-8b51-0f524aed4d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:54.582283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd752d3-6a49-485f-8b51-0f524aed4d2e admin admin] PATH_INFO: `/v3/projects/c33bb86097044d40b41078392b87160c/users/d3094a373cdd414ca66485d33b8e106e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:54.584607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd752d3-6a49-485f-8b51-0f524aed4d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c33bb86097044d40b41078392b87160c, user_id=d3094a373cdd414ca66485d33b8e106e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:54.588109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd752d3-6a49-485f-8b51-0f524aed4d2e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:54.625091 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 895/1811] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:54 2022] PUT /identity/v3/projects/c33bb86097044d40b41078392b87160c/users/d3094a373cdd414ca66485d33b8e106e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:54.629656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4d32d-fa67-4e2b-aec2-83654f33337b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:54.630188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4d32d-fa67-4e2b-aec2-83654f33337b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:54.630624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4d32d-fa67-4e2b-aec2-83654f33337b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:54.641258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:54.703372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c8a4d32d-fa67-4e2b-aec2-83654f33337b None None] MFA Rules not processed for user `d3094a373cdd414ca66485d33b8e106e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:54.704765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c8a4d32d-fa67-4e2b-aec2-83654f33337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:54.729931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3ce9c-deb0-4d07-b039-8b1c7dd3f12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:54.730717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3ce9c-deb0-4d07-b039-8b1c7dd3f12c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:54.731154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3ce9c-deb0-4d07-b039-8b1c7dd3f12c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:54.731574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3ce9c-deb0-4d07-b039-8b1c7dd3f12c admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:54.732619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3ce9c-deb0-4d07-b039-8b1c7dd3f12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:54.734814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3ce9c-deb0-4d07-b039-8b1c7dd3f12c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:54.807889 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 917/1812] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:54 2022] POST /identity/v3/users => generated 300 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:54.808982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3ce9c-deb0-4d07-b039-8b1c7dd3f12c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:54.838839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3ce9c-deb0-4d07-b039-8b1c7dd3f12c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:54.845727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:54.886697 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 896/1813] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:54.968011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb46d5b-6014-467c-8187-c3f7f0ca9b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:54.970359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb46d5b-6014-467c-8187-c3f7f0ca9b4d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:54.970821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb46d5b-6014-467c-8187-c3f7f0ca9b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:54.971271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb46d5b-6014-467c-8187-c3f7f0ca9b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:54.971945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb46d5b-6014-467c-8187-c3f7f0ca9b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:54.974170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb46d5b-6014-467c-8187-c3f7f0ca9b4d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:55.010274 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 918/1814] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:55.012835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4d32d-fa67-4e2b-aec2-83654f33337b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:55.042091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4d32d-fa67-4e2b-aec2-83654f33337b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:55.044771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb46d5b-6014-467c-8187-c3f7f0ca9b4d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:55.047167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cfb46d5b-6014-467c-8187-c3f7f0ca9b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:55.058801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d39f7e-bf23-4527-95a2-5426e56d1d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:55.059450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d39f7e-bf23-4527-95a2-5426e56d1d9d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:55.059723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d39f7e-bf23-4527-95a2-5426e56d1d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:55.059987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d39f7e-bf23-4527-95a2-5426e56d1d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:55.060627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d39f7e-bf23-4527-95a2-5426e56d1d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:55.063972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d39f7e-bf23-4527-95a2-5426e56d1d9d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:55.096524 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 897/1815] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:55.097687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d39f7e-bf23-4527-95a2-5426e56d1d9d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:55.127288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d39f7e-bf23-4527-95a2-5426e56d1d9d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:55.133780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf9e66-beb1-40cf-b692-d61b54f175f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:55.134653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf9e66-beb1-40cf-b692-d61b54f175f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:55.134954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf9e66-beb1-40cf-b692-d61b54f175f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:55.135282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf9e66-beb1-40cf-b692-d61b54f175f3 admin admin] PATH_INFO: `/v3/projects/d6c560f9235945a7af176a454b799ea2/users/f050455c8e5247cfb77c1f9044efc534/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:55.157167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf9e66-beb1-40cf-b692-d61b54f175f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c560f9235945a7af176a454b799ea2, user_id=f050455c8e5247cfb77c1f9044efc534, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:55.159032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb46d5b-6014-467c-8187-c3f7f0ca9b4d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:55.159926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf9e66-beb1-40cf-b692-d61b54f175f3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:55.167336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9648a-8f1f-4eed-8918-ddf39bcb3083 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:55.168131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9648a-8f1f-4eed-8918-ddf39bcb3083 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:55.168811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9648a-8f1f-4eed-8918-ddf39bcb3083 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:55.169072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9648a-8f1f-4eed-8918-ddf39bcb3083 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:55.181099 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 898/1816] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:55 2022] PUT /identity/v3/projects/d6c560f9235945a7af176a454b799ea2/users/f050455c8e5247cfb77c1f9044efc534/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:55.187244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9648a-8f1f-4eed-8918-ddf39bcb3083 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:55.195327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9648a-8f1f-4eed-8918-ddf39bcb3083 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:55.199093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd9888-edeb-4472-ad0e-4101ba1b3f78 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:55.199354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd9888-edeb-4472-ad0e-4101ba1b3f78 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:55.199633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd9888-edeb-4472-ad0e-4101ba1b3f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:55.210646 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 919/1817] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:55.255086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b3cd9888-edeb-4472-ad0e-4101ba1b3f78 None None] MFA Rules not processed for user `f050455c8e5247cfb77c1f9044efc534`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:55.256360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b3cd9888-edeb-4472-ad0e-4101ba1b3f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:55.323839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9648a-8f1f-4eed-8918-ddf39bcb3083 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:55.345249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9648a-8f1f-4eed-8918-ddf39bcb3083 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:55.413282 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 899/1818] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:48:55.418231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd9888-edeb-4472-ad0e-4101ba1b3f78 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:55.441020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:55.441737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:55.442011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:55.442285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] PATH_INFO: `/v3/users/e7c171cfd1f442289a3e59c54cbd1421` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:55.443427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c171cfd1f442289a3e59c54cbd1421)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:55.446159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:55.475400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:55.502750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd9888-edeb-4472-ad0e-4101ba1b3f78 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:55.509709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97665ce9-e327-45d3-a31a-8c66d8d2f18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:55.510387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97665ce9-e327-45d3-a31a-8c66d8d2f18c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:55.510667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97665ce9-e327-45d3-a31a-8c66d8d2f18c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:55.510986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97665ce9-e327-45d3-a31a-8c66d8d2f18c admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:55.511753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97665ce9-e327-45d3-a31a-8c66d8d2f18c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:55.514800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97665ce9-e327-45d3-a31a-8c66d8d2f18c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:55.516435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:55.544250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c171cfd1f442289a3e59c54cbd1421'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:55.552413 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 900/1819] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:55 2022] POST /identity/v3/projects => generated 399 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:55.557146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97665ce9-e327-45d3-a31a-8c66d8d2f18c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:55.567075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c171cfd1f442289a3e59c54cbd1421'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:55.585623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c171cfd1f442289a3e59c54cbd1421'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:55.596420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c171cfd1f442289a3e59c54cbd1421'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:55.600012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5004599-085c-4e35-b722-ae030c243540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c171cfd1f442289a3e59c54cbd1421'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:55.616150 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 920/1820] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:55 2022] DELETE /identity/v3/users/e7c171cfd1f442289a3e59c54cbd1421 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:55.620919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f2abc-538c-431a-9257-5dfb3dbeda71 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:55.621293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f2abc-538c-431a-9257-5dfb3dbeda71 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:55.621801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f2abc-538c-431a-9257-5dfb3dbeda71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:55.661884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97665ce9-e327-45d3-a31a-8c66d8d2f18c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:55.686471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bf3f2abc-538c-431a-9257-5dfb3dbeda71 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:55.692843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bf3f2abc-538c-431a-9257-5dfb3dbeda71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:55.780509 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 921/1821] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:55.792003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f2abc-538c-431a-9257-5dfb3dbeda71 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:55.794959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bf3f2abc-538c-431a-9257-5dfb3dbeda71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:55.841668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15765377-92a8-4534-b320-61ac5c5025b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:55.842397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15765377-92a8-4534-b320-61ac5c5025b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:55.842729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15765377-92a8-4534-b320-61ac5c5025b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:55.843010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15765377-92a8-4534-b320-61ac5c5025b8 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:55.843669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15765377-92a8-4534-b320-61ac5c5025b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:55.849399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15765377-92a8-4534-b320-61ac5c5025b8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:55.889607 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 901/1822] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:55 2022] POST /identity/v3/users => generated 306 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:55.892376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f2abc-538c-431a-9257-5dfb3dbeda71 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:55.895904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15765377-92a8-4534-b320-61ac5c5025b8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:55.898476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfde6d8-8ce2-46a1-8dcd-2dab13d9a338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:55.899341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfde6d8-8ce2-46a1-8dcd-2dab13d9a338 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:55.899718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfde6d8-8ce2-46a1-8dcd-2dab13d9a338 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:55.900084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfde6d8-8ce2-46a1-8dcd-2dab13d9a338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:55.900759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-15765377-92a8-4534-b320-61ac5c5025b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:55.914785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfde6d8-8ce2-46a1-8dcd-2dab13d9a338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:55.919766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfde6d8-8ce2-46a1-8dcd-2dab13d9a338 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:55.932592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15765377-92a8-4534-b320-61ac5c5025b8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:55.950930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23c703-3725-488f-81d5-c14c70f2337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:55.952697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23c703-3725-488f-81d5-c14c70f2337b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:55.953406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23c703-3725-488f-81d5-c14c70f2337b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:55.953876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23c703-3725-488f-81d5-c14c70f2337b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:55.954897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23c703-3725-488f-81d5-c14c70f2337b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:55.958022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23c703-3725-488f-81d5-c14c70f2337b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:55.966844 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 922/1823] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:55.977476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfde6d8-8ce2-46a1-8dcd-2dab13d9a338 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:55.980277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0bfde6d8-8ce2-46a1-8dcd-2dab13d9a338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:55.986678 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 902/1824] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:55.994067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23c703-3725-488f-81d5-c14c70f2337b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:55.998002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfde6d8-8ce2-46a1-8dcd-2dab13d9a338 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:56.003665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:56.004704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.005065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.005519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] PATH_INFO: `/v3/projects/1b4645ec7c584e549cdc3cbf9965a11d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.006862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b4645ec7c584e549cdc3cbf9965a11d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:56.010063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:56.028641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23c703-3725-488f-81d5-c14c70f2337b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:56.047927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6123605-b984-4fa7-8cd7-3989676f5d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:56.048630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6123605-b984-4fa7-8cd7-3989676f5d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.049573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6123605-b984-4fa7-8cd7-3989676f5d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.049830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6123605-b984-4fa7-8cd7-3989676f5d91 admin admin] PATH_INFO: `/v3/projects/5a194c610fdf40dd849fc78fc4aa07f2/users/c123f19c891d4ad6b8844d785c2c06d3/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.071998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:56.080535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b4645ec7c584e549cdc3cbf9965a11d', 'request_id': 'req-d4ec8d58-441e-44e4-bb80-e2372d4e2373'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:56.090651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6123605-b984-4fa7-8cd7-3989676f5d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a194c610fdf40dd849fc78fc4aa07f2, user_id=c123f19c891d4ad6b8844d785c2c06d3, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:56.094778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6123605-b984-4fa7-8cd7-3989676f5d91 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:56.106898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b4645ec7c584e549cdc3cbf9965a11d', 'request_id': 'req-d4ec8d58-441e-44e4-bb80-e2372d4e2373'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:56.116265 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 903/1825] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:55 2022] PUT /identity/v3/projects/5a194c610fdf40dd849fc78fc4aa07f2/users/c123f19c891d4ad6b8844d785c2c06d3/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:56.126461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6123605-b984-4fa7-8cd7-3989676f5d91 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:56.149919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b4645ec7c584e549cdc3cbf9965a11d', 'request_id': 'req-d4ec8d58-441e-44e4-bb80-e2372d4e2373'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:56.183348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] The token cache is being invalidate because project 1b4645ec7c584e549cdc3cbf9965a11d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:56.183961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:56.187084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4ec8d58-441e-44e4-bb80-e2372d4e2373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:56.198256 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 923/1826] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:55 2022] DELETE /identity/v3/projects/1b4645ec7c584e549cdc3cbf9965a11d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:56.202972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384f5f-a7ad-4a33-a54a-0c9124579843 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.203262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384f5f-a7ad-4a33-a54a-0c9124579843 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.203616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384f5f-a7ad-4a33-a54a-0c9124579843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.251542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e1384f5f-a7ad-4a33-a54a-0c9124579843 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:56.252813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e1384f5f-a7ad-4a33-a54a-0c9124579843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:56.259689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:56.304507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6123605-b984-4fa7-8cd7-3989676f5d91 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:56.310648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea03672-8cc3-49e1-84ac-c351af7e44ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:56.311371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea03672-8cc3-49e1-84ac-c351af7e44ac admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.311715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea03672-8cc3-49e1-84ac-c351af7e44ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.312055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea03672-8cc3-49e1-84ac-c351af7e44ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.314705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea03672-8cc3-49e1-84ac-c351af7e44ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:56.318229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea03672-8cc3-49e1-84ac-c351af7e44ac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:56.334797 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 904/1827] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:56.334957 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 924/1828] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:56.342935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249c725-c2c6-4a56-b022-20bec7b15c85 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.343808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249c725-c2c6-4a56-b022-20bec7b15c85 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.343808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249c725-c2c6-4a56-b022-20bec7b15c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.344003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea03672-8cc3-49e1-84ac-c351af7e44ac admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:56.352449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1ea03672-8cc3-49e1-84ac-c351af7e44ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:56.374008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1249c725-c2c6-4a56-b022-20bec7b15c85 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:56.375992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1249c725-c2c6-4a56-b022-20bec7b15c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:56.386046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea03672-8cc3-49e1-84ac-c351af7e44ac admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:56.390018 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 925/1829] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:56.393808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249c725-c2c6-4a56-b022-20bec7b15c85 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:56.396236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a1450-13f3-40bf-a631-54ecfface48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:56.397204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a1450-13f3-40bf-a631-54ecfface48a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.397603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a1450-13f3-40bf-a631-54ecfface48a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.397976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a1450-13f3-40bf-a631-54ecfface48a admin admin] PATH_INFO: `/v3/projects/5a194c610fdf40dd849fc78fc4aa07f2/users/c123f19c891d4ad6b8844d785c2c06d3/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.400436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a1450-13f3-40bf-a631-54ecfface48a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a194c610fdf40dd849fc78fc4aa07f2, user_id=c123f19c891d4ad6b8844d785c2c06d3, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:56.404472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a1450-13f3-40bf-a631-54ecfface48a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:56.408391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249c725-c2c6-4a56-b022-20bec7b15c85 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:56.418366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50274f-31cd-4678-b2fe-da7d8082b80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:56.419290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50274f-31cd-4678-b2fe-da7d8082b80d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.419731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50274f-31cd-4678-b2fe-da7d8082b80d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.420151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50274f-31cd-4678-b2fe-da7d8082b80d admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.421624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50274f-31cd-4678-b2fe-da7d8082b80d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:56.422319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50274f-31cd-4678-b2fe-da7d8082b80d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:56.425151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50274f-31cd-4678-b2fe-da7d8082b80d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:56.430748 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 905/1830] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:56 2022] PUT /identity/v3/projects/5a194c610fdf40dd849fc78fc4aa07f2/users/c123f19c891d4ad6b8844d785c2c06d3/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:56.432248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623147f6-604a-4fc9-ae36-aa6e14dd22a4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.432529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623147f6-604a-4fc9-ae36-aa6e14dd22a4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.433539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623147f6-604a-4fc9-ae36-aa6e14dd22a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.445184 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 926/1831] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:56.447021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50274f-31cd-4678-b2fe-da7d8082b80d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:56.460408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50274f-31cd-4678-b2fe-da7d8082b80d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:56.485072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-623147f6-604a-4fc9-ae36-aa6e14dd22a4 None None] MFA Rules not processed for user `c123f19c891d4ad6b8844d785c2c06d3`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:56.486340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-623147f6-604a-4fc9-ae36-aa6e14dd22a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:56.555457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca3143-6cdd-498f-b195-956409e10408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:56.556331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3143-6cdd-498f-b195-956409e10408 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.556611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3143-6cdd-498f-b195-956409e10408 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.557101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3143-6cdd-498f-b195-956409e10408 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.558012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca3143-6cdd-498f-b195-956409e10408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:56.560201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca3143-6cdd-498f-b195-956409e10408 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:56.578650 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 906/1832] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:56.584360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623147f6-604a-4fc9-ae36-aa6e14dd22a4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:56.586301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-623147f6-604a-4fc9-ae36-aa6e14dd22a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:56.594741 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 927/1833] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:56 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:56.595858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca3143-6cdd-498f-b195-956409e10408 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:56.598923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7cca3143-6cdd-498f-b195-956409e10408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:56.608260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623147f6-604a-4fc9-ae36-aa6e14dd22a4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:56.666441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:56.697146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:56.697834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.698484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.698810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] PATH_INFO: `/v3/projects/4a94a4756e9f452f82d7a91ae2a887d2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.701118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a94a4756e9f452f82d7a91ae2a887d2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:56.703819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:56.709422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca3143-6cdd-498f-b195-956409e10408 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:56.717286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c721b8b-ddf4-41a6-9b46-f5dd64f3a065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:56.718019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c721b8b-ddf4-41a6-9b46-f5dd64f3a065 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.718366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c721b8b-ddf4-41a6-9b46-f5dd64f3a065 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.718745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c721b8b-ddf4-41a6-9b46-f5dd64f3a065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.720659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9c721b8b-ddf4-41a6-9b46-f5dd64f3a065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:56.752790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:56.765293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a94a4756e9f452f82d7a91ae2a887d2', 'request_id': 'req-ae3b3062-c076-42d2-95c2-0839d1a9a177'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:56.776958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a94a4756e9f452f82d7a91ae2a887d2', 'request_id': 'req-ae3b3062-c076-42d2-95c2-0839d1a9a177'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:56.785393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a94a4756e9f452f82d7a91ae2a887d2', 'request_id': 'req-ae3b3062-c076-42d2-95c2-0839d1a9a177'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:56.794117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c721b8b-ddf4-41a6-9b46-f5dd64f3a065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:56.795890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c721b8b-ddf4-41a6-9b46-f5dd64f3a065 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:56.801104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] The token cache is being invalidate because project 4a94a4756e9f452f82d7a91ae2a887d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:56.801459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:56.801882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:56.803428 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 907/1834] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:56 2022] DELETE /identity/v3/projects/4a94a4756e9f452f82d7a91ae2a887d2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:56.804230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:56.805823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:56.821249 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 928/1835] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:56 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:56.823157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c721b8b-ddf4-41a6-9b46-f5dd64f3a065 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:56.853443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c721b8b-ddf4-41a6-9b46-f5dd64f3a065 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:56.896262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b3062-c076-42d2-95c2-0839d1a9a177 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:56.901450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5553a-d981-4283-af30-9012390ca704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:56.902131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5553a-d981-4283-af30-9012390ca704 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.902616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5553a-d981-4283-af30-9012390ca704 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.902957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5553a-d981-4283-af30-9012390ca704 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.903630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5553a-d981-4283-af30-9012390ca704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:56.905987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5553a-d981-4283-af30-9012390ca704 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:56.935931 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 908/1836] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:56 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:56.937115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5553a-d981-4283-af30-9012390ca704 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:56.939379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-24a5553a-d981-4283-af30-9012390ca704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:56.943671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:56.944721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:56.944835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:56.945208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] PATH_INFO: `/v3/users/d3094a373cdd414ca66485d33b8e106e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:56.947531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3094a373cdd414ca66485d33b8e106e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:56.949355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:57.001826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3094a373cdd414ca66485d33b8e106e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.017847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3094a373cdd414ca66485d33b8e106e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.027071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5553a-d981-4283-af30-9012390ca704 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:57.027261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3094a373cdd414ca66485d33b8e106e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.032894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66289c3-b61e-4116-9a56-67191767c455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:57.033557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66289c3-b61e-4116-9a56-67191767c455 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:57.033805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66289c3-b61e-4116-9a56-67191767c455 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:57.034076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66289c3-b61e-4116-9a56-67191767c455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:57.034745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:57.036713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3094a373cdd414ca66485d33b8e106e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.036829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b66289c3-b61e-4116-9a56-67191767c455 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:57.037431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3094a373cdd414ca66485d33b8e106e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.045286 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 929/1837] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:56 2022] DELETE /identity/v3/users/d3094a373cdd414ca66485d33b8e106e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:57.046060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:57.048846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:57.152488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66438e6b-f722-4516-91b6-384949f73370 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:57.162268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:57.163667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7328d-2514-485b-8428-044fef3c297e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:57.164795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7328d-2514-485b-8428-044fef3c297e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:57.165099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7328d-2514-485b-8428-044fef3c297e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:57.165807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7328d-2514-485b-8428-044fef3c297e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:57.166642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7328d-2514-485b-8428-044fef3c297e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:57.168736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7328d-2514-485b-8428-044fef3c297e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:57.191001 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 930/1838] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:57.197170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7328d-2514-485b-8428-044fef3c297e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:57.197332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66289c3-b61e-4116-9a56-67191767c455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:57.197332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66289c3-b61e-4116-9a56-67191767c455 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:57.205183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7328d-2514-485b-8428-044fef3c297e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:57.217499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a7ac6-c1dc-4cea-af4a-2aa96a1db6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:57.218138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a7ac6-c1dc-4cea-af4a-2aa96a1db6bf admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:57.218417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a7ac6-c1dc-4cea-af4a-2aa96a1db6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:57.218732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a7ac6-c1dc-4cea-af4a-2aa96a1db6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:57.219309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a7ac6-c1dc-4cea-af4a-2aa96a1db6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:57.220945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a7ac6-c1dc-4cea-af4a-2aa96a1db6bf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:57.222233 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 909/1839] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:56 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:57.245588 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 931/1840] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:57.249354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66289c3-b61e-4116-9a56-67191767c455 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:57.263402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66289c3-b61e-4116-9a56-67191767c455 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:57.268817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01270a-6d0d-4a8f-8cc6-49275784c5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:57.270783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01270a-6d0d-4a8f-8cc6-49275784c5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:57.271277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01270a-6d0d-4a8f-8cc6-49275784c5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:57.271821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01270a-6d0d-4a8f-8cc6-49275784c5d1 admin admin] PATH_INFO: `/v3/projects/90478ad396024d92baf1f96159d9d7e0/users/3493560ca6c9451a940c11eac852ef21/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:57.295530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01270a-6d0d-4a8f-8cc6-49275784c5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90478ad396024d92baf1f96159d9d7e0, user_id=3493560ca6c9451a940c11eac852ef21, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:57.299161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01270a-6d0d-4a8f-8cc6-49275784c5d1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:57.346790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d869d29-ee7e-4332-84db-c922a67f6932 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:57.351362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d869d29-ee7e-4332-84db-c922a67f6932 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:57.351755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d869d29-ee7e-4332-84db-c922a67f6932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:57.354046 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 910/1841] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:57 2022] PUT /identity/v3/projects/90478ad396024d92baf1f96159d9d7e0/users/3493560ca6c9451a940c11eac852ef21/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:57.401508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7d869d29-ee7e-4332-84db-c922a67f6932 None None] MFA Rules not processed for user `3493560ca6c9451a940c11eac852ef21`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:57.402711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7d869d29-ee7e-4332-84db-c922a67f6932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:57.465821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01270a-6d0d-4a8f-8cc6-49275784c5d1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:57.468048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ed01270a-6d0d-4a8f-8cc6-49275784c5d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:57.542438 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 932/1842] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:57.547846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d869d29-ee7e-4332-84db-c922a67f6932 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:57.574776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01270a-6d0d-4a8f-8cc6-49275784c5d1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:57.580924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:57.581488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:57.581835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:57.582208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] PATH_INFO: `/v3/users/f050455c8e5247cfb77c1f9044efc534` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:57.583570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f050455c8e5247cfb77c1f9044efc534)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:57.587653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:57.638678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d869d29-ee7e-4332-84db-c922a67f6932 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:57.645725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687525b-e790-4275-966c-f2567c4c4b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:57.646345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687525b-e790-4275-966c-f2567c4c4b13 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:57.646622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687525b-e790-4275-966c-f2567c4c4b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:57.646908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687525b-e790-4275-966c-f2567c4c4b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:57.647621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687525b-e790-4275-966c-f2567c4c4b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:57.650503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687525b-e790-4275-966c-f2567c4c4b13 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:57.650877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f050455c8e5247cfb77c1f9044efc534'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.660708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f050455c8e5247cfb77c1f9044efc534'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.677758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:57.680350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f050455c8e5247cfb77c1f9044efc534'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.694277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f050455c8e5247cfb77c1f9044efc534'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.695643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f050455c8e5247cfb77c1f9044efc534'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.702340 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 933/1843] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:57 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:48:57.702724 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 911/1844] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:48:57 2022] DELETE /identity/v3/users/f050455c8e5247cfb77c1f9044efc534 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:57.703590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:57.705161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:57.706362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687525b-e790-4275-966c-f2567c4c4b13 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:57.708183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9687525b-e790-4275-966c-f2567c4c4b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:57.760958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:57.787131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0dab4-fc67-4c97-b8f5-f37195e18e39 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:57.796720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e5f9d-169f-4ad4-a59e-7d1ac6597899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:57.799189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e5f9d-169f-4ad4-a59e-7d1ac6597899 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:57.799550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e5f9d-169f-4ad4-a59e-7d1ac6597899 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:57.801573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e5f9d-169f-4ad4-a59e-7d1ac6597899 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:57.802830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e5f9d-169f-4ad4-a59e-7d1ac6597899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:57.804830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e5f9d-169f-4ad4-a59e-7d1ac6597899 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:57.805186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687525b-e790-4275-966c-f2567c4c4b13 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:57.827605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:57.828414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:57.828693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:57.829062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] PATH_INFO: `/v3/projects/c33bb86097044d40b41078392b87160c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:57.833102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c33bb86097044d40b41078392b87160c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:57.835069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:57.841390 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 912/1845] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:57 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:57.843909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36655ad-57bb-4883-b5a8-aa79cd68487f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:57.844327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36655ad-57bb-4883-b5a8-aa79cd68487f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:57.844646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36655ad-57bb-4883-b5a8-aa79cd68487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:57.875671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d36655ad-57bb-4883-b5a8-aa79cd68487f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:57.877937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d36655ad-57bb-4883-b5a8-aa79cd68487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:57.891501 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 913/1846] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:57.893119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36655ad-57bb-4883-b5a8-aa79cd68487f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:57.927021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36655ad-57bb-4883-b5a8-aa79cd68487f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:57.937191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c33bb86097044d40b41078392b87160c', 'request_id': 'req-455cb901-ed81-4961-8ed1-c9f61793cf2c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.959054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c33bb86097044d40b41078392b87160c', 'request_id': 'req-455cb901-ed81-4961-8ed1-c9f61793cf2c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.974373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c33bb86097044d40b41078392b87160c', 'request_id': 'req-455cb901-ed81-4961-8ed1-c9f61793cf2c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:57.997254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] The token cache is being invalidate because project c33bb86097044d40b41078392b87160c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:57.997645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:58.002217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:58.003204 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 934/1847] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:57 2022] DELETE /identity/v3/projects/c33bb86097044d40b41078392b87160c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:58.007245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:58.008111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:58.041937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ccbf6-4b4a-41d9-a7cb-e05887f04851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:58.043093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ccbf6-4b4a-41d9-a7cb-e05887f04851 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.043524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ccbf6-4b4a-41d9-a7cb-e05887f04851 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.043939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ccbf6-4b4a-41d9-a7cb-e05887f04851 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.044750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ccbf6-4b4a-41d9-a7cb-e05887f04851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:58.047170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ccbf6-4b4a-41d9-a7cb-e05887f04851 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:58.066531 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 914/1848] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:58.069345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1c164-f1f1-4bda-a026-daaf0797f429 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.070085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1c164-f1f1-4bda-a026-daaf0797f429 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.070683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1c164-f1f1-4bda-a026-daaf0797f429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.104140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-40d1c164-f1f1-4bda-a026-daaf0797f429 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:58.106096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-40d1c164-f1f1-4bda-a026-daaf0797f429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:58.127219 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 915/1849] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:58.128454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1c164-f1f1-4bda-a026-daaf0797f429 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:58.134947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cb901-ed81-4961-8ed1-c9f61793cf2c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:58.136315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-40d1c164-f1f1-4bda-a026-daaf0797f429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:58.148717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683f58c-9fd2-48b0-b430-8802f6fd4846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:58.149240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683f58c-9fd2-48b0-b430-8802f6fd4846 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.149539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683f58c-9fd2-48b0-b430-8802f6fd4846 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.149809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683f58c-9fd2-48b0-b430-8802f6fd4846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.151554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b683f58c-9fd2-48b0-b430-8802f6fd4846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:58.162575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1c164-f1f1-4bda-a026-daaf0797f429 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:58.175626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683f58c-9fd2-48b0-b430-8802f6fd4846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:58.177171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683f58c-9fd2-48b0-b430-8802f6fd4846 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:58.177995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a0a4c-3961-44c7-984b-3659d577c34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:58.178912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a0a4c-3961-44c7-984b-3659d577c34e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.179372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a0a4c-3961-44c7-984b-3659d577c34e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.179821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a0a4c-3961-44c7-984b-3659d577c34e admin admin] PATH_INFO: `/v3/projects/d490afb03206412eb69a868d0458d6a5/users/0c6a45f3a0c943d9a6510760a1a5af72/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.196998 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 935/1850] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:58.200646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b083ad-ec3c-4c19-904b-779c90b33a2e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.200879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b083ad-ec3c-4c19-904b-779c90b33a2e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.203653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b083ad-ec3c-4c19-904b-779c90b33a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.225674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a0a4c-3961-44c7-984b-3659d577c34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d490afb03206412eb69a868d0458d6a5, user_id=0c6a45f3a0c943d9a6510760a1a5af72, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:58.232438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a0a4c-3961-44c7-984b-3659d577c34e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:58.242024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-32b083ad-ec3c-4c19-904b-779c90b33a2e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:58.247287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-32b083ad-ec3c-4c19-904b-779c90b33a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:58.261442 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 916/1851] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:58 2022] PUT /identity/v3/projects/d490afb03206412eb69a868d0458d6a5/users/0c6a45f3a0c943d9a6510760a1a5af72/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:58.265245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a0a4c-3961-44c7-984b-3659d577c34e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:58.298652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:58.366977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a0a4c-3961-44c7-984b-3659d577c34e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:58.376089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cc75f-3b95-4352-9328-ea03d440e1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:58.376716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cc75f-3b95-4352-9328-ea03d440e1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.376995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cc75f-3b95-4352-9328-ea03d440e1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.381462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cc75f-3b95-4352-9328-ea03d440e1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.382607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4cc75f-3b95-4352-9328-ea03d440e1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:58.386590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4cc75f-3b95-4352-9328-ea03d440e1c1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:58.391853 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 936/1852] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:58.394077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b083ad-ec3c-4c19-904b-779c90b33a2e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:58.409216 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 917/1853] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:48:58.413838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cc75f-3b95-4352-9328-ea03d440e1c1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:58.432455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b083ad-ec3c-4c19-904b-779c90b33a2e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:58.438409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cc75f-3b95-4352-9328-ea03d440e1c1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:58.448800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f880d-6cc9-4a74-9f1b-9ba90d897f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:58.451154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f880d-6cc9-4a74-9f1b-9ba90d897f27 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.451535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f880d-6cc9-4a74-9f1b-9ba90d897f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.459071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f880d-6cc9-4a74-9f1b-9ba90d897f27 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.460864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f880d-6cc9-4a74-9f1b-9ba90d897f27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:48:58.461746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec67d01-164c-482b-a0dd-e47b5ab90c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:58.463260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f880d-6cc9-4a74-9f1b-9ba90d897f27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:58.464442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec67d01-164c-482b-a0dd-e47b5ab90c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.464757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec67d01-164c-482b-a0dd-e47b5ab90c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.466053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f880d-6cc9-4a74-9f1b-9ba90d897f27 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:58.468137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec67d01-164c-482b-a0dd-e47b5ab90c40 admin admin] PATH_INFO: `/v3/projects/d490afb03206412eb69a868d0458d6a5/users/0c6a45f3a0c943d9a6510760a1a5af72/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.471997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec67d01-164c-482b-a0dd-e47b5ab90c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d490afb03206412eb69a868d0458d6a5, user_id=0c6a45f3a0c943d9a6510760a1a5af72, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:58.479119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec67d01-164c-482b-a0dd-e47b5ab90c40 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:58.495120 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 937/1854] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:48:58.501759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f880d-6cc9-4a74-9f1b-9ba90d897f27 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:58.510108 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 918/1855] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:58 2022] PUT /identity/v3/projects/d490afb03206412eb69a868d0458d6a5/users/0c6a45f3a0c943d9a6510760a1a5af72/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:58.521549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6722408-26b1-4d3a-92de-997784d895de None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.521838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6722408-26b1-4d3a-92de-997784d895de None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.522097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6722408-26b1-4d3a-92de-997784d895de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.528880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f880d-6cc9-4a74-9f1b-9ba90d897f27 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:58.605305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d6722408-26b1-4d3a-92de-997784d895de None None] MFA Rules not processed for user `0c6a45f3a0c943d9a6510760a1a5af72`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:58.611148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d6722408-26b1-4d3a-92de-997784d895de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:58.620323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:58.696327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170414d-03d4-4575-835d-f87982043563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:58.697987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170414d-03d4-4575-835d-f87982043563 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.698416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170414d-03d4-4575-835d-f87982043563 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.698691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170414d-03d4-4575-835d-f87982043563 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.699677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170414d-03d4-4575-835d-f87982043563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:58.708697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170414d-03d4-4575-835d-f87982043563 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:58.773230 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 938/1856] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:58 2022] POST /identity/v3/projects => generated 375 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:58.786382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170414d-03d4-4575-835d-f87982043563 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:58.806347 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 919/1857] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:58.963369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170414d-03d4-4575-835d-f87982043563 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:58.977806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986885d9-7e1a-4aa3-aa18-2e5e8f7dd3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:58.979392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986885d9-7e1a-4aa3-aa18-2e5e8f7dd3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:58.979741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986885d9-7e1a-4aa3-aa18-2e5e8f7dd3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:58.980098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986885d9-7e1a-4aa3-aa18-2e5e8f7dd3c4 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:58.981899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986885d9-7e1a-4aa3-aa18-2e5e8f7dd3c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:58.988997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986885d9-7e1a-4aa3-aa18-2e5e8f7dd3c4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:59.049746 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 939/1858] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:58 2022] POST /identity/v3/users => generated 288 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:59.051929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6722408-26b1-4d3a-92de-997784d895de None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:59.068319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6722408-26b1-4d3a-92de-997784d895de None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:59.075047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e3bbb-5ea0-4c9d-aa72-aed97fe138d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:59.075917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e3bbb-5ea0-4c9d-aa72-aed97fe138d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:59.077127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e3bbb-5ea0-4c9d-aa72-aed97fe138d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:59.077545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e3bbb-5ea0-4c9d-aa72-aed97fe138d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:59.078403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e3bbb-5ea0-4c9d-aa72-aed97fe138d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:59.080381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e3bbb-5ea0-4c9d-aa72-aed97fe138d7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:59.102699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986885d9-7e1a-4aa3-aa18-2e5e8f7dd3c4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:59.102877 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 920/1859] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:59.122158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986885d9-7e1a-4aa3-aa18-2e5e8f7dd3c4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:59.138047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c844e76-135a-46a9-9807-1ec73eb29502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:59.138928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c844e76-135a-46a9-9807-1ec73eb29502 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:59.139383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c844e76-135a-46a9-9807-1ec73eb29502 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:59.139802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c844e76-135a-46a9-9807-1ec73eb29502 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:59.140623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c844e76-135a-46a9-9807-1ec73eb29502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:59.142353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c844e76-135a-46a9-9807-1ec73eb29502 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:59.161088 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 940/1860] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:59.164349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e3bbb-5ea0-4c9d-aa72-aed97fe138d7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:59.178697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e3bbb-5ea0-4c9d-aa72-aed97fe138d7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:59.186332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f47200-e275-4b25-95c1-96fdb9fda5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:59.187220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f47200-e275-4b25-95c1-96fdb9fda5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:59.187808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f47200-e275-4b25-95c1-96fdb9fda5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:59.188257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f47200-e275-4b25-95c1-96fdb9fda5e9 admin admin] PATH_INFO: `/v3/projects/ed39803cd13045ac9d772d1e738e2adf/users/46bf8420943b4313a3ae4ce5dce9e6bf/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:59.218270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f47200-e275-4b25-95c1-96fdb9fda5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed39803cd13045ac9d772d1e738e2adf, user_id=46bf8420943b4313a3ae4ce5dce9e6bf, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:59.224161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f47200-e275-4b25-95c1-96fdb9fda5e9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:59.249899 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 921/1861] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:59 2022] PUT /identity/v3/projects/ed39803cd13045ac9d772d1e738e2adf/users/46bf8420943b4313a3ae4ce5dce9e6bf/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:48:59.262321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9da10-4a48-4ca5-806f-69a89dcdb9af None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:59.262906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9da10-4a48-4ca5-806f-69a89dcdb9af None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:59.262906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9da10-4a48-4ca5-806f-69a89dcdb9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:59.309405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-77f9da10-4a48-4ca5-806f-69a89dcdb9af None None] MFA Rules not processed for user `46bf8420943b4313a3ae4ce5dce9e6bf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:48:59.311330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-77f9da10-4a48-4ca5-806f-69a89dcdb9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:59.324511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:59.380523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f47200-e275-4b25-95c1-96fdb9fda5e9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:59.382572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f1f47200-e275-4b25-95c1-96fdb9fda5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:59.436026 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 941/1862] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:48:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:48:59.436939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9da10-4a48-4ca5-806f-69a89dcdb9af None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:59.525107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9da10-4a48-4ca5-806f-69a89dcdb9af None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:59.542628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6767904-11bf-4911-b846-f28c94a2b0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:59.543655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6767904-11bf-4911-b846-f28c94a2b0a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:59.544025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6767904-11bf-4911-b846-f28c94a2b0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:59.544389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6767904-11bf-4911-b846-f28c94a2b0a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:59.545156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6767904-11bf-4911-b846-f28c94a2b0a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:59.549835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6767904-11bf-4911-b846-f28c94a2b0a9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:59.559002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f47200-e275-4b25-95c1-96fdb9fda5e9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:59.570425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a2976-024c-425d-9aac-84a98a539457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:59.572427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a2976-024c-425d-9aac-84a98a539457 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:59.572802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a2976-024c-425d-9aac-84a98a539457 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:59.573197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a2976-024c-425d-9aac-84a98a539457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:59.641377 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 942/1863] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:48:59 2022] POST /identity/v3/projects => generated 371 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:59.647308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6767904-11bf-4911-b846-f28c94a2b0a9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:59.697089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a2976-024c-425d-9aac-84a98a539457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:59.700645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a2976-024c-425d-9aac-84a98a539457 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:59.705092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:59.719983 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 922/1864] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:48:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:48:59.793505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a2976-024c-425d-9aac-84a98a539457 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:59.795424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-644a2976-024c-425d-9aac-84a98a539457 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:48:59.804425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6767904-11bf-4911-b846-f28c94a2b0a9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:59.813307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827da5ec-9e61-4e45-94b0-a6a449472b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:59.814530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827da5ec-9e61-4e45-94b0-a6a449472b06 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:59.818001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a2976-024c-425d-9aac-84a98a539457 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:59.826701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827da5ec-9e61-4e45-94b0-a6a449472b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:59.828921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:59.829683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:59.829961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:59.830342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] PATH_INFO: `/v3/projects/d6c560f9235945a7af176a454b799ea2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:59.831536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6c560f9235945a7af176a454b799ea2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:59.835679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827da5ec-9e61-4e45-94b0-a6a449472b06 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:59.836451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827da5ec-9e61-4e45-94b0-a6a449472b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:59.838848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827da5ec-9e61-4e45-94b0-a6a449472b06 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:59.839495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:59.894313 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 943/1865] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:48:59 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:48:59.897101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827da5ec-9e61-4e45-94b0-a6a449472b06 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:59.912543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827da5ec-9e61-4e45-94b0-a6a449472b06 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:59.923620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7128a3-46c1-47df-bb67-7fb5f86cfa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:48:59.925068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7128a3-46c1-47df-bb67-7fb5f86cfa8b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:48:59.925517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7128a3-46c1-47df-bb67-7fb5f86cfa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:48:59.927274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7128a3-46c1-47df-bb67-7fb5f86cfa8b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:48:59.929471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7128a3-46c1-47df-bb67-7fb5f86cfa8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:48:59.931933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7128a3-46c1-47df-bb67-7fb5f86cfa8b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:48:59.950675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6c560f9235945a7af176a454b799ea2', 'request_id': 'req-0f6f94cf-38f4-4895-ac5b-e7371401a5de'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:59.957447 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 944/1866] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:48:59.960969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7128a3-46c1-47df-bb67-7fb5f86cfa8b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:48:59.969403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6c560f9235945a7af176a454b799ea2', 'request_id': 'req-0f6f94cf-38f4-4895-ac5b-e7371401a5de'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:59.979941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6c560f9235945a7af176a454b799ea2', 'request_id': 'req-0f6f94cf-38f4-4895-ac5b-e7371401a5de'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:59.987225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7128a3-46c1-47df-bb67-7fb5f86cfa8b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:48:59.996013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] The token cache is being invalidate because project d6c560f9235945a7af176a454b799ea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:48:59.996298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:48:59.996954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.002003 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 923/1867] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:48:59 2022] DELETE /identity/v3/projects/d6c560f9235945a7af176a454b799ea2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:49:00.002944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:00.005029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:00.005587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:00.089267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57c6a2-908d-4acd-b0d4-9fbc92f11bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:00.090028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57c6a2-908d-4acd-b0d4-9fbc92f11bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:00.090441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57c6a2-908d-4acd-b0d4-9fbc92f11bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:00.090722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57c6a2-908d-4acd-b0d4-9fbc92f11bdb admin admin] PATH_INFO: `/v3/projects/e2db412f18324573b4b60f10c2cf22b3/users/aed8c6d10ce144e6bee13f43c28e4319/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:00.125931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57c6a2-908d-4acd-b0d4-9fbc92f11bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2db412f18324573b4b60f10c2cf22b3, user_id=aed8c6d10ce144e6bee13f43c28e4319, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:00.130207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57c6a2-908d-4acd-b0d4-9fbc92f11bdb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:00.141158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f94cf-38f4-4895-ac5b-e7371401a5de admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:00.154403 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 945/1868] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:48:59 2022] PUT /identity/v3/projects/e2db412f18324573b4b60f10c2cf22b3/users/aed8c6d10ce144e6bee13f43c28e4319/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:00.154403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57c6a2-908d-4acd-b0d4-9fbc92f11bdb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:00.156513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9b57c6a2-908d-4acd-b0d4-9fbc92f11bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:00.280415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:00.281135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:00.287075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:00.287610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] PATH_INFO: `/v3/users/c123f19c891d4ad6b8844d785c2c06d3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:00.292110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c123f19c891d4ad6b8844d785c2c06d3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:00.294141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:00.294556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57c6a2-908d-4acd-b0d4-9fbc92f11bdb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:00.303785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:00.305401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:00.306583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:00.306865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] PATH_INFO: `/v3/users/3493560ca6c9451a940c11eac852ef21` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:00.308607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3493560ca6c9451a940c11eac852ef21)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:00.311289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:00.349404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:00.401322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c123f19c891d4ad6b8844d785c2c06d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.413318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c123f19c891d4ad6b8844d785c2c06d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.461710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c123f19c891d4ad6b8844d785c2c06d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.482640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c123f19c891d4ad6b8844d785c2c06d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.483875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c123f19c891d4ad6b8844d785c2c06d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.485743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3493560ca6c9451a940c11eac852ef21'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.497229 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 924/1869] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:00 2022] DELETE /identity/v3/users/c123f19c891d4ad6b8844d785c2c06d3 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:00.497229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:00.499588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:00.511863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3493560ca6c9451a940c11eac852ef21'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.527310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3493560ca6c9451a940c11eac852ef21'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.544061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3493560ca6c9451a940c11eac852ef21'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.544988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce77a3a-20a4-4da9-b532-210beec973c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3493560ca6c9451a940c11eac852ef21'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:00.554530 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 946/1870] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:00 2022] DELETE /identity/v3/users/3493560ca6c9451a940c11eac852ef21 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:00.572685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9980-f6a4-4a3f-8028-795c591e7019 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:00.573377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9980-f6a4-4a3f-8028-795c591e7019 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:00.574039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9980-f6a4-4a3f-8028-795c591e7019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:00.634308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fc2d9980-f6a4-4a3f-8028-795c591e7019 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:00.635945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fc2d9980-f6a4-4a3f-8028-795c591e7019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:00.648239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0924b2f-496a-4c9b-8201-3ff6ca63079a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:00.659495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:00.778195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195cb42-33f4-43d3-a6f9-3ed9d5ee68b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:00.778874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195cb42-33f4-43d3-a6f9-3ed9d5ee68b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:00.779108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195cb42-33f4-43d3-a6f9-3ed9d5ee68b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:00.779379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195cb42-33f4-43d3-a6f9-3ed9d5ee68b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:00.780237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195cb42-33f4-43d3-a6f9-3ed9d5ee68b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:00.781861 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 947/1871] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:00.782812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195cb42-33f4-43d3-a6f9-3ed9d5ee68b7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:00.796611 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 925/1872] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:00.801096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9980-f6a4-4a3f-8028-795c591e7019 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:00.802586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fc2d9980-f6a4-4a3f-8028-795c591e7019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:00.816994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9980-f6a4-4a3f-8028-795c591e7019 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:00.826556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d7a86-159e-4d3d-901d-d7bfca82d446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:00.827242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d7a86-159e-4d3d-901d-d7bfca82d446 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:00.827518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d7a86-159e-4d3d-901d-d7bfca82d446 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:00.827773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d7a86-159e-4d3d-901d-d7bfca82d446 admin admin] PATH_INFO: `/v3/projects/e2db412f18324573b4b60f10c2cf22b3/users/aed8c6d10ce144e6bee13f43c28e4319/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:00.829810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d7a86-159e-4d3d-901d-d7bfca82d446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2db412f18324573b4b60f10c2cf22b3, user_id=aed8c6d10ce144e6bee13f43c28e4319, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:00.833769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d7a86-159e-4d3d-901d-d7bfca82d446 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:00.848675 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 948/1873] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:00 2022] PUT /identity/v3/projects/e2db412f18324573b4b60f10c2cf22b3/users/aed8c6d10ce144e6bee13f43c28e4319/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:00.862556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97be18-0238-4251-916b-1ffc77b4e293 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:00.863040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97be18-0238-4251-916b-1ffc77b4e293 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:00.863826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97be18-0238-4251-916b-1ffc77b4e293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:00.864517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d7a86-159e-4d3d-901d-d7bfca82d446 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:00.866345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-079d7a86-159e-4d3d-901d-d7bfca82d446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:00.918003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-de97be18-0238-4251-916b-1ffc77b4e293 None None] MFA Rules not processed for user `aed8c6d10ce144e6bee13f43c28e4319`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:00.920412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de97be18-0238-4251-916b-1ffc77b4e293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:01.008020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:01.010152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d7a86-159e-4d3d-901d-d7bfca82d446 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:01.020449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7198a-f438-4d96-85b7-319c641c3da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:01.021258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7198a-f438-4d96-85b7-319c641c3da7 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:01.021552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7198a-f438-4d96-85b7-319c641c3da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:01.021980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7198a-f438-4d96-85b7-319c641c3da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:01.074854 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 926/1874] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:01.085778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97be18-0238-4251-916b-1ffc77b4e293 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:01.087747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de97be18-0238-4251-916b-1ffc77b4e293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:01.172102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7198a-f438-4d96-85b7-319c641c3da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:01.174390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7198a-f438-4d96-85b7-319c641c3da7 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:01.191812 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 949/1875] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:49:01.241785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97be18-0238-4251-916b-1ffc77b4e293 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:01.248515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977193-b400-4a0f-a2c9-5ff68d098a6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:01.249385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977193-b400-4a0f-a2c9-5ff68d098a6a service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:01.249660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977193-b400-4a0f-a2c9-5ff68d098a6a service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:01.249977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977193-b400-4a0f-a2c9-5ff68d098a6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:01.260910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe977193-b400-4a0f-a2c9-5ff68d098a6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:01.262846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe977193-b400-4a0f-a2c9-5ff68d098a6a service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:01.297087 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 927/1876] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:49:01.456578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7198a-f438-4d96-85b7-319c641c3da7 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:01.458778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f6c7198a-f438-4d96-85b7-319c641c3da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:01.478767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7198a-f438-4d96-85b7-319c641c3da7 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:01.484964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bc69d-838e-4882-8517-47e97ffe8aac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:01.485840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bc69d-838e-4882-8517-47e97ffe8aac service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:01.486324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bc69d-838e-4882-8517-47e97ffe8aac service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:01.486879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bc69d-838e-4882-8517-47e97ffe8aac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:01.489647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-495bc69d-838e-4882-8517-47e97ffe8aac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:01.606197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:01.635375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bc69d-838e-4882-8517-47e97ffe8aac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:01.637539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bc69d-838e-4882-8517-47e97ffe8aac service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:01.670967 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 950/1877] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:01.826438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977193-b400-4a0f-a2c9-5ff68d098a6a service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:01.827888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fe977193-b400-4a0f-a2c9-5ff68d098a6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:01.854302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977193-b400-4a0f-a2c9-5ff68d098a6a service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:01.866829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:01.867571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:01.867901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:01.868287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] PATH_INFO: `/v3/projects/5a194c610fdf40dd849fc78fc4aa07f2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:01.869671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a194c610fdf40dd849fc78fc4aa07f2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:01.872458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:02.001612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a194c610fdf40dd849fc78fc4aa07f2', 'request_id': 'req-347722c2-c2ea-415d-aac5-727966865d69'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:02.032110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a194c610fdf40dd849fc78fc4aa07f2', 'request_id': 'req-347722c2-c2ea-415d-aac5-727966865d69'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:02.033771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bc69d-838e-4882-8517-47e97ffe8aac service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:02.053747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a194c610fdf40dd849fc78fc4aa07f2', 'request_id': 'req-347722c2-c2ea-415d-aac5-727966865d69'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:02.060531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bc69d-838e-4882-8517-47e97ffe8aac service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:02.096491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] The token cache is being invalidate because project 5a194c610fdf40dd849fc78fc4aa07f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:02.096948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:02.097747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-347722c2-c2ea-415d-aac5-727966865d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:02.106908 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 928/1878] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:01 2022] DELETE /identity/v3/projects/5a194c610fdf40dd849fc78fc4aa07f2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:02.113474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d03eb-81ad-46b7-9ce9-b21e429f0cf7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:02.113777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d03eb-81ad-46b7-9ce9-b21e429f0cf7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:02.114035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d03eb-81ad-46b7-9ce9-b21e429f0cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:02.171769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1d1d03eb-81ad-46b7-9ce9-b21e429f0cf7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:02.173205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1d1d03eb-81ad-46b7-9ce9-b21e429f0cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:02.251680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:02.252724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:02.253024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:02.253347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] PATH_INFO: `/v3/users/46bf8420943b4313a3ae4ce5dce9e6bf` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:02.255302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] RBAC: Authorizing `identity:delete_user(user_id=46bf8420943b4313a3ae4ce5dce9e6bf)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:02.259716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:02.285777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:02.328423 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 929/1879] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:02.336820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371208c-d6e8-4eb7-853c-7336208346f7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:02.337210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371208c-d6e8-4eb7-853c-7336208346f7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:02.337720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371208c-d6e8-4eb7-853c-7336208346f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:02.406033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46bf8420943b4313a3ae4ce5dce9e6bf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:02.407395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f371208c-d6e8-4eb7-853c-7336208346f7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:02.409932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f371208c-d6e8-4eb7-853c-7336208346f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:02.433925 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 930/1880] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:02.436337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46bf8420943b4313a3ae4ce5dce9e6bf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:02.438738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371208c-d6e8-4eb7-853c-7336208346f7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:02.460335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371208c-d6e8-4eb7-853c-7336208346f7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:02.465379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46bf8420943b4313a3ae4ce5dce9e6bf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:02.479339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5556b-672c-4e26-a8c2-cbb80de4f1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:02.479999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5556b-672c-4e26-a8c2-cbb80de4f1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:02.480282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5556b-672c-4e26-a8c2-cbb80de4f1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:02.480550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5556b-672c-4e26-a8c2-cbb80de4f1e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:02.481227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5556b-672c-4e26-a8c2-cbb80de4f1e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:02.481530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5556b-672c-4e26-a8c2-cbb80de4f1e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:02.482116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46bf8420943b4313a3ae4ce5dce9e6bf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:02.486508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9600aca1-b5ad-4541-8fdb-d8ede8cbe8df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46bf8420943b4313a3ae4ce5dce9e6bf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:02.490756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5556b-672c-4e26-a8c2-cbb80de4f1e2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:02.504756 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 951/1881] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:02 2022] DELETE /identity/v3/users/46bf8420943b4313a3ae4ce5dce9e6bf => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:02.509308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6835c7-25be-4af7-8277-e6c55683d3b1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:02.514773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6835c7-25be-4af7-8277-e6c55683d3b1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:02.516458 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 931/1882] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:02.523371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6835c7-25be-4af7-8277-e6c55683d3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:02.526808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5556b-672c-4e26-a8c2-cbb80de4f1e2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:02.530042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0ab5556b-672c-4e26-a8c2-cbb80de4f1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:02.552482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5556b-672c-4e26-a8c2-cbb80de4f1e2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:02.559316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1e6835c7-25be-4af7-8277-e6c55683d3b1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:02.563058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e6835c7-25be-4af7-8277-e6c55683d3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:02.666559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62cf4-a228-44f7-9529-78101787ae17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:02.667282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62cf4-a228-44f7-9529-78101787ae17 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:02.667627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62cf4-a228-44f7-9529-78101787ae17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:02.667968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62cf4-a228-44f7-9529-78101787ae17 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:02.668888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d62cf4-a228-44f7-9529-78101787ae17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:02.670914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d62cf4-a228-44f7-9529-78101787ae17 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:02.690775 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 952/1883] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:02.690775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac59e1-d4fa-4b64-8d6f-7315c9eb6b8f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:02.690775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac59e1-d4fa-4b64-8d6f-7315c9eb6b8f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:02.690775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac59e1-d4fa-4b64-8d6f-7315c9eb6b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:02.722287 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 932/1884] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:02 2022] POST /identity/v3/projects => generated 369 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:02.727226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:02.728075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62cf4-a228-44f7-9529-78101787ae17 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:02.731258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d7d62cf4-a228-44f7-9529-78101787ae17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:02.740784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5cac59e1-d4fa-4b64-8d6f-7315c9eb6b8f None None] MFA Rules not processed for user `507faa1518b34884b3e095bfae431bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:02.754671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5cac59e1-d4fa-4b64-8d6f-7315c9eb6b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:02.878475 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 953/1885] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:02.880338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac59e1-d4fa-4b64-8d6f-7315c9eb6b8f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:02.888847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62cf4-a228-44f7-9529-78101787ae17 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:02.894265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6ea5a-5d44-4f18-aaea-2b84da9ad762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:02.895349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6ea5a-5d44-4f18-aaea-2b84da9ad762 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:02.895896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6ea5a-5d44-4f18-aaea-2b84da9ad762 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:02.896386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6ea5a-5d44-4f18-aaea-2b84da9ad762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:02.972106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac59e1-d4fa-4b64-8d6f-7315c9eb6b8f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:02.978747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd99914-d008-4cdb-b1ed-972b467a58cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:02.979448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd99914-d008-4cdb-b1ed-972b467a58cb admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:02.979849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd99914-d008-4cdb-b1ed-972b467a58cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:02.980188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd99914-d008-4cdb-b1ed-972b467a58cb admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:02.980894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd99914-d008-4cdb-b1ed-972b467a58cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:02.983334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd99914-d008-4cdb-b1ed-972b467a58cb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:02.994942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6ea5a-5d44-4f18-aaea-2b84da9ad762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:02.996774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6ea5a-5d44-4f18-aaea-2b84da9ad762 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:03.007466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:03.026931 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 954/1886] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:02 2022] POST /identity/v3/users => generated 285 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:03.027126 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 933/1887] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:03.027206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968ec37-485b-42c8-8a6c-313ccd0faa91 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:03.027206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968ec37-485b-42c8-8a6c-313ccd0faa91 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:03.027206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968ec37-485b-42c8-8a6c-313ccd0faa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:03.028013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6ea5a-5d44-4f18-aaea-2b84da9ad762 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:03.049429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6ea5a-5d44-4f18-aaea-2b84da9ad762 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:03.054475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4968ec37-485b-42c8-8a6c-313ccd0faa91 None None] MFA Rules not processed for user `507faa1518b34884b3e095bfae431bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:03.057242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4968ec37-485b-42c8-8a6c-313ccd0faa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:03.064416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8b3eb-b655-4aad-9d92-6f2c2a73a15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:03.065303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8b3eb-b655-4aad-9d92-6f2c2a73a15b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:03.065594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8b3eb-b655-4aad-9d92-6f2c2a73a15b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:03.066150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8b3eb-b655-4aad-9d92-6f2c2a73a15b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:03.067164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8b3eb-b655-4aad-9d92-6f2c2a73a15b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:03.070597 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 955/1888] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:03.071564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8b3eb-b655-4aad-9d92-6f2c2a73a15b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:03.094627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968ec37-485b-42c8-8a6c-313ccd0faa91 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:03.096909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4968ec37-485b-42c8-8a6c-313ccd0faa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:03.097273 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 934/1889] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:49:03.102310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8b3eb-b655-4aad-9d92-6f2c2a73a15b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:03.118013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8b3eb-b655-4aad-9d92-6f2c2a73a15b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:03.138083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33383f23-0fbe-49ed-be61-0133b4e6981a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:03.138758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33383f23-0fbe-49ed-be61-0133b4e6981a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:03.139037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33383f23-0fbe-49ed-be61-0133b4e6981a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:03.139363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33383f23-0fbe-49ed-be61-0133b4e6981a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:03.140007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33383f23-0fbe-49ed-be61-0133b4e6981a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:03.141656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33383f23-0fbe-49ed-be61-0133b4e6981a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:03.174557 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 935/1890] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:49:03.175270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33383f23-0fbe-49ed-be61-0133b4e6981a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:03.188281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33383f23-0fbe-49ed-be61-0133b4e6981a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:03.209331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ac07e-5112-4330-b785-72c690cb8287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:03.209800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968ec37-485b-42c8-8a6c-313ccd0faa91 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:03.210014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ac07e-5112-4330-b785-72c690cb8287 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:03.210295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ac07e-5112-4330-b785-72c690cb8287 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:03.210569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ac07e-5112-4330-b785-72c690cb8287 admin admin] PATH_INFO: `/v3/projects/fe14ede74b2140269cd7dfc32fe14f9c/users/c613201f55b94a528c6c36e5b5772799/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:03.217228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc4c51-9c46-4274-8cda-7eabed951e11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:03.217964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc4c51-9c46-4274-8cda-7eabed951e11 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:03.218311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc4c51-9c46-4274-8cda-7eabed951e11 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:03.218651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc4c51-9c46-4274-8cda-7eabed951e11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:03.228390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc4c51-9c46-4274-8cda-7eabed951e11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:03.230210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc4c51-9c46-4274-8cda-7eabed951e11 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:03.232784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ac07e-5112-4330-b785-72c690cb8287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe14ede74b2140269cd7dfc32fe14f9c, user_id=c613201f55b94a528c6c36e5b5772799, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:03.236297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ac07e-5112-4330-b785-72c690cb8287 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:03.242250 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 956/1891] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:49:03.243116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc4c51-9c46-4274-8cda-7eabed951e11 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:03.244937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1bfc4c51-9c46-4274-8cda-7eabed951e11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:03.261236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc4c51-9c46-4274-8cda-7eabed951e11 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:03.264278 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 936/1892] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:03 2022] PUT /identity/v3/projects/fe14ede74b2140269cd7dfc32fe14f9c/users/c613201f55b94a528c6c36e5b5772799/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:03.270488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b11df1-8a07-4e7a-8207-a2b6745ff105 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:03.270786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b11df1-8a07-4e7a-8207-a2b6745ff105 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:03.271079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b11df1-8a07-4e7a-8207-a2b6745ff105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:03.274049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:03.318693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-80b11df1-8a07-4e7a-8207-a2b6745ff105 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:03.319933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80b11df1-8a07-4e7a-8207-a2b6745ff105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:03.392962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:03.393631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:03.394944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:03.394944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] PATH_INFO: `/v3/projects/90478ad396024d92baf1f96159d9d7e0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:03.400630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90478ad396024d92baf1f96159d9d7e0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:03.403494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:03.471707 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 937/1893] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:03.497478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90478ad396024d92baf1f96159d9d7e0', 'request_id': 'req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:03.510486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90478ad396024d92baf1f96159d9d7e0', 'request_id': 'req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:03.515716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b11df1-8a07-4e7a-8207-a2b6745ff105 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:03.517845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80b11df1-8a07-4e7a-8207-a2b6745ff105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:03.541272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90478ad396024d92baf1f96159d9d7e0', 'request_id': 'req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:03.568190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] The token cache is being invalidate because project 90478ad396024d92baf1f96159d9d7e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:03.568866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:03.571006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:03.577139 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 957/1894] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:03 2022] DELETE /identity/v3/projects/90478ad396024d92baf1f96159d9d7e0 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:03.599597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:03.605944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:03.682411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b11df1-8a07-4e7a-8207-a2b6745ff105 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:03.802469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ebb5a-1799-467a-bc1c-2d90e8bdd0c5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:03.808628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:03.809905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:03.810259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:03.810603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] PATH_INFO: `/v3/users/0c6a45f3a0c943d9a6510760a1a5af72` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:03.811855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c6a45f3a0c943d9a6510760a1a5af72)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:03.816934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:03.841313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0685a-e63a-4de3-b405-799fc2b12c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:03.842133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0685a-e63a-4de3-b405-799fc2b12c80 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:03.843145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0685a-e63a-4de3-b405-799fc2b12c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:03.845843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0685a-e63a-4de3-b405-799fc2b12c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:03.856436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-36f0685a-e63a-4de3-b405-799fc2b12c80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:03.873682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0685a-e63a-4de3-b405-799fc2b12c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:03.875246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0685a-e63a-4de3-b405-799fc2b12c80 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:03.890350 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 938/1895] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:49:03.891328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0685a-e63a-4de3-b405-799fc2b12c80 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:03.892882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-36f0685a-e63a-4de3-b405-799fc2b12c80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:03.907266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0685a-e63a-4de3-b405-799fc2b12c80 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:03.913169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:03.913918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:03.914352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:03.914719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] PATH_INFO: `/v3/projects/ed39803cd13045ac9d772d1e738e2adf` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:03.915937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed39803cd13045ac9d772d1e738e2adf)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:03.918346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:03.924257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:03.932838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:03.944605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c6a45f3a0c943d9a6510760a1a5af72'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:03.987458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c6a45f3a0c943d9a6510760a1a5af72'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:03.999000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed39803cd13045ac9d772d1e738e2adf', 'request_id': 'req-5abc7658-79d6-4bb0-b00c-a56076d41b7d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.007962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c6a45f3a0c943d9a6510760a1a5af72'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.015914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed39803cd13045ac9d772d1e738e2adf', 'request_id': 'req-5abc7658-79d6-4bb0-b00c-a56076d41b7d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.018851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c6a45f3a0c943d9a6510760a1a5af72'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.020244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c6a45f3a0c943d9a6510760a1a5af72'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.023793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed39803cd13045ac9d772d1e738e2adf', 'request_id': 'req-5abc7658-79d6-4bb0-b00c-a56076d41b7d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.030938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 958/1896] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:03 2022] DELETE /identity/v3/users/0c6a45f3a0c943d9a6510760a1a5af72 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:04.046134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] The token cache is being invalidate because project ed39803cd13045ac9d772d1e738e2adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:04.046706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.048616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.051432 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 939/1897] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:03 2022] DELETE /identity/v3/projects/ed39803cd13045ac9d772d1e738e2adf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:04.056721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:04.059765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:04.172937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604f35e-8eb3-4ae2-8abb-8a2d1c2210ba admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:04.179512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:04.180140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:04.180414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:04.180826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] PATH_INFO: `/v3/users/aed8c6d10ce144e6bee13f43c28e4319` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:04.183113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed8c6d10ce144e6bee13f43c28e4319)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:04.185240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:04.293097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed8c6d10ce144e6bee13f43c28e4319'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.316824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed8c6d10ce144e6bee13f43c28e4319'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.331123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed8c6d10ce144e6bee13f43c28e4319'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.343854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed8c6d10ce144e6bee13f43c28e4319'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.349085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed8c6d10ce144e6bee13f43c28e4319'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:04.372008 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 959/1898] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:04 2022] DELETE /identity/v3/users/aed8c6d10ce144e6bee13f43c28e4319 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:04.834662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:04.836186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:04.977465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc7658-79d6-4bb0-b00c-a56076d41b7d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:05.076755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:05.077905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:05.078363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:05.078813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] PATH_INFO: `/v3/projects/d490afb03206412eb69a868d0458d6a5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:05.080499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d490afb03206412eb69a868d0458d6a5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:05.084582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:05.161909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:05.164949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:05.173767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:05.182437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd490afb03206412eb69a868d0458d6a5', 'request_id': 'req-9dde9b91-a59e-4ca4-819e-8895baa200dd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:05.186716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fab6c-ed7d-426f-be82-1813256d588f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:05.195119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd490afb03206412eb69a868d0458d6a5', 'request_id': 'req-9dde9b91-a59e-4ca4-819e-8895baa200dd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:05.208742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd490afb03206412eb69a868d0458d6a5', 'request_id': 'req-9dde9b91-a59e-4ca4-819e-8895baa200dd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:05.224790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] The token cache is being invalidate because project d490afb03206412eb69a868d0458d6a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:05.225244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:05.226214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9dde9b91-a59e-4ca4-819e-8895baa200dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:05.228376 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 940/1899] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:04 2022] DELETE /identity/v3/projects/d490afb03206412eb69a868d0458d6a5 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:05.234636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31f70f-bfbc-4e75-b9bc-8f1bb0dfcdc0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:05.235009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31f70f-bfbc-4e75-b9bc-8f1bb0dfcdc0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:05.235349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31f70f-bfbc-4e75-b9bc-8f1bb0dfcdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:05.242684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:05.267900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4d31f70f-bfbc-4e75-b9bc-8f1bb0dfcdc0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:05.274544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4d31f70f-bfbc-4e75-b9bc-8f1bb0dfcdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:05.289060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:05.290394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:05.291965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:05.292249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] PATH_INFO: `/v3/projects/e2db412f18324573b4b60f10c2cf22b3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:05.294992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2db412f18324573b4b60f10c2cf22b3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:05.296247 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 941/1900] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:05.298060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:05.304663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51252f6f-75c4-4ace-86a8-3e58fcc957f1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:05.304944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51252f6f-75c4-4ace-86a8-3e58fcc957f1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:05.305237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51252f6f-75c4-4ace-86a8-3e58fcc957f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:05.364457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-51252f6f-75c4-4ace-86a8-3e58fcc957f1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:05.365889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-51252f6f-75c4-4ace-86a8-3e58fcc957f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:05.380517 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 942/1901] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:05.381877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51252f6f-75c4-4ace-86a8-3e58fcc957f1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:05.393173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51252f6f-75c4-4ace-86a8-3e58fcc957f1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:05.406373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338f10c-17ce-4d18-beb9-7e8c8c20bb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:05.407052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338f10c-17ce-4d18-beb9-7e8c8c20bb52 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:05.407412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338f10c-17ce-4d18-beb9-7e8c8c20bb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:05.408067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338f10c-17ce-4d18-beb9-7e8c8c20bb52 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:05.408415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3338f10c-17ce-4d18-beb9-7e8c8c20bb52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:05.409229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3338f10c-17ce-4d18-beb9-7e8c8c20bb52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:05.412461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3338f10c-17ce-4d18-beb9-7e8c8c20bb52 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:05.448266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2db412f18324573b4b60f10c2cf22b3', 'request_id': 'req-ef656933-26c5-4b45-9c67-817508ceb1d8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:05.449781 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 943/1902] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:05.450479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338f10c-17ce-4d18-beb9-7e8c8c20bb52 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:05.474474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338f10c-17ce-4d18-beb9-7e8c8c20bb52 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:05.476001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2db412f18324573b4b60f10c2cf22b3', 'request_id': 'req-ef656933-26c5-4b45-9c67-817508ceb1d8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:05.485871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2db412f18324573b4b60f10c2cf22b3', 'request_id': 'req-ef656933-26c5-4b45-9c67-817508ceb1d8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:05.506258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] The token cache is being invalidate because project e2db412f18324573b4b60f10c2cf22b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:05.506401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:05.507229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef656933-26c5-4b45-9c67-817508ceb1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:05.512419 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 960/1903] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:05 2022] DELETE /identity/v3/projects/e2db412f18324573b4b60f10c2cf22b3 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:05.515086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd751279-8f6b-4a0c-aa6b-09d67bb1b6fb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:05.515487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd751279-8f6b-4a0c-aa6b-09d67bb1b6fb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:05.515862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd751279-8f6b-4a0c-aa6b-09d67bb1b6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:05.547207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dd751279-8f6b-4a0c-aa6b-09d67bb1b6fb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:05.552411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dd751279-8f6b-4a0c-aa6b-09d67bb1b6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:05.611527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba0516-6404-4e7b-b0ef-323a00b670e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:05.612355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba0516-6404-4e7b-b0ef-323a00b670e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:05.613009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba0516-6404-4e7b-b0ef-323a00b670e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:05.613631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba0516-6404-4e7b-b0ef-323a00b670e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:05.614508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba0516-6404-4e7b-b0ef-323a00b670e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:05.616885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba0516-6404-4e7b-b0ef-323a00b670e4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:05.679202 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 944/1904] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:05 2022] POST /identity/v3/projects => generated 387 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:05.681501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba0516-6404-4e7b-b0ef-323a00b670e4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:05.684866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-caba0516-6404-4e7b-b0ef-323a00b670e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:05.712459 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 961/1905] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:05.725615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b54402-02bb-4b82-80cd-416711f9fcb0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:05.728292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b54402-02bb-4b82-80cd-416711f9fcb0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:05.728519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b54402-02bb-4b82-80cd-416711f9fcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:05.793784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-59b54402-02bb-4b82-80cd-416711f9fcb0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:05.795832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-59b54402-02bb-4b82-80cd-416711f9fcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:05.854601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba0516-6404-4e7b-b0ef-323a00b670e4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:05.876005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78756db4-dcfb-4c10-9b41-1ad337203c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:05.876866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78756db4-dcfb-4c10-9b41-1ad337203c01 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:05.877237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78756db4-dcfb-4c10-9b41-1ad337203c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:05.877605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78756db4-dcfb-4c10-9b41-1ad337203c01 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:05.878976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78756db4-dcfb-4c10-9b41-1ad337203c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:05.880998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78756db4-dcfb-4c10-9b41-1ad337203c01 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:05.930888 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 962/1906] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:05.932018 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 945/1907] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:05 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:05.935784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78756db4-dcfb-4c10-9b41-1ad337203c01 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:05.941853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b54402-02bb-4b82-80cd-416711f9fcb0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:05.989096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78756db4-dcfb-4c10-9b41-1ad337203c01 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:05.994878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b54402-02bb-4b82-80cd-416711f9fcb0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.002434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b9389-f490-40a5-bd10-868f5a6ca2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.005410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b9389-f490-40a5-bd10-868f5a6ca2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.005880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b9389-f490-40a5-bd10-868f5a6ca2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.006846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b9389-f490-40a5-bd10-868f5a6ca2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.008162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b9389-f490-40a5-bd10-868f5a6ca2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.011278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b9389-f490-40a5-bd10-868f5a6ca2b6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.014174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f31fa-1639-4d93-867e-260d46e44e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.015000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f31fa-1639-4d93-867e-260d46e44e63 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.015455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f31fa-1639-4d93-867e-260d46e44e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.015968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f31fa-1639-4d93-867e-260d46e44e63 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.017632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f31fa-1639-4d93-867e-260d46e44e63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:06.018819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f31fa-1639-4d93-867e-260d46e44e63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.025227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f31fa-1639-4d93-867e-260d46e44e63 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.033396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:06.045656 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 946/1908] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:49:06.050093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b9389-f490-40a5-bd10-868f5a6ca2b6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:06.061772 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 963/1909] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:06.066800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f31fa-1639-4d93-867e-260d46e44e63 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:06.079872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b9389-f490-40a5-bd10-868f5a6ca2b6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.092706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f31fa-1639-4d93-867e-260d46e44e63 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.099808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79025f79-62a6-42b9-9f57-ebbfb8d43c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.100902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79025f79-62a6-42b9-9f57-ebbfb8d43c04 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.101358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79025f79-62a6-42b9-9f57-ebbfb8d43c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.101698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79025f79-62a6-42b9-9f57-ebbfb8d43c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.102410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79025f79-62a6-42b9-9f57-ebbfb8d43c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.106026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79025f79-62a6-42b9-9f57-ebbfb8d43c04 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.106594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be2f30-16a8-4317-ac2a-b58f3c5db747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.107983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be2f30-16a8-4317-ac2a-b58f3c5db747 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.108410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be2f30-16a8-4317-ac2a-b58f3c5db747 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.108748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be2f30-16a8-4317-ac2a-b58f3c5db747 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.111330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be2f30-16a8-4317-ac2a-b58f3c5db747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.115353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be2f30-16a8-4317-ac2a-b58f3c5db747 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.118618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:06.145443 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 947/1910] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:06.146226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79025f79-62a6-42b9-9f57-ebbfb8d43c04 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:06.192214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79025f79-62a6-42b9-9f57-ebbfb8d43c04 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.216609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f61f8-3b0f-4986-98f2-b0621876bfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.217596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f61f8-3b0f-4986-98f2-b0621876bfca admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.218041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f61f8-3b0f-4986-98f2-b0621876bfca admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.218430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f61f8-3b0f-4986-98f2-b0621876bfca admin admin] PATH_INFO: `/v3/projects/22a57f99fc634af4b37a892143bf6665/users/a62d07846dda4594a29a627c97cd236c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.248306 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 964/1911] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:06 2022] POST /identity/v3/projects => generated 409 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:06.250907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be2f30-16a8-4317-ac2a-b58f3c5db747 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:06.258725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f61f8-3b0f-4986-98f2-b0621876bfca admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a57f99fc634af4b37a892143bf6665, user_id=a62d07846dda4594a29a627c97cd236c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.262161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f61f8-3b0f-4986-98f2-b0621876bfca admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.309977 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 948/1912] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:06 2022] PUT /identity/v3/projects/22a57f99fc634af4b37a892143bf6665/users/a62d07846dda4594a29a627c97cd236c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:06.333979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db395c22-0950-49a0-978e-f2a2a49bf846 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.334406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db395c22-0950-49a0-978e-f2a2a49bf846 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.334849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db395c22-0950-49a0-978e-f2a2a49bf846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.382945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be2f30-16a8-4317-ac2a-b58f3c5db747 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.409448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-db395c22-0950-49a0-978e-f2a2a49bf846 None None] MFA Rules not processed for user `a62d07846dda4594a29a627c97cd236c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:06.410828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-db395c22-0950-49a0-978e-f2a2a49bf846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:06.490753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e7351-167c-40a2-8736-e4d5f21e4d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.491365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e7351-167c-40a2-8736-e4d5f21e4d08 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.491771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e7351-167c-40a2-8736-e4d5f21e4d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.492110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e7351-167c-40a2-8736-e4d5f21e4d08 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.493136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e7351-167c-40a2-8736-e4d5f21e4d08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.495372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e7351-167c-40a2-8736-e4d5f21e4d08 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.528044 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 965/1913] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:06 2022] POST /identity/v3/users => generated 305 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:06.529137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e7351-167c-40a2-8736-e4d5f21e4d08 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:06.559215 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 949/1914] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:06.560974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e7351-167c-40a2-8736-e4d5f21e4d08 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.562534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db395c22-0950-49a0-978e-f2a2a49bf846 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:06.568921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f06f8c-209a-4bd4-b8b7-07527957f4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.569687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f06f8c-209a-4bd4-b8b7-07527957f4cd admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.570065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f06f8c-209a-4bd4-b8b7-07527957f4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.570592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f06f8c-209a-4bd4-b8b7-07527957f4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.572215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f06f8c-209a-4bd4-b8b7-07527957f4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.574524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f06f8c-209a-4bd4-b8b7-07527957f4cd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.589664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db395c22-0950-49a0-978e-f2a2a49bf846 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.596392 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 966/1915] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:06.597872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f06f8c-209a-4bd4-b8b7-07527957f4cd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:06.614229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f06f8c-209a-4bd4-b8b7-07527957f4cd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.615748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c7da4-e538-4fc1-b1ce-fd57641a1898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.616412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c7da4-e538-4fc1-b1ce-fd57641a1898 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.616720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c7da4-e538-4fc1-b1ce-fd57641a1898 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.616952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c7da4-e538-4fc1-b1ce-fd57641a1898 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.617628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c7da4-e538-4fc1-b1ce-fd57641a1898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.619546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c7da4-e538-4fc1-b1ce-fd57641a1898 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.622216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cca543-ea4f-4b0a-a686-7b08593e820d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.623160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cca543-ea4f-4b0a-a686-7b08593e820d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.624148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cca543-ea4f-4b0a-a686-7b08593e820d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.624526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cca543-ea4f-4b0a-a686-7b08593e820d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.625396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cca543-ea4f-4b0a-a686-7b08593e820d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.627150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cca543-ea4f-4b0a-a686-7b08593e820d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.648172 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 967/1916] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:06.649362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cca543-ea4f-4b0a-a686-7b08593e820d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:06.669822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cca543-ea4f-4b0a-a686-7b08593e820d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.674946 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 950/1917] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:06 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:06.676810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c7da4-e538-4fc1-b1ce-fd57641a1898 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:06.759028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:06.783073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b31ed-c120-440b-94a0-84000abfbe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.785113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b31ed-c120-440b-94a0-84000abfbe38 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.785552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b31ed-c120-440b-94a0-84000abfbe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.785949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b31ed-c120-440b-94a0-84000abfbe38 admin admin] PATH_INFO: `/v3/projects/89ed78bfa690487c816835dd282e2970/users/bd2aa13e39d644319136fd102c55cb95/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.810597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b31ed-c120-440b-94a0-84000abfbe38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ed78bfa690487c816835dd282e2970, user_id=bd2aa13e39d644319136fd102c55cb95, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.818883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b31ed-c120-440b-94a0-84000abfbe38 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.822352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c7da4-e538-4fc1-b1ce-fd57641a1898 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.831362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cb17-9cc8-4b3c-a53f-31a878a88fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:06.832526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cb17-9cc8-4b3c-a53f-31a878a88fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.835283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cb17-9cc8-4b3c-a53f-31a878a88fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.835884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cb17-9cc8-4b3c-a53f-31a878a88fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.837070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0cb17-9cc8-4b3c-a53f-31a878a88fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:06.840198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0cb17-9cc8-4b3c-a53f-31a878a88fc8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:06.842702 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 968/1918] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:06 2022] PUT /identity/v3/projects/89ed78bfa690487c816835dd282e2970/users/bd2aa13e39d644319136fd102c55cb95/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:06.854861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555921f7-963e-4a0b-8449-e58a1076d77d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:06.855147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555921f7-963e-4a0b-8449-e58a1076d77d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:06.855423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555921f7-963e-4a0b-8449-e58a1076d77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:06.887165 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 951/1919] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:06 2022] POST /identity/v3/users => generated 301 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:06.888257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cb17-9cc8-4b3c-a53f-31a878a88fc8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:06.916952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cb17-9cc8-4b3c-a53f-31a878a88fc8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:06.934423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-555921f7-963e-4a0b-8449-e58a1076d77d None None] MFA Rules not processed for user `bd2aa13e39d644319136fd102c55cb95`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:06.937372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-555921f7-963e-4a0b-8449-e58a1076d77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:06.984697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:07.058211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22224edd-bcfa-4848-8804-7ad479869b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:07.058839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22224edd-bcfa-4848-8804-7ad479869b94 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:07.059112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22224edd-bcfa-4848-8804-7ad479869b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:07.059382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22224edd-bcfa-4848-8804-7ad479869b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:07.060031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22224edd-bcfa-4848-8804-7ad479869b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:07.062163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22224edd-bcfa-4848-8804-7ad479869b94 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:07.086249 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 969/1920] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:06 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:07.095918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555921f7-963e-4a0b-8449-e58a1076d77d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:07.103615 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 952/1921] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:07.106996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22224edd-bcfa-4848-8804-7ad479869b94 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:07.123969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555921f7-963e-4a0b-8449-e58a1076d77d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:07.143094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c3edf-0edf-47f8-a9d9-7c2b61e8495d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:07.143094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c3edf-0edf-47f8-a9d9-7c2b61e8495d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:07.143478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c3edf-0edf-47f8-a9d9-7c2b61e8495d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:07.144204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c3edf-0edf-47f8-a9d9-7c2b61e8495d admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:07.145069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c3edf-0edf-47f8-a9d9-7c2b61e8495d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:07.147384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c3edf-0edf-47f8-a9d9-7c2b61e8495d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:07.165806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22224edd-bcfa-4848-8804-7ad479869b94 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:07.171797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28eeed-4136-419a-9e92-1d096999cf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:07.172961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28eeed-4136-419a-9e92-1d096999cf8e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:07.172961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28eeed-4136-419a-9e92-1d096999cf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:07.173479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28eeed-4136-419a-9e92-1d096999cf8e admin admin] PATH_INFO: `/v3/projects/62478424b22c42c88c3d53b21bac76cc/users/7e4d521d64424a37ab5a12e58a752a1a/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:07.203878 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 970/1922] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:07 2022] POST /identity/v3/projects => generated 411 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:07.208885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c3edf-0edf-47f8-a9d9-7c2b61e8495d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:07.228756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28eeed-4136-419a-9e92-1d096999cf8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=62478424b22c42c88c3d53b21bac76cc, user_id=7e4d521d64424a37ab5a12e58a752a1a, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:07.232363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28eeed-4136-419a-9e92-1d096999cf8e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:07.271591 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 953/1923] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:07 2022] PUT /identity/v3/projects/62478424b22c42c88c3d53b21bac76cc/users/7e4d521d64424a37ab5a12e58a752a1a/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:07.281687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28eeed-4136-419a-9e92-1d096999cf8e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:07.314224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c3edf-0edf-47f8-a9d9-7c2b61e8495d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:07.465492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:07.486867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28eeed-4136-419a-9e92-1d096999cf8e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:07.503047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc692f83-ccff-44bc-983c-8c65aa0a9a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:07.506078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff0a3-bc1a-4f76-9668-a6288b80d9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:07.506734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff0a3-bc1a-4f76-9668-a6288b80d9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:07.507009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff0a3-bc1a-4f76-9668-a6288b80d9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:07.507502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692f83-ccff-44bc-983c-8c65aa0a9a47 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:07.508285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692f83-ccff-44bc-983c-8c65aa0a9a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:07.508557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692f83-ccff-44bc-983c-8c65aa0a9a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:07.510220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff0a3-bc1a-4f76-9668-a6288b80d9f4 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:07.510878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eff0a3-bc1a-4f76-9668-a6288b80d9f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:07.512352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc692f83-ccff-44bc-983c-8c65aa0a9a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:07.515656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc692f83-ccff-44bc-983c-8c65aa0a9a47 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:07.520163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eff0a3-bc1a-4f76-9668-a6288b80d9f4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:07.549178 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 954/1924] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:49:07.557001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc692f83-ccff-44bc-983c-8c65aa0a9a47 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:07.572505 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 971/1925] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:07 2022] POST /identity/v3/users => generated 312 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:07.576303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff0a3-bc1a-4f76-9668-a6288b80d9f4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:07.576676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc692f83-ccff-44bc-983c-8c65aa0a9a47 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:07.595351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff0a3-bc1a-4f76-9668-a6288b80d9f4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:07.602890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08d5d9-f26c-4149-82e3-73266e62cbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:07.603649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08d5d9-f26c-4149-82e3-73266e62cbca admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:07.604032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08d5d9-f26c-4149-82e3-73266e62cbca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:07.604411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08d5d9-f26c-4149-82e3-73266e62cbca admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:07.605207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08d5d9-f26c-4149-82e3-73266e62cbca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:07.608938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08d5d9-f26c-4149-82e3-73266e62cbca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:07.612711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64d066-78a1-46b4-a359-63ee9eb60fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:07.614503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64d066-78a1-46b4-a359-63ee9eb60fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:07.614810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64d066-78a1-46b4-a359-63ee9eb60fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:07.615087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64d066-78a1-46b4-a359-63ee9eb60fb6 admin admin] PATH_INFO: `/v3/projects/62478424b22c42c88c3d53b21bac76cc/users/7e4d521d64424a37ab5a12e58a752a1a/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:07.627554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64d066-78a1-46b4-a359-63ee9eb60fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62478424b22c42c88c3d53b21bac76cc, user_id=7e4d521d64424a37ab5a12e58a752a1a, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:07.635478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64d066-78a1-46b4-a359-63ee9eb60fb6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:07.637345 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 972/1926] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:07.642111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08d5d9-f26c-4149-82e3-73266e62cbca admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:07.681016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08d5d9-f26c-4149-82e3-73266e62cbca admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:07.702714 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 955/1927] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:07 2022] PUT /identity/v3/projects/62478424b22c42c88c3d53b21bac76cc/users/7e4d521d64424a37ab5a12e58a752a1a/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:07.705865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad375-6ab1-4325-8d77-956e9288bc5c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:07.706253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad375-6ab1-4325-8d77-956e9288bc5c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:07.706470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad375-6ab1-4325-8d77-956e9288bc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:07.813058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bd5ad375-6ab1-4325-8d77-956e9288bc5c None None] MFA Rules not processed for user `7e4d521d64424a37ab5a12e58a752a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:07.813058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bd5ad375-6ab1-4325-8d77-956e9288bc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:07.875510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a90c6-8deb-40ab-b090-614af8520647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:07.876166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a90c6-8deb-40ab-b090-614af8520647 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:07.877398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a90c6-8deb-40ab-b090-614af8520647 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:07.877712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a90c6-8deb-40ab-b090-614af8520647 admin admin] PATH_INFO: `/v3/projects/2aa40e24e7b7418eac62e7e71bee0b69/users/d798e3c137b8489fb4c0a66dcaa9fff6/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:07.945968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a90c6-8deb-40ab-b090-614af8520647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa40e24e7b7418eac62e7e71bee0b69, user_id=d798e3c137b8489fb4c0a66dcaa9fff6, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:07.965165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a90c6-8deb-40ab-b090-614af8520647 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:07.992903 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 973/1928] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:07 2022] PUT /identity/v3/projects/2aa40e24e7b7418eac62e7e71bee0b69/users/d798e3c137b8489fb4c0a66dcaa9fff6/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:07.994173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a90c6-8deb-40ab-b090-614af8520647 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:08.017608 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 956/1929] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:08.027994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad375-6ab1-4325-8d77-956e9288bc5c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:08.030603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bd5ad375-6ab1-4325-8d77-956e9288bc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:08.066943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:08.118165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a90c6-8deb-40ab-b090-614af8520647 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:08.123731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3082bd3-f53d-4346-a838-c160d2a24142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:08.124334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3082bd3-f53d-4346-a838-c160d2a24142 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:08.125150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3082bd3-f53d-4346-a838-c160d2a24142 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:08.125150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3082bd3-f53d-4346-a838-c160d2a24142 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:08.125853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3082bd3-f53d-4346-a838-c160d2a24142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:08.128065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3082bd3-f53d-4346-a838-c160d2a24142 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:08.152742 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 974/1930] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:08.154117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3082bd3-f53d-4346-a838-c160d2a24142 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:08.166971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3082bd3-f53d-4346-a838-c160d2a24142 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:08.181474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edd1f5-2c94-47e4-a348-eb11af197457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:08.182228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edd1f5-2c94-47e4-a348-eb11af197457 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:08.182571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edd1f5-2c94-47e4-a348-eb11af197457 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:08.182908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edd1f5-2c94-47e4-a348-eb11af197457 admin admin] PATH_INFO: `/v3/projects/2aa40e24e7b7418eac62e7e71bee0b69/users/d798e3c137b8489fb4c0a66dcaa9fff6/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:08.188611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edd1f5-2c94-47e4-a348-eb11af197457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa40e24e7b7418eac62e7e71bee0b69, user_id=d798e3c137b8489fb4c0a66dcaa9fff6, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:08.208885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edd1f5-2c94-47e4-a348-eb11af197457 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:08.222219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad375-6ab1-4325-8d77-956e9288bc5c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:08.247337 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 975/1931] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:08 2022] PUT /identity/v3/projects/2aa40e24e7b7418eac62e7e71bee0b69/users/d798e3c137b8489fb4c0a66dcaa9fff6/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:08.251418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefdb57-f9fe-4449-a76f-ab5f406efdc0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:08.253864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefdb57-f9fe-4449-a76f-ab5f406efdc0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:08.255316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefdb57-f9fe-4449-a76f-ab5f406efdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:08.343558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0cefdb57-f9fe-4449-a76f-ab5f406efdc0 None None] MFA Rules not processed for user `d798e3c137b8489fb4c0a66dcaa9fff6`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:08.347077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0cefdb57-f9fe-4449-a76f-ab5f406efdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:08.370050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08455cf1-86d8-499d-9b88-8704711a6216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:08.372699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455cf1-86d8-499d-9b88-8704711a6216 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:08.376574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455cf1-86d8-499d-9b88-8704711a6216 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:08.376999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455cf1-86d8-499d-9b88-8704711a6216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:08.426844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:08.530559 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 976/1932] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:08 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:08.537795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08455cf1-86d8-499d-9b88-8704711a6216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:08.539076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08455cf1-86d8-499d-9b88-8704711a6216 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:08.550813 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 957/1933] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:49:08.664868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefdb57-f9fe-4449-a76f-ab5f406efdc0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:08.682817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefdb57-f9fe-4449-a76f-ab5f406efdc0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:08.699473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06561d58-dc74-4f98-bea4-f73acaba9e90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:08.700100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06561d58-dc74-4f98-bea4-f73acaba9e90 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:08.700376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06561d58-dc74-4f98-bea4-f73acaba9e90 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:08.700642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06561d58-dc74-4f98-bea4-f73acaba9e90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:08.718813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06561d58-dc74-4f98-bea4-f73acaba9e90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:08.720904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06561d58-dc74-4f98-bea4-f73acaba9e90 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:08.747276 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 977/1934] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:08 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:08.787490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08455cf1-86d8-499d-9b88-8704711a6216 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:08.801420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08455cf1-86d8-499d-9b88-8704711a6216 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:08.807381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7daea-dcf9-42ef-a2c0-148ed92e51da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:08.807996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7daea-dcf9-42ef-a2c0-148ed92e51da service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:08.808411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7daea-dcf9-42ef-a2c0-148ed92e51da service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:08.808662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7daea-dcf9-42ef-a2c0-148ed92e51da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:08.810488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-83c7daea-dcf9-42ef-a2c0-148ed92e51da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:08.878823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:08.919236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7daea-dcf9-42ef-a2c0-148ed92e51da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:08.921087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7daea-dcf9-42ef-a2c0-148ed92e51da service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:08.945521 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 958/1935] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:09.069750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06561d58-dc74-4f98-bea4-f73acaba9e90 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:09.072734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-06561d58-dc74-4f98-bea4-f73acaba9e90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:09.093408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06561d58-dc74-4f98-bea4-f73acaba9e90 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:09.101500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1567d31-9c07-4206-83d4-0361fc4087f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:09.102449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567d31-9c07-4206-83d4-0361fc4087f2 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:09.102868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567d31-9c07-4206-83d4-0361fc4087f2 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:09.103341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567d31-9c07-4206-83d4-0361fc4087f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:09.195339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1567d31-9c07-4206-83d4-0361fc4087f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:09.198101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1567d31-9c07-4206-83d4-0361fc4087f2 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:09.242346 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 978/1936] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:09.283911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7daea-dcf9-42ef-a2c0-148ed92e51da service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:09.300325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7daea-dcf9-42ef-a2c0-148ed92e51da service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:09.307509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f0dec-1f95-4b05-bc4f-33bb89d9e065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:09.308143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f0dec-1f95-4b05-bc4f-33bb89d9e065 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:09.308401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f0dec-1f95-4b05-bc4f-33bb89d9e065 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:09.308946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f0dec-1f95-4b05-bc4f-33bb89d9e065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:09.431402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f0dec-1f95-4b05-bc4f-33bb89d9e065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:09.435875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f0dec-1f95-4b05-bc4f-33bb89d9e065 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:09.459361 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 959/1937] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:09 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:10.259424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1567d31-9c07-4206-83d4-0361fc4087f2 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:10.395590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1567d31-9c07-4206-83d4-0361fc4087f2 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:10.408415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:10.413349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:10.413722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:10.414068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] PATH_INFO: `/v3/users/bd2aa13e39d644319136fd102c55cb95` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:10.415549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd2aa13e39d644319136fd102c55cb95)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:10.420835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:10.437003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f7926-2d6c-4fcd-959e-04218df623b3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:10.437540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f7926-2d6c-4fcd-959e-04218df623b3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:10.437793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f7926-2d6c-4fcd-959e-04218df623b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:10.478893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-104f7926-2d6c-4fcd-959e-04218df623b3 None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:10.485373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-104f7926-2d6c-4fcd-959e-04218df623b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:10.574046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:10.586054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd2aa13e39d644319136fd102c55cb95'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:10.622732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd2aa13e39d644319136fd102c55cb95'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:10.639649 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 960/1938] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:10.646417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd2aa13e39d644319136fd102c55cb95'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:10.663853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd2aa13e39d644319136fd102c55cb95'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:10.668945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-23332d18-8687-4eec-a831-b6a031dbfadd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd2aa13e39d644319136fd102c55cb95'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:10.681634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f7926-2d6c-4fcd-959e-04218df623b3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:10.684092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-104f7926-2d6c-4fcd-959e-04218df623b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:10.726063 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 979/1939] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:10 2022] DELETE /identity/v3/users/bd2aa13e39d644319136fd102c55cb95 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:10.732458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27835be6-9e3b-4869-be64-63d0f55f85c7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:10.733780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27835be6-9e3b-4869-be64-63d0f55f85c7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:10.736592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27835be6-9e3b-4869-be64-63d0f55f85c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:10.743093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f7926-2d6c-4fcd-959e-04218df623b3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:10.791634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-27835be6-9e3b-4869-be64-63d0f55f85c7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:10.795013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-27835be6-9e3b-4869-be64-63d0f55f85c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:10.888172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97550a4-4879-4454-9b45-78e9af02e358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:10.889719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97550a4-4879-4454-9b45-78e9af02e358 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:10.892045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97550a4-4879-4454-9b45-78e9af02e358 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:10.892702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97550a4-4879-4454-9b45-78e9af02e358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:10.898114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a97550a4-4879-4454-9b45-78e9af02e358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:10.923221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97550a4-4879-4454-9b45-78e9af02e358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:10.925003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97550a4-4879-4454-9b45-78e9af02e358 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:10.955974 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 961/1940] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:10.964710 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 980/1941] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:11.024265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97550a4-4879-4454-9b45-78e9af02e358 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:11.026605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a97550a4-4879-4454-9b45-78e9af02e358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:11.047774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97550a4-4879-4454-9b45-78e9af02e358 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:11.067573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbe374-c198-466d-9507-09dde1472e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:11.068390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbe374-c198-466d-9507-09dde1472e05 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:11.069061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbe374-c198-466d-9507-09dde1472e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:11.069520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbe374-c198-466d-9507-09dde1472e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:11.105545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbe374-c198-466d-9507-09dde1472e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:11.107648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbe374-c198-466d-9507-09dde1472e05 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:11.144506 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 962/1942] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:11.185453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27835be6-9e3b-4869-be64-63d0f55f85c7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:11.187143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-27835be6-9e3b-4869-be64-63d0f55f85c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:11.205951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27835be6-9e3b-4869-be64-63d0f55f85c7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:11.213935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:11.214899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:11.215242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:11.215574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] PATH_INFO: `/v3/users/a62d07846dda4594a29a627c97cd236c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:11.228882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] RBAC: Authorizing `identity:delete_user(user_id=a62d07846dda4594a29a627c97cd236c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:11.236029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:11.365524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a62d07846dda4594a29a627c97cd236c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:11.405932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a62d07846dda4594a29a627c97cd236c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:11.427327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a62d07846dda4594a29a627c97cd236c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:11.452187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a62d07846dda4594a29a627c97cd236c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:11.453587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-46350e06-0a00-44bf-8d96-460a32f654de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a62d07846dda4594a29a627c97cd236c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:11.471200 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 981/1943] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:11 2022] DELETE /identity/v3/users/a62d07846dda4594a29a627c97cd236c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:11.477086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ede5a-6890-46bb-9e40-85d5e50b837a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:11.477577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ede5a-6890-46bb-9e40-85d5e50b837a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:11.477654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ede5a-6890-46bb-9e40-85d5e50b837a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:11.524706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d20ede5a-6890-46bb-9e40-85d5e50b837a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:11.525997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d20ede5a-6890-46bb-9e40-85d5e50b837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:11.548117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:11.558070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efcc37-aaa0-4739-b18d-fe5b49647f49 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:11.558750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efcc37-aaa0-4739-b18d-fe5b49647f49 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:11.559432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efcc37-aaa0-4739-b18d-fe5b49647f49 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:11.562143 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 982/1944] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:49:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:11.572088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f8b5b-be1a-424b-8bd8-54f6093eea96 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:11.572803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f8b5b-be1a-424b-8bd8-54f6093eea96 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:11.573325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f8b5b-be1a-424b-8bd8-54f6093eea96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:11.619836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3b0f8b5b-be1a-424b-8bd8-54f6093eea96 None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:11.633083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3b0f8b5b-be1a-424b-8bd8-54f6093eea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:11.731360 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 963/1945] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:11.748220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:11.751161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ede5a-6890-46bb-9e40-85d5e50b837a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:11.753802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d20ede5a-6890-46bb-9e40-85d5e50b837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:11.822231 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 983/1946] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:11.887440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f8b5b-be1a-424b-8bd8-54f6093eea96 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:11.896593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ede5a-6890-46bb-9e40-85d5e50b837a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:11.898289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f8b5b-be1a-424b-8bd8-54f6093eea96 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:11.907412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e66e-95dd-4c66-af0f-7a98fb566637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:11.908047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e66e-95dd-4c66-af0f-7a98fb566637 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:11.908306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e66e-95dd-4c66-af0f-7a98fb566637 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:11.908570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e66e-95dd-4c66-af0f-7a98fb566637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:11.909252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e915403-14ba-43db-b010-73aca80795a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:11.910081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e915403-14ba-43db-b010-73aca80795a0 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:11.910808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e915403-14ba-43db-b010-73aca80795a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:11.911270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e915403-14ba-43db-b010-73aca80795a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:11.930045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e915403-14ba-43db-b010-73aca80795a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:11.932944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e915403-14ba-43db-b010-73aca80795a0 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:11.936943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139e66e-95dd-4c66-af0f-7a98fb566637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:11.938775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139e66e-95dd-4c66-af0f-7a98fb566637 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:11.956250 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 984/1947] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:11.966521 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 964/1948] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:12.341527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e915403-14ba-43db-b010-73aca80795a0 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:12.343385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2e915403-14ba-43db-b010-73aca80795a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:12.523580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e66e-95dd-4c66-af0f-7a98fb566637 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:12.526253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8139e66e-95dd-4c66-af0f-7a98fb566637 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:12.573027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e66e-95dd-4c66-af0f-7a98fb566637 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:12.584628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:12.586943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:12.587527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:12.588734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] PATH_INFO: `/v3/projects/89ed78bfa690487c816835dd282e2970` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:12.591280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ed78bfa690487c816835dd282e2970)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:12.597330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:12.608329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e915403-14ba-43db-b010-73aca80795a0 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:12.623436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff334d30-273a-4da9-909e-be1bf9cbcd92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:12.624405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff334d30-273a-4da9-909e-be1bf9cbcd92 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:12.625305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff334d30-273a-4da9-909e-be1bf9cbcd92 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:12.626288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff334d30-273a-4da9-909e-be1bf9cbcd92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:12.633004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ff334d30-273a-4da9-909e-be1bf9cbcd92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:12.660134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff334d30-273a-4da9-909e-be1bf9cbcd92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:12.662177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff334d30-273a-4da9-909e-be1bf9cbcd92 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:12.699830 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 985/1949] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:12.712546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ed78bfa690487c816835dd282e2970', 'request_id': 'req-13e85d82-0f90-46c9-903b-e992f6e73a87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:12.738826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ed78bfa690487c816835dd282e2970', 'request_id': 'req-13e85d82-0f90-46c9-903b-e992f6e73a87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:12.753778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ed78bfa690487c816835dd282e2970', 'request_id': 'req-13e85d82-0f90-46c9-903b-e992f6e73a87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:12.766711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] The token cache is being invalidate because project 89ed78bfa690487c816835dd282e2970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:12.767259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:12.768111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13e85d82-0f90-46c9-903b-e992f6e73a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:12.775891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff334d30-273a-4da9-909e-be1bf9cbcd92 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:12.783252 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 965/1950] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:12 2022] DELETE /identity/v3/projects/89ed78bfa690487c816835dd282e2970 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:12.800483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff334d30-273a-4da9-909e-be1bf9cbcd92 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:12.930219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:12.931766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:12.934716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:12.935089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] PATH_INFO: `/v3/users/d798e3c137b8489fb4c0a66dcaa9fff6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:12.937709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d798e3c137b8489fb4c0a66dcaa9fff6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:12.942596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:12.989335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49baa872-5d74-4924-88dd-c648db32d92d None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:12.990145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49baa872-5d74-4924-88dd-c648db32d92d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:12.990695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49baa872-5d74-4924-88dd-c648db32d92d None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:12.993186 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 966/1951] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:49:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 20:49:13.009773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d2b2e-0bc2-46ab-8a33-4f36a2bd7732 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:13.010451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d2b2e-0bc2-46ab-8a33-4f36a2bd7732 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:13.010573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d2b2e-0bc2-46ab-8a33-4f36a2bd7732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:13.043710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-813d2b2e-0bc2-46ab-8a33-4f36a2bd7732 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:13.045221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-813d2b2e-0bc2-46ab-8a33-4f36a2bd7732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:13.057537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd798e3c137b8489fb4c0a66dcaa9fff6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:13.074860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:13.078953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd798e3c137b8489fb4c0a66dcaa9fff6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:13.115248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd798e3c137b8489fb4c0a66dcaa9fff6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:13.116401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:13.130532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd798e3c137b8489fb4c0a66dcaa9fff6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:13.133947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd798e3c137b8489fb4c0a66dcaa9fff6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:13.150546 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 986/1952] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:12 2022] DELETE /identity/v3/users/d798e3c137b8489fb4c0a66dcaa9fff6 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:13.181191 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 967/1953] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:13.226445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:13.229067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:13.335192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b68973-d7f3-4d9e-8daf-7ec52959ac38 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:13.339509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369801f-ee88-4df8-93a5-5751adbce552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:13.340249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369801f-ee88-4df8-93a5-5751adbce552 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:13.340587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369801f-ee88-4df8-93a5-5751adbce552 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:13.340900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369801f-ee88-4df8-93a5-5751adbce552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:13.458607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82dbaf-7aac-4739-b1c1-816dab26dd9a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:13.458860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82dbaf-7aac-4739-b1c1-816dab26dd9a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:13.459162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82dbaf-7aac-4739-b1c1-816dab26dd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:13.494297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369801f-ee88-4df8-93a5-5751adbce552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:13.495875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369801f-ee88-4df8-93a5-5751adbce552 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:13.511456 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 987/1954] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:13.515343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2d82dbaf-7aac-4739-b1c1-816dab26dd9a None None] MFA Rules not processed for user `c613201f55b94a528c6c36e5b5772799`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:13.516637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2d82dbaf-7aac-4739-b1c1-816dab26dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:13.542335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369801f-ee88-4df8-93a5-5751adbce552 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:13.544011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4369801f-ee88-4df8-93a5-5751adbce552 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:13.662003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369801f-ee88-4df8-93a5-5751adbce552 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:13.665811 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 968/1955] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:13.670182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82dbaf-7aac-4739-b1c1-816dab26dd9a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:13.688483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2d82dbaf-7aac-4739-b1c1-816dab26dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:13.693156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:13.693802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:13.694073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:13.694368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] PATH_INFO: `/v3/projects/22a57f99fc634af4b37a892143bf6665` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:13.698559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a57f99fc634af4b37a892143bf6665)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:13.702497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:13.740309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82dbaf-7aac-4739-b1c1-816dab26dd9a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:13.771557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53c991-7d83-483c-b295-b0fee0dfd155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:13.771842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:13.782121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53c991-7d83-483c-b295-b0fee0dfd155 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:13.783246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53c991-7d83-483c-b295-b0fee0dfd155 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:13.783698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53c991-7d83-483c-b295-b0fee0dfd155 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:13.785697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53c991-7d83-483c-b295-b0fee0dfd155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:13.791602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53c991-7d83-483c-b295-b0fee0dfd155 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:13.824680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a57f99fc634af4b37a892143bf6665', 'request_id': 'req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:13.835826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a57f99fc634af4b37a892143bf6665', 'request_id': 'req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:13.852454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a57f99fc634af4b37a892143bf6665', 'request_id': 'req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:13.863155 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 969/1956] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:13 2022] POST /identity/v3/projects => generated 369 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:49:13.872732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53c991-7d83-483c-b295-b0fee0dfd155 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:13.885911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] The token cache is being invalidate because project 22a57f99fc634af4b37a892143bf6665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:13.887042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:13.887873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:13.893599 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 988/1957] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:13 2022] DELETE /identity/v3/projects/22a57f99fc634af4b37a892143bf6665 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:49:13.897648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:13.900865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:14.049326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e7504-1aff-46d8-a537-cc4ed5bd6d48 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:14.054623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53c991-7d83-483c-b295-b0fee0dfd155 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:14.061372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:14.061584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.061845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.062103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] PATH_INFO: `/v3/users/7e4d521d64424a37ab5a12e58a752a1a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.063015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3c349-9169-484b-9d78-b002f6961b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:14.065594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3c349-9169-484b-9d78-b002f6961b7f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.068505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3c349-9169-484b-9d78-b002f6961b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.068505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3c349-9169-484b-9d78-b002f6961b7f admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.069229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3c349-9169-484b-9d78-b002f6961b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:14.069873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e4d521d64424a37ab5a12e58a752a1a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:14.072543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:14.076009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3c349-9169-484b-9d78-b002f6961b7f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:14.130696 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 970/1958] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:13 2022] POST /identity/v3/users => generated 291 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:14.133352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3c349-9169-484b-9d78-b002f6961b7f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:14.135743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1bf3c349-9169-484b-9d78-b002f6961b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:14.158739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3c349-9169-484b-9d78-b002f6961b7f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:14.176549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:14.177798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.178254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.178794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] PATH_INFO: `/v3/projects/2aa40e24e7b7418eac62e7e71bee0b69` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.180683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa40e24e7b7418eac62e7e71bee0b69)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:14.186907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:14.256299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e4d521d64424a37ab5a12e58a752a1a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:14.279517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e4d521d64424a37ab5a12e58a752a1a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:14.290765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:14.306863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e4d521d64424a37ab5a12e58a752a1a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:14.319825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e4d521d64424a37ab5a12e58a752a1a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:14.320543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e4d521d64424a37ab5a12e58a752a1a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:14.333152 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 989/1959] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:13 2022] DELETE /identity/v3/users/7e4d521d64424a37ab5a12e58a752a1a => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:14.334373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:14.336048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:14.336464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa40e24e7b7418eac62e7e71bee0b69', 'request_id': 'req-86883b23-6e70-4266-9e94-1c616a3ba387'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:14.352161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa40e24e7b7418eac62e7e71bee0b69', 'request_id': 'req-86883b23-6e70-4266-9e94-1c616a3ba387'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:14.361807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa40e24e7b7418eac62e7e71bee0b69', 'request_id': 'req-86883b23-6e70-4266-9e94-1c616a3ba387'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:14.389632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] The token cache is being invalidate because project 2aa40e24e7b7418eac62e7e71bee0b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:14.389925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:14.393895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-86883b23-6e70-4266-9e94-1c616a3ba387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:14.400912 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 971/1960] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:14 2022] DELETE /identity/v3/projects/2aa40e24e7b7418eac62e7e71bee0b69 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:14.402373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07858170-5a54-4e24-bee0-fcbce18de209 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.402649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07858170-5a54-4e24-bee0-fcbce18de209 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.402915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07858170-5a54-4e24-bee0-fcbce18de209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.446190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:14.455079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-07858170-5a54-4e24-bee0-fcbce18de209 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:14.461260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-07858170-5a54-4e24-bee0-fcbce18de209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:14.547019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5509eb-8660-4420-b955-2739da3c9044 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:14.560275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fc650-a3dd-4c2b-860e-90943cd1d910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:14.560957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fc650-a3dd-4c2b-860e-90943cd1d910 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.561302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fc650-a3dd-4c2b-860e-90943cd1d910 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.561571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fc650-a3dd-4c2b-860e-90943cd1d910 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.562223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9fc650-a3dd-4c2b-860e-90943cd1d910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:14.566452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9fc650-a3dd-4c2b-860e-90943cd1d910 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:14.587235 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 972/1961] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:14.597918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30646c16-62fe-4f71-a2c2-37a005a941c0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.601823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30646c16-62fe-4f71-a2c2-37a005a941c0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.602274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30646c16-62fe-4f71-a2c2-37a005a941c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.621380 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 990/1962] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:14.624974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fc650-a3dd-4c2b-860e-90943cd1d910 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:14.635247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9a9fc650-a3dd-4c2b-860e-90943cd1d910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:14.653387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-30646c16-62fe-4f71-a2c2-37a005a941c0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:14.654685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-30646c16-62fe-4f71-a2c2-37a005a941c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:14.666852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fc650-a3dd-4c2b-860e-90943cd1d910 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:14.669845 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 973/1963] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:14.673509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39366058-88bf-4caa-8fba-1ddc612b6041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:14.674180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39366058-88bf-4caa-8fba-1ddc612b6041 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.674455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39366058-88bf-4caa-8fba-1ddc612b6041 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.674996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30646c16-62fe-4f71-a2c2-37a005a941c0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:14.675125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39366058-88bf-4caa-8fba-1ddc612b6041 admin admin] PATH_INFO: `/v3/projects/1e95d86475074ada9ee35335e52b62dd/users/92d7ca8b7e1a4ca6acc797364290c607/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.698609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39366058-88bf-4caa-8fba-1ddc612b6041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e95d86475074ada9ee35335e52b62dd, user_id=92d7ca8b7e1a4ca6acc797364290c607, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:14.701663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39366058-88bf-4caa-8fba-1ddc612b6041 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:14.717954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30646c16-62fe-4f71-a2c2-37a005a941c0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:14.725179 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 991/1964] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:14 2022] PUT /identity/v3/projects/1e95d86475074ada9ee35335e52b62dd/users/92d7ca8b7e1a4ca6acc797364290c607/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:14.726972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39366058-88bf-4caa-8fba-1ddc612b6041 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:14.825240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39366058-88bf-4caa-8fba-1ddc612b6041 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:14.833331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c46e8a-aa0b-4b3c-83f4-295c46b70d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:14.834276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c46e8a-aa0b-4b3c-83f4-295c46b70d0c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.834697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c46e8a-aa0b-4b3c-83f4-295c46b70d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.835354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c46e8a-aa0b-4b3c-83f4-295c46b70d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.836656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c46e8a-aa0b-4b3c-83f4-295c46b70d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:14.839644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c46e8a-aa0b-4b3c-83f4-295c46b70d0c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:14.860201 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 992/1965] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:14.864960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c46e8a-aa0b-4b3c-83f4-295c46b70d0c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:14.899782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c46e8a-aa0b-4b3c-83f4-295c46b70d0c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:14.904653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168189c8-994e-43ee-bc2e-40d3aee6544a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:14.905970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168189c8-994e-43ee-bc2e-40d3aee6544a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.906334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168189c8-994e-43ee-bc2e-40d3aee6544a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.906568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168189c8-994e-43ee-bc2e-40d3aee6544a admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.907291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168189c8-994e-43ee-bc2e-40d3aee6544a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:14.907630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168189c8-994e-43ee-bc2e-40d3aee6544a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:14.909576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168189c8-994e-43ee-bc2e-40d3aee6544a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:14.926046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83346d-6d09-4625-b609-9d2a9a87069f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:14.929595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83346d-6d09-4625-b609-9d2a9a87069f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.929880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83346d-6d09-4625-b609-9d2a9a87069f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.930147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83346d-6d09-4625-b609-9d2a9a87069f admin admin] PATH_INFO: `/v3/projects/1e95d86475074ada9ee35335e52b62dd/users/92d7ca8b7e1a4ca6acc797364290c607/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.933471 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 974/1966] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:14.934591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168189c8-994e-43ee-bc2e-40d3aee6544a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:14.938686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83346d-6d09-4625-b609-9d2a9a87069f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e95d86475074ada9ee35335e52b62dd, user_id=92d7ca8b7e1a4ca6acc797364290c607, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:14.946531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83346d-6d09-4625-b609-9d2a9a87069f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:14.947357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168189c8-994e-43ee-bc2e-40d3aee6544a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:14.972443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bdf4f-5b32-449f-9c58-ee3bb1998585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:14.973884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bdf4f-5b32-449f-9c58-ee3bb1998585 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:14.975902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bdf4f-5b32-449f-9c58-ee3bb1998585 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:14.976339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bdf4f-5b32-449f-9c58-ee3bb1998585 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:14.978156 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 993/1967] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:14 2022] PUT /identity/v3/projects/1e95d86475074ada9ee35335e52b62dd/users/92d7ca8b7e1a4ca6acc797364290c607/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:14.979432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bdf4f-5b32-449f-9c58-ee3bb1998585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:14.982429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bdf4f-5b32-449f-9c58-ee3bb1998585 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:15.019557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83346d-6d09-4625-b609-9d2a9a87069f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:15.021609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f83346d-6d09-4625-b609-9d2a9a87069f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:15.040220 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 975/1968] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:14 2022] POST /identity/v3/projects => generated 401 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:15.045713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bdf4f-5b32-449f-9c58-ee3bb1998585 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:15.076723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:15.196083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bdf4f-5b32-449f-9c58-ee3bb1998585 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:15.204050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83346d-6d09-4625-b609-9d2a9a87069f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:15.209986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13594d-eaca-4c81-bb94-fb8dac1b9b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:15.211775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13594d-eaca-4c81-bb94-fb8dac1b9b4f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:15.212206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13594d-eaca-4c81-bb94-fb8dac1b9b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:15.212755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13594d-eaca-4c81-bb94-fb8dac1b9b4f admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:15.213658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:15.214084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13594d-eaca-4c81-bb94-fb8dac1b9b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:15.216187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:15.216594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:15.216967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] PATH_INFO: `/v3/projects/62478424b22c42c88c3d53b21bac76cc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:15.217781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13594d-eaca-4c81-bb94-fb8dac1b9b4f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:15.222834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=62478424b22c42c88c3d53b21bac76cc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:15.225835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:15.259214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 976/1969] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:15 2022] POST /identity/v3/users => generated 301 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:15.260147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13594d-eaca-4c81-bb94-fb8dac1b9b4f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:15.271602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13594d-eaca-4c81-bb94-fb8dac1b9b4f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:15.277601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb158a19-65d7-40e3-963e-4081e0d5d179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:15.278230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb158a19-65d7-40e3-963e-4081e0d5d179 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:15.278542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb158a19-65d7-40e3-963e-4081e0d5d179 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:15.279064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb158a19-65d7-40e3-963e-4081e0d5d179 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:15.279686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb158a19-65d7-40e3-963e-4081e0d5d179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:15.282119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb158a19-65d7-40e3-963e-4081e0d5d179 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:15.293452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:15.320056 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 977/1970] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:15.323518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb158a19-65d7-40e3-963e-4081e0d5d179 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:15.339276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb158a19-65d7-40e3-963e-4081e0d5d179 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:15.346053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62478424b22c42c88c3d53b21bac76cc', 'request_id': 'req-b2df05c9-8de9-4c46-9abd-56f08b113b3a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:15.365182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62478424b22c42c88c3d53b21bac76cc', 'request_id': 'req-b2df05c9-8de9-4c46-9abd-56f08b113b3a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:15.383121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62478424b22c42c88c3d53b21bac76cc', 'request_id': 'req-b2df05c9-8de9-4c46-9abd-56f08b113b3a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:15.406413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] The token cache is being invalidate because project 62478424b22c42c88c3d53b21bac76cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:15.406413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:15.407634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b2df05c9-8de9-4c46-9abd-56f08b113b3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:15.410489 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 994/1971] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:15 2022] DELETE /identity/v3/projects/62478424b22c42c88c3d53b21bac76cc => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:15.415281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ece00f-cc76-4639-a3b5-ab522ce96347 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:15.415557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ece00f-cc76-4639-a3b5-ab522ce96347 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:15.415838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ece00f-cc76-4639-a3b5-ab522ce96347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:15.476444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-26ece00f-cc76-4639-a3b5-ab522ce96347 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:15.480438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-26ece00f-cc76-4639-a3b5-ab522ce96347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:15.493958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51ea7b-60e4-4067-ad55-5733c508ee0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:15.494590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51ea7b-60e4-4067-ad55-5733c508ee0c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:15.495460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51ea7b-60e4-4067-ad55-5733c508ee0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:15.495460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51ea7b-60e4-4067-ad55-5733c508ee0c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:15.495757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51ea7b-60e4-4067-ad55-5733c508ee0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:15.497470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51ea7b-60e4-4067-ad55-5733c508ee0c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:15.502330 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 995/1972] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:15.511276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4508f03-dbd7-4947-bd67-af3012ac6496 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:15.511679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4508f03-dbd7-4947-bd67-af3012ac6496 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:15.511845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4508f03-dbd7-4947-bd67-af3012ac6496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:15.526602 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 978/1973] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:15.529400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51ea7b-60e4-4067-ad55-5733c508ee0c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:15.536715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e51ea7b-60e4-4067-ad55-5733c508ee0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:15.551815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e4508f03-dbd7-4947-bd67-af3012ac6496 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:15.553169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e4508f03-dbd7-4947-bd67-af3012ac6496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:15.563175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51ea7b-60e4-4067-ad55-5733c508ee0c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:15.581186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023113e3-9176-4185-9378-6e0194594586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:15.581186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023113e3-9176-4185-9378-6e0194594586 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:15.581186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023113e3-9176-4185-9378-6e0194594586 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:15.581186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023113e3-9176-4185-9378-6e0194594586 admin admin] PATH_INFO: `/v3/projects/f9e809ddc6b7418485892059e70926f4/users/d1929f640b7348d6901067dfa6354c49/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:15.587380 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 996/1974] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:15.590753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4508f03-dbd7-4947-bd67-af3012ac6496 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:15.603043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4508f03-dbd7-4947-bd67-af3012ac6496 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:15.609316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb588c-49c6-4d19-ab1b-afbc6c8bc639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:15.610032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb588c-49c6-4d19-ab1b-afbc6c8bc639 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:15.610293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb588c-49c6-4d19-ab1b-afbc6c8bc639 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:15.610564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb588c-49c6-4d19-ab1b-afbc6c8bc639 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:15.611211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb588c-49c6-4d19-ab1b-afbc6c8bc639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:15.611507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb588c-49c6-4d19-ab1b-afbc6c8bc639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:15.613677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023113e3-9176-4185-9378-6e0194594586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e809ddc6b7418485892059e70926f4, user_id=d1929f640b7348d6901067dfa6354c49, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:15.615532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb588c-49c6-4d19-ab1b-afbc6c8bc639 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:15.618910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023113e3-9176-4185-9378-6e0194594586 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:15.636544 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 997/1975] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:49:15.638709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb588c-49c6-4d19-ab1b-afbc6c8bc639 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:15.652464 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 979/1976] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:15 2022] PUT /identity/v3/projects/f9e809ddc6b7418485892059e70926f4/users/d1929f640b7348d6901067dfa6354c49/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:15.653554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb588c-49c6-4d19-ab1b-afbc6c8bc639 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:15.660649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71eac4-1494-47fc-962a-5346d0b6f566 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:15.661264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71eac4-1494-47fc-962a-5346d0b6f566 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:15.661510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71eac4-1494-47fc-962a-5346d0b6f566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:15.725284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9f71eac4-1494-47fc-962a-5346d0b6f566 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:15.726496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9f71eac4-1494-47fc-962a-5346d0b6f566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:15.740071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa0243-0124-4a2e-bcba-fd70a8c3879e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:15.741980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa0243-0124-4a2e-bcba-fd70a8c3879e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:15.741980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa0243-0124-4a2e-bcba-fd70a8c3879e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:15.741980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa0243-0124-4a2e-bcba-fd70a8c3879e admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:15.742580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa0243-0124-4a2e-bcba-fd70a8c3879e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:15.745508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa0243-0124-4a2e-bcba-fd70a8c3879e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:15.827064 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 998/1977] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:15 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:15.830015 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 980/1978] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:15.833653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa0243-0124-4a2e-bcba-fd70a8c3879e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:15.843618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71eac4-1494-47fc-962a-5346d0b6f566 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:15.845688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9f71eac4-1494-47fc-962a-5346d0b6f566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:15.864825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:15.994240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa0243-0124-4a2e-bcba-fd70a8c3879e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:15.999650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519c62b-f0b4-494b-a1fa-b4b06b994f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:16.000449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519c62b-f0b4-494b-a1fa-b4b06b994f18 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:16.000750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519c62b-f0b4-494b-a1fa-b4b06b994f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:16.001016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519c62b-f0b4-494b-a1fa-b4b06b994f18 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:16.001665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519c62b-f0b4-494b-a1fa-b4b06b994f18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:16.004403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519c62b-f0b4-494b-a1fa-b4b06b994f18 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:16.030308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71eac4-1494-47fc-962a-5346d0b6f566 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:16.033209 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 999/1979] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:15 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:16.040385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519c62b-f0b4-494b-a1fa-b4b06b994f18 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:16.047172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:16.058990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519c62b-f0b4-494b-a1fa-b4b06b994f18 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:16.078151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cf6c9-16de-4c40-860d-620aaf4e8fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:16.078379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cf6c9-16de-4c40-860d-620aaf4e8fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:16.078583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cf6c9-16de-4c40-860d-620aaf4e8fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:16.078922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cf6c9-16de-4c40-860d-620aaf4e8fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:16.079579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cf6c9-16de-4c40-860d-620aaf4e8fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:16.081447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cf6c9-16de-4c40-860d-620aaf4e8fd9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:16.103554 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1000/1980] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:16.107855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cf6c9-16de-4c40-860d-620aaf4e8fd9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:16.120973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd0c9f-efba-4986-b93d-ec6590b258ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:16.122179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0c9f-efba-4986-b93d-ec6590b258ab service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:16.122503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cf6c9-16de-4c40-860d-620aaf4e8fd9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:16.122797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0c9f-efba-4986-b93d-ec6590b258ab service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:16.123447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0c9f-efba-4986-b93d-ec6590b258ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:16.127766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e514eaf-21dd-4680-9379-be959a3f3327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:16.128355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e514eaf-21dd-4680-9379-be959a3f3327 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:16.128626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e514eaf-21dd-4680-9379-be959a3f3327 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:16.128903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e514eaf-21dd-4680-9379-be959a3f3327 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:16.129580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e514eaf-21dd-4680-9379-be959a3f3327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:16.131173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e514eaf-21dd-4680-9379-be959a3f3327 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:16.141558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd0c9f-efba-4986-b93d-ec6590b258ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:16.144519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd0c9f-efba-4986-b93d-ec6590b258ab service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:16.147302 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1001/1981] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:49:16.152054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e514eaf-21dd-4680-9379-be959a3f3327 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:16.161537 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 981/1982] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:49:16.167301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e514eaf-21dd-4680-9379-be959a3f3327 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:16.172940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44da702-5fe6-481f-bb64-567c97dff90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:16.173729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44da702-5fe6-481f-bb64-567c97dff90d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:16.174235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44da702-5fe6-481f-bb64-567c97dff90d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:16.175389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44da702-5fe6-481f-bb64-567c97dff90d admin admin] PATH_INFO: `/v3/projects/60259bf80cc041a286453bc99293125c/users/75281761f21648b8a1e2f6e9890fe72c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:16.213296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44da702-5fe6-481f-bb64-567c97dff90d admin admin] RBAC: Authorizing `identity:create_grant(project_id=60259bf80cc041a286453bc99293125c, user_id=75281761f21648b8a1e2f6e9890fe72c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:16.217066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44da702-5fe6-481f-bb64-567c97dff90d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:16.238984 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1002/1983] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:16 2022] PUT /identity/v3/projects/60259bf80cc041a286453bc99293125c/users/75281761f21648b8a1e2f6e9890fe72c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:49:16.251333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88441063-2f5f-4482-9f85-9e988f821bb0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:16.251863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88441063-2f5f-4482-9f85-9e988f821bb0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:16.253984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88441063-2f5f-4482-9f85-9e988f821bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:16.294211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-88441063-2f5f-4482-9f85-9e988f821bb0 None None] MFA Rules not processed for user `75281761f21648b8a1e2f6e9890fe72c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:16.295752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-88441063-2f5f-4482-9f85-9e988f821bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:16.416426 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 982/1984] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:16.421706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44da702-5fe6-481f-bb64-567c97dff90d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:16.512542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44da702-5fe6-481f-bb64-567c97dff90d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:16.520284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b378ef-be74-4f6c-8c46-1f00b81b699c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:16.521056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b378ef-be74-4f6c-8c46-1f00b81b699c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:16.521837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b378ef-be74-4f6c-8c46-1f00b81b699c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:16.521956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b378ef-be74-4f6c-8c46-1f00b81b699c admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:16.522914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b378ef-be74-4f6c-8c46-1f00b81b699c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:16.525720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b378ef-be74-4f6c-8c46-1f00b81b699c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:16.568878 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1003/1985] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:16 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:49:16.572577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88441063-2f5f-4482-9f85-9e988f821bb0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:16.590611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88441063-2f5f-4482-9f85-9e988f821bb0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:16.682949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a60e8d-99ef-4c52-aa1a-1b80a016178c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:16.683886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a60e8d-99ef-4c52-aa1a-1b80a016178c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:16.684262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a60e8d-99ef-4c52-aa1a-1b80a016178c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:16.684626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a60e8d-99ef-4c52-aa1a-1b80a016178c admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:16.685506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a60e8d-99ef-4c52-aa1a-1b80a016178c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:16.687787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a60e8d-99ef-4c52-aa1a-1b80a016178c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:16.698978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:16.735777 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 983/1986] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:16 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:16.740465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b378ef-be74-4f6c-8c46-1f00b81b699c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:16.754895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b378ef-be74-4f6c-8c46-1f00b81b699c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:16.771707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e26dc7-5106-4395-9100-a47a086b1777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:16.772550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26dc7-5106-4395-9100-a47a086b1777 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:16.773176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26dc7-5106-4395-9100-a47a086b1777 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:16.773620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26dc7-5106-4395-9100-a47a086b1777 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:16.774643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e26dc7-5106-4395-9100-a47a086b1777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:16.779728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e26dc7-5106-4395-9100-a47a086b1777 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:16.813146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a60e8d-99ef-4c52-aa1a-1b80a016178c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:16.813687 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1004/1987] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:16.833116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a60e8d-99ef-4c52-aa1a-1b80a016178c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:16.840061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a824db-b3ad-4a83-a69e-191a81b8ad7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:16.841005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a824db-b3ad-4a83-a69e-191a81b8ad7c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:16.841641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a824db-b3ad-4a83-a69e-191a81b8ad7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:16.842353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a824db-b3ad-4a83-a69e-191a81b8ad7c admin admin] PATH_INFO: `/v3/projects/c62d63a04c2440c48068d825b7378b24/users/1b8b091703d545819b1e020a96850ec9/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:16.879987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a824db-b3ad-4a83-a69e-191a81b8ad7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62d63a04c2440c48068d825b7378b24, user_id=1b8b091703d545819b1e020a96850ec9, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:16.886733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a824db-b3ad-4a83-a69e-191a81b8ad7c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:16.942172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e26dc7-5106-4395-9100-a47a086b1777 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:16.942480 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 984/1988] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:16 2022] PUT /identity/v3/projects/c62d63a04c2440c48068d825b7378b24/users/1b8b091703d545819b1e020a96850ec9/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:49:16.958986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e26dc7-5106-4395-9100-a47a086b1777 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:17.110636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecdb7a-1ef6-4693-a532-8fdb367af156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:17.111544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecdb7a-1ef6-4693-a532-8fdb367af156 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:17.112564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecdb7a-1ef6-4693-a532-8fdb367af156 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:17.112986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecdb7a-1ef6-4693-a532-8fdb367af156 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:17.113804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecdb7a-1ef6-4693-a532-8fdb367af156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:17.115845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecdb7a-1ef6-4693-a532-8fdb367af156 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:17.135914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a824db-b3ad-4a83-a69e-191a81b8ad7c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:17.137187 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1005/1989] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:17.149983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a824db-b3ad-4a83-a69e-191a81b8ad7c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:17.156187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799309f4-8cbc-471a-b041-6746463fa889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:17.156858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799309f4-8cbc-471a-b041-6746463fa889 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:17.157160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799309f4-8cbc-471a-b041-6746463fa889 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:17.157428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799309f4-8cbc-471a-b041-6746463fa889 admin admin] PATH_INFO: `/v3/projects/c62d63a04c2440c48068d825b7378b24/users/1b8b091703d545819b1e020a96850ec9/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:17.160164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799309f4-8cbc-471a-b041-6746463fa889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62d63a04c2440c48068d825b7378b24, user_id=1b8b091703d545819b1e020a96850ec9, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:17.163741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799309f4-8cbc-471a-b041-6746463fa889 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:17.198917 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 985/1990] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:17 2022] PUT /identity/v3/projects/c62d63a04c2440c48068d825b7378b24/users/1b8b091703d545819b1e020a96850ec9/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:49:17.209306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de34a7-49a9-4928-bfdd-a7a1adb9d20b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:17.209827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de34a7-49a9-4928-bfdd-a7a1adb9d20b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:17.210394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de34a7-49a9-4928-bfdd-a7a1adb9d20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:17.260914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b8de34a7-49a9-4928-bfdd-a7a1adb9d20b None None] MFA Rules not processed for user `1b8b091703d545819b1e020a96850ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:17.263009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b8de34a7-49a9-4928-bfdd-a7a1adb9d20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:17.284937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:17.467980 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1006/1991] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:17.475711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799309f4-8cbc-471a-b041-6746463fa889 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:17.503862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799309f4-8cbc-471a-b041-6746463fa889 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:17.510644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7e702-21f9-4bc9-ac15-d6a22cb06b9c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:17.511343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7e702-21f9-4bc9-ac15-d6a22cb06b9c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:17.511872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7e702-21f9-4bc9-ac15-d6a22cb06b9c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:17.512824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7e702-21f9-4bc9-ac15-d6a22cb06b9c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:17.513559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7e702-21f9-4bc9-ac15-d6a22cb06b9c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:17.514380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7e702-21f9-4bc9-ac15-d6a22cb06b9c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:17.516152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7e702-21f9-4bc9-ac15-d6a22cb06b9c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:17.561597 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 986/1992] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:17.566575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de34a7-49a9-4928-bfdd-a7a1adb9d20b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:17.587591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de34a7-49a9-4928-bfdd-a7a1adb9d20b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:17.613526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e1a7e-b9fe-4800-9ec5-3f72836528d8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:17.614648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e1a7e-b9fe-4800-9ec5-3f72836528d8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:17.615195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e1a7e-b9fe-4800-9ec5-3f72836528d8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:17.615689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e1a7e-b9fe-4800-9ec5-3f72836528d8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:17.616390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e1a7e-b9fe-4800-9ec5-3f72836528d8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:17.627059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e1a7e-b9fe-4800-9ec5-3f72836528d8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:17.739947 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1007/1993] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:17 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:17.741572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7e702-21f9-4bc9-ac15-d6a22cb06b9c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:17.756706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7e702-21f9-4bc9-ac15-d6a22cb06b9c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:17.893116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f997bd-723e-4fc8-a4ec-33ecff04032e tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:17.894198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f997bd-723e-4fc8-a4ec-33ecff04032e tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:17.894868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f997bd-723e-4fc8-a4ec-33ecff04032e tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:17.894868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f997bd-723e-4fc8-a4ec-33ecff04032e tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:17.895829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f997bd-723e-4fc8-a4ec-33ecff04032e tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:17.896243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f997bd-723e-4fc8-a4ec-33ecff04032e tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:17.898990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f997bd-723e-4fc8-a4ec-33ecff04032e tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:17.922376 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 987/1994] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:17.931382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e1a7e-b9fe-4800-9ec5-3f72836528d8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:17.934790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e9e1a7e-b9fe-4800-9ec5-3f72836528d8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:18.093991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e1a7e-b9fe-4800-9ec5-3f72836528d8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:18.101398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c1e1a-6c54-4482-86a5-1f69197fc69b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:18.102028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c1e1a-6c54-4482-86a5-1f69197fc69b service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:18.102301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c1e1a-6c54-4482-86a5-1f69197fc69b service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:18.102572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c1e1a-6c54-4482-86a5-1f69197fc69b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:18.119300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c1e1a-6c54-4482-86a5-1f69197fc69b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:18.120954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c1e1a-6c54-4482-86a5-1f69197fc69b service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:18.146902 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1008/1995] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:18.178020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063f4ac-2a02-416a-bd5e-ffdd4b276140 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:18.179268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063f4ac-2a02-416a-bd5e-ffdd4b276140 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:18.179553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063f4ac-2a02-416a-bd5e-ffdd4b276140 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:18.181967 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 988/1996] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:49:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:18.189417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c1e1a-6c54-4482-86a5-1f69197fc69b service nova] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:49:18.229646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c1e1a-6c54-4482-86a5-1f69197fc69b service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:18.244876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c1e1a-6c54-4482-86a5-1f69197fc69b service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:18.251551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027390d-cd8e-45cf-a5f9-7f3d50696dc8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:18.252620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027390d-cd8e-45cf-a5f9-7f3d50696dc8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:18.253562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027390d-cd8e-45cf-a5f9-7f3d50696dc8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:18.253972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027390d-cd8e-45cf-a5f9-7f3d50696dc8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/projects/d6d1303ef7b949da9374cf07859ccfd4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:18.255356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027390d-cd8e-45cf-a5f9-7f3d50696dc8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:get_project(project_id=d6d1303ef7b949da9374cf07859ccfd4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:18.274227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027390d-cd8e-45cf-a5f9-7f3d50696dc8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:18.282320 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1009/1997] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:49:18 2022] GET /identity/v3/projects/d6d1303ef7b949da9374cf07859ccfd4 => generated 369 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:18.665536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea585a-4e07-448b-8af1-4a938b595b12 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:18.665930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea585a-4e07-448b-8af1-4a938b595b12 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:18.670656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea585a-4e07-448b-8af1-4a938b595b12 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:18.672765 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 989/1998] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:49:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:18.681086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027390d-cd8e-45cf-a5f9-7f3d50696dc8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:49:18.683672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5027390d-cd8e-45cf-a5f9-7f3d50696dc8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:18.711071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027390d-cd8e-45cf-a5f9-7f3d50696dc8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:18.734227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027390d-cd8e-45cf-a5f9-7f3d50696dc8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:18.748491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198915d-f3e9-463f-8238-5cd0936c5c0f tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:18.749368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198915d-f3e9-463f-8238-5cd0936c5c0f tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:18.749758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198915d-f3e9-463f-8238-5cd0936c5c0f tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:18.750821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198915d-f3e9-463f-8238-5cd0936c5c0f tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/projects/d6d1303ef7b949da9374cf07859ccfd4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:18.754028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198915d-f3e9-463f-8238-5cd0936c5c0f tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:get_project(project_id=d6d1303ef7b949da9374cf07859ccfd4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:18.760451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198915d-f3e9-463f-8238-5cd0936c5c0f tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:18.762598 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1010/1999] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:49:18 2022] GET /identity/v3/projects/d6d1303ef7b949da9374cf07859ccfd4 => generated 369 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:18.867375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea585a-4e07-448b-8af1-4a938b595b12 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:18.887726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea585a-4e07-448b-8af1-4a938b595b12 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:18.899850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044a4ab-f842-495f-a119-32e0b5666cf4 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:18.900911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044a4ab-f842-495f-a119-32e0b5666cf4 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:18.901599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044a4ab-f842-495f-a119-32e0b5666cf4 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:18.902100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044a4ab-f842-495f-a119-32e0b5666cf4 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:18.903024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044a4ab-f842-495f-a119-32e0b5666cf4 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:18.903561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044a4ab-f842-495f-a119-32e0b5666cf4 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:18.907188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044a4ab-f842-495f-a119-32e0b5666cf4 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:18.937820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198915d-f3e9-463f-8238-5cd0936c5c0f tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:18.947511 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 990/2000] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:18.961571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198915d-f3e9-463f-8238-5cd0936c5c0f tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:18.969891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d7c6a-d83f-47ca-96b5-9b1f99e77637 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:18.972030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d7c6a-d83f-47ca-96b5-9b1f99e77637 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:18.972647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d7c6a-d83f-47ca-96b5-9b1f99e77637 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:18.973406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d7c6a-d83f-47ca-96b5-9b1f99e77637 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:18.975110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d7c6a-d83f-47ca-96b5-9b1f99e77637 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:18.980599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d7c6a-d83f-47ca-96b5-9b1f99e77637 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:19.020897 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1011/2001] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:18 2022] POST /identity/v3/users => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:19.025190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044a4ab-f842-495f-a119-32e0b5666cf4 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:19.047144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044a4ab-f842-495f-a119-32e0b5666cf4 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:19.061722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67a965-76a0-4aeb-b421-f9ad1f3711dc tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:19.062718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67a965-76a0-4aeb-b421-f9ad1f3711dc tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:19.063140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67a965-76a0-4aeb-b421-f9ad1f3711dc tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:19.063698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67a965-76a0-4aeb-b421-f9ad1f3711dc tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:19.064527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67a965-76a0-4aeb-b421-f9ad1f3711dc tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:19.065616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67a965-76a0-4aeb-b421-f9ad1f3711dc tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:19.067971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67a965-76a0-4aeb-b421-f9ad1f3711dc tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:19.084835 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 991/2002] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:19.609722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421a86a-f9f5-4eee-920c-2d8ca2c7d10d None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:19.610010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421a86a-f9f5-4eee-920c-2d8ca2c7d10d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:19.610286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421a86a-f9f5-4eee-920c-2d8ca2c7d10d None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:19.611729 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1012/2003] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:19.623857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67a965-76a0-4aeb-b421-f9ad1f3711dc tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:49:19.646399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67a965-76a0-4aeb-b421-f9ad1f3711dc tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:19.693106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67a965-76a0-4aeb-b421-f9ad1f3711dc tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:19.707446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b1be9-e7e2-4119-bab7-624645bb70ee tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:19.708380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b1be9-e7e2-4119-bab7-624645bb70ee tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:19.708843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b1be9-e7e2-4119-bab7-624645bb70ee tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:19.709334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b1be9-e7e2-4119-bab7-624645bb70ee tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/projects/d6d1303ef7b949da9374cf07859ccfd4` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:19.710707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b1be9-e7e2-4119-bab7-624645bb70ee tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:get_project(project_id=d6d1303ef7b949da9374cf07859ccfd4)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:19.713946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b1be9-e7e2-4119-bab7-624645bb70ee tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:19.716284 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 992/2004] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:49:19 2022] GET /identity/v3/projects/d6d1303ef7b949da9374cf07859ccfd4 => generated 369 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:19.904400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421a86a-f9f5-4eee-920c-2d8ca2c7d10d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:19.927508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421a86a-f9f5-4eee-920c-2d8ca2c7d10d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:19.933174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d4cde-5d05-4c3e-87a7-397447b35384 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:19.933842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4cde-5d05-4c3e-87a7-397447b35384 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:19.934123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4cde-5d05-4c3e-87a7-397447b35384 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:19.934388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4cde-5d05-4c3e-87a7-397447b35384 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:19.936265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-109d4cde-5d05-4c3e-87a7-397447b35384 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:20.028170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:20.102908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d4cde-5d05-4c3e-87a7-397447b35384 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:20.104613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d4cde-5d05-4c3e-87a7-397447b35384 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:20.165681 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1013/2005] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:20.567168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ff1ec-1c1f-4fe1-abee-bb8821ffe117 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:20.568013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ff1ec-1c1f-4fe1-abee-bb8821ffe117 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:20.568454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ff1ec-1c1f-4fe1-abee-bb8821ffe117 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:20.570336 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 993/2006] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:49:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:20.580078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d4cde-5d05-4c3e-87a7-397447b35384 service nova] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:49:20.607657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d4cde-5d05-4c3e-87a7-397447b35384 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:20.649241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d4cde-5d05-4c3e-87a7-397447b35384 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:20.664360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51160-4f32-453b-976b-799b0a9959a5 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:20.665014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51160-4f32-453b-976b-799b0a9959a5 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:20.670866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51160-4f32-453b-976b-799b0a9959a5 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:20.671163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51160-4f32-453b-976b-799b0a9959a5 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/projects/d6d1303ef7b949da9374cf07859ccfd4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:20.673793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d51160-4f32-453b-976b-799b0a9959a5 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:get_project(project_id=d6d1303ef7b949da9374cf07859ccfd4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:20.676728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d51160-4f32-453b-976b-799b0a9959a5 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:20.692144 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1014/2007] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:49:20 2022] GET /identity/v3/projects/d6d1303ef7b949da9374cf07859ccfd4 => generated 369 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:20.822929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ff1ec-1c1f-4fe1-abee-bb8821ffe117 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:20.851915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ff1ec-1c1f-4fe1-abee-bb8821ffe117 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:20.865649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:20.866312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:20.866590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:20.866856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/users/dd8d07607d68445a81bbcd3b3c2dfe2b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:20.892433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:delete_user(user_id=dd8d07607d68445a81bbcd3b3c2dfe2b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:20.895765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:20.950571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=5,keystone:SELECT=89,keystone:DELETE=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:20.959490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd8d07607d68445a81bbcd3b3c2dfe2b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:20.971787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd8d07607d68445a81bbcd3b3c2dfe2b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:20.992121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd8d07607d68445a81bbcd3b3c2dfe2b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.008943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd8d07607d68445a81bbcd3b3c2dfe2b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.010530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd8d07607d68445a81bbcd3b3c2dfe2b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.024271 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 994/2008] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:20 2022] DELETE /identity/v3/users/dd8d07607d68445a81bbcd3b3c2dfe2b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:21.025560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51160-4f32-453b-976b-799b0a9959a5 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:21.027620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-27d51160-4f32-453b-976b-799b0a9959a5 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:21.051315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51160-4f32-453b-976b-799b0a9959a5 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:21.198009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b8b091703d545819b1e020a96850ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62d63a04c2440c48068d825b7378b24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:21.198833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:21.199245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:21.199649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] PATH_INFO: `/v3/projects/d6d1303ef7b949da9374cf07859ccfd4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:21.201083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d6d1303ef7b949da9374cf07859ccfd4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:21.203215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:21.307477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6d1303ef7b949da9374cf07859ccfd4', 'request_id': 'req-39be6045-fde0-4385-97dd-04a4d3f489a8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.317636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6d1303ef7b949da9374cf07859ccfd4', 'request_id': 'req-39be6045-fde0-4385-97dd-04a4d3f489a8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.326981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6d1303ef7b949da9374cf07859ccfd4', 'request_id': 'req-39be6045-fde0-4385-97dd-04a4d3f489a8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.344934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] The token cache is being invalidate because project d6d1303ef7b949da9374cf07859ccfd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:21.345354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.346745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39be6045-fde0-4385-97dd-04a4d3f489a8 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.348835 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1015/2009] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:21 2022] DELETE /identity/v3/projects/d6d1303ef7b949da9374cf07859ccfd4 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:21.354750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:21.357465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:21.494507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e951b-cba7-438d-805d-919e3d94ffc1 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:21.666933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:21.668807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:21.670275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:21.670859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] PATH_INFO: `/v3/users/75281761f21648b8a1e2f6e9890fe72c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:21.675985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75281761f21648b8a1e2f6e9890fe72c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:21.684425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:21.851680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75281761f21648b8a1e2f6e9890fe72c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.863899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75281761f21648b8a1e2f6e9890fe72c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.876358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:21.889744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75281761f21648b8a1e2f6e9890fe72c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.903608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75281761f21648b8a1e2f6e9890fe72c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.904600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75281761f21648b8a1e2f6e9890fe72c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:21.914274 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 995/2010] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:21 2022] DELETE /identity/v3/users/75281761f21648b8a1e2f6e9890fe72c => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:21.918696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bded33a-82d3-4ca2-89e3-2c35922b27c4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:21.920012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bded33a-82d3-4ca2-89e3-2c35922b27c4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:21.920012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bded33a-82d3-4ca2-89e3-2c35922b27c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:21.984505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9bded33a-82d3-4ca2-89e3-2c35922b27c4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:21.986334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9bded33a-82d3-4ca2-89e3-2c35922b27c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:22.099959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:22.136117 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1016/2011] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:22.233309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:22.236011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:22.373332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaf380-a9f1-444c-a6ac-2cec73e7d730 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:22.378812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20796ed4-6cbc-4b43-8139-7c6fe0eb0331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:22.379890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20796ed4-6cbc-4b43-8139-7c6fe0eb0331 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:22.380386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20796ed4-6cbc-4b43-8139-7c6fe0eb0331 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:22.380866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20796ed4-6cbc-4b43-8139-7c6fe0eb0331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:22.434326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20796ed4-6cbc-4b43-8139-7c6fe0eb0331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:22.435866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20796ed4-6cbc-4b43-8139-7c6fe0eb0331 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:22.459800 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 996/2012] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:23.319281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bded33a-82d3-4ca2-89e3-2c35922b27c4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:23.321497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9bded33a-82d3-4ca2-89e3-2c35922b27c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:23.341573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20796ed4-6cbc-4b43-8139-7c6fe0eb0331 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:23.343641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-20796ed4-6cbc-4b43-8139-7c6fe0eb0331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:23.343913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bded33a-82d3-4ca2-89e3-2c35922b27c4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:23.360067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:23.362714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:23.363097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:23.364305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] PATH_INFO: `/v3/users/c5a178b6a74140d4910f77d811febc08` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:23.366335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a178b6a74140d4910f77d811febc08)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:23.366727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20796ed4-6cbc-4b43-8139-7c6fe0eb0331 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:23.370576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:23.380052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:23.380859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:23.382567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:23.382956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] PATH_INFO: `/v3/projects/60259bf80cc041a286453bc99293125c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:23.385571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60259bf80cc041a286453bc99293125c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:23.391537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:23.520791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a178b6a74140d4910f77d811febc08'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.544918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60259bf80cc041a286453bc99293125c', 'request_id': 'req-21bfaa60-18a2-453d-bbed-c040193412b3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.555191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a178b6a74140d4910f77d811febc08'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.566167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a178b6a74140d4910f77d811febc08'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.575488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60259bf80cc041a286453bc99293125c', 'request_id': 'req-21bfaa60-18a2-453d-bbed-c040193412b3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.575887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a178b6a74140d4910f77d811febc08'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.577019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f5377f24-2ce7-4c42-aa1e-4c0a47549f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a178b6a74140d4910f77d811febc08'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.583963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60259bf80cc041a286453bc99293125c', 'request_id': 'req-21bfaa60-18a2-453d-bbed-c040193412b3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.590145 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1017/2013] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:23 2022] DELETE /identity/v3/users/c5a178b6a74140d4910f77d811febc08 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:23.592115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240a69-60a2-45d4-b029-518b9aaea9ae None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:23.592559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240a69-60a2-45d4-b029-518b9aaea9ae None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:23.593016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240a69-60a2-45d4-b029-518b9aaea9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:23.601952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] The token cache is being invalidate because project 60259bf80cc041a286453bc99293125c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:23.602395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.603692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.608656 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 997/2014] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:23 2022] DELETE /identity/v3/projects/60259bf80cc041a286453bc99293125c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:23.609995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:23.613410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:23.624340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-97240a69-60a2-45d4-b029-518b9aaea9ae None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:23.625983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-97240a69-60a2-45d4-b029-518b9aaea9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:23.753490 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1018/2015] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:23.762102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfaa60-18a2-453d-bbed-c040193412b3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:23.781382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:23.783904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97240a69-60a2-45d4-b029-518b9aaea9ae None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:23.784492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:23.784837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:23.786343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-97240a69-60a2-45d4-b029-518b9aaea9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:23.788803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] PATH_INFO: `/v3/users/1b8b091703d545819b1e020a96850ec9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:23.791630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8b091703d545819b1e020a96850ec9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:23.795978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:23.829437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:23.929235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8b091703d545819b1e020a96850ec9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.948214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97240a69-60a2-45d4-b029-518b9aaea9ae None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:23.948488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8b091703d545819b1e020a96850ec9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.959271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161e683-82eb-499f-b17b-db204e057cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:23.960971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161e683-82eb-499f-b17b-db204e057cea service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:23.967477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161e683-82eb-499f-b17b-db204e057cea service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:23.967791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161e683-82eb-499f-b17b-db204e057cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:23.986205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8b091703d545819b1e020a96850ec9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:23.993254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161e683-82eb-499f-b17b-db204e057cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:24.000036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161e683-82eb-499f-b17b-db204e057cea service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:24.007951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8b091703d545819b1e020a96850ec9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:24.008634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8b091703d545819b1e020a96850ec9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:24.029825 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 998/2017] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:23 2022] DELETE /identity/v3/users/1b8b091703d545819b1e020a96850ec9 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:24.030025 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1019/2017] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:25.032524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:25.037959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:25.235106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a4508-8a23-4eec-8b0e-c4972ee06881 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:25.256815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:25.257613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:25.257893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:25.258168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] PATH_INFO: `/v3/projects/c62d63a04c2440c48068d825b7378b24` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:25.263567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62d63a04c2440c48068d825b7378b24)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:25.274799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:25.443334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62d63a04c2440c48068d825b7378b24', 'request_id': 'req-8446e20d-c65a-4490-9291-c3afc32563ee'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:25.444737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161e683-82eb-499f-b17b-db204e057cea service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:25.446640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e161e683-82eb-499f-b17b-db204e057cea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:25.465408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62d63a04c2440c48068d825b7378b24', 'request_id': 'req-8446e20d-c65a-4490-9291-c3afc32563ee'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:25.492443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161e683-82eb-499f-b17b-db204e057cea service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:25.497208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62d63a04c2440c48068d825b7378b24', 'request_id': 'req-8446e20d-c65a-4490-9291-c3afc32563ee'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:25.530636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] The token cache is being invalidate because project c62d63a04c2440c48068d825b7378b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:25.530839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:25.532379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:25.532590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8446e20d-c65a-4490-9291-c3afc32563ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:25.543547 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 999/2018] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:25 2022] DELETE /identity/v3/projects/c62d63a04c2440c48068d825b7378b24 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:25.545080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a2b41-eb11-4248-a54a-fe6f01acfa80 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:25.545370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a2b41-eb11-4248-a54a-fe6f01acfa80 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:25.545643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a2b41-eb11-4248-a54a-fe6f01acfa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:25.592666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f44a2b41-eb11-4248-a54a-fe6f01acfa80 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:25.593993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f44a2b41-eb11-4248-a54a-fe6f01acfa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:25.653512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:25.740838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:25.742214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:25.742602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:25.742974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] PATH_INFO: `/v3/projects/c126854c21954928af16db403a0b8919` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:25.746065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c126854c21954928af16db403a0b8919)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:25.748310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:25.790615 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1000/2019] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:25.792403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dce5a-1a43-4f67-ac80-0c28393b86b7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:25.792846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dce5a-1a43-4f67-ac80-0c28393b86b7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:25.793283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dce5a-1a43-4f67-ac80-0c28393b86b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:25.835376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4d6dce5a-1a43-4f67-ac80-0c28393b86b7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:25.844145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4d6dce5a-1a43-4f67-ac80-0c28393b86b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:25.865733 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1001/2020] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:25.870774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6dce5a-1a43-4f67-ac80-0c28393b86b7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:25.891194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c126854c21954928af16db403a0b8919', 'request_id': 'req-786c379d-db53-45a8-9338-b992befd4530'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:25.896665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6dce5a-1a43-4f67-ac80-0c28393b86b7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:25.912530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c126854c21954928af16db403a0b8919', 'request_id': 'req-786c379d-db53-45a8-9338-b992befd4530'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:25.924357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c126854c21954928af16db403a0b8919', 'request_id': 'req-786c379d-db53-45a8-9338-b992befd4530'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:25.963825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] The token cache is being invalidate because project c126854c21954928af16db403a0b8919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:25.964119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:25.967384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:25.970333 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1020/2021] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:25 2022] DELETE /identity/v3/projects/c126854c21954928af16db403a0b8919 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:25.974976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:25.979892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:26.019388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435201-5e29-427e-aa0e-51005a748dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:26.026356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435201-5e29-427e-aa0e-51005a748dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:26.026894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435201-5e29-427e-aa0e-51005a748dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:26.027173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435201-5e29-427e-aa0e-51005a748dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:26.027878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435201-5e29-427e-aa0e-51005a748dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:26.028752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435201-5e29-427e-aa0e-51005a748dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:26.030432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435201-5e29-427e-aa0e-51005a748dc6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:26.049587 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1002/2022] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:26.056821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435201-5e29-427e-aa0e-51005a748dc6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:26.060602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-21435201-5e29-427e-aa0e-51005a748dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:26.101579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435201-5e29-427e-aa0e-51005a748dc6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:26.122920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd1836-8d77-424b-a80f-1867765626b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:26.123539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd1836-8d77-424b-a80f-1867765626b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:26.124058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd1836-8d77-424b-a80f-1867765626b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:26.127277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd1836-8d77-424b-a80f-1867765626b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:26.127950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd1836-8d77-424b-a80f-1867765626b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:26.129901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd1836-8d77-424b-a80f-1867765626b7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:26.153700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c379d-db53-45a8-9338-b992befd4530 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:26.169767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:26.170422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:26.170697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:26.170966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] PATH_INFO: `/v3/users/e25fb2f175bd434b824ec70f41abe8ec` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:26.180219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e25fb2f175bd434b824ec70f41abe8ec)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:26.182250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:26.242415 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1003/2023] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:26 2022] POST /identity/v3/projects => generated 391 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:26.246016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd1836-8d77-424b-a80f-1867765626b7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:26.306580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:26.317027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e25fb2f175bd434b824ec70f41abe8ec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:26.366982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e25fb2f175bd434b824ec70f41abe8ec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:26.388186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e25fb2f175bd434b824ec70f41abe8ec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:26.406838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e25fb2f175bd434b824ec70f41abe8ec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:26.410395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7c7150-31fd-4e78-9ffe-09cbe3b5bb21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e25fb2f175bd434b824ec70f41abe8ec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:26.433025 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1021/2024] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:25 2022] DELETE /identity/v3/users/e25fb2f175bd434b824ec70f41abe8ec => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:26.541832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd1836-8d77-424b-a80f-1867765626b7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:26.626072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0cabf-771d-4d3a-891a-669e8e43358d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:26.626574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0cabf-771d-4d3a-891a-669e8e43358d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:26.626574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0cabf-771d-4d3a-891a-669e8e43358d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:26.654763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:26.701184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5df2b4-e7e9-4880-9007-8d340df6aa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:26.702135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5df2b4-e7e9-4880-9007-8d340df6aa83 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:26.702472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5df2b4-e7e9-4880-9007-8d340df6aa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:26.702828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5df2b4-e7e9-4880-9007-8d340df6aa83 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:26.703510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5df2b4-e7e9-4880-9007-8d340df6aa83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:26.706262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5df2b4-e7e9-4880-9007-8d340df6aa83 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:26.728824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-14f0cabf-771d-4d3a-891a-669e8e43358d None None] MFA Rules not processed for user `d1929f640b7348d6901067dfa6354c49`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:26.733536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-14f0cabf-771d-4d3a-891a-669e8e43358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:26.750764 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1004/2025] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:26 2022] POST /identity/v3/users => generated 296 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:26.752877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d026941-9fc1-4714-9760-975334ccff61 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:26.753370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d026941-9fc1-4714-9760-975334ccff61 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:26.753741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d026941-9fc1-4714-9760-975334ccff61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:26.792068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8d026941-9fc1-4714-9760-975334ccff61 None None] MFA Rules not processed for user `14bd717b05ce4e40a1cd54c2db8474a6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:26.793491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8d026941-9fc1-4714-9760-975334ccff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:26.896545 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1022/2026] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:26.897758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0cabf-771d-4d3a-891a-669e8e43358d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:26.901288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-14f0cabf-771d-4d3a-891a-669e8e43358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:26.922151 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1005/2027] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:26.922151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d026941-9fc1-4714-9760-975334ccff61 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:26.922151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8d026941-9fc1-4714-9760-975334ccff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:26.925435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0cabf-771d-4d3a-891a-669e8e43358d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:26.938473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d026941-9fc1-4714-9760-975334ccff61 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:26.939139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621be800-bcef-4a9b-94fb-be5dbf5ae338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:26.940555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621be800-bcef-4a9b-94fb-be5dbf5ae338 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:26.940983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621be800-bcef-4a9b-94fb-be5dbf5ae338 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:26.941355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621be800-bcef-4a9b-94fb-be5dbf5ae338 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:26.942209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621be800-bcef-4a9b-94fb-be5dbf5ae338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:26.944040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621be800-bcef-4a9b-94fb-be5dbf5ae338 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:26.947118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f731b3b-7aeb-4551-99f2-ef79a3b85621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:26.947864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731b3b-7aeb-4551-99f2-ef79a3b85621 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:26.948751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731b3b-7aeb-4551-99f2-ef79a3b85621 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:26.949156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731b3b-7aeb-4551-99f2-ef79a3b85621 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:26.949862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f731b3b-7aeb-4551-99f2-ef79a3b85621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:26.953704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f731b3b-7aeb-4551-99f2-ef79a3b85621 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:26.973546 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1023/2028] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:26.977167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621be800-bcef-4a9b-94fb-be5dbf5ae338 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:27.006052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621be800-bcef-4a9b-94fb-be5dbf5ae338 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:27.020999 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1006/2029] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:26 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:27.021483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4b8c4-8c94-4fc6-9448-0af26d430208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:27.022466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4b8c4-8c94-4fc6-9448-0af26d430208 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:27.022891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4b8c4-8c94-4fc6-9448-0af26d430208 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:27.023177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f731b3b-7aeb-4551-99f2-ef79a3b85621 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:27.023754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4b8c4-8c94-4fc6-9448-0af26d430208 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:27.024574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4b8c4-8c94-4fc6-9448-0af26d430208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:27.026376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4b8c4-8c94-4fc6-9448-0af26d430208 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:27.081376 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1024/2030] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:27.085854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4b8c4-8c94-4fc6-9448-0af26d430208 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:27.110526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4b8c4-8c94-4fc6-9448-0af26d430208 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:27.210846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f731b3b-7aeb-4551-99f2-ef79a3b85621 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:27.225152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b17f0-3e4f-49d5-b9e4-b3fa20f38eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:27.227140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b17f0-3e4f-49d5-b9e4-b3fa20f38eab admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:27.227445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b17f0-3e4f-49d5-b9e4-b3fa20f38eab admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:27.227720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b17f0-3e4f-49d5-b9e4-b3fa20f38eab admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:27.231128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b17f0-3e4f-49d5-b9e4-b3fa20f38eab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:27.236167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b17f0-3e4f-49d5-b9e4-b3fa20f38eab admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:27.287001 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1007/2031] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:27 2022] POST /identity/v3/users => generated 309 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:27.290329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46125c-5204-46bb-b263-73abc3ac281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:27.291145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46125c-5204-46bb-b263-73abc3ac281b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:27.291268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b17f0-3e4f-49d5-b9e4-b3fa20f38eab admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:27.291437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46125c-5204-46bb-b263-73abc3ac281b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:27.294630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46125c-5204-46bb-b263-73abc3ac281b admin admin] PATH_INFO: `/v3/projects/ab4db46c126d4607943fef56702f5eaf/users/58842ce40beb4def95f41884b083ec49/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:27.314337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b17f0-3e4f-49d5-b9e4-b3fa20f38eab admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:27.337591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7310d-9b7e-4e7c-9091-2c7780d340c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:27.338269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7310d-9b7e-4e7c-9091-2c7780d340c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:27.338551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7310d-9b7e-4e7c-9091-2c7780d340c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:27.338826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7310d-9b7e-4e7c-9091-2c7780d340c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:27.339510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7310d-9b7e-4e7c-9091-2c7780d340c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:27.343468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46125c-5204-46bb-b263-73abc3ac281b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4db46c126d4607943fef56702f5eaf, user_id=58842ce40beb4def95f41884b083ec49, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:27.344685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7310d-9b7e-4e7c-9091-2c7780d340c6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:27.350468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46125c-5204-46bb-b263-73abc3ac281b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:27.377254 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1008/2032] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:27.381799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7310d-9b7e-4e7c-9091-2c7780d340c6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:27.409647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7310d-9b7e-4e7c-9091-2c7780d340c6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:27.416390 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1025/2033] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:27 2022] PUT /identity/v3/projects/ab4db46c126d4607943fef56702f5eaf/users/58842ce40beb4def95f41884b083ec49/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:27.434392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b014e-6eee-4429-85b4-1bb5d71ae81f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:27.434940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b014e-6eee-4429-85b4-1bb5d71ae81f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:27.435374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b014e-6eee-4429-85b4-1bb5d71ae81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:27.468510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:27.550803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:27.612272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a60b014e-6eee-4429-85b4-1bb5d71ae81f None None] MFA Rules not processed for user `58842ce40beb4def95f41884b083ec49`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:27.613664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a60b014e-6eee-4429-85b4-1bb5d71ae81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:27.642208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af378b-af52-4415-af53-eb2354d4c8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:27.643483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af378b-af52-4415-af53-eb2354d4c8ad admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:27.643836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af378b-af52-4415-af53-eb2354d4c8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:27.644111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af378b-af52-4415-af53-eb2354d4c8ad admin admin] PATH_INFO: `/v3/projects/784d87e5cec947028825114a56719f9f/users/5acc23498df94c3886781c23cfc7d6de/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:27.696900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af378b-af52-4415-af53-eb2354d4c8ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=784d87e5cec947028825114a56719f9f, user_id=5acc23498df94c3886781c23cfc7d6de, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:27.703214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af378b-af52-4415-af53-eb2354d4c8ad admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:27.733708 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1009/2034] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:27 2022] PUT /identity/v3/projects/784d87e5cec947028825114a56719f9f/users/5acc23498df94c3886781c23cfc7d6de/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:27.735039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63a714-ef10-480a-b797-2346112273b7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:27.735330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63a714-ef10-480a-b797-2346112273b7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:27.736899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63a714-ef10-480a-b797-2346112273b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:27.810600 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1026/2035] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:27.810600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b014e-6eee-4429-85b4-1bb5d71ae81f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:27.811620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2d63a714-ef10-480a-b797-2346112273b7 None None] MFA Rules not processed for user `92d7ca8b7e1a4ca6acc797364290c607`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:27.812901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2d63a714-ef10-480a-b797-2346112273b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:27.814112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a60b014e-6eee-4429-85b4-1bb5d71ae81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:27.852400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b014e-6eee-4429-85b4-1bb5d71ae81f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:28.017126 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1010/2036] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:28.017126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63a714-ef10-480a-b797-2346112273b7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:28.018541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:28.018689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.018943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.019212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] PATH_INFO: `/v3/projects/08e758aebf084d609d8183f2644b54fe` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.021440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e758aebf084d609d8183f2644b54fe)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:28.023398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:28.040513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63a714-ef10-480a-b797-2346112273b7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:28.054950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cdc5f-9a51-4a12-ab91-134b21e34323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:28.055879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cdc5f-9a51-4a12-ab91-134b21e34323 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.056550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cdc5f-9a51-4a12-ab91-134b21e34323 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.057024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cdc5f-9a51-4a12-ab91-134b21e34323 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.058030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3cdc5f-9a51-4a12-ab91-134b21e34323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:28.070251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3cdc5f-9a51-4a12-ab91-134b21e34323 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:28.098354 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1011/2037] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:28.099424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cdc5f-9a51-4a12-ab91-134b21e34323 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:28.123993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cdc5f-9a51-4a12-ab91-134b21e34323 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:28.137394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e758aebf084d609d8183f2644b54fe', 'request_id': 'req-a059dd27-444f-4013-82ba-3dbe1828d899'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:28.153582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e758aebf084d609d8183f2644b54fe', 'request_id': 'req-a059dd27-444f-4013-82ba-3dbe1828d899'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:28.170649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e758aebf084d609d8183f2644b54fe', 'request_id': 'req-a059dd27-444f-4013-82ba-3dbe1828d899'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:28.214868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] The token cache is being invalidate because project 08e758aebf084d609d8183f2644b54fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:28.215182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:28.216640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a059dd27-444f-4013-82ba-3dbe1828d899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:28.225596 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1027/2038] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:27 2022] DELETE /identity/v3/projects/08e758aebf084d609d8183f2644b54fe => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:28.227774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40832f72-ac74-4e69-a182-8bc69a7136be None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.228054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40832f72-ac74-4e69-a182-8bc69a7136be None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.228319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40832f72-ac74-4e69-a182-8bc69a7136be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.270800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-40832f72-ac74-4e69-a182-8bc69a7136be None None] MFA Rules not processed for user `c613201f55b94a528c6c36e5b5772799`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:28.280223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-40832f72-ac74-4e69-a182-8bc69a7136be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:28.294916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9d526-7ac7-4081-9802-1c91edc86eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:28.295629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9d526-7ac7-4081-9802-1c91edc86eed admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.295972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9d526-7ac7-4081-9802-1c91edc86eed admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.297296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9d526-7ac7-4081-9802-1c91edc86eed admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.298167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9d526-7ac7-4081-9802-1c91edc86eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:28.301382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9d526-7ac7-4081-9802-1c91edc86eed admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:28.332437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:28.364842 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1012/2039] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:28 2022] POST /identity/v3/projects => generated 391 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:28.370444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9d526-7ac7-4081-9802-1c91edc86eed admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:28.373060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-37b9d526-7ac7-4081-9802-1c91edc86eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:28.387227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:28.413358 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1028/2040] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:28.416374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e1cab-7fcf-4e0a-b169-3d9f924061bb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.416652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e1cab-7fcf-4e0a-b169-3d9f924061bb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.416922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e1cab-7fcf-4e0a-b169-3d9f924061bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.498789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e66e1cab-7fcf-4e0a-b169-3d9f924061bb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:28.500132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e66e1cab-7fcf-4e0a-b169-3d9f924061bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:28.566092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9d526-7ac7-4081-9802-1c91edc86eed admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:28.581544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd8e06-aec0-440b-8e49-4df39b6fb48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:28.582238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8e06-aec0-440b-8e49-4df39b6fb48b admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.582514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8e06-aec0-440b-8e49-4df39b6fb48b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.582853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8e06-aec0-440b-8e49-4df39b6fb48b admin admin] PATH_INFO: `/v3/projects/784d87e5cec947028825114a56719f9f/users/5acc23498df94c3886781c23cfc7d6de/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.586143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd8e06-aec0-440b-8e49-4df39b6fb48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=784d87e5cec947028825114a56719f9f, user_id=5acc23498df94c3886781c23cfc7d6de, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:28.588800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd8e06-aec0-440b-8e49-4df39b6fb48b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:28.613421 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1013/2041] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:28 2022] PUT /identity/v3/projects/784d87e5cec947028825114a56719f9f/users/5acc23498df94c3886781c23cfc7d6de/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:28.617699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd8e06-aec0-440b-8e49-4df39b6fb48b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:28.622236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f1fd8e06-aec0-440b-8e49-4df39b6fb48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:28.662691 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1029/2042] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:28.671022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfd6ae-0260-4185-b2d5-3fb79f28521f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.671637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfd6ae-0260-4185-b2d5-3fb79f28521f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.672120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfd6ae-0260-4185-b2d5-3fb79f28521f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.727266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c2cfd6ae-0260-4185-b2d5-3fb79f28521f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:28.730815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c2cfd6ae-0260-4185-b2d5-3fb79f28521f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:28.769332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd8e06-aec0-440b-8e49-4df39b6fb48b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:28.779028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f59fb5-1bf4-4a05-be5d-1f2dfaa519cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:28.779668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f59fb5-1bf4-4a05-be5d-1f2dfaa519cd admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.779943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f59fb5-1bf4-4a05-be5d-1f2dfaa519cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.780596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f59fb5-1bf4-4a05-be5d-1f2dfaa519cd admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.781124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f59fb5-1bf4-4a05-be5d-1f2dfaa519cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:28.783274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f59fb5-1bf4-4a05-be5d-1f2dfaa519cd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:28.834381 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1014/2043] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:28 2022] POST /identity/v3/users => generated 302 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:28.835726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f59fb5-1bf4-4a05-be5d-1f2dfaa519cd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:28.849569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f59fb5-1bf4-4a05-be5d-1f2dfaa519cd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:28.855636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439773bc-4610-4f88-b90c-e818427b62c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:28.856258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439773bc-4610-4f88-b90c-e818427b62c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.856533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439773bc-4610-4f88-b90c-e818427b62c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.856804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439773bc-4610-4f88-b90c-e818427b62c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.857588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439773bc-4610-4f88-b90c-e818427b62c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:28.859328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439773bc-4610-4f88-b90c-e818427b62c0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:28.873897 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1015/2044] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:28.879109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439773bc-4610-4f88-b90c-e818427b62c0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:28.890822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439773bc-4610-4f88-b90c-e818427b62c0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:28.899030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d8ed8-76f2-46cf-b76b-27c87590ddd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:28.899702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8ed8-76f2-46cf-b76b-27c87590ddd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.900021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8ed8-76f2-46cf-b76b-27c87590ddd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.900293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8ed8-76f2-46cf-b76b-27c87590ddd5 admin admin] PATH_INFO: `/v3/projects/d9987eb652194122b3026f0b65c2b536/users/ef5ff17425f743f5866cafa5dc7e0da7/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.909910 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1030/2045] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:28.911306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfd6ae-0260-4185-b2d5-3fb79f28521f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:28.932566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfd6ae-0260-4185-b2d5-3fb79f28521f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:28.951310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d8ed8-76f2-46cf-b76b-27c87590ddd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9987eb652194122b3026f0b65c2b536, user_id=ef5ff17425f743f5866cafa5dc7e0da7, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:28.952704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72c1c5-8a8a-417e-99f2-6b6d50e63d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:28.953497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72c1c5-8a8a-417e-99f2-6b6d50e63d55 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:28.954120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72c1c5-8a8a-417e-99f2-6b6d50e63d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:28.954246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d8ed8-76f2-46cf-b76b-27c87590ddd5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:28.954417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72c1c5-8a8a-417e-99f2-6b6d50e63d55 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:28.956201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72c1c5-8a8a-417e-99f2-6b6d50e63d55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:28.956837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72c1c5-8a8a-417e-99f2-6b6d50e63d55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:28.962825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72c1c5-8a8a-417e-99f2-6b6d50e63d55 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:28.978900 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1016/2046] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:28 2022] PUT /identity/v3/projects/d9987eb652194122b3026f0b65c2b536/users/ef5ff17425f743f5866cafa5dc7e0da7/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:28.979773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d8ed8-76f2-46cf-b76b-27c87590ddd5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:28.982476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8e0d8ed8-76f2-46cf-b76b-27c87590ddd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:28.995605 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1031/2047] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:28.995605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72c1c5-8a8a-417e-99f2-6b6d50e63d55 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:29.020315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72c1c5-8a8a-417e-99f2-6b6d50e63d55 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:29.115598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:29.172051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d8ed8-76f2-46cf-b76b-27c87590ddd5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:29.177247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147807ee-cb0c-4662-bb31-067f545dd3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:29.179001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147807ee-cb0c-4662-bb31-067f545dd3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:29.180063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147807ee-cb0c-4662-bb31-067f545dd3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:29.181143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147807ee-cb0c-4662-bb31-067f545dd3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:29.182859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147807ee-cb0c-4662-bb31-067f545dd3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:29.187719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147807ee-cb0c-4662-bb31-067f545dd3c3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:29.188722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6bc539-d7a6-4f24-b40a-a25bf2d1b9cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:29.189942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bc539-d7a6-4f24-b40a-a25bf2d1b9cc service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:29.190492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bc539-d7a6-4f24-b40a-a25bf2d1b9cc service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:29.190990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bc539-d7a6-4f24-b40a-a25bf2d1b9cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:29.197532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f6bc539-d7a6-4f24-b40a-a25bf2d1b9cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:29.209667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:29.237862 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1032/2048] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:29.239845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147807ee-cb0c-4662-bb31-067f545dd3c3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:29.266312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147807ee-cb0c-4662-bb31-067f545dd3c3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:29.285018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd3547-8fb1-4d03-9cc0-10e5d13469d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:29.287692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd3547-8fb1-4d03-9cc0-10e5d13469d7 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:29.288448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd3547-8fb1-4d03-9cc0-10e5d13469d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:29.289646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd3547-8fb1-4d03-9cc0-10e5d13469d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:29.291228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd3547-8fb1-4d03-9cc0-10e5d13469d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:29.302078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd3547-8fb1-4d03-9cc0-10e5d13469d7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:29.393477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6bc539-d7a6-4f24-b40a-a25bf2d1b9cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:29.398572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6bc539-d7a6-4f24-b40a-a25bf2d1b9cc service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:29.399182 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1033/2049] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:29 2022] POST /identity/v3/projects => generated 395 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:29.404873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd3547-8fb1-4d03-9cc0-10e5d13469d7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:29.475093 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1017/2050] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:29.475852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6bc539-d7a6-4f24-b40a-a25bf2d1b9cc service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:29.522275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6bc539-d7a6-4f24-b40a-a25bf2d1b9cc service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:29.676013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd3547-8fb1-4d03-9cc0-10e5d13469d7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:29.693070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef257d1-ced9-42e8-b79e-e01834ceec60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:29.694775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef257d1-ced9-42e8-b79e-e01834ceec60 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:29.695347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef257d1-ced9-42e8-b79e-e01834ceec60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:29.697310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef257d1-ced9-42e8-b79e-e01834ceec60 admin admin] PATH_INFO: `/v3/projects/d9987eb652194122b3026f0b65c2b536/users/ef5ff17425f743f5866cafa5dc7e0da7/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:29.702521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef257d1-ced9-42e8-b79e-e01834ceec60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9987eb652194122b3026f0b65c2b536, user_id=ef5ff17425f743f5866cafa5dc7e0da7, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:29.707122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef257d1-ced9-42e8-b79e-e01834ceec60 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:29.750441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f82a43-4fb2-461a-8865-8d5df0ddcbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:29.754863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f82a43-4fb2-461a-8865-8d5df0ddcbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:29.757828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f82a43-4fb2-461a-8865-8d5df0ddcbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:29.758225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f82a43-4fb2-461a-8865-8d5df0ddcbb5 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:29.759018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f82a43-4fb2-461a-8865-8d5df0ddcbb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:29.761027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f82a43-4fb2-461a-8865-8d5df0ddcbb5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:29.761974 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1034/2051] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:29 2022] PUT /identity/v3/projects/d9987eb652194122b3026f0b65c2b536/users/ef5ff17425f743f5866cafa5dc7e0da7/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:29.773068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fc375-7cde-4755-98c9-bfd07cec23c3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:29.773498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fc375-7cde-4755-98c9-bfd07cec23c3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:29.773887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fc375-7cde-4755-98c9-bfd07cec23c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:29.838618 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1018/2052] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:29 2022] POST /identity/v3/users => generated 298 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:29.845705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f82a43-4fb2-461a-8865-8d5df0ddcbb5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:29.867879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f82a43-4fb2-461a-8865-8d5df0ddcbb5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:29.874286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-750fc375-7cde-4755-98c9-bfd07cec23c3 None None] MFA Rules not processed for user `ef5ff17425f743f5866cafa5dc7e0da7`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:29.875800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-750fc375-7cde-4755-98c9-bfd07cec23c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:30.055145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc89192-9d61-45f5-91c9-a3d0df1e8464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:30.055694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc89192-9d61-45f5-91c9-a3d0df1e8464 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:30.055969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc89192-9d61-45f5-91c9-a3d0df1e8464 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:30.056235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc89192-9d61-45f5-91c9-a3d0df1e8464 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:30.056886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc89192-9d61-45f5-91c9-a3d0df1e8464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:30.062193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc89192-9d61-45f5-91c9-a3d0df1e8464 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:30.101274 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1035/2053] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:29 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:30.110186 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1019/2054] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:30.110407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cad2e-6899-4ccd-98a1-1a561e5dc193 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:30.110407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cad2e-6899-4ccd-98a1-1a561e5dc193 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:30.110407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cad2e-6899-4ccd-98a1-1a561e5dc193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:30.113245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc89192-9d61-45f5-91c9-a3d0df1e8464 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:30.134712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc89192-9d61-45f5-91c9-a3d0df1e8464 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:30.149853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec287a-4c62-441a-a5e3-8ddaaf266755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:30.150585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec287a-4c62-441a-a5e3-8ddaaf266755 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:30.150863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec287a-4c62-441a-a5e3-8ddaaf266755 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:30.151130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec287a-4c62-441a-a5e3-8ddaaf266755 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:30.151786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec287a-4c62-441a-a5e3-8ddaaf266755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:30.154358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3f9cad2e-6899-4ccd-98a1-1a561e5dc193 None None] MFA Rules not processed for user `58842ce40beb4def95f41884b083ec49`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:30.156178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3f9cad2e-6899-4ccd-98a1-1a561e5dc193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:30.159207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec287a-4c62-441a-a5e3-8ddaaf266755 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:30.214459 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1020/2055] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:30.217849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec287a-4c62-441a-a5e3-8ddaaf266755 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:30.262811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec287a-4c62-441a-a5e3-8ddaaf266755 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:30.289473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef776b3-138c-4bc1-80b0-8bdd345df77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:30.290377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef776b3-138c-4bc1-80b0-8bdd345df77b admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:30.290377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef776b3-138c-4bc1-80b0-8bdd345df77b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:30.290663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef776b3-138c-4bc1-80b0-8bdd345df77b admin admin] PATH_INFO: `/v3/projects/43aaf2ff5067452883510aad3a05adc3/users/94d53e544c7047539a60d1cd0dd681e3/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:30.307052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:30.360181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef776b3-138c-4bc1-80b0-8bdd345df77b admin admin] RBAC: Authorizing `identity:create_grant(project_id=43aaf2ff5067452883510aad3a05adc3, user_id=94d53e544c7047539a60d1cd0dd681e3, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:30.364393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef776b3-138c-4bc1-80b0-8bdd345df77b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:30.478812 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1036/2056] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:30.481207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1021/2057] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:30 2022] PUT /identity/v3/projects/43aaf2ff5067452883510aad3a05adc3/users/94d53e544c7047539a60d1cd0dd681e3/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:30.493786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04823ae-382d-4a65-9af1-8eaa612bb7c4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:30.494365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04823ae-382d-4a65-9af1-8eaa612bb7c4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:30.494815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04823ae-382d-4a65-9af1-8eaa612bb7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:30.509889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:30.631803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c04823ae-382d-4a65-9af1-8eaa612bb7c4 None None] MFA Rules not processed for user `94d53e544c7047539a60d1cd0dd681e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:30.633783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c04823ae-382d-4a65-9af1-8eaa612bb7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:30.848343 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1037/2058] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:30.852485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef776b3-138c-4bc1-80b0-8bdd345df77b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:31.007933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef776b3-138c-4bc1-80b0-8bdd345df77b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:31.023560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97f9b7-e036-4aeb-b128-b51a64301990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:31.024744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97f9b7-e036-4aeb-b128-b51a64301990 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:31.025158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97f9b7-e036-4aeb-b128-b51a64301990 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:31.025717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97f9b7-e036-4aeb-b128-b51a64301990 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:31.026279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97f9b7-e036-4aeb-b128-b51a64301990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:31.032320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97f9b7-e036-4aeb-b128-b51a64301990 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:31.103633 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1022/2059] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:30 2022] POST /identity/v3/projects => generated 393 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:31.107684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04823ae-382d-4a65-9af1-8eaa612bb7c4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:31.124247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04823ae-382d-4a65-9af1-8eaa612bb7c4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:31.253415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31647994-9691-4b41-8268-d38360860a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:31.254368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647994-9691-4b41-8268-d38360860a6f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:31.254916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647994-9691-4b41-8268-d38360860a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:31.255389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647994-9691-4b41-8268-d38360860a6f admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:31.256274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31647994-9691-4b41-8268-d38360860a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:31.258772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31647994-9691-4b41-8268-d38360860a6f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:31.292181 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1038/2060] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:31 2022] POST /identity/v3/users => generated 303 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:31.296811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97f9b7-e036-4aeb-b128-b51a64301990 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:31.310947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97f9b7-e036-4aeb-b128-b51a64301990 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:31.318080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de123d9-2244-480e-bd25-76a748a2b1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:31.318972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de123d9-2244-480e-bd25-76a748a2b1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:31.319739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de123d9-2244-480e-bd25-76a748a2b1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:31.320182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de123d9-2244-480e-bd25-76a748a2b1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:31.320981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de123d9-2244-480e-bd25-76a748a2b1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:31.322722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de123d9-2244-480e-bd25-76a748a2b1a2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:31.344866 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1023/2061] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:31.351269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31647994-9691-4b41-8268-d38360860a6f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:31.374336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31647994-9691-4b41-8268-d38360860a6f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:31.388829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01280455-089a-4131-9183-6b0aa91bf901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:31.390323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01280455-089a-4131-9183-6b0aa91bf901 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:31.391073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01280455-089a-4131-9183-6b0aa91bf901 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:31.392852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01280455-089a-4131-9183-6b0aa91bf901 admin admin] PATH_INFO: `/v3/projects/d66f78230e7f4217a939bda430a3d9f6/users/fa3564c93bca41fc947c9d0762320f25/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:31.437521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01280455-089a-4131-9183-6b0aa91bf901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d66f78230e7f4217a939bda430a3d9f6, user_id=fa3564c93bca41fc947c9d0762320f25, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:31.440434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01280455-089a-4131-9183-6b0aa91bf901 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:31.473741 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1039/2062] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:31 2022] PUT /identity/v3/projects/d66f78230e7f4217a939bda430a3d9f6/users/fa3564c93bca41fc947c9d0762320f25/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:31.475781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de123d9-2244-480e-bd25-76a748a2b1a2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:31.493787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de123d9-2244-480e-bd25-76a748a2b1a2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:31.686728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5b0aa-fc1b-4054-a5f3-639fd4f75ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:31.688040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5b0aa-fc1b-4054-a5f3-639fd4f75ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:31.688328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5b0aa-fc1b-4054-a5f3-639fd4f75ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:31.688603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5b0aa-fc1b-4054-a5f3-639fd4f75ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:31.689305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5b0aa-fc1b-4054-a5f3-639fd4f75ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:31.690987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5b0aa-fc1b-4054-a5f3-639fd4f75ee6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:31.702931 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1024/2063] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:31.707196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01280455-089a-4131-9183-6b0aa91bf901 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:31.720110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01280455-089a-4131-9183-6b0aa91bf901 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:31.750219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c0e9a-12fe-47dc-9a18-afb92b05bafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:31.752831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c0e9a-12fe-47dc-9a18-afb92b05bafa admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:31.754021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c0e9a-12fe-47dc-9a18-afb92b05bafa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:31.757194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c0e9a-12fe-47dc-9a18-afb92b05bafa admin admin] PATH_INFO: `/v3/projects/d66f78230e7f4217a939bda430a3d9f6/users/fa3564c93bca41fc947c9d0762320f25/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:31.761436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c0e9a-12fe-47dc-9a18-afb92b05bafa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d66f78230e7f4217a939bda430a3d9f6, user_id=fa3564c93bca41fc947c9d0762320f25, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:31.767136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c0e9a-12fe-47dc-9a18-afb92b05bafa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:31.777633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5b0aa-fc1b-4054-a5f3-639fd4f75ee6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:31.780092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c6f5b0aa-fc1b-4054-a5f3-639fd4f75ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:31.799800 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1040/2064] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:31 2022] PUT /identity/v3/projects/d66f78230e7f4217a939bda430a3d9f6/users/fa3564c93bca41fc947c9d0762320f25/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:31.810839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523e05-22bf-4b06-a7a4-9beda728f76c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:31.812580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523e05-22bf-4b06-a7a4-9beda728f76c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:31.813008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523e05-22bf-4b06-a7a4-9beda728f76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:31.929895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:31.965402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-48523e05-22bf-4b06-a7a4-9beda728f76c None None] MFA Rules not processed for user `fa3564c93bca41fc947c9d0762320f25`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:31.973741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-48523e05-22bf-4b06-a7a4-9beda728f76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:32.029542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5b0aa-fc1b-4054-a5f3-639fd4f75ee6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:32.155466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:32.222743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37aac2-98b7-48ab-ac12-5a9141646d89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:32.225852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37aac2-98b7-48ab-ac12-5a9141646d89 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:32.229349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37aac2-98b7-48ab-ac12-5a9141646d89 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:32.229642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37aac2-98b7-48ab-ac12-5a9141646d89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:32.234925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1f37aac2-98b7-48ab-ac12-5a9141646d89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:32.388096 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1041/2065] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:31 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 578 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:32.389425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523e05-22bf-4b06-a7a4-9beda728f76c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:32.392111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37aac2-98b7-48ab-ac12-5a9141646d89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:32.393800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37aac2-98b7-48ab-ac12-5a9141646d89 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:32.417987 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1025/2066] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:32.745766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523e05-22bf-4b06-a7a4-9beda728f76c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:32.781552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58b6cc-3126-4833-a852-654b2665261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:32.782554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58b6cc-3126-4833-a852-654b2665261a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:32.783140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58b6cc-3126-4833-a852-654b2665261a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:32.784232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58b6cc-3126-4833-a852-654b2665261a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:32.785137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58b6cc-3126-4833-a852-654b2665261a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:32.787764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58b6cc-3126-4833-a852-654b2665261a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:32.899481 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1042/2067] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:32 2022] POST /identity/v3/projects => generated 395 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:32.903643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37aac2-98b7-48ab-ac12-5a9141646d89 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:32.931881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37aac2-98b7-48ab-ac12-5a9141646d89 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:33.169078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af38f86-597e-4d94-91f0-340b1e74c319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:33.169929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38f86-597e-4d94-91f0-340b1e74c319 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:33.170455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38f86-597e-4d94-91f0-340b1e74c319 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:33.170897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38f86-597e-4d94-91f0-340b1e74c319 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:33.171688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af38f86-597e-4d94-91f0-340b1e74c319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:33.188597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af38f86-597e-4d94-91f0-340b1e74c319 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:33.296437 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1026/2068] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:32 2022] POST /identity/v3/users => generated 298 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:49:33.298205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58b6cc-3126-4833-a852-654b2665261a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:33.316200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58b6cc-3126-4833-a852-654b2665261a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:33.341875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0927fd9-f157-4b37-8426-7ea08bd4e90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:33.342501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0927fd9-f157-4b37-8426-7ea08bd4e90b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:33.342775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0927fd9-f157-4b37-8426-7ea08bd4e90b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:33.343043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0927fd9-f157-4b37-8426-7ea08bd4e90b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:33.343698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0927fd9-f157-4b37-8426-7ea08bd4e90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:33.345967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0927fd9-f157-4b37-8426-7ea08bd4e90b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:33.393359 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1043/2069] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:33.397024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af38f86-597e-4d94-91f0-340b1e74c319 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:33.414296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af38f86-597e-4d94-91f0-340b1e74c319 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:33.439031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d3225-83c2-4cbe-92e5-4b348e358d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:33.439794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d3225-83c2-4cbe-92e5-4b348e358d1e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:33.440007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d3225-83c2-4cbe-92e5-4b348e358d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:33.440277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d3225-83c2-4cbe-92e5-4b348e358d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:33.440944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d3225-83c2-4cbe-92e5-4b348e358d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:33.447477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d3225-83c2-4cbe-92e5-4b348e358d1e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:33.481515 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1027/2070] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:33.485785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0927fd9-f157-4b37-8426-7ea08bd4e90b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:33.500172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0927fd9-f157-4b37-8426-7ea08bd4e90b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:33.526610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94310680-d343-409f-937c-13d11456bc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:33.527291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310680-d343-409f-937c-13d11456bc3c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:33.527567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310680-d343-409f-937c-13d11456bc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:33.527835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310680-d343-409f-937c-13d11456bc3c admin admin] PATH_INFO: `/v3/projects/2f2cdf5281e744898df90c7e32baca94/users/ecb1091d48fc4b3f84d56f81a46c5792/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:33.591328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94310680-d343-409f-937c-13d11456bc3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f2cdf5281e744898df90c7e32baca94, user_id=ecb1091d48fc4b3f84d56f81a46c5792, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:33.609298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94310680-d343-409f-937c-13d11456bc3c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:33.664782 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1044/2071] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:33 2022] PUT /identity/v3/projects/2f2cdf5281e744898df90c7e32baca94/users/ecb1091d48fc4b3f84d56f81a46c5792/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:33.690379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fabfc4-0985-4742-b6cb-76fce9adb8a1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:33.690671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fabfc4-0985-4742-b6cb-76fce9adb8a1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:33.692307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fabfc4-0985-4742-b6cb-76fce9adb8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:33.726100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:33.775005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-40fabfc4-0985-4742-b6cb-76fce9adb8a1 None None] MFA Rules not processed for user `ecb1091d48fc4b3f84d56f81a46c5792`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:33.779798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-40fabfc4-0985-4742-b6cb-76fce9adb8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:33.997696 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1028/2072] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:34.626841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94310680-d343-409f-937c-13d11456bc3c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:34.820859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94310680-d343-409f-937c-13d11456bc3c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:34.838833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf7b53-3450-4cc2-843b-f4618afd824b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:34.839457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf7b53-3450-4cc2-843b-f4618afd824b service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:34.839733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf7b53-3450-4cc2-843b-f4618afd824b service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:34.840011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf7b53-3450-4cc2-843b-f4618afd824b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:34.870247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:34.969917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf7b53-3450-4cc2-843b-f4618afd824b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:34.976992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf7b53-3450-4cc2-843b-f4618afd824b service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:35.011705 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1045/2073] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:34 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:35.767858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cb0b9-c7d9-4b9c-9445-aa25a75bb8b5 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:35.767858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cb0b9-c7d9-4b9c-9445-aa25a75bb8b5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:35.769124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cb0b9-c7d9-4b9c-9445-aa25a75bb8b5 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:35.769394 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1029/2074] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:49:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:35.780044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf7b53-3450-4cc2-843b-f4618afd824b service nova] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:49:35.799561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf7b53-3450-4cc2-843b-f4618afd824b service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:35.829469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf7b53-3450-4cc2-843b-f4618afd824b service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:35.841360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97477a-63a9-4eb8-9ebd-823734aa45e1 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3564c93bca41fc947c9d0762320f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd66f78230e7f4217a939bda430a3d9f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:35.842193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97477a-63a9-4eb8-9ebd-823734aa45e1 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:35.842575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97477a-63a9-4eb8-9ebd-823734aa45e1 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:35.842917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97477a-63a9-4eb8-9ebd-823734aa45e1 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] PATH_INFO: `/v3/projects/43aaf2ff5067452883510aad3a05adc3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:35.848475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97477a-63a9-4eb8-9ebd-823734aa45e1 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorizing `identity:get_project(project_id=43aaf2ff5067452883510aad3a05adc3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:35.850709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97477a-63a9-4eb8-9ebd-823734aa45e1 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:35.861893 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1046/2075] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:49:35 2022] GET /identity/v3/projects/43aaf2ff5067452883510aad3a05adc3 => generated 395 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:36.859238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717dffd-f8ce-4677-9237-fe738a737bcf None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:36.860256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717dffd-f8ce-4677-9237-fe738a737bcf None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:36.861293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717dffd-f8ce-4677-9237-fe738a737bcf None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:36.863229 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1030/2076] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:49:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:36.868629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97477a-63a9-4eb8-9ebd-823734aa45e1 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:49:36.893657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97477a-63a9-4eb8-9ebd-823734aa45e1 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:36.910218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97477a-63a9-4eb8-9ebd-823734aa45e1 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:36.935451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2017c0-a42a-4584-a7b1-53df1d0f79cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3564c93bca41fc947c9d0762320f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd66f78230e7f4217a939bda430a3d9f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:36.950499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2017c0-a42a-4584-a7b1-53df1d0f79cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:36.951148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2017c0-a42a-4584-a7b1-53df1d0f79cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:36.951676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2017c0-a42a-4584-a7b1-53df1d0f79cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] PATH_INFO: `/v3/projects/43aaf2ff5067452883510aad3a05adc3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:36.955464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2017c0-a42a-4584-a7b1-53df1d0f79cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorizing `identity:get_project(project_id=43aaf2ff5067452883510aad3a05adc3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:36.962083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2017c0-a42a-4584-a7b1-53df1d0f79cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:36.967853 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1047/2077] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:49:36 2022] GET /identity/v3/projects/43aaf2ff5067452883510aad3a05adc3 => generated 395 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:37.530763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c0483-9177-4859-900b-8670bc1a64b6 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:37.531243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c0483-9177-4859-900b-8670bc1a64b6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:37.532012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c0483-9177-4859-900b-8670bc1a64b6 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:37.534857 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1031/2078] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:49:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:37.540916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2017c0-a42a-4584-a7b1-53df1d0f79cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:49:37.565385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2017c0-a42a-4584-a7b1-53df1d0f79cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:37.584425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2017c0-a42a-4584-a7b1-53df1d0f79cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:37.609272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3ac5b-ff41-406d-b968-62ac03eddadc tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3564c93bca41fc947c9d0762320f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd66f78230e7f4217a939bda430a3d9f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:37.610696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3ac5b-ff41-406d-b968-62ac03eddadc tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:37.611210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3ac5b-ff41-406d-b968-62ac03eddadc tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:37.611768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3ac5b-ff41-406d-b968-62ac03eddadc tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] PATH_INFO: `/v3/projects/43aaf2ff5067452883510aad3a05adc3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:37.616323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3ac5b-ff41-406d-b968-62ac03eddadc tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorizing `identity:get_project(project_id=43aaf2ff5067452883510aad3a05adc3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:37.618684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3ac5b-ff41-406d-b968-62ac03eddadc tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:37.636778 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1048/2079] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:49:37 2022] GET /identity/v3/projects/43aaf2ff5067452883510aad3a05adc3 => generated 395 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:39.570939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c0483-9177-4859-900b-8670bc1a64b6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:39.598960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c0483-9177-4859-900b-8670bc1a64b6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:39.605859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6102c3a-6afa-4545-baad-54a39cea685e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:39.606747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102c3a-6afa-4545-baad-54a39cea685e service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:39.607217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102c3a-6afa-4545-baad-54a39cea685e service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:39.607734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102c3a-6afa-4545-baad-54a39cea685e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:39.766034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6102c3a-6afa-4545-baad-54a39cea685e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:39.768239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6102c3a-6afa-4545-baad-54a39cea685e service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:39.787312 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1032/2080] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:39 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:41.627719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3ac5b-ff41-406d-b968-62ac03eddadc tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:41.652893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3ac5b-ff41-406d-b968-62ac03eddadc tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:41.667267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fe9b6-c244-4abd-ba6d-4276ce8801fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:41.668929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fe9b6-c244-4abd-ba6d-4276ce8801fc service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:41.669554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fe9b6-c244-4abd-ba6d-4276ce8801fc service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:41.670400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fe9b6-c244-4abd-ba6d-4276ce8801fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:41.800653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fe9b6-c244-4abd-ba6d-4276ce8801fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:41.802711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fe9b6-c244-4abd-ba6d-4276ce8801fc service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:41.826070 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1049/2081] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:41 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:42.983815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae0e73-35b1-4d4b-a5f0-05e343426607 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:42.984392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae0e73-35b1-4d4b-a5f0-05e343426607 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:42.984653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae0e73-35b1-4d4b-a5f0-05e343426607 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:42.987293 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1033/2082] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:49:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:42.997921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fe9b6-c244-4abd-ba6d-4276ce8801fc service nova] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:49:43.018176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fe9b6-c244-4abd-ba6d-4276ce8801fc service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:43.033539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fe9b6-c244-4abd-ba6d-4276ce8801fc service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:43.048743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895ec3b-226e-4e5f-b17e-6f2cd837d374 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3564c93bca41fc947c9d0762320f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd66f78230e7f4217a939bda430a3d9f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:43.049626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895ec3b-226e-4e5f-b17e-6f2cd837d374 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:43.069023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be5f5b-ace5-4dad-9857-4ec820785707 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:43.069783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be5f5b-ace5-4dad-9857-4ec820785707 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:43.070250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be5f5b-ace5-4dad-9857-4ec820785707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:43.070874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895ec3b-226e-4e5f-b17e-6f2cd837d374 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:43.071266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895ec3b-226e-4e5f-b17e-6f2cd837d374 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] PATH_INFO: `/v3/projects/43aaf2ff5067452883510aad3a05adc3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:43.076132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895ec3b-226e-4e5f-b17e-6f2cd837d374 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorizing `identity:get_project(project_id=43aaf2ff5067452883510aad3a05adc3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:43.086504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895ec3b-226e-4e5f-b17e-6f2cd837d374 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:43.102813 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1050/2083] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:49:42 2022] GET /identity/v3/projects/43aaf2ff5067452883510aad3a05adc3 => generated 395 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:43.130062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-34be5f5b-ace5-4dad-9857-4ec820785707 None None] MFA Rules not processed for user `5acc23498df94c3886781c23cfc7d6de`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:43.131289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-34be5f5b-ace5-4dad-9857-4ec820785707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:43.306518 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1034/2084] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:43 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:43.993529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647934ca-fba7-47a8-8ec4-1069ac0503b4 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:43.994989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647934ca-fba7-47a8-8ec4-1069ac0503b4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:43.996095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647934ca-fba7-47a8-8ec4-1069ac0503b4 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:44.002179 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1051/2085] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:49:43 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:44.009082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be5f5b-ace5-4dad-9857-4ec820785707 None None] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:49:44.022481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be5f5b-ace5-4dad-9857-4ec820785707 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:44.034894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be5f5b-ace5-4dad-9857-4ec820785707 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:44.041056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdfee2-8370-4820-8bcf-1a2cf9bc5ad0 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3564c93bca41fc947c9d0762320f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd66f78230e7f4217a939bda430a3d9f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:44.042098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdfee2-8370-4820-8bcf-1a2cf9bc5ad0 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:44.042686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdfee2-8370-4820-8bcf-1a2cf9bc5ad0 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:44.043493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdfee2-8370-4820-8bcf-1a2cf9bc5ad0 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] PATH_INFO: `/v3/projects/43aaf2ff5067452883510aad3a05adc3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:44.053915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fdfee2-8370-4820-8bcf-1a2cf9bc5ad0 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorizing `identity:get_project(project_id=43aaf2ff5067452883510aad3a05adc3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:44.056384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fdfee2-8370-4820-8bcf-1a2cf9bc5ad0 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:44.058659 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1035/2086] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:49:44 2022] GET /identity/v3/projects/43aaf2ff5067452883510aad3a05adc3 => generated 395 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:44.809749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f656e-47b8-464c-ae6f-2836b21a896b None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:44.810164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f656e-47b8-464c-ae6f-2836b21a896b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:44.810531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f656e-47b8-464c-ae6f-2836b21a896b None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:44.812214 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1052/2087] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:44.818161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdfee2-8370-4820-8bcf-1a2cf9bc5ad0 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:49:44.834822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdfee2-8370-4820-8bcf-1a2cf9bc5ad0 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:44.839827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:44.846965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdfee2-8370-4820-8bcf-1a2cf9bc5ad0 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:44.852525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0d55b-5a78-45cc-b016-1dd37338e2bd tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3564c93bca41fc947c9d0762320f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd66f78230e7f4217a939bda430a3d9f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:44.853471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0d55b-5a78-45cc-b016-1dd37338e2bd tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:44.853909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0d55b-5a78-45cc-b016-1dd37338e2bd tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:44.854329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0d55b-5a78-45cc-b016-1dd37338e2bd tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] PATH_INFO: `/v3/projects/2f2cdf5281e744898df90c7e32baca94` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:44.855988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0d55b-5a78-45cc-b016-1dd37338e2bd tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorizing `identity:get_project(project_id=2f2cdf5281e744898df90c7e32baca94)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:44.858322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0d55b-5a78-45cc-b016-1dd37338e2bd tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:44.867653 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1036/2088] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:49:44 2022] GET /identity/v3/projects/2f2cdf5281e744898df90c7e32baca94 => generated 395 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:45.021668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f656e-47b8-464c-ae6f-2836b21a896b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:45.177211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f656e-47b8-464c-ae6f-2836b21a896b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:45.187174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:45.187949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:45.188544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:45.190535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] PATH_INFO: `/v3/users/94d53e544c7047539a60d1cd0dd681e3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:45.193776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94d53e544c7047539a60d1cd0dd681e3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:45.200946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:45.230556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:45.305388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94d53e544c7047539a60d1cd0dd681e3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:45.341385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94d53e544c7047539a60d1cd0dd681e3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:45.354109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94d53e544c7047539a60d1cd0dd681e3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:45.365906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94d53e544c7047539a60d1cd0dd681e3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:45.367355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94d53e544c7047539a60d1cd0dd681e3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:45.375489 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1053/2089] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:45 2022] DELETE /identity/v3/users/94d53e544c7047539a60d1cd0dd681e3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:45.381326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd517e-08e1-46e9-8f4a-b8fb628b5544 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:45.381780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd517e-08e1-46e9-8f4a-b8fb628b5544 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:45.381875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd517e-08e1-46e9-8f4a-b8fb628b5544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:45.417550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a7cd517e-08e1-46e9-8f4a-b8fb628b5544 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:45.418782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a7cd517e-08e1-46e9-8f4a-b8fb628b5544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:45.557335 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1037/2090] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:45.650815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:45.652968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:45.742123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918ade9-67d3-4435-82c7-cb0341f96683 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:45.747093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1189d-8e9f-43f0-9645-5b0b074276d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:45.748293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1189d-8e9f-43f0-9645-5b0b074276d2 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:45.748787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1189d-8e9f-43f0-9645-5b0b074276d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:45.749644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1189d-8e9f-43f0-9645-5b0b074276d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:45.763470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1189d-8e9f-43f0-9645-5b0b074276d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:45.765291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1189d-8e9f-43f0-9645-5b0b074276d2 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:45.792743 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1054/2091] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:46.927192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd517e-08e1-46e9-8f4a-b8fb628b5544 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:46.929733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a7cd517e-08e1-46e9-8f4a-b8fb628b5544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:46.956114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd517e-08e1-46e9-8f4a-b8fb628b5544 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:46.976667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:46.977890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:46.978327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:46.978789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] PATH_INFO: `/v3/projects/43aaf2ff5067452883510aad3a05adc3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:46.979903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] RBAC: Authorizing `identity:delete_project(project_id=43aaf2ff5067452883510aad3a05adc3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:46.983023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:47.119696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43aaf2ff5067452883510aad3a05adc3', 'request_id': 'req-4342c9af-f958-443a-bc18-326a94c2cbac'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:47.136622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43aaf2ff5067452883510aad3a05adc3', 'request_id': 'req-4342c9af-f958-443a-bc18-326a94c2cbac'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:47.184045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43aaf2ff5067452883510aad3a05adc3', 'request_id': 'req-4342c9af-f958-443a-bc18-326a94c2cbac'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:47.225200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] The token cache is being invalidate because project 43aaf2ff5067452883510aad3a05adc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:47.225527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:47.226202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4342c9af-f958-443a-bc18-326a94c2cbac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:47.235724 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1038/2092] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:46 2022] DELETE /identity/v3/projects/43aaf2ff5067452883510aad3a05adc3 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:47.237757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1189d-8e9f-43f0-9645-5b0b074276d2 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:47.255398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1189d-8e9f-43f0-9645-5b0b074276d2 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:47.436803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:47.437765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:47.438405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:47.438953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] PATH_INFO: `/v3/users/fa3564c93bca41fc947c9d0762320f25` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:47.441364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa3564c93bca41fc947c9d0762320f25)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:47.443694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:47.514377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:47.528595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa3564c93bca41fc947c9d0762320f25'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:47.559105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa3564c93bca41fc947c9d0762320f25'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:47.582417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa3564c93bca41fc947c9d0762320f25'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:47.598305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa3564c93bca41fc947c9d0762320f25'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:47.599068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fdb6d117-671c-4659-8791-dae477d75484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa3564c93bca41fc947c9d0762320f25'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:47.608651 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1055/2093] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:47 2022] DELETE /identity/v3/users/fa3564c93bca41fc947c9d0762320f25 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:48.736861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ce46b-da75-40ff-bfb8-5e8d3cd168cb None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:48.737707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ce46b-da75-40ff-bfb8-5e8d3cd168cb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:48.738477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ce46b-da75-40ff-bfb8-5e8d3cd168cb None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:48.740211 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1039/2094] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:49:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:49:48.768677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8161dc-9373-4432-b6dc-8bb772d591c6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:48.769267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8161dc-9373-4432-b6dc-8bb772d591c6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:48.769267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8161dc-9373-4432-b6dc-8bb772d591c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:48.799676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8a8161dc-9373-4432-b6dc-8bb772d591c6 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:48.800992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a8161dc-9373-4432-b6dc-8bb772d591c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:48.890680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ce46b-da75-40ff-bfb8-5e8d3cd168cb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:48.892433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-987ce46b-da75-40ff-bfb8-5e8d3cd168cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:49.028334 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1056/2095] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:49:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:49.030477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8161dc-9373-4432-b6dc-8bb772d591c6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:49.034221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a8161dc-9373-4432-b6dc-8bb772d591c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:49.059111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8161dc-9373-4432-b6dc-8bb772d591c6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:49.070217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:49.071889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:49.072383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:49.072879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] PATH_INFO: `/v3/users/507faa1518b34884b3e095bfae431bc2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:49.077237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] RBAC: Authorizing `identity:delete_user(user_id=507faa1518b34884b3e095bfae431bc2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:49.077399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ce46b-da75-40ff-bfb8-5e8d3cd168cb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:49.082021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:49.107696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:49.111298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:49.111598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:49.112694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] PATH_INFO: `/v3/projects/d66f78230e7f4217a939bda430a3d9f6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:49.122967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d66f78230e7f4217a939bda430a3d9f6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:49.124732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:49.198991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '507faa1518b34884b3e095bfae431bc2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.204887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:49.228849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '507faa1518b34884b3e095bfae431bc2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.253845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '507faa1518b34884b3e095bfae431bc2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.264408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd66f78230e7f4217a939bda430a3d9f6', 'request_id': 'req-4be12aad-6332-4aca-aa11-0a1ffa797edb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.271719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '507faa1518b34884b3e095bfae431bc2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.273019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '507faa1518b34884b3e095bfae431bc2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.281132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd66f78230e7f4217a939bda430a3d9f6', 'request_id': 'req-4be12aad-6332-4aca-aa11-0a1ffa797edb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.293189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd66f78230e7f4217a939bda430a3d9f6', 'request_id': 'req-4be12aad-6332-4aca-aa11-0a1ffa797edb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.307713 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1057/2096] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:49 2022] DELETE /identity/v3/users/507faa1518b34884b3e095bfae431bc2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:49.308988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:49.313655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:49.319687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] The token cache is being invalidate because project d66f78230e7f4217a939bda430a3d9f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:49.319996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.323367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.329670 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1040/2097] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:48 2022] DELETE /identity/v3/projects/d66f78230e7f4217a939bda430a3d9f6 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:49.333985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:49.337300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:49.403868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:49.523445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942b6ec-ea8c-41fd-9488-1b497c4f504a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:49.525380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be12aad-6332-4aca-aa11-0a1ffa797edb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:49.538774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dec40-5d69-4ded-bda5-ebff54bdeb93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:49.541510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dec40-5d69-4ded-bda5-ebff54bdeb93 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:49.544338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dec40-5d69-4ded-bda5-ebff54bdeb93 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:49.544605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dec40-5d69-4ded-bda5-ebff54bdeb93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:49.550649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d16dec40-5d69-4ded-bda5-ebff54bdeb93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:49.581009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dec40-5d69-4ded-bda5-ebff54bdeb93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:49.584510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dec40-5d69-4ded-bda5-ebff54bdeb93 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:49.618577 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1058/2098] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:49.655044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:49.655990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:49.656413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:49.657669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] PATH_INFO: `/v3/users/ecb1091d48fc4b3f84d56f81a46c5792` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:49.659019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb1091d48fc4b3f84d56f81a46c5792)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:49.661268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:49.793242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb1091d48fc4b3f84d56f81a46c5792'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.809928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb1091d48fc4b3f84d56f81a46c5792'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.815712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:49.827460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb1091d48fc4b3f84d56f81a46c5792'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.848944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb1091d48fc4b3f84d56f81a46c5792'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.849680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c819847-639b-4d4a-b676-eedc3feaea25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb1091d48fc4b3f84d56f81a46c5792'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:49.859216 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1041/2099] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:49 2022] DELETE /identity/v3/users/ecb1091d48fc4b3f84d56f81a46c5792 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:49:50.313856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dec40-5d69-4ded-bda5-ebff54bdeb93 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:50.315479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d16dec40-5d69-4ded-bda5-ebff54bdeb93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:50.331899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dec40-5d69-4ded-bda5-ebff54bdeb93 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:50.485480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:50.486435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:50.486791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:50.487221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] PATH_INFO: `/v3/projects/86408ec8ca9f4e33a16bdf9a23aa9faa` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:50.490816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86408ec8ca9f4e33a16bdf9a23aa9faa)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:50.496067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:50.573430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'request_id': 'req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:50.583933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'request_id': 'req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:50.604142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'request_id': 'req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:50.620584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] The token cache is being invalidate because project 86408ec8ca9f4e33a16bdf9a23aa9faa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:50.621441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:50.622144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ae9f118-b4ed-4897-8ebe-8793d0f6d667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:50.624402 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1059/2100] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:50 2022] DELETE /identity/v3/projects/86408ec8ca9f4e33a16bdf9a23aa9faa => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:50.632626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbe8b8-ae08-494a-a08e-4ec165bd02ba None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:50.633020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbe8b8-ae08-494a-a08e-4ec165bd02ba None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:50.633943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbe8b8-ae08-494a-a08e-4ec165bd02ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:50.662151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-19cbe8b8-ae08-494a-a08e-4ec165bd02ba None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:50.663800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-19cbe8b8-ae08-494a-a08e-4ec165bd02ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:50.755976 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1042/2101] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:50.763837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716496b6-c3b2-4194-b5b3-a6300e474d59 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:50.764536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716496b6-c3b2-4194-b5b3-a6300e474d59 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:50.764939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716496b6-c3b2-4194-b5b3-a6300e474d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:50.805232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-716496b6-c3b2-4194-b5b3-a6300e474d59 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:50.807043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-716496b6-c3b2-4194-b5b3-a6300e474d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:50.818336 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1060/2102] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:50.822236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbe8b8-ae08-494a-a08e-4ec165bd02ba None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:50.867807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbe8b8-ae08-494a-a08e-4ec165bd02ba None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:50.894061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8b511-0844-4edd-ad47-f6144fb83e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:50.894891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8b511-0844-4edd-ad47-f6144fb83e17 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:50.895165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8b511-0844-4edd-ad47-f6144fb83e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:50.895428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8b511-0844-4edd-ad47-f6144fb83e17 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:50.897474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8b511-0844-4edd-ad47-f6144fb83e17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:50.897788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8b511-0844-4edd-ad47-f6144fb83e17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:50.899387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8b511-0844-4edd-ad47-f6144fb83e17 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:50.920950 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1043/2103] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:50.925488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716496b6-c3b2-4194-b5b3-a6300e474d59 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:50.935656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716496b6-c3b2-4194-b5b3-a6300e474d59 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:50.942790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0a245-8bc8-4ffa-8091-cfed82856792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:50.943536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0a245-8bc8-4ffa-8091-cfed82856792 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:50.943909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0a245-8bc8-4ffa-8091-cfed82856792 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:50.944278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0a245-8bc8-4ffa-8091-cfed82856792 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:50.945011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0a245-8bc8-4ffa-8091-cfed82856792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:50.947074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0a245-8bc8-4ffa-8091-cfed82856792 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:50.986418 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1061/2104] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:50 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:50.990933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8b511-0844-4edd-ad47-f6144fb83e17 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:51.002995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8b511-0844-4edd-ad47-f6144fb83e17 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:51.122697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaf0cd-2e01-4c92-965e-f8bcfe42d645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:51.123995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaf0cd-2e01-4c92-965e-f8bcfe42d645 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.124604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaf0cd-2e01-4c92-965e-f8bcfe42d645 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.126256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaf0cd-2e01-4c92-965e-f8bcfe42d645 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.127374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacaf0cd-2e01-4c92-965e-f8bcfe42d645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:51.133072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacaf0cd-2e01-4c92-965e-f8bcfe42d645 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:51.185956 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1044/2105] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:50 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:49:51.190365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0a245-8bc8-4ffa-8091-cfed82856792 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:51.205381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0a245-8bc8-4ffa-8091-cfed82856792 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:51.213246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3acbb6-9956-44c0-af78-7a77be8949d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:51.213884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3acbb6-9956-44c0-af78-7a77be8949d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.214309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3acbb6-9956-44c0-af78-7a77be8949d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.215112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3acbb6-9956-44c0-af78-7a77be8949d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.215808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3acbb6-9956-44c0-af78-7a77be8949d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:51.217523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3acbb6-9956-44c0-af78-7a77be8949d5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:51.232475 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1062/2106] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:51.237320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaf0cd-2e01-4c92-965e-f8bcfe42d645 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:51.253571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaf0cd-2e01-4c92-965e-f8bcfe42d645 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:51.259003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb29bdd-5967-400c-a6b7-1efa6e5c1b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:51.259702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb29bdd-5967-400c-a6b7-1efa6e5c1b9a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.260096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb29bdd-5967-400c-a6b7-1efa6e5c1b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.260477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb29bdd-5967-400c-a6b7-1efa6e5c1b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.261292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb29bdd-5967-400c-a6b7-1efa6e5c1b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:51.263011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb29bdd-5967-400c-a6b7-1efa6e5c1b9a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:51.275983 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1045/2107] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:51.282432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3acbb6-9956-44c0-af78-7a77be8949d5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:51.284197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb29bdd-5967-400c-a6b7-1efa6e5c1b9a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:51.285577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fa3acbb6-9956-44c0-af78-7a77be8949d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:51.295970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb29bdd-5967-400c-a6b7-1efa6e5c1b9a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:51.304468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:51.307378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8da38-1bca-4202-b932-2093bc21ad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:51.308144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8da38-1bca-4202-b932-2093bc21ad04 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.308417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8da38-1bca-4202-b932-2093bc21ad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.308697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8da38-1bca-4202-b932-2093bc21ad04 admin admin] PATH_INFO: `/v3/projects/d4a19929b33948cd810e5d2f6afcdc0c/users/2d089a9833444a38a880f60c0aa1feb3/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.309454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3acbb6-9956-44c0-af78-7a77be8949d5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:51.316176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:51.317837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.318779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.321260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] PATH_INFO: `/v3/projects/2f2cdf5281e744898df90c7e32baca94` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.322781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f2cdf5281e744898df90c7e32baca94)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:51.325288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:51.329827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8da38-1bca-4202-b932-2093bc21ad04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4a19929b33948cd810e5d2f6afcdc0c, user_id=2d089a9833444a38a880f60c0aa1feb3, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:51.332582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8da38-1bca-4202-b932-2093bc21ad04 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:51.350260 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1046/2108] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:51 2022] PUT /identity/v3/projects/d4a19929b33948cd810e5d2f6afcdc0c/users/2d089a9833444a38a880f60c0aa1feb3/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:51.362808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cea17-b664-4428-a187-7ea5ad02405e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.363112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cea17-b664-4428-a187-7ea5ad02405e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.363387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cea17-b664-4428-a187-7ea5ad02405e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.392083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f2cdf5281e744898df90c7e32baca94', 'request_id': 'req-603c1f1e-8b96-41d9-bff8-8b550a05c6be'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:51.396114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c58cea17-b664-4428-a187-7ea5ad02405e None None] MFA Rules not processed for user `2d089a9833444a38a880f60c0aa1feb3`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:51.397345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c58cea17-b664-4428-a187-7ea5ad02405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:51.409959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:51.413410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f2cdf5281e744898df90c7e32baca94', 'request_id': 'req-603c1f1e-8b96-41d9-bff8-8b550a05c6be'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:51.433319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f2cdf5281e744898df90c7e32baca94', 'request_id': 'req-603c1f1e-8b96-41d9-bff8-8b550a05c6be'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:51.456995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] The token cache is being invalidate because project 2f2cdf5281e744898df90c7e32baca94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:51.457347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:51.459633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-603c1f1e-8b96-41d9-bff8-8b550a05c6be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:51.461949 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1063/2109] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:51 2022] DELETE /identity/v3/projects/2f2cdf5281e744898df90c7e32baca94 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:51.468684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d2a1e-84c2-4390-b260-5e35fb313d9e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.470163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d2a1e-84c2-4390-b260-5e35fb313d9e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.470643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d2a1e-84c2-4390-b260-5e35fb313d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.511273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b39d2a1e-84c2-4390-b260-5e35fb313d9e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:51.512522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b39d2a1e-84c2-4390-b260-5e35fb313d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:51.549370 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1047/2110] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:51.632065 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1064/2111] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:51.633946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9c619-18d5-498e-aaac-bfedcd73f6a1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.634240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9c619-18d5-498e-aaac-bfedcd73f6a1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.634511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9c619-18d5-498e-aaac-bfedcd73f6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.667106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-82b9c619-18d5-498e-aaac-bfedcd73f6a1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:51.668749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-82b9c619-18d5-498e-aaac-bfedcd73f6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:51.676538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d2a1e-84c2-4390-b260-5e35fb313d9e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:51.678072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b39d2a1e-84c2-4390-b260-5e35fb313d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:51.681076 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1048/2112] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:51.685664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9c619-18d5-498e-aaac-bfedcd73f6a1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:51.697549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9c619-18d5-498e-aaac-bfedcd73f6a1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:51.702114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1c386-9ebf-49e7-9dd6-79086fd824be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:51.704229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c386-9ebf-49e7-9dd6-79086fd824be admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.704764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c386-9ebf-49e7-9dd6-79086fd824be admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.705569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c386-9ebf-49e7-9dd6-79086fd824be admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.706339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1c386-9ebf-49e7-9dd6-79086fd824be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:51.706740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1c386-9ebf-49e7-9dd6-79086fd824be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:51.708476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1c386-9ebf-49e7-9dd6-79086fd824be admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:51.731540 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1049/2113] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:51.732568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1c386-9ebf-49e7-9dd6-79086fd824be admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:51.745382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1c386-9ebf-49e7-9dd6-79086fd824be admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:51.753245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c72de-c747-4ace-8c48-de590d1b7c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:51.754282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c72de-c747-4ace-8c48-de590d1b7c0f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.754764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c72de-c747-4ace-8c48-de590d1b7c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.755242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c72de-c747-4ace-8c48-de590d1b7c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.756097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c72de-c747-4ace-8c48-de590d1b7c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:51.758924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c72de-c747-4ace-8c48-de590d1b7c0f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:51.775208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d2a1e-84c2-4390-b260-5e35fb313d9e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:51.788409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd844a0b-ebe7-4da4-ace1-7117703f1416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:51.789029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd844a0b-ebe7-4da4-ace1-7117703f1416 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.789329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd844a0b-ebe7-4da4-ace1-7117703f1416 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.790131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd844a0b-ebe7-4da4-ace1-7117703f1416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.801868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd844a0b-ebe7-4da4-ace1-7117703f1416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:51.804380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd844a0b-ebe7-4da4-ace1-7117703f1416 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:51.816949 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1050/2114] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:51 2022] POST /identity/v3/projects => generated 391 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:51.818731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c72de-c747-4ace-8c48-de590d1b7c0f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:51.820391 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1065/2115] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:51.922942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c72de-c747-4ace-8c48-de590d1b7c0f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:51.932835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db47e3-cc6a-4929-8357-367d3470e47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:51.933744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db47e3-cc6a-4929-8357-367d3470e47a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:51.934271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db47e3-cc6a-4929-8357-367d3470e47a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:51.934763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db47e3-cc6a-4929-8357-367d3470e47a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:51.935622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db47e3-cc6a-4929-8357-367d3470e47a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:51.939733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db47e3-cc6a-4929-8357-367d3470e47a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:51.976035 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1051/2116] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:51 2022] POST /identity/v3/users => generated 296 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:51.982225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd844a0b-ebe7-4da4-ace1-7117703f1416 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:52.001617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd844a0b-ebe7-4da4-ace1-7117703f1416 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:52.008331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee20be0-f17f-498e-b438-482bc4b1fb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:52.009374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee20be0-f17f-498e-b438-482bc4b1fb28 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:52.009756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee20be0-f17f-498e-b438-482bc4b1fb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:52.010116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee20be0-f17f-498e-b438-482bc4b1fb28 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:52.010854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee20be0-f17f-498e-b438-482bc4b1fb28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:52.013177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee20be0-f17f-498e-b438-482bc4b1fb28 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:52.031425 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1066/2117] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:52.033577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db47e3-cc6a-4929-8357-367d3470e47a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:52.050251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db47e3-cc6a-4929-8357-367d3470e47a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:52.055327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efbaa1-c1f0-4487-90f8-9b0059fe142d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:52.056167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efbaa1-c1f0-4487-90f8-9b0059fe142d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:52.056536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efbaa1-c1f0-4487-90f8-9b0059fe142d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:52.056894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efbaa1-c1f0-4487-90f8-9b0059fe142d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:52.057935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efbaa1-c1f0-4487-90f8-9b0059fe142d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:52.059760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efbaa1-c1f0-4487-90f8-9b0059fe142d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:52.078020 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1052/2118] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:52.080763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee20be0-f17f-498e-b438-482bc4b1fb28 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:52.087585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:52.106934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee20be0-f17f-498e-b438-482bc4b1fb28 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:52.135318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8df43-be4e-4681-890c-f77d6a0da1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:52.135721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8df43-be4e-4681-890c-f77d6a0da1e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:52.135995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8df43-be4e-4681-890c-f77d6a0da1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:52.136263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8df43-be4e-4681-890c-f77d6a0da1e0 admin admin] PATH_INFO: `/v3/projects/6c295540b8aa426d9c0ba8de59f6db86/users/5948a633cdea4adfbca508540f1e2710/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:52.174106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8df43-be4e-4681-890c-f77d6a0da1e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c295540b8aa426d9c0ba8de59f6db86, user_id=5948a633cdea4adfbca508540f1e2710, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:52.178211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8df43-be4e-4681-890c-f77d6a0da1e0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:52.219884 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1067/2119] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:52 2022] PUT /identity/v3/projects/6c295540b8aa426d9c0ba8de59f6db86/users/5948a633cdea4adfbca508540f1e2710/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:52.234274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c952489-13eb-499a-a790-bfa3fb559cfb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:52.235548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c952489-13eb-499a-a790-bfa3fb559cfb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:52.235980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c952489-13eb-499a-a790-bfa3fb559cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:52.305537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5c952489-13eb-499a-a790-bfa3fb559cfb None None] MFA Rules not processed for user `5948a633cdea4adfbca508540f1e2710`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:52.309740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5c952489-13eb-499a-a790-bfa3fb559cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:52.378274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:52.441077 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1053/2120] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:52.443383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8df43-be4e-4681-890c-f77d6a0da1e0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:52.523876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8df43-be4e-4681-890c-f77d6a0da1e0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:52.529309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96748118-1793-40a8-bdc7-35f262de9d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:52.529977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748118-1793-40a8-bdc7-35f262de9d1d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:52.530320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748118-1793-40a8-bdc7-35f262de9d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:52.530820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748118-1793-40a8-bdc7-35f262de9d1d admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:52.531548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96748118-1793-40a8-bdc7-35f262de9d1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:52.533794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96748118-1793-40a8-bdc7-35f262de9d1d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:52.575462 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1068/2121] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:52 2022] POST /identity/v3/projects => generated 393 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:52.576316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c952489-13eb-499a-a790-bfa3fb559cfb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:52.589322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c952489-13eb-499a-a790-bfa3fb559cfb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:52.668772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d5c91-5615-4624-915a-79515b93f56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:52.669566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d5c91-5615-4624-915a-79515b93f56f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:52.669900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d5c91-5615-4624-915a-79515b93f56f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:52.670632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d5c91-5615-4624-915a-79515b93f56f admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:52.671120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d5c91-5615-4624-915a-79515b93f56f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:52.674771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d5c91-5615-4624-915a-79515b93f56f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:52.739403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96748118-1793-40a8-bdc7-35f262de9d1d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:52.740593 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1054/2122] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:52 2022] POST /identity/v3/users => generated 303 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:49:52.756360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96748118-1793-40a8-bdc7-35f262de9d1d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:52.794838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d5c91-5615-4624-915a-79515b93f56f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:52.797752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af697ad-73d8-4881-a699-ac0b231fff5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:52.798190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ef4d5c91-5615-4624-915a-79515b93f56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:52.799356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af697ad-73d8-4881-a699-ac0b231fff5b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:52.799834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af697ad-73d8-4881-a699-ac0b231fff5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:52.800250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af697ad-73d8-4881-a699-ac0b231fff5b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:52.801125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af697ad-73d8-4881-a699-ac0b231fff5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:52.802920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af697ad-73d8-4881-a699-ac0b231fff5b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:52.816800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d5c91-5615-4624-915a-79515b93f56f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:52.821331 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1069/2123] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:52.822261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:52.823114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af697ad-73d8-4881-a699-ac0b231fff5b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:52.823628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:52.824000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:52.824361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] PATH_INFO: `/v3/users/2d089a9833444a38a880f60c0aa1feb3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:52.825685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d089a9833444a38a880f60c0aa1feb3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:52.827623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:52.839094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af697ad-73d8-4881-a699-ac0b231fff5b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:52.846577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553011e8-2dd8-429e-8b09-82219622cb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:52.849527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553011e8-2dd8-429e-8b09-82219622cb8b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:52.850402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553011e8-2dd8-429e-8b09-82219622cb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:52.850704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553011e8-2dd8-429e-8b09-82219622cb8b admin admin] PATH_INFO: `/v3/projects/09b5e23eb3eb4a79ae61820b6b333d58/users/40c6d0c6dc654842be06e5b0bd82068f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:52.869189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553011e8-2dd8-429e-8b09-82219622cb8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b5e23eb3eb4a79ae61820b6b333d58, user_id=40c6d0c6dc654842be06e5b0bd82068f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:52.872367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553011e8-2dd8-429e-8b09-82219622cb8b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:52.904690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d089a9833444a38a880f60c0aa1feb3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:52.910550 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1070/2124] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:52 2022] PUT /identity/v3/projects/09b5e23eb3eb4a79ae61820b6b333d58/users/40c6d0c6dc654842be06e5b0bd82068f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:52.914784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553011e8-2dd8-429e-8b09-82219622cb8b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:52.916932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d089a9833444a38a880f60c0aa1feb3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:52.935446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d089a9833444a38a880f60c0aa1feb3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:52.945173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d089a9833444a38a880f60c0aa1feb3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:52.947007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3270091c-3b3a-4664-bea7-4282b93dba02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d089a9833444a38a880f60c0aa1feb3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:52.955367 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1055/2125] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:52 2022] DELETE /identity/v3/users/2d089a9833444a38a880f60c0aa1feb3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:49:52.960069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b659df6-2c0f-4b6b-a38e-79ef85fe0248 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:52.960376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b659df6-2c0f-4b6b-a38e-79ef85fe0248 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:52.960821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b659df6-2c0f-4b6b-a38e-79ef85fe0248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:52.982407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4b659df6-2c0f-4b6b-a38e-79ef85fe0248 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:52.983632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4b659df6-2c0f-4b6b-a38e-79ef85fe0248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:53.003088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553011e8-2dd8-429e-8b09-82219622cb8b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:53.020865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:53.053998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:53.065249 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1056/2126] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:53.075665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b659df6-2c0f-4b6b-a38e-79ef85fe0248 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:53.077551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4b659df6-2c0f-4b6b-a38e-79ef85fe0248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:53.087315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1d1ec-5355-44b8-a9c0-f96c021cc168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:53.088022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d1ec-5355-44b8-a9c0-f96c021cc168 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:53.088371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d1ec-5355-44b8-a9c0-f96c021cc168 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:53.088702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d1ec-5355-44b8-a9c0-f96c021cc168 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:53.089468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1d1ec-5355-44b8-a9c0-f96c021cc168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:53.091173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1d1ec-5355-44b8-a9c0-f96c021cc168 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:53.103735 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1071/2127] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:53.107703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1d1ec-5355-44b8-a9c0-f96c021cc168 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:53.110415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a0f1d1ec-5355-44b8-a9c0-f96c021cc168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:53.129478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1d1ec-5355-44b8-a9c0-f96c021cc168 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:53.143590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6102ae5-eed5-4e44-add3-f3ab94c01596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:53.144363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102ae5-eed5-4e44-add3-f3ab94c01596 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:53.144714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102ae5-eed5-4e44-add3-f3ab94c01596 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:53.145112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102ae5-eed5-4e44-add3-f3ab94c01596 admin admin] PATH_INFO: `/v3/projects/09b5e23eb3eb4a79ae61820b6b333d58/users/40c6d0c6dc654842be06e5b0bd82068f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:53.147485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6102ae5-eed5-4e44-add3-f3ab94c01596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b5e23eb3eb4a79ae61820b6b333d58, user_id=40c6d0c6dc654842be06e5b0bd82068f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:53.150402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6102ae5-eed5-4e44-add3-f3ab94c01596 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:53.163270 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1072/2128] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:53 2022] PUT /identity/v3/projects/09b5e23eb3eb4a79ae61820b6b333d58/users/40c6d0c6dc654842be06e5b0bd82068f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:53.173303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b659df6-2c0f-4b6b-a38e-79ef85fe0248 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:53.174877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc39e6e-d55c-48f2-abe0-93475dfd508b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:53.175239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc39e6e-d55c-48f2-abe0-93475dfd508b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:53.175594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc39e6e-d55c-48f2-abe0-93475dfd508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:53.223271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cdc39e6e-d55c-48f2-abe0-93475dfd508b None None] MFA Rules not processed for user `40c6d0c6dc654842be06e5b0bd82068f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:53.224812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cdc39e6e-d55c-48f2-abe0-93475dfd508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:53.236834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad892e-2706-46e7-ac10-aa818fb84bdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:53.237878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad892e-2706-46e7-ac10-aa818fb84bdc service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:53.238257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad892e-2706-46e7-ac10-aa818fb84bdc service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:53.239369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad892e-2706-46e7-ac10-aa818fb84bdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:53.353966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad892e-2706-46e7-ac10-aa818fb84bdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:53.355726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad892e-2706-46e7-ac10-aa818fb84bdc service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:53.396924 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1057/2129] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:53.399219 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1073/2130] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:53 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:53.593760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad892e-2706-46e7-ac10-aa818fb84bdc service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:53.597141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-02ad892e-2706-46e7-ac10-aa818fb84bdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:53.719584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:53.782879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad892e-2706-46e7-ac10-aa818fb84bdc service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:53.800195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac23b4-5c43-4ada-87a5-58144786fd92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:53.800967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac23b4-5c43-4ada-87a5-58144786fd92 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:53.801267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac23b4-5c43-4ada-87a5-58144786fd92 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:53.801533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac23b4-5c43-4ada-87a5-58144786fd92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:53.818010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac23b4-5c43-4ada-87a5-58144786fd92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:53.820170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac23b4-5c43-4ada-87a5-58144786fd92 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:53.843949 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1058/2131] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:53 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:54.302371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc39e6e-d55c-48f2-abe0-93475dfd508b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:54.304373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cdc39e6e-d55c-48f2-abe0-93475dfd508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:54.326659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc39e6e-d55c-48f2-abe0-93475dfd508b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:54.345414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:54.346272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:54.346558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:54.347162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] PATH_INFO: `/v3/projects/d4a19929b33948cd810e5d2f6afcdc0c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:54.348527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4a19929b33948cd810e5d2f6afcdc0c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:54.351002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:54.428290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4a19929b33948cd810e5d2f6afcdc0c', 'request_id': 'req-6b99114e-a283-4560-9a1d-64eee3a393b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:54.444906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4a19929b33948cd810e5d2f6afcdc0c', 'request_id': 'req-6b99114e-a283-4560-9a1d-64eee3a393b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:54.462501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4a19929b33948cd810e5d2f6afcdc0c', 'request_id': 'req-6b99114e-a283-4560-9a1d-64eee3a393b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:54.479024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] The token cache is being invalidate because project d4a19929b33948cd810e5d2f6afcdc0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:54.479319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:54.480028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6b99114e-a283-4560-9a1d-64eee3a393b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:54.481561 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1074/2132] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:54 2022] DELETE /identity/v3/projects/d4a19929b33948cd810e5d2f6afcdc0c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:54.488469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c233b-adb2-46ad-a7b7-32edd0d88c10 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:54.489055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c233b-adb2-46ad-a7b7-32edd0d88c10 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:54.489329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c233b-adb2-46ad-a7b7-32edd0d88c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:54.525848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e35c233b-adb2-46ad-a7b7-32edd0d88c10 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:54.528063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e35c233b-adb2-46ad-a7b7-32edd0d88c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:54.680727 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1059/2133] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:54.681887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1178ac-b96c-4b6b-9a17-2a649ae37675 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:54.682171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1178ac-b96c-4b6b-9a17-2a649ae37675 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:54.682488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1178ac-b96c-4b6b-9a17-2a649ae37675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:54.728644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ea1178ac-b96c-4b6b-9a17-2a649ae37675 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:54.730090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ea1178ac-b96c-4b6b-9a17-2a649ae37675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:54.732521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c233b-adb2-46ad-a7b7-32edd0d88c10 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:54.735351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e35c233b-adb2-46ad-a7b7-32edd0d88c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:54.762385 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1075/2134] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:54.762571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c233b-adb2-46ad-a7b7-32edd0d88c10 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:54.768189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1178ac-b96c-4b6b-9a17-2a649ae37675 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:54.777802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:54.778453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:54.778729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:54.778999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] PATH_INFO: `/v3/users/5948a633cdea4adfbca508540f1e2710` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:54.780686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5948a633cdea4adfbca508540f1e2710)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:54.785090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:54.795005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1178ac-b96c-4b6b-9a17-2a649ae37675 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:54.807170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937609b5-443d-4652-8a30-994bb6745e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:54.808628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937609b5-443d-4652-8a30-994bb6745e51 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:54.808916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937609b5-443d-4652-8a30-994bb6745e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:54.809297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937609b5-443d-4652-8a30-994bb6745e51 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:54.809955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937609b5-443d-4652-8a30-994bb6745e51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:54.810539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937609b5-443d-4652-8a30-994bb6745e51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:54.811909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937609b5-443d-4652-8a30-994bb6745e51 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:54.856223 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1076/2135] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:54.858412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937609b5-443d-4652-8a30-994bb6745e51 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:54.884806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937609b5-443d-4652-8a30-994bb6745e51 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:54.898770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8de73-2157-44aa-a779-9e2b9592fe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:54.899730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8de73-2157-44aa-a779-9e2b9592fe4f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:54.900355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8de73-2157-44aa-a779-9e2b9592fe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:54.903205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8de73-2157-44aa-a779-9e2b9592fe4f admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:54.904446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8de73-2157-44aa-a779-9e2b9592fe4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:54.904857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5948a633cdea4adfbca508540f1e2710'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:54.907904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8de73-2157-44aa-a779-9e2b9592fe4f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:54.917643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5948a633cdea4adfbca508540f1e2710'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:54.920775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:54.936776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5948a633cdea4adfbca508540f1e2710'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:54.965346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5948a633cdea4adfbca508540f1e2710'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:54.970610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2099de47-3a77-4879-9fc9-1df68d1511a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5948a633cdea4adfbca508540f1e2710'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:54.980172 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1077/2136] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:54 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:54.986602 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1060/2137] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:54 2022] DELETE /identity/v3/users/5948a633cdea4adfbca508540f1e2710 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:49:54.987798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8de73-2157-44aa-a779-9e2b9592fe4f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:54.990885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470bcd4-b670-4fc0-b994-203dd9125774 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:54.991172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470bcd4-b670-4fc0-b994-203dd9125774 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:54.991444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470bcd4-b670-4fc0-b994-203dd9125774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:54.993133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-18a8de73-2157-44aa-a779-9e2b9592fe4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:55.035016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3470bcd4-b670-4fc0-b994-203dd9125774 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:55.036332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3470bcd4-b670-4fc0-b994-203dd9125774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:55.141832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:55.186970 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1061/2138] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:55.193903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8de73-2157-44aa-a779-9e2b9592fe4f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:55.202124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9b53e-cbe1-42c9-bd6e-ac172dc3e802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:55.202755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9b53e-cbe1-42c9-bd6e-ac172dc3e802 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:55.203022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9b53e-cbe1-42c9-bd6e-ac172dc3e802 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:55.206015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9b53e-cbe1-42c9-bd6e-ac172dc3e802 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:55.206771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9b53e-cbe1-42c9-bd6e-ac172dc3e802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:55.213601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9b53e-cbe1-42c9-bd6e-ac172dc3e802 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:55.228063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470bcd4-b670-4fc0-b994-203dd9125774 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:55.229842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3470bcd4-b670-4fc0-b994-203dd9125774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:55.263219 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1078/2139] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:54 2022] POST /identity/v3/users => generated 291 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:55.268325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9b53e-cbe1-42c9-bd6e-ac172dc3e802 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:55.294798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9b53e-cbe1-42c9-bd6e-ac172dc3e802 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:55.306338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa54e4-ee59-496d-9cd3-ac0d17436866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:55.307032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa54e4-ee59-496d-9cd3-ac0d17436866 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:55.307310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa54e4-ee59-496d-9cd3-ac0d17436866 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:55.307584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa54e4-ee59-496d-9cd3-ac0d17436866 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:55.310253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa54e4-ee59-496d-9cd3-ac0d17436866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:55.315161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa54e4-ee59-496d-9cd3-ac0d17436866 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:55.355139 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1079/2140] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:55.356678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa54e4-ee59-496d-9cd3-ac0d17436866 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:55.373877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470bcd4-b670-4fc0-b994-203dd9125774 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:55.377772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa54e4-ee59-496d-9cd3-ac0d17436866 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:55.378972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8988d3-b746-48d0-a828-7e2e0b091c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:55.380187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8988d3-b746-48d0-a828-7e2e0b091c65 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:55.380716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8988d3-b746-48d0-a828-7e2e0b091c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:55.381458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8988d3-b746-48d0-a828-7e2e0b091c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:55.394912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160311a-3b5b-4c77-8bb9-7ffefea1d921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:55.397718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160311a-3b5b-4c77-8bb9-7ffefea1d921 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:55.398063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160311a-3b5b-4c77-8bb9-7ffefea1d921 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:55.398376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160311a-3b5b-4c77-8bb9-7ffefea1d921 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:55.399154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160311a-3b5b-4c77-8bb9-7ffefea1d921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:55.400847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160311a-3b5b-4c77-8bb9-7ffefea1d921 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:55.401957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8988d3-b746-48d0-a828-7e2e0b091c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:55.404296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8988d3-b746-48d0-a828-7e2e0b091c65 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:55.418591 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1062/2141] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:55.423735 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1080/2142] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:55.429769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8988d3-b746-48d0-a828-7e2e0b091c65 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:55.442204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8988d3-b746-48d0-a828-7e2e0b091c65 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:55.447717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d775c-34ac-40f2-b665-3cfa87eeb525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:55.448434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d775c-34ac-40f2-b665-3cfa87eeb525 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:55.448706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d775c-34ac-40f2-b665-3cfa87eeb525 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:55.448974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d775c-34ac-40f2-b665-3cfa87eeb525 admin admin] PATH_INFO: `/v3/projects/869f1ee4c91c4067a3e32e39155b30a6/users/03bc9235fe1c4fc1b3097a913cbe1052/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:55.501355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d775c-34ac-40f2-b665-3cfa87eeb525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=869f1ee4c91c4067a3e32e39155b30a6, user_id=03bc9235fe1c4fc1b3097a913cbe1052, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:55.505003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d775c-34ac-40f2-b665-3cfa87eeb525 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:55.531058 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1063/2143] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:55 2022] PUT /identity/v3/projects/869f1ee4c91c4067a3e32e39155b30a6/users/03bc9235fe1c4fc1b3097a913cbe1052/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:55.536632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c3389-b0a7-479c-ac9a-15bc6756a8ee None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:55.536898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c3389-b0a7-479c-ac9a-15bc6756a8ee None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:55.537837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c3389-b0a7-479c-ac9a-15bc6756a8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:55.561346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ac1c3389-b0a7-479c-ac9a-15bc6756a8ee None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:55.561592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac1c3389-b0a7-479c-ac9a-15bc6756a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:55.659286 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1081/2144] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:55.687469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d775c-34ac-40f2-b665-3cfa87eeb525 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:55.690775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4f7d775c-34ac-40f2-b665-3cfa87eeb525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:55.804717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d775c-34ac-40f2-b665-3cfa87eeb525 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:55.811373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70468f8-81d9-4468-87b0-6b25cb74c3a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:55.812211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70468f8-81d9-4468-87b0-6b25cb74c3a0 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:55.813354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70468f8-81d9-4468-87b0-6b25cb74c3a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:55.813935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70468f8-81d9-4468-87b0-6b25cb74c3a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:55.831494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70468f8-81d9-4468-87b0-6b25cb74c3a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:55.833743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70468f8-81d9-4468-87b0-6b25cb74c3a0 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:55.858648 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1064/2145] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:56.032788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c3389-b0a7-479c-ac9a-15bc6756a8ee None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:56.034882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac1c3389-b0a7-479c-ac9a-15bc6756a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:56.137399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:56.197928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c3389-b0a7-479c-ac9a-15bc6756a8ee None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:56.206522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7098f-1cba-45aa-9a33-d2e2e46672b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:56.207236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7098f-1cba-45aa-9a33-d2e2e46672b3 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:56.207525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7098f-1cba-45aa-9a33-d2e2e46672b3 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:56.207794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7098f-1cba-45aa-9a33-d2e2e46672b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:56.209676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9dc7098f-1cba-45aa-9a33-d2e2e46672b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:56.289554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7098f-1cba-45aa-9a33-d2e2e46672b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:56.291193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7098f-1cba-45aa-9a33-d2e2e46672b3 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:56.303732 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1082/2146] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:56.807378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70468f8-81d9-4468-87b0-6b25cb74c3a0 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:56.817114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c70468f8-81d9-4468-87b0-6b25cb74c3a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:56.831488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70468f8-81d9-4468-87b0-6b25cb74c3a0 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:56.847826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:56.848813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:56.849329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:56.849604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] PATH_INFO: `/v3/projects/6c295540b8aa426d9c0ba8de59f6db86` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:56.853992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c295540b8aa426d9c0ba8de59f6db86)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:56.856062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:57.077131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c295540b8aa426d9c0ba8de59f6db86', 'request_id': 'req-bbfba56d-2feb-4274-b8af-9e82ff16e912'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:57.092248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c295540b8aa426d9c0ba8de59f6db86', 'request_id': 'req-bbfba56d-2feb-4274-b8af-9e82ff16e912'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:57.094813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:57.104690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c295540b8aa426d9c0ba8de59f6db86', 'request_id': 'req-bbfba56d-2feb-4274-b8af-9e82ff16e912'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:57.132491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] The token cache is being invalidate because project 6c295540b8aa426d9c0ba8de59f6db86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:57.132806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:57.136008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:57.142568 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1065/2147] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:56 2022] DELETE /identity/v3/projects/6c295540b8aa426d9c0ba8de59f6db86 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:57.143627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7098f-1cba-45aa-9a33-d2e2e46672b3 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:57.159799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7098f-1cba-45aa-9a33-d2e2e46672b3 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:57.347865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:57.348508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:57.348783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:57.349067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] PATH_INFO: `/v3/users/40c6d0c6dc654842be06e5b0bd82068f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:57.351390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c6d0c6dc654842be06e5b0bd82068f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:57.355056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:57.472616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c6d0c6dc654842be06e5b0bd82068f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:57.493482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c6d0c6dc654842be06e5b0bd82068f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:57.514981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c6d0c6dc654842be06e5b0bd82068f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:57.525645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c6d0c6dc654842be06e5b0bd82068f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:57.526671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f725679-d994-4206-8135-cf2c5ea8d32b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c6d0c6dc654842be06e5b0bd82068f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:57.545998 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1083/2148] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:57 2022] DELETE /identity/v3/users/40c6d0c6dc654842be06e5b0bd82068f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:57.928341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:57.929938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:57.946724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfba56d-2feb-4274-b8af-9e82ff16e912 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:58.035348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:58.036238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:58.036665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:58.037121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] PATH_INFO: `/v3/projects/09b5e23eb3eb4a79ae61820b6b333d58` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:58.038598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b5e23eb3eb4a79ae61820b6b333d58)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:58.041121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:58.148140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b5e23eb3eb4a79ae61820b6b333d58', 'request_id': 'req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:58.167491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b5e23eb3eb4a79ae61820b6b333d58', 'request_id': 'req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:58.183757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b5e23eb3eb4a79ae61820b6b333d58', 'request_id': 'req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:58.223158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] The token cache is being invalidate because project 09b5e23eb3eb4a79ae61820b6b333d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:49:58.223490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:58.225395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba9c2856-8181-4c9d-a3b0-54d7e25387f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:49:58.232372 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1066/2149] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:57 2022] DELETE /identity/v3/projects/09b5e23eb3eb4a79ae61820b6b333d58 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:58.233582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77f76d-3e8f-4fe6-9a5d-0a1f6d345aef None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:58.234048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77f76d-3e8f-4fe6-9a5d-0a1f6d345aef None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:58.234138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77f76d-3e8f-4fe6-9a5d-0a1f6d345aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:58.249565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:58.273822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bd77f76d-3e8f-4fe6-9a5d-0a1f6d345aef None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:58.276189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bd77f76d-3e8f-4fe6-9a5d-0a1f6d345aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:58.397379 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1084/2150] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:58.403437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c990-598d-4a32-ae00-97a4ad3f8f62 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:58.403723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c990-598d-4a32-ae00-97a4ad3f8f62 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:58.403999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c990-598d-4a32-ae00-97a4ad3f8f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:58.430752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1fd5c990-598d-4a32-ae00-97a4ad3f8f62 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:58.432471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1fd5c990-598d-4a32-ae00-97a4ad3f8f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:58.448232 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1067/2151] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:49:58.454577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77f76d-3e8f-4fe6-9a5d-0a1f6d345aef None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:58.476200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77f76d-3e8f-4fe6-9a5d-0a1f6d345aef None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:58.490967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619828ac-160d-42a4-b9a0-1efbd6516819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:58.493699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619828ac-160d-42a4-b9a0-1efbd6516819 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:58.494819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619828ac-160d-42a4-b9a0-1efbd6516819 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:58.495264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619828ac-160d-42a4-b9a0-1efbd6516819 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:58.497314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619828ac-160d-42a4-b9a0-1efbd6516819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:49:58.497795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619828ac-160d-42a4-b9a0-1efbd6516819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:58.502395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619828ac-160d-42a4-b9a0-1efbd6516819 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:58.560373 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1085/2152] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:49:58.565051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5c990-598d-4a32-ae00-97a4ad3f8f62 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:58.591115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5c990-598d-4a32-ae00-97a4ad3f8f62 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:58.619617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972c35b-6646-4319-a003-fbaaab77959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:58.623079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972c35b-6646-4319-a003-fbaaab77959b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:58.624074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972c35b-6646-4319-a003-fbaaab77959b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:58.625931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972c35b-6646-4319-a003-fbaaab77959b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:58.628586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972c35b-6646-4319-a003-fbaaab77959b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:58.632829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972c35b-6646-4319-a003-fbaaab77959b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:58.723641 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1068/2153] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:58 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:58.727875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619828ac-160d-42a4-b9a0-1efbd6516819 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:58.743304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619828ac-160d-42a4-b9a0-1efbd6516819 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:58.876887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915986e-ac9e-483a-bbf9-27b00f0410bc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:58.877193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915986e-ac9e-483a-bbf9-27b00f0410bc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:58.877911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915986e-ac9e-483a-bbf9-27b00f0410bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:58.881466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce520be-bb3f-4f06-b8a5-bb3c19903ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:58.882098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce520be-bb3f-4f06-b8a5-bb3c19903ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:58.882372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce520be-bb3f-4f06-b8a5-bb3c19903ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:58.882643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce520be-bb3f-4f06-b8a5-bb3c19903ff2 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:58.883281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce520be-bb3f-4f06-b8a5-bb3c19903ff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:58.885251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce520be-bb3f-4f06-b8a5-bb3c19903ff2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:58.911158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7915986e-ac9e-483a-bbf9-27b00f0410bc None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:58.913099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7915986e-ac9e-483a-bbf9-27b00f0410bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:58.935701 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1086/2154] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:58 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:58.938164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce520be-bb3f-4f06-b8a5-bb3c19903ff2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:58.973464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce520be-bb3f-4f06-b8a5-bb3c19903ff2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:58.993545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b3259-7521-4526-bec0-f1f58bdaa92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:58.994167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b3259-7521-4526-bec0-f1f58bdaa92e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:58.994445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b3259-7521-4526-bec0-f1f58bdaa92e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:58.994714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b3259-7521-4526-bec0-f1f58bdaa92e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:58.995362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b3259-7521-4526-bec0-f1f58bdaa92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:58.997002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b3259-7521-4526-bec0-f1f58bdaa92e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:59.039880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:59.045455 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1087/2155] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:49:59.050548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b3259-7521-4526-bec0-f1f58bdaa92e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:59.073095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b3259-7521-4526-bec0-f1f58bdaa92e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:59.092192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19416c1a-3c5d-4553-8b9c-fd4dce83e078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:59.092814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19416c1a-3c5d-4553-8b9c-fd4dce83e078 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:59.093114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19416c1a-3c5d-4553-8b9c-fd4dce83e078 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:59.093389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19416c1a-3c5d-4553-8b9c-fd4dce83e078 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:59.094149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19416c1a-3c5d-4553-8b9c-fd4dce83e078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:59.096563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19416c1a-3c5d-4553-8b9c-fd4dce83e078 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:59.107653 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1069/2156] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:49:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:59.124331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915986e-ac9e-483a-bbf9-27b00f0410bc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:59.128630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7915986e-ac9e-483a-bbf9-27b00f0410bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:59.145881 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1088/2157] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:49:59.150304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19416c1a-3c5d-4553-8b9c-fd4dce83e078 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:59.170880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19416c1a-3c5d-4553-8b9c-fd4dce83e078 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:59.196046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319723eb-468b-40a8-8f40-1afbe1ae1f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:59.196726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319723eb-468b-40a8-8f40-1afbe1ae1f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:59.197001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319723eb-468b-40a8-8f40-1afbe1ae1f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:59.197303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319723eb-468b-40a8-8f40-1afbe1ae1f11 admin admin] PATH_INFO: `/v3/projects/d35a8fc12f58435a9950cc7436eafb3b/users/febedcafbc7a487dbe6778d54387d78c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:59.245998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319723eb-468b-40a8-8f40-1afbe1ae1f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d35a8fc12f58435a9950cc7436eafb3b, user_id=febedcafbc7a487dbe6778d54387d78c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:59.248729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319723eb-468b-40a8-8f40-1afbe1ae1f11 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:59.301500 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1089/2158] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:49:59 2022] PUT /identity/v3/projects/d35a8fc12f58435a9950cc7436eafb3b/users/febedcafbc7a487dbe6778d54387d78c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:49:59.307158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf430b1-32d2-4682-bea7-ba6db01f9005 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:59.307545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf430b1-32d2-4682-bea7-ba6db01f9005 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:59.307819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf430b1-32d2-4682-bea7-ba6db01f9005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:59.348902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915986e-ac9e-483a-bbf9-27b00f0410bc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:59.377253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4cf430b1-32d2-4682-bea7-ba6db01f9005 None None] MFA Rules not processed for user `febedcafbc7a487dbe6778d54387d78c`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:49:59.380718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4cf430b1-32d2-4682-bea7-ba6db01f9005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:49:59.490051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:59.554089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9f13d-033c-410e-9d59-e1a64123cba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:59.554726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9f13d-033c-410e-9d59-e1a64123cba6 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:59.555006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9f13d-033c-410e-9d59-e1a64123cba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:59.555277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9f13d-033c-410e-9d59-e1a64123cba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:59.582763 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1090/2159] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:49:59.587401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf430b1-32d2-4682-bea7-ba6db01f9005 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:59.680452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9f13d-033c-410e-9d59-e1a64123cba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:59.682048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9f13d-033c-410e-9d59-e1a64123cba6 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:59.704659 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1070/2160] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:49:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:59.722400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf430b1-32d2-4682-bea7-ba6db01f9005 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:59.728017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa38e7-3819-4081-afcd-16a0fd95bd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:59.728955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa38e7-3819-4081-afcd-16a0fd95bd42 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:59.729478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa38e7-3819-4081-afcd-16a0fd95bd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:59.729966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa38e7-3819-4081-afcd-16a0fd95bd42 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:59.730808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa38e7-3819-4081-afcd-16a0fd95bd42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:59.733287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa38e7-3819-4081-afcd-16a0fd95bd42 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:59.777932 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1091/2161] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:49:59 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:59.779023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9f13d-033c-410e-9d59-e1a64123cba6 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:59.793377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9f13d-033c-410e-9d59-e1a64123cba6 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:49:59.856134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:59.932095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9871a-8ea8-4a49-881b-86baf9389b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:49:59.932803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9871a-8ea8-4a49-881b-86baf9389b90 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:49:59.933215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9871a-8ea8-4a49-881b-86baf9389b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:49:59.933556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9871a-8ea8-4a49-881b-86baf9389b90 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:49:59.934267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9871a-8ea8-4a49-881b-86baf9389b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:49:59.936246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9871a-8ea8-4a49-881b-86baf9389b90 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:49:59.973878 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1071/2162] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:49:59 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:49:59.978174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa38e7-3819-4081-afcd-16a0fd95bd42 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:49:59.993674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa38e7-3819-4081-afcd-16a0fd95bd42 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:00.007909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8ec0c-3925-40fb-be51-dd8247a341fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:00.008756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8ec0c-3925-40fb-be51-dd8247a341fb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:00.009074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8ec0c-3925-40fb-be51-dd8247a341fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:00.009601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8ec0c-3925-40fb-be51-dd8247a341fb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:00.011010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8ec0c-3925-40fb-be51-dd8247a341fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:00.012849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8ec0c-3925-40fb-be51-dd8247a341fb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:00.028021 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1092/2163] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:00.033463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9871a-8ea8-4a49-881b-86baf9389b90 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:00.052489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9871a-8ea8-4a49-881b-86baf9389b90 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:00.067912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606da3d-031e-4dc7-8e3b-7b5ab7940c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:00.068722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606da3d-031e-4dc7-8e3b-7b5ab7940c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:00.069260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606da3d-031e-4dc7-8e3b-7b5ab7940c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:00.069606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606da3d-031e-4dc7-8e3b-7b5ab7940c6c admin admin] PATH_INFO: `/v3/projects/0a367b44e2994e5d9803aeca3b402d50/users/22547753ad6640b9819a96d848aa31f3/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:00.098389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606da3d-031e-4dc7-8e3b-7b5ab7940c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a367b44e2994e5d9803aeca3b402d50, user_id=22547753ad6640b9819a96d848aa31f3, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:00.104479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606da3d-031e-4dc7-8e3b-7b5ab7940c6c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:00.131988 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1072/2164] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:00 2022] PUT /identity/v3/projects/0a367b44e2994e5d9803aeca3b402d50/users/22547753ad6640b9819a96d848aa31f3/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:00.136468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8ec0c-3925-40fb-be51-dd8247a341fb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:00.167786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8ec0c-3925-40fb-be51-dd8247a341fb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:00.264214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff7a1-463b-4c99-bbba-45ff65fc3547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:00.264900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff7a1-463b-4c99-bbba-45ff65fc3547 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:00.265196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff7a1-463b-4c99-bbba-45ff65fc3547 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:00.265411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff7a1-463b-4c99-bbba-45ff65fc3547 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:00.266062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ff7a1-463b-4c99-bbba-45ff65fc3547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:00.267680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ff7a1-463b-4c99-bbba-45ff65fc3547 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:00.282691 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1093/2165] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:00.283760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606da3d-031e-4dc7-8e3b-7b5ab7940c6c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:00.295393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606da3d-031e-4dc7-8e3b-7b5ab7940c6c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:00.302018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d5f8-2024-407f-8afc-3dc6a3f440ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:00.302945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d5f8-2024-407f-8afc-3dc6a3f440ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:00.303091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d5f8-2024-407f-8afc-3dc6a3f440ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:00.303488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d5f8-2024-407f-8afc-3dc6a3f440ee admin admin] PATH_INFO: `/v3/projects/0a367b44e2994e5d9803aeca3b402d50/users/22547753ad6640b9819a96d848aa31f3/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:00.305529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d5f8-2024-407f-8afc-3dc6a3f440ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a367b44e2994e5d9803aeca3b402d50, user_id=22547753ad6640b9819a96d848aa31f3, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:00.308298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d5f8-2024-407f-8afc-3dc6a3f440ee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:00.331516 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1073/2166] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:00 2022] PUT /identity/v3/projects/0a367b44e2994e5d9803aeca3b402d50/users/22547753ad6640b9819a96d848aa31f3/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:00.333592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5715f-3d9f-42b0-8cd5-f2faee104b3f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:00.333957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5715f-3d9f-42b0-8cd5-f2faee104b3f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:00.334291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5715f-3d9f-42b0-8cd5-f2faee104b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:00.372583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6ac5715f-3d9f-42b0-8cd5-f2faee104b3f None None] MFA Rules not processed for user `22547753ad6640b9819a96d848aa31f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:00.374030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6ac5715f-3d9f-42b0-8cd5-f2faee104b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:00.408621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:00.477343 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1094/2167] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:50:00.489794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d5f8-2024-407f-8afc-3dc6a3f440ee admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:00.492259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0750d5f8-2024-407f-8afc-3dc6a3f440ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:00.502459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b3cec-f965-4ac5-8373-ced1cddea758 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:00.502735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b3cec-f965-4ac5-8373-ced1cddea758 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:00.503009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b3cec-f965-4ac5-8373-ced1cddea758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:00.537003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2c3b3cec-f965-4ac5-8373-ced1cddea758 None None] MFA Rules not processed for user `e32a1c53b4be44188b07c768405961ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:00.538318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2c3b3cec-f965-4ac5-8373-ced1cddea758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:00.638607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d5f8-2024-407f-8afc-3dc6a3f440ee admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:00.648809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11545140-72f3-4128-bca8-4a50c350d2ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:00.650767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11545140-72f3-4128-bca8-4a50c350d2ee service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:00.651900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11545140-72f3-4128-bca8-4a50c350d2ee service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:00.653075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11545140-72f3-4128-bca8-4a50c350d2ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:00.667555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11545140-72f3-4128-bca8-4a50c350d2ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:00.669311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11545140-72f3-4128-bca8-4a50c350d2ee service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:00.696525 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1074/2168] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:00.703163 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1095/2169] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:01.049791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11545140-72f3-4128-bca8-4a50c350d2ee service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:01.218857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11545140-72f3-4128-bca8-4a50c350d2ee service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:01.241679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:01.241679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:01.241679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:01.241679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] PATH_INFO: `/v3/users/febedcafbc7a487dbe6778d54387d78c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:01.242188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=febedcafbc7a487dbe6778d54387d78c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:01.242188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:01.259620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:01.316128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'febedcafbc7a487dbe6778d54387d78c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:01.326707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'febedcafbc7a487dbe6778d54387d78c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:01.343294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'febedcafbc7a487dbe6778d54387d78c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:01.356484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'febedcafbc7a487dbe6778d54387d78c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:01.358319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'febedcafbc7a487dbe6778d54387d78c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:01.367223 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1075/2170] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:50:01 2022] DELETE /identity/v3/users/febedcafbc7a487dbe6778d54387d78c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:01.374124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83eca3-3f68-4cdb-91d9-65ef59f70603 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:01.375457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83eca3-3f68-4cdb-91d9-65ef59f70603 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:01.375935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83eca3-3f68-4cdb-91d9-65ef59f70603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:01.426995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8c83eca3-3f68-4cdb-91d9-65ef59f70603 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:01.427807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8c83eca3-3f68-4cdb-91d9-65ef59f70603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:01.528629 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1096/2171] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:01.579286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:01.582531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:01.671031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb06e5-9808-4ab3-9a47-67e07ee01d0b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:01.675724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e65174-04c4-4539-a33b-cc33c651a239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:01.676619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e65174-04c4-4539-a33b-cc33c651a239 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:01.677229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e65174-04c4-4539-a33b-cc33c651a239 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:01.678672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e65174-04c4-4539-a33b-cc33c651a239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:01.691534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e65174-04c4-4539-a33b-cc33c651a239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:01.693413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e65174-04c4-4539-a33b-cc33c651a239 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:01.709311 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1076/2172] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:02.787610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83eca3-3f68-4cdb-91d9-65ef59f70603 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:02.789233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8c83eca3-3f68-4cdb-91d9-65ef59f70603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:02.808633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83eca3-3f68-4cdb-91d9-65ef59f70603 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:02.814892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:02.816472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:02.817076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:02.817750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] PATH_INFO: `/v3/users/d1929f640b7348d6901067dfa6354c49` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:02.842516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1929f640b7348d6901067dfa6354c49)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:02.862000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:02.993539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1929f640b7348d6901067dfa6354c49'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.013958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:03.018340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1929f640b7348d6901067dfa6354c49'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.029426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1929f640b7348d6901067dfa6354c49'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.039252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1929f640b7348d6901067dfa6354c49'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.040223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1929f640b7348d6901067dfa6354c49'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.048899 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1097/2173] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:50:02 2022] DELETE /identity/v3/users/d1929f640b7348d6901067dfa6354c49 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:03.287018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e65174-04c4-4539-a33b-cc33c651a239 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:03.307588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e65174-04c4-4539-a33b-cc33c651a239 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:03.340581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:03.342557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:03.444319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d766ac-42fd-4ba2-8fe7-ec3c942eefa2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:03.450977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:03.451710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:03.452188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:03.452922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] PATH_INFO: `/v3/projects/d35a8fc12f58435a9950cc7436eafb3b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:03.454225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d35a8fc12f58435a9950cc7436eafb3b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:03.456495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:03.484666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:03.485667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:03.485966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:03.486356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] PATH_INFO: `/v3/projects/f9e809ddc6b7418485892059e70926f4` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:03.488012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e809ddc6b7418485892059e70926f4)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:03.490105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:03.573712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd35a8fc12f58435a9950cc7436eafb3b', 'request_id': 'req-7eee2b4d-7327-4149-b556-84dbc852a335'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.592041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:03.615668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e809ddc6b7418485892059e70926f4', 'request_id': 'req-f9fee459-deb9-4b3b-854b-49ef35e76746'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.627804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd35a8fc12f58435a9950cc7436eafb3b', 'request_id': 'req-7eee2b4d-7327-4149-b556-84dbc852a335'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.629595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e809ddc6b7418485892059e70926f4', 'request_id': 'req-f9fee459-deb9-4b3b-854b-49ef35e76746'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.639757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e809ddc6b7418485892059e70926f4', 'request_id': 'req-f9fee459-deb9-4b3b-854b-49ef35e76746'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.659651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd35a8fc12f58435a9950cc7436eafb3b', 'request_id': 'req-7eee2b4d-7327-4149-b556-84dbc852a335'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.673216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] The token cache is being invalidate because project f9e809ddc6b7418485892059e70926f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:50:03.673685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.674418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.676350 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1077/2174] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:50:03 2022] DELETE /identity/v3/projects/f9e809ddc6b7418485892059e70926f4 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:03.686868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:03.687966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] The token cache is being invalidate because project d35a8fc12f58435a9950cc7436eafb3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:50:03.688258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.688397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:03.689824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:03.697451 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1098/2175] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:50:03 2022] DELETE /identity/v3/projects/d35a8fc12f58435a9950cc7436eafb3b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:03.698888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:03.701645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:03.852357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fee459-deb9-4b3b-854b-49ef35e76746 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:03.864000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:03.864654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:03.864923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:03.865230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] PATH_INFO: `/v3/users/5acc23498df94c3886781c23cfc7d6de` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:03.867673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee2b4d-7327-4149-b556-84dbc852a335 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:03.869149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5acc23498df94c3886781c23cfc7d6de)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:03.871243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:03.880136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:03.880770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:03.881065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:03.881476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] PATH_INFO: `/v3/users/22547753ad6640b9819a96d848aa31f3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:03.890673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=22547753ad6640b9819a96d848aa31f3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:03.893370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:03.912178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:03.990399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5acc23498df94c3886781c23cfc7d6de'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:04.002461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5acc23498df94c3886781c23cfc7d6de'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:04.008356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22547753ad6640b9819a96d848aa31f3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:04.020436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5acc23498df94c3886781c23cfc7d6de'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:04.033472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5acc23498df94c3886781c23cfc7d6de'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:04.034577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5acc23498df94c3886781c23cfc7d6de'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:04.041708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22547753ad6640b9819a96d848aa31f3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:04.058007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22547753ad6640b9819a96d848aa31f3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:04.063514 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1078/2176] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:50:03 2022] DELETE /identity/v3/users/5acc23498df94c3886781c23cfc7d6de => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:04.070457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22547753ad6640b9819a96d848aa31f3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:04.071213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22547753ad6640b9819a96d848aa31f3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:04.082233 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1099/2177] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:50:03 2022] DELETE /identity/v3/users/22547753ad6640b9819a96d848aa31f3 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:50:04.841940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:04.843604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:04.933465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:04.994952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad097a6-6b71-4e39-a9f0-099087384a25 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:05.130035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:05.134281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:05.134751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:05.135303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] PATH_INFO: `/v3/projects/784d87e5cec947028825114a56719f9f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:05.139336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] RBAC: Authorizing `identity:delete_project(project_id=784d87e5cec947028825114a56719f9f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:05.146156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:05.214906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:05.216904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:05.286833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3b616-63e1-499e-b7ac-6b43a255cf9c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:05.325411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '784d87e5cec947028825114a56719f9f', 'request_id': 'req-045fee0b-0dc5-4712-ad90-bdba168b571d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:05.339018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '784d87e5cec947028825114a56719f9f', 'request_id': 'req-045fee0b-0dc5-4712-ad90-bdba168b571d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:05.355565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '784d87e5cec947028825114a56719f9f', 'request_id': 'req-045fee0b-0dc5-4712-ad90-bdba168b571d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:05.376489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] The token cache is being invalidate because project 784d87e5cec947028825114a56719f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:50:05.376865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:05.377639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-045fee0b-0dc5-4712-ad90-bdba168b571d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:05.382524 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1079/2178] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:50:04 2022] DELETE /identity/v3/projects/784d87e5cec947028825114a56719f9f => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:50:05.384160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3124c-001e-4bc0-a823-77cf17823b69 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:05.386730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3124c-001e-4bc0-a823-77cf17823b69 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:05.389189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3124c-001e-4bc0-a823-77cf17823b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:05.449214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-aab3124c-001e-4bc0-a823-77cf17823b69 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:05.454951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aab3124c-001e-4bc0-a823-77cf17823b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:05.456465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:05.458377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:05.459194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:05.459549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] PATH_INFO: `/v3/projects/0a367b44e2994e5d9803aeca3b402d50` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:05.461301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a367b44e2994e5d9803aeca3b402d50)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:05.467061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:05.474287 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1080/2179] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:05.480682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113538a-3008-41fc-a250-763bdd5a1345 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:05.481028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113538a-3008-41fc-a250-763bdd5a1345 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:05.482141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113538a-3008-41fc-a250-763bdd5a1345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:05.530110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0113538a-3008-41fc-a250-763bdd5a1345 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:05.534867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0113538a-3008-41fc-a250-763bdd5a1345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:05.552912 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1081/2180] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:50:05.557114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113538a-3008-41fc-a250-763bdd5a1345 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:05.567944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113538a-3008-41fc-a250-763bdd5a1345 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:05.586796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a367b44e2994e5d9803aeca3b402d50', 'request_id': 'req-798bb1d0-c67a-423b-8491-0a13d9f1569b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:05.601570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a367b44e2994e5d9803aeca3b402d50', 'request_id': 'req-798bb1d0-c67a-423b-8491-0a13d9f1569b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:05.611250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a367b44e2994e5d9803aeca3b402d50', 'request_id': 'req-798bb1d0-c67a-423b-8491-0a13d9f1569b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:05.639176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] The token cache is being invalidate because project 0a367b44e2994e5d9803aeca3b402d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:50:05.639578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:05.643714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-798bb1d0-c67a-423b-8491-0a13d9f1569b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:05.652458 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1100/2181] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:50:05 2022] DELETE /identity/v3/projects/0a367b44e2994e5d9803aeca3b402d50 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:50:05.654193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d169f3-1e09-4dee-86f0-90d972d76143 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:05.654478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d169f3-1e09-4dee-86f0-90d972d76143 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:05.654910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d169f3-1e09-4dee-86f0-90d972d76143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:05.685994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b6d169f3-1e09-4dee-86f0-90d972d76143 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:05.687341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b6d169f3-1e09-4dee-86f0-90d972d76143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:05.703477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:05.708838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb720d-8d58-4099-ae0e-58dd76039210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:05.709797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb720d-8d58-4099-ae0e-58dd76039210 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:05.710272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb720d-8d58-4099-ae0e-58dd76039210 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:05.710872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb720d-8d58-4099-ae0e-58dd76039210 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:05.711759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb720d-8d58-4099-ae0e-58dd76039210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:50:05.712324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb720d-8d58-4099-ae0e-58dd76039210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:05.717375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb720d-8d58-4099-ae0e-58dd76039210 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:05.741786 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1082/2182] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:50:05.746437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb720d-8d58-4099-ae0e-58dd76039210 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:05.752005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5fbb720d-8d58-4099-ae0e-58dd76039210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:05.791272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:05.811669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb720d-8d58-4099-ae0e-58dd76039210 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:05.837703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5e6e-e584-417f-ac32-4faf4d88bbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:05.838669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5e6e-e584-417f-ac32-4faf4d88bbcb admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:05.838943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5e6e-e584-417f-ac32-4faf4d88bbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:05.839206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5e6e-e584-417f-ac32-4faf4d88bbcb admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:05.840858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb5e6e-e584-417f-ac32-4faf4d88bbcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:05.844248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb5e6e-e584-417f-ac32-4faf4d88bbcb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:05.874384 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1101/2183] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:05.883033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2769b-7e13-4d6b-9798-0508fe4de16e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:05.883373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2769b-7e13-4d6b-9798-0508fe4de16e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:05.883579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2769b-7e13-4d6b-9798-0508fe4de16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:05.903032 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1083/2184] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:50:05 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:05.903964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5e6e-e584-417f-ac32-4faf4d88bbcb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:05.932559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d3f2769b-7e13-4d6b-9798-0508fe4de16e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:05.933841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d3f2769b-7e13-4d6b-9798-0508fe4de16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:06.008821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5e6e-e584-417f-ac32-4faf4d88bbcb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:06.019649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0ba64-94bf-476e-b6bb-57e0679c1220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:06.020787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0ba64-94bf-476e-b6bb-57e0679c1220 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.035399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0ba64-94bf-476e-b6bb-57e0679c1220 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.035988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0ba64-94bf-476e-b6bb-57e0679c1220 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.038500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0ba64-94bf-476e-b6bb-57e0679c1220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:06.054754 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1102/2185] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:06.055746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2769b-7e13-4d6b-9798-0508fe4de16e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:06.070507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2769b-7e13-4d6b-9798-0508fe4de16e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:06.072283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0ba64-94bf-476e-b6bb-57e0679c1220 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:06.078673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c875b-20ee-44f5-a996-6ba8db2fca13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:06.079289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c875b-20ee-44f5-a996-6ba8db2fca13 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.079559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c875b-20ee-44f5-a996-6ba8db2fca13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.079825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c875b-20ee-44f5-a996-6ba8db2fca13 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.080470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c875b-20ee-44f5-a996-6ba8db2fca13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:50:06.080769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c875b-20ee-44f5-a996-6ba8db2fca13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:06.086082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c875b-20ee-44f5-a996-6ba8db2fca13 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:06.116845 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1103/2186] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:06.121649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c875b-20ee-44f5-a996-6ba8db2fca13 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:06.125715 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1084/2187] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:50:05 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:06.141293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0ba64-94bf-476e-b6bb-57e0679c1220 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:06.163114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c875b-20ee-44f5-a996-6ba8db2fca13 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:06.170142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0ba64-94bf-476e-b6bb-57e0679c1220 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:06.178633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd4167-f7ae-4c31-8a3c-99f3acd987c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:06.180994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd4167-f7ae-4c31-8a3c-99f3acd987c8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.181367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd4167-f7ae-4c31-8a3c-99f3acd987c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.181636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd4167-f7ae-4c31-8a3c-99f3acd987c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.182208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101c853-feb3-4d02-b24e-09d493e7b99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:06.182337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd4167-f7ae-4c31-8a3c-99f3acd987c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:06.183089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c853-feb3-4d02-b24e-09d493e7b99c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.183681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c853-feb3-4d02-b24e-09d493e7b99c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.184097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c853-feb3-4d02-b24e-09d493e7b99c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.184895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101c853-feb3-4d02-b24e-09d493e7b99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:06.192287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd4167-f7ae-4c31-8a3c-99f3acd987c8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:06.205722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101c853-feb3-4d02-b24e-09d493e7b99c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:06.250423 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1085/2188] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:06.254406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101c853-feb3-4d02-b24e-09d493e7b99c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:06.283725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101c853-feb3-4d02-b24e-09d493e7b99c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:06.293542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ff073-079a-418f-b966-5905f6c4b757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:06.294189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ff073-079a-418f-b966-5905f6c4b757 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.294529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ff073-079a-418f-b966-5905f6c4b757 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.294904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ff073-079a-418f-b966-5905f6c4b757 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.295596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ff073-079a-418f-b966-5905f6c4b757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:06.298986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ff073-079a-418f-b966-5905f6c4b757 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:06.318796 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1086/2189] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:06.319800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ff073-079a-418f-b966-5905f6c4b757 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:06.340600 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1104/2190] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:50:06 2022] POST /identity/v3/projects => generated 395 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:06.341831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd4167-f7ae-4c31-8a3c-99f3acd987c8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:06.345421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ff073-079a-418f-b966-5905f6c4b757 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:06.473084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd4167-f7ae-4c31-8a3c-99f3acd987c8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:06.482604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01daac-be9b-45b7-a123-400df59c9076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:06.483765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01daac-be9b-45b7-a123-400df59c9076 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.485235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01daac-be9b-45b7-a123-400df59c9076 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.485565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01daac-be9b-45b7-a123-400df59c9076 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.486212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01daac-be9b-45b7-a123-400df59c9076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:06.505110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01daac-be9b-45b7-a123-400df59c9076 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:06.514683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65762d-4c37-4d50-bc23-fb249a4eb792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:06.514683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65762d-4c37-4d50-bc23-fb249a4eb792 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.514683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65762d-4c37-4d50-bc23-fb249a4eb792 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.514683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65762d-4c37-4d50-bc23-fb249a4eb792 admin admin] PATH_INFO: `/v3/projects/358f9c122d6c4f49ad13d0d08e224306/users/21a0c7d1d6e7430cbe92810613087865/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.544711 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1105/2191] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:50:06 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:06.545864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01daac-be9b-45b7-a123-400df59c9076 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:06.558657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01daac-be9b-45b7-a123-400df59c9076 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:06.566314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10909-fd1c-4b0e-a6ed-2645535a5e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:06.567822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10909-fd1c-4b0e-a6ed-2645535a5e5b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.568251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10909-fd1c-4b0e-a6ed-2645535a5e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.568681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10909-fd1c-4b0e-a6ed-2645535a5e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.569529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d10909-fd1c-4b0e-a6ed-2645535a5e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:06.570379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65762d-4c37-4d50-bc23-fb249a4eb792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=358f9c122d6c4f49ad13d0d08e224306, user_id=21a0c7d1d6e7430cbe92810613087865, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:06.571572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d10909-fd1c-4b0e-a6ed-2645535a5e5b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:06.574411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65762d-4c37-4d50-bc23-fb249a4eb792 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:06.610162 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1106/2192] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:50:06.614010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10909-fd1c-4b0e-a6ed-2645535a5e5b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:06.614607 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1087/2193] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:06 2022] PUT /identity/v3/projects/358f9c122d6c4f49ad13d0d08e224306/users/21a0c7d1d6e7430cbe92810613087865/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:06.623442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4ffbd-0064-4593-9a7e-985868687d82 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.623736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4ffbd-0064-4593-9a7e-985868687d82 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.624006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4ffbd-0064-4593-9a7e-985868687d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.626215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:06.643551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10909-fd1c-4b0e-a6ed-2645535a5e5b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:06.710449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-12c4ffbd-0064-4593-9a7e-985868687d82 None None] MFA Rules not processed for user `21a0c7d1d6e7430cbe92810613087865`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:06.711726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-12c4ffbd-0064-4593-9a7e-985868687d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:06.742620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2249d1-f7d7-423d-9363-e0ec67e80bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:06.742983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2249d1-f7d7-423d-9363-e0ec67e80bdb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.743195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2249d1-f7d7-423d-9363-e0ec67e80bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.743780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2249d1-f7d7-423d-9363-e0ec67e80bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.744503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2249d1-f7d7-423d-9363-e0ec67e80bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:06.746171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2249d1-f7d7-423d-9363-e0ec67e80bdb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:06.753199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:06.759461 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1107/2194] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:06.761270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d921c2e-419a-445e-a716-77504323f0bd None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.761399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d921c2e-419a-445e-a716-77504323f0bd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.761579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d921c2e-419a-445e-a716-77504323f0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.818038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3d921c2e-419a-445e-a716-77504323f0bd None None] MFA Rules not processed for user `03bc9235fe1c4fc1b3097a913cbe1052`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:06.819588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3d921c2e-419a-445e-a716-77504323f0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:06.831936 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1088/2195] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:06.833338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4ffbd-0064-4593-9a7e-985868687d82 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:06.877901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4ffbd-0064-4593-9a7e-985868687d82 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:06.896559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd00c0-b8c8-4f91-8a15-92a02630a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:06.897273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd00c0-b8c8-4f91-8a15-92a02630a9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:06.897548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd00c0-b8c8-4f91-8a15-92a02630a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:06.897814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd00c0-b8c8-4f91-8a15-92a02630a9f3 admin admin] PATH_INFO: `/v3/projects/f400a79cb647438f968a895142c5410d/users/180d9b7141ff4792ab7ab19e21a3d7ac/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:06.949873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd00c0-b8c8-4f91-8a15-92a02630a9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f400a79cb647438f968a895142c5410d, user_id=180d9b7141ff4792ab7ab19e21a3d7ac, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:06.953083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd00c0-b8c8-4f91-8a15-92a02630a9f3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:07.018064 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1089/2196] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:06 2022] PUT /identity/v3/projects/f400a79cb647438f968a895142c5410d/users/180d9b7141ff4792ab7ab19e21a3d7ac/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:07.035617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd00c0-b8c8-4f91-8a15-92a02630a9f3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:07.065022 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1108/2197] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:07.068313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5ae05-43b9-443e-910f-06737161ba86 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:07.068766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5ae05-43b9-443e-910f-06737161ba86 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:07.068903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5ae05-43b9-443e-910f-06737161ba86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:07.135341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f8b5ae05-43b9-443e-910f-06737161ba86 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:07.137278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f8b5ae05-43b9-443e-910f-06737161ba86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:07.214818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd00c0-b8c8-4f91-8a15-92a02630a9f3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:07.223652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c294fc2-972f-4e45-a3d1-234b563f4608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:07.224273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c294fc2-972f-4e45-a3d1-234b563f4608 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:07.224535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c294fc2-972f-4e45-a3d1-234b563f4608 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:07.224803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c294fc2-972f-4e45-a3d1-234b563f4608 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:07.225463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c294fc2-972f-4e45-a3d1-234b563f4608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:07.227349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c294fc2-972f-4e45-a3d1-234b563f4608 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:07.240886 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1109/2198] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:07.242107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5ae05-43b9-443e-910f-06737161ba86 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:07.245371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f8b5ae05-43b9-443e-910f-06737161ba86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:07.289538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:07.321851 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1090/2199] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:50:07 2022] POST /identity/v3/projects => generated 375 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:07.322996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c294fc2-972f-4e45-a3d1-234b563f4608 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:07.326141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6c294fc2-972f-4e45-a3d1-234b563f4608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:07.361367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5ae05-43b9-443e-910f-06737161ba86 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:07.412950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c294fc2-972f-4e45-a3d1-234b563f4608 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:07.416930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b7e0a-9a04-45e1-adc3-46ccea445724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:07.417613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b7e0a-9a04-45e1-adc3-46ccea445724 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:07.417878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b7e0a-9a04-45e1-adc3-46ccea445724 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:07.418141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b7e0a-9a04-45e1-adc3-46ccea445724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:07.481021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:07.484599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9378a6-c592-4773-aab0-6491ca154373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:07.487091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9378a6-c592-4773-aab0-6491ca154373 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:07.487529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9378a6-c592-4773-aab0-6491ca154373 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:07.488030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9378a6-c592-4773-aab0-6491ca154373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:07.531342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b7e0a-9a04-45e1-adc3-46ccea445724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:07.532885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b7e0a-9a04-45e1-adc3-46ccea445724 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:07.591963 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1091/2200] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:07.595991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b7e0a-9a04-45e1-adc3-46ccea445724 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:07.606009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9378a6-c592-4773-aab0-6491ca154373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:07.608166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9378a6-c592-4773-aab0-6491ca154373 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:07.617430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b7e0a-9a04-45e1-adc3-46ccea445724 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:07.631753 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1110/2201] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:07.641423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04ec72-33e9-41ad-a058-26e0dcc488d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:07.645101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04ec72-33e9-41ad-a058-26e0dcc488d2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:07.645513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04ec72-33e9-41ad-a058-26e0dcc488d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:07.646895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04ec72-33e9-41ad-a058-26e0dcc488d2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:07.647652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04ec72-33e9-41ad-a058-26e0dcc488d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:07.652705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04ec72-33e9-41ad-a058-26e0dcc488d2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:07.717245 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1092/2202] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:50:07 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:07.719369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9378a6-c592-4773-aab0-6491ca154373 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:07.744042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9378a6-c592-4773-aab0-6491ca154373 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:07.759325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7c117-ccaa-40d2-af47-e783312bb0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:07.760052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7c117-ccaa-40d2-af47-e783312bb0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:07.760395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7c117-ccaa-40d2-af47-e783312bb0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:07.760730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7c117-ccaa-40d2-af47-e783312bb0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:07.762711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7c117-ccaa-40d2-af47-e783312bb0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:07.766180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7c117-ccaa-40d2-af47-e783312bb0e6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:07.799585 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1111/2203] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:07.800806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04ec72-33e9-41ad-a058-26e0dcc488d2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:07.812582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04ec72-33e9-41ad-a058-26e0dcc488d2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:07.822670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002541d-af01-4167-9460-8dcec0e901c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:07.823481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002541d-af01-4167-9460-8dcec0e901c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:07.825136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002541d-af01-4167-9460-8dcec0e901c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:07.825713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002541d-af01-4167-9460-8dcec0e901c4 admin admin] PATH_INFO: `/v3/projects/5a84a87d983f400faf2761a5a5b0fef5/users/305f573e4a7b483eb929fc47f6b59d93/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:07.853554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002541d-af01-4167-9460-8dcec0e901c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a84a87d983f400faf2761a5a5b0fef5, user_id=305f573e4a7b483eb929fc47f6b59d93, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:07.857761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002541d-af01-4167-9460-8dcec0e901c4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:07.897927 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1093/2204] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:07 2022] PUT /identity/v3/projects/5a84a87d983f400faf2761a5a5b0fef5/users/305f573e4a7b483eb929fc47f6b59d93/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:07.900820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7c117-ccaa-40d2-af47-e783312bb0e6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:07.913783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7c117-ccaa-40d2-af47-e783312bb0e6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:08.008194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:08.028601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6261b30-b6d9-47a5-bdc4-726ef9121dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:08.029249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6261b30-b6d9-47a5-bdc4-726ef9121dbf admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:08.029524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6261b30-b6d9-47a5-bdc4-726ef9121dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:08.029789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6261b30-b6d9-47a5-bdc4-726ef9121dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:08.030501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6261b30-b6d9-47a5-bdc4-726ef9121dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:08.032444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6261b30-b6d9-47a5-bdc4-726ef9121dbf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:08.045319 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1112/2205] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:50:08.049796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002541d-af01-4167-9460-8dcec0e901c4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:08.064854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002541d-af01-4167-9460-8dcec0e901c4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:08.070460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192832b7-a963-4ce9-9183-c75f415e9560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:08.071268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192832b7-a963-4ce9-9183-c75f415e9560 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:08.071616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192832b7-a963-4ce9-9183-c75f415e9560 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:08.071957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192832b7-a963-4ce9-9183-c75f415e9560 admin admin] PATH_INFO: `/v3/projects/5a84a87d983f400faf2761a5a5b0fef5/users/305f573e4a7b483eb929fc47f6b59d93/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:08.074060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192832b7-a963-4ce9-9183-c75f415e9560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a84a87d983f400faf2761a5a5b0fef5, user_id=305f573e4a7b483eb929fc47f6b59d93, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:08.076946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192832b7-a963-4ce9-9183-c75f415e9560 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:08.091426 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1094/2206] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:08 2022] PUT /identity/v3/projects/5a84a87d983f400faf2761a5a5b0fef5/users/305f573e4a7b483eb929fc47f6b59d93/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:50:08.104937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6534b-08ec-4f11-b6d4-7e7f582d20d8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:08.105532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6534b-08ec-4f11-b6d4-7e7f582d20d8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:08.105952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6534b-08ec-4f11-b6d4-7e7f582d20d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:08.166440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fdf6534b-08ec-4f11-b6d4-7e7f582d20d8 None None] MFA Rules not processed for user `305f573e4a7b483eb929fc47f6b59d93`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:08.167666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fdf6534b-08ec-4f11-b6d4-7e7f582d20d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:08.341384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192832b7-a963-4ce9-9183-c75f415e9560 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:08.342583 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1113/2207] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:08.356553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192832b7-a963-4ce9-9183-c75f415e9560 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:08.374452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce060c-7bf0-419e-8d5d-01e2abf0dbf8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:08.375105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce060c-7bf0-419e-8d5d-01e2abf0dbf8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:08.375878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce060c-7bf0-419e-8d5d-01e2abf0dbf8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:08.375878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce060c-7bf0-419e-8d5d-01e2abf0dbf8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:08.376473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce060c-7bf0-419e-8d5d-01e2abf0dbf8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:50:08.376788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce060c-7bf0-419e-8d5d-01e2abf0dbf8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:08.378538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce060c-7bf0-419e-8d5d-01e2abf0dbf8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:08.402729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6534b-08ec-4f11-b6d4-7e7f582d20d8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:08.409097 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1095/2208] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:08.415293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6534b-08ec-4f11-b6d4-7e7f582d20d8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:08.425755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf88d9-b273-40eb-b4c1-3dff114a2d42 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:08.426787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf88d9-b273-40eb-b4c1-3dff114a2d42 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:08.427301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf88d9-b273-40eb-b4c1-3dff114a2d42 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:08.427905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf88d9-b273-40eb-b4c1-3dff114a2d42 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:08.428584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf88d9-b273-40eb-b4c1-3dff114a2d42 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:08.430539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf88d9-b273-40eb-b4c1-3dff114a2d42 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:08.490263 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1114/2209] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:50:08 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:50:08.492303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce060c-7bf0-419e-8d5d-01e2abf0dbf8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:08.505497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce060c-7bf0-419e-8d5d-01e2abf0dbf8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:08.642404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92dc6a-7a1e-4f18-a66e-e89626fbe7b7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:08.643727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92dc6a-7a1e-4f18-a66e-e89626fbe7b7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:08.643727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92dc6a-7a1e-4f18-a66e-e89626fbe7b7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:08.644023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92dc6a-7a1e-4f18-a66e-e89626fbe7b7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:08.644905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92dc6a-7a1e-4f18-a66e-e89626fbe7b7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:50:08.645316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92dc6a-7a1e-4f18-a66e-e89626fbe7b7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:08.647492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92dc6a-7a1e-4f18-a66e-e89626fbe7b7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:08.678973 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1096/2210] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:08.776077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf88d9-b273-40eb-b4c1-3dff114a2d42 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:08.777344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79457a56-2a99-4221-84ce-832ab69b3fac None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:08.777643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79457a56-2a99-4221-84ce-832ab69b3fac None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:08.777894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79457a56-2a99-4221-84ce-832ab69b3fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:08.778461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-48cf88d9-b273-40eb-b4c1-3dff114a2d42 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:08.816946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-79457a56-2a99-4221-84ce-832ab69b3fac None None] MFA Rules not processed for user `03bc9235fe1c4fc1b3097a913cbe1052`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:08.818493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-79457a56-2a99-4221-84ce-832ab69b3fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:08.912568 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1097/2211] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:50:08.983670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf88d9-b273-40eb-b4c1-3dff114a2d42 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:08.992471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce61ce-2296-46d1-a7fc-d7dcbc64b1d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:08.993419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce61ce-2296-46d1-a7fc-d7dcbc64b1d1 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:08.993611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce61ce-2296-46d1-a7fc-d7dcbc64b1d1 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:08.993883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce61ce-2296-46d1-a7fc-d7dcbc64b1d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:09.005489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce61ce-2296-46d1-a7fc-d7dcbc64b1d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:09.008200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce61ce-2296-46d1-a7fc-d7dcbc64b1d1 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:09.023348 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1115/2212] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:09.046754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61626f9a-93f7-4161-abe7-c76f95e77fb0 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:09.047272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61626f9a-93f7-4161-abe7-c76f95e77fb0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:09.048151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61626f9a-93f7-4161-abe7-c76f95e77fb0 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:09.050103 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1098/2213] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:09.055405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce61ce-2296-46d1-a7fc-d7dcbc64b1d1 service nova] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:09.073504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce61ce-2296-46d1-a7fc-d7dcbc64b1d1 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:09.088891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce61ce-2296-46d1-a7fc-d7dcbc64b1d1 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:09.096229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931eb326-9f22-403d-8f0f-724ce30e429c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:09.098401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eb326-9f22-403d-8f0f-724ce30e429c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:09.098921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eb326-9f22-403d-8f0f-724ce30e429c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:09.099182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eb326-9f22-403d-8f0f-724ce30e429c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/5f70e93455584211995b4a24d8cd71a8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:09.100826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931eb326-9f22-403d-8f0f-724ce30e429c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=5f70e93455584211995b4a24d8cd71a8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:09.102937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931eb326-9f22-403d-8f0f-724ce30e429c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:09.105021 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1116/2214] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:09 2022] GET /identity/v3/projects/5f70e93455584211995b4a24d8cd71a8 => generated 371 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:50:09.459928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2264f-b899-4c25-bcee-581fd47e2fc6 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:09.460253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2264f-b899-4c25-bcee-581fd47e2fc6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:09.460472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2264f-b899-4c25-bcee-581fd47e2fc6 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:09.461873 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1099/2215] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:09.479230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931eb326-9f22-403d-8f0f-724ce30e429c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:09.504801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931eb326-9f22-403d-8f0f-724ce30e429c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:09.520237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931eb326-9f22-403d-8f0f-724ce30e429c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:09.536577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752131f-0c58-4854-9ef8-d272fcbe88a4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:09.537260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752131f-0c58-4854-9ef8-d272fcbe88a4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:09.537540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752131f-0c58-4854-9ef8-d272fcbe88a4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:09.537811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752131f-0c58-4854-9ef8-d272fcbe88a4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/5f70e93455584211995b4a24d8cd71a8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:09.539462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752131f-0c58-4854-9ef8-d272fcbe88a4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=5f70e93455584211995b4a24d8cd71a8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:09.542609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752131f-0c58-4854-9ef8-d272fcbe88a4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:09.546998 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1117/2216] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:09 2022] GET /identity/v3/projects/5f70e93455584211995b4a24d8cd71a8 => generated 371 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:10.674483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efd7f5-0c20-4c45-9c50-feb76a6b1690 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:10.675021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efd7f5-0c20-4c45-9c50-feb76a6b1690 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:10.675021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efd7f5-0c20-4c45-9c50-feb76a6b1690 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:10.676323 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1100/2217] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:10.685342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752131f-0c58-4854-9ef8-d272fcbe88a4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:10.743575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752131f-0c58-4854-9ef8-d272fcbe88a4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:10.776748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752131f-0c58-4854-9ef8-d272fcbe88a4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:10.786341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8a9ac-9417-48fc-b5e4-50d84bf19fb2 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:10.787190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8a9ac-9417-48fc-b5e4-50d84bf19fb2 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:10.787652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8a9ac-9417-48fc-b5e4-50d84bf19fb2 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:10.788785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8a9ac-9417-48fc-b5e4-50d84bf19fb2 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/5f70e93455584211995b4a24d8cd71a8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:10.790219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8a9ac-9417-48fc-b5e4-50d84bf19fb2 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=5f70e93455584211995b4a24d8cd71a8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:10.792652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8a9ac-9417-48fc-b5e4-50d84bf19fb2 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:10.794798 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1118/2218] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:10 2022] GET /identity/v3/projects/5f70e93455584211995b4a24d8cd71a8 => generated 371 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:10.846633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efd7f5-0c20-4c45-9c50-feb76a6b1690 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:10.858162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:10.867027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efd7f5-0c20-4c45-9c50-feb76a6b1690 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:10.876515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:10.877700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:10.878134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:10.878600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/5f70e93455584211995b4a24d8cd71a8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:10.880018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5f70e93455584211995b4a24d8cd71a8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:10.882209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:11.011137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f70e93455584211995b4a24d8cd71a8', 'request_id': 'req-18760909-c7e4-4d46-8d89-91e1d892f745'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:11.027707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f70e93455584211995b4a24d8cd71a8', 'request_id': 'req-18760909-c7e4-4d46-8d89-91e1d892f745'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:11.056228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f70e93455584211995b4a24d8cd71a8', 'request_id': 'req-18760909-c7e4-4d46-8d89-91e1d892f745'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:11.078840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] The token cache is being invalidate because project 5f70e93455584211995b4a24d8cd71a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:50:11.079693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:11.080161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:11.082070 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1101/2219] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:50:10 2022] DELETE /identity/v3/projects/5f70e93455584211995b4a24d8cd71a8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:11.101384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04b522-c442-4c71-b19b-647054bcc06a None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:11.101660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04b522-c442-4c71-b19b-647054bcc06a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:11.102227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04b522-c442-4c71-b19b-647054bcc06a None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:11.103653 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1119/2220] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:11.112945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:11.115770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:11.220037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:11.221699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:11.310090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18760909-c7e4-4d46-8d89-91e1d892f745 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:11.315773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61fb5d-f811-42f3-b8a9-e7587a934a28 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:11.316505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61fb5d-f811-42f3-b8a9-e7587a934a28 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:11.316983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61fb5d-f811-42f3-b8a9-e7587a934a28 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:11.317360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61fb5d-f811-42f3-b8a9-e7587a934a28 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/358f9c122d6c4f49ad13d0d08e224306` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:11.318612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61fb5d-f811-42f3-b8a9-e7587a934a28 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=358f9c122d6c4f49ad13d0d08e224306)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:11.320963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61fb5d-f811-42f3-b8a9-e7587a934a28 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:11.322715 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1102/2221] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:11 2022] GET /identity/v3/projects/358f9c122d6c4f49ad13d0d08e224306 => generated 373 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:11.341888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04b522-c442-4c71-b19b-647054bcc06a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:11.355894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04b522-c442-4c71-b19b-647054bcc06a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:11.361694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d1268-21f9-4b2c-a5c2-2386d65c8bfa tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:11.362314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d1268-21f9-4b2c-a5c2-2386d65c8bfa tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:11.362596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d1268-21f9-4b2c-a5c2-2386d65c8bfa tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:11.362869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d1268-21f9-4b2c-a5c2-2386d65c8bfa tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:11.363521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d1268-21f9-4b2c-a5c2-2386d65c8bfa tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:50:11.363826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d1268-21f9-4b2c-a5c2-2386d65c8bfa tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:11.365515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d1268-21f9-4b2c-a5c2-2386d65c8bfa tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:11.376772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:11.392598 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1120/2222] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:11.392885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61fb5d-f811-42f3-b8a9-e7587a934a28 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:11.408928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61fb5d-f811-42f3-b8a9-e7587a934a28 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:11.413757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9fa31-d0db-4d7d-93fb-e9c5b3ebad17 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:11.414366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9fa31-d0db-4d7d-93fb-e9c5b3ebad17 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:11.414637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9fa31-d0db-4d7d-93fb-e9c5b3ebad17 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:11.415034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9fa31-d0db-4d7d-93fb-e9c5b3ebad17 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:11.415675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9fa31-d0db-4d7d-93fb-e9c5b3ebad17 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:11.417796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9fa31-d0db-4d7d-93fb-e9c5b3ebad17 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:11.463671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d1268-21f9-4b2c-a5c2-2386d65c8bfa tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:11.492364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d1268-21f9-4b2c-a5c2-2386d65c8bfa tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:11.492772 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1103/2223] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:50:11 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:11.684720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:11.686080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:11.687489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:11.687489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:11.698119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:50:11.698566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:11.700313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:11.722530 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1121/2224] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:12.228862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ff24d-3666-427a-948d-29eebfd01c8f None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:12.229428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ff24d-3666-427a-948d-29eebfd01c8f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:12.229428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ff24d-3666-427a-948d-29eebfd01c8f None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:12.230692 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1104/2225] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:12.241840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:12.244560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:12.434881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:12.460971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2449e-cb93-418f-8d59-648bd5f36fb9 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:12.470673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a36918-e4ab-4797-8994-41ad68a25878 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:12.471599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a36918-e4ab-4797-8994-41ad68a25878 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:12.471880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a36918-e4ab-4797-8994-41ad68a25878 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:12.472150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a36918-e4ab-4797-8994-41ad68a25878 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/7c6248db5a234d1f92f68484f827a9f9` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:12.473412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a36918-e4ab-4797-8994-41ad68a25878 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=7c6248db5a234d1f92f68484f827a9f9)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:12.475654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a36918-e4ab-4797-8994-41ad68a25878 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:12.477803 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1122/2226] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:12 2022] GET /identity/v3/projects/7c6248db5a234d1f92f68484f827a9f9 => generated 371 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:12.731306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53dd76-1384-43a0-aeb9-fcf20440913f None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:12.731841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53dd76-1384-43a0-aeb9-fcf20440913f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:12.732217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53dd76-1384-43a0-aeb9-fcf20440913f None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:12.734313 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1105/2227] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:12.739752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a36918-e4ab-4797-8994-41ad68a25878 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:12.761988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a36918-e4ab-4797-8994-41ad68a25878 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:12.805626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a36918-e4ab-4797-8994-41ad68a25878 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:12.824278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc61b25-9b4c-4380-9ea1-154511423add tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:12.825630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc61b25-9b4c-4380-9ea1-154511423add tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:12.826364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc61b25-9b4c-4380-9ea1-154511423add tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:12.827053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc61b25-9b4c-4380-9ea1-154511423add tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/7c6248db5a234d1f92f68484f827a9f9` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:12.832322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc61b25-9b4c-4380-9ea1-154511423add tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=7c6248db5a234d1f92f68484f827a9f9)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:12.836023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc61b25-9b4c-4380-9ea1-154511423add tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:12.854467 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1123/2228] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:12 2022] GET /identity/v3/projects/7c6248db5a234d1f92f68484f827a9f9 => generated 371 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:12.949837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53dd76-1384-43a0-aeb9-fcf20440913f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:12.961575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:12.975988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53dd76-1384-43a0-aeb9-fcf20440913f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:12.982456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a809a2-fa1f-420d-b7da-02c43767e290 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:12.983776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a809a2-fa1f-420d-b7da-02c43767e290 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:12.985221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a809a2-fa1f-420d-b7da-02c43767e290 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:12.985771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a809a2-fa1f-420d-b7da-02c43767e290 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:12.990014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a809a2-fa1f-420d-b7da-02c43767e290 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:50:12.990605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a809a2-fa1f-420d-b7da-02c43767e290 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:12.992481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a809a2-fa1f-420d-b7da-02c43767e290 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:13.028819 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1106/2229] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:13.029859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc61b25-9b4c-4380-9ea1-154511423add tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:13.048848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc61b25-9b4c-4380-9ea1-154511423add tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:13.068239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734560a-e860-41e4-a0bc-b68550821b61 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:13.069241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734560a-e860-41e4-a0bc-b68550821b61 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:13.069767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734560a-e860-41e4-a0bc-b68550821b61 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:13.071834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734560a-e860-41e4-a0bc-b68550821b61 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:13.072482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734560a-e860-41e4-a0bc-b68550821b61 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:13.078831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734560a-e860-41e4-a0bc-b68550821b61 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:13.158812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a809a2-fa1f-420d-b7da-02c43767e290 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:13.171762 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1124/2230] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:50:13 2022] POST /identity/v3/users => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:13.178237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a809a2-fa1f-420d-b7da-02c43767e290 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:13.190344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdff06-447d-4186-8ba2-7607f5f989f0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:13.191436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdff06-447d-4186-8ba2-7607f5f989f0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:13.191926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdff06-447d-4186-8ba2-7607f5f989f0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:13.192414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdff06-447d-4186-8ba2-7607f5f989f0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:13.193635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdff06-447d-4186-8ba2-7607f5f989f0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:50:13.194169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdff06-447d-4186-8ba2-7607f5f989f0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:13.197893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdff06-447d-4186-8ba2-7607f5f989f0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:13.227682 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1107/2231] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:13.566204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ee837-da9b-4832-8aaf-92985e2a23f7 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:13.566661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ee837-da9b-4832-8aaf-92985e2a23f7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:13.567174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ee837-da9b-4832-8aaf-92985e2a23f7 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:13.569345 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1125/2232] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:13.576363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdff06-447d-4186-8ba2-7607f5f989f0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:13.601959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdff06-447d-4186-8ba2-7607f5f989f0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:13.622413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdff06-447d-4186-8ba2-7607f5f989f0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:13.629426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba00f37-27ad-4eb8-b802-8f7360e0d77d tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:13.630427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba00f37-27ad-4eb8-b802-8f7360e0d77d tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:13.631181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba00f37-27ad-4eb8-b802-8f7360e0d77d tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:13.632070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba00f37-27ad-4eb8-b802-8f7360e0d77d tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/7c6248db5a234d1f92f68484f827a9f9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:13.633571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba00f37-27ad-4eb8-b802-8f7360e0d77d tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=7c6248db5a234d1f92f68484f827a9f9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:13.636060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba00f37-27ad-4eb8-b802-8f7360e0d77d tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:13.639474 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1108/2233] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:13 2022] GET /identity/v3/projects/7c6248db5a234d1f92f68484f827a9f9 => generated 371 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:13.878583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa36cdc-2718-45ba-a255-fc4b481b19e5 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:13.879596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa36cdc-2718-45ba-a255-fc4b481b19e5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:13.879864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa36cdc-2718-45ba-a255-fc4b481b19e5 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:13.881541 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1126/2234] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:13.887085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba00f37-27ad-4eb8-b802-8f7360e0d77d tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:13.902591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba00f37-27ad-4eb8-b802-8f7360e0d77d tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:13.913729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba00f37-27ad-4eb8-b802-8f7360e0d77d tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:13.919597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaa69d-cc59-402f-b72a-82aba07e8ee5 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:13.920437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaa69d-cc59-402f-b72a-82aba07e8ee5 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:13.921719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaa69d-cc59-402f-b72a-82aba07e8ee5 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:13.922197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaa69d-cc59-402f-b72a-82aba07e8ee5 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/7c6248db5a234d1f92f68484f827a9f9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:13.923815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaa69d-cc59-402f-b72a-82aba07e8ee5 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=7c6248db5a234d1f92f68484f827a9f9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:13.926979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaa69d-cc59-402f-b72a-82aba07e8ee5 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:13.941439 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1109/2235] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:13 2022] GET /identity/v3/projects/7c6248db5a234d1f92f68484f827a9f9 => generated 371 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:13.971517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa36cdc-2718-45ba-a255-fc4b481b19e5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:13.996507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa36cdc-2718-45ba-a255-fc4b481b19e5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:14.012833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:14.013817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:14.014094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:14.014485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/users/0fc384ae0e2c446e93677acd6c342657` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:14.046993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:delete_user(user_id=0fc384ae0e2c446e93677acd6c342657)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:14.050080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:14.121911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:14.135091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc384ae0e2c446e93677acd6c342657'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:14.161339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc384ae0e2c446e93677acd6c342657'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:14.176952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc384ae0e2c446e93677acd6c342657'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:14.190722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc384ae0e2c446e93677acd6c342657'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:14.192725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea2a261b-cea9-4810-bf45-cb25206b6b25 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc384ae0e2c446e93677acd6c342657'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:14.208415 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1127/2236] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:50:13 2022] DELETE /identity/v3/users/0fc384ae0e2c446e93677acd6c342657 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:14.213138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaa69d-cc59-402f-b72a-82aba07e8ee5 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:14.215809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8ebaa69d-cc59-402f-b72a-82aba07e8ee5 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:14.231544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaa69d-cc59-402f-b72a-82aba07e8ee5 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:14.330402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:14.332414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:14.332928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:14.333425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/7c6248db5a234d1f92f68484f827a9f9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:14.334984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7c6248db5a234d1f92f68484f827a9f9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:14.338524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:14.416433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c6248db5a234d1f92f68484f827a9f9', 'request_id': 'req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:14.430270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c6248db5a234d1f92f68484f827a9f9', 'request_id': 'req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:14.446591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c6248db5a234d1f92f68484f827a9f9', 'request_id': 'req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:14.487403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] The token cache is being invalidate because project 7c6248db5a234d1f92f68484f827a9f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:50:14.487712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:14.488443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:14.492724 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1110/2237] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:50:14 2022] DELETE /identity/v3/projects/7c6248db5a234d1f92f68484f827a9f9 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:14.510842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6db1c-3b15-4616-bb65-fdbfae35222f None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:14.511123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6db1c-3b15-4616-bb65-fdbfae35222f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:14.511394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6db1c-3b15-4616-bb65-fdbfae35222f None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:14.512919 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1128/2238] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:14.528403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:14.529905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:14.630469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:14.682997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:14.684936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:14.861909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390072f-7b9b-4f11-9f58-bbc5d08f32bb tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:14.875483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de8c0d-ffa9-44d8-adae-738ff3d6ae8c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:14.876056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de8c0d-ffa9-44d8-adae-738ff3d6ae8c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:14.876407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de8c0d-ffa9-44d8-adae-738ff3d6ae8c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:14.876579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de8c0d-ffa9-44d8-adae-738ff3d6ae8c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/358f9c122d6c4f49ad13d0d08e224306` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:14.877754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de8c0d-ffa9-44d8-adae-738ff3d6ae8c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=358f9c122d6c4f49ad13d0d08e224306)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:14.880437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de8c0d-ffa9-44d8-adae-738ff3d6ae8c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:14.886546 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1111/2239] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:14 2022] GET /identity/v3/projects/358f9c122d6c4f49ad13d0d08e224306 => generated 373 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:15.013925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d349ddb-600f-4063-8237-9c200450f340 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:15.015033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d349ddb-600f-4063-8237-9c200450f340 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:15.016248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d349ddb-600f-4063-8237-9c200450f340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:15.054272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7d349ddb-600f-4063-8237-9c200450f340 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:15.060885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7d349ddb-600f-4063-8237-9c200450f340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:15.186545 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1129/2240] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:50:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:15.284369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de8c0d-ffa9-44d8-adae-738ff3d6ae8c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:15.292790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-69de8c0d-ffa9-44d8-adae-738ff3d6ae8c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:15.417843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de8c0d-ffa9-44d8-adae-738ff3d6ae8c tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:15.428429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b0cbd-8a62-49fb-af73-388be4935463 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:15.429366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b0cbd-8a62-49fb-af73-388be4935463 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:15.429712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b0cbd-8a62-49fb-af73-388be4935463 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:15.430047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b0cbd-8a62-49fb-af73-388be4935463 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:15.440766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b0cbd-8a62-49fb-af73-388be4935463 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:15.443242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b0cbd-8a62-49fb-af73-388be4935463 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:15.453846 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1112/2241] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:50:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:50:15.613052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827250-471c-4c32-a026-a40c15b60e89 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:15.613375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827250-471c-4c32-a026-a40c15b60e89 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:15.613587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827250-471c-4c32-a026-a40c15b60e89 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:15.615085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1130/2242] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:15.626722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b0cbd-8a62-49fb-af73-388be4935463 service swift] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:15.663695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b0cbd-8a62-49fb-af73-388be4935463 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:15.718271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b0cbd-8a62-49fb-af73-388be4935463 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:15.751222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1c3d2-c33b-4c10-90f9-da1f45acdeb4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:15.751869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1c3d2-c33b-4c10-90f9-da1f45acdeb4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:15.752170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1c3d2-c33b-4c10-90f9-da1f45acdeb4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:15.752440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1c3d2-c33b-4c10-90f9-da1f45acdeb4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/358f9c122d6c4f49ad13d0d08e224306` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:15.753779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1c3d2-c33b-4c10-90f9-da1f45acdeb4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=358f9c122d6c4f49ad13d0d08e224306)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:15.756120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1c3d2-c33b-4c10-90f9-da1f45acdeb4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:15.758068 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1113/2243] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:15 2022] GET /identity/v3/projects/358f9c122d6c4f49ad13d0d08e224306 => generated 373 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:16.328023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59149e11-e33e-4cbe-bfbd-d2f455265283 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:16.329018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59149e11-e33e-4cbe-bfbd-d2f455265283 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:16.329514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59149e11-e33e-4cbe-bfbd-d2f455265283 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:16.331123 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1131/2244] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:50:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:16.338638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1c3d2-c33b-4c10-90f9-da1f45acdeb4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:50:16.355051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1c3d2-c33b-4c10-90f9-da1f45acdeb4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:16.365612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1c3d2-c33b-4c10-90f9-da1f45acdeb4 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:16.378278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a52ad-731f-4b29-b207-e4e9568292b0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '305f573e4a7b483eb929fc47f6b59d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a84a87d983f400faf2761a5a5b0fef5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:16.378975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a52ad-731f-4b29-b207-e4e9568292b0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:16.379257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a52ad-731f-4b29-b207-e4e9568292b0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:16.379526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a52ad-731f-4b29-b207-e4e9568292b0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] PATH_INFO: `/v3/projects/358f9c122d6c4f49ad13d0d08e224306` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:16.381059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a52ad-731f-4b29-b207-e4e9568292b0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorizing `identity:get_project(project_id=358f9c122d6c4f49ad13d0d08e224306)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:16.383105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a52ad-731f-4b29-b207-e4e9568292b0 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:16.385075 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1114/2245] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:50:16 2022] GET /identity/v3/projects/358f9c122d6c4f49ad13d0d08e224306 => generated 373 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:17.099746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59149e11-e33e-4cbe-bfbd-d2f455265283 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:17.101464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-59149e11-e33e-4cbe-bfbd-d2f455265283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:17.368578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59149e11-e33e-4cbe-bfbd-d2f455265283 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:17.380393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:17.383659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:17.384131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:17.384812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] PATH_INFO: `/v3/users/21a0c7d1d6e7430cbe92810613087865` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:17.387203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a0c7d1d6e7430cbe92810613087865)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:17.391603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:17.524964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a0c7d1d6e7430cbe92810613087865'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:17.558996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:17.569150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a0c7d1d6e7430cbe92810613087865'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:17.581079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a0c7d1d6e7430cbe92810613087865'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:17.594504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a0c7d1d6e7430cbe92810613087865'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:17.595690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a0c7d1d6e7430cbe92810613087865'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:17.624657 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1132/2246] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:50:17 2022] DELETE /identity/v3/users/21a0c7d1d6e7430cbe92810613087865 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:17.626706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1564d0-6480-4258-9ce3-d83c7472e601 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:17.627781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1564d0-6480-4258-9ce3-d83c7472e601 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:17.628410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1564d0-6480-4258-9ce3-d83c7472e601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:17.671559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5c1564d0-6480-4258-9ce3-d83c7472e601 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:17.672790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5c1564d0-6480-4258-9ce3-d83c7472e601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:17.894696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:17.923462 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1115/2247] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:50:17.958276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:17.960792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:18.069775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb8a92-97b8-45ea-8b8d-9be9c456039a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:18.075328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205047e4-e8b2-4ac5-b7e3-fc54f3a66126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:18.075950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205047e4-e8b2-4ac5-b7e3-fc54f3a66126 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:18.076219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205047e4-e8b2-4ac5-b7e3-fc54f3a66126 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:18.076483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205047e4-e8b2-4ac5-b7e3-fc54f3a66126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:18.089952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205047e4-e8b2-4ac5-b7e3-fc54f3a66126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:18.091330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205047e4-e8b2-4ac5-b7e3-fc54f3a66126 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:18.105297 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1133/2248] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:18.316066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078c5e9-c0ad-4f48-bf1c-aed2a21a742a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:18.316373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078c5e9-c0ad-4f48-bf1c-aed2a21a742a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:18.316756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078c5e9-c0ad-4f48-bf1c-aed2a21a742a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:18.376225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2078c5e9-c0ad-4f48-bf1c-aed2a21a742a None None] MFA Rules not processed for user `180d9b7141ff4792ab7ab19e21a3d7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:18.377605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2078c5e9-c0ad-4f48-bf1c-aed2a21a742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:18.505385 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1116/2249] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:18 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:18.506608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205047e4-e8b2-4ac5-b7e3-fc54f3a66126 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:18.508677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-205047e4-e8b2-4ac5-b7e3-fc54f3a66126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:18.535844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205047e4-e8b2-4ac5-b7e3-fc54f3a66126 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:18.566876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dc6df-d7bf-4059-9c56-3fcba0aff847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:18.567799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dc6df-d7bf-4059-9c56-3fcba0aff847 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:18.567799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dc6df-d7bf-4059-9c56-3fcba0aff847 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:18.568127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dc6df-d7bf-4059-9c56-3fcba0aff847 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:18.568673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759dc6df-d7bf-4059-9c56-3fcba0aff847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:18.570635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759dc6df-d7bf-4059-9c56-3fcba0aff847 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:18.638497 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1134/2250] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:50:18 2022] POST /identity/v3/projects => generated 395 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:18.644098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078c5e9-c0ad-4f48-bf1c-aed2a21a742a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:18.662068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078c5e9-c0ad-4f48-bf1c-aed2a21a742a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:18.795933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf85016-6cd1-4a2b-8c70-87143a7604fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:18.801830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf85016-6cd1-4a2b-8c70-87143a7604fa admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:18.802485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf85016-6cd1-4a2b-8c70-87143a7604fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:18.803133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf85016-6cd1-4a2b-8c70-87143a7604fa admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:18.804014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf85016-6cd1-4a2b-8c70-87143a7604fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:18.808400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf85016-6cd1-4a2b-8c70-87143a7604fa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:18.863483 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1117/2251] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:50:18 2022] POST /identity/v3/users => generated 304 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:18.864511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dc6df-d7bf-4059-9c56-3fcba0aff847 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:18.899160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dc6df-d7bf-4059-9c56-3fcba0aff847 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:18.906573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b226a3-eaab-4373-b776-427c886a153b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:18.907192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b226a3-eaab-4373-b776-427c886a153b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:18.907889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b226a3-eaab-4373-b776-427c886a153b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:18.908892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b226a3-eaab-4373-b776-427c886a153b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:18.909600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b226a3-eaab-4373-b776-427c886a153b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:18.911727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b226a3-eaab-4373-b776-427c886a153b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:18.926507 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1135/2252] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:18.930491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf85016-6cd1-4a2b-8c70-87143a7604fa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:18.933311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b226a3-eaab-4373-b776-427c886a153b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:18.937394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-88b226a3-eaab-4373-b776-427c886a153b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:18.954104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf85016-6cd1-4a2b-8c70-87143a7604fa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:18.961895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b226a3-eaab-4373-b776-427c886a153b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:18.968191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca976cec-18a0-4318-a3a5-06c21d58f8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:18.971010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca976cec-18a0-4318-a3a5-06c21d58f8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:18.972257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca976cec-18a0-4318-a3a5-06c21d58f8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:18.972743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca976cec-18a0-4318-a3a5-06c21d58f8f1 admin admin] PATH_INFO: `/v3/projects/4a5d6aa09fb0449ea9078a380c02ac87/users/c2394c816d8c47b887c085027a81b7a9/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:18.985823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:18.992406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:18.992406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:18.992406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] PATH_INFO: `/v3/projects/358f9c122d6c4f49ad13d0d08e224306` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:18.994688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=358f9c122d6c4f49ad13d0d08e224306)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:19.000425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:19.021453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca976cec-18a0-4318-a3a5-06c21d58f8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5d6aa09fb0449ea9078a380c02ac87, user_id=c2394c816d8c47b887c085027a81b7a9, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:19.024909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca976cec-18a0-4318-a3a5-06c21d58f8f1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:19.048493 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1118/2253] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:18 2022] PUT /identity/v3/projects/4a5d6aa09fb0449ea9078a380c02ac87/users/c2394c816d8c47b887c085027a81b7a9/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:50:19.052323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca976cec-18a0-4318-a3a5-06c21d58f8f1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:19.114742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '358f9c122d6c4f49ad13d0d08e224306', 'request_id': 'req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:19.134511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '358f9c122d6c4f49ad13d0d08e224306', 'request_id': 'req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:19.142704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:19.149909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '358f9c122d6c4f49ad13d0d08e224306', 'request_id': 'req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:19.161427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca976cec-18a0-4318-a3a5-06c21d58f8f1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:19.171658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] The token cache is being invalidate because project 358f9c122d6c4f49ad13d0d08e224306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:50:19.171886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:19.172705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:19.177138 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1136/2254] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:50:18 2022] DELETE /identity/v3/projects/358f9c122d6c4f49ad13d0d08e224306 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:19.181382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:19.182779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:19.257268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:19.300095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672953a5-a83d-4dd1-af9d-5d1dee5cff02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:19.312550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672953a5-a83d-4dd1-af9d-5d1dee5cff02 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:19.312873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672953a5-a83d-4dd1-af9d-5d1dee5cff02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:19.313177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672953a5-a83d-4dd1-af9d-5d1dee5cff02 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:19.329074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672953a5-a83d-4dd1-af9d-5d1dee5cff02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:19.329074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672953a5-a83d-4dd1-af9d-5d1dee5cff02 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:19.334602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8940d2-ce2d-4ace-a6aa-f0deee820e7c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:19.343402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:19.347729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:19.347729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:19.347729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] PATH_INFO: `/v3/users/305f573e4a7b483eb929fc47f6b59d93` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:19.347729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=305f573e4a7b483eb929fc47f6b59d93)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:19.348512 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1119/2255] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:50:19.349717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672953a5-a83d-4dd1-af9d-5d1dee5cff02 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:19.351503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:19.352142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-672953a5-a83d-4dd1-af9d-5d1dee5cff02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:19.367040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672953a5-a83d-4dd1-af9d-5d1dee5cff02 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:19.381841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe98dc-557d-4b51-b4ce-59110124f4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:19.382883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe98dc-557d-4b51-b4ce-59110124f4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:19.383363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe98dc-557d-4b51-b4ce-59110124f4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:19.384006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe98dc-557d-4b51-b4ce-59110124f4fe admin admin] PATH_INFO: `/v3/projects/4a5d6aa09fb0449ea9078a380c02ac87/users/c2394c816d8c47b887c085027a81b7a9/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:19.386617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe98dc-557d-4b51-b4ce-59110124f4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5d6aa09fb0449ea9078a380c02ac87, user_id=c2394c816d8c47b887c085027a81b7a9, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:19.390026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe98dc-557d-4b51-b4ce-59110124f4fe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:19.405276 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1120/2256] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:19 2022] PUT /identity/v3/projects/4a5d6aa09fb0449ea9078a380c02ac87/users/c2394c816d8c47b887c085027a81b7a9/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:19.440606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305f573e4a7b483eb929fc47f6b59d93'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:19.467858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305f573e4a7b483eb929fc47f6b59d93'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:19.480916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305f573e4a7b483eb929fc47f6b59d93'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:19.501618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305f573e4a7b483eb929fc47f6b59d93'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:19.503119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-99b2b631-a4fe-4b5b-b0ae-bc8540a29113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305f573e4a7b483eb929fc47f6b59d93'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:19.513851 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1137/2257] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:50:19 2022] DELETE /identity/v3/users/305f573e4a7b483eb929fc47f6b59d93 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:21.354344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe98dc-557d-4b51-b4ce-59110124f4fe admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:21.356746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bafe98dc-557d-4b51-b4ce-59110124f4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:21.518851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe98dc-557d-4b51-b4ce-59110124f4fe admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:21.730599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:21.731265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:21.731541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:21.731812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] PATH_INFO: `/v3/projects/5a84a87d983f400faf2761a5a5b0fef5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:21.741913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a84a87d983f400faf2761a5a5b0fef5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:21.745548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:21.845726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:21.876795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a84a87d983f400faf2761a5a5b0fef5', 'request_id': 'req-cc74781e-08a8-422e-8585-88ba880b6f8e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:21.893356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a84a87d983f400faf2761a5a5b0fef5', 'request_id': 'req-cc74781e-08a8-422e-8585-88ba880b6f8e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:21.904386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a84a87d983f400faf2761a5a5b0fef5', 'request_id': 'req-cc74781e-08a8-422e-8585-88ba880b6f8e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:21.938406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] The token cache is being invalidate because project 5a84a87d983f400faf2761a5a5b0fef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:50:21.938867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:21.939542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cc74781e-08a8-422e-8585-88ba880b6f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:21.943570 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1121/2258] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:50:21 2022] DELETE /identity/v3/projects/5a84a87d983f400faf2761a5a5b0fef5 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:21.949567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371274bb-a86c-448a-89c3-0bade7718a82 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:21.950487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371274bb-a86c-448a-89c3-0bade7718a82 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:21.950968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371274bb-a86c-448a-89c3-0bade7718a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:22.024095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-371274bb-a86c-448a-89c3-0bade7718a82 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:22.024680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-371274bb-a86c-448a-89c3-0bade7718a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:22.221877 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1138/2259] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:22.224190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42381fcf-57fe-4b47-8d4b-2769b245ec49 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:22.224190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42381fcf-57fe-4b47-8d4b-2769b245ec49 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:22.224190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42381fcf-57fe-4b47-8d4b-2769b245ec49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:22.263157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-42381fcf-57fe-4b47-8d4b-2769b245ec49 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:22.264568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-42381fcf-57fe-4b47-8d4b-2769b245ec49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:22.286629 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1122/2260] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:22.288244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371274bb-a86c-448a-89c3-0bade7718a82 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:22.308482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371274bb-a86c-448a-89c3-0bade7718a82 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:22.330852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57bdcd-515a-434c-9448-259f1e848d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:22.345524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57bdcd-515a-434c-9448-259f1e848d34 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:22.347675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57bdcd-515a-434c-9448-259f1e848d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:22.347936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57bdcd-515a-434c-9448-259f1e848d34 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:22.349050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57bdcd-515a-434c-9448-259f1e848d34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:50:22.353433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57bdcd-515a-434c-9448-259f1e848d34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:22.360281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57bdcd-515a-434c-9448-259f1e848d34 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:22.410008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42381fcf-57fe-4b47-8d4b-2769b245ec49 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:22.414445 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1139/2261] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:22.422531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42381fcf-57fe-4b47-8d4b-2769b245ec49 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:22.437560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76b84-5fd7-431c-8cbf-a6d935687a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:22.438198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76b84-5fd7-431c-8cbf-a6d935687a00 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:22.438468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76b84-5fd7-431c-8cbf-a6d935687a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:22.438735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76b84-5fd7-431c-8cbf-a6d935687a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:22.439387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab76b84-5fd7-431c-8cbf-a6d935687a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:22.441312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab76b84-5fd7-431c-8cbf-a6d935687a00 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:22.508507 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1123/2262] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:50:22 2022] POST /identity/v3/projects => generated 407 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:22.513144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57bdcd-515a-434c-9448-259f1e848d34 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:22.533827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57bdcd-515a-434c-9448-259f1e848d34 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:22.602746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:22.651127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc09d28-4a6e-488c-9506-57214a52364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:22.651738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09d28-4a6e-488c-9506-57214a52364e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:22.652012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09d28-4a6e-488c-9506-57214a52364e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:22.652278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09d28-4a6e-488c-9506-57214a52364e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:22.652904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc09d28-4a6e-488c-9506-57214a52364e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:22.660151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc09d28-4a6e-488c-9506-57214a52364e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:22.708083 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1140/2263] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:50:22 2022] POST /identity/v3/users => generated 304 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:22.709373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76b84-5fd7-431c-8cbf-a6d935687a00 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:22.725876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76b84-5fd7-431c-8cbf-a6d935687a00 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:22.745797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250889c-3d1c-4b1e-803b-6c3a6b89139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:22.746580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250889c-3d1c-4b1e-803b-6c3a6b89139b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:22.746958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250889c-3d1c-4b1e-803b-6c3a6b89139b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:22.747715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250889c-3d1c-4b1e-803b-6c3a6b89139b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:22.751015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250889c-3d1c-4b1e-803b-6c3a6b89139b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:22.754893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250889c-3d1c-4b1e-803b-6c3a6b89139b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:22.784439 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1124/2264] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:22.788376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc09d28-4a6e-488c-9506-57214a52364e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:22.822092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc09d28-4a6e-488c-9506-57214a52364e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:22.875787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab4a37-a491-4801-8718-5ec69bd96c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:22.875787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab4a37-a491-4801-8718-5ec69bd96c0f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:22.875787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab4a37-a491-4801-8718-5ec69bd96c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:22.875787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab4a37-a491-4801-8718-5ec69bd96c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:22.876393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab4a37-a491-4801-8718-5ec69bd96c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:22.876393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab4a37-a491-4801-8718-5ec69bd96c0f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:22.902384 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1141/2265] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:22.902629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250889c-3d1c-4b1e-803b-6c3a6b89139b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:22.917257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250889c-3d1c-4b1e-803b-6c3a6b89139b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:22.936292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6d37a-2bc4-4c33-86a2-a1e2cea2b7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:22.936658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6d37a-2bc4-4c33-86a2-a1e2cea2b7d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:22.938462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6d37a-2bc4-4c33-86a2-a1e2cea2b7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:22.938950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6d37a-2bc4-4c33-86a2-a1e2cea2b7d5 admin admin] PATH_INFO: `/v3/projects/b6a3e3b54fc94c19b60e3da80737ac14/users/c38734f558c8459f8bc8cc4a11ebeb4d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:22.993872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6d37a-2bc4-4c33-86a2-a1e2cea2b7d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a3e3b54fc94c19b60e3da80737ac14, user_id=c38734f558c8459f8bc8cc4a11ebeb4d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:22.997516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6d37a-2bc4-4c33-86a2-a1e2cea2b7d5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:23.074557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe12b2-5861-4f76-8c6e-19ddf59d996d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:23.075105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe12b2-5861-4f76-8c6e-19ddf59d996d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:23.075209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe12b2-5861-4f76-8c6e-19ddf59d996d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:23.079121 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1125/2266] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:22 2022] PUT /identity/v3/projects/b6a3e3b54fc94c19b60e3da80737ac14/users/c38734f558c8459f8bc8cc4a11ebeb4d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:23.134540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-87fe12b2-5861-4f76-8c6e-19ddf59d996d None None] MFA Rules not processed for user `c38734f558c8459f8bc8cc4a11ebeb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:23.136011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-87fe12b2-5861-4f76-8c6e-19ddf59d996d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:23.356872 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1142/2267] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:23.362382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6d37a-2bc4-4c33-86a2-a1e2cea2b7d5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:23.531295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6d37a-2bc4-4c33-86a2-a1e2cea2b7d5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:23.551287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ebf2c-322b-4462-90a1-a3be1469e616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:23.551909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ebf2c-322b-4462-90a1-a3be1469e616 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:23.552181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ebf2c-322b-4462-90a1-a3be1469e616 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:23.552443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ebf2c-322b-4462-90a1-a3be1469e616 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:23.553095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ebf2c-322b-4462-90a1-a3be1469e616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:23.570309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ebf2c-322b-4462-90a1-a3be1469e616 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:23.682144 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1126/2268] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:50:23 2022] POST /identity/v3/projects => generated 407 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:23.686851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe12b2-5861-4f76-8c6e-19ddf59d996d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:23.718793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe12b2-5861-4f76-8c6e-19ddf59d996d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:23.957568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143a96b-0148-47cf-a16d-27098526f9d8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:23.958065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143a96b-0148-47cf-a16d-27098526f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:23.958408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143a96b-0148-47cf-a16d-27098526f9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:24.033320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8ea3-75b0-4d91-8bd4-4da5b53f737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:24.034369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8ea3-75b0-4d91-8bd4-4da5b53f737e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:24.034793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8ea3-75b0-4d91-8bd4-4da5b53f737e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:24.036221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8ea3-75b0-4d91-8bd4-4da5b53f737e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:24.039181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f143a96b-0148-47cf-a16d-27098526f9d8 None None] MFA Rules not processed for user `c613201f55b94a528c6c36e5b5772799`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:24.040109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba8ea3-75b0-4d91-8bd4-4da5b53f737e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:24.041504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f143a96b-0148-47cf-a16d-27098526f9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:24.046166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba8ea3-75b0-4d91-8bd4-4da5b53f737e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:24.106008 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1143/2269] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:50:23 2022] POST /identity/v3/users => generated 310 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:24.107952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8ea3-75b0-4d91-8bd4-4da5b53f737e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:24.159853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8ea3-75b0-4d91-8bd4-4da5b53f737e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:24.166839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53743c-7c21-4acc-8ab0-ffdfdc7833ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:24.167781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53743c-7c21-4acc-8ab0-ffdfdc7833ca admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:24.168055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53743c-7c21-4acc-8ab0-ffdfdc7833ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:24.168324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53743c-7c21-4acc-8ab0-ffdfdc7833ca admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:24.170513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53743c-7c21-4acc-8ab0-ffdfdc7833ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:24.174521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53743c-7c21-4acc-8ab0-ffdfdc7833ca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:24.204043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:24.211394 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1144/2270] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:24.218276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53743c-7c21-4acc-8ab0-ffdfdc7833ca admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:24.242047 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1127/2271] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:24.246379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53743c-7c21-4acc-8ab0-ffdfdc7833ca admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:24.255025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a78be-6816-4543-adde-bef563f09f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:24.255734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a78be-6816-4543-adde-bef563f09f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:24.256178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a78be-6816-4543-adde-bef563f09f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:24.256767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a78be-6816-4543-adde-bef563f09f01 admin admin] PATH_INFO: `/v3/projects/b771ca70349d47ec9ef65b837a25b2b2/users/5350e89ee4104986be48f22e5f393935/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:24.291574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a78be-6816-4543-adde-bef563f09f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b771ca70349d47ec9ef65b837a25b2b2, user_id=5350e89ee4104986be48f22e5f393935, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:24.294351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a78be-6816-4543-adde-bef563f09f01 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:24.317471 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1145/2272] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:24 2022] PUT /identity/v3/projects/b771ca70349d47ec9ef65b837a25b2b2/users/5350e89ee4104986be48f22e5f393935/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:24.321136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143a96b-0148-47cf-a16d-27098526f9d8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:24.340685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143a96b-0148-47cf-a16d-27098526f9d8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:24.467917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a92a9-d6a4-451f-a4af-b3e59abc0482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:24.469552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a92a9-d6a4-451f-a4af-b3e59abc0482 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:24.470638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a92a9-d6a4-451f-a4af-b3e59abc0482 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:24.471738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a92a9-d6a4-451f-a4af-b3e59abc0482 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:24.472664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a92a9-d6a4-451f-a4af-b3e59abc0482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:24.474638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a92a9-d6a4-451f-a4af-b3e59abc0482 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:24.489786 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1128/2273] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:24.494508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a78be-6816-4543-adde-bef563f09f01 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:24.500944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:24.534641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a78be-6816-4543-adde-bef563f09f01 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:24.552071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516613f-ebf1-4496-8964-e8dd38972a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:24.553237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516613f-ebf1-4496-8964-e8dd38972a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:24.553609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516613f-ebf1-4496-8964-e8dd38972a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:24.555374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516613f-ebf1-4496-8964-e8dd38972a06 admin admin] PATH_INFO: `/v3/projects/b771ca70349d47ec9ef65b837a25b2b2/users/5350e89ee4104986be48f22e5f393935/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:24.560444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516613f-ebf1-4496-8964-e8dd38972a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b771ca70349d47ec9ef65b837a25b2b2, user_id=5350e89ee4104986be48f22e5f393935, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:24.573606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516613f-ebf1-4496-8964-e8dd38972a06 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:24.603984 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1146/2274] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:24 2022] PUT /identity/v3/projects/b771ca70349d47ec9ef65b837a25b2b2/users/5350e89ee4104986be48f22e5f393935/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:24.630736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ed639-cefe-4e8c-99fa-a4a44a9cb512 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:24.631209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ed639-cefe-4e8c-99fa-a4a44a9cb512 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:24.631304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ed639-cefe-4e8c-99fa-a4a44a9cb512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:24.707012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-790ed639-cefe-4e8c-99fa-a4a44a9cb512 None None] MFA Rules not processed for user `5350e89ee4104986be48f22e5f393935`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:24.708232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-790ed639-cefe-4e8c-99fa-a4a44a9cb512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:24.895949 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1129/2275] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:24.898482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516613f-ebf1-4496-8964-e8dd38972a06 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:25.080056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516613f-ebf1-4496-8964-e8dd38972a06 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:25.099433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:25.100201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:25.100714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:25.101053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] PATH_INFO: `/v3/users/c38734f558c8459f8bc8cc4a11ebeb4d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:25.102927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c38734f558c8459f8bc8cc4a11ebeb4d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:25.105407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:25.206495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c38734f558c8459f8bc8cc4a11ebeb4d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:25.229887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c38734f558c8459f8bc8cc4a11ebeb4d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:25.250866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c38734f558c8459f8bc8cc4a11ebeb4d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:25.264919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c38734f558c8459f8bc8cc4a11ebeb4d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:25.265958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7492f1bc-d12c-489e-a92f-0706e0714d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c38734f558c8459f8bc8cc4a11ebeb4d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:25.278179 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1147/2276] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:50:24 2022] DELETE /identity/v3/users/c38734f558c8459f8bc8cc4a11ebeb4d => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:25.283393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130a4a0-ae4c-47be-806d-34be1c90dbb5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:25.283667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130a4a0-ae4c-47be-806d-34be1c90dbb5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:25.284488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130a4a0-ae4c-47be-806d-34be1c90dbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:25.339569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6130a4a0-ae4c-47be-806d-34be1c90dbb5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:25.344475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6130a4a0-ae4c-47be-806d-34be1c90dbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:25.363266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3cbca-2e70-46af-b6e2-dc169f496a32 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:25.363976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3cbca-2e70-46af-b6e2-dc169f496a32 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:25.364440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3cbca-2e70-46af-b6e2-dc169f496a32 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:25.366325 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1148/2277] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:25.373779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d03b9-9fb4-42d7-9282-6a39fc6958c7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:25.374456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d03b9-9fb4-42d7-9282-6a39fc6958c7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:25.375082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d03b9-9fb4-42d7-9282-6a39fc6958c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:25.416177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4f2d03b9-9fb4-42d7-9282-6a39fc6958c7 None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:25.417776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4f2d03b9-9fb4-42d7-9282-6a39fc6958c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:25.508660 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1130/2278] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:25.523780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130a4a0-ae4c-47be-806d-34be1c90dbb5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:25.526503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6130a4a0-ae4c-47be-806d-34be1c90dbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:25.550046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:25.561639 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1149/2279] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:25.606360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d03b9-9fb4-42d7-9282-6a39fc6958c7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:25.612077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4f2d03b9-9fb4-42d7-9282-6a39fc6958c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:25.628573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d03b9-9fb4-42d7-9282-6a39fc6958c7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:25.633654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc98aa1-0108-4f0e-9a02-b388dec94ab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:25.634402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc98aa1-0108-4f0e-9a02-b388dec94ab9 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:25.634686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc98aa1-0108-4f0e-9a02-b388dec94ab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:25.634954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc98aa1-0108-4f0e-9a02-b388dec94ab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:25.644704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc98aa1-0108-4f0e-9a02-b388dec94ab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:25.646188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc98aa1-0108-4f0e-9a02-b388dec94ab9 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:25.656759 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1150/2280] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:25.742358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130a4a0-ae4c-47be-806d-34be1c90dbb5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:25.754999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67573aa9-147a-4d11-992d-3f602a1975bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:25.755624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67573aa9-147a-4d11-992d-3f602a1975bc service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:25.755965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67573aa9-147a-4d11-992d-3f602a1975bc service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:25.756153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67573aa9-147a-4d11-992d-3f602a1975bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:25.778679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67573aa9-147a-4d11-992d-3f602a1975bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:25.780321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67573aa9-147a-4d11-992d-3f602a1975bc service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:25.801511 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1131/2281] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:26.473017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c591a6-77df-4398-ab2d-decaa56f7717 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:26.474363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c591a6-77df-4398-ab2d-decaa56f7717 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:26.474910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c591a6-77df-4398-ab2d-decaa56f7717 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:26.476794 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1151/2282] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:50:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:26.490107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39eacc-9521-4292-9848-8f5780c76546 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:26.490576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39eacc-9521-4292-9848-8f5780c76546 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:26.490720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39eacc-9521-4292-9848-8f5780c76546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:26.569874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3c39eacc-9521-4292-9848-8f5780c76546 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:26.577233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3c39eacc-9521-4292-9848-8f5780c76546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:26.785125 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1132/2283] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:26.823846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c591a6-77df-4398-ab2d-decaa56f7717 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:26.831228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:26.839347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c591a6-77df-4398-ab2d-decaa56f7717 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:26.851901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1642c7-df0f-4013-9264-ae3adb4a59c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:26.852630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1642c7-df0f-4013-9264-ae3adb4a59c7 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:26.852910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1642c7-df0f-4013-9264-ae3adb4a59c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:26.853222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1642c7-df0f-4013-9264-ae3adb4a59c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:26.867023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1642c7-df0f-4013-9264-ae3adb4a59c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:26.868530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1642c7-df0f-4013-9264-ae3adb4a59c7 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:26.889946 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1152/2284] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:27.481302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39eacc-9521-4292-9848-8f5780c76546 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:27.483826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3c39eacc-9521-4292-9848-8f5780c76546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:27.522262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39eacc-9521-4292-9848-8f5780c76546 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:27.540247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:27.559764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:27.560253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:27.562644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] PATH_INFO: `/v3/projects/b6a3e3b54fc94c19b60e3da80737ac14` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:27.570792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a3e3b54fc94c19b60e3da80737ac14)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:27.572718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:27.763208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a3e3b54fc94c19b60e3da80737ac14', 'request_id': 'req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:27.798336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a3e3b54fc94c19b60e3da80737ac14', 'request_id': 'req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:27.825107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a3e3b54fc94c19b60e3da80737ac14', 'request_id': 'req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:27.885109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] The token cache is being invalidate because project b6a3e3b54fc94c19b60e3da80737ac14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:50:27.885400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:27.889623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:27.893651 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1133/2285] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:50:27 2022] DELETE /identity/v3/projects/b6a3e3b54fc94c19b60e3da80737ac14 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:27.897568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1642c7-df0f-4013-9264-ae3adb4a59c7 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:27.926977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1642c7-df0f-4013-9264-ae3adb4a59c7 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:28.086151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:28.086842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:28.086957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:28.087171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] PATH_INFO: `/v3/users/5350e89ee4104986be48f22e5f393935` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:28.090855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5350e89ee4104986be48f22e5f393935)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:28.098817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:28.217462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5350e89ee4104986be48f22e5f393935'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:28.254279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5350e89ee4104986be48f22e5f393935'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:28.267130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5350e89ee4104986be48f22e5f393935'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:28.282179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5350e89ee4104986be48f22e5f393935'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:28.284034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0270a0ae-3837-44c3-ad78-7397000ba4eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5350e89ee4104986be48f22e5f393935'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:28.306923 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1153/2286] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:50:27 2022] DELETE /identity/v3/users/5350e89ee4104986be48f22e5f393935 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:29.059065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:29.060646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:29.092575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa2f72-9dfe-4911-a5d9-a4e9e56977cb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:29.183902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:29.286089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:29.286825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:29.287203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:29.287573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] PATH_INFO: `/v3/projects/b771ca70349d47ec9ef65b837a25b2b2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:29.289581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b771ca70349d47ec9ef65b837a25b2b2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:29.291853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:29.381004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b771ca70349d47ec9ef65b837a25b2b2', 'request_id': 'req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:29.397735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b771ca70349d47ec9ef65b837a25b2b2', 'request_id': 'req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:29.408011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b771ca70349d47ec9ef65b837a25b2b2', 'request_id': 'req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:29.436080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] The token cache is being invalidate because project b771ca70349d47ec9ef65b837a25b2b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:50:29.437897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:29.441007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2697d48f-d136-45aa-b5ec-5c2bb91f4e9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:50:29.454277 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1134/2287] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:50:29 2022] DELETE /identity/v3/projects/b771ca70349d47ec9ef65b837a25b2b2 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:29.455422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390888c-deff-4e3c-b4fc-99c4346cf23d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:29.455899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390888c-deff-4e3c-b4fc-99c4346cf23d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:29.456148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390888c-deff-4e3c-b4fc-99c4346cf23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:29.501763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3390888c-deff-4e3c-b4fc-99c4346cf23d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:29.503271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3390888c-deff-4e3c-b4fc-99c4346cf23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:29.617156 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1154/2288] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:50:29.625446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7e595-9626-4601-947f-8b40756a8f92 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:29.625740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7e595-9626-4601-947f-8b40756a8f92 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:29.626064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7e595-9626-4601-947f-8b40756a8f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:29.726631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-02f7e595-9626-4601-947f-8b40756a8f92 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:29.733268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-02f7e595-9626-4601-947f-8b40756a8f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:29.786944 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1135/2289] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:29.790063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390888c-deff-4e3c-b4fc-99c4346cf23d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:29.803589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390888c-deff-4e3c-b4fc-99c4346cf23d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:29.811487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bd85d-58ac-4386-8c33-3ff324fe76de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:29.812099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bd85d-58ac-4386-8c33-3ff324fe76de admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:29.812367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bd85d-58ac-4386-8c33-3ff324fe76de admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:29.812629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bd85d-58ac-4386-8c33-3ff324fe76de admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:29.814690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bd85d-58ac-4386-8c33-3ff324fe76de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:50:29.814991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bd85d-58ac-4386-8c33-3ff324fe76de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:29.816632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bd85d-58ac-4386-8c33-3ff324fe76de admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:29.839183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:29.850257 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1155/2290] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:50:29.853528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7e595-9626-4601-947f-8b40756a8f92 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:29.870482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7e595-9626-4601-947f-8b40756a8f92 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:29.891878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb0038-c13a-49de-90a0-cfe5b2a6eea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:29.892696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb0038-c13a-49de-90a0-cfe5b2a6eea8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:29.893282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb0038-c13a-49de-90a0-cfe5b2a6eea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:29.894092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb0038-c13a-49de-90a0-cfe5b2a6eea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:29.896244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb0038-c13a-49de-90a0-cfe5b2a6eea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:29.898517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb0038-c13a-49de-90a0-cfe5b2a6eea8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:29.940046 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1136/2291] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:50:29 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:29.943969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bd85d-58ac-4386-8c33-3ff324fe76de admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:29.957948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bd85d-58ac-4386-8c33-3ff324fe76de admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:30.073796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d90a0-0e74-490c-bd07-d3e311f9ac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:30.074604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d90a0-0e74-490c-bd07-d3e311f9ac71 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:30.074983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d90a0-0e74-490c-bd07-d3e311f9ac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:30.075352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d90a0-0e74-490c-bd07-d3e311f9ac71 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:30.076616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d90a0-0e74-490c-bd07-d3e311f9ac71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:30.079173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d90a0-0e74-490c-bd07-d3e311f9ac71 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:30.112533 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1156/2292] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:50:29 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:30.117280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb0038-c13a-49de-90a0-cfe5b2a6eea8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:30.137751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb0038-c13a-49de-90a0-cfe5b2a6eea8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:30.144528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb5555-23db-4c99-b885-14c11fb0a7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:30.145488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb5555-23db-4c99-b885-14c11fb0a7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:30.146045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb5555-23db-4c99-b885-14c11fb0a7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:30.146521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb5555-23db-4c99-b885-14c11fb0a7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:30.147494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb5555-23db-4c99-b885-14c11fb0a7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:30.149964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb5555-23db-4c99-b885-14c11fb0a7c0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:30.178358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d90a0-0e74-490c-bd07-d3e311f9ac71 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:30.179717 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1137/2293] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:30.191338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d90a0-0e74-490c-bd07-d3e311f9ac71 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:30.199542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495770a-0d98-4349-ac3c-6b66c0f75232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:30.202113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495770a-0d98-4349-ac3c-6b66c0f75232 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:30.202782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495770a-0d98-4349-ac3c-6b66c0f75232 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:30.203584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495770a-0d98-4349-ac3c-6b66c0f75232 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:30.204455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495770a-0d98-4349-ac3c-6b66c0f75232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:30.206370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495770a-0d98-4349-ac3c-6b66c0f75232 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:30.228731 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1157/2294] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:30.229511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb5555-23db-4c99-b885-14c11fb0a7c0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:30.259749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb5555-23db-4c99-b885-14c11fb0a7c0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:30.274446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060897f-76d2-4fb0-aba0-77dcde94bcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:30.275371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060897f-76d2-4fb0-aba0-77dcde94bcb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:30.275810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060897f-76d2-4fb0-aba0-77dcde94bcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:30.276227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060897f-76d2-4fb0-aba0-77dcde94bcb9 admin admin] PATH_INFO: `/v3/projects/b107efbbeb064e1fbf3c8bf31da12cbe/users/1ddcfe1b0eb5431dbbe4777b8dd77436/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:30.308194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f060897f-76d2-4fb0-aba0-77dcde94bcb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b107efbbeb064e1fbf3c8bf31da12cbe, user_id=1ddcfe1b0eb5431dbbe4777b8dd77436, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:30.313180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f060897f-76d2-4fb0-aba0-77dcde94bcb9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:30.354240 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1138/2295] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:30 2022] PUT /identity/v3/projects/b107efbbeb064e1fbf3c8bf31da12cbe/users/1ddcfe1b0eb5431dbbe4777b8dd77436/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:30.446193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401beb81-f41c-4023-9ab4-b1b0b0e7de36 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:30.446696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401beb81-f41c-4023-9ab4-b1b0b0e7de36 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:30.447082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401beb81-f41c-4023-9ab4-b1b0b0e7de36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:30.495146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-401beb81-f41c-4023-9ab4-b1b0b0e7de36 None None] MFA Rules not processed for user `1ddcfe1b0eb5431dbbe4777b8dd77436`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:30.497224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-401beb81-f41c-4023-9ab4-b1b0b0e7de36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:30.717793 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1158/2296] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:30.721692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060897f-76d2-4fb0-aba0-77dcde94bcb9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:30.829547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:30.851132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060897f-76d2-4fb0-aba0-77dcde94bcb9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:30.857577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6cf29-1b03-44c9-a806-2ec4b2efd0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:30.858465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6cf29-1b03-44c9-a806-2ec4b2efd0cc admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:30.859933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6cf29-1b03-44c9-a806-2ec4b2efd0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:30.860587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6cf29-1b03-44c9-a806-2ec4b2efd0cc admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:30.861452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6cf29-1b03-44c9-a806-2ec4b2efd0cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:30.865831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6cf29-1b03-44c9-a806-2ec4b2efd0cc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:30.915462 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1139/2297] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:50:30 2022] POST /identity/v3/projects => generated 383 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:50:30.915871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401beb81-f41c-4023-9ab4-b1b0b0e7de36 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:30.933767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401beb81-f41c-4023-9ab4-b1b0b0e7de36 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:31.046267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d119cd4-a0c0-42d5-9a85-39f075a29d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:31.047342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d119cd4-a0c0-42d5-9a85-39f075a29d24 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:31.047772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d119cd4-a0c0-42d5-9a85-39f075a29d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:31.048201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d119cd4-a0c0-42d5-9a85-39f075a29d24 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:31.049001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d119cd4-a0c0-42d5-9a85-39f075a29d24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:31.051412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d119cd4-a0c0-42d5-9a85-39f075a29d24 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:31.069156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:31.096381 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1159/2298] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:50:30 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:50:31.099212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6cf29-1b03-44c9-a806-2ec4b2efd0cc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:31.119182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6cf29-1b03-44c9-a806-2ec4b2efd0cc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:31.127758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39788bc9-5a8c-46a3-912f-b5c96924a679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:31.128745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39788bc9-5a8c-46a3-912f-b5c96924a679 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:31.129277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39788bc9-5a8c-46a3-912f-b5c96924a679 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:31.129840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39788bc9-5a8c-46a3-912f-b5c96924a679 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:31.130710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39788bc9-5a8c-46a3-912f-b5c96924a679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:31.132882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39788bc9-5a8c-46a3-912f-b5c96924a679 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:31.153053 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1140/2299] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:31.153246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d119cd4-a0c0-42d5-9a85-39f075a29d24 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:31.173295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d119cd4-a0c0-42d5-9a85-39f075a29d24 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:31.188719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d5bf0-7338-432e-a2b9-f1fd0daa8abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:31.189426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d5bf0-7338-432e-a2b9-f1fd0daa8abb admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:31.189704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d5bf0-7338-432e-a2b9-f1fd0daa8abb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:31.189973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d5bf0-7338-432e-a2b9-f1fd0daa8abb admin admin] PATH_INFO: `/v3/projects/0c63de51873b4e1c93aae1c7980bc2be/users/4024451bc4d04ab1b2230e0da780ef32/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:31.236697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d5bf0-7338-432e-a2b9-f1fd0daa8abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c63de51873b4e1c93aae1c7980bc2be, user_id=4024451bc4d04ab1b2230e0da780ef32, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:31.239429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d5bf0-7338-432e-a2b9-f1fd0daa8abb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:31.265436 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1160/2300] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:31 2022] PUT /identity/v3/projects/0c63de51873b4e1c93aae1c7980bc2be/users/4024451bc4d04ab1b2230e0da780ef32/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:31.267543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39788bc9-5a8c-46a3-912f-b5c96924a679 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:31.289013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39788bc9-5a8c-46a3-912f-b5c96924a679 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:31.558141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6d537-f66f-4537-9849-7db1cb714349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:31.559761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6d537-f66f-4537-9849-7db1cb714349 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:31.560356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6d537-f66f-4537-9849-7db1cb714349 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:31.561196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6d537-f66f-4537-9849-7db1cb714349 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:31.563126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6d537-f66f-4537-9849-7db1cb714349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:31.565449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6d537-f66f-4537-9849-7db1cb714349 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:31.600746 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1141/2301] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:50:31.605279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d5bf0-7338-432e-a2b9-f1fd0daa8abb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:31.644429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d5bf0-7338-432e-a2b9-f1fd0daa8abb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:31.701609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259873b9-aa54-46cc-a403-e07ecb639ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:31.713870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259873b9-aa54-46cc-a403-e07ecb639ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:31.713870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259873b9-aa54-46cc-a403-e07ecb639ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:31.713870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259873b9-aa54-46cc-a403-e07ecb639ce1 admin admin] PATH_INFO: `/v3/projects/0c63de51873b4e1c93aae1c7980bc2be/users/4024451bc4d04ab1b2230e0da780ef32/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:31.713870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259873b9-aa54-46cc-a403-e07ecb639ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c63de51873b4e1c93aae1c7980bc2be, user_id=4024451bc4d04ab1b2230e0da780ef32, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:31.720329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259873b9-aa54-46cc-a403-e07ecb639ce1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:31.808423 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1161/2302] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:50:31 2022] PUT /identity/v3/projects/0c63de51873b4e1c93aae1c7980bc2be/users/4024451bc4d04ab1b2230e0da780ef32/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:50:31.825722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14db28-6f86-4763-aa1e-b1b4bde761d1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:31.826352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14db28-6f86-4763-aa1e-b1b4bde761d1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:31.826852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14db28-6f86-4763-aa1e-b1b4bde761d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:31.944217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7e14db28-6f86-4763-aa1e-b1b4bde761d1 None None] MFA Rules not processed for user `4024451bc4d04ab1b2230e0da780ef32`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:31.947987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7e14db28-6f86-4763-aa1e-b1b4bde761d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:32.146181 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1142/2303] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:32.159197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd21aa-4f40-4a8f-ad87-7507832e9190 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:32.159512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd21aa-4f40-4a8f-ad87-7507832e9190 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:32.159730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd21aa-4f40-4a8f-ad87-7507832e9190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:32.206827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e0dd21aa-4f40-4a8f-ad87-7507832e9190 None None] MFA Rules not processed for user `1ddcfe1b0eb5431dbbe4777b8dd77436`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:32.213992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e0dd21aa-4f40-4a8f-ad87-7507832e9190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:32.392790 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1162/2304] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:32.733210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14db28-6f86-4763-aa1e-b1b4bde761d1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:32.734631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7e14db28-6f86-4763-aa1e-b1b4bde761d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:32.895169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:32.959911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14db28-6f86-4763-aa1e-b1b4bde761d1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:32.978543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d9a05-779c-4b5e-9d0e-8c7f4d88d2d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:32.979272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d9a05-779c-4b5e-9d0e-8c7f4d88d2d4 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:32.979659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d9a05-779c-4b5e-9d0e-8c7f4d88d2d4 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:32.980031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d9a05-779c-4b5e-9d0e-8c7f4d88d2d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:32.997939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d9a05-779c-4b5e-9d0e-8c7f4d88d2d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:32.999087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d9a05-779c-4b5e-9d0e-8c7f4d88d2d4 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:33.015897 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1143/2305] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:33.923270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfcb85-5376-41e2-90c9-8e8f10b4708d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:33.923830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfcb85-5376-41e2-90c9-8e8f10b4708d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:33.923830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfcb85-5376-41e2-90c9-8e8f10b4708d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:33.991216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-20bfcb85-5376-41e2-90c9-8e8f10b4708d None None] MFA Rules not processed for user `c2394c816d8c47b887c085027a81b7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:33.994322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-20bfcb85-5376-41e2-90c9-8e8f10b4708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:34.184112 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1163/2306] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:34.195527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988230-7729-451a-b4b9-9024faceb37b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:34.196416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988230-7729-451a-b4b9-9024faceb37b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:34.196827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988230-7729-451a-b4b9-9024faceb37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:34.239418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-83988230-7729-451a-b4b9-9024faceb37b None None] MFA Rules not processed for user `180d9b7141ff4792ab7ab19e21a3d7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:34.241375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-83988230-7729-451a-b4b9-9024faceb37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:34.393395 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1144/2307] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:34 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:35.497975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfcb85-5376-41e2-90c9-8e8f10b4708d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:35.500427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-20bfcb85-5376-41e2-90c9-8e8f10b4708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:35.550114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfcb85-5376-41e2-90c9-8e8f10b4708d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:35.558470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c981cb-23cf-45d8-8c21-ebcad8ab6feb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:35.559438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c981cb-23cf-45d8-8c21-ebcad8ab6feb service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:35.560178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c981cb-23cf-45d8-8c21-ebcad8ab6feb service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:35.560686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c981cb-23cf-45d8-8c21-ebcad8ab6feb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:35.563272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-26c981cb-23cf-45d8-8c21-ebcad8ab6feb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:35.605189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c981cb-23cf-45d8-8c21-ebcad8ab6feb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:35.607747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c981cb-23cf-45d8-8c21-ebcad8ab6feb service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:35.620157 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1164/2308] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:35 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:39.759535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83988230-7729-451a-b4b9-9024faceb37b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:39.761303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-83988230-7729-451a-b4b9-9024faceb37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:40.004029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83988230-7729-451a-b4b9-9024faceb37b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:40.015398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e2e2f-4750-444e-b8dc-e74a8e9be34a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:40.016763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e2e2f-4750-444e-b8dc-e74a8e9be34a service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:40.017279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e2e2f-4750-444e-b8dc-e74a8e9be34a service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:40.017879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e2e2f-4750-444e-b8dc-e74a8e9be34a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:40.020280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4b6e2e2f-4750-444e-b8dc-e74a8e9be34a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:40.043330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e2e2f-4750-444e-b8dc-e74a8e9be34a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:40.046079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e2e2f-4750-444e-b8dc-e74a8e9be34a service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:40.071115 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1145/2309] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:44.264717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0bf96-ef55-4028-a3bd-9c866b4290e6 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:44.265597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0bf96-ef55-4028-a3bd-9c866b4290e6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:44.266234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0bf96-ef55-4028-a3bd-9c866b4290e6 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:44.267707 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1165/2310] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:50:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:50:44.273338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40cc20-1353-4461-b0f7-5908c60ef711 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:44.282538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40cc20-1353-4461-b0f7-5908c60ef711 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:44.283050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40cc20-1353-4461-b0f7-5908c60ef711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:44.306930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3b40cc20-1353-4461-b0f7-5908c60ef711 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:44.308246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3b40cc20-1353-4461-b0f7-5908c60ef711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:44.319394 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1146/2311] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:50:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:44.338203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0bf96-ef55-4028-a3bd-9c866b4290e6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:44.340483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4e0bf96-ef55-4028-a3bd-9c866b4290e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:44.389001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:44.427341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0bf96-ef55-4028-a3bd-9c866b4290e6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:44.433311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aee468-e9a3-44be-a10c-951e85421d8c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:44.434084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aee468-e9a3-44be-a10c-951e85421d8c service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:44.435269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aee468-e9a3-44be-a10c-951e85421d8c service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:44.435776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aee468-e9a3-44be-a10c-951e85421d8c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:44.460909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aee468-e9a3-44be-a10c-951e85421d8c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:44.462393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aee468-e9a3-44be-a10c-951e85421d8c service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:44.476342 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1166/2312] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:44.664045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40cc20-1353-4461-b0f7-5908c60ef711 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:44.676648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3b40cc20-1353-4461-b0f7-5908c60ef711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:44.703454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40cc20-1353-4461-b0f7-5908c60ef711 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:44.712639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54620b80-00fe-45ba-b4f3-2e51a4905e0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:44.713384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54620b80-00fe-45ba-b4f3-2e51a4905e0c service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:44.714213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54620b80-00fe-45ba-b4f3-2e51a4905e0c service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:44.714692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54620b80-00fe-45ba-b4f3-2e51a4905e0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:44.718166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-54620b80-00fe-45ba-b4f3-2e51a4905e0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:44.740007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54620b80-00fe-45ba-b4f3-2e51a4905e0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:44.742275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54620b80-00fe-45ba-b4f3-2e51a4905e0c service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:44.759065 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1147/2313] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:46.575531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7349-f33c-4dfa-b5b0-304efc60f770 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:46.576002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7349-f33c-4dfa-b5b0-304efc60f770 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:46.576109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7349-f33c-4dfa-b5b0-304efc60f770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:46.607810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-197d7349-f33c-4dfa-b5b0-304efc60f770 None None] MFA Rules not processed for user `03bc9235fe1c4fc1b3097a913cbe1052`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:46.609126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-197d7349-f33c-4dfa-b5b0-304efc60f770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:46.676270 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1167/2314] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:50:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:47.671772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce8383-d06a-400c-ad52-52d3d9340a0b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:47.672652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce8383-d06a-400c-ad52-52d3d9340a0b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:47.673150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce8383-d06a-400c-ad52-52d3d9340a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:47.703061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-76ce8383-d06a-400c-ad52-52d3d9340a0b None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:50:47.704271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76ce8383-d06a-400c-ad52-52d3d9340a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:47.761197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:47.774262 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1148/2315] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:50:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:50:47.786625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7349-f33c-4dfa-b5b0-304efc60f770 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:47.788009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-197d7349-f33c-4dfa-b5b0-304efc60f770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:50:47.923763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7349-f33c-4dfa-b5b0-304efc60f770 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:47.931104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15677e-e6bd-4512-8a8e-8e086d198c52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:47.931782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15677e-e6bd-4512-8a8e-8e086d198c52 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:47.932045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15677e-e6bd-4512-8a8e-8e086d198c52 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:47.932781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15677e-e6bd-4512-8a8e-8e086d198c52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:47.945664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15677e-e6bd-4512-8a8e-8e086d198c52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:47.947662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15677e-e6bd-4512-8a8e-8e086d198c52 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:47.963088 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1168/2316] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:50:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:50:55.784109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce8383-d06a-400c-ad52-52d3d9340a0b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:50:55.798314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce8383-d06a-400c-ad52-52d3d9340a0b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:50:55.804377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ac160-04a9-47f0-9469-492644fa518b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:50:55.805111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ac160-04a9-47f0-9469-492644fa518b service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:50:55.805389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ac160-04a9-47f0-9469-492644fa518b service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:50:55.805665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ac160-04a9-47f0-9469-492644fa518b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:50:55.815033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ac160-04a9-47f0-9469-492644fa518b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:50:55.816627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ac160-04a9-47f0-9469-492644fa518b service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:50:55.827863 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1149/2317] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:50:55 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:50:57.957752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=67 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:00.688638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15677e-e6bd-4512-8a8e-8e086d198c52 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:00.750021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15677e-e6bd-4512-8a8e-8e086d198c52 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:00.810292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eed9f2-2306-47cc-97ec-b43fcefa28d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:00.811545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eed9f2-2306-47cc-97ec-b43fcefa28d4 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:00.811881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eed9f2-2306-47cc-97ec-b43fcefa28d4 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:00.812057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eed9f2-2306-47cc-97ec-b43fcefa28d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:00.825985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eed9f2-2306-47cc-97ec-b43fcefa28d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:00.830209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eed9f2-2306-47cc-97ec-b43fcefa28d4 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:00.840951 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1169/2318] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:51:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:51:01.015824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc07cf2-ace2-4650-962a-f96a9d36f392 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:01.016826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc07cf2-ace2-4650-962a-f96a9d36f392 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:01.019253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc07cf2-ace2-4650-962a-f96a9d36f392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:01.043505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-adc07cf2-ace2-4650-962a-f96a9d36f392 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:01.044717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-adc07cf2-ace2-4650-962a-f96a9d36f392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:01.054374 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1150/2319] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:51:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:01.068747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eed9f2-2306-47cc-97ec-b43fcefa28d4 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:01.086212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eed9f2-2306-47cc-97ec-b43fcefa28d4 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:01.094085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b6fb8-ff18-4c49-ae0d-e4276219bc69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:01.095203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b6fb8-ff18-4c49-ae0d-e4276219bc69 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:01.096218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b6fb8-ff18-4c49-ae0d-e4276219bc69 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:01.096691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b6fb8-ff18-4c49-ae0d-e4276219bc69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:01.115575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b6fb8-ff18-4c49-ae0d-e4276219bc69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:01.117317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b6fb8-ff18-4c49-ae0d-e4276219bc69 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:01.127470 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1170/2320] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:51:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:02.492099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0a67d-2550-44c0-9651-f74a9df092cd None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:02.492644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0a67d-2550-44c0-9651-f74a9df092cd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:02.492964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0a67d-2550-44c0-9651-f74a9df092cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:02.513739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f0c0a67d-2550-44c0-9651-f74a9df092cd None None] MFA Rules not processed for user `1ddcfe1b0eb5431dbbe4777b8dd77436`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:02.515393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f0c0a67d-2550-44c0-9651-f74a9df092cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:02.527865 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1151/2321] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:10.385412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b6fb8-ff18-4c49-ae0d-e4276219bc69 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:10.387308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f95b6fb8-ff18-4c49-ae0d-e4276219bc69 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:10.466146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b6fb8-ff18-4c49-ae0d-e4276219bc69 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:10.472104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:10.472735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:10.473226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:10.473505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] PATH_INFO: `/v3/users/c84b11d2d54b43cbac87115e227ef50e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:10.474896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c84b11d2d54b43cbac87115e227ef50e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:10.476904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:10.557605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c84b11d2d54b43cbac87115e227ef50e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:10.574334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c84b11d2d54b43cbac87115e227ef50e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:10.583433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c84b11d2d54b43cbac87115e227ef50e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:10.595700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c84b11d2d54b43cbac87115e227ef50e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:10.596790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c84b11d2d54b43cbac87115e227ef50e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:10.604603 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1171/2322] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:51:10 2022] DELETE /identity/v3/users/c84b11d2d54b43cbac87115e227ef50e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:10.609887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d35a9-c36a-42a8-b434-f5fc3f571ae2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:10.610364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d35a9-c36a-42a8-b434-f5fc3f571ae2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:10.610460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d35a9-c36a-42a8-b434-f5fc3f571ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:10.646549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cf8d35a9-c36a-42a8-b434-f5fc3f571ae2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:10.648034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cf8d35a9-c36a-42a8-b434-f5fc3f571ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:10.743852 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1152/2323] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:10.753170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:10.755736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:10.830316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578d8cc-afec-4f96-89f9-88d93c9b668c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:10.834828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ae266-8828-4807-92ae-78ec47a7ba24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:10.836109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ae266-8828-4807-92ae-78ec47a7ba24 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:10.836109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ae266-8828-4807-92ae-78ec47a7ba24 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:10.836109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ae266-8828-4807-92ae-78ec47a7ba24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:10.846431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:10.849243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ae266-8828-4807-92ae-78ec47a7ba24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:10.851357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ae266-8828-4807-92ae-78ec47a7ba24 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:10.862802 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1172/2324] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:51:11.034827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d35a9-c36a-42a8-b434-f5fc3f571ae2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:11.036610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cf8d35a9-c36a-42a8-b434-f5fc3f571ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:11.050577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d35a9-c36a-42a8-b434-f5fc3f571ae2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:11.056060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:11.057175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:11.057623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:11.058057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] PATH_INFO: `/v3/projects/1f0b466076164a309233726db31b0efd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:11.059733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f0b466076164a309233726db31b0efd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:11.061767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:11.116965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f0b466076164a309233726db31b0efd', 'request_id': 'req-17ffa7ef-96af-4ee7-950d-4741a475bf25'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.124381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f0b466076164a309233726db31b0efd', 'request_id': 'req-17ffa7ef-96af-4ee7-950d-4741a475bf25'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.131394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f0b466076164a309233726db31b0efd', 'request_id': 'req-17ffa7ef-96af-4ee7-950d-4741a475bf25'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.148007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] The token cache is being invalidate because project 1f0b466076164a309233726db31b0efd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:51:11.148396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.149204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.150778 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1153/2325] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:11 2022] DELETE /identity/v3/projects/1f0b466076164a309233726db31b0efd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:51:11.155627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ae266-8828-4807-92ae-78ec47a7ba24 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:11.167029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ae266-8828-4807-92ae-78ec47a7ba24 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:11.257695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:11.265340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:11.265865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:11.266448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] PATH_INFO: `/v3/users/15b61f670adb4431b2445f6c89e21b18` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:11.268285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15b61f670adb4431b2445f6c89e21b18)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:11.270827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:11.350583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15b61f670adb4431b2445f6c89e21b18'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.382518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15b61f670adb4431b2445f6c89e21b18'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.394320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15b61f670adb4431b2445f6c89e21b18'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.404957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15b61f670adb4431b2445f6c89e21b18'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.405789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e357ba5-68f2-4dd9-b19b-5cbe5c03975d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15b61f670adb4431b2445f6c89e21b18'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.417086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1173/2326] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:51:11 2022] DELETE /identity/v3/users/15b61f670adb4431b2445f6c89e21b18 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:51:11.728019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:11.730758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:11.755925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa7ef-96af-4ee7-950d-4741a475bf25 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:11.818371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:11.835592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:11.837402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:11.837923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:11.838392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] PATH_INFO: `/v3/projects/5c7b5e1d939a4b4592b1a81e4a30bcab` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:11.840143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c7b5e1d939a4b4592b1a81e4a30bcab)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:11.842760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:11.955787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c7b5e1d939a4b4592b1a81e4a30bcab', 'request_id': 'req-c5076f38-5f43-4240-acdc-7223decf64dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.969176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c7b5e1d939a4b4592b1a81e4a30bcab', 'request_id': 'req-c5076f38-5f43-4240-acdc-7223decf64dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:11.997071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c7b5e1d939a4b4592b1a81e4a30bcab', 'request_id': 'req-c5076f38-5f43-4240-acdc-7223decf64dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:12.034537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] The token cache is being invalidate because project 5c7b5e1d939a4b4592b1a81e4a30bcab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:51:12.034882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:12.035608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c5076f38-5f43-4240-acdc-7223decf64dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:12.043934 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1154/2327] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:11 2022] DELETE /identity/v3/projects/5c7b5e1d939a4b4592b1a81e4a30bcab => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:12.044649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6d3d8-2169-4c74-bbf1-41e07ec03d04 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:12.044938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6d3d8-2169-4c74-bbf1-41e07ec03d04 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:12.045225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6d3d8-2169-4c74-bbf1-41e07ec03d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:12.075058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b5f6d3d8-2169-4c74-bbf1-41e07ec03d04 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:12.076720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b5f6d3d8-2169-4c74-bbf1-41e07ec03d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:12.197528 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1174/2328] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:12.208429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd45360-0167-4f40-9c92-1b9ba2fac7b0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:12.208719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd45360-0167-4f40-9c92-1b9ba2fac7b0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:12.208994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd45360-0167-4f40-9c92-1b9ba2fac7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:12.238065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0cd45360-0167-4f40-9c92-1b9ba2fac7b0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:12.239298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0cd45360-0167-4f40-9c92-1b9ba2fac7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:12.268976 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1155/2329] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:12.270919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6d3d8-2169-4c74-bbf1-41e07ec03d04 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:12.288670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6d3d8-2169-4c74-bbf1-41e07ec03d04 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:12.295201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9884c1-8e1d-429c-8835-f703c3a9ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:12.295821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9884c1-8e1d-429c-8835-f703c3a9ed9e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:12.296142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9884c1-8e1d-429c-8835-f703c3a9ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:12.296410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9884c1-8e1d-429c-8835-f703c3a9ed9e admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:12.297431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9884c1-8e1d-429c-8835-f703c3a9ed9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:51:12.297736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9884c1-8e1d-429c-8835-f703c3a9ed9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:12.299606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9884c1-8e1d-429c-8835-f703c3a9ed9e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:12.321282 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1175/2330] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:51:12.326418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd45360-0167-4f40-9c92-1b9ba2fac7b0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:12.338819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd45360-0167-4f40-9c92-1b9ba2fac7b0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:12.344835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bac49-c52f-4945-b0d8-31a2eb6a02f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:12.345537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bac49-c52f-4945-b0d8-31a2eb6a02f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:12.345877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bac49-c52f-4945-b0d8-31a2eb6a02f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:12.346206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bac49-c52f-4945-b0d8-31a2eb6a02f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:12.347162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bac49-c52f-4945-b0d8-31a2eb6a02f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:12.348846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bac49-c52f-4945-b0d8-31a2eb6a02f0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:12.380865 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1156/2331] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:51:12 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:51:12.384705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9884c1-8e1d-429c-8835-f703c3a9ed9e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:12.390117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:12.400572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9884c1-8e1d-429c-8835-f703c3a9ed9e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:12.489648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961a6cb-6f55-4d5b-a91f-36f71e54a362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:12.490310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a6cb-6f55-4d5b-a91f-36f71e54a362 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:12.490592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a6cb-6f55-4d5b-a91f-36f71e54a362 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:12.490884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a6cb-6f55-4d5b-a91f-36f71e54a362 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:12.491521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961a6cb-6f55-4d5b-a91f-36f71e54a362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:12.493579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961a6cb-6f55-4d5b-a91f-36f71e54a362 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:12.519810 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1176/2332] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:51:12 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:51:12.523886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bac49-c52f-4945-b0d8-31a2eb6a02f0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:12.539640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bac49-c52f-4945-b0d8-31a2eb6a02f0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:12.546389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19275e4c-b963-4800-a09e-80890014155e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:12.547004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19275e4c-b963-4800-a09e-80890014155e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:12.547274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19275e4c-b963-4800-a09e-80890014155e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:12.547974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19275e4c-b963-4800-a09e-80890014155e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:12.548192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19275e4c-b963-4800-a09e-80890014155e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:12.549802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19275e4c-b963-4800-a09e-80890014155e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:12.584664 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1157/2333] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:51:12.586452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961a6cb-6f55-4d5b-a91f-36f71e54a362 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:12.603038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961a6cb-6f55-4d5b-a91f-36f71e54a362 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:12.608808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4f699-d8fe-4714-97f6-f62537f58a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:12.609579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f699-d8fe-4714-97f6-f62537f58a8a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:12.609960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f699-d8fe-4714-97f6-f62537f58a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:12.610331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f699-d8fe-4714-97f6-f62537f58a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:12.611076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4f699-d8fe-4714-97f6-f62537f58a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:12.613149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4f699-d8fe-4714-97f6-f62537f58a8a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:12.629988 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1177/2334] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:51:12.631232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19275e4c-b963-4800-a09e-80890014155e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:12.642491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19275e4c-b963-4800-a09e-80890014155e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:12.647679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf433a95-82f9-4688-bdc0-5fef27d9944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:12.648394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf433a95-82f9-4688-bdc0-5fef27d9944a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:12.648735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf433a95-82f9-4688-bdc0-5fef27d9944a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:12.649194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf433a95-82f9-4688-bdc0-5fef27d9944a admin admin] PATH_INFO: `/v3/projects/5545306632c2429a98f479af041c9db9/users/b90d5845757e4dc58e884ca9f59a71f0/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:12.667005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf433a95-82f9-4688-bdc0-5fef27d9944a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5545306632c2429a98f479af041c9db9, user_id=b90d5845757e4dc58e884ca9f59a71f0, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:12.670634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf433a95-82f9-4688-bdc0-5fef27d9944a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:12.687633 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1158/2335] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:51:12 2022] PUT /identity/v3/projects/5545306632c2429a98f479af041c9db9/users/b90d5845757e4dc58e884ca9f59a71f0/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:12.699431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17209e72-d3f3-46f7-9976-db4f719c9554 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:12.700619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17209e72-d3f3-46f7-9976-db4f719c9554 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:12.701827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17209e72-d3f3-46f7-9976-db4f719c9554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:12.764440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-17209e72-d3f3-46f7-9976-db4f719c9554 None None] MFA Rules not processed for user `b90d5845757e4dc58e884ca9f59a71f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:12.765809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-17209e72-d3f3-46f7-9976-db4f719c9554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:12.881496 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1178/2336] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:12.885973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf433a95-82f9-4688-bdc0-5fef27d9944a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:13.002273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf433a95-82f9-4688-bdc0-5fef27d9944a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:13.008383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407c8b6-d715-4e9f-b9fb-d7f060129227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:13.009378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407c8b6-d715-4e9f-b9fb-d7f060129227 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:13.009882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407c8b6-d715-4e9f-b9fb-d7f060129227 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:13.010534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407c8b6-d715-4e9f-b9fb-d7f060129227 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:13.011541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407c8b6-d715-4e9f-b9fb-d7f060129227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:13.014155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407c8b6-d715-4e9f-b9fb-d7f060129227 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:13.064946 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1159/2337] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:51:12 2022] POST /identity/v3/projects => generated 389 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:51:13.068666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17209e72-d3f3-46f7-9976-db4f719c9554 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:13.092322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17209e72-d3f3-46f7-9976-db4f719c9554 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:13.217348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53911e7-8c88-407d-bfd8-33dcd238ae03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:13.218679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53911e7-8c88-407d-bfd8-33dcd238ae03 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:13.218679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53911e7-8c88-407d-bfd8-33dcd238ae03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:13.222678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53911e7-8c88-407d-bfd8-33dcd238ae03 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:13.223668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53911e7-8c88-407d-bfd8-33dcd238ae03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:13.231096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53911e7-8c88-407d-bfd8-33dcd238ae03 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:13.264743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:13.270831 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1179/2338] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:51:13 2022] POST /identity/v3/users => generated 301 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:51:13.274685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407c8b6-d715-4e9f-b9fb-d7f060129227 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:13.283657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:13.287841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407c8b6-d715-4e9f-b9fb-d7f060129227 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:13.295492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bc81a-98b8-40d6-b758-b5db5661fc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:13.296770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bc81a-98b8-40d6-b758-b5db5661fc81 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:13.297196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bc81a-98b8-40d6-b758-b5db5661fc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:13.297582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bc81a-98b8-40d6-b758-b5db5661fc81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:13.298357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bc81a-98b8-40d6-b758-b5db5661fc81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:13.300674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bc81a-98b8-40d6-b758-b5db5661fc81 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:13.312976 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1160/2339] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:51:13.316995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53911e7-8c88-407d-bfd8-33dcd238ae03 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:13.329278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53911e7-8c88-407d-bfd8-33dcd238ae03 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:13.335056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f87af5-d905-4db3-9b5f-63dc1e64d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:13.336099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f87af5-d905-4db3-9b5f-63dc1e64d931 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:13.337526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f87af5-d905-4db3-9b5f-63dc1e64d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:13.337831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f87af5-d905-4db3-9b5f-63dc1e64d931 admin admin] PATH_INFO: `/v3/projects/354cd0c497e74e66a0d7284157067a9c/users/e64b5f7e10054320a67b6147553391cd/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:13.359988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f87af5-d905-4db3-9b5f-63dc1e64d931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=354cd0c497e74e66a0d7284157067a9c, user_id=e64b5f7e10054320a67b6147553391cd, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:13.362616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f87af5-d905-4db3-9b5f-63dc1e64d931 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:13.382299 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1180/2340] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:51:13 2022] PUT /identity/v3/projects/354cd0c497e74e66a0d7284157067a9c/users/e64b5f7e10054320a67b6147553391cd/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:13.384012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bc81a-98b8-40d6-b758-b5db5661fc81 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:13.396607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bc81a-98b8-40d6-b758-b5db5661fc81 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:13.562672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5e99b-d2d4-417c-bf23-4094bee06119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:13.564859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5e99b-d2d4-417c-bf23-4094bee06119 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:13.566380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5e99b-d2d4-417c-bf23-4094bee06119 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:13.567291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5e99b-d2d4-417c-bf23-4094bee06119 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:13.568495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5e99b-d2d4-417c-bf23-4094bee06119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:13.572581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5e99b-d2d4-417c-bf23-4094bee06119 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:13.637319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f87af5-d905-4db3-9b5f-63dc1e64d931 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:13.638514 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1161/2341] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:51:13.661884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f87af5-d905-4db3-9b5f-63dc1e64d931 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:13.669874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0353000-7488-4f9a-8a7f-b7f63fd94faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:13.670641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0353000-7488-4f9a-8a7f-b7f63fd94faa admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:13.670918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0353000-7488-4f9a-8a7f-b7f63fd94faa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:13.671482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0353000-7488-4f9a-8a7f-b7f63fd94faa admin admin] PATH_INFO: `/v3/projects/354cd0c497e74e66a0d7284157067a9c/users/e64b5f7e10054320a67b6147553391cd/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:13.673637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0353000-7488-4f9a-8a7f-b7f63fd94faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=354cd0c497e74e66a0d7284157067a9c, user_id=e64b5f7e10054320a67b6147553391cd, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:13.677776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0353000-7488-4f9a-8a7f-b7f63fd94faa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:13.695066 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1181/2342] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:51:13 2022] PUT /identity/v3/projects/354cd0c497e74e66a0d7284157067a9c/users/e64b5f7e10054320a67b6147553391cd/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:13.710417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a00f5b-37e1-48a0-92f8-ff2e08216695 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:13.710702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a00f5b-37e1-48a0-92f8-ff2e08216695 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:13.710980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a00f5b-37e1-48a0-92f8-ff2e08216695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:13.746004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-91a00f5b-37e1-48a0-92f8-ff2e08216695 None None] MFA Rules not processed for user `e64b5f7e10054320a67b6147553391cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:13.747700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-91a00f5b-37e1-48a0-92f8-ff2e08216695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:13.894124 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1162/2343] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:13 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:13.941464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0353000-7488-4f9a-8a7f-b7f63fd94faa admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:13.943959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f0353000-7488-4f9a-8a7f-b7f63fd94faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:14.057243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0353000-7488-4f9a-8a7f-b7f63fd94faa admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:14.063743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0df268-df2a-4f1f-81d5-3a5c6b8926da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:14.064340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0df268-df2a-4f1f-81d5-3a5c6b8926da service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:14.064613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0df268-df2a-4f1f-81d5-3a5c6b8926da service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:14.064878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0df268-df2a-4f1f-81d5-3a5c6b8926da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:14.073562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0df268-df2a-4f1f-81d5-3a5c6b8926da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:14.075098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0df268-df2a-4f1f-81d5-3a5c6b8926da service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:14.085318 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1182/2344] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:51:13 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:51:14.125209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a00f5b-37e1-48a0-92f8-ff2e08216695 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:14.137975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a00f5b-37e1-48a0-92f8-ff2e08216695 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:14.144383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f98f9-1fbe-43bc-8459-9bb946d08afa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:14.145094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f98f9-1fbe-43bc-8459-9bb946d08afa service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:14.145375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f98f9-1fbe-43bc-8459-9bb946d08afa service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:14.145744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f98f9-1fbe-43bc-8459-9bb946d08afa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:14.205717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f98f9-1fbe-43bc-8459-9bb946d08afa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:14.207387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f98f9-1fbe-43bc-8459-9bb946d08afa service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:14.217719 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1163/2345] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:51:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:51:14.236679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0df268-df2a-4f1f-81d5-3a5c6b8926da service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:14.301243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0df268-df2a-4f1f-81d5-3a5c6b8926da service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:14.306196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:14.306967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:14.307241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:14.307509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] PATH_INFO: `/v3/users/b90d5845757e4dc58e884ca9f59a71f0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:14.308634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b90d5845757e4dc58e884ca9f59a71f0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:14.311234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:14.373285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b90d5845757e4dc58e884ca9f59a71f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:14.381336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:14.394760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b90d5845757e4dc58e884ca9f59a71f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:14.403111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b90d5845757e4dc58e884ca9f59a71f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:14.412886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b90d5845757e4dc58e884ca9f59a71f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:14.414143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b90d5845757e4dc58e884ca9f59a71f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:14.427432 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1183/2346] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:51:14 2022] DELETE /identity/v3/users/b90d5845757e4dc58e884ca9f59a71f0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:14.429492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b108-e9b8-4fd5-8576-a9a16f085037 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:14.429941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b108-e9b8-4fd5-8576-a9a16f085037 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:14.430296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b108-e9b8-4fd5-8576-a9a16f085037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:14.452014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f242b108-e9b8-4fd5-8576-a9a16f085037 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:14.453558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f242b108-e9b8-4fd5-8576-a9a16f085037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:14.459856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:14.522574 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1164/2347] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:14.541831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:14.545089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:14.632302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2103b-87b2-4850-8cd7-171811142908 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:14.637087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd89e0-eb9e-4bd8-8020-c58a754fa232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:14.637087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd89e0-eb9e-4bd8-8020-c58a754fa232 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:14.637500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd89e0-eb9e-4bd8-8020-c58a754fa232 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:14.638203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd89e0-eb9e-4bd8-8020-c58a754fa232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:14.648448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd89e0-eb9e-4bd8-8020-c58a754fa232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:14.649928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd89e0-eb9e-4bd8-8020-c58a754fa232 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:14.663872 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1184/2348] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:51:14.979679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242b108-e9b8-4fd5-8576-a9a16f085037 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:14.982197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f242b108-e9b8-4fd5-8576-a9a16f085037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:15.001838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242b108-e9b8-4fd5-8576-a9a16f085037 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:15.016986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:15.018062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:15.018340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:15.018607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] PATH_INFO: `/v3/projects/5545306632c2429a98f479af041c9db9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:15.019862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5545306632c2429a98f479af041c9db9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:15.022063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:15.087490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5545306632c2429a98f479af041c9db9', 'request_id': 'req-cefbffe0-176d-4375-ad74-4e27b87d2b53'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:15.103569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5545306632c2429a98f479af041c9db9', 'request_id': 'req-cefbffe0-176d-4375-ad74-4e27b87d2b53'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:15.115203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5545306632c2429a98f479af041c9db9', 'request_id': 'req-cefbffe0-176d-4375-ad74-4e27b87d2b53'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:15.135988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] The token cache is being invalidate because project 5545306632c2429a98f479af041c9db9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:51:15.136462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:15.137335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cefbffe0-176d-4375-ad74-4e27b87d2b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:15.143258 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1165/2349] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:14 2022] DELETE /identity/v3/projects/5545306632c2429a98f479af041c9db9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:15.143932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd89e0-eb9e-4bd8-8020-c58a754fa232 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:15.157602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd89e0-eb9e-4bd8-8020-c58a754fa232 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:15.238662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:15.239290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:15.239562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:15.239828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] PATH_INFO: `/v3/users/e64b5f7e10054320a67b6147553391cd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:15.240939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e64b5f7e10054320a67b6147553391cd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:15.242876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:15.250924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044845d5-7ede-4fb5-ac28-cd32eb1faa5f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:15.251202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044845d5-7ede-4fb5-ac28-cd32eb1faa5f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:15.251468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044845d5-7ede-4fb5-ac28-cd32eb1faa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:15.271291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-044845d5-7ede-4fb5-ac28-cd32eb1faa5f None None] MFA Rules not processed for user `e32a1c53b4be44188b07c768405961ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:15.272476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-044845d5-7ede-4fb5-ac28-cd32eb1faa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:15.293019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e64b5f7e10054320a67b6147553391cd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:15.306167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e64b5f7e10054320a67b6147553391cd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:15.314294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e64b5f7e10054320a67b6147553391cd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:15.315821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:15.324283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e64b5f7e10054320a67b6147553391cd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:15.325472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c3743d6-4f01-4c2b-b9b0-3a49f6a50d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e64b5f7e10054320a67b6147553391cd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:15.331320 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1166/2350] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:51:15.331697 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1185/2351] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:51:15 2022] DELETE /identity/v3/users/e64b5f7e10054320a67b6147553391cd => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:15.842174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044845d5-7ede-4fb5-ac28-cd32eb1faa5f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:15.845267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-044845d5-7ede-4fb5-ac28-cd32eb1faa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:15.910828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044845d5-7ede-4fb5-ac28-cd32eb1faa5f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:16.007017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:16.070354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:16.072641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:16.073180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:16.073650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] PATH_INFO: `/v3/projects/354cd0c497e74e66a0d7284157067a9c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:16.078226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=354cd0c497e74e66a0d7284157067a9c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:16.080773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:16.165268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354cd0c497e74e66a0d7284157067a9c', 'request_id': 'req-9ee1a2a9-6b29-4166-9909-d5bff00f1924'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:16.180647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354cd0c497e74e66a0d7284157067a9c', 'request_id': 'req-9ee1a2a9-6b29-4166-9909-d5bff00f1924'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:16.194942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354cd0c497e74e66a0d7284157067a9c', 'request_id': 'req-9ee1a2a9-6b29-4166-9909-d5bff00f1924'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:16.232586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] The token cache is being invalidate because project 354cd0c497e74e66a0d7284157067a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:51:16.232861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:16.234211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ee1a2a9-6b29-4166-9909-d5bff00f1924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:16.241801 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1167/2352] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:15 2022] DELETE /identity/v3/projects/354cd0c497e74e66a0d7284157067a9c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:16.242973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723f0e7-b134-43dc-a701-d6e958bc9b43 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:16.243419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723f0e7-b134-43dc-a701-d6e958bc9b43 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:16.243520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723f0e7-b134-43dc-a701-d6e958bc9b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:16.269743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4723f0e7-b134-43dc-a701-d6e958bc9b43 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:16.271271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4723f0e7-b134-43dc-a701-d6e958bc9b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:16.349778 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1186/2353] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:51:16.357984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd5e96-336d-4152-a087-d0aca3217080 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:16.358187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd5e96-336d-4152-a087-d0aca3217080 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:16.358549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd5e96-336d-4152-a087-d0aca3217080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:16.387925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-78bd5e96-336d-4152-a087-d0aca3217080 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:16.389526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-78bd5e96-336d-4152-a087-d0aca3217080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:16.403494 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1168/2354] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:16.403989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723f0e7-b134-43dc-a701-d6e958bc9b43 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:16.414123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723f0e7-b134-43dc-a701-d6e958bc9b43 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:16.420610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d870-bbba-4e5d-86cd-49cf497a0dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:16.421291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d870-bbba-4e5d-86cd-49cf497a0dca admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:16.421563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d870-bbba-4e5d-86cd-49cf497a0dca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:16.421826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d870-bbba-4e5d-86cd-49cf497a0dca admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:16.423014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776d870-bbba-4e5d-86cd-49cf497a0dca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:51:16.423378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776d870-bbba-4e5d-86cd-49cf497a0dca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:16.424995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776d870-bbba-4e5d-86cd-49cf497a0dca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:16.439899 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1187/2355] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:51:16.446121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd5e96-336d-4152-a087-d0aca3217080 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:16.461348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd5e96-336d-4152-a087-d0aca3217080 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:16.469219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74389-2f64-4ab6-b936-b2b185e4ee60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:16.470097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74389-2f64-4ab6-b936-b2b185e4ee60 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:16.470375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74389-2f64-4ab6-b936-b2b185e4ee60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:16.471166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74389-2f64-4ab6-b936-b2b185e4ee60 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:16.471871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e74389-2f64-4ab6-b936-b2b185e4ee60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:16.474058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e74389-2f64-4ab6-b936-b2b185e4ee60 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:16.514659 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1169/2356] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:51:16 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:51:16.519244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d870-bbba-4e5d-86cd-49cf497a0dca admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:16.530988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d870-bbba-4e5d-86cd-49cf497a0dca admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:16.639511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e68dde-70b0-44df-9b33-24f64b941b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:16.640393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e68dde-70b0-44df-9b33-24f64b941b38 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:16.640857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e68dde-70b0-44df-9b33-24f64b941b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:16.641309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e68dde-70b0-44df-9b33-24f64b941b38 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:16.642091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e68dde-70b0-44df-9b33-24f64b941b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:16.644771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e68dde-70b0-44df-9b33-24f64b941b38 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:16.680654 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1188/2357] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:51:16 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:51:16.681912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74389-2f64-4ab6-b936-b2b185e4ee60 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:16.694826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74389-2f64-4ab6-b936-b2b185e4ee60 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:16.703735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9266ba2-52ff-41f7-93e3-9d16985bab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:16.703735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9266ba2-52ff-41f7-93e3-9d16985bab1e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:16.703735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9266ba2-52ff-41f7-93e3-9d16985bab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:16.703735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9266ba2-52ff-41f7-93e3-9d16985bab1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:16.704208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9266ba2-52ff-41f7-93e3-9d16985bab1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:16.705094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9266ba2-52ff-41f7-93e3-9d16985bab1e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:16.720432 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1170/2358] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:51:16.722497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e68dde-70b0-44df-9b33-24f64b941b38 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:16.733852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e68dde-70b0-44df-9b33-24f64b941b38 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:16.739615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942b15-1606-4b19-a3ae-14ec393756fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:16.740306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942b15-1606-4b19-a3ae-14ec393756fe admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:16.740588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942b15-1606-4b19-a3ae-14ec393756fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:16.740999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942b15-1606-4b19-a3ae-14ec393756fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:16.741694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c942b15-1606-4b19-a3ae-14ec393756fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:16.743401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c942b15-1606-4b19-a3ae-14ec393756fe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:16.768389 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1189/2359] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:51:16.770230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9266ba2-52ff-41f7-93e3-9d16985bab1e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:16.792597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9266ba2-52ff-41f7-93e3-9d16985bab1e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:16.797440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d52338-c794-4edf-b9df-8647a399a9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:16.798092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d52338-c794-4edf-b9df-8647a399a9f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:16.798424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d52338-c794-4edf-b9df-8647a399a9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:16.798706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d52338-c794-4edf-b9df-8647a399a9f5 admin admin] PATH_INFO: `/v3/projects/e5c50f30f706437ea737a3d5d20ea8a1/users/3e76b25814474b9db035954b3a437c96/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:16.816933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d52338-c794-4edf-b9df-8647a399a9f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c50f30f706437ea737a3d5d20ea8a1, user_id=3e76b25814474b9db035954b3a437c96, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:16.820188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d52338-c794-4edf-b9df-8647a399a9f5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:16.849108 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1171/2360] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:51:16 2022] PUT /identity/v3/projects/e5c50f30f706437ea737a3d5d20ea8a1/users/3e76b25814474b9db035954b3a437c96/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:16.858026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501d645-1bb7-4eaf-b5d6-96450e095c31 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:16.859221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501d645-1bb7-4eaf-b5d6-96450e095c31 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:16.859632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501d645-1bb7-4eaf-b5d6-96450e095c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:16.897844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2501d645-1bb7-4eaf-b5d6-96450e095c31 None None] MFA Rules not processed for user `3e76b25814474b9db035954b3a437c96`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:16.901955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2501d645-1bb7-4eaf-b5d6-96450e095c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:17.062285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:17.109381 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1190/2361] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:17.119642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d52338-c794-4edf-b9df-8647a399a9f5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:17.121718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b2d52338-c794-4edf-b9df-8647a399a9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:17.261423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d52338-c794-4edf-b9df-8647a399a9f5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:17.266832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec1593-83f9-42af-b117-5abf639ff1f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:17.267691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec1593-83f9-42af-b117-5abf639ff1f9 service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:17.268213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec1593-83f9-42af-b117-5abf639ff1f9 service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:17.268751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec1593-83f9-42af-b117-5abf639ff1f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:17.279513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec1593-83f9-42af-b117-5abf639ff1f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:17.281145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec1593-83f9-42af-b117-5abf639ff1f9 service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:17.292066 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1172/2362] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:51:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:51:18.472274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3ab71-b676-4c29-8b72-d04ab8730f4a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:18.472274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3ab71-b676-4c29-8b72-d04ab8730f4a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:18.472854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3ab71-b676-4c29-8b72-d04ab8730f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:18.514452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-92c3ab71-b676-4c29-8b72-d04ab8730f4a None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:18.516239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-92c3ab71-b676-4c29-8b72-d04ab8730f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:18.677253 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1191/2363] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:51:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:18.680612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec1593-83f9-42af-b117-5abf639ff1f9 service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:18.682477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2fec1593-83f9-42af-b117-5abf639ff1f9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:18.701080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:18.839472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec1593-83f9-42af-b117-5abf639ff1f9 service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:18.852922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed2db3-1edb-45a4-9838-f47456097f42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:18.855121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed2db3-1edb-45a4-9838-f47456097f42 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:18.857321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed2db3-1edb-45a4-9838-f47456097f42 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:18.857838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed2db3-1edb-45a4-9838-f47456097f42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:18.895897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed2db3-1edb-45a4-9838-f47456097f42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:18.897963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed2db3-1edb-45a4-9838-f47456097f42 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:18.911347 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1173/2364] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:51:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:20.662269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7146094-baee-4d92-928e-1b8f15f1a5d0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:20.662819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7146094-baee-4d92-928e-1b8f15f1a5d0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:20.662819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7146094-baee-4d92-928e-1b8f15f1a5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:20.683376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f7146094-baee-4d92-928e-1b8f15f1a5d0 None None] MFA Rules not processed for user `180d9b7141ff4792ab7ab19e21a3d7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:20.685216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f7146094-baee-4d92-928e-1b8f15f1a5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:20.771438 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1192/2365] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:20 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:21.417932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5f868-9e1d-4dce-8e62-370f309087fe None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:21.418411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5f868-9e1d-4dce-8e62-370f309087fe None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:21.419414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5f868-9e1d-4dce-8e62-370f309087fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:21.440998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d7d5f868-9e1d-4dce-8e62-370f309087fe None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:21.443000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d7d5f868-9e1d-4dce-8e62-370f309087fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:21.451879 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1174/2366] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:51:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:21.467643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7146094-baee-4d92-928e-1b8f15f1a5d0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:21.487268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7146094-baee-4d92-928e-1b8f15f1a5d0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:21.496008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721ea7a-bb3d-4ca8-8642-b51c8158c35f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:21.497298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721ea7a-bb3d-4ca8-8642-b51c8158c35f service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:21.497731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721ea7a-bb3d-4ca8-8642-b51c8158c35f service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:21.498156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721ea7a-bb3d-4ca8-8642-b51c8158c35f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:21.514985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721ea7a-bb3d-4ca8-8642-b51c8158c35f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:21.517900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721ea7a-bb3d-4ca8-8642-b51c8158c35f service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:21.532615 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1193/2367] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:51:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:23.480752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b43ac-7afc-4eda-ba88-0d09b7dc4a5a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:23.481418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b43ac-7afc-4eda-ba88-0d09b7dc4a5a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:23.481418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b43ac-7afc-4eda-ba88-0d09b7dc4a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:23.521896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a22b43ac-7afc-4eda-ba88-0d09b7dc4a5a None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:23.523671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a22b43ac-7afc-4eda-ba88-0d09b7dc4a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:23.566550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1175/2368] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:51:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:23.567348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721ea7a-bb3d-4ca8-8642-b51c8158c35f service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:23.580497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721ea7a-bb3d-4ca8-8642-b51c8158c35f service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:23.598825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8cc1c-1f8f-47ef-961b-0a134bd48d16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:23.599753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8cc1c-1f8f-47ef-961b-0a134bd48d16 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:23.600190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8cc1c-1f8f-47ef-961b-0a134bd48d16 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:23.601326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8cc1c-1f8f-47ef-961b-0a134bd48d16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:23.638698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8cc1c-1f8f-47ef-961b-0a134bd48d16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:23.644794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8cc1c-1f8f-47ef-961b-0a134bd48d16 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:23.663137 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1194/2369] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:51:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:23.977482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c055c-2fa0-45f4-89ed-bf3a6abe2c8e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:23.977789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c055c-2fa0-45f4-89ed-bf3a6abe2c8e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:23.978052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c055c-2fa0-45f4-89ed-bf3a6abe2c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:24.010567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-902c055c-2fa0-45f4-89ed-bf3a6abe2c8e None None] MFA Rules not processed for user `c613201f55b94a528c6c36e5b5772799`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:24.011773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-902c055c-2fa0-45f4-89ed-bf3a6abe2c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:24.118762 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1176/2370] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:24.865303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81053e-b2d0-4ba7-bf67-5e8e6ba33daf None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:24.865804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81053e-b2d0-4ba7-bf67-5e8e6ba33daf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:24.865889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81053e-b2d0-4ba7-bf67-5e8e6ba33daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:24.888946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6f81053e-b2d0-4ba7-bf67-5e8e6ba33daf None None] MFA Rules not processed for user `3e76b25814474b9db035954b3a437c96`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:24.890345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f81053e-b2d0-4ba7-bf67-5e8e6ba33daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:24.910434 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1195/2371] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:25.617232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8029665-88ba-4c65-96d3-0f613e05a538 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:25.618052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8029665-88ba-4c65-96d3-0f613e05a538 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:25.618712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8029665-88ba-4c65-96d3-0f613e05a538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:25.648303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d8029665-88ba-4c65-96d3-0f613e05a538 None None] MFA Rules not processed for user `3e76b25814474b9db035954b3a437c96`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:25.649619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8029665-88ba-4c65-96d3-0f613e05a538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:25.663860 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1177/2372] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:32.413630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b659ce-a853-4d04-a5da-1f60af5cb7fb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:32.414540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b659ce-a853-4d04-a5da-1f60af5cb7fb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:32.415021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b659ce-a853-4d04-a5da-1f60af5cb7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:32.440312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-32b659ce-a853-4d04-a5da-1f60af5cb7fb None None] MFA Rules not processed for user `1ddcfe1b0eb5431dbbe4777b8dd77436`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:32.442171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-32b659ce-a853-4d04-a5da-1f60af5cb7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:32.508112 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1196/2373] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:51:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:35.641356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=62 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:36.659519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0880a4b-4e10-4bb1-9443-be902f5d2de4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:36.660129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0880a4b-4e10-4bb1-9443-be902f5d2de4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:36.660129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0880a4b-4e10-4bb1-9443-be902f5d2de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:36.702696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c0880a4b-4e10-4bb1-9443-be902f5d2de4 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:36.702696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c0880a4b-4e10-4bb1-9443-be902f5d2de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:36.705136 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1178/2374] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:51:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:36.725731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b659ce-a853-4d04-a5da-1f60af5cb7fb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:36.739076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b659ce-a853-4d04-a5da-1f60af5cb7fb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:36.744398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b23be2-c225-4737-a33b-303634b262cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:36.745223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b23be2-c225-4737-a33b-303634b262cb service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:36.745794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b23be2-c225-4737-a33b-303634b262cb service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:36.746347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b23be2-c225-4737-a33b-303634b262cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:36.753698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:36.759054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b23be2-c225-4737-a33b-303634b262cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:36.761740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b23be2-c225-4737-a33b-303634b262cb service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:36.796258 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1197/2375] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:51:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:46.683379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:46.769071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:47.174126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1499a3-0205-4f89-9c21-c6cd743b6afd None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:47.174405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1499a3-0205-4f89-9c21-c6cd743b6afd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:47.174740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1499a3-0205-4f89-9c21-c6cd743b6afd None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:47.176500 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1179/2376] 10.222.0.16 () {54 vars in 921 bytes} [Sat Oct 15 20:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:51:47.190525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cd986-8741-4fd6-bf7e-297f1c3bfd3b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:47.191052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cd986-8741-4fd6-bf7e-297f1c3bfd3b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:47.191479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cd986-8741-4fd6-bf7e-297f1c3bfd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:47.235907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-527cd986-8741-4fd6-bf7e-297f1c3bfd3b None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:51:47.237412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-527cd986-8741-4fd6-bf7e-297f1c3bfd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:47.351625 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1198/2377] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 20:51:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:51:47.701081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1499a3-0205-4f89-9c21-c6cd743b6afd None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:47.704021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7b1499a3-0205-4f89-9c21-c6cd743b6afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:47.759182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1499a3-0205-4f89-9c21-c6cd743b6afd None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:47.769335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d647d4-a471-4090-9aae-6ee5dee11033 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:47.770299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d647d4-a471-4090-9aae-6ee5dee11033 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:47.771035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d647d4-a471-4090-9aae-6ee5dee11033 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:47.771465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d647d4-a471-4090-9aae-6ee5dee11033 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:47.821512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d647d4-a471-4090-9aae-6ee5dee11033 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:47.823768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d647d4-a471-4090-9aae-6ee5dee11033 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:47.846163 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1180/2378] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:51:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:51:57.337252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=29 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:57.841764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:58.730158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cd986-8741-4fd6-bf7e-297f1c3bfd3b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:58.731641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-527cd986-8741-4fd6-bf7e-297f1c3bfd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:58.805200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cd986-8741-4fd6-bf7e-297f1c3bfd3b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:58.810110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:58.810769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:58.811041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:58.811308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] PATH_INFO: `/v3/users/14bd717b05ce4e40a1cd54c2db8474a6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:58.812420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14bd717b05ce4e40a1cd54c2db8474a6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:58.814381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:58.875459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14bd717b05ce4e40a1cd54c2db8474a6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:58.894160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14bd717b05ce4e40a1cd54c2db8474a6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:58.904894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14bd717b05ce4e40a1cd54c2db8474a6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:58.914402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14bd717b05ce4e40a1cd54c2db8474a6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:58.915117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14bd717b05ce4e40a1cd54c2db8474a6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:58.923251 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1199/2379] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:51:58 2022] DELETE /identity/v3/users/14bd717b05ce4e40a1cd54c2db8474a6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:59.548739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d647d4-a471-4090-9aae-6ee5dee11033 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:59.574438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d647d4-a471-4090-9aae-6ee5dee11033 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:59.680208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:59.681168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:59.682335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:59.682715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] PATH_INFO: `/v3/projects/f856d315b3b84bdcac94b6d300ef8b2b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:59.685817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f856d315b3b84bdcac94b6d300ef8b2b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:59.687902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:51:59.754251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f856d315b3b84bdcac94b6d300ef8b2b', 'request_id': 'req-0565d1fe-9dcb-421b-bb20-0096b09f4864'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:59.766514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f856d315b3b84bdcac94b6d300ef8b2b', 'request_id': 'req-0565d1fe-9dcb-421b-bb20-0096b09f4864'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:59.776136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f856d315b3b84bdcac94b6d300ef8b2b', 'request_id': 'req-0565d1fe-9dcb-421b-bb20-0096b09f4864'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:59.818541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] The token cache is being invalidate because project f856d315b3b84bdcac94b6d300ef8b2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:51:59.818541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:59.818541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0565d1fe-9dcb-421b-bb20-0096b09f4864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:51:59.823870 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1181/2380] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:59 2022] DELETE /identity/v3/projects/f856d315b3b84bdcac94b6d300ef8b2b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:51:59.827833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:51:59.830457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:51:59.847728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba07c72-8de3-4ea1-8984-b22d61b85959 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:51:59.977592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:51:59.979391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:51:59.979857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:51:59.980407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] PATH_INFO: `/v3/users/b8343eb55cd148ef94141af734ef6abf` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:51:59.982715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8343eb55cd148ef94141af734ef6abf)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:51:59.989545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:00.033381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:00.115661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8343eb55cd148ef94141af734ef6abf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:00.152970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8343eb55cd148ef94141af734ef6abf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:00.175348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8343eb55cd148ef94141af734ef6abf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:00.189007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8343eb55cd148ef94141af734ef6abf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:00.190391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8343eb55cd148ef94141af734ef6abf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:00.212045 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1200/2381] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:51:59 2022] DELETE /identity/v3/users/b8343eb55cd148ef94141af734ef6abf => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:52:00.292020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aea8e7-5f84-46e8-970e-15b4e18c591c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:00.292305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aea8e7-5f84-46e8-970e-15b4e18c591c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:00.292593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aea8e7-5f84-46e8-970e-15b4e18c591c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:00.334576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a6aea8e7-5f84-46e8-970e-15b4e18c591c None None] MFA Rules not processed for user `1ddcfe1b0eb5431dbbe4777b8dd77436`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:00.336094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a6aea8e7-5f84-46e8-970e-15b4e18c591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:00.496941 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1182/2382] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:00.515016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:00.517368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:00.696287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ccbf0-8793-4e23-8ced-ecb8373dee5f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:00.716642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:00.717823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:00.718561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:00.719031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] PATH_INFO: `/v3/projects/e43279179c3b432cbf49db0581d3b8cc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:00.725342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43279179c3b432cbf49db0581d3b8cc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:00.731289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:00.799376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43279179c3b432cbf49db0581d3b8cc', 'request_id': 'req-39a5ff6b-a561-4904-9bc1-efc50e263ca9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:00.812684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43279179c3b432cbf49db0581d3b8cc', 'request_id': 'req-39a5ff6b-a561-4904-9bc1-efc50e263ca9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:00.833151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43279179c3b432cbf49db0581d3b8cc', 'request_id': 'req-39a5ff6b-a561-4904-9bc1-efc50e263ca9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:00.850375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] The token cache is being invalidate because project e43279179c3b432cbf49db0581d3b8cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:00.850907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:00.851813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39a5ff6b-a561-4904-9bc1-efc50e263ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:00.855740 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1201/2383] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:00 2022] DELETE /identity/v3/projects/e43279179c3b432cbf49db0581d3b8cc => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:00.861356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe628195-7f13-435c-8207-64bba123e7af None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:00.861853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe628195-7f13-435c-8207-64bba123e7af None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:00.861937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe628195-7f13-435c-8207-64bba123e7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:00.896877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fe628195-7f13-435c-8207-64bba123e7af None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:00.898768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fe628195-7f13-435c-8207-64bba123e7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:00.962084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:01.008948 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1183/2384] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:01.018186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f9c9b-2c5d-4262-ba99-64ae3abbc7a1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:01.018599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f9c9b-2c5d-4262-ba99-64ae3abbc7a1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:01.018967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f9c9b-2c5d-4262-ba99-64ae3abbc7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:01.050000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-402f9c9b-2c5d-4262-ba99-64ae3abbc7a1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:01.062002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-402f9c9b-2c5d-4262-ba99-64ae3abbc7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:01.076541 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1202/2385] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:01.077868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe628195-7f13-435c-8207-64bba123e7af None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:01.088698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe628195-7f13-435c-8207-64bba123e7af None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:01.096157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88d423-415a-41c7-8d65-48cbf065b657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:01.097088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88d423-415a-41c7-8d65-48cbf065b657 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:01.097587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88d423-415a-41c7-8d65-48cbf065b657 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:01.098124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88d423-415a-41c7-8d65-48cbf065b657 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:01.099104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88d423-415a-41c7-8d65-48cbf065b657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:52:01.099625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88d423-415a-41c7-8d65-48cbf065b657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:01.101891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88d423-415a-41c7-8d65-48cbf065b657 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:01.118432 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1184/2386] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:01.122347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f9c9b-2c5d-4262-ba99-64ae3abbc7a1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:01.132168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f9c9b-2c5d-4262-ba99-64ae3abbc7a1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:01.139264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539218e-b8d0-4854-8a45-cac1fccd4de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:01.139784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539218e-b8d0-4854-8a45-cac1fccd4de8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:01.140065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539218e-b8d0-4854-8a45-cac1fccd4de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:01.140331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539218e-b8d0-4854-8a45-cac1fccd4de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:01.141126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539218e-b8d0-4854-8a45-cac1fccd4de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:01.143907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539218e-b8d0-4854-8a45-cac1fccd4de8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:01.215721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:01.240316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88d423-415a-41c7-8d65-48cbf065b657 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:01.250666 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1203/2387] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:01 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:52:01.252736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88d423-415a-41c7-8d65-48cbf065b657 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:01.270535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539218e-b8d0-4854-8a45-cac1fccd4de8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:01.274154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f539218e-b8d0-4854-8a45-cac1fccd4de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:01.379131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539218e-b8d0-4854-8a45-cac1fccd4de8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:01.386807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:01.387508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:01.388706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:01.388706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] PATH_INFO: `/v3/users/03bc9235fe1c4fc1b3097a913cbe1052` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:01.390771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03bc9235fe1c4fc1b3097a913cbe1052)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:01.394610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:01.441870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d441a05-8215-4d4c-a7ff-1a9786f16596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:01.443191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d441a05-8215-4d4c-a7ff-1a9786f16596 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:01.443722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d441a05-8215-4d4c-a7ff-1a9786f16596 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:01.444213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d441a05-8215-4d4c-a7ff-1a9786f16596 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:01.445498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d441a05-8215-4d4c-a7ff-1a9786f16596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:01.449702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d441a05-8215-4d4c-a7ff-1a9786f16596 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:01.492569 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1185/2388] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:01 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:52:01.499863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d441a05-8215-4d4c-a7ff-1a9786f16596 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:01.522865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d441a05-8215-4d4c-a7ff-1a9786f16596 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:01.529774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200e01a-99fd-40a1-aa94-405c3a73c49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:01.532776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200e01a-99fd-40a1-aa94-405c3a73c49c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:01.533466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200e01a-99fd-40a1-aa94-405c3a73c49c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:01.534050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200e01a-99fd-40a1-aa94-405c3a73c49c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:01.535869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200e01a-99fd-40a1-aa94-405c3a73c49c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:01.537904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200e01a-99fd-40a1-aa94-405c3a73c49c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:01.554402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03bc9235fe1c4fc1b3097a913cbe1052'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:01.570116 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1186/2389] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:01.574773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200e01a-99fd-40a1-aa94-405c3a73c49c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:01.590275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03bc9235fe1c4fc1b3097a913cbe1052'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:01.602635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200e01a-99fd-40a1-aa94-405c3a73c49c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:01.603129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03bc9235fe1c4fc1b3097a913cbe1052'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:01.611474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd1646-3a4f-411d-bf98-8c3cb5cc6394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:01.613428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd1646-3a4f-411d-bf98-8c3cb5cc6394 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:01.613428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd1646-3a4f-411d-bf98-8c3cb5cc6394 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:01.613428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd1646-3a4f-411d-bf98-8c3cb5cc6394 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:01.615255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd1646-3a4f-411d-bf98-8c3cb5cc6394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:01.616082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd1646-3a4f-411d-bf98-8c3cb5cc6394 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:01.618856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03bc9235fe1c4fc1b3097a913cbe1052'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:01.619718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03bc9235fe1c4fc1b3097a913cbe1052'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:01.629583 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1204/2390] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:01 2022] DELETE /identity/v3/users/03bc9235fe1c4fc1b3097a913cbe1052 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:52:01.630576 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1187/2391] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:01.638642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:01.640290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:01.790963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d711c-635a-47c7-b312-6ec4a67c8027 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:01.811271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b4707-9d19-4a10-be81-41ad553bcef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:01.813449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b4707-9d19-4a10-be81-41ad553bcef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:01.813587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b4707-9d19-4a10-be81-41ad553bcef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:01.814290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b4707-9d19-4a10-be81-41ad553bcef2 admin admin] PATH_INFO: `/v3/projects/6d53a952b9a04786a687eb543c8615c3/users/10843cee66164ad294a67be5dc5c3ff1/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:01.860912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b4707-9d19-4a10-be81-41ad553bcef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d53a952b9a04786a687eb543c8615c3, user_id=10843cee66164ad294a67be5dc5c3ff1, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:01.864179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b4707-9d19-4a10-be81-41ad553bcef2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:01.896949 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1205/2392] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:01 2022] PUT /identity/v3/projects/6d53a952b9a04786a687eb543c8615c3/users/10843cee66164ad294a67be5dc5c3ff1/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:01.908228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa2aa5-eddb-4407-b945-ab7a2ef64892 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:01.911065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa2aa5-eddb-4407-b945-ab7a2ef64892 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:01.911536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa2aa5-eddb-4407-b945-ab7a2ef64892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:01.956457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-eeaa2aa5-eddb-4407-b945-ab7a2ef64892 None None] MFA Rules not processed for user `10843cee66164ad294a67be5dc5c3ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:01.958254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eeaa2aa5-eddb-4407-b945-ab7a2ef64892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:01.975298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b4707-9d19-4a10-be81-41ad553bcef2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:01.977297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-156b4707-9d19-4a10-be81-41ad553bcef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:01.984813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:02.067715 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1188/2393] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:02.071901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa2aa5-eddb-4407-b945-ab7a2ef64892 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.074695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b4707-9d19-4a10-be81-41ad553bcef2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.080112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.081651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.081942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.082288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] PATH_INFO: `/v3/projects/869f1ee4c91c4067a3e32e39155b30a6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.083843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=869f1ee4c91c4067a3e32e39155b30a6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.085825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.089166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa2aa5-eddb-4407-b945-ab7a2ef64892 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.097052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120c6db-47eb-4f9f-b7c0-21dfe6fd9ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.097787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120c6db-47eb-4f9f-b7c0-21dfe6fd9ebb admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.098129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120c6db-47eb-4f9f-b7c0-21dfe6fd9ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.098512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120c6db-47eb-4f9f-b7c0-21dfe6fd9ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.099369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120c6db-47eb-4f9f-b7c0-21dfe6fd9ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.101673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120c6db-47eb-4f9f-b7c0-21dfe6fd9ebb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.127658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:02.141779 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1189/2394] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:02 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:02.144567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120c6db-47eb-4f9f-b7c0-21dfe6fd9ebb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.152355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '869f1ee4c91c4067a3e32e39155b30a6', 'request_id': 'req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:02.162661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '869f1ee4c91c4067a3e32e39155b30a6', 'request_id': 'req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:02.172150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '869f1ee4c91c4067a3e32e39155b30a6', 'request_id': 'req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:02.185004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] The token cache is being invalidate because project 869f1ee4c91c4067a3e32e39155b30a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:02.185439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:02.186179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5c0ae1a1-18c7-48f8-adca-3aba9a7cc400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:02.187799 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1206/2395] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:01 2022] DELETE /identity/v3/projects/869f1ee4c91c4067a3e32e39155b30a6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:02.193852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874efa4-6c17-4200-9418-05b6e03bb499 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.194133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874efa4-6c17-4200-9418-05b6e03bb499 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.194414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874efa4-6c17-4200-9418-05b6e03bb499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.221285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1874efa4-6c17-4200-9418-05b6e03bb499 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:02.222524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1874efa4-6c17-4200-9418-05b6e03bb499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:02.233395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120c6db-47eb-4f9f-b7c0-21dfe6fd9ebb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.236128 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1207/2396] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:02.245638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1e2dc-1989-44d1-92ac-2a06bca3e4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.246656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e2dc-1989-44d1-92ac-2a06bca3e4be admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.247064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e2dc-1989-44d1-92ac-2a06bca3e4be admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.247313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e2dc-1989-44d1-92ac-2a06bca3e4be admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.248603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1e2dc-1989-44d1-92ac-2a06bca3e4be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.251339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1e2dc-1989-44d1-92ac-2a06bca3e4be admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.303681 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1190/2397] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:02 2022] POST /identity/v3/users => generated 300 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:02.305205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874efa4-6c17-4200-9418-05b6e03bb499 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.307991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1874efa4-6c17-4200-9418-05b6e03bb499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:02.313260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d95f27-63b2-459e-a10f-e75021384f11 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.313790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d95f27-63b2-459e-a10f-e75021384f11 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.314583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d95f27-63b2-459e-a10f-e75021384f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.323965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874efa4-6c17-4200-9418-05b6e03bb499 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.329676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94af131-72ab-4833-90c6-de0c77c51cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.330282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94af131-72ab-4833-90c6-de0c77c51cfd admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.330560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94af131-72ab-4833-90c6-de0c77c51cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.330829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94af131-72ab-4833-90c6-de0c77c51cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.334343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94af131-72ab-4833-90c6-de0c77c51cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.336089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94af131-72ab-4833-90c6-de0c77c51cfd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.342655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a7d95f27-63b2-459e-a10f-e75021384f11 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:02.344496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a7d95f27-63b2-459e-a10f-e75021384f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:02.349078 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1208/2398] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:02.352837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94af131-72ab-4833-90c6-de0c77c51cfd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.355722 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1191/2399] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:02.359973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d95f27-63b2-459e-a10f-e75021384f11 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.372983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94af131-72ab-4833-90c6-de0c77c51cfd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.376968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d95f27-63b2-459e-a10f-e75021384f11 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.378294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d938-c589-4a7a-b774-f3a979608c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.378951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d938-c589-4a7a-b774-f3a979608c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.379373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d938-c589-4a7a-b774-f3a979608c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.379651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d938-c589-4a7a-b774-f3a979608c9e admin admin] PATH_INFO: `/v3/projects/8f6ce97c7b4b4778a6ad48e3566146a2/users/5246ab457b9a4086bd667e7436ab2951/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.382692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455a729-ab7c-48a2-9d1c-f133c0c4ae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.383473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455a729-ab7c-48a2-9d1c-f133c0c4ae47 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.383811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455a729-ab7c-48a2-9d1c-f133c0c4ae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.384500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455a729-ab7c-48a2-9d1c-f133c0c4ae47 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.384922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455a729-ab7c-48a2-9d1c-f133c0c4ae47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:52:02.385400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455a729-ab7c-48a2-9d1c-f133c0c4ae47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.387225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455a729-ab7c-48a2-9d1c-f133c0c4ae47 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.406036 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1192/2400] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:02.408422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455a729-ab7c-48a2-9d1c-f133c0c4ae47 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.410329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4d938-c589-4a7a-b774-f3a979608c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6ce97c7b4b4778a6ad48e3566146a2, user_id=5246ab457b9a4086bd667e7436ab2951, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.413458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4d938-c589-4a7a-b774-f3a979608c9e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.429271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455a729-ab7c-48a2-9d1c-f133c0c4ae47 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.429938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1209/2401] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:02 2022] PUT /identity/v3/projects/8f6ce97c7b4b4778a6ad48e3566146a2/users/5246ab457b9a4086bd667e7436ab2951/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:52:02.433976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d938-c589-4a7a-b774-f3a979608c9e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.505719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:02.519869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf17c2b-4afa-4277-ad5b-cb9b74b7394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.520871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf17c2b-4afa-4277-ad5b-cb9b74b7394b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.521187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf17c2b-4afa-4277-ad5b-cb9b74b7394b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.522541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf17c2b-4afa-4277-ad5b-cb9b74b7394b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.523483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf17c2b-4afa-4277-ad5b-cb9b74b7394b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.525266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d938-c589-4a7a-b774-f3a979608c9e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.526605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf17c2b-4afa-4277-ad5b-cb9b74b7394b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.531334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c53e1-e44e-4fbe-b1c1-7018e43a2d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.532424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c53e1-e44e-4fbe-b1c1-7018e43a2d89 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.532640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c53e1-e44e-4fbe-b1c1-7018e43a2d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.533003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c53e1-e44e-4fbe-b1c1-7018e43a2d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.533743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c53e1-e44e-4fbe-b1c1-7018e43a2d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.535447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c53e1-e44e-4fbe-b1c1-7018e43a2d89 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.547341 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1210/2402] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:02.551570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c53e1-e44e-4fbe-b1c1-7018e43a2d89 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.564224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c53e1-e44e-4fbe-b1c1-7018e43a2d89 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.569461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5418c84-d2e7-4196-b002-4a85ceb38165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.570178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5418c84-d2e7-4196-b002-4a85ceb38165 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.570656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5418c84-d2e7-4196-b002-4a85ceb38165 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.570906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5418c84-d2e7-4196-b002-4a85ceb38165 admin admin] PATH_INFO: `/v3/projects/8f6ce97c7b4b4778a6ad48e3566146a2/users/5246ab457b9a4086bd667e7436ab2951/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.572948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5418c84-d2e7-4196-b002-4a85ceb38165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6ce97c7b4b4778a6ad48e3566146a2, user_id=5246ab457b9a4086bd667e7436ab2951, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.573608 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1193/2403] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:02 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:02.576077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5418c84-d2e7-4196-b002-4a85ceb38165 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.578206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf17c2b-4afa-4277-ad5b-cb9b74b7394b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.590685 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1211/2404] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:02 2022] PUT /identity/v3/projects/8f6ce97c7b4b4778a6ad48e3566146a2/users/5246ab457b9a4086bd667e7436ab2951/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:02.602449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c505b-b0f6-4ac6-9b7c-499c06e6f21b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.602794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c505b-b0f6-4ac6-9b7c-499c06e6f21b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.603102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c505b-b0f6-4ac6-9b7c-499c06e6f21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.670682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-461c505b-b0f6-4ac6-9b7c-499c06e6f21b None None] MFA Rules not processed for user `5246ab457b9a4086bd667e7436ab2951`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:02.672322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-461c505b-b0f6-4ac6-9b7c-499c06e6f21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:02.674199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:02.688612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf17c2b-4afa-4277-ad5b-cb9b74b7394b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.754115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999e8f0-3a13-411e-8814-2d75dafe9f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.754731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999e8f0-3a13-411e-8814-2d75dafe9f15 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.755007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999e8f0-3a13-411e-8814-2d75dafe9f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.755321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999e8f0-3a13-411e-8814-2d75dafe9f15 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.755962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999e8f0-3a13-411e-8814-2d75dafe9f15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.757897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999e8f0-3a13-411e-8814-2d75dafe9f15 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.765270 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1212/2405] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:02.776129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c505b-b0f6-4ac6-9b7c-499c06e6f21b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.777775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-461c505b-b0f6-4ac6-9b7c-499c06e6f21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:02.785329 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1194/2406] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:02 2022] POST /identity/v3/users => generated 281 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:02.789539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999e8f0-3a13-411e-8814-2d75dafe9f15 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.802458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999e8f0-3a13-411e-8814-2d75dafe9f15 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.807771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58aaf7-6faf-4c27-83cb-25ee55fa273c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.808368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58aaf7-6faf-4c27-83cb-25ee55fa273c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.808645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58aaf7-6faf-4c27-83cb-25ee55fa273c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.809006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58aaf7-6faf-4c27-83cb-25ee55fa273c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.809644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58aaf7-6faf-4c27-83cb-25ee55fa273c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.811321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58aaf7-6faf-4c27-83cb-25ee55fa273c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.822257 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1195/2407] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:02.826403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58aaf7-6faf-4c27-83cb-25ee55fa273c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.837438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58aaf7-6faf-4c27-83cb-25ee55fa273c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.847268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb77439-b640-4c1a-8ff5-ef60c1f09d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.848097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb77439-b640-4c1a-8ff5-ef60c1f09d0a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.848289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb77439-b640-4c1a-8ff5-ef60c1f09d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.848743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb77439-b640-4c1a-8ff5-ef60c1f09d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.849415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb77439-b640-4c1a-8ff5-ef60c1f09d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.851192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb77439-b640-4c1a-8ff5-ef60c1f09d0a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.864208 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1196/2408] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:52:02.868203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb77439-b640-4c1a-8ff5-ef60c1f09d0a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:02.872862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c505b-b0f6-4ac6-9b7c-499c06e6f21b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.879905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7fe92-9112-47f5-95a9-e61a81b18c70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.880514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7fe92-9112-47f5-95a9-e61a81b18c70 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.880792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7fe92-9112-47f5-95a9-e61a81b18c70 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.881085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7fe92-9112-47f5-95a9-e61a81b18c70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.881197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb77439-b640-4c1a-8ff5-ef60c1f09d0a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:02.886033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050f274-0757-4ba8-b5fc-4865be498c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:02.886672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050f274-0757-4ba8-b5fc-4865be498c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.886942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050f274-0757-4ba8-b5fc-4865be498c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.887693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050f274-0757-4ba8-b5fc-4865be498c59 admin admin] PATH_INFO: `/v3/projects/02d8a479a539401f8fdce3a47b92784b/users/cc73c9008b0c429ca555647dddc62435/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.890996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7fe92-9112-47f5-95a9-e61a81b18c70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.893002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7fe92-9112-47f5-95a9-e61a81b18c70 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.904224 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1213/2409] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:52:02.910467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050f274-0757-4ba8-b5fc-4865be498c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d8a479a539401f8fdce3a47b92784b, user_id=cc73c9008b0c429ca555647dddc62435, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:02.914098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050f274-0757-4ba8-b5fc-4865be498c59 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:02.937133 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1197/2410] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:02 2022] PUT /identity/v3/projects/02d8a479a539401f8fdce3a47b92784b/users/cc73c9008b0c429ca555647dddc62435/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:02.941220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f38ba-00f1-44e1-bb21-1628e9b1f9da None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:02.941500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f38ba-00f1-44e1-bb21-1628e9b1f9da None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:02.941764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f38ba-00f1-44e1-bb21-1628e9b1f9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:02.973301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-524f38ba-00f1-44e1-bb21-1628e9b1f9da None None] MFA Rules not processed for user `cc73c9008b0c429ca555647dddc62435`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:02.974593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-524f38ba-00f1-44e1-bb21-1628e9b1f9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:02.988570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050f274-0757-4ba8-b5fc-4865be498c59 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:03.012382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:03.050692 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1214/2411] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:03.060901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f38ba-00f1-44e1-bb21-1628e9b1f9da None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:03.062412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-524f38ba-00f1-44e1-bb21-1628e9b1f9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:03.064979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050f274-0757-4ba8-b5fc-4865be498c59 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:03.070578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6a904-b0a1-484f-91eb-6121c45b35d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:03.071177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6a904-b0a1-484f-91eb-6121c45b35d3 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:03.071448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6a904-b0a1-484f-91eb-6121c45b35d3 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:03.071718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6a904-b0a1-484f-91eb-6121c45b35d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:03.073480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bdd6a904-b0a1-484f-91eb-6121c45b35d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:03.079095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f38ba-00f1-44e1-bb21-1628e9b1f9da None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:03.085253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451c7cf-672b-4097-9037-92d772a13a55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:03.085840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451c7cf-672b-4097-9037-92d772a13a55 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:03.086112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451c7cf-672b-4097-9037-92d772a13a55 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:03.086375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451c7cf-672b-4097-9037-92d772a13a55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:03.095134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451c7cf-672b-4097-9037-92d772a13a55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:03.096720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451c7cf-672b-4097-9037-92d772a13a55 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:03.106150 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1215/2412] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:52:03.117633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:03.136120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6a904-b0a1-484f-91eb-6121c45b35d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:03.137731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6a904-b0a1-484f-91eb-6121c45b35d3 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:03.147495 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1198/2413] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:52:03.149465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451c7cf-672b-4097-9037-92d772a13a55 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:03.226492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451c7cf-672b-4097-9037-92d772a13a55 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:03.232733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:03.233396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:03.233669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:03.233935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] PATH_INFO: `/v3/users/cc73c9008b0c429ca555647dddc62435` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:03.235069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc73c9008b0c429ca555647dddc62435)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:03.237003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:03.312053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc73c9008b0c429ca555647dddc62435'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:03.328522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc73c9008b0c429ca555647dddc62435'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:03.341733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc73c9008b0c429ca555647dddc62435'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:03.351619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc73c9008b0c429ca555647dddc62435'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:03.352549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc73c9008b0c429ca555647dddc62435'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:03.359852 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1216/2414] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:03 2022] DELETE /identity/v3/users/cc73c9008b0c429ca555647dddc62435 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:52:03.364565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a52109-68ec-4784-a57b-5c2056a20d9e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:03.364957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a52109-68ec-4784-a57b-5c2056a20d9e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:03.365348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a52109-68ec-4784-a57b-5c2056a20d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:03.397898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a1a52109-68ec-4784-a57b-5c2056a20d9e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:03.399224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a1a52109-68ec-4784-a57b-5c2056a20d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:03.507298 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1199/2415] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:03.527872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:03.531456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:03.584421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a52109-68ec-4784-a57b-5c2056a20d9e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:03.586139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a1a52109-68ec-4784-a57b-5c2056a20d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:03.600462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a52109-68ec-4784-a57b-5c2056a20d9e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:03.607459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:03.608588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:03.609207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:03.609486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] PATH_INFO: `/v3/users/10843cee66164ad294a67be5dc5c3ff1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:03.610663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10843cee66164ad294a67be5dc5c3ff1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:03.612621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:03.623731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:03.643736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173c625-18ed-4111-8dbf-0a18c79ccc46 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:03.660339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b265b4-e69d-46c6-806f-aa6e465fa9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:03.662360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b265b4-e69d-46c6-806f-aa6e465fa9da service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:03.662643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b265b4-e69d-46c6-806f-aa6e465fa9da service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:03.662909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b265b4-e69d-46c6-806f-aa6e465fa9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:03.682287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b265b4-e69d-46c6-806f-aa6e465fa9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:03.683811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b265b4-e69d-46c6-806f-aa6e465fa9da service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:03.701552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10843cee66164ad294a67be5dc5c3ff1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:03.710345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10843cee66164ad294a67be5dc5c3ff1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:03.715427 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1217/2416] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:03.732914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10843cee66164ad294a67be5dc5c3ff1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:03.762444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10843cee66164ad294a67be5dc5c3ff1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:03.764011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea21dc8c-6f3f-4b91-9336-67a274cfc9b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10843cee66164ad294a67be5dc5c3ff1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:03.780139 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1200/2417] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:03 2022] DELETE /identity/v3/users/10843cee66164ad294a67be5dc5c3ff1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:52:03.781720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1b32-ee27-4f45-a6bf-487e2eebb350 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:03.781997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1b32-ee27-4f45-a6bf-487e2eebb350 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:03.782325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1b32-ee27-4f45-a6bf-487e2eebb350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:03.784816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f4150-2fd4-4383-baef-e14a0f080a6f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:03.785461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f4150-2fd4-4383-baef-e14a0f080a6f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:03.785961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f4150-2fd4-4383-baef-e14a0f080a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:03.823534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a83f4150-2fd4-4383-baef-e14a0f080a6f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:03.824066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-578d1b32-ee27-4f45-a6bf-487e2eebb350 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:03.824782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a83f4150-2fd4-4383-baef-e14a0f080a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:03.825717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-578d1b32-ee27-4f45-a6bf-487e2eebb350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:03.971336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:03.976002 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1218/2418] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:52:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:04.000349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d1b32-ee27-4f45-a6bf-487e2eebb350 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:04.003046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-578d1b32-ee27-4f45-a6bf-487e2eebb350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:04.006475 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1201/2419] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:04.017143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f4150-2fd4-4383-baef-e14a0f080a6f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:04.018556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a83f4150-2fd4-4383-baef-e14a0f080a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:04.162865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f4150-2fd4-4383-baef-e14a0f080a6f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:04.170498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054f072-2528-45b2-96d8-b99b774b626b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:04.171207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054f072-2528-45b2-96d8-b99b774b626b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:04.171545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054f072-2528-45b2-96d8-b99b774b626b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:04.171872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054f072-2528-45b2-96d8-b99b774b626b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:04.186431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d1b32-ee27-4f45-a6bf-487e2eebb350 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:04.188320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054f072-2528-45b2-96d8-b99b774b626b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:04.191676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054f072-2528-45b2-96d8-b99b774b626b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:04.208373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5802239-3a1c-444b-b0c5-27919df3be7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:04.209110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5802239-3a1c-444b-b0c5-27919df3be7b service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:04.209504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5802239-3a1c-444b-b0c5-27919df3be7b service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:04.209677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5802239-3a1c-444b-b0c5-27919df3be7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:04.219543 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1202/2420] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:52:04.222930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5802239-3a1c-444b-b0c5-27919df3be7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:04.224908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5802239-3a1c-444b-b0c5-27919df3be7b service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:04.235826 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1219/2421] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:52:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 20:52:04.246425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054f072-2528-45b2-96d8-b99b774b626b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:04.248661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f054f072-2528-45b2-96d8-b99b774b626b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:04.275175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054f072-2528-45b2-96d8-b99b774b626b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:04.281611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:04.282252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:04.282527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:04.282797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] PATH_INFO: `/v3/projects/02d8a479a539401f8fdce3a47b92784b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:04.284860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d8a479a539401f8fdce3a47b92784b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:04.286990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:04.389457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d8a479a539401f8fdce3a47b92784b', 'request_id': 'req-5cd1bf12-a452-4579-a284-4be534e67911'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:04.402667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d8a479a539401f8fdce3a47b92784b', 'request_id': 'req-5cd1bf12-a452-4579-a284-4be534e67911'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:04.413779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d8a479a539401f8fdce3a47b92784b', 'request_id': 'req-5cd1bf12-a452-4579-a284-4be534e67911'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:04.426876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] The token cache is being invalidate because project 02d8a479a539401f8fdce3a47b92784b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:04.427315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:04.428461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cd1bf12-a452-4579-a284-4be534e67911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:04.430359 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1203/2422] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:04 2022] DELETE /identity/v3/projects/02d8a479a539401f8fdce3a47b92784b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:04.435932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2769c0-c33f-4ff6-a98a-b233a48613ad None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:04.436214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2769c0-c33f-4ff6-a98a-b233a48613ad None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:04.436490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2769c0-c33f-4ff6-a98a-b233a48613ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:04.457058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6d2769c0-c33f-4ff6-a98a-b233a48613ad None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:04.458289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6d2769c0-c33f-4ff6-a98a-b233a48613ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:04.584338 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1220/2423] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:04.585677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd4a2-65bf-4c43-b855-217eade24eb2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:04.586164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd4a2-65bf-4c43-b855-217eade24eb2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:04.586699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd4a2-65bf-4c43-b855-217eade24eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:04.625784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-56fcd4a2-65bf-4c43-b855-217eade24eb2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:04.631695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-56fcd4a2-65bf-4c43-b855-217eade24eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:04.654164 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1204/2424] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:04.655160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2769c0-c33f-4ff6-a98a-b233a48613ad None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:04.668716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2769c0-c33f-4ff6-a98a-b233a48613ad None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:04.683779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28ab1-6103-476a-8dff-829235112b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:04.685098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28ab1-6103-476a-8dff-829235112b7b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:04.686039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28ab1-6103-476a-8dff-829235112b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:04.686771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28ab1-6103-476a-8dff-829235112b7b admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:04.689118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28ab1-6103-476a-8dff-829235112b7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:52:04.689506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28ab1-6103-476a-8dff-829235112b7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:04.692607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28ab1-6103-476a-8dff-829235112b7b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:04.700358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcd4a2-65bf-4c43-b855-217eade24eb2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:04.701844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-56fcd4a2-65bf-4c43-b855-217eade24eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:04.720462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcd4a2-65bf-4c43-b855-217eade24eb2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:04.726188 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1221/2425] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:04.726339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:04.726339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:04.726339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:04.726636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] PATH_INFO: `/v3/projects/6d53a952b9a04786a687eb543c8615c3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:04.727621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d53a952b9a04786a687eb543c8615c3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:04.729128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28ab1-6103-476a-8dff-829235112b7b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:04.731043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:04.778998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28ab1-6103-476a-8dff-829235112b7b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:04.786256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:04.792203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e2148-7af3-45ab-a253-0f7a88ded823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:04.793299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e2148-7af3-45ab-a253-0f7a88ded823 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:04.800054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e2148-7af3-45ab-a253-0f7a88ded823 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:04.800324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e2148-7af3-45ab-a253-0f7a88ded823 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:04.800994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e2148-7af3-45ab-a253-0f7a88ded823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:04.802951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e2148-7af3-45ab-a253-0f7a88ded823 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:04.881696 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1222/2426] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:04 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:52:04.890994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e2148-7af3-45ab-a253-0f7a88ded823 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:04.896815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d53a952b9a04786a687eb543c8615c3', 'request_id': 'req-5946fbb3-1d39-448b-af1a-7c095fa40c51'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:04.918539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d53a952b9a04786a687eb543c8615c3', 'request_id': 'req-5946fbb3-1d39-448b-af1a-7c095fa40c51'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:04.936485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d53a952b9a04786a687eb543c8615c3', 'request_id': 'req-5946fbb3-1d39-448b-af1a-7c095fa40c51'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:04.943207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:04.956050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] The token cache is being invalidate because project 6d53a952b9a04786a687eb543c8615c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:04.963648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:04.965126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:04.966817 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1205/2427] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:04 2022] DELETE /identity/v3/projects/6d53a952b9a04786a687eb543c8615c3 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:04.971407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:04.974022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:05.093712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e2148-7af3-45ab-a253-0f7a88ded823 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:05.098917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68704086-629a-479a-8022-91cdfa4a23fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:05.099423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68704086-629a-479a-8022-91cdfa4a23fa admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:05.099969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68704086-629a-479a-8022-91cdfa4a23fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:05.100247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68704086-629a-479a-8022-91cdfa4a23fa admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:05.100881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68704086-629a-479a-8022-91cdfa4a23fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:05.102861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68704086-629a-479a-8022-91cdfa4a23fa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:05.147704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946fbb3-1d39-448b-af1a-7c095fa40c51 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:05.154810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:05.155800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:05.156233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:05.156712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] PATH_INFO: `/v3/users/5246ab457b9a4086bd667e7436ab2951` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:05.158571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5246ab457b9a4086bd667e7436ab2951)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:05.159818 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1223/2428] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:04 2022] POST /identity/v3/users => generated 287 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:05.160885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68704086-629a-479a-8022-91cdfa4a23fa admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:05.161878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:05.167702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-68704086-629a-479a-8022-91cdfa4a23fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:05.201584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68704086-629a-479a-8022-91cdfa4a23fa admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:05.209384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f3ffb-aa20-45b9-882b-a0cb521a7a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:05.209896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f3ffb-aa20-45b9-882b-a0cb521a7a6d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:05.210451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f3ffb-aa20-45b9-882b-a0cb521a7a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:05.210709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f3ffb-aa20-45b9-882b-a0cb521a7a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:05.211361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f3ffb-aa20-45b9-882b-a0cb521a7a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:05.213727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f3ffb-aa20-45b9-882b-a0cb521a7a6d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:05.226446 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1224/2429] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:05.232117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f3ffb-aa20-45b9-882b-a0cb521a7a6d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:05.250459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f3ffb-aa20-45b9-882b-a0cb521a7a6d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:05.255683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5246ab457b9a4086bd667e7436ab2951'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:05.259114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24147f63-992c-4036-a1b2-98558d786c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:05.260184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24147f63-992c-4036-a1b2-98558d786c01 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:05.260566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24147f63-992c-4036-a1b2-98558d786c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:05.260936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24147f63-992c-4036-a1b2-98558d786c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:05.261716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24147f63-992c-4036-a1b2-98558d786c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:05.263845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24147f63-992c-4036-a1b2-98558d786c01 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:05.273859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5246ab457b9a4086bd667e7436ab2951'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:05.281089 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1225/2430] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:52:05.285656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24147f63-992c-4036-a1b2-98558d786c01 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:05.297530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5246ab457b9a4086bd667e7436ab2951'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:05.301079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24147f63-992c-4036-a1b2-98558d786c01 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:05.309603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceee69d-a392-456f-8b9d-3ef8dde1c4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:05.310509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceee69d-a392-456f-8b9d-3ef8dde1c4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:05.310994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceee69d-a392-456f-8b9d-3ef8dde1c4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:05.311698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceee69d-a392-456f-8b9d-3ef8dde1c4e9 admin admin] PATH_INFO: `/v3/projects/1c30aa11fd244564be716550bfe5aaa6/users/8bbc891924f84d99aac22518f69e7b99/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:05.311892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5246ab457b9a4086bd667e7436ab2951'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:05.313529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40acaa58-0f69-4c36-a4c2-ff0075b269b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5246ab457b9a4086bd667e7436ab2951'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:05.325592 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1206/2431] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:04 2022] DELETE /identity/v3/users/5246ab457b9a4086bd667e7436ab2951 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:05.333760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceee69d-a392-456f-8b9d-3ef8dde1c4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c30aa11fd244564be716550bfe5aaa6, user_id=8bbc891924f84d99aac22518f69e7b99, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:05.338145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceee69d-a392-456f-8b9d-3ef8dde1c4e9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:05.372674 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1226/2432] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:05 2022] PUT /identity/v3/projects/1c30aa11fd244564be716550bfe5aaa6/users/8bbc891924f84d99aac22518f69e7b99/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:05.374581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86156361-9b2b-4a73-9871-94510cc3d842 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:05.375790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86156361-9b2b-4a73-9871-94510cc3d842 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:05.376133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86156361-9b2b-4a73-9871-94510cc3d842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:05.414734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-86156361-9b2b-4a73-9871-94510cc3d842 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:05.416726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-86156361-9b2b-4a73-9871-94510cc3d842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:05.594608 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1207/2433] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:05.622049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceee69d-a392-456f-8b9d-3ef8dde1c4e9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:05.627164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dceee69d-a392-456f-8b9d-3ef8dde1c4e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:05.759713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceee69d-a392-456f-8b9d-3ef8dde1c4e9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:05.764333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef360992-65ec-4ef1-8f3a-bfa44f21104e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:05.778890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef360992-65ec-4ef1-8f3a-bfa44f21104e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:05.780172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef360992-65ec-4ef1-8f3a-bfa44f21104e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:05.780640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef360992-65ec-4ef1-8f3a-bfa44f21104e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:05.805647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef360992-65ec-4ef1-8f3a-bfa44f21104e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:05.812746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef360992-65ec-4ef1-8f3a-bfa44f21104e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:05.840346 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1227/2434] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:06.598845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86156361-9b2b-4a73-9871-94510cc3d842 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:06.600326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-86156361-9b2b-4a73-9871-94510cc3d842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:06.617637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86156361-9b2b-4a73-9871-94510cc3d842 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:06.642069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:06.642715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:06.642983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:06.643245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] PATH_INFO: `/v3/projects/8f6ce97c7b4b4778a6ad48e3566146a2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:06.644460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6ce97c7b4b4778a6ad48e3566146a2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:06.646373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:06.671278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:06.751774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6ce97c7b4b4778a6ad48e3566146a2', 'request_id': 'req-4144f1fa-e914-4bf9-b3d6-44352fc77706'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:06.778555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6ce97c7b4b4778a6ad48e3566146a2', 'request_id': 'req-4144f1fa-e914-4bf9-b3d6-44352fc77706'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:06.791595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6ce97c7b4b4778a6ad48e3566146a2', 'request_id': 'req-4144f1fa-e914-4bf9-b3d6-44352fc77706'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:06.824198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] The token cache is being invalidate because project 8f6ce97c7b4b4778a6ad48e3566146a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:06.824505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:06.831116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4144f1fa-e914-4bf9-b3d6-44352fc77706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:06.834784 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1208/2435] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:06 2022] DELETE /identity/v3/projects/8f6ce97c7b4b4778a6ad48e3566146a2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:06.840225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8d22a-3a36-42a3-bb3b-e64ecc36e24d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:06.840719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8d22a-3a36-42a3-bb3b-e64ecc36e24d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:06.840802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8d22a-3a36-42a3-bb3b-e64ecc36e24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:06.895446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-13f8d22a-3a36-42a3-bb3b-e64ecc36e24d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:06.898368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-13f8d22a-3a36-42a3-bb3b-e64ecc36e24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:07.002109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:07.102877 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1228/2436] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:07.111087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8883fc1-d493-4b7c-8927-b3a98dd3664b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:07.111728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8883fc1-d493-4b7c-8927-b3a98dd3664b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:07.112248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8883fc1-d493-4b7c-8927-b3a98dd3664b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:07.149768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f8883fc1-d493-4b7c-8927-b3a98dd3664b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:07.153057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f8883fc1-d493-4b7c-8927-b3a98dd3664b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:07.183732 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1209/2437] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:07.188283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8d22a-3a36-42a3-bb3b-e64ecc36e24d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:07.221167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8d22a-3a36-42a3-bb3b-e64ecc36e24d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:07.227125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a6802-ea4c-4a3b-b04f-c333346b0d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:07.227858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a6802-ea4c-4a3b-b04f-c333346b0d22 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:07.228429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a6802-ea4c-4a3b-b04f-c333346b0d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:07.228907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a6802-ea4c-4a3b-b04f-c333346b0d22 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:07.232781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a6802-ea4c-4a3b-b04f-c333346b0d22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:52:07.232781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a6802-ea4c-4a3b-b04f-c333346b0d22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:07.232781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a6802-ea4c-4a3b-b04f-c333346b0d22 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:07.246693 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1229/2438] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:07.251235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8883fc1-d493-4b7c-8927-b3a98dd3664b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:07.265885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8883fc1-d493-4b7c-8927-b3a98dd3664b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:07.273549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8183da-e5c3-44f7-b52a-49c919df5d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:07.274168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8183da-e5c3-44f7-b52a-49c919df5d04 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:07.274454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8183da-e5c3-44f7-b52a-49c919df5d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:07.276253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8183da-e5c3-44f7-b52a-49c919df5d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:07.276910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8183da-e5c3-44f7-b52a-49c919df5d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:07.279734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8183da-e5c3-44f7-b52a-49c919df5d04 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:07.340762 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1210/2439] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:07 2022] POST /identity/v3/projects => generated 391 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:07.345947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a6802-ea4c-4a3b-b04f-c333346b0d22 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:07.359002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a6802-ea4c-4a3b-b04f-c333346b0d22 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:07.502460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64295fd-5357-46c3-b2f5-133f8f2709e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:07.504237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64295fd-5357-46c3-b2f5-133f8f2709e6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:07.506702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64295fd-5357-46c3-b2f5-133f8f2709e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:07.506987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64295fd-5357-46c3-b2f5-133f8f2709e6 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:07.523238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64295fd-5357-46c3-b2f5-133f8f2709e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:07.535826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64295fd-5357-46c3-b2f5-133f8f2709e6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:07.627036 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1230/2440] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:07 2022] POST /identity/v3/users => generated 296 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:07.628235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8183da-e5c3-44f7-b52a-49c919df5d04 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:07.642918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8183da-e5c3-44f7-b52a-49c919df5d04 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:07.671726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1933d-1987-4101-aed6-e79f31f6e62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:07.672619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1933d-1987-4101-aed6-e79f31f6e62d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:07.673146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1933d-1987-4101-aed6-e79f31f6e62d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:07.673367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1933d-1987-4101-aed6-e79f31f6e62d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:07.674015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1933d-1987-4101-aed6-e79f31f6e62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:07.676452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1933d-1987-4101-aed6-e79f31f6e62d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:07.731077 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1211/2441] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:07.733876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64295fd-5357-46c3-b2f5-133f8f2709e6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:07.744713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64295fd-5357-46c3-b2f5-133f8f2709e6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:07.766057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb91daf-365f-4cac-8dc8-9ae1970c4e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:07.766846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb91daf-365f-4cac-8dc8-9ae1970c4e24 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:07.767286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb91daf-365f-4cac-8dc8-9ae1970c4e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:07.767720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb91daf-365f-4cac-8dc8-9ae1970c4e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:07.768589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb91daf-365f-4cac-8dc8-9ae1970c4e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:07.770369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb91daf-365f-4cac-8dc8-9ae1970c4e24 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:07.799131 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1231/2442] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:52:07.803475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1933d-1987-4101-aed6-e79f31f6e62d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:07.814533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1933d-1987-4101-aed6-e79f31f6e62d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:07.834153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a8fcb-4b62-4983-afb3-0309ea30ec01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:07.834943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a8fcb-4b62-4983-afb3-0309ea30ec01 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:07.835248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a8fcb-4b62-4983-afb3-0309ea30ec01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:07.835503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a8fcb-4b62-4983-afb3-0309ea30ec01 admin admin] PATH_INFO: `/v3/projects/1bee0be68ff7479d815da6fd79b1547f/users/c4ef5c31f3804698805b7d38c1868938/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:07.870808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a8fcb-4b62-4983-afb3-0309ea30ec01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bee0be68ff7479d815da6fd79b1547f, user_id=c4ef5c31f3804698805b7d38c1868938, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:07.877191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a8fcb-4b62-4983-afb3-0309ea30ec01 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:07.910606 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1212/2443] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:07 2022] PUT /identity/v3/projects/1bee0be68ff7479d815da6fd79b1547f/users/c4ef5c31f3804698805b7d38c1868938/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:07.921320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199718a-18d7-4c03-b68d-12e60cf9af52 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:07.922152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199718a-18d7-4c03-b68d-12e60cf9af52 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:07.922577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199718a-18d7-4c03-b68d-12e60cf9af52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:08.005209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6199718a-18d7-4c03-b68d-12e60cf9af52 None None] MFA Rules not processed for user `c4ef5c31f3804698805b7d38c1868938`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:08.006623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6199718a-18d7-4c03-b68d-12e60cf9af52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:08.142871 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1232/2444] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:08.144080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a8fcb-4b62-4983-afb3-0309ea30ec01 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:08.332721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a8fcb-4b62-4983-afb3-0309ea30ec01 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:08.339097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c551a9-3f8f-424d-bf99-e3575eb4c573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:08.339708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c551a9-3f8f-424d-bf99-e3575eb4c573 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:08.339978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c551a9-3f8f-424d-bf99-e3575eb4c573 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:08.340382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c551a9-3f8f-424d-bf99-e3575eb4c573 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:08.341030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c551a9-3f8f-424d-bf99-e3575eb4c573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:08.343913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c551a9-3f8f-424d-bf99-e3575eb4c573 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:08.403643 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1213/2445] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:08 2022] POST /identity/v3/projects => generated 391 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:52:08.416030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199718a-18d7-4c03-b68d-12e60cf9af52 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:08.439432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199718a-18d7-4c03-b68d-12e60cf9af52 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:08.516756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:08.630555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dead64-ba5b-413c-aa3b-1d9236a1da05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:08.630904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dead64-ba5b-413c-aa3b-1d9236a1da05 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:08.631137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dead64-ba5b-413c-aa3b-1d9236a1da05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:08.631404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dead64-ba5b-413c-aa3b-1d9236a1da05 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:08.632030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dead64-ba5b-413c-aa3b-1d9236a1da05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:08.633991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dead64-ba5b-413c-aa3b-1d9236a1da05 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:08.668292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c551a9-3f8f-424d-bf99-e3575eb4c573 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:08.673996 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1233/2446] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:08 2022] POST /identity/v3/users => generated 302 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:08.686917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c551a9-3f8f-424d-bf99-e3575eb4c573 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:08.697648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abef44b-83e0-44c1-af10-9deb6d4d75b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:08.698536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef44b-83e0-44c1-af10-9deb6d4d75b9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:08.699503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef44b-83e0-44c1-af10-9deb6d4d75b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:08.699503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef44b-83e0-44c1-af10-9deb6d4d75b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:08.700597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abef44b-83e0-44c1-af10-9deb6d4d75b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:08.702170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abef44b-83e0-44c1-af10-9deb6d4d75b9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:08.734892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1214/2447] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:08.736627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dead64-ba5b-413c-aa3b-1d9236a1da05 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:08.750760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dead64-ba5b-413c-aa3b-1d9236a1da05 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:08.765073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c551ab-3e47-4113-81bb-a217cec838b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:08.766895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c551ab-3e47-4113-81bb-a217cec838b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:08.766895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c551ab-3e47-4113-81bb-a217cec838b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:08.766895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c551ab-3e47-4113-81bb-a217cec838b0 admin admin] PATH_INFO: `/v3/projects/df717e6d14ce4cb4852e3078c33478a0/users/1385481e4d1b4bea9c0860fa42ed13e0/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:08.805551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c551ab-3e47-4113-81bb-a217cec838b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df717e6d14ce4cb4852e3078c33478a0, user_id=1385481e4d1b4bea9c0860fa42ed13e0, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:08.810007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c551ab-3e47-4113-81bb-a217cec838b0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:08.841061 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1234/2448] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:08 2022] PUT /identity/v3/projects/df717e6d14ce4cb4852e3078c33478a0/users/1385481e4d1b4bea9c0860fa42ed13e0/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:08.843053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abef44b-83e0-44c1-af10-9deb6d4d75b9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:08.859107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abef44b-83e0-44c1-af10-9deb6d4d75b9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:08.872384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:08.985132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb584e00-9269-4e56-8588-d48654263e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:08.986371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb584e00-9269-4e56-8588-d48654263e8a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:08.986967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb584e00-9269-4e56-8588-d48654263e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:08.988248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb584e00-9269-4e56-8588-d48654263e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:08.989200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb584e00-9269-4e56-8588-d48654263e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:08.991289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb584e00-9269-4e56-8588-d48654263e8a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:09.023459 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1215/2449] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:09.028653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c551ab-3e47-4113-81bb-a217cec838b0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:09.046281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c551ab-3e47-4113-81bb-a217cec838b0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:09.052348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d6563-ffb0-4c48-a794-da60cc1d66cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:09.053386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d6563-ffb0-4c48-a794-da60cc1d66cf admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:09.053863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d6563-ffb0-4c48-a794-da60cc1d66cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:09.054526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d6563-ffb0-4c48-a794-da60cc1d66cf admin admin] PATH_INFO: `/v3/projects/df717e6d14ce4cb4852e3078c33478a0/users/1385481e4d1b4bea9c0860fa42ed13e0/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:09.058429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d6563-ffb0-4c48-a794-da60cc1d66cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=df717e6d14ce4cb4852e3078c33478a0, user_id=1385481e4d1b4bea9c0860fa42ed13e0, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:09.078424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d6563-ffb0-4c48-a794-da60cc1d66cf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:09.118430 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1235/2450] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:09 2022] PUT /identity/v3/projects/df717e6d14ce4cb4852e3078c33478a0/users/1385481e4d1b4bea9c0860fa42ed13e0/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:09.131729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd9710-30f7-47f2-9f60-fdd23be65703 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:09.132452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd9710-30f7-47f2-9f60-fdd23be65703 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:09.133201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd9710-30f7-47f2-9f60-fdd23be65703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:09.227413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1bbd9710-30f7-47f2-9f60-fdd23be65703 None None] MFA Rules not processed for user `1385481e4d1b4bea9c0860fa42ed13e0`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:09.229725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1bbd9710-30f7-47f2-9f60-fdd23be65703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:09.438299 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1216/2451] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:09 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:09.439211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d6563-ffb0-4c48-a794-da60cc1d66cf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:09.441827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e1d6563-ffb0-4c48-a794-da60cc1d66cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:09.651559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d6563-ffb0-4c48-a794-da60cc1d66cf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:09.708118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5bf55-1e0d-487e-bd94-b23b80844118 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:09.709917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5bf55-1e0d-487e-bd94-b23b80844118 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:09.709917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5bf55-1e0d-487e-bd94-b23b80844118 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:09.709917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5bf55-1e0d-487e-bd94-b23b80844118 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:09.721135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5bf55-1e0d-487e-bd94-b23b80844118 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:09.722865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5bf55-1e0d-487e-bd94-b23b80844118 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:09.735576 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1236/2452] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:09 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:52:10.431358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd9710-30f7-47f2-9f60-fdd23be65703 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:10.655589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd9710-30f7-47f2-9f60-fdd23be65703 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:10.661072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:10.663237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:10.663237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:10.663237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] PATH_INFO: `/v3/users/c4ef5c31f3804698805b7d38c1868938` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:10.663664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ef5c31f3804698805b7d38c1868938)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:10.665272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:10.716994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:10.739577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ef5c31f3804698805b7d38c1868938'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:10.764295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ef5c31f3804698805b7d38c1868938'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:10.782413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ef5c31f3804698805b7d38c1868938'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:10.798170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ef5c31f3804698805b7d38c1868938'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:10.800151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ef5c31f3804698805b7d38c1868938'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:10.822124 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1217/2453] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:10 2022] DELETE /identity/v3/users/c4ef5c31f3804698805b7d38c1868938 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:10.826271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92736f4-f84c-4b6f-b1d8-25216b28b490 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:10.827258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92736f4-f84c-4b6f-b1d8-25216b28b490 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:10.827681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92736f4-f84c-4b6f-b1d8-25216b28b490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:10.864139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c92736f4-f84c-4b6f-b1d8-25216b28b490 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:10.865692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c92736f4-f84c-4b6f-b1d8-25216b28b490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:11.007075 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1237/2454] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:11.024436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:11.026561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:11.186467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb890379-5ee5-4979-88a7-702d58117985 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:11.211141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09875ae-9f8d-4915-ba02-6ac01a3dbb74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:11.215045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09875ae-9f8d-4915-ba02-6ac01a3dbb74 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:11.219362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09875ae-9f8d-4915-ba02-6ac01a3dbb74 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:11.219628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09875ae-9f8d-4915-ba02-6ac01a3dbb74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:11.254156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09875ae-9f8d-4915-ba02-6ac01a3dbb74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:11.255674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09875ae-9f8d-4915-ba02-6ac01a3dbb74 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:11.283826 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1218/2455] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:12.179826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92736f4-f84c-4b6f-b1d8-25216b28b490 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:12.182071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c92736f4-f84c-4b6f-b1d8-25216b28b490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:12.207394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:12.220850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92736f4-f84c-4b6f-b1d8-25216b28b490 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:12.241711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:12.242639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:12.243173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:12.243819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] PATH_INFO: `/v3/projects/1bee0be68ff7479d815da6fd79b1547f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:12.246147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bee0be68ff7479d815da6fd79b1547f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:12.248378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:12.355833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bee0be68ff7479d815da6fd79b1547f', 'request_id': 'req-53c776f2-7ae4-4cfd-9b80-900daf1213cc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:12.371062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bee0be68ff7479d815da6fd79b1547f', 'request_id': 'req-53c776f2-7ae4-4cfd-9b80-900daf1213cc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:12.381996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bee0be68ff7479d815da6fd79b1547f', 'request_id': 'req-53c776f2-7ae4-4cfd-9b80-900daf1213cc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:12.406846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] The token cache is being invalidate because project 1bee0be68ff7479d815da6fd79b1547f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:12.407530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:12.409794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:12.412596 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1238/2456] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:12 2022] DELETE /identity/v3/projects/1bee0be68ff7479d815da6fd79b1547f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:12.416661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09875ae-9f8d-4915-ba02-6ac01a3dbb74 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:12.436932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09875ae-9f8d-4915-ba02-6ac01a3dbb74 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:12.564277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:12.564556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:12.564940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:12.565333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] PATH_INFO: `/v3/users/1385481e4d1b4bea9c0860fa42ed13e0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:12.570437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1385481e4d1b4bea9c0860fa42ed13e0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:12.572499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:12.682279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1385481e4d1b4bea9c0860fa42ed13e0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:12.688952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:12.698007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1385481e4d1b4bea9c0860fa42ed13e0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:12.742895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1385481e4d1b4bea9c0860fa42ed13e0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:12.753019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1385481e4d1b4bea9c0860fa42ed13e0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:12.757938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f89d83c0-ed98-4014-add9-b3523dc44856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1385481e4d1b4bea9c0860fa42ed13e0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:12.767139 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1219/2457] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:12 2022] DELETE /identity/v3/users/1385481e4d1b4bea9c0860fa42ed13e0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:13.295580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:13.299967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:13.321162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c776f2-7ae4-4cfd-9b80-900daf1213cc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:13.467676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:13.470420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:13.471048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:13.471791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] PATH_INFO: `/v3/projects/df717e6d14ce4cb4852e3078c33478a0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:13.477870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df717e6d14ce4cb4852e3078c33478a0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:13.484405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:13.695804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df717e6d14ce4cb4852e3078c33478a0', 'request_id': 'req-c99d3efc-dd5f-422b-96e3-952c0af25fe1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:13.714609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df717e6d14ce4cb4852e3078c33478a0', 'request_id': 'req-c99d3efc-dd5f-422b-96e3-952c0af25fe1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:13.742227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df717e6d14ce4cb4852e3078c33478a0', 'request_id': 'req-c99d3efc-dd5f-422b-96e3-952c0af25fe1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:13.778985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] The token cache is being invalidate because project df717e6d14ce4cb4852e3078c33478a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:13.779736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:13.782167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c99d3efc-dd5f-422b-96e3-952c0af25fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:13.786503 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1239/2458] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:13 2022] DELETE /identity/v3/projects/df717e6d14ce4cb4852e3078c33478a0 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:13.791973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f81e99-8bec-46cc-a895-70122d678255 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:13.792891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f81e99-8bec-46cc-a895-70122d678255 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:13.793356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f81e99-8bec-46cc-a895-70122d678255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:13.864286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-91f81e99-8bec-46cc-a895-70122d678255 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:13.867202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-91f81e99-8bec-46cc-a895-70122d678255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:14.218299 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1220/2459] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:14.220450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42ca10-15ba-433d-8990-f33336c6e3ae None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:14.222045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42ca10-15ba-433d-8990-f33336c6e3ae None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:14.224403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42ca10-15ba-433d-8990-f33336c6e3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:14.295566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ed42ca10-15ba-433d-8990-f33336c6e3ae None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:14.297980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ed42ca10-15ba-433d-8990-f33336c6e3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:14.339880 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1240/2460] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:14.340941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f81e99-8bec-46cc-a895-70122d678255 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:14.358387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f81e99-8bec-46cc-a895-70122d678255 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:14.376766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b046318-e623-49bb-b758-ef9f9daec934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:14.380131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b046318-e623-49bb-b758-ef9f9daec934 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:14.380548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b046318-e623-49bb-b758-ef9f9daec934 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:14.381369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b046318-e623-49bb-b758-ef9f9daec934 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:14.388786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b046318-e623-49bb-b758-ef9f9daec934 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:52:14.396621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b046318-e623-49bb-b758-ef9f9daec934 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:14.396621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b046318-e623-49bb-b758-ef9f9daec934 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:14.431355 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1221/2461] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:52:14.436406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42ca10-15ba-433d-8990-f33336c6e3ae None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:14.453488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42ca10-15ba-433d-8990-f33336c6e3ae None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:14.482692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243e6a6-5d4e-4bbc-8cb5-c6cc45568afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:14.484771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243e6a6-5d4e-4bbc-8cb5-c6cc45568afb admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:14.486613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243e6a6-5d4e-4bbc-8cb5-c6cc45568afb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:14.488550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243e6a6-5d4e-4bbc-8cb5-c6cc45568afb admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:14.490841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243e6a6-5d4e-4bbc-8cb5-c6cc45568afb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:14.494822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243e6a6-5d4e-4bbc-8cb5-c6cc45568afb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:14.526264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:14.602749 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1241/2462] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:14 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:14.605231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b046318-e623-49bb-b758-ef9f9daec934 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:14.626922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b046318-e623-49bb-b758-ef9f9daec934 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:14.780073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a1a9-fb7b-4e8a-abde-9dbc79d3ae32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:14.780704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a1a9-fb7b-4e8a-abde-9dbc79d3ae32 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:14.821423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a1a9-fb7b-4e8a-abde-9dbc79d3ae32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:14.823722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a1a9-fb7b-4e8a-abde-9dbc79d3ae32 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:14.826333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54a1a9-fb7b-4e8a-abde-9dbc79d3ae32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:14.831779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54a1a9-fb7b-4e8a-abde-9dbc79d3ae32 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:14.919342 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1222/2463] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:14 2022] POST /identity/v3/users => generated 281 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:14.920669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243e6a6-5d4e-4bbc-8cb5-c6cc45568afb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:14.935415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243e6a6-5d4e-4bbc-8cb5-c6cc45568afb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:14.974315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79d177-ca70-46c0-886f-03e4d6c6003c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:14.974945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79d177-ca70-46c0-886f-03e4d6c6003c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:14.975224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79d177-ca70-46c0-886f-03e4d6c6003c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:14.975493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79d177-ca70-46c0-886f-03e4d6c6003c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:14.980622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79d177-ca70-46c0-886f-03e4d6c6003c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:14.982374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79d177-ca70-46c0-886f-03e4d6c6003c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:15.023938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1242/2464] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:15.028288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a1a9-fb7b-4e8a-abde-9dbc79d3ae32 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:15.045596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a1a9-fb7b-4e8a-abde-9dbc79d3ae32 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:15.067090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3cbea-5e9e-4727-a865-284139b7ce5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:15.067906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3cbea-5e9e-4727-a865-284139b7ce5f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:15.068342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3cbea-5e9e-4727-a865-284139b7ce5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:15.068899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3cbea-5e9e-4727-a865-284139b7ce5f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:15.069737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3cbea-5e9e-4727-a865-284139b7ce5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:15.074971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3cbea-5e9e-4727-a865-284139b7ce5f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:15.130687 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1223/2465] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:15.135284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79d177-ca70-46c0-886f-03e4d6c6003c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:15.164140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79d177-ca70-46c0-886f-03e4d6c6003c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:15.180105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a88712-12d5-40b5-a9b8-62e4f130ab0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:15.180105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88712-12d5-40b5-a9b8-62e4f130ab0f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:15.184342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88712-12d5-40b5-a9b8-62e4f130ab0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:15.184800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88712-12d5-40b5-a9b8-62e4f130ab0f admin admin] PATH_INFO: `/v3/projects/f052606f63d64258a31961e11816a261/users/ecbe63acee0c48fdb5ae4f3adc7f57a1/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:15.264225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a88712-12d5-40b5-a9b8-62e4f130ab0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f052606f63d64258a31961e11816a261, user_id=ecbe63acee0c48fdb5ae4f3adc7f57a1, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:15.273925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a88712-12d5-40b5-a9b8-62e4f130ab0f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:15.306734 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1243/2466] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:15 2022] PUT /identity/v3/projects/f052606f63d64258a31961e11816a261/users/ecbe63acee0c48fdb5ae4f3adc7f57a1/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:15.318388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a71eb6-4b0d-4438-8f34-abc902e47ba2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:15.318673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a71eb6-4b0d-4438-8f34-abc902e47ba2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:15.319027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a71eb6-4b0d-4438-8f34-abc902e47ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:15.391904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-28a71eb6-4b0d-4438-8f34-abc902e47ba2 None None] MFA Rules not processed for user `ecbe63acee0c48fdb5ae4f3adc7f57a1`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:15.393248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-28a71eb6-4b0d-4438-8f34-abc902e47ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:15.437205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:15.493879 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1224/2467] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:15.498892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a88712-12d5-40b5-a9b8-62e4f130ab0f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:15.606068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a88712-12d5-40b5-a9b8-62e4f130ab0f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:15.612234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb934e8d-5224-4f5e-bc13-d339a96d42cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:15.613362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb934e8d-5224-4f5e-bc13-d339a96d42cc admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:15.613798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb934e8d-5224-4f5e-bc13-d339a96d42cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:15.614397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb934e8d-5224-4f5e-bc13-d339a96d42cc admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:15.615287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb934e8d-5224-4f5e-bc13-d339a96d42cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:15.617549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb934e8d-5224-4f5e-bc13-d339a96d42cc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:15.656723 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1244/2468] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:15 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:15.660571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a71eb6-4b0d-4438-8f34-abc902e47ba2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:15.680126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a71eb6-4b0d-4438-8f34-abc902e47ba2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:15.803833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899d48d-a2eb-4493-86c4-b992293195de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:15.804462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899d48d-a2eb-4493-86c4-b992293195de admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:15.804737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899d48d-a2eb-4493-86c4-b992293195de admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:15.805004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899d48d-a2eb-4493-86c4-b992293195de admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:15.806493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899d48d-a2eb-4493-86c4-b992293195de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:15.809945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899d48d-a2eb-4493-86c4-b992293195de admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:15.841942 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1225/2469] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:15 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:52:15.842945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb934e8d-5224-4f5e-bc13-d339a96d42cc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:15.873239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb934e8d-5224-4f5e-bc13-d339a96d42cc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:15.878762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c147d39-bf54-4d69-9344-f5b2af9e625b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:15.879702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c147d39-bf54-4d69-9344-f5b2af9e625b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:15.880270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c147d39-bf54-4d69-9344-f5b2af9e625b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:15.880647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c147d39-bf54-4d69-9344-f5b2af9e625b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:15.886341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c147d39-bf54-4d69-9344-f5b2af9e625b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:15.891467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c147d39-bf54-4d69-9344-f5b2af9e625b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:15.927178 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1245/2470] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:15.933009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899d48d-a2eb-4493-86c4-b992293195de admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:15.946741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899d48d-a2eb-4493-86c4-b992293195de admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:15.952862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe1d86-40a3-4725-a45b-2e61b3641614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:15.953506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1d86-40a3-4725-a45b-2e61b3641614 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:15.953905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1d86-40a3-4725-a45b-2e61b3641614 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:15.954599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1d86-40a3-4725-a45b-2e61b3641614 admin admin] PATH_INFO: `/v3/projects/a7cc21bb0a3949d8a989940a07a85f84/users/730bccefd42b4379af09bd391c6f4baf/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:15.994983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe1d86-40a3-4725-a45b-2e61b3641614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7cc21bb0a3949d8a989940a07a85f84, user_id=730bccefd42b4379af09bd391c6f4baf, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:16.005992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe1d86-40a3-4725-a45b-2e61b3641614 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:16.045310 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1226/2471] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:15 2022] PUT /identity/v3/projects/a7cc21bb0a3949d8a989940a07a85f84/users/730bccefd42b4379af09bd391c6f4baf/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:16.046135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c147d39-bf54-4d69-9344-f5b2af9e625b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:16.064049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c147d39-bf54-4d69-9344-f5b2af9e625b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:16.137694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383c02b-e9b6-471c-99df-8f9b44d2e1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:16.138632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383c02b-e9b6-471c-99df-8f9b44d2e1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:16.138960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383c02b-e9b6-471c-99df-8f9b44d2e1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:16.139283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383c02b-e9b6-471c-99df-8f9b44d2e1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:16.139934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383c02b-e9b6-471c-99df-8f9b44d2e1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:16.141948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383c02b-e9b6-471c-99df-8f9b44d2e1c4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:16.161391 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1246/2472] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:16.162587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe1d86-40a3-4725-a45b-2e61b3641614 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:16.179419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe1d86-40a3-4725-a45b-2e61b3641614 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:16.202673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e3b21-1f3f-4116-9f58-850d9fd0bc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:16.203453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e3b21-1f3f-4116-9f58-850d9fd0bc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:16.206268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e3b21-1f3f-4116-9f58-850d9fd0bc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:16.206268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e3b21-1f3f-4116-9f58-850d9fd0bc10 admin admin] PATH_INFO: `/v3/projects/a7cc21bb0a3949d8a989940a07a85f84/users/730bccefd42b4379af09bd391c6f4baf/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:16.210028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e3b21-1f3f-4116-9f58-850d9fd0bc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7cc21bb0a3949d8a989940a07a85f84, user_id=730bccefd42b4379af09bd391c6f4baf, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:16.214878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e3b21-1f3f-4116-9f58-850d9fd0bc10 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:16.232264 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1227/2473] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:16 2022] PUT /identity/v3/projects/a7cc21bb0a3949d8a989940a07a85f84/users/730bccefd42b4379af09bd391c6f4baf/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:16.242463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa274e-ee01-4d7d-9996-7ae81b5823ac None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:16.243096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa274e-ee01-4d7d-9996-7ae81b5823ac None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:16.243480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa274e-ee01-4d7d-9996-7ae81b5823ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:16.283599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-eefa274e-ee01-4d7d-9996-7ae81b5823ac None None] MFA Rules not processed for user `730bccefd42b4379af09bd391c6f4baf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:16.284996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eefa274e-ee01-4d7d-9996-7ae81b5823ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:16.298763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:16.403213 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1247/2474] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:16.510737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e3b21-1f3f-4116-9f58-850d9fd0bc10 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:16.514675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-415e3b21-1f3f-4116-9f58-850d9fd0bc10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:16.646422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e3b21-1f3f-4116-9f58-850d9fd0bc10 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:16.652657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e89734-a6a9-4c3a-9bb2-8d4c1d9ff0b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:16.653310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89734-a6a9-4c3a-9bb2-8d4c1d9ff0b1 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:16.653585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89734-a6a9-4c3a-9bb2-8d4c1d9ff0b1 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:16.653856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89734-a6a9-4c3a-9bb2-8d4c1d9ff0b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:16.667170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e89734-a6a9-4c3a-9bb2-8d4c1d9ff0b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:16.668974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e89734-a6a9-4c3a-9bb2-8d4c1d9ff0b1 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:16.679457 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1228/2475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:16 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:16.764582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d7cff-9bdb-4f4d-a2aa-4991d49058c0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:16.764871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d7cff-9bdb-4f4d-a2aa-4991d49058c0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:16.765238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d7cff-9bdb-4f4d-a2aa-4991d49058c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:16.813547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-467d7cff-9bdb-4f4d-a2aa-4991d49058c0 None None] MFA Rules not processed for user `8bbc891924f84d99aac22518f69e7b99`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:16.814749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-467d7cff-9bdb-4f4d-a2aa-4991d49058c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:16.899721 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1248/2476] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:16.938375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e89734-a6a9-4c3a-9bb2-8d4c1d9ff0b1 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:16.939867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f0e89734-a6a9-4c3a-9bb2-8d4c1d9ff0b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:16.946343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:17.018280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e89734-a6a9-4c3a-9bb2-8d4c1d9ff0b1 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:17.022177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08b633-2204-4a9f-b4fe-016debe37097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:17.022774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08b633-2204-4a9f-b4fe-016debe37097 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:17.023525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08b633-2204-4a9f-b4fe-016debe37097 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:17.023525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08b633-2204-4a9f-b4fe-016debe37097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:17.033761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08b633-2204-4a9f-b4fe-016debe37097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:17.035354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08b633-2204-4a9f-b4fe-016debe37097 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:17.046468 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1229/2477] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:17.218146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d7cff-9bdb-4f4d-a2aa-4991d49058c0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:17.320779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d7cff-9bdb-4f4d-a2aa-4991d49058c0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:17.327436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:17.328597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:17.329157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:17.329594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] PATH_INFO: `/v3/users/ecbe63acee0c48fdb5ae4f3adc7f57a1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:17.331026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecbe63acee0c48fdb5ae4f3adc7f57a1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:17.333635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:17.383453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:17.392986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecbe63acee0c48fdb5ae4f3adc7f57a1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:17.412834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecbe63acee0c48fdb5ae4f3adc7f57a1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:17.421781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecbe63acee0c48fdb5ae4f3adc7f57a1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:17.431419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecbe63acee0c48fdb5ae4f3adc7f57a1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:17.432589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecbe63acee0c48fdb5ae4f3adc7f57a1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:17.443229 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1249/2478] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:17 2022] DELETE /identity/v3/users/ecbe63acee0c48fdb5ae4f3adc7f57a1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:17.448552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f402e-e0d5-464e-aa1c-2099eddfe7f3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:17.449171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f402e-e0d5-464e-aa1c-2099eddfe7f3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:17.449636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f402e-e0d5-464e-aa1c-2099eddfe7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:17.500416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6e3f402e-e0d5-464e-aa1c-2099eddfe7f3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:17.508441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e3f402e-e0d5-464e-aa1c-2099eddfe7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:17.613243 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1230/2479] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:17.641221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:17.641221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:17.773588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f01a3-9278-4076-89ec-ebe495158e3f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:17.789294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1827f1-4bae-4ae0-8ae6-fce11ae4aef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:17.798475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1827f1-4bae-4ae0-8ae6-fce11ae4aef2 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:17.798740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1827f1-4bae-4ae0-8ae6-fce11ae4aef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:17.799066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1827f1-4bae-4ae0-8ae6-fce11ae4aef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:17.811502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1827f1-4bae-4ae0-8ae6-fce11ae4aef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:17.813297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1827f1-4bae-4ae0-8ae6-fce11ae4aef2 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:17.825517 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1250/2480] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:18.341889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97c7b2-f099-4f47-a5e0-6c6e31a44689 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:18.343185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97c7b2-f099-4f47-a5e0-6c6e31a44689 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:18.343841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97c7b2-f099-4f47-a5e0-6c6e31a44689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:18.371178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ac97c7b2-f099-4f47-a5e0-6c6e31a44689 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:18.372870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ac97c7b2-f099-4f47-a5e0-6c6e31a44689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:18.498892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1231/2481] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:52:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:18.532055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1827f1-4bae-4ae0-8ae6-fce11ae4aef2 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:18.533688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eb1827f1-4bae-4ae0-8ae6-fce11ae4aef2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:18.673661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1827f1-4bae-4ae0-8ae6-fce11ae4aef2 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:18.679542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16dca5-135d-4300-aa8f-4aea3876998d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:18.680440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16dca5-135d-4300-aa8f-4aea3876998d service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:18.680876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16dca5-135d-4300-aa8f-4aea3876998d service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:18.681323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16dca5-135d-4300-aa8f-4aea3876998d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:18.691441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16dca5-135d-4300-aa8f-4aea3876998d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:18.693510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16dca5-135d-4300-aa8f-4aea3876998d service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:18.704933 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1251/2482] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:52:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 20:52:18.766269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97c7b2-f099-4f47-a5e0-6c6e31a44689 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:18.768005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ac97c7b2-f099-4f47-a5e0-6c6e31a44689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:18.811054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97c7b2-f099-4f47-a5e0-6c6e31a44689 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:18.825922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:18.826793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:18.827261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:18.827766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] PATH_INFO: `/v3/projects/f052606f63d64258a31961e11816a261` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:18.829800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f052606f63d64258a31961e11816a261)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:18.835668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:18.930886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:18.934570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f052606f63d64258a31961e11816a261', 'request_id': 'req-0416590c-a8f3-4f59-b784-4658b5ad636d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:18.957122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f052606f63d64258a31961e11816a261', 'request_id': 'req-0416590c-a8f3-4f59-b784-4658b5ad636d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:18.968119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f052606f63d64258a31961e11816a261', 'request_id': 'req-0416590c-a8f3-4f59-b784-4658b5ad636d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:19.027501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] The token cache is being invalidate because project f052606f63d64258a31961e11816a261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:19.027791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:19.028475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:19.037214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1232/2483] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:18 2022] DELETE /identity/v3/projects/f052606f63d64258a31961e11816a261 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:52:19.038884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16dca5-135d-4300-aa8f-4aea3876998d service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:19.059679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16dca5-135d-4300-aa8f-4aea3876998d service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:19.178247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:19.207362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:19.208175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:19.208457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:19.208730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] PATH_INFO: `/v3/users/730bccefd42b4379af09bd391c6f4baf` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:19.209911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=730bccefd42b4379af09bd391c6f4baf)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:19.213549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:19.341857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730bccefd42b4379af09bd391c6f4baf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:19.382638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730bccefd42b4379af09bd391c6f4baf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:19.403067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730bccefd42b4379af09bd391c6f4baf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:19.414234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730bccefd42b4379af09bd391c6f4baf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:19.416188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a0178a8-601f-46f6-a246-e4e1e8ffca9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730bccefd42b4379af09bd391c6f4baf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:19.431031 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1252/2484] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:19 2022] DELETE /identity/v3/users/730bccefd42b4379af09bd391c6f4baf => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:20.145344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:20.145344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:20.186116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416590c-a8f3-4f59-b784-4658b5ad636d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:20.281496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:20.283602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:20.284053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:20.284488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] PATH_INFO: `/v3/projects/a7cc21bb0a3949d8a989940a07a85f84` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:20.286110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7cc21bb0a3949d8a989940a07a85f84)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:20.289463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:20.375763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7cc21bb0a3949d8a989940a07a85f84', 'request_id': 'req-910114ab-8d6f-4f26-9914-8a1975a34659'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:20.397658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7cc21bb0a3949d8a989940a07a85f84', 'request_id': 'req-910114ab-8d6f-4f26-9914-8a1975a34659'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:20.411087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7cc21bb0a3949d8a989940a07a85f84', 'request_id': 'req-910114ab-8d6f-4f26-9914-8a1975a34659'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:20.426286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] The token cache is being invalidate because project a7cc21bb0a3949d8a989940a07a85f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:20.426665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:20.427587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-910114ab-8d6f-4f26-9914-8a1975a34659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:20.450575 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1233/2485] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:20 2022] DELETE /identity/v3/projects/a7cc21bb0a3949d8a989940a07a85f84 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:20.452868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8857c-be6a-4846-b102-4224658c3c0c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:20.455035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8857c-be6a-4846-b102-4224658c3c0c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:20.455494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8857c-be6a-4846-b102-4224658c3c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:20.505695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-86f8857c-be6a-4846-b102-4224658c3c0c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:20.508036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-86f8857c-be6a-4846-b102-4224658c3c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:20.644517 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1253/2486] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:20.652225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7360-c07c-498d-ad89-cda33209d6f4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:20.652872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7360-c07c-498d-ad89-cda33209d6f4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:20.653336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7360-c07c-498d-ad89-cda33209d6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:20.687034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b5aa7360-c07c-498d-ad89-cda33209d6f4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:20.688845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b5aa7360-c07c-498d-ad89-cda33209d6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:20.705590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8857c-be6a-4846-b102-4224658c3c0c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:20.710566 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1234/2487] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:20.721982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8857c-be6a-4846-b102-4224658c3c0c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:20.728080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e750f7f-8d7a-467a-874f-883edd3be944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:20.729321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e750f7f-8d7a-467a-874f-883edd3be944 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:20.729677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e750f7f-8d7a-467a-874f-883edd3be944 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:20.730021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e750f7f-8d7a-467a-874f-883edd3be944 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:20.730749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e750f7f-8d7a-467a-874f-883edd3be944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:52:20.731124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e750f7f-8d7a-467a-874f-883edd3be944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:20.733090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e750f7f-8d7a-467a-874f-883edd3be944 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:20.759756 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1254/2488] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:52:20.761310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7360-c07c-498d-ad89-cda33209d6f4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:20.778776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7360-c07c-498d-ad89-cda33209d6f4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:20.793790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f046b5-cc81-4e59-bd3c-d9c5614a6acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:20.794406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f046b5-cc81-4e59-bd3c-d9c5614a6acb admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:20.794677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f046b5-cc81-4e59-bd3c-d9c5614a6acb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:20.794936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f046b5-cc81-4e59-bd3c-d9c5614a6acb admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:20.795577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f046b5-cc81-4e59-bd3c-d9c5614a6acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:20.804025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f046b5-cc81-4e59-bd3c-d9c5614a6acb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:20.867151 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1235/2489] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:20 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:20.870963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e750f7f-8d7a-467a-874f-883edd3be944 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:20.886377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e750f7f-8d7a-467a-874f-883edd3be944 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:21.032891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:21.047774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a273eb3-2396-4d40-a05e-8b6e62b099cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:21.048666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a273eb3-2396-4d40-a05e-8b6e62b099cc admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:21.049202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a273eb3-2396-4d40-a05e-8b6e62b099cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:21.049813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a273eb3-2396-4d40-a05e-8b6e62b099cc admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:21.050585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a273eb3-2396-4d40-a05e-8b6e62b099cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:21.053527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a273eb3-2396-4d40-a05e-8b6e62b099cc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:21.101682 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1255/2490] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:20 2022] POST /identity/v3/users => generated 291 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:21.103398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f046b5-cc81-4e59-bd3c-d9c5614a6acb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:21.117184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f046b5-cc81-4e59-bd3c-d9c5614a6acb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:21.128878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8489bcf-879a-48e3-be5a-99d80cfbb14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:21.129855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8489bcf-879a-48e3-be5a-99d80cfbb14e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:21.130840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8489bcf-879a-48e3-be5a-99d80cfbb14e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:21.131524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8489bcf-879a-48e3-be5a-99d80cfbb14e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:21.133058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8489bcf-879a-48e3-be5a-99d80cfbb14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:21.135560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8489bcf-879a-48e3-be5a-99d80cfbb14e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:21.155717 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1236/2491] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:21.159759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a273eb3-2396-4d40-a05e-8b6e62b099cc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:21.175366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a273eb3-2396-4d40-a05e-8b6e62b099cc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:21.185021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a4f2-9265-446b-9e29-98c363f953ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:21.185937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a4f2-9265-446b-9e29-98c363f953ef admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:21.186385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a4f2-9265-446b-9e29-98c363f953ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:21.186811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a4f2-9265-446b-9e29-98c363f953ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:21.188383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a4f2-9265-446b-9e29-98c363f953ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:21.190637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a4f2-9265-446b-9e29-98c363f953ef admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:21.228081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96aead6-ff2f-4071-8d77-156bca593b26 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:21.229246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96aead6-ff2f-4071-8d77-156bca593b26 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:21.229833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96aead6-ff2f-4071-8d77-156bca593b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:21.250278 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1256/2492] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:21.253588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a4f2-9265-446b-9e29-98c363f953ef admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:21.300194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f96aead6-ff2f-4071-8d77-156bca593b26 None None] MFA Rules not processed for user `8bbc891924f84d99aac22518f69e7b99`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:21.302740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a4f2-9265-446b-9e29-98c363f953ef admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:21.304229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f96aead6-ff2f-4071-8d77-156bca593b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:21.317623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba9be8-c4f0-4b09-9e02-7495f3902242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:21.321175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba9be8-c4f0-4b09-9e02-7495f3902242 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:21.322963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba9be8-c4f0-4b09-9e02-7495f3902242 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:21.324250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba9be8-c4f0-4b09-9e02-7495f3902242 admin admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff/users/2d8b0084c7654102b93527cb74465443/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:21.355311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:21.361463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba9be8-c4f0-4b09-9e02-7495f3902242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c5ca0165104167a90e69f6f45fdeff, user_id=2d8b0084c7654102b93527cb74465443, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:21.369834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba9be8-c4f0-4b09-9e02-7495f3902242 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:21.414852 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1257/2493] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:21 2022] PUT /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff/users/2d8b0084c7654102b93527cb74465443/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:21.427372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d6db7-8a77-4367-8b88-b3e5e2936695 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:21.428855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d6db7-8a77-4367-8b88-b3e5e2936695 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:21.429355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d6db7-8a77-4367-8b88-b3e5e2936695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:21.483023 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1237/2494] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:21.509132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-679d6db7-8a77-4367-8b88-b3e5e2936695 None None] MFA Rules not processed for user `2d8b0084c7654102b93527cb74465443`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:21.512859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-679d6db7-8a77-4367-8b88-b3e5e2936695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:21.723307 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1258/2495] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:21.726705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96aead6-ff2f-4071-8d77-156bca593b26 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:21.903046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96aead6-ff2f-4071-8d77-156bca593b26 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:22.162185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549a335-a1c3-4193-8940-41c6ffbb444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:22.162899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549a335-a1c3-4193-8940-41c6ffbb444d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:22.163070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549a335-a1c3-4193-8940-41c6ffbb444d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:22.169271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549a335-a1c3-4193-8940-41c6ffbb444d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:22.171104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549a335-a1c3-4193-8940-41c6ffbb444d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:22.177298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549a335-a1c3-4193-8940-41c6ffbb444d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:22.325913 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1238/2496] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:21 2022] POST /identity/v3/projects => generated 381 bytes in 602 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:22.326879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d6db7-8a77-4367-8b88-b3e5e2936695 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:22.348634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d6db7-8a77-4367-8b88-b3e5e2936695 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:22.504045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0e129-3663-4a08-9321-49162898ad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:22.504915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0e129-3663-4a08-9321-49162898ad12 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:22.505279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0e129-3663-4a08-9321-49162898ad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:22.505652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0e129-3663-4a08-9321-49162898ad12 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:22.506612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0e129-3663-4a08-9321-49162898ad12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:22.514648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0e129-3663-4a08-9321-49162898ad12 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:22.572993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549a335-a1c3-4193-8940-41c6ffbb444d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:22.584422 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1259/2497] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:22 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:52:22.586357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549a335-a1c3-4193-8940-41c6ffbb444d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:22.600280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58058c-0df4-4b64-bf00-95e4d59d1dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:22.601062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58058c-0df4-4b64-bf00-95e4d59d1dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:22.601444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58058c-0df4-4b64-bf00-95e4d59d1dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:22.601979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58058c-0df4-4b64-bf00-95e4d59d1dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:22.603121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58058c-0df4-4b64-bf00-95e4d59d1dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:22.607736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58058c-0df4-4b64-bf00-95e4d59d1dc1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:22.633252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0e129-3663-4a08-9321-49162898ad12 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:22.634213 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1239/2498] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:22.649028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0e129-3663-4a08-9321-49162898ad12 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:22.663459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53adefa-4b8d-49ec-91d8-6a85437f0844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:22.664588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53adefa-4b8d-49ec-91d8-6a85437f0844 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:22.665120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53adefa-4b8d-49ec-91d8-6a85437f0844 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:22.665591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53adefa-4b8d-49ec-91d8-6a85437f0844 admin admin] PATH_INFO: `/v3/projects/af5ed95125704b8f96fa971a9aa70586/users/b3c71ea02dc94b89a79fc78b56bb84b5/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:22.678022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:22.690835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53adefa-4b8d-49ec-91d8-6a85437f0844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af5ed95125704b8f96fa971a9aa70586, user_id=b3c71ea02dc94b89a79fc78b56bb84b5, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:22.694067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53adefa-4b8d-49ec-91d8-6a85437f0844 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:22.721739 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1260/2499] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:22 2022] PUT /identity/v3/projects/af5ed95125704b8f96fa971a9aa70586/users/b3c71ea02dc94b89a79fc78b56bb84b5/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:22.727658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58058c-0df4-4b64-bf00-95e4d59d1dc1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:22.742037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58058c-0df4-4b64-bf00-95e4d59d1dc1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:22.829379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:22.858529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cf032-6b82-4798-a76b-1f7c26bcfc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:22.859256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cf032-6b82-4798-a76b-1f7c26bcfc6f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:22.860026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cf032-6b82-4798-a76b-1f7c26bcfc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:22.860614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cf032-6b82-4798-a76b-1f7c26bcfc6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:22.861869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cf032-6b82-4798-a76b-1f7c26bcfc6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:22.865702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cf032-6b82-4798-a76b-1f7c26bcfc6f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:22.907006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53adefa-4b8d-49ec-91d8-6a85437f0844 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:22.919497 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1240/2500] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:22.921179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53adefa-4b8d-49ec-91d8-6a85437f0844 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:22.935214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c96e97-11c1-4124-a332-738a65f16d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:22.936778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c96e97-11c1-4124-a332-738a65f16d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:22.938098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c96e97-11c1-4124-a332-738a65f16d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:22.939271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c96e97-11c1-4124-a332-738a65f16d2e admin admin] PATH_INFO: `/v3/projects/af5ed95125704b8f96fa971a9aa70586/users/b3c71ea02dc94b89a79fc78b56bb84b5/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:22.944482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c96e97-11c1-4124-a332-738a65f16d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=af5ed95125704b8f96fa971a9aa70586, user_id=b3c71ea02dc94b89a79fc78b56bb84b5, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:22.952570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c96e97-11c1-4124-a332-738a65f16d2e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:22.996446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b458c-3dd8-4ccc-84c0-7ea840c372b1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:22.997117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b458c-3dd8-4ccc-84c0-7ea840c372b1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:22.997574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b458c-3dd8-4ccc-84c0-7ea840c372b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:23.013620 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1261/2501] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:22 2022] PUT /identity/v3/projects/af5ed95125704b8f96fa971a9aa70586/users/b3c71ea02dc94b89a79fc78b56bb84b5/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:23.112057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-210b458c-3dd8-4ccc-84c0-7ea840c372b1 None None] MFA Rules not processed for user `b3c71ea02dc94b89a79fc78b56bb84b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:23.117202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-210b458c-3dd8-4ccc-84c0-7ea840c372b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:23.390323 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1241/2502] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:23.411077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128e584-d205-4a9d-85b9-023f67ac5930 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:23.411360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128e584-d205-4a9d-85b9-023f67ac5930 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:23.411624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128e584-d205-4a9d-85b9-023f67ac5930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:23.508992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c128e584-d205-4a9d-85b9-023f67ac5930 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:23.513411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c128e584-d205-4a9d-85b9-023f67ac5930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:23.771681 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1262/2503] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:23.788110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b458c-3dd8-4ccc-84c0-7ea840c372b1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:23.790527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-210b458c-3dd8-4ccc-84c0-7ea840c372b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:24.069259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b458c-3dd8-4ccc-84c0-7ea840c372b1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:24.103205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5056c-9e61-4136-a6d3-6d419f30feb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:24.103997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5056c-9e61-4136-a6d3-6d419f30feb3 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:24.104447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5056c-9e61-4136-a6d3-6d419f30feb3 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:24.135977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5056c-9e61-4136-a6d3-6d419f30feb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:24.186786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5056c-9e61-4136-a6d3-6d419f30feb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:24.188419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5056c-9e61-4136-a6d3-6d419f30feb3 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:24.252929 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1242/2504] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:52:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:52:25.639259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128e584-d205-4a9d-85b9-023f67ac5930 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:25.641894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c128e584-d205-4a9d-85b9-023f67ac5930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:25.849942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128e584-d205-4a9d-85b9-023f67ac5930 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:25.865645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f96ca-27ec-490b-9f4a-f44d5ed2a844 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:25.866407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f96ca-27ec-490b-9f4a-f44d5ed2a844 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:25.866790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f96ca-27ec-490b-9f4a-f44d5ed2a844 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:25.867161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f96ca-27ec-490b-9f4a-f44d5ed2a844 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:25.884885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f96ca-27ec-490b-9f4a-f44d5ed2a844 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:25.887518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f96ca-27ec-490b-9f4a-f44d5ed2a844 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:25.910494 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1263/2505] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:25.938818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6d2db-7951-497e-a168-fe84b0948b16 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:25.939391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6d2db-7951-497e-a168-fe84b0948b16 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:25.939391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6d2db-7951-497e-a168-fe84b0948b16 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:25.941019 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1243/2506] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:52:25.951882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f96ca-27ec-490b-9f4a-f44d5ed2a844 service nova] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:52:25.969774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f96ca-27ec-490b-9f4a-f44d5ed2a844 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:25.999237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f96ca-27ec-490b-9f4a-f44d5ed2a844 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:26.021753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9244a1-efbf-40c6-b252-551339afb98d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c71ea02dc94b89a79fc78b56bb84b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af5ed95125704b8f96fa971a9aa70586', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:26.022559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9244a1-efbf-40c6-b252-551339afb98d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:26.022945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9244a1-efbf-40c6-b252-551339afb98d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:26.023340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9244a1-efbf-40c6-b252-551339afb98d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:26.030471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9244a1-efbf-40c6-b252-551339afb98d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorizing `identity:get_project(project_id=43c5ca0165104167a90e69f6f45fdeff)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:26.037608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9244a1-efbf-40c6-b252-551339afb98d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:26.047905 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1264/2507] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:52:25 2022] GET /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff => generated 381 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:26.668903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3c290-84a5-47b0-b324-c333b98c5f15 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:26.669460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3c290-84a5-47b0-b324-c333b98c5f15 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:26.669908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3c290-84a5-47b0-b324-c333b98c5f15 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:26.671774 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1244/2508] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:52:26.677289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9244a1-efbf-40c6-b252-551339afb98d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:52:26.679699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2f9244a1-efbf-40c6-b252-551339afb98d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:26.715615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9244a1-efbf-40c6-b252-551339afb98d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:26.732180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9244a1-efbf-40c6-b252-551339afb98d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:26.741338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611728b9-9d94-4a73-8884-b6e33f4773a9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c71ea02dc94b89a79fc78b56bb84b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af5ed95125704b8f96fa971a9aa70586', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:26.742061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611728b9-9d94-4a73-8884-b6e33f4773a9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:26.742516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611728b9-9d94-4a73-8884-b6e33f4773a9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:26.742860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611728b9-9d94-4a73-8884-b6e33f4773a9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:26.747081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611728b9-9d94-4a73-8884-b6e33f4773a9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorizing `identity:get_project(project_id=43c5ca0165104167a90e69f6f45fdeff)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:26.756663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611728b9-9d94-4a73-8884-b6e33f4773a9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:26.777584 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1265/2509] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:52:26 2022] GET /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff => generated 381 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:26.968283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe23b1-eb8e-43fe-a971-5c8a470d1773 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:26.968832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe23b1-eb8e-43fe-a971-5c8a470d1773 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:26.968832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe23b1-eb8e-43fe-a971-5c8a470d1773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:27.061664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9cfe23b1-eb8e-43fe-a971-5c8a470d1773 None None] MFA Rules not processed for user `2d8b0084c7654102b93527cb74465443`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:27.067713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9cfe23b1-eb8e-43fe-a971-5c8a470d1773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:27.341479 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1245/2510] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:27.432037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611728b9-9d94-4a73-8884-b6e33f4773a9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:27.457065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb870-183a-4741-97aa-d46a6bbafbdb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:27.457065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb870-183a-4741-97aa-d46a6bbafbdb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:27.457065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb870-183a-4741-97aa-d46a6bbafbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:27.466492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:27.475647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611728b9-9d94-4a73-8884-b6e33f4773a9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:27.497381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5e8fb870-183a-4741-97aa-d46a6bbafbdb None None] MFA Rules not processed for user `e32a1c53b4be44188b07c768405961ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:27.501767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5e8fb870-183a-4741-97aa-d46a6bbafbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:27.532798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028121e-3296-4578-b2ad-acfe3b6529c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:27.536067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028121e-3296-4578-b2ad-acfe3b6529c1 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:27.539797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028121e-3296-4578-b2ad-acfe3b6529c1 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:27.540156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028121e-3296-4578-b2ad-acfe3b6529c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:27.579361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028121e-3296-4578-b2ad-acfe3b6529c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:27.584085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028121e-3296-4578-b2ad-acfe3b6529c1 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:27.620533 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1266/2511] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:27.686710 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1246/2512] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:28.731996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6051e-d67c-4e33-8ecd-97df4480cd14 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:28.732908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6051e-d67c-4e33-8ecd-97df4480cd14 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:28.733374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6051e-d67c-4e33-8ecd-97df4480cd14 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:28.734938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1267/2513] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:52:28 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:52:28.750919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb870-183a-4741-97aa-d46a6bbafbdb None None] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:52:28.773598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb870-183a-4741-97aa-d46a6bbafbdb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:28.791226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb870-183a-4741-97aa-d46a6bbafbdb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:28.797181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ade16-f662-4227-bf20-828c77949937 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c71ea02dc94b89a79fc78b56bb84b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af5ed95125704b8f96fa971a9aa70586', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:28.797880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ade16-f662-4227-bf20-828c77949937 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:28.798398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ade16-f662-4227-bf20-828c77949937 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:28.798906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ade16-f662-4227-bf20-828c77949937 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:28.800244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ade16-f662-4227-bf20-828c77949937 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorizing `identity:get_project(project_id=43c5ca0165104167a90e69f6f45fdeff)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:28.802924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ade16-f662-4227-bf20-828c77949937 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:28.809782 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1247/2514] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:52:28 2022] GET /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff => generated 381 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:29.076555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70446d72-818b-4e98-a23f-116e1cbfd1f2 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:29.077109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70446d72-818b-4e98-a23f-116e1cbfd1f2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:29.077607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70446d72-818b-4e98-a23f-116e1cbfd1f2 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:29.080947 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1268/2515] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:52:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:52:29.090686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ade16-f662-4227-bf20-828c77949937 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:52:29.115692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ade16-f662-4227-bf20-828c77949937 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:29.151705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ade16-f662-4227-bf20-828c77949937 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:29.157264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054b0c8-3e6e-4659-bf1c-6d6c79351b0c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c71ea02dc94b89a79fc78b56bb84b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af5ed95125704b8f96fa971a9aa70586', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:29.158095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054b0c8-3e6e-4659-bf1c-6d6c79351b0c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:29.158373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054b0c8-3e6e-4659-bf1c-6d6c79351b0c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:29.158639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054b0c8-3e6e-4659-bf1c-6d6c79351b0c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:29.159808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054b0c8-3e6e-4659-bf1c-6d6c79351b0c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorizing `identity:get_project(project_id=43c5ca0165104167a90e69f6f45fdeff)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:29.162172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054b0c8-3e6e-4659-bf1c-6d6c79351b0c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:29.174901 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1248/2516] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:52:29 2022] GET /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff => generated 381 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:29.249361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8013eb-01f8-4253-b709-99d87c0e17bc None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:29.249757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8013eb-01f8-4253-b709-99d87c0e17bc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:29.250221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8013eb-01f8-4253-b709-99d87c0e17bc None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:29.252145 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1269/2517] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:52:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:52:29.257206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054b0c8-3e6e-4659-bf1c-6d6c79351b0c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:52:29.280294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054b0c8-3e6e-4659-bf1c-6d6c79351b0c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:29.304641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054b0c8-3e6e-4659-bf1c-6d6c79351b0c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:29.311941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feedb3-4db7-465e-84a1-cfde35ddcbf9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c71ea02dc94b89a79fc78b56bb84b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af5ed95125704b8f96fa971a9aa70586', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:29.313157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feedb3-4db7-465e-84a1-cfde35ddcbf9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:29.313609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feedb3-4db7-465e-84a1-cfde35ddcbf9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:29.327121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feedb3-4db7-465e-84a1-cfde35ddcbf9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:29.328908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32feedb3-4db7-465e-84a1-cfde35ddcbf9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorizing `identity:get_project(project_id=43c5ca0165104167a90e69f6f45fdeff)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:29.331492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32feedb3-4db7-465e-84a1-cfde35ddcbf9 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:29.333896 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1249/2518] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:52:29 2022] GET /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff => generated 381 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:29.471701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad9c94-3067-4f20-b16e-3a4fa87fe69d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:29.477310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad9c94-3067-4f20-b16e-3a4fa87fe69d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:29.477776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad9c94-3067-4f20-b16e-3a4fa87fe69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:29.549564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b4ad9c94-3067-4f20-b16e-3a4fa87fe69d None None] MFA Rules not processed for user `2d8b0084c7654102b93527cb74465443`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:29.559041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b4ad9c94-3067-4f20-b16e-3a4fa87fe69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:29.600020 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1270/2519] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:30.793028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef880b-8fbb-42e5-a34d-d63ad654331a None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:30.793592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef880b-8fbb-42e5-a34d-d63ad654331a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:30.793592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef880b-8fbb-42e5-a34d-d63ad654331a None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:30.795435 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1250/2520] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:52:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:52:30.800256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad9c94-3067-4f20-b16e-3a4fa87fe69d None None] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:52:30.807855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:30.825653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad9c94-3067-4f20-b16e-3a4fa87fe69d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:30.865514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad9c94-3067-4f20-b16e-3a4fa87fe69d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:30.879526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f550a06-ed2d-4225-8938-c02473ba8699 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c71ea02dc94b89a79fc78b56bb84b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af5ed95125704b8f96fa971a9aa70586', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:30.881156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f550a06-ed2d-4225-8938-c02473ba8699 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:30.899784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f550a06-ed2d-4225-8938-c02473ba8699 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:30.901450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f550a06-ed2d-4225-8938-c02473ba8699 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:30.904974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f550a06-ed2d-4225-8938-c02473ba8699 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorizing `identity:get_project(project_id=43c5ca0165104167a90e69f6f45fdeff)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:30.933065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f550a06-ed2d-4225-8938-c02473ba8699 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:30.936132 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1271/2521] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:52:30 2022] GET /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff => generated 381 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:31.243312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6100-d598-4582-b443-fd01dd71c7ec None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:31.243738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6100-d598-4582-b443-fd01dd71c7ec None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:31.244176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6100-d598-4582-b443-fd01dd71c7ec None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:31.246319 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1251/2522] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:52:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:52:31.251776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f550a06-ed2d-4225-8938-c02473ba8699 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:52:31.302913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f550a06-ed2d-4225-8938-c02473ba8699 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:31.325344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f550a06-ed2d-4225-8938-c02473ba8699 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:31.335611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2af80d-c38f-4eca-bbf0-ba2bf74fb584 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c71ea02dc94b89a79fc78b56bb84b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af5ed95125704b8f96fa971a9aa70586', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:31.336346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2af80d-c38f-4eca-bbf0-ba2bf74fb584 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:31.336850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2af80d-c38f-4eca-bbf0-ba2bf74fb584 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:31.337223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2af80d-c38f-4eca-bbf0-ba2bf74fb584 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:31.339227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2af80d-c38f-4eca-bbf0-ba2bf74fb584 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorizing `identity:get_project(project_id=43c5ca0165104167a90e69f6f45fdeff)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:31.342470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2af80d-c38f-4eca-bbf0-ba2bf74fb584 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:31.345955 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1272/2523] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:52:31 2022] GET /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff => generated 381 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:31.406194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36338ec-601b-4a49-b2b0-72bb2e935f04 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:31.406489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36338ec-601b-4a49-b2b0-72bb2e935f04 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:31.406747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36338ec-601b-4a49-b2b0-72bb2e935f04 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:31.408072 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1252/2524] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:52:31.417285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2af80d-c38f-4eca-bbf0-ba2bf74fb584 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:52:31.438932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2af80d-c38f-4eca-bbf0-ba2bf74fb584 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:31.470371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2af80d-c38f-4eca-bbf0-ba2bf74fb584 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:31.477833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddf5bc-d991-4e65-8e32-1a266e6fc7be tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c71ea02dc94b89a79fc78b56bb84b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af5ed95125704b8f96fa971a9aa70586', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:31.478747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddf5bc-d991-4e65-8e32-1a266e6fc7be tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:31.479401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddf5bc-d991-4e65-8e32-1a266e6fc7be tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:31.479895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddf5bc-d991-4e65-8e32-1a266e6fc7be tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:31.481647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ddf5bc-d991-4e65-8e32-1a266e6fc7be tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorizing `identity:get_project(project_id=43c5ca0165104167a90e69f6f45fdeff)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:31.484960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ddf5bc-d991-4e65-8e32-1a266e6fc7be tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:31.513180 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1273/2525] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:52:31 2022] GET /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff => generated 381 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:31.656102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d8fad-f545-4795-901c-24d561e6da7b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:31.656629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d8fad-f545-4795-901c-24d561e6da7b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:31.657347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d8fad-f545-4795-901c-24d561e6da7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:31.700786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b88d8fad-f545-4795-901c-24d561e6da7b None None] MFA Rules not processed for user `2d8b0084c7654102b93527cb74465443`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:31.707193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b88d8fad-f545-4795-901c-24d561e6da7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:31.734243 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1253/2526] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:33.393636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e03d98-6bd5-498c-ac08-6391074fee2a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:33.402842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e03d98-6bd5-498c-ac08-6391074fee2a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:33.403361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e03d98-6bd5-498c-ac08-6391074fee2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:33.454706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-91e03d98-6bd5-498c-ac08-6391074fee2a None None] MFA Rules not processed for user `180d9b7141ff4792ab7ab19e21a3d7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:33.462246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-91e03d98-6bd5-498c-ac08-6391074fee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:33.648480 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1274/2527] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:33 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:33.654063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5865f-5cc6-4733-a3e3-b526d23fb111 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:33.654995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5865f-5cc6-4733-a3e3-b526d23fb111 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:33.655475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5865f-5cc6-4733-a3e3-b526d23fb111 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:33.657014 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1254/2528] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 20:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 20:52:33.669066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e03d98-6bd5-498c-ac08-6391074fee2a None None] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 20:52:33.694695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e03d98-6bd5-498c-ac08-6391074fee2a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:33.732534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e03d98-6bd5-498c-ac08-6391074fee2a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:33.747346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd3765-7c4d-4c2c-a539-ee887e94db2c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c71ea02dc94b89a79fc78b56bb84b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af5ed95125704b8f96fa971a9aa70586', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:33.748026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd3765-7c4d-4c2c-a539-ee887e94db2c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:33.750165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd3765-7c4d-4c2c-a539-ee887e94db2c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:33.750534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd3765-7c4d-4c2c-a539-ee887e94db2c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:33.752749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd3765-7c4d-4c2c-a539-ee887e94db2c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorizing `identity:get_project(project_id=43c5ca0165104167a90e69f6f45fdeff)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:33.755455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd3765-7c4d-4c2c-a539-ee887e94db2c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:33.762119 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1275/2529] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 20:52:33 2022] GET /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff => generated 381 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:34.424001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5865f-5cc6-4733-a3e3-b526d23fb111 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:34.617079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5865f-5cc6-4733-a3e3-b526d23fb111 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:34.627358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:34.628400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:34.628889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:34.629399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] PATH_INFO: `/v3/users/2d8b0084c7654102b93527cb74465443` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:34.633551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d8b0084c7654102b93527cb74465443)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:34.636379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:34.741382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d8b0084c7654102b93527cb74465443'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:34.752192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d8b0084c7654102b93527cb74465443'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:34.770645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd3765-7c4d-4c2c-a539-ee887e94db2c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:34.775611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-acbd3765-7c4d-4c2c-a539-ee887e94db2c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:34.795107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d8b0084c7654102b93527cb74465443'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:34.804862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:34.814995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd3765-7c4d-4c2c-a539-ee887e94db2c tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:34.821567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d8b0084c7654102b93527cb74465443'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:34.822461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acc4ab79-0549-449e-bc96-c637d8bb2743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d8b0084c7654102b93527cb74465443'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:34.832996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:34.834065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:34.834488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:34.835143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] PATH_INFO: `/v3/users/58842ce40beb4def95f41884b083ec49` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:34.837250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58842ce40beb4def95f41884b083ec49)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:34.841703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:34.845741 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1255/2530] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:34 2022] DELETE /identity/v3/users/2d8b0084c7654102b93527cb74465443 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:34.860633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9b023-6e8b-44ac-afa4-43a99ec9ab59 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:34.862900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9b023-6e8b-44ac-afa4-43a99ec9ab59 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:34.863345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9b023-6e8b-44ac-afa4-43a99ec9ab59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:34.922052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-56f9b023-6e8b-44ac-afa4-43a99ec9ab59 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:34.930173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-56f9b023-6e8b-44ac-afa4-43a99ec9ab59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:34.953958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58842ce40beb4def95f41884b083ec49'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:34.985306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58842ce40beb4def95f41884b083ec49'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:35.000206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58842ce40beb4def95f41884b083ec49'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:35.017055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58842ce40beb4def95f41884b083ec49'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:35.018536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1011feb-dee3-48ab-ad0e-3fee7c52e790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58842ce40beb4def95f41884b083ec49'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:35.032917 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1276/2531] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:34 2022] DELETE /identity/v3/users/58842ce40beb4def95f41884b083ec49 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:52:35.037324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fa553-d011-4b00-9fb0-3622d3bed847 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:35.037605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fa553-d011-4b00-9fb0-3622d3bed847 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:35.037884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fa553-d011-4b00-9fb0-3622d3bed847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:35.085137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1256/2532] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:35.086124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-842fa553-d011-4b00-9fb0-3622d3bed847 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:35.087982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-842fa553-d011-4b00-9fb0-3622d3bed847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:35.149962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:35.156691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9b023-6e8b-44ac-afa4-43a99ec9ab59 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:35.158591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-56f9b023-6e8b-44ac-afa4-43a99ec9ab59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:35.239181 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1277/2533] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:35.251839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fa553-d011-4b00-9fb0-3622d3bed847 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:35.253308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-842fa553-d011-4b00-9fb0-3622d3bed847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:35.348056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9b023-6e8b-44ac-afa4-43a99ec9ab59 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:35.367009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fa553-d011-4b00-9fb0-3622d3bed847 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:35.384228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a8e95-eb71-4de4-ac95-942bdafc2026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:35.384849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a8e95-eb71-4de4-ac95-942bdafc2026 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:35.385180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a8e95-eb71-4de4-ac95-942bdafc2026 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:35.386130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a8e95-eb71-4de4-ac95-942bdafc2026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:35.420594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a8e95-eb71-4de4-ac95-942bdafc2026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:35.424532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a8e95-eb71-4de4-ac95-942bdafc2026 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:35.455614 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1278/2534] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:52:35.497531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562efe79-5cc2-4324-b24d-d80a1ac5ac64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:35.498259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562efe79-5cc2-4324-b24d-d80a1ac5ac64 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:35.498658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562efe79-5cc2-4324-b24d-d80a1ac5ac64 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:35.498870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562efe79-5cc2-4324-b24d-d80a1ac5ac64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:35.543108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562efe79-5cc2-4324-b24d-d80a1ac5ac64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:35.548687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562efe79-5cc2-4324-b24d-d80a1ac5ac64 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:35.603270 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1257/2535] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:36.415719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a8e95-eb71-4de4-ac95-942bdafc2026 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:36.418566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a63a8e95-eb71-4de4-ac95-942bdafc2026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:36.475974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a8e95-eb71-4de4-ac95-942bdafc2026 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:36.499831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:36.500543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:36.500829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:36.501128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] PATH_INFO: `/v3/projects/43c5ca0165104167a90e69f6f45fdeff` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:36.503756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c5ca0165104167a90e69f6f45fdeff)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:36.506944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:36.649017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c5ca0165104167a90e69f6f45fdeff', 'request_id': 'req-eb293afb-cf07-4965-bc4f-3ef50fbdecac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:36.665748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c5ca0165104167a90e69f6f45fdeff', 'request_id': 'req-eb293afb-cf07-4965-bc4f-3ef50fbdecac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:36.683294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c5ca0165104167a90e69f6f45fdeff', 'request_id': 'req-eb293afb-cf07-4965-bc4f-3ef50fbdecac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:36.696935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562efe79-5cc2-4324-b24d-d80a1ac5ac64 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:36.699082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-562efe79-5cc2-4324-b24d-d80a1ac5ac64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:36.709628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] The token cache is being invalidate because project 43c5ca0165104167a90e69f6f45fdeff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:36.710280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:36.711155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:36.713508 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1279/2536] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:36 2022] DELETE /identity/v3/projects/43c5ca0165104167a90e69f6f45fdeff => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:36.721695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:36.725125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:36.731153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562efe79-5cc2-4324-b24d-d80a1ac5ac64 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:36.855795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:36.880807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb293afb-cf07-4965-bc4f-3ef50fbdecac admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:36.896199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:36.900555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:36.901867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:36.903050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] PATH_INFO: `/v3/users/b3c71ea02dc94b89a79fc78b56bb84b5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:36.910178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c71ea02dc94b89a79fc78b56bb84b5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:36.919852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:36.930258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:36.932070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:36.933330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:36.933780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] PATH_INFO: `/v3/projects/ab4db46c126d4607943fef56702f5eaf` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:36.935246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab4db46c126d4607943fef56702f5eaf)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:36.938719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:36.996368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:37.024374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab4db46c126d4607943fef56702f5eaf', 'request_id': 'req-3fbb1803-921d-474d-9eab-757f6ce855ec'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.036428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab4db46c126d4607943fef56702f5eaf', 'request_id': 'req-3fbb1803-921d-474d-9eab-757f6ce855ec'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.047865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab4db46c126d4607943fef56702f5eaf', 'request_id': 'req-3fbb1803-921d-474d-9eab-757f6ce855ec'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.089587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c71ea02dc94b89a79fc78b56bb84b5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.092441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] The token cache is being invalidate because project ab4db46c126d4607943fef56702f5eaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:37.092735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.098424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.105817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c71ea02dc94b89a79fc78b56bb84b5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.108006 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1258/2537] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:36 2022] DELETE /identity/v3/projects/ab4db46c126d4607943fef56702f5eaf => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:37.112421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:37.117188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:37.121907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c71ea02dc94b89a79fc78b56bb84b5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.131138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c71ea02dc94b89a79fc78b56bb84b5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.132113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c72623fd-fde1-49b7-90e6-eee45f7efb23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c71ea02dc94b89a79fc78b56bb84b5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.141194 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1280/2538] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:36 2022] DELETE /identity/v3/users/b3c71ea02dc94b89a79fc78b56bb84b5 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:37.257503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb1803-921d-474d-9eab-757f6ce855ec admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:37.371632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:37.372631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:37.373430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:37.373808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] PATH_INFO: `/v3/users/ef5ff17425f743f5866cafa5dc7e0da7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:37.375336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef5ff17425f743f5866cafa5dc7e0da7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:37.377590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:37.386072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:37.511597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef5ff17425f743f5866cafa5dc7e0da7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.534910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef5ff17425f743f5866cafa5dc7e0da7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.563395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef5ff17425f743f5866cafa5dc7e0da7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.596147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef5ff17425f743f5866cafa5dc7e0da7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.596866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef5ff17425f743f5866cafa5dc7e0da7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:37.614133 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1259/2539] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:37 2022] DELETE /identity/v3/users/ef5ff17425f743f5866cafa5dc7e0da7 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:37.895774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258407d-6c1d-4fea-8460-c38458ae3161 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:37.896241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258407d-6c1d-4fea-8460-c38458ae3161 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:37.896526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258407d-6c1d-4fea-8460-c38458ae3161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:37.932977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b258407d-6c1d-4fea-8460-c38458ae3161 None None] MFA Rules not processed for user `1ddcfe1b0eb5431dbbe4777b8dd77436`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:37.937163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b258407d-6c1d-4fea-8460-c38458ae3161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:38.118325 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1281/2540] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:38.492399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:38.501528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:38.638365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8fb3b-bfaa-49d1-8d90-27f2e70e82c6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:38.649414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:38.652505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:38.656221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:38.656221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] PATH_INFO: `/v3/projects/af5ed95125704b8f96fa971a9aa70586` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:38.656221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af5ed95125704b8f96fa971a9aa70586)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:38.657018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:38.773163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af5ed95125704b8f96fa971a9aa70586', 'request_id': 'req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:38.793636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af5ed95125704b8f96fa971a9aa70586', 'request_id': 'req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:38.809869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af5ed95125704b8f96fa971a9aa70586', 'request_id': 'req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:38.836137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] The token cache is being invalidate because project af5ed95125704b8f96fa971a9aa70586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:38.839596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:38.839596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:38.845232 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1260/2541] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:38 2022] DELETE /identity/v3/projects/af5ed95125704b8f96fa971a9aa70586 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:38.845780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf0501-4660-4912-bebe-2d107008d981 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:38.846066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf0501-4660-4912-bebe-2d107008d981 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:38.846340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf0501-4660-4912-bebe-2d107008d981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:38.905619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f7cf0501-4660-4912-bebe-2d107008d981 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:38.906858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f7cf0501-4660-4912-bebe-2d107008d981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:39.036238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:39.038704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:39.106203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:39.185610 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1282/2542] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:39.187249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3ad73-0305-46d0-8b6a-e3f717341a56 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:39.187531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3ad73-0305-46d0-8b6a-e3f717341a56 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:39.187802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3ad73-0305-46d0-8b6a-e3f717341a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:39.227702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a69ba3-f209-4a7e-a3ed-e001fe6ca984 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:39.227935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b3a3ad73-0305-46d0-8b6a-e3f717341a56 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:39.230210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b3a3ad73-0305-46d0-8b6a-e3f717341a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:39.235633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:39.237602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:39.237920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:39.238193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] PATH_INFO: `/v3/projects/d9987eb652194122b3026f0b65c2b536` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:39.242305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9987eb652194122b3026f0b65c2b536)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:39.246159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:39.260302 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1283/2543] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:39.262459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3ad73-0305-46d0-8b6a-e3f717341a56 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:39.278573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3ad73-0305-46d0-8b6a-e3f717341a56 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:39.296435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916fe66-7643-49a9-aeba-491bced8c409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:39.297975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916fe66-7643-49a9-aeba-491bced8c409 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:39.298336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916fe66-7643-49a9-aeba-491bced8c409 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:39.298612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916fe66-7643-49a9-aeba-491bced8c409 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:39.299334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916fe66-7643-49a9-aeba-491bced8c409 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:52:39.299677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916fe66-7643-49a9-aeba-491bced8c409 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:39.303755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916fe66-7643-49a9-aeba-491bced8c409 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:39.333918 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1284/2544] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:52:39.358696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916fe66-7643-49a9-aeba-491bced8c409 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:39.401919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9987eb652194122b3026f0b65c2b536', 'request_id': 'req-7b5ff6f2-8578-462b-8e01-84b92359a6e8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:39.412574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916fe66-7643-49a9-aeba-491bced8c409 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:39.423727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9987eb652194122b3026f0b65c2b536', 'request_id': 'req-7b5ff6f2-8578-462b-8e01-84b92359a6e8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:39.424089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:39.439648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9987eb652194122b3026f0b65c2b536', 'request_id': 'req-7b5ff6f2-8578-462b-8e01-84b92359a6e8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:39.485136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] The token cache is being invalidate because project d9987eb652194122b3026f0b65c2b536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:39.485915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:39.487696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7b5ff6f2-8578-462b-8e01-84b92359a6e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:39.493064 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1261/2545] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:39 2022] DELETE /identity/v3/projects/d9987eb652194122b3026f0b65c2b536 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:39.498151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0fe3-f559-4145-b1b4-c95e0f69ac80 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:39.498438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0fe3-f559-4145-b1b4-c95e0f69ac80 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:39.498756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0fe3-f559-4145-b1b4-c95e0f69ac80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:39.551516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1e3a0fe3-f559-4145-b1b4-c95e0f69ac80 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:39.552741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1e3a0fe3-f559-4145-b1b4-c95e0f69ac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:39.594220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d54f43-9397-45f0-b2e3-3146a9a8f5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:39.594850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d54f43-9397-45f0-b2e3-3146a9a8f5dd admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:39.595127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d54f43-9397-45f0-b2e3-3146a9a8f5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:39.595704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d54f43-9397-45f0-b2e3-3146a9a8f5dd admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:39.598267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d54f43-9397-45f0-b2e3-3146a9a8f5dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:39.600813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d54f43-9397-45f0-b2e3-3146a9a8f5dd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:39.663030 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1285/2546] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:39 2022] POST /identity/v3/projects => generated 377 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:39.668301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d54f43-9397-45f0-b2e3-3146a9a8f5dd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:39.671230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-88d54f43-9397-45f0-b2e3-3146a9a8f5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:39.708268 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1262/2547] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:39.718782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598d28c-60ba-4bf8-8753-0f5e02d9d8a1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:39.719188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598d28c-60ba-4bf8-8753-0f5e02d9d8a1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:39.719530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598d28c-60ba-4bf8-8753-0f5e02d9d8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:39.770896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9598d28c-60ba-4bf8-8753-0f5e02d9d8a1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:39.772134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9598d28c-60ba-4bf8-8753-0f5e02d9d8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:39.867584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d54f43-9397-45f0-b2e3-3146a9a8f5dd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:39.879869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:39.882072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01966b9-d1d7-4c29-ba9c-bc80aa589f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:39.883405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01966b9-d1d7-4c29-ba9c-bc80aa589f45 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:39.886281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01966b9-d1d7-4c29-ba9c-bc80aa589f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:39.886946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01966b9-d1d7-4c29-ba9c-bc80aa589f45 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:39.888017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01966b9-d1d7-4c29-ba9c-bc80aa589f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:39.890235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01966b9-d1d7-4c29-ba9c-bc80aa589f45 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:39.952969 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1286/2548] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:39 2022] POST /identity/v3/users => generated 289 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:39.953627 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1263/2549] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:39.956284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598d28c-60ba-4bf8-8753-0f5e02d9d8a1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:39.960066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01966b9-d1d7-4c29-ba9c-bc80aa589f45 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:39.974738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598d28c-60ba-4bf8-8753-0f5e02d9d8a1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:39.976501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01966b9-d1d7-4c29-ba9c-bc80aa589f45 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:39.982253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166a616-a87f-4406-9a32-3ff4c5a6b9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:39.982984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166a616-a87f-4406-9a32-3ff4c5a6b9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:39.983391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166a616-a87f-4406-9a32-3ff4c5a6b9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:39.983707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166a616-a87f-4406-9a32-3ff4c5a6b9a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:39.984546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166a616-a87f-4406-9a32-3ff4c5a6b9a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:52:39.985146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166a616-a87f-4406-9a32-3ff4c5a6b9a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:39.987551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166a616-a87f-4406-9a32-3ff4c5a6b9a0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:39.991492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f515694-3ae3-4e08-8eaf-25b019036939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:39.992048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f515694-3ae3-4e08-8eaf-25b019036939 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:39.992324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f515694-3ae3-4e08-8eaf-25b019036939 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:39.992593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f515694-3ae3-4e08-8eaf-25b019036939 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:39.993261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f515694-3ae3-4e08-8eaf-25b019036939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:39.996002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f515694-3ae3-4e08-8eaf-25b019036939 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:40.029300 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1287/2550] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:52:40.029507 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1264/2551] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:40.030815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166a616-a87f-4406-9a32-3ff4c5a6b9a0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:40.031993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f515694-3ae3-4e08-8eaf-25b019036939 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:40.055486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166a616-a87f-4406-9a32-3ff4c5a6b9a0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:40.066473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f515694-3ae3-4e08-8eaf-25b019036939 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:40.077579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac926fd4-7367-4595-917d-f331584f2671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:40.078192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac926fd4-7367-4595-917d-f331584f2671 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:40.078464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac926fd4-7367-4595-917d-f331584f2671 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:40.078733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac926fd4-7367-4595-917d-f331584f2671 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:40.079906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac926fd4-7367-4595-917d-f331584f2671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:40.081558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac926fd4-7367-4595-917d-f331584f2671 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:40.091611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a8c42-e653-4516-a2b1-da745559422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:40.092226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a8c42-e653-4516-a2b1-da745559422c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:40.092499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a8c42-e653-4516-a2b1-da745559422c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:40.092767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a8c42-e653-4516-a2b1-da745559422c admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:40.093480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a8c42-e653-4516-a2b1-da745559422c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:40.096939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a8c42-e653-4516-a2b1-da745559422c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:40.114216 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1288/2552] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:40.115113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac926fd4-7367-4595-917d-f331584f2671 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:40.137182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac926fd4-7367-4595-917d-f331584f2671 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:40.149677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6eca70-0845-4d44-aba6-1b456f06fb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:40.151917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eca70-0845-4d44-aba6-1b456f06fb46 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:40.152200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eca70-0845-4d44-aba6-1b456f06fb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:40.152473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eca70-0845-4d44-aba6-1b456f06fb46 admin admin] PATH_INFO: `/v3/projects/4e067bceee8647ca8398901c42a00357/users/5812447072ef4cf897a4c54cc023a112/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:40.162191 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1265/2553] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:40 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:40.163380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a8c42-e653-4516-a2b1-da745559422c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:40.194082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6eca70-0845-4d44-aba6-1b456f06fb46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e067bceee8647ca8398901c42a00357, user_id=5812447072ef4cf897a4c54cc023a112, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:40.201129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6eca70-0845-4d44-aba6-1b456f06fb46 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:40.231753 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1289/2554] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:40 2022] PUT /identity/v3/projects/4e067bceee8647ca8398901c42a00357/users/5812447072ef4cf897a4c54cc023a112/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:40.245906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9066168-3f4d-481f-91e6-6f89fb14170c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:40.246473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9066168-3f4d-481f-91e6-6f89fb14170c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:40.246922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9066168-3f4d-481f-91e6-6f89fb14170c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:40.326285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a8c42-e653-4516-a2b1-da745559422c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:40.330809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f9066168-3f4d-481f-91e6-6f89fb14170c None None] MFA Rules not processed for user `5812447072ef4cf897a4c54cc023a112`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:40.332141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f9066168-3f4d-481f-91e6-6f89fb14170c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:40.423219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:40.506197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39e234-c258-4bc0-80a5-36a4b374dc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:40.506992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39e234-c258-4bc0-80a5-36a4b374dc57 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:40.509529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39e234-c258-4bc0-80a5-36a4b374dc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:40.510888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39e234-c258-4bc0-80a5-36a4b374dc57 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:40.512758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39e234-c258-4bc0-80a5-36a4b374dc57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:40.518639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39e234-c258-4bc0-80a5-36a4b374dc57 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:40.576665 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1266/2555] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:40 2022] POST /identity/v3/users => generated 292 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:40.580509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39e234-c258-4bc0-80a5-36a4b374dc57 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:40.591635 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1290/2556] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:40.592861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9066168-3f4d-481f-91e6-6f89fb14170c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:40.612202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39e234-c258-4bc0-80a5-36a4b374dc57 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:40.619039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9066168-3f4d-481f-91e6-6f89fb14170c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:40.626371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c86a6-8213-4fdd-8960-571e18456145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:40.629237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c86a6-8213-4fdd-8960-571e18456145 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:40.629751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c86a6-8213-4fdd-8960-571e18456145 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:40.630251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c86a6-8213-4fdd-8960-571e18456145 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:40.630446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456dc18-4c80-455a-857d-8ad5e5d7a160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:40.632529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456dc18-4c80-455a-857d-8ad5e5d7a160 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:40.632806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456dc18-4c80-455a-857d-8ad5e5d7a160 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:40.633182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c86a6-8213-4fdd-8960-571e18456145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:40.633378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456dc18-4c80-455a-857d-8ad5e5d7a160 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:40.634936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456dc18-4c80-455a-857d-8ad5e5d7a160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:40.638663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c86a6-8213-4fdd-8960-571e18456145 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:40.652903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456dc18-4c80-455a-857d-8ad5e5d7a160 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:40.665445 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1267/2557] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:40.667670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c86a6-8213-4fdd-8960-571e18456145 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:40.697011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c86a6-8213-4fdd-8960-571e18456145 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:40.720172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c70c18-37d5-49a1-beea-43e4926dd208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:40.721074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c70c18-37d5-49a1-beea-43e4926dd208 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:40.721398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c70c18-37d5-49a1-beea-43e4926dd208 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:40.721732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c70c18-37d5-49a1-beea-43e4926dd208 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:40.722444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c70c18-37d5-49a1-beea-43e4926dd208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:40.727023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c70c18-37d5-49a1-beea-43e4926dd208 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:40.747660 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1291/2558] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:40 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:52:40.758442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456dc18-4c80-455a-857d-8ad5e5d7a160 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:40.771771 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1268/2559] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:40.781709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c70c18-37d5-49a1-beea-43e4926dd208 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:40.809395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c70c18-37d5-49a1-beea-43e4926dd208 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:40.853535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:40.976773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456dc18-4c80-455a-857d-8ad5e5d7a160 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:40.993154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3588b-84ee-492f-bc89-c6d348e7e45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:40.994118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3588b-84ee-492f-bc89-c6d348e7e45a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:40.994300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b40ece-1167-4005-90ea-12ec1d28c51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:40.994599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3588b-84ee-492f-bc89-c6d348e7e45a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:40.995403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b40ece-1167-4005-90ea-12ec1d28c51b admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:40.997308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3588b-84ee-492f-bc89-c6d348e7e45a admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:40.997633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b40ece-1167-4005-90ea-12ec1d28c51b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:40.998070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b40ece-1167-4005-90ea-12ec1d28c51b admin admin] PATH_INFO: `/v3/projects/e67382792a7d4ccf8100febea9052e1b/users/999bc03f52824573bb3c8bdc6a1179cf/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:40.998949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3588b-84ee-492f-bc89-c6d348e7e45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:41.005632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3588b-84ee-492f-bc89-c6d348e7e45a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:41.049846 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1292/2560] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:40 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:41.054813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3588b-84ee-492f-bc89-c6d348e7e45a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:41.069661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b40ece-1167-4005-90ea-12ec1d28c51b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67382792a7d4ccf8100febea9052e1b, user_id=999bc03f52824573bb3c8bdc6a1179cf, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:41.072579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b40ece-1167-4005-90ea-12ec1d28c51b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:41.073750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3588b-84ee-492f-bc89-c6d348e7e45a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:41.080909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51dea8-b602-46f6-b04d-d9fb5ad90ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:41.081822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51dea8-b602-46f6-b04d-d9fb5ad90ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:41.082180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51dea8-b602-46f6-b04d-d9fb5ad90ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:41.082580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51dea8-b602-46f6-b04d-d9fb5ad90ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:41.083392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae51dea8-b602-46f6-b04d-d9fb5ad90ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:41.085469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae51dea8-b602-46f6-b04d-d9fb5ad90ee4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:41.117449 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1293/2561] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:52:41.122643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51dea8-b602-46f6-b04d-d9fb5ad90ee4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:41.125096 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1269/2562] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:40 2022] PUT /identity/v3/projects/e67382792a7d4ccf8100febea9052e1b/users/999bc03f52824573bb3c8bdc6a1179cf/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:52:41.130086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be5433-7d8d-4ca7-8f29-4d814b0d8068 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:41.130372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be5433-7d8d-4ca7-8f29-4d814b0d8068 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:41.130643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be5433-7d8d-4ca7-8f29-4d814b0d8068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:41.152300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51dea8-b602-46f6-b04d-d9fb5ad90ee4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:41.175784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a6be5433-7d8d-4ca7-8f29-4d814b0d8068 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:41.177126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a6be5433-7d8d-4ca7-8f29-4d814b0d8068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:41.349090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0cfb8-bc59-4347-a050-ccdbb5b9f463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:41.349953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0cfb8-bc59-4347-a050-ccdbb5b9f463 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:41.350414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0cfb8-bc59-4347-a050-ccdbb5b9f463 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:41.350779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0cfb8-bc59-4347-a050-ccdbb5b9f463 admin admin] PATH_INFO: `/v3/projects/5eef5e00245e421eaef1580b5373126c/users/c240bffb09b049adb7c7acc452aa37d3/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:41.371839 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1270/2563] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:41.396904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0cfb8-bc59-4347-a050-ccdbb5b9f463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eef5e00245e421eaef1580b5373126c, user_id=c240bffb09b049adb7c7acc452aa37d3, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:41.406849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0cfb8-bc59-4347-a050-ccdbb5b9f463 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:41.416102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be5433-7d8d-4ca7-8f29-4d814b0d8068 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:41.417790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a6be5433-7d8d-4ca7-8f29-4d814b0d8068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:41.460830 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1294/2564] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:41 2022] PUT /identity/v3/projects/5eef5e00245e421eaef1580b5373126c/users/c240bffb09b049adb7c7acc452aa37d3/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:41.475345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0cfb8-bc59-4347-a050-ccdbb5b9f463 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:41.481061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:41.587644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be5433-7d8d-4ca7-8f29-4d814b0d8068 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:41.609473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0cfb8-bc59-4347-a050-ccdbb5b9f463 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:41.625779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d4893-9f1c-4083-a027-76b1a2c84675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:41.626374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d4893-9f1c-4083-a027-76b1a2c84675 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:41.626650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d4893-9f1c-4083-a027-76b1a2c84675 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:41.626916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d4893-9f1c-4083-a027-76b1a2c84675 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:41.627575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d4893-9f1c-4083-a027-76b1a2c84675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:41.629514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d4893-9f1c-4083-a027-76b1a2c84675 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:41.652432 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1295/2565] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:41.658361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d4893-9f1c-4083-a027-76b1a2c84675 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:41.741783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d4893-9f1c-4083-a027-76b1a2c84675 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:41.769498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab53a8f-3a11-4d78-b25b-bfef8a2bef12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:41.774209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab53a8f-3a11-4d78-b25b-bfef8a2bef12 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:41.776792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeab754-3f48-4b1b-8257-e70014d31ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:41.777265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab53a8f-3a11-4d78-b25b-bfef8a2bef12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:41.777506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeab754-3f48-4b1b-8257-e70014d31ecd service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:41.777783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeab754-3f48-4b1b-8257-e70014d31ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:41.778060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeab754-3f48-4b1b-8257-e70014d31ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:41.778833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab53a8f-3a11-4d78-b25b-bfef8a2bef12 admin admin] PATH_INFO: `/v3/projects/5eef5e00245e421eaef1580b5373126c/users/c240bffb09b049adb7c7acc452aa37d3/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:41.786181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab53a8f-3a11-4d78-b25b-bfef8a2bef12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eef5e00245e421eaef1580b5373126c, user_id=c240bffb09b049adb7c7acc452aa37d3, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:41.794896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeab754-3f48-4b1b-8257-e70014d31ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:41.796523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeab754-3f48-4b1b-8257-e70014d31ecd service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:41.796970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab53a8f-3a11-4d78-b25b-bfef8a2bef12 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:41.861712 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1271/2566] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:41.926305 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1296/2567] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:41 2022] PUT /identity/v3/projects/5eef5e00245e421eaef1580b5373126c/users/c240bffb09b049adb7c7acc452aa37d3/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:41.929778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb64e0-4595-4f4a-b523-f7764abfcb9a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:41.930151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb64e0-4595-4f4a-b523-f7764abfcb9a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:41.930512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb64e0-4595-4f4a-b523-f7764abfcb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:41.998484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:42.025311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2ffb64e0-4595-4f4a-b523-f7764abfcb9a None None] MFA Rules not processed for user `c240bffb09b049adb7c7acc452aa37d3`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:42.026737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2ffb64e0-4595-4f4a-b523-f7764abfcb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:42.254892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1272/2568] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:42.267823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab53a8f-3a11-4d78-b25b-bfef8a2bef12 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:42.270236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6ab53a8f-3a11-4d78-b25b-bfef8a2bef12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:42.442520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab53a8f-3a11-4d78-b25b-bfef8a2bef12 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:42.450507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02085a-330e-4aca-92db-dffe73ddfec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:42.451251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02085a-330e-4aca-92db-dffe73ddfec3 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:42.451655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02085a-330e-4aca-92db-dffe73ddfec3 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:42.451993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02085a-330e-4aca-92db-dffe73ddfec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:42.468389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02085a-330e-4aca-92db-dffe73ddfec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:42.471993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02085a-330e-4aca-92db-dffe73ddfec3 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:42.489476 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1297/2569] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:43.334526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb64e0-4595-4f4a-b523-f7764abfcb9a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:43.443322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb64e0-4595-4f4a-b523-f7764abfcb9a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:43.449563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:43.450769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:43.451148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:43.451516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] PATH_INFO: `/v3/users/5812447072ef4cf897a4c54cc023a112` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:43.452892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5812447072ef4cf897a4c54cc023a112)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:43.454953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:43.521585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5812447072ef4cf897a4c54cc023a112'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:43.531587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5812447072ef4cf897a4c54cc023a112'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:43.548391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5812447072ef4cf897a4c54cc023a112'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:43.577703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5812447072ef4cf897a4c54cc023a112'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:43.583060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5812447072ef4cf897a4c54cc023a112'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:43.596603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035b506-7d5f-46ba-9c63-1c3171fa73bb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:43.597187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035b506-7d5f-46ba-9c63-1c3171fa73bb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:43.597187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035b506-7d5f-46ba-9c63-1c3171fa73bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:43.599971 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1273/2570] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:43 2022] DELETE /identity/v3/users/5812447072ef4cf897a4c54cc023a112 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:43.619740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1035b506-7d5f-46ba-9c63-1c3171fa73bb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:43.621339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1035b506-7d5f-46ba-9c63-1c3171fa73bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:43.693477 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1298/2571] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:43.732954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:43.737393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:43.786085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:43.863719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2094eab-2db6-457d-a832-a13c8b8d926a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:43.867304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62b709-b1e6-4e07-9e29-06eb2dd79172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:43.868772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62b709-b1e6-4e07-9e29-06eb2dd79172 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:43.868772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62b709-b1e6-4e07-9e29-06eb2dd79172 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:43.868772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62b709-b1e6-4e07-9e29-06eb2dd79172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:43.880154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62b709-b1e6-4e07-9e29-06eb2dd79172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:43.882790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62b709-b1e6-4e07-9e29-06eb2dd79172 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:43.896868 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1274/2572] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:45.906755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035b506-7d5f-46ba-9c63-1c3171fa73bb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:45.908963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1035b506-7d5f-46ba-9c63-1c3171fa73bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:45.938818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035b506-7d5f-46ba-9c63-1c3171fa73bb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:45.953802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:45.954758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:45.956056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:45.956619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] PATH_INFO: `/v3/projects/4e067bceee8647ca8398901c42a00357` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:45.958260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e067bceee8647ca8398901c42a00357)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:45.964956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:45.977256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:46.109232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e067bceee8647ca8398901c42a00357', 'request_id': 'req-c6e85bd2-7890-4a5b-bd25-453a791933b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:46.129571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e067bceee8647ca8398901c42a00357', 'request_id': 'req-c6e85bd2-7890-4a5b-bd25-453a791933b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:46.139097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e067bceee8647ca8398901c42a00357', 'request_id': 'req-c6e85bd2-7890-4a5b-bd25-453a791933b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:46.168194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] The token cache is being invalidate because project 4e067bceee8647ca8398901c42a00357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:46.168522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:46.171428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:46.180778 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1299/2573] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:45 2022] DELETE /identity/v3/projects/4e067bceee8647ca8398901c42a00357 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:46.182023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62b709-b1e6-4e07-9e29-06eb2dd79172 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:46.202013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62b709-b1e6-4e07-9e29-06eb2dd79172 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:46.311417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:46.312444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:46.312898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:46.313382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] PATH_INFO: `/v3/users/c240bffb09b049adb7c7acc452aa37d3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:46.315384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c240bffb09b049adb7c7acc452aa37d3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:46.317649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:46.405267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c240bffb09b049adb7c7acc452aa37d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:46.415280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c240bffb09b049adb7c7acc452aa37d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:46.438066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c240bffb09b049adb7c7acc452aa37d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:46.448921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c240bffb09b049adb7c7acc452aa37d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:46.449810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-57628292-6610-45f1-8de1-b5ba5bbda813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c240bffb09b049adb7c7acc452aa37d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:46.460392 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1275/2574] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:52:46 2022] DELETE /identity/v3/users/c240bffb09b049adb7c7acc452aa37d3 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:47.230987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:47.232553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:47.263246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e85bd2-7890-4a5b-bd25-453a791933b3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:47.399923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:47.401000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:47.403304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:47.403608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] PATH_INFO: `/v3/projects/5eef5e00245e421eaef1580b5373126c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:47.406393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eef5e00245e421eaef1580b5373126c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:47.409914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:47.538989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eef5e00245e421eaef1580b5373126c', 'request_id': 'req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:47.565146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eef5e00245e421eaef1580b5373126c', 'request_id': 'req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:47.576056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eef5e00245e421eaef1580b5373126c', 'request_id': 'req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:47.605948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] The token cache is being invalidate because project 5eef5e00245e421eaef1580b5373126c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:52:47.606331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:47.608568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1c08d05f-5ea4-4c89-8f0f-04eb030bf279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:52:47.611513 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1300/2575] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:47 2022] DELETE /identity/v3/projects/5eef5e00245e421eaef1580b5373126c => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:47.616616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0490a4-3f45-4740-8f5c-97576c01f732 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:47.617028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0490a4-3f45-4740-8f5c-97576c01f732 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:47.617261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0490a4-3f45-4740-8f5c-97576c01f732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:47.647954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5e0490a4-3f45-4740-8f5c-97576c01f732 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:47.649308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5e0490a4-3f45-4740-8f5c-97576c01f732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:47.716977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:47.789273 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1276/2576] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:47.796981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e37c9-8f90-47d6-909a-dfd64d9a57ec None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:47.797442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e37c9-8f90-47d6-909a-dfd64d9a57ec None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:47.797720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e37c9-8f90-47d6-909a-dfd64d9a57ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:47.834655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-de2e37c9-8f90-47d6-909a-dfd64d9a57ec None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:47.835930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-de2e37c9-8f90-47d6-909a-dfd64d9a57ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:47.845520 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1301/2577] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:47.850059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0490a4-3f45-4740-8f5c-97576c01f732 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:47.860432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0490a4-3f45-4740-8f5c-97576c01f732 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:47.865316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e6888-a0da-4c31-b871-f4a27099467c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:47.865996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e6888-a0da-4c31-b871-f4a27099467c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:47.866340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e6888-a0da-4c31-b871-f4a27099467c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:47.866612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e6888-a0da-4c31-b871-f4a27099467c admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:47.867353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e6888-a0da-4c31-b871-f4a27099467c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:52:47.867730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e6888-a0da-4c31-b871-f4a27099467c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:47.869611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e6888-a0da-4c31-b871-f4a27099467c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:47.884382 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1277/2578] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:52:47.889404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e37c9-8f90-47d6-909a-dfd64d9a57ec None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:47.899375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e37c9-8f90-47d6-909a-dfd64d9a57ec None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:47.904877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37262b-4cc3-4df1-a1be-5b38057fddef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:47.905533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37262b-4cc3-4df1-a1be-5b38057fddef admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:47.905805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37262b-4cc3-4df1-a1be-5b38057fddef admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:47.906072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37262b-4cc3-4df1-a1be-5b38057fddef admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:47.906755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37262b-4cc3-4df1-a1be-5b38057fddef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:47.908630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37262b-4cc3-4df1-a1be-5b38057fddef admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:47.928811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:47.958907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1302/2579] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:47 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:47.960432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e6888-a0da-4c31-b871-f4a27099467c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:47.978129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e6888-a0da-4c31-b871-f4a27099467c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:48.081575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ef7b6-4fda-4114-b3e8-683fe1b9b1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:48.083139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ef7b6-4fda-4114-b3e8-683fe1b9b1ad admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:48.085375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ef7b6-4fda-4114-b3e8-683fe1b9b1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:48.085828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ef7b6-4fda-4114-b3e8-683fe1b9b1ad admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:48.087151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ef7b6-4fda-4114-b3e8-683fe1b9b1ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:48.090258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ef7b6-4fda-4114-b3e8-683fe1b9b1ad admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:48.135506 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1278/2580] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:47 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:48.136348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37262b-4cc3-4df1-a1be-5b38057fddef admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:48.150902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37262b-4cc3-4df1-a1be-5b38057fddef admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:48.156164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac655eb-b236-4d44-b045-f8dca7c5955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:48.157145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac655eb-b236-4d44-b045-f8dca7c5955d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:48.157599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac655eb-b236-4d44-b045-f8dca7c5955d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:48.158142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac655eb-b236-4d44-b045-f8dca7c5955d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:48.158948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac655eb-b236-4d44-b045-f8dca7c5955d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:48.160542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac655eb-b236-4d44-b045-f8dca7c5955d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:48.182222 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1303/2581] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:48.186623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ef7b6-4fda-4114-b3e8-683fe1b9b1ad admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:48.200942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ef7b6-4fda-4114-b3e8-683fe1b9b1ad admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:48.209136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe5c8e-f29d-4f83-91cd-610fe0322184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:48.209969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe5c8e-f29d-4f83-91cd-610fe0322184 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:48.210327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe5c8e-f29d-4f83-91cd-610fe0322184 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:48.210715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe5c8e-f29d-4f83-91cd-610fe0322184 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:48.211431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe5c8e-f29d-4f83-91cd-610fe0322184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:48.213597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe5c8e-f29d-4f83-91cd-610fe0322184 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:48.229304 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1279/2582] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:48.233784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac655eb-b236-4d44-b045-f8dca7c5955d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:48.248080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac655eb-b236-4d44-b045-f8dca7c5955d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:48.256622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfb0d3-9ea5-4346-89cb-a977d63dba0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:48.257588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfb0d3-9ea5-4346-89cb-a977d63dba0c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:48.258035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfb0d3-9ea5-4346-89cb-a977d63dba0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:48.258455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfb0d3-9ea5-4346-89cb-a977d63dba0c admin admin] PATH_INFO: `/v3/projects/b2227fe5e2aa4eddba19afff48b1d11a/users/e92c87e9b9c840e6a4bc73ee0593a2c6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:48.287264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfb0d3-9ea5-4346-89cb-a977d63dba0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2227fe5e2aa4eddba19afff48b1d11a, user_id=e92c87e9b9c840e6a4bc73ee0593a2c6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:48.289950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfb0d3-9ea5-4346-89cb-a977d63dba0c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:48.321005 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1304/2583] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:48 2022] PUT /identity/v3/projects/b2227fe5e2aa4eddba19afff48b1d11a/users/e92c87e9b9c840e6a4bc73ee0593a2c6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:48.400673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e33bc2-5077-42a2-9d7a-34a93e6bd9b4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:48.400955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e33bc2-5077-42a2-9d7a-34a93e6bd9b4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:48.401936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e33bc2-5077-42a2-9d7a-34a93e6bd9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:48.442496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-57e33bc2-5077-42a2-9d7a-34a93e6bd9b4 None None] MFA Rules not processed for user `e92c87e9b9c840e6a4bc73ee0593a2c6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:48.444859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-57e33bc2-5077-42a2-9d7a-34a93e6bd9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:48.536249 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1280/2584] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:48.540805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfb0d3-9ea5-4346-89cb-a977d63dba0c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:48.646605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfb0d3-9ea5-4346-89cb-a977d63dba0c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:48.652860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c944a72-8ea4-4454-bb2e-d9911a2443bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:48.653838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c944a72-8ea4-4454-bb2e-d9911a2443bf admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:48.654382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c944a72-8ea4-4454-bb2e-d9911a2443bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:48.654800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c944a72-8ea4-4454-bb2e-d9911a2443bf admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:48.655678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c944a72-8ea4-4454-bb2e-d9911a2443bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:48.658112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c944a72-8ea4-4454-bb2e-d9911a2443bf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:48.708730 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1305/2585] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:48 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:48.712765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e33bc2-5077-42a2-9d7a-34a93e6bd9b4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:48.729667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e33bc2-5077-42a2-9d7a-34a93e6bd9b4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:48.789100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:48.911374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b20572-e603-4a9d-89b6-df14e0abe801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:48.911825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b20572-e603-4a9d-89b6-df14e0abe801 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:48.912096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b20572-e603-4a9d-89b6-df14e0abe801 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:48.912362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b20572-e603-4a9d-89b6-df14e0abe801 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:48.916389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b20572-e603-4a9d-89b6-df14e0abe801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:48.919175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b20572-e603-4a9d-89b6-df14e0abe801 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:48.968774 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1281/2586] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:48 2022] POST /identity/v3/users => generated 295 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:52:48.970271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c944a72-8ea4-4454-bb2e-d9911a2443bf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:48.988022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c944a72-8ea4-4454-bb2e-d9911a2443bf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:48.996710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fcfb1-5ac1-44d7-a4c7-3044ef03b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:48.998567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fcfb1-5ac1-44d7-a4c7-3044ef03b431 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:49.001950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fcfb1-5ac1-44d7-a4c7-3044ef03b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:49.002279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fcfb1-5ac1-44d7-a4c7-3044ef03b431 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:49.003363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fcfb1-5ac1-44d7-a4c7-3044ef03b431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:49.006325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fcfb1-5ac1-44d7-a4c7-3044ef03b431 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:49.037070 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1306/2587] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:49.039756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b20572-e603-4a9d-89b6-df14e0abe801 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:49.063306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b20572-e603-4a9d-89b6-df14e0abe801 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:49.071241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82084d-ef56-497b-9ade-999e8edbf1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:49.071892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82084d-ef56-497b-9ade-999e8edbf1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:49.072616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82084d-ef56-497b-9ade-999e8edbf1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:49.072616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82084d-ef56-497b-9ade-999e8edbf1a9 admin admin] PATH_INFO: `/v3/projects/6754d0bbe3d44f868289bfbbe6af6e67/users/54e203121335401c9d746b54d1a68b00/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:49.112346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82084d-ef56-497b-9ade-999e8edbf1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6754d0bbe3d44f868289bfbbe6af6e67, user_id=54e203121335401c9d746b54d1a68b00, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:49.115257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82084d-ef56-497b-9ade-999e8edbf1a9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:49.154956 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1282/2588] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:49 2022] PUT /identity/v3/projects/6754d0bbe3d44f868289bfbbe6af6e67/users/54e203121335401c9d746b54d1a68b00/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:49.159273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fcfb1-5ac1-44d7-a4c7-3044ef03b431 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:49.184089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fcfb1-5ac1-44d7-a4c7-3044ef03b431 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:49.300834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65d99d-32c0-4ee6-9258-81c359a9bc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:49.303446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65d99d-32c0-4ee6-9258-81c359a9bc2c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:49.303726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65d99d-32c0-4ee6-9258-81c359a9bc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:49.304129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65d99d-32c0-4ee6-9258-81c359a9bc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:49.305331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65d99d-32c0-4ee6-9258-81c359a9bc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:49.308379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65d99d-32c0-4ee6-9258-81c359a9bc2c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:49.335922 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1307/2589] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:49.340725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82084d-ef56-497b-9ade-999e8edbf1a9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:49.351623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82084d-ef56-497b-9ade-999e8edbf1a9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:49.357534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66680cac-e120-44e6-8e9a-739bb90d955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:49.358225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680cac-e120-44e6-8e9a-739bb90d955b admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:49.358539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680cac-e120-44e6-8e9a-739bb90d955b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:49.358869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680cac-e120-44e6-8e9a-739bb90d955b admin admin] PATH_INFO: `/v3/projects/6754d0bbe3d44f868289bfbbe6af6e67/users/54e203121335401c9d746b54d1a68b00/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:49.360831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66680cac-e120-44e6-8e9a-739bb90d955b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6754d0bbe3d44f868289bfbbe6af6e67, user_id=54e203121335401c9d746b54d1a68b00, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:49.363578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66680cac-e120-44e6-8e9a-739bb90d955b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:49.376758 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1283/2590] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:49 2022] PUT /identity/v3/projects/6754d0bbe3d44f868289bfbbe6af6e67/users/54e203121335401c9d746b54d1a68b00/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:52:49.387394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe2ba-66b2-42c2-8913-1c9673e2521b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:49.387675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe2ba-66b2-42c2-8913-1c9673e2521b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:49.387944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe2ba-66b2-42c2-8913-1c9673e2521b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:49.421524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-013fe2ba-66b2-42c2-8913-1c9673e2521b None None] MFA Rules not processed for user `54e203121335401c9d746b54d1a68b00`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:49.422737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-013fe2ba-66b2-42c2-8913-1c9673e2521b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:49.444071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:49.497607 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1308/2591] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:52:49.510406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9bf52-fac8-470a-ad61-df311c9c5703 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:49.511211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9bf52-fac8-470a-ad61-df311c9c5703 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:49.511481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9bf52-fac8-470a-ad61-df311c9c5703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:49.530972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7db9bf52-fac8-470a-ad61-df311c9c5703 None None] MFA Rules not processed for user `e92c87e9b9c840e6a4bc73ee0593a2c6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:49.532175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7db9bf52-fac8-470a-ad61-df311c9c5703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:49.604930 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1284/2592] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:49.616241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fe2ba-66b2-42c2-8913-1c9673e2521b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:49.618061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-013fe2ba-66b2-42c2-8913-1c9673e2521b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:49.708307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fe2ba-66b2-42c2-8913-1c9673e2521b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:49.715028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aeeda-dd32-4bad-98e0-7951f6353887 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:49.715647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aeeda-dd32-4bad-98e0-7951f6353887 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:49.715941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aeeda-dd32-4bad-98e0-7951f6353887 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:49.716214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aeeda-dd32-4bad-98e0-7951f6353887 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:49.725241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aeeda-dd32-4bad-98e0-7951f6353887 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:49.727160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aeeda-dd32-4bad-98e0-7951f6353887 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:49.738476 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1309/2593] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:52:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:52:54.002342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f6e1-6dc3-4f63-9990-bbe3d19d60cc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:54.003036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f6e1-6dc3-4f63-9990-bbe3d19d60cc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:54.003036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f6e1-6dc3-4f63-9990-bbe3d19d60cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:54.103258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b811f6e1-6dc3-4f63-9990-bbe3d19d60cc None None] MFA Rules not processed for user `999bc03f52824573bb3c8bdc6a1179cf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:54.108911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b811f6e1-6dc3-4f63-9990-bbe3d19d60cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:54.266492 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1285/2594] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:52:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:54.267720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aeeda-dd32-4bad-98e0-7951f6353887 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:54.269211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2c6aeeda-dd32-4bad-98e0-7951f6353887 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:54.352658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aeeda-dd32-4bad-98e0-7951f6353887 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:54.358564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784cd3b-69af-48e8-9c40-2111ef936de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:54.359458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784cd3b-69af-48e8-9c40-2111ef936de2 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:54.359792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784cd3b-69af-48e8-9c40-2111ef936de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:54.360119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784cd3b-69af-48e8-9c40-2111ef936de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:54.360955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784cd3b-69af-48e8-9c40-2111ef936de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:54.363174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784cd3b-69af-48e8-9c40-2111ef936de2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:54.405081 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1310/2595] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:52:54 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:54.406443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811f6e1-6dc3-4f63-9990-bbe3d19d60cc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:54.420528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811f6e1-6dc3-4f63-9990-bbe3d19d60cc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:54.440989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:54.527614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443335ae-c9a4-4763-89fc-75b32a89cdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:54.528434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443335ae-c9a4-4763-89fc-75b32a89cdd9 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:54.528789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443335ae-c9a4-4763-89fc-75b32a89cdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:54.529261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443335ae-c9a4-4763-89fc-75b32a89cdd9 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:54.529984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443335ae-c9a4-4763-89fc-75b32a89cdd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:54.533357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443335ae-c9a4-4763-89fc-75b32a89cdd9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:54.565509 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1286/2596] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:52:54 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:52:54.569623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784cd3b-69af-48e8-9c40-2111ef936de2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:54.581705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784cd3b-69af-48e8-9c40-2111ef936de2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:54.587950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572b3ca-3610-46d2-a41f-202070e12c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:54.588572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572b3ca-3610-46d2-a41f-202070e12c1b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:54.588951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572b3ca-3610-46d2-a41f-202070e12c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:54.589439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572b3ca-3610-46d2-a41f-202070e12c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:54.590362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572b3ca-3610-46d2-a41f-202070e12c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:54.592096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572b3ca-3610-46d2-a41f-202070e12c1b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:54.609007 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1311/2597] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:52:54.610154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443335ae-c9a4-4763-89fc-75b32a89cdd9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:54.622204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443335ae-c9a4-4763-89fc-75b32a89cdd9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:54.627452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a32d19-3b33-4962-b0b6-402bbcf7b174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:54.628150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a32d19-3b33-4962-b0b6-402bbcf7b174 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:54.628484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a32d19-3b33-4962-b0b6-402bbcf7b174 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:54.629377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a32d19-3b33-4962-b0b6-402bbcf7b174 admin admin] PATH_INFO: `/v3/projects/7544347d450845ef8157b63b71c83959/users/6435555fee024871b05e04d4a0a7cd4f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:54.648999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a32d19-3b33-4962-b0b6-402bbcf7b174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7544347d450845ef8157b63b71c83959, user_id=6435555fee024871b05e04d4a0a7cd4f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:54.651716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a32d19-3b33-4962-b0b6-402bbcf7b174 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:54.695666 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1287/2598] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:54 2022] PUT /identity/v3/projects/7544347d450845ef8157b63b71c83959/users/6435555fee024871b05e04d4a0a7cd4f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:54.697191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572b3ca-3610-46d2-a41f-202070e12c1b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:54.709538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572b3ca-3610-46d2-a41f-202070e12c1b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:54.740507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:54.866469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1c8c5-5e93-43db-995e-ecd540b5b7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:54.870278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1c8c5-5e93-43db-995e-ecd540b5b7fa admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:54.871790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1c8c5-5e93-43db-995e-ecd540b5b7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:54.872097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1c8c5-5e93-43db-995e-ecd540b5b7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:54.872966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1c8c5-5e93-43db-995e-ecd540b5b7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:54.878693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1c8c5-5e93-43db-995e-ecd540b5b7fa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:54.905586 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1312/2599] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:52:54.909419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a32d19-3b33-4962-b0b6-402bbcf7b174 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:54.922928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a32d19-3b33-4962-b0b6-402bbcf7b174 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:54.928007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dfcd4-09ef-41f3-bc12-1b0b2707df16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:54.928738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dfcd4-09ef-41f3-bc12-1b0b2707df16 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:54.929135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dfcd4-09ef-41f3-bc12-1b0b2707df16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:54.929573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dfcd4-09ef-41f3-bc12-1b0b2707df16 admin admin] PATH_INFO: `/v3/projects/7544347d450845ef8157b63b71c83959/users/6435555fee024871b05e04d4a0a7cd4f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:54.931968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01dfcd4-09ef-41f3-bc12-1b0b2707df16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7544347d450845ef8157b63b71c83959, user_id=6435555fee024871b05e04d4a0a7cd4f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:54.935158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01dfcd4-09ef-41f3-bc12-1b0b2707df16 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:54.949502 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1288/2600] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:52:54 2022] PUT /identity/v3/projects/7544347d450845ef8157b63b71c83959/users/6435555fee024871b05e04d4a0a7cd4f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:52:58.880443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4a490-ea50-4ee0-9c6c-2573a262289a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:58.881413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4a490-ea50-4ee0-9c6c-2573a262289a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:58.881860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4a490-ea50-4ee0-9c6c-2573a262289a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:58.938269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-01e4a490-ea50-4ee0-9c6c-2573a262289a None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:52:58.939522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-01e4a490-ea50-4ee0-9c6c-2573a262289a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:59.137991 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1313/2601] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:52:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:52:59.156199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dfcd4-09ef-41f3-bc12-1b0b2707df16 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:52:59.157955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f01dfcd4-09ef-41f3-bc12-1b0b2707df16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:52:59.349101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dfcd4-09ef-41f3-bc12-1b0b2707df16 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:52:59.365839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b3fcb-377f-420b-8fdb-35256aeb5f17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:52:59.367611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b3fcb-377f-420b-8fdb-35256aeb5f17 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:52:59.367611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b3fcb-377f-420b-8fdb-35256aeb5f17 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:52:59.367611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b3fcb-377f-420b-8fdb-35256aeb5f17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:52:59.387024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b3fcb-377f-420b-8fdb-35256aeb5f17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:52:59.392499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b3fcb-377f-420b-8fdb-35256aeb5f17 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:52:59.404665 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1289/2602] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:52:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:53:01.460584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4a490-ea50-4ee0-9c6c-2573a262289a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:01.462100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-01e4a490-ea50-4ee0-9c6c-2573a262289a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:01.650389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4a490-ea50-4ee0-9c6c-2573a262289a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:01.658562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0368d4-4591-4190-affe-0d78dd2be8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:01.659249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0368d4-4591-4190-affe-0d78dd2be8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:01.660678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0368d4-4591-4190-affe-0d78dd2be8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:01.660985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0368d4-4591-4190-affe-0d78dd2be8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:01.674083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4f0368d4-4591-4190-affe-0d78dd2be8d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:01.988193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0368d4-4591-4190-affe-0d78dd2be8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:01.989805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0368d4-4591-4190-affe-0d78dd2be8d3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:01.995252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:02.025916 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1314/2603] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:04.163636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca098bbd-8485-49de-b82b-48b175bb41df None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:04.164141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca098bbd-8485-49de-b82b-48b175bb41df None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:04.164362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca098bbd-8485-49de-b82b-48b175bb41df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:04.208713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a6692-eebc-4310-a877-7a88bc87211e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:04.209970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a6692-eebc-4310-a877-7a88bc87211e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:04.210447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a6692-eebc-4310-a877-7a88bc87211e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:04.223873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ca098bbd-8485-49de-b82b-48b175bb41df None None] MFA Rules not processed for user `6435555fee024871b05e04d4a0a7cd4f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:04.224972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ca098bbd-8485-49de-b82b-48b175bb41df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:04.244123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-025a6692-eebc-4310-a877-7a88bc87211e None None] MFA Rules not processed for user `1ddcfe1b0eb5431dbbe4777b8dd77436`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:04.246649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-025a6692-eebc-4310-a877-7a88bc87211e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:04.316214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:04.339802 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1315/2604] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:53:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:53:04.361897 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1290/2605] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:53:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:04.373632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a6692-eebc-4310-a877-7a88bc87211e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:04.375132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-025a6692-eebc-4310-a877-7a88bc87211e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:04.491919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a6692-eebc-4310-a877-7a88bc87211e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:04.499015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02400a-6976-404e-b8a1-b71baf9618f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:04.499686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02400a-6976-404e-b8a1-b71baf9618f9 service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:04.499986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02400a-6976-404e-b8a1-b71baf9618f9 service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:04.500519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02400a-6976-404e-b8a1-b71baf9618f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:04.515450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02400a-6976-404e-b8a1-b71baf9618f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:04.517211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02400a-6976-404e-b8a1-b71baf9618f9 service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:04.529940 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1316/2606] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:08.829213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca098bbd-8485-49de-b82b-48b175bb41df None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:08.848919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca098bbd-8485-49de-b82b-48b175bb41df None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:08.859259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e9731-7ee8-4153-a510-b5b75bc4da37 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:08.861370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e9731-7ee8-4153-a510-b5b75bc4da37 service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:08.865101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e9731-7ee8-4153-a510-b5b75bc4da37 service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:08.865101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e9731-7ee8-4153-a510-b5b75bc4da37 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:08.867547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-849e9731-7ee8-4153-a510-b5b75bc4da37 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:08.910333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e9731-7ee8-4153-a510-b5b75bc4da37 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:08.911895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e9731-7ee8-4153-a510-b5b75bc4da37 service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:08.961332 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1291/2607] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:09.283453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02400a-6976-404e-b8a1-b71baf9618f9 service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:09.284920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fc02400a-6976-404e-b8a1-b71baf9618f9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:09.314667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02400a-6976-404e-b8a1-b71baf9618f9 service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:09.331612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ada74-f3aa-43d7-b8b2-eceb4a3f4396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:09.332538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ada74-f3aa-43d7-b8b2-eceb4a3f4396 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:09.332880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ada74-f3aa-43d7-b8b2-eceb4a3f4396 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:09.333352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ada74-f3aa-43d7-b8b2-eceb4a3f4396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:09.337503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bf1ada74-f3aa-43d7-b8b2-eceb4a3f4396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:09.362860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ada74-f3aa-43d7-b8b2-eceb4a3f4396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:09.364417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ada74-f3aa-43d7-b8b2-eceb4a3f4396 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:09.379872 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1317/2608] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:09.628611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e9731-7ee8-4153-a510-b5b75bc4da37 service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:09.630783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-849e9731-7ee8-4153-a510-b5b75bc4da37 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:09.781724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e9731-7ee8-4153-a510-b5b75bc4da37 service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:09.788320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de8f38-bcd9-4670-9427-2c271691b6d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:09.789202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de8f38-bcd9-4670-9427-2c271691b6d3 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:09.789617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de8f38-bcd9-4670-9427-2c271691b6d3 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:09.790256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de8f38-bcd9-4670-9427-2c271691b6d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:09.811487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d2de8f38-bcd9-4670-9427-2c271691b6d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:09.852842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de8f38-bcd9-4670-9427-2c271691b6d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:09.860583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de8f38-bcd9-4670-9427-2c271691b6d3 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:09.910120 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1292/2609] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:15.716154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ada74-f3aa-43d7-b8b2-eceb4a3f4396 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:15.718552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bf1ada74-f3aa-43d7-b8b2-eceb4a3f4396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:15.750643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ada74-f3aa-43d7-b8b2-eceb4a3f4396 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:15.767066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89992a84-3888-4f60-ab93-06ce5eed4870 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:15.769775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89992a84-3888-4f60-ab93-06ce5eed4870 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:15.770312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89992a84-3888-4f60-ab93-06ce5eed4870 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:15.770741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89992a84-3888-4f60-ab93-06ce5eed4870 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:15.787603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-89992a84-3888-4f60-ab93-06ce5eed4870 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:15.819895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89992a84-3888-4f60-ab93-06ce5eed4870 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:15.825737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89992a84-3888-4f60-ab93-06ce5eed4870 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:15.866254 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1318/2610] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:17.640625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de8f38-bcd9-4670-9427-2c271691b6d3 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:17.681281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de8f38-bcd9-4670-9427-2c271691b6d3 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:17.726600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661be99-3a51-46d2-a217-65482fefd43f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:17.727231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661be99-3a51-46d2-a217-65482fefd43f service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:17.727503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661be99-3a51-46d2-a217-65482fefd43f service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:17.727767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661be99-3a51-46d2-a217-65482fefd43f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:17.902533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661be99-3a51-46d2-a217-65482fefd43f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:17.904239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661be99-3a51-46d2-a217-65482fefd43f service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:17.926847 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1293/2611] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:18.270887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b036a5-47c2-47ab-b4b1-57aeedc675df None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:18.271570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b036a5-47c2-47ab-b4b1-57aeedc675df None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:18.271570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b036a5-47c2-47ab-b4b1-57aeedc675df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:18.309973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d3b036a5-47c2-47ab-b4b1-57aeedc675df None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:18.311550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d3b036a5-47c2-47ab-b4b1-57aeedc675df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:18.325149 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1319/2612] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:53:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:18.359335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661be99-3a51-46d2-a217-65482fefd43f service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:18.375929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661be99-3a51-46d2-a217-65482fefd43f service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:18.386881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87680ad-d628-4efd-b852-1be865688630 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:18.387506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87680ad-d628-4efd-b852-1be865688630 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:18.387959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87680ad-d628-4efd-b852-1be865688630 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:18.388380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87680ad-d628-4efd-b852-1be865688630 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:18.399945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87680ad-d628-4efd-b852-1be865688630 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:18.407564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87680ad-d628-4efd-b852-1be865688630 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:18.424090 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1294/2613] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:53:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:53:23.753410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b036a5-47c2-47ab-b4b1-57aeedc675df None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:23.770912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b036a5-47c2-47ab-b4b1-57aeedc675df None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:23.784379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c18ad-915e-4a33-afc4-7842177e5e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:23.785757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c18ad-915e-4a33-afc4-7842177e5e88 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:23.786198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c18ad-915e-4a33-afc4-7842177e5e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:23.786923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c18ad-915e-4a33-afc4-7842177e5e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:23.789694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f9c18ad-915e-4a33-afc4-7842177e5e88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:23.819490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c18ad-915e-4a33-afc4-7842177e5e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:23.824896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c18ad-915e-4a33-afc4-7842177e5e88 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:23.890158 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1320/2614] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:26.256720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87680ad-d628-4efd-b852-1be865688630 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:26.327662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:26.362843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87680ad-d628-4efd-b852-1be865688630 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:26.368462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:26.370274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:26.370812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:26.371840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] PATH_INFO: `/v3/users/e92c87e9b9c840e6a4bc73ee0593a2c6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:26.373584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e92c87e9b9c840e6a4bc73ee0593a2c6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:26.375964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:26.548522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e92c87e9b9c840e6a4bc73ee0593a2c6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:26.567171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e92c87e9b9c840e6a4bc73ee0593a2c6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:26.602412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e92c87e9b9c840e6a4bc73ee0593a2c6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:26.622131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e92c87e9b9c840e6a4bc73ee0593a2c6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:26.623050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e92c87e9b9c840e6a4bc73ee0593a2c6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:26.634525 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1295/2615] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:53:26 2022] DELETE /identity/v3/users/e92c87e9b9c840e6a4bc73ee0593a2c6 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:53:26.636114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa6b77-93c8-4c43-8c30-e9beb52e9138 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:26.636723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa6b77-93c8-4c43-8c30-e9beb52e9138 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:26.636723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa6b77-93c8-4c43-8c30-e9beb52e9138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:26.674547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1dfa6b77-93c8-4c43-8c30-e9beb52e9138 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:26.675790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1dfa6b77-93c8-4c43-8c30-e9beb52e9138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:26.833409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:26.863624 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1321/2616] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:26.910076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:26.911575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:27.076874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd708d14-5c67-48dc-b0a2-5d81894ad134 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:27.087561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed8650-baed-48ef-b3ce-382f06ee5b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:27.088134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed8650-baed-48ef-b3ce-382f06ee5b2f service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:27.088415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed8650-baed-48ef-b3ce-382f06ee5b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:27.088684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed8650-baed-48ef-b3ce-382f06ee5b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:27.149576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed8650-baed-48ef-b3ce-382f06ee5b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:27.159303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed8650-baed-48ef-b3ce-382f06ee5b2f service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:27.219160 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1296/2617] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:27.691783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa6b77-93c8-4c43-8c30-e9beb52e9138 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:27.693320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1dfa6b77-93c8-4c43-8c30-e9beb52e9138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:27.718836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa6b77-93c8-4c43-8c30-e9beb52e9138 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:27.743833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:27.744724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:27.745246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:27.745725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] PATH_INFO: `/v3/projects/b2227fe5e2aa4eddba19afff48b1d11a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:27.747673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2227fe5e2aa4eddba19afff48b1d11a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:27.751314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:27.841651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2227fe5e2aa4eddba19afff48b1d11a', 'request_id': 'req-a1d314f4-80be-4b92-8253-b928f5a9a3fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:27.856497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2227fe5e2aa4eddba19afff48b1d11a', 'request_id': 'req-a1d314f4-80be-4b92-8253-b928f5a9a3fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:27.865324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2227fe5e2aa4eddba19afff48b1d11a', 'request_id': 'req-a1d314f4-80be-4b92-8253-b928f5a9a3fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:27.883499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] The token cache is being invalidate because project b2227fe5e2aa4eddba19afff48b1d11a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:53:27.884014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:27.884898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:27.888998 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1322/2618] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:27 2022] DELETE /identity/v3/projects/b2227fe5e2aa4eddba19afff48b1d11a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:53:27.892822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed8650-baed-48ef-b3ce-382f06ee5b2f service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:27.908246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed8650-baed-48ef-b3ce-382f06ee5b2f service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:28.041212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:28.042302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:28.042684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:28.043242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] PATH_INFO: `/v3/users/54e203121335401c9d746b54d1a68b00` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:28.044884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e203121335401c9d746b54d1a68b00)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:28.048539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:28.067085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:28.156387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e203121335401c9d746b54d1a68b00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:28.167202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e203121335401c9d746b54d1a68b00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:28.186959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e203121335401c9d746b54d1a68b00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:28.209449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e203121335401c9d746b54d1a68b00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:28.211184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62f3c5f0-6e2c-4ebd-ab73-521d894c7241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e203121335401c9d746b54d1a68b00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:28.223724 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1297/2619] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:53:27 2022] DELETE /identity/v3/users/54e203121335401c9d746b54d1a68b00 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:53:28.852536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:28.854087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:28.868176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d314f4-80be-4b92-8253-b928f5a9a3fe admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:28.968028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:28.971191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:28.971581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:28.971953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] PATH_INFO: `/v3/projects/6754d0bbe3d44f868289bfbbe6af6e67` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:28.978069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6754d0bbe3d44f868289bfbbe6af6e67)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:28.980375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:29.073886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6754d0bbe3d44f868289bfbbe6af6e67', 'request_id': 'req-1fa870eb-7aea-4392-be06-440f9ba0f2d5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:29.099313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6754d0bbe3d44f868289bfbbe6af6e67', 'request_id': 'req-1fa870eb-7aea-4392-be06-440f9ba0f2d5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:29.113416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6754d0bbe3d44f868289bfbbe6af6e67', 'request_id': 'req-1fa870eb-7aea-4392-be06-440f9ba0f2d5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:29.150811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] The token cache is being invalidate because project 6754d0bbe3d44f868289bfbbe6af6e67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:53:29.151944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:29.154123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1fa870eb-7aea-4392-be06-440f9ba0f2d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:53:29.162528 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1323/2620] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:28 2022] DELETE /identity/v3/projects/6754d0bbe3d44f868289bfbbe6af6e67 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:53:29.172402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5528c9e-e608-41a9-8688-fa3649d984bb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:29.172933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5528c9e-e608-41a9-8688-fa3649d984bb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:29.172933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5528c9e-e608-41a9-8688-fa3649d984bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:29.234688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f5528c9e-e608-41a9-8688-fa3649d984bb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:29.236426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f5528c9e-e608-41a9-8688-fa3649d984bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:29.339257 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1298/2621] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:29.348377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9669aa-ecd8-47a6-aa30-f2d522e8f231 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:29.348945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9669aa-ecd8-47a6-aa30-f2d522e8f231 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:29.349226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9669aa-ecd8-47a6-aa30-f2d522e8f231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:29.390148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4c9669aa-ecd8-47a6-aa30-f2d522e8f231 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:29.391572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4c9669aa-ecd8-47a6-aa30-f2d522e8f231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:29.430750 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1324/2622] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:29.434354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5528c9e-e608-41a9-8688-fa3649d984bb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:29.454334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5528c9e-e608-41a9-8688-fa3649d984bb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:29.467054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124ddb8-a7c3-4072-808d-0df39ecba3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:29.469411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124ddb8-a7c3-4072-808d-0df39ecba3bb admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:29.469791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124ddb8-a7c3-4072-808d-0df39ecba3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:29.470156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124ddb8-a7c3-4072-808d-0df39ecba3bb admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:29.474653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124ddb8-a7c3-4072-808d-0df39ecba3bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:53:29.476232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124ddb8-a7c3-4072-808d-0df39ecba3bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:29.481400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124ddb8-a7c3-4072-808d-0df39ecba3bb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:29.515481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9669aa-ecd8-47a6-aa30-f2d522e8f231 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:29.515775 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1299/2623] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:53:29.525633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9669aa-ecd8-47a6-aa30-f2d522e8f231 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:29.543574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36982c-77be-40b4-a81d-f4807f42ab15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:29.544201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36982c-77be-40b4-a81d-f4807f42ab15 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:29.544472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36982c-77be-40b4-a81d-f4807f42ab15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:29.544733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36982c-77be-40b4-a81d-f4807f42ab15 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:29.545402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36982c-77be-40b4-a81d-f4807f42ab15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:29.547355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36982c-77be-40b4-a81d-f4807f42ab15 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:29.564246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:29.629537 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1325/2624] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:53:29 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:53:29.630885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124ddb8-a7c3-4072-808d-0df39ecba3bb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:29.698487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124ddb8-a7c3-4072-808d-0df39ecba3bb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:29.915294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ef1ba-eda2-4438-bfc5-416110339d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:29.916085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ef1ba-eda2-4438-bfc5-416110339d5d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:29.916363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ef1ba-eda2-4438-bfc5-416110339d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:29.916677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ef1ba-eda2-4438-bfc5-416110339d5d admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:29.917332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ef1ba-eda2-4438-bfc5-416110339d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:29.919545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ef1ba-eda2-4438-bfc5-416110339d5d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:29.970572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:29.983494 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1300/2625] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:53:29 2022] POST /identity/v3/users => generated 292 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:53:29.985353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36982c-77be-40b4-a81d-f4807f42ab15 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:30.000461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36982c-77be-40b4-a81d-f4807f42ab15 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:30.012485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d698f9-d7a7-4fad-869a-3004ff14b098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:30.014727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d698f9-d7a7-4fad-869a-3004ff14b098 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:30.014996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d698f9-d7a7-4fad-869a-3004ff14b098 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:30.016850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d698f9-d7a7-4fad-869a-3004ff14b098 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:30.017964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d698f9-d7a7-4fad-869a-3004ff14b098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:30.022671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d698f9-d7a7-4fad-869a-3004ff14b098 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:30.068665 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1326/2626] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:53:30.070191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ef1ba-eda2-4438-bfc5-416110339d5d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:30.089443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ef1ba-eda2-4438-bfc5-416110339d5d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:30.105016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943765c1-ab8b-4a40-a0b3-c41e3a7a3f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:30.107792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943765c1-ab8b-4a40-a0b3-c41e3a7a3f83 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:30.108069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943765c1-ab8b-4a40-a0b3-c41e3a7a3f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:30.108340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943765c1-ab8b-4a40-a0b3-c41e3a7a3f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:30.108989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943765c1-ab8b-4a40-a0b3-c41e3a7a3f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:30.116287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943765c1-ab8b-4a40-a0b3-c41e3a7a3f83 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:30.163738 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1301/2627] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:53:30.170115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d698f9-d7a7-4fad-869a-3004ff14b098 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:30.186893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d698f9-d7a7-4fad-869a-3004ff14b098 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:30.220357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a37b58-6b2d-43dd-86fe-0a3d257c564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:30.222422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a37b58-6b2d-43dd-86fe-0a3d257c564c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:30.222769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a37b58-6b2d-43dd-86fe-0a3d257c564c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:30.224236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a37b58-6b2d-43dd-86fe-0a3d257c564c admin admin] PATH_INFO: `/v3/projects/e08b85b4f6cf47a88fe24ead8e6d45e9/users/ec8dcd7615cc43b3866034d5af34a9a1/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:30.262437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a37b58-6b2d-43dd-86fe-0a3d257c564c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08b85b4f6cf47a88fe24ead8e6d45e9, user_id=ec8dcd7615cc43b3866034d5af34a9a1, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:30.265705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a37b58-6b2d-43dd-86fe-0a3d257c564c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:30.322383 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1327/2628] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:53:30 2022] PUT /identity/v3/projects/e08b85b4f6cf47a88fe24ead8e6d45e9/users/ec8dcd7615cc43b3866034d5af34a9a1/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:53:30.327618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4558a-843f-4716-9645-bf43c300ca28 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:30.328759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4558a-843f-4716-9645-bf43c300ca28 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:30.329287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4558a-843f-4716-9645-bf43c300ca28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:30.340989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a37b58-6b2d-43dd-86fe-0a3d257c564c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:30.347329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-56a37b58-6b2d-43dd-86fe-0a3d257c564c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:30.367370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7fd4558a-843f-4716-9645-bf43c300ca28 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:30.373427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7fd4558a-843f-4716-9645-bf43c300ca28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:30.500422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a37b58-6b2d-43dd-86fe-0a3d257c564c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:30.506739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcffeb9-6e83-40ae-a94b-b23873d19208 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:30.507486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcffeb9-6e83-40ae-a94b-b23873d19208 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:30.507967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcffeb9-6e83-40ae-a94b-b23873d19208 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:30.508326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcffeb9-6e83-40ae-a94b-b23873d19208 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:30.522260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8fcffeb9-6e83-40ae-a94b-b23873d19208 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:30.546060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcffeb9-6e83-40ae-a94b-b23873d19208 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:30.547441 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1302/2629] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:30.548735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcffeb9-6e83-40ae-a94b-b23873d19208 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:30.562213 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1328/2630] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:53:30.567273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4558a-843f-4716-9645-bf43c300ca28 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:30.571384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7fd4558a-843f-4716-9645-bf43c300ca28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:30.695433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7569f7c-a57f-485b-8223-733002054251 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:30.696132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7569f7c-a57f-485b-8223-733002054251 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:30.696499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7569f7c-a57f-485b-8223-733002054251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:30.714262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4558a-843f-4716-9645-bf43c300ca28 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:30.725833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3de93-aa91-47a3-93d4-df979a076980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:30.726465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3de93-aa91-47a3-93d4-df979a076980 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:30.726742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3de93-aa91-47a3-93d4-df979a076980 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:30.727013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3de93-aa91-47a3-93d4-df979a076980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:30.745366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e7569f7c-a57f-485b-8223-733002054251 None None] MFA Rules not processed for user `8bbc891924f84d99aac22518f69e7b99`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:30.748890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e7569f7c-a57f-485b-8223-733002054251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:30.782487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3de93-aa91-47a3-93d4-df979a076980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:30.784710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3de93-aa91-47a3-93d4-df979a076980 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:30.826505 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1303/2631] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:30.860540 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1329/2632] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:53:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:35.850078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3de93-aa91-47a3-93d4-df979a076980 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:35.882932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3de93-aa91-47a3-93d4-df979a076980 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:35.896770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21ae3d-3050-4e43-a2f2-b3b3c629bd08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:35.900389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ae3d-3050-4e43-a2f2-b3b3c629bd08 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:35.901876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ae3d-3050-4e43-a2f2-b3b3c629bd08 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:35.902314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ae3d-3050-4e43-a2f2-b3b3c629bd08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:35.909261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3b21ae3d-3050-4e43-a2f2-b3b3c629bd08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:35.936679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21ae3d-3050-4e43-a2f2-b3b3c629bd08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:35.940281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21ae3d-3050-4e43-a2f2-b3b3c629bd08 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:35.956467 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1304/2633] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:37.858483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c218d8-e949-46ba-8a7d-30250935e2b9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:37.859087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c218d8-e949-46ba-8a7d-30250935e2b9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:37.859448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c218d8-e949-46ba-8a7d-30250935e2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:37.900879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b9c218d8-e949-46ba-8a7d-30250935e2b9 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:37.903297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9c218d8-e949-46ba-8a7d-30250935e2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:37.960181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:38.016249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21ae3d-3050-4e43-a2f2-b3b3c629bd08 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:38.018659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3b21ae3d-3050-4e43-a2f2-b3b3c629bd08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:38.029992 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1330/2634] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:53:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:38.238537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21ae3d-3050-4e43-a2f2-b3b3c629bd08 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:38.252511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155547d5-0cb0-41e3-ad25-98c021223fc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:38.256695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155547d5-0cb0-41e3-ad25-98c021223fc8 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:38.257716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155547d5-0cb0-41e3-ad25-98c021223fc8 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:38.261093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155547d5-0cb0-41e3-ad25-98c021223fc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:38.282110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155547d5-0cb0-41e3-ad25-98c021223fc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:38.286900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155547d5-0cb0-41e3-ad25-98c021223fc8 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:38.308758 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1305/2635] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:53:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 20:53:41.425323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c218d8-e949-46ba-8a7d-30250935e2b9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:41.426707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9c218d8-e949-46ba-8a7d-30250935e2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:41.468429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c218d8-e949-46ba-8a7d-30250935e2b9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:41.492125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3ad3c-eb6f-4e83-9bf0-c4ba56359455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:41.492586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3ad3c-eb6f-4e83-9bf0-c4ba56359455 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:41.494114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3ad3c-eb6f-4e83-9bf0-c4ba56359455 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:41.494395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3ad3c-eb6f-4e83-9bf0-c4ba56359455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:41.502612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d2b3ad3c-eb6f-4e83-9bf0-c4ba56359455 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:41.719638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3ad3c-eb6f-4e83-9bf0-c4ba56359455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:41.725977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3ad3c-eb6f-4e83-9bf0-c4ba56359455 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:41.767889 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1331/2636] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:53:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:44.302185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155547d5-0cb0-41e3-ad25-98c021223fc8 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:44.317634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155547d5-0cb0-41e3-ad25-98c021223fc8 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:44.324061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843b7df-d6e1-43c3-baa7-c1d2a73c7930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:44.325242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843b7df-d6e1-43c3-baa7-c1d2a73c7930 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:44.325521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843b7df-d6e1-43c3-baa7-c1d2a73c7930 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:44.325788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843b7df-d6e1-43c3-baa7-c1d2a73c7930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:44.330561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3843b7df-d6e1-43c3-baa7-c1d2a73c7930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:44.351519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:44.512782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843b7df-d6e1-43c3-baa7-c1d2a73c7930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:44.516670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843b7df-d6e1-43c3-baa7-c1d2a73c7930 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:44.547432 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1306/2637] 10.222.0.16 () {60 vars in 1396 bytes} [Sat Oct 15 20:53:44 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:53:50.327065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b00d6-d6d0-4d32-b843-2f6c22908c82 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:50.327601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b00d6-d6d0-4d32-b843-2f6c22908c82 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:50.327709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b00d6-d6d0-4d32-b843-2f6c22908c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:50.397735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd5bf3-5ad5-4080-981b-221f346819c2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:50.398275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd5bf3-5ad5-4080-981b-221f346819c2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:50.398275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd5bf3-5ad5-4080-981b-221f346819c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:50.442826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c52b00d6-d6d0-4d32-b843-2f6c22908c82 None None] MFA Rules not processed for user `ec8dcd7615cc43b3866034d5af34a9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:50.444080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c52b00d6-d6d0-4d32-b843-2f6c22908c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:50.456406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-21dd5bf3-5ad5-4080-981b-221f346819c2 None None] MFA Rules not processed for user `999bc03f52824573bb3c8bdc6a1179cf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:50.457756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-21dd5bf3-5ad5-4080-981b-221f346819c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:50.483629 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1307/2638] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:50.670643 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1332/2639] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:50.675931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd5bf3-5ad5-4080-981b-221f346819c2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:50.694923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd5bf3-5ad5-4080-981b-221f346819c2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:50.715990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa523650-77ea-4d9a-9546-f9222eaa4aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:50.721638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523650-77ea-4d9a-9546-f9222eaa4aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:50.723080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523650-77ea-4d9a-9546-f9222eaa4aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:50.723584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523650-77ea-4d9a-9546-f9222eaa4aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:50.727699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa523650-77ea-4d9a-9546-f9222eaa4aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:50.739938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa523650-77ea-4d9a-9546-f9222eaa4aa8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:50.793027 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1308/2640] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:53:50 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:53:50.797507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b00d6-d6d0-4d32-b843-2f6c22908c82 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:50.818437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b00d6-d6d0-4d32-b843-2f6c22908c82 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:51.000051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbee61-30c2-49a8-98ea-b69562068637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:51.000813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbee61-30c2-49a8-98ea-b69562068637 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:51.001293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbee61-30c2-49a8-98ea-b69562068637 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:51.001656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbee61-30c2-49a8-98ea-b69562068637 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:51.002379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbee61-30c2-49a8-98ea-b69562068637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:51.004590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbee61-30c2-49a8-98ea-b69562068637 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:51.050180 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1333/2641] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:53:50 2022] POST /identity/v3/users => generated 298 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:53:51.054065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa523650-77ea-4d9a-9546-f9222eaa4aa8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:51.091679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa523650-77ea-4d9a-9546-f9222eaa4aa8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:51.099633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06378766-1187-4d2c-b012-a084d777c1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:51.100854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06378766-1187-4d2c-b012-a084d777c1ba admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:51.101988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06378766-1187-4d2c-b012-a084d777c1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:51.102464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06378766-1187-4d2c-b012-a084d777c1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:51.103261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06378766-1187-4d2c-b012-a084d777c1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:51.107187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06378766-1187-4d2c-b012-a084d777c1ba admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:51.138942 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1309/2642] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:53:51.140710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbee61-30c2-49a8-98ea-b69562068637 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:51.151765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:51.172681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbee61-30c2-49a8-98ea-b69562068637 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:51.178878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5da2c-f7ac-4cef-8ea1-04fc6fdd3d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:51.179909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5da2c-f7ac-4cef-8ea1-04fc6fdd3d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:51.180254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5da2c-f7ac-4cef-8ea1-04fc6fdd3d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:51.180587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5da2c-f7ac-4cef-8ea1-04fc6fdd3d06 admin admin] PATH_INFO: `/v3/projects/971ec797f8d94d0989654cfa2d952a80/users/9749157277104da9bc3ffeef563adeb4/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:51.236876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5da2c-f7ac-4cef-8ea1-04fc6fdd3d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=971ec797f8d94d0989654cfa2d952a80, user_id=9749157277104da9bc3ffeef563adeb4, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:51.244195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5da2c-f7ac-4cef-8ea1-04fc6fdd3d06 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:51.273206 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1334/2643] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:53:51 2022] PUT /identity/v3/projects/971ec797f8d94d0989654cfa2d952a80/users/9749157277104da9bc3ffeef563adeb4/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:53:51.278178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06378766-1187-4d2c-b012-a084d777c1ba admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:51.311089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06378766-1187-4d2c-b012-a084d777c1ba admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:51.484698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e28c9d-0b96-4ab6-9480-9cd74cba3abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:51.485372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e28c9d-0b96-4ab6-9480-9cd74cba3abe admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:51.485597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e28c9d-0b96-4ab6-9480-9cd74cba3abe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:51.485862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e28c9d-0b96-4ab6-9480-9cd74cba3abe admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:51.486543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e28c9d-0b96-4ab6-9480-9cd74cba3abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:51.488123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e28c9d-0b96-4ab6-9480-9cd74cba3abe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:51.516460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5da2c-f7ac-4cef-8ea1-04fc6fdd3d06 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:51.521904 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1310/2644] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:53:51.531283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5da2c-f7ac-4cef-8ea1-04fc6fdd3d06 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:51.540525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca3759-c1af-46b9-9233-484e013699d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:51.541376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca3759-c1af-46b9-9233-484e013699d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:51.541746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca3759-c1af-46b9-9233-484e013699d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:51.542107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca3759-c1af-46b9-9233-484e013699d3 admin admin] PATH_INFO: `/v3/projects/971ec797f8d94d0989654cfa2d952a80/users/9749157277104da9bc3ffeef563adeb4/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:51.545028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca3759-c1af-46b9-9233-484e013699d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=971ec797f8d94d0989654cfa2d952a80, user_id=9749157277104da9bc3ffeef563adeb4, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:51.550620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca3759-c1af-46b9-9233-484e013699d3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:51.566998 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1335/2645] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:53:51 2022] PUT /identity/v3/projects/971ec797f8d94d0989654cfa2d952a80/users/9749157277104da9bc3ffeef563adeb4/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:53:56.473472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2f9a2-3ef2-4bde-b4ef-067fd8e59769 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:56.473992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2f9a2-3ef2-4bde-b4ef-067fd8e59769 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:56.473992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2f9a2-3ef2-4bde-b4ef-067fd8e59769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:56.516726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2fd2f9a2-3ef2-4bde-b4ef-067fd8e59769 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:53:56.520364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2fd2f9a2-3ef2-4bde-b4ef-067fd8e59769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:53:56.647847 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1311/2646] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:53:56.656258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca3759-c1af-46b9-9233-484e013699d3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:53:56.849629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca3759-c1af-46b9-9233-484e013699d3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:53:56.855624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4066fe-1834-4fd3-9e85-a61dc2eeb472 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:53:56.856285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4066fe-1834-4fd3-9e85-a61dc2eeb472 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:53:56.856701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4066fe-1834-4fd3-9e85-a61dc2eeb472 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:53:56.856977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4066fe-1834-4fd3-9e85-a61dc2eeb472 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:53:56.889269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4066fe-1834-4fd3-9e85-a61dc2eeb472 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:53:56.891129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4066fe-1834-4fd3-9e85-a61dc2eeb472 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:53:56.914479 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1336/2647] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:53:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:06.405727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267521fb-a541-48c2-a389-7bf83983bf84 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:06.406210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267521fb-a541-48c2-a389-7bf83983bf84 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:06.408181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267521fb-a541-48c2-a389-7bf83983bf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:06.457028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:06.504522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-267521fb-a541-48c2-a389-7bf83983bf84 None None] MFA Rules not processed for user `9749157277104da9bc3ffeef563adeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:06.505831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-267521fb-a541-48c2-a389-7bf83983bf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:06.735331 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1312/2648] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:06.909767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:07.202628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4066fe-1834-4fd3-9e85-a61dc2eeb472 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:07.384382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4066fe-1834-4fd3-9e85-a61dc2eeb472 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:07.391115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9300d7-b1e2-4756-9362-1feb306af939 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:07.391857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9300d7-b1e2-4756-9362-1feb306af939 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:07.393062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9300d7-b1e2-4756-9362-1feb306af939 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:07.393415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9300d7-b1e2-4756-9362-1feb306af939 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:07.491417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9300d7-b1e2-4756-9362-1feb306af939 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:07.493284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9300d7-b1e2-4756-9362-1feb306af939 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:07.505514 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1337/2649] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:14.301191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd763-4f6f-429e-b209-be962cc991c8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:14.302531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd763-4f6f-429e-b209-be962cc991c8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:14.302795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd763-4f6f-429e-b209-be962cc991c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:14.323671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3fabd763-4f6f-429e-b209-be962cc991c8 None None] MFA Rules not processed for user `ec8dcd7615cc43b3866034d5af34a9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:14.324935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3fabd763-4f6f-429e-b209-be962cc991c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:14.336869 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1313/2650] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:15.103005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9300d7-b1e2-4756-9362-1feb306af939 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:15.104501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac9300d7-b1e2-4756-9362-1feb306af939 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:15.249795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9300d7-b1e2-4756-9362-1feb306af939 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:15.259034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74d828-b7ed-4779-83f9-af4b1ae941fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:15.259926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74d828-b7ed-4779-83f9-af4b1ae941fb service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:15.260392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74d828-b7ed-4779-83f9-af4b1ae941fb service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:15.260886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74d828-b7ed-4779-83f9-af4b1ae941fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:15.263736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1f74d828-b7ed-4779-83f9-af4b1ae941fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:15.279012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74d828-b7ed-4779-83f9-af4b1ae941fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:15.282001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74d828-b7ed-4779-83f9-af4b1ae941fb service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:15.305649 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1338/2651] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:16.189027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22849d6e-3f38-44ec-8d27-d8471f800b7e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:16.190153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22849d6e-3f38-44ec-8d27-d8471f800b7e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:16.190651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22849d6e-3f38-44ec-8d27-d8471f800b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:16.226688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-22849d6e-3f38-44ec-8d27-d8471f800b7e None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:16.228498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-22849d6e-3f38-44ec-8d27-d8471f800b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:16.264911 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1314/2652] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:54:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:54:16.278758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74d828-b7ed-4779-83f9-af4b1ae941fb service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:16.299637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74d828-b7ed-4779-83f9-af4b1ae941fb service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:16.305471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bca6c-b8a5-4290-921a-1dca1638ed2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:16.306197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bca6c-b8a5-4290-921a-1dca1638ed2b service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:16.306532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bca6c-b8a5-4290-921a-1dca1638ed2b service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:16.306879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bca6c-b8a5-4290-921a-1dca1638ed2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:16.317171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bca6c-b8a5-4290-921a-1dca1638ed2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:16.319359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bca6c-b8a5-4290-921a-1dca1638ed2b service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:16.332097 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1339/2653] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:54:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:17.004393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095cb58-33ac-44fe-88ab-c230a7ae8179 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:17.004929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095cb58-33ac-44fe-88ab-c230a7ae8179 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:17.005445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095cb58-33ac-44fe-88ab-c230a7ae8179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:17.043218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f095cb58-33ac-44fe-88ab-c230a7ae8179 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:17.045118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f095cb58-33ac-44fe-88ab-c230a7ae8179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:17.063221 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1315/2654] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:54:17.073560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bca6c-b8a5-4290-921a-1dca1638ed2b service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:17.111985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bca6c-b8a5-4290-921a-1dca1638ed2b service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:17.122432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095cb58-33ac-44fe-88ab-c230a7ae8179 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:17.123871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f095cb58-33ac-44fe-88ab-c230a7ae8179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:17.141517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e316a52-45f0-4825-889e-8681bdf6ef8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:17.142430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e316a52-45f0-4825-889e-8681bdf6ef8e service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:17.142882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e316a52-45f0-4825-889e-8681bdf6ef8e service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:17.143315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e316a52-45f0-4825-889e-8681bdf6ef8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:17.163665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e316a52-45f0-4825-889e-8681bdf6ef8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:17.165318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e316a52-45f0-4825-889e-8681bdf6ef8e service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:17.187138 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1340/2655] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:54:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:17.251390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095cb58-33ac-44fe-88ab-c230a7ae8179 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:17.256780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a06a5-ba64-42af-8299-3af13261e7af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:17.257501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a06a5-ba64-42af-8299-3af13261e7af service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:17.257845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a06a5-ba64-42af-8299-3af13261e7af service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:17.258112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a06a5-ba64-42af-8299-3af13261e7af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:17.259985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1f6a06a5-ba64-42af-8299-3af13261e7af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:17.274346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a06a5-ba64-42af-8299-3af13261e7af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:17.276146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a06a5-ba64-42af-8299-3af13261e7af service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:17.291408 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1316/2656] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:54:18.259489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f0ad5-e7ab-47ea-a7aa-dfdb59ff7809 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:18.259952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f0ad5-e7ab-47ea-a7aa-dfdb59ff7809 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:18.260062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f0ad5-e7ab-47ea-a7aa-dfdb59ff7809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:18.279924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-560f0ad5-e7ab-47ea-a7aa-dfdb59ff7809 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:18.281375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-560f0ad5-e7ab-47ea-a7aa-dfdb59ff7809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:18.292262 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1341/2657] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:18.317703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a06a5-ba64-42af-8299-3af13261e7af service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:18.329565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a06a5-ba64-42af-8299-3af13261e7af service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:18.334787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b966b7-4322-40e6-9008-144af5aab651 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:18.336017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b966b7-4322-40e6-9008-144af5aab651 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:18.336294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b966b7-4322-40e6-9008-144af5aab651 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:18.336561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b966b7-4322-40e6-9008-144af5aab651 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:18.346264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b966b7-4322-40e6-9008-144af5aab651 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:18.347831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b966b7-4322-40e6-9008-144af5aab651 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:18.357268 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1317/2658] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:54:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:20.113588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f0ad5-e7ab-47ea-a7aa-dfdb59ff7809 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:20.115123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-560f0ad5-e7ab-47ea-a7aa-dfdb59ff7809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:20.127042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:20.199895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f0ad5-e7ab-47ea-a7aa-dfdb59ff7809 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:20.211168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:20.211728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:20.211998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:20.212266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] PATH_INFO: `/v3/users/e32a1c53b4be44188b07c768405961ed` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:20.213653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32a1c53b4be44188b07c768405961ed)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:20.215641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:20.296815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32a1c53b4be44188b07c768405961ed'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:20.324482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32a1c53b4be44188b07c768405961ed'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:20.336861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32a1c53b4be44188b07c768405961ed'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:20.348060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32a1c53b4be44188b07c768405961ed'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:20.348929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32a1c53b4be44188b07c768405961ed'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:20.358305 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1342/2659] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:20 2022] DELETE /identity/v3/users/e32a1c53b4be44188b07c768405961ed => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:20.362774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72e009-e0a8-4998-b1b2-091493ef5aec None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:20.363304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72e009-e0a8-4998-b1b2-091493ef5aec None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:20.363304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72e009-e0a8-4998-b1b2-091493ef5aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:20.385253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fd72e009-e0a8-4998-b1b2-091493ef5aec None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:20.386655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd72e009-e0a8-4998-b1b2-091493ef5aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:20.445979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:20.485893 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1318/2660] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:54:20.497944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:20.500096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:20.587415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b46fab-5fea-43fd-8afc-0d39adda80a4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:20.594771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d9614-e7de-4e48-b8a6-84cbaf709753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:20.595381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d9614-e7de-4e48-b8a6-84cbaf709753 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:20.595827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d9614-e7de-4e48-b8a6-84cbaf709753 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:20.596091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d9614-e7de-4e48-b8a6-84cbaf709753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:20.613698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d9614-e7de-4e48-b8a6-84cbaf709753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:20.615379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d9614-e7de-4e48-b8a6-84cbaf709753 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:20.626646 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1343/2661] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:21.217712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72e009-e0a8-4998-b1b2-091493ef5aec None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:21.219304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd72e009-e0a8-4998-b1b2-091493ef5aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:21.256435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72e009-e0a8-4998-b1b2-091493ef5aec None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:21.274639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:21.276004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:21.276382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:21.276755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] PATH_INFO: `/v3/projects/94e961feee714587bb2ffe545d286c5c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:21.285470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e961feee714587bb2ffe545d286c5c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:21.292337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:21.390596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e961feee714587bb2ffe545d286c5c', 'request_id': 'req-c59ab578-47c1-4537-810e-9444c31eaf9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:21.403779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e961feee714587bb2ffe545d286c5c', 'request_id': 'req-c59ab578-47c1-4537-810e-9444c31eaf9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:21.411331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e961feee714587bb2ffe545d286c5c', 'request_id': 'req-c59ab578-47c1-4537-810e-9444c31eaf9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:21.436946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] The token cache is being invalidate because project 94e961feee714587bb2ffe545d286c5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:21.437822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:21.440113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:21.445559 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1319/2662] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:21 2022] DELETE /identity/v3/projects/94e961feee714587bb2ffe545d286c5c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:21.447056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d9614-e7de-4e48-b8a6-84cbaf709753 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:21.461010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d9614-e7de-4e48-b8a6-84cbaf709753 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:21.496741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:21.555974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:21.556973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:21.557452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:21.557874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] PATH_INFO: `/v3/users/81fdbd60e924457692039abcb456cb89` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:21.561967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81fdbd60e924457692039abcb456cb89)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:21.564141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:21.647449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81fdbd60e924457692039abcb456cb89'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:21.663686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81fdbd60e924457692039abcb456cb89'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:21.680265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81fdbd60e924457692039abcb456cb89'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:21.690993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81fdbd60e924457692039abcb456cb89'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:21.691718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f604bbae-6705-4c50-928c-734be64da965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81fdbd60e924457692039abcb456cb89'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:21.698367 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1344/2663] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:21 2022] DELETE /identity/v3/users/81fdbd60e924457692039abcb456cb89 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:22.632352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:22.633832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:22.660426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ab578-47c1-4537-810e-9444c31eaf9c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:22.769195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:22.773578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:22.774137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:22.774650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] PATH_INFO: `/v3/projects/08e2c30e5266478893afa293c836a3ed` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:22.776375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e2c30e5266478893afa293c836a3ed)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:22.780486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:22.875679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e2c30e5266478893afa293c836a3ed', 'request_id': 'req-595d684c-09d2-40cd-934a-b68e8991dfb5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:22.901904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e2c30e5266478893afa293c836a3ed', 'request_id': 'req-595d684c-09d2-40cd-934a-b68e8991dfb5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:22.915091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e2c30e5266478893afa293c836a3ed', 'request_id': 'req-595d684c-09d2-40cd-934a-b68e8991dfb5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:22.938440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] The token cache is being invalidate because project 08e2c30e5266478893afa293c836a3ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:22.939142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:22.941319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-595d684c-09d2-40cd-934a-b68e8991dfb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:22.943516 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1320/2664] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:22 2022] DELETE /identity/v3/projects/08e2c30e5266478893afa293c836a3ed => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:22.952177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eefc7-d4e9-47d4-8a7f-7642637490ef None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:22.953225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eefc7-d4e9-47d4-8a7f-7642637490ef None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:22.953710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eefc7-d4e9-47d4-8a7f-7642637490ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:22.990728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-085eefc7-d4e9-47d4-8a7f-7642637490ef None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:22.992220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-085eefc7-d4e9-47d4-8a7f-7642637490ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:23.155752 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1345/2665] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:23.230923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35361c00-77bc-4152-a8d0-9ef847b10188 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:23.231182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35361c00-77bc-4152-a8d0-9ef847b10188 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:23.231987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35361c00-77bc-4152-a8d0-9ef847b10188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:23.253087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:23.282816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-35361c00-77bc-4152-a8d0-9ef847b10188 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:23.284064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-35361c00-77bc-4152-a8d0-9ef847b10188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:23.307919 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1321/2666] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:23.313338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085eefc7-d4e9-47d4-8a7f-7642637490ef None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:23.332691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085eefc7-d4e9-47d4-8a7f-7642637490ef None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:23.348127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422d141-2f1a-4103-a6f2-4bf3d58dc667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:23.348774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422d141-2f1a-4103-a6f2-4bf3d58dc667 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:23.349066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422d141-2f1a-4103-a6f2-4bf3d58dc667 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:23.360240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422d141-2f1a-4103-a6f2-4bf3d58dc667 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:23.363242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422d141-2f1a-4103-a6f2-4bf3d58dc667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:54:23.363916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422d141-2f1a-4103-a6f2-4bf3d58dc667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:23.368964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422d141-2f1a-4103-a6f2-4bf3d58dc667 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:23.429250 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1346/2667] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:54:23.432145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35361c00-77bc-4152-a8d0-9ef847b10188 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:23.455418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35361c00-77bc-4152-a8d0-9ef847b10188 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:23.481509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff35d4-e5da-4b4d-bf70-4bd98622b873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:23.485989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff35d4-e5da-4b4d-bf70-4bd98622b873 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:23.486847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff35d4-e5da-4b4d-bf70-4bd98622b873 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:23.488333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff35d4-e5da-4b4d-bf70-4bd98622b873 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:23.490370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff35d4-e5da-4b4d-bf70-4bd98622b873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:23.495471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff35d4-e5da-4b4d-bf70-4bd98622b873 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:23.569252 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1322/2668] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:23 2022] POST /identity/v3/projects => generated 403 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:23.573702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422d141-2f1a-4103-a6f2-4bf3d58dc667 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:23.586302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422d141-2f1a-4103-a6f2-4bf3d58dc667 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:23.668514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:23.729946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f691416-7f98-4934-a49a-b5d6ce7f2ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:23.730823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f691416-7f98-4934-a49a-b5d6ce7f2ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:23.731461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f691416-7f98-4934-a49a-b5d6ce7f2ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:23.732185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f691416-7f98-4934-a49a-b5d6ce7f2ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:23.743401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f691416-7f98-4934-a49a-b5d6ce7f2ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:23.750076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f691416-7f98-4934-a49a-b5d6ce7f2ef4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:23.795610 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1347/2669] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:23 2022] POST /identity/v3/users => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:23.799457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff35d4-e5da-4b4d-bf70-4bd98622b873 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:23.811900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff35d4-e5da-4b4d-bf70-4bd98622b873 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:23.817548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b530a7b-01c4-413a-94a3-169b22f2cafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:23.818455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b530a7b-01c4-413a-94a3-169b22f2cafe admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:23.819075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b530a7b-01c4-413a-94a3-169b22f2cafe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:23.819583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b530a7b-01c4-413a-94a3-169b22f2cafe admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:23.820442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b530a7b-01c4-413a-94a3-169b22f2cafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:23.822279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b530a7b-01c4-413a-94a3-169b22f2cafe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:23.834683 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1323/2670] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:23.838735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f691416-7f98-4934-a49a-b5d6ce7f2ef4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:23.859494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f691416-7f98-4934-a49a-b5d6ce7f2ef4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:23.865509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ef014-14a8-47e9-97c7-b403fe61ddbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:23.866124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ef014-14a8-47e9-97c7-b403fe61ddbf admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:23.866396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ef014-14a8-47e9-97c7-b403fe61ddbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:23.867293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ef014-14a8-47e9-97c7-b403fe61ddbf admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:23.867997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ef014-14a8-47e9-97c7-b403fe61ddbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:23.869706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ef014-14a8-47e9-97c7-b403fe61ddbf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:23.896554 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1348/2671] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:23.897176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b530a7b-01c4-413a-94a3-169b22f2cafe admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:23.912898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b530a7b-01c4-413a-94a3-169b22f2cafe admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:23.920025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e4668-7eda-4651-b2c9-4b2f05a3d368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:23.922643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e4668-7eda-4651-b2c9-4b2f05a3d368 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:23.923228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e4668-7eda-4651-b2c9-4b2f05a3d368 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:23.923666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e4668-7eda-4651-b2c9-4b2f05a3d368 admin admin] PATH_INFO: `/v3/projects/b53a4be470b7451b88037a82e4657a12/users/93a6541758aa4eefb5c5c2407a899943/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:23.948285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e4668-7eda-4651-b2c9-4b2f05a3d368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b53a4be470b7451b88037a82e4657a12, user_id=93a6541758aa4eefb5c5c2407a899943, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:23.952498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e4668-7eda-4651-b2c9-4b2f05a3d368 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:23.978098 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1324/2672] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:23 2022] PUT /identity/v3/projects/b53a4be470b7451b88037a82e4657a12/users/93a6541758aa4eefb5c5c2407a899943/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:54:23.989410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a31ff3-4888-4127-8718-cc16d6a001ac None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:23.989924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a31ff3-4888-4127-8718-cc16d6a001ac None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:23.990061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a31ff3-4888-4127-8718-cc16d6a001ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:24.054123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c7a31ff3-4888-4127-8718-cc16d6a001ac None None] MFA Rules not processed for user `93a6541758aa4eefb5c5c2407a899943`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:24.055393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c7a31ff3-4888-4127-8718-cc16d6a001ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:24.158232 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1349/2673] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:24.162841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e4668-7eda-4651-b2c9-4b2f05a3d368 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:24.301177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e4668-7eda-4651-b2c9-4b2f05a3d368 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:24.308558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50e247-8d27-41e2-b7e8-2654df75d360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:24.309310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50e247-8d27-41e2-b7e8-2654df75d360 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:24.310208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50e247-8d27-41e2-b7e8-2654df75d360 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:24.310586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50e247-8d27-41e2-b7e8-2654df75d360 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:24.311406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50e247-8d27-41e2-b7e8-2654df75d360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:24.313648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50e247-8d27-41e2-b7e8-2654df75d360 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:24.361483 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1325/2674] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:24 2022] POST /identity/v3/projects => generated 401 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:24.365613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a31ff3-4888-4127-8718-cc16d6a001ac None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:24.387005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a31ff3-4888-4127-8718-cc16d6a001ac None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:24.489142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87387cc3-d3b1-4e46-a888-47992f61d10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:24.489774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87387cc3-d3b1-4e46-a888-47992f61d10e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:24.490055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87387cc3-d3b1-4e46-a888-47992f61d10e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:24.490329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87387cc3-d3b1-4e46-a888-47992f61d10e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:24.491377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87387cc3-d3b1-4e46-a888-47992f61d10e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:24.494299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87387cc3-d3b1-4e46-a888-47992f61d10e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:24.525818 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1350/2675] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:24 2022] POST /identity/v3/users => generated 307 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:24.527002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50e247-8d27-41e2-b7e8-2654df75d360 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:24.548188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50e247-8d27-41e2-b7e8-2654df75d360 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:24.556623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057be098-270b-4a49-8192-f5f56e01582c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:24.557456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057be098-270b-4a49-8192-f5f56e01582c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:24.557736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057be098-270b-4a49-8192-f5f56e01582c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:24.558122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057be098-270b-4a49-8192-f5f56e01582c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:24.558890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057be098-270b-4a49-8192-f5f56e01582c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:24.561324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057be098-270b-4a49-8192-f5f56e01582c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:24.578486 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1326/2676] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:24.579477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87387cc3-d3b1-4e46-a888-47992f61d10e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:24.618568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87387cc3-d3b1-4e46-a888-47992f61d10e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:24.633416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228e098-68fe-4558-b0b9-105ec325d118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:24.635285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228e098-68fe-4558-b0b9-105ec325d118 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:24.635837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228e098-68fe-4558-b0b9-105ec325d118 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:24.636326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228e098-68fe-4558-b0b9-105ec325d118 admin admin] PATH_INFO: `/v3/projects/752227df24174003a236d784f047db75/users/d8f9414680164912ba902c7a6f2a2f83/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:24.655312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228e098-68fe-4558-b0b9-105ec325d118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=752227df24174003a236d784f047db75, user_id=d8f9414680164912ba902c7a6f2a2f83, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:24.658013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228e098-68fe-4558-b0b9-105ec325d118 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:24.678365 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1351/2677] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:24 2022] PUT /identity/v3/projects/752227df24174003a236d784f047db75/users/d8f9414680164912ba902c7a6f2a2f83/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:54:24.682568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057be098-270b-4a49-8192-f5f56e01582c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:24.693583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057be098-270b-4a49-8192-f5f56e01582c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:24.771272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89dce28-0b21-4860-b2d9-beac9db056a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:24.771940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dce28-0b21-4860-b2d9-beac9db056a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:24.772196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dce28-0b21-4860-b2d9-beac9db056a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:24.772463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dce28-0b21-4860-b2d9-beac9db056a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:24.773649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dce28-0b21-4860-b2d9-beac9db056a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:24.775407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dce28-0b21-4860-b2d9-beac9db056a2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:24.803109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228e098-68fe-4558-b0b9-105ec325d118 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:24.813429 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1327/2678] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:54:24.815236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228e098-68fe-4558-b0b9-105ec325d118 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:24.835026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046783f9-9671-4bea-8eae-00fa378a32d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:24.835648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046783f9-9671-4bea-8eae-00fa378a32d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:24.835921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046783f9-9671-4bea-8eae-00fa378a32d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:24.836183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046783f9-9671-4bea-8eae-00fa378a32d9 admin admin] PATH_INFO: `/v3/projects/752227df24174003a236d784f047db75/users/d8f9414680164912ba902c7a6f2a2f83/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:24.838235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046783f9-9671-4bea-8eae-00fa378a32d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=752227df24174003a236d784f047db75, user_id=d8f9414680164912ba902c7a6f2a2f83, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:24.841337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046783f9-9671-4bea-8eae-00fa378a32d9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:24.847433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:24.857495 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1352/2679] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:24 2022] PUT /identity/v3/projects/752227df24174003a236d784f047db75/users/d8f9414680164912ba902c7a6f2a2f83/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:24.869530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b69787-fbb5-4528-a6b2-e38de9fe0e46 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:24.869804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b69787-fbb5-4528-a6b2-e38de9fe0e46 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:24.870073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b69787-fbb5-4528-a6b2-e38de9fe0e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:24.939784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:24.948313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b7b69787-fbb5-4528-a6b2-e38de9fe0e46 None None] MFA Rules not processed for user `d8f9414680164912ba902c7a6f2a2f83`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:24.949732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b7b69787-fbb5-4528-a6b2-e38de9fe0e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:25.178690 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1328/2680] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:24 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:25.484458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046783f9-9671-4bea-8eae-00fa378a32d9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:25.486948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-046783f9-9671-4bea-8eae-00fa378a32d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:25.595687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046783f9-9671-4bea-8eae-00fa378a32d9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:25.605110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e42713-406d-4897-bde4-e640c271c604 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:25.606164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e42713-406d-4897-bde4-e640c271c604 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:25.608295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e42713-406d-4897-bde4-e640c271c604 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:25.608876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e42713-406d-4897-bde4-e640c271c604 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:25.619186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e42713-406d-4897-bde4-e640c271c604 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:25.620999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e42713-406d-4897-bde4-e640c271c604 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:25.633026 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1353/2681] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:25 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:26.690045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b69787-fbb5-4528-a6b2-e38de9fe0e46 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:26.704175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b69787-fbb5-4528-a6b2-e38de9fe0e46 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:26.709707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8a0de-ea96-4633-90d8-20ad6924d792 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:26.710554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a0de-ea96-4633-90d8-20ad6924d792 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:26.711027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a0de-ea96-4633-90d8-20ad6924d792 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:26.711289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a0de-ea96-4633-90d8-20ad6924d792 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:26.769126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8a0de-ea96-4633-90d8-20ad6924d792 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:26.770793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8a0de-ea96-4633-90d8-20ad6924d792 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:26.781161 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1329/2682] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:26 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:26.814665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e42713-406d-4897-bde4-e640c271c604 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:26.878151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e42713-406d-4897-bde4-e640c271c604 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:26.882563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:26.883177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:26.883454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:26.883713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] PATH_INFO: `/v3/users/93a6541758aa4eefb5c5c2407a899943` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:26.884911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a6541758aa4eefb5c5c2407a899943)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:26.887109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:26.936626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a6541758aa4eefb5c5c2407a899943'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:26.966283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a6541758aa4eefb5c5c2407a899943'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:26.977062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a6541758aa4eefb5c5c2407a899943'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:26.986683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a6541758aa4eefb5c5c2407a899943'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:26.987365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a6541758aa4eefb5c5c2407a899943'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:26.990881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:27.000180 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1354/2683] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:26 2022] DELETE /identity/v3/users/93a6541758aa4eefb5c5c2407a899943 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:27.001406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f02f8-aa80-4d12-8f88-6b73fe2e9a9b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:27.003702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f02f8-aa80-4d12-8f88-6b73fe2e9a9b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:27.004177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f02f8-aa80-4d12-8f88-6b73fe2e9a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:27.044389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fe1f02f8-aa80-4d12-8f88-6b73fe2e9a9b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:27.049014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fe1f02f8-aa80-4d12-8f88-6b73fe2e9a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:27.192700 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1330/2684] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:27.194043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:27.199208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:27.283883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067eeec7-58f9-4ff2-ac39-1063684923f9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:27.288311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c79f22-6e42-4ef2-97ae-1547f6c12402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:27.288931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c79f22-6e42-4ef2-97ae-1547f6c12402 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:27.289248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c79f22-6e42-4ef2-97ae-1547f6c12402 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:27.289513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c79f22-6e42-4ef2-97ae-1547f6c12402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:27.307150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c79f22-6e42-4ef2-97ae-1547f6c12402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:27.308890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c79f22-6e42-4ef2-97ae-1547f6c12402 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:27.324753 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1355/2685] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:27.711384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f02f8-aa80-4d12-8f88-6b73fe2e9a9b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:27.711384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fe1f02f8-aa80-4d12-8f88-6b73fe2e9a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:27.733665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f02f8-aa80-4d12-8f88-6b73fe2e9a9b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:27.750810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:27.752259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:27.752715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:27.753161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] PATH_INFO: `/v3/users/1ddcfe1b0eb5431dbbe4777b8dd77436` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:27.754486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ddcfe1b0eb5431dbbe4777b8dd77436)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:27.756726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:27.827066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:27.862797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ddcfe1b0eb5431dbbe4777b8dd77436'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:27.880972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ddcfe1b0eb5431dbbe4777b8dd77436'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:27.894696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ddcfe1b0eb5431dbbe4777b8dd77436'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:27.904022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ddcfe1b0eb5431dbbe4777b8dd77436'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:27.904964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ddcfe1b0eb5431dbbe4777b8dd77436'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:27.913997 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1331/2686] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:27 2022] DELETE /identity/v3/users/1ddcfe1b0eb5431dbbe4777b8dd77436 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:27.921869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd4830-b7b9-4a76-a13f-fa71c05e0cdc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:27.922387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd4830-b7b9-4a76-a13f-fa71c05e0cdc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:27.922387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd4830-b7b9-4a76-a13f-fa71c05e0cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:27.955284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d4fd4830-b7b9-4a76-a13f-fa71c05e0cdc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:27.956522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4fd4830-b7b9-4a76-a13f-fa71c05e0cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:28.008177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:28.009928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:28.106122 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1356/2687] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:28.122836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2cf2f-01ba-46dd-96a5-c904f8f1c711 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:28.123159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd4830-b7b9-4a76-a13f-fa71c05e0cdc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:28.124638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4fd4830-b7b9-4a76-a13f-fa71c05e0cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:28.128376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:28.129134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:28.129494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:28.129838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] PATH_INFO: `/v3/projects/b53a4be470b7451b88037a82e4657a12` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:28.131115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b53a4be470b7451b88037a82e4657a12)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:28.133147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:28.184927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd4830-b7b9-4a76-a13f-fa71c05e0cdc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:28.236315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b53a4be470b7451b88037a82e4657a12', 'request_id': 'req-8c42a279-c3ec-4122-b935-c313db0c2483'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:28.251547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b53a4be470b7451b88037a82e4657a12', 'request_id': 'req-8c42a279-c3ec-4122-b935-c313db0c2483'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:28.273602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b53a4be470b7451b88037a82e4657a12', 'request_id': 'req-8c42a279-c3ec-4122-b935-c313db0c2483'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:28.276064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fcec1-897b-4dd1-aca0-71c728abd9d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:28.276710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fcec1-897b-4dd1-aca0-71c728abd9d5 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:28.276968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fcec1-897b-4dd1-aca0-71c728abd9d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:28.277255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fcec1-897b-4dd1-aca0-71c728abd9d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:28.288469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] The token cache is being invalidate because project b53a4be470b7451b88037a82e4657a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:28.289217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:28.290107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:28.292568 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1332/2688] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:28 2022] DELETE /identity/v3/projects/b53a4be470b7451b88037a82e4657a12 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:28.297153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:28.298924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:28.304171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:28.391277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fcec1-897b-4dd1-aca0-71c728abd9d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:28.392749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fcec1-897b-4dd1-aca0-71c728abd9d5 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:28.405981 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1357/2689] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:54:28.421403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:28.488720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42a279-c3ec-4122-b935-c313db0c2483 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:28.505589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:28.507685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:28.507685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:28.507961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] PATH_INFO: `/v3/users/d8f9414680164912ba902c7a6f2a2f83` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:28.509344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8f9414680164912ba902c7a6f2a2f83)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:28.511479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:28.578973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8f9414680164912ba902c7a6f2a2f83'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:28.595146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8f9414680164912ba902c7a6f2a2f83'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:28.616816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8f9414680164912ba902c7a6f2a2f83'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:28.629882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8f9414680164912ba902c7a6f2a2f83'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:28.630910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b0cdff08-18ee-4550-a4fe-924792b6fcf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8f9414680164912ba902c7a6f2a2f83'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:28.641202 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1333/2690] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:28 2022] DELETE /identity/v3/users/d8f9414680164912ba902c7a6f2a2f83 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:54:29.914760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fcec1-897b-4dd1-aca0-71c728abd9d5 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:29.935427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fcec1-897b-4dd1-aca0-71c728abd9d5 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:30.062118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:30.063208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:30.063849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:30.064592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] PATH_INFO: `/v3/projects/752227df24174003a236d784f047db75` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:30.066279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=752227df24174003a236d784f047db75)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:30.069137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:30.204773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '752227df24174003a236d784f047db75', 'request_id': 'req-751214ff-858a-44d8-91b3-8718d390c207'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:30.237608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '752227df24174003a236d784f047db75', 'request_id': 'req-751214ff-858a-44d8-91b3-8718d390c207'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:30.249089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '752227df24174003a236d784f047db75', 'request_id': 'req-751214ff-858a-44d8-91b3-8718d390c207'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:30.266017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] The token cache is being invalidate because project 752227df24174003a236d784f047db75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:30.267300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:30.268905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:30.271838 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1358/2691] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:29 2022] DELETE /identity/v3/projects/752227df24174003a236d784f047db75 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:30.276495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095f677-a27d-4e12-93b9-57b3aac5c083 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:30.277028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095f677-a27d-4e12-93b9-57b3aac5c083 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:30.277418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095f677-a27d-4e12-93b9-57b3aac5c083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:30.302684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5095f677-a27d-4e12-93b9-57b3aac5c083 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:30.304049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5095f677-a27d-4e12-93b9-57b3aac5c083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:30.403894 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1334/2692] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:30.410009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:30.413502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:30.419739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921be1b-b59d-4a18-ba32-dd6b74b46668 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:30.420284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921be1b-b59d-4a18-ba32-dd6b74b46668 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:30.420815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921be1b-b59d-4a18-ba32-dd6b74b46668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:30.452700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751214ff-858a-44d8-91b3-8718d390c207 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:30.467244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:30.468343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:30.470371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:30.470747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] PATH_INFO: `/v3/projects/b107efbbeb064e1fbf3c8bf31da12cbe` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:30.474022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b107efbbeb064e1fbf3c8bf31da12cbe)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:30.476427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:30.477972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8921be1b-b59d-4a18-ba32-dd6b74b46668 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:30.482996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8921be1b-b59d-4a18-ba32-dd6b74b46668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:30.507520 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1335/2693] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:30.512386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921be1b-b59d-4a18-ba32-dd6b74b46668 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:30.539322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921be1b-b59d-4a18-ba32-dd6b74b46668 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:30.556002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdde7f-f891-4f9d-99ac-3be839012988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:30.559059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdde7f-f891-4f9d-99ac-3be839012988 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:30.560531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdde7f-f891-4f9d-99ac-3be839012988 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:30.561079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdde7f-f891-4f9d-99ac-3be839012988 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:30.577305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdde7f-f891-4f9d-99ac-3be839012988 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:54:30.577305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdde7f-f891-4f9d-99ac-3be839012988 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:30.577305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdde7f-f891-4f9d-99ac-3be839012988 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:30.602352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:30.626692 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1336/2694] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:54:30.629275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdde7f-f891-4f9d-99ac-3be839012988 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:30.645953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'request_id': 'req-ce53c65c-d523-49e8-af36-4a9b925ecb2f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:30.660689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdde7f-f891-4f9d-99ac-3be839012988 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:30.666345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'request_id': 'req-ce53c65c-d523-49e8-af36-4a9b925ecb2f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:30.681841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'request_id': 'req-ce53c65c-d523-49e8-af36-4a9b925ecb2f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:30.713949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] The token cache is being invalidate because project b107efbbeb064e1fbf3c8bf31da12cbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:30.714216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:30.717368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:30.719430 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1359/2695] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:30 2022] DELETE /identity/v3/projects/b107efbbeb064e1fbf3c8bf31da12cbe => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:30.723222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:30.724644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:30.748341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:30.797898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01cdf8-0c70-4057-b1c4-9d8899ec40c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:30.798541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01cdf8-0c70-4057-b1c4-9d8899ec40c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:30.798928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01cdf8-0c70-4057-b1c4-9d8899ec40c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:30.799209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01cdf8-0c70-4057-b1c4-9d8899ec40c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:30.811696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01cdf8-0c70-4057-b1c4-9d8899ec40c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:30.818107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01cdf8-0c70-4057-b1c4-9d8899ec40c3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:30.883223 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1337/2696] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:30 2022] POST /identity/v3/projects => generated 393 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:30.899692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01cdf8-0c70-4057-b1c4-9d8899ec40c3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:30.906991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3c01cdf8-0c70-4057-b1c4-9d8899ec40c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:30.908257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53c65c-d523-49e8-af36-4a9b925ecb2f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:31.036340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01cdf8-0c70-4057-b1c4-9d8899ec40c3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:31.064404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a8881-65ac-4eec-adc7-dd2ea63dfa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:31.067015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a8881-65ac-4eec-adc7-dd2ea63dfa11 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:31.067369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a8881-65ac-4eec-adc7-dd2ea63dfa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:31.067818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a8881-65ac-4eec-adc7-dd2ea63dfa11 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:31.070024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a8881-65ac-4eec-adc7-dd2ea63dfa11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:31.072027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a8881-65ac-4eec-adc7-dd2ea63dfa11 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:31.077834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:31.078592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:31.078928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:31.079289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] PATH_INFO: `/v3/users/4024451bc4d04ab1b2230e0da780ef32` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:31.083408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4024451bc4d04ab1b2230e0da780ef32)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:31.085483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:31.142531 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1338/2697] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:30 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:31.143604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a8881-65ac-4eec-adc7-dd2ea63dfa11 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:31.158809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a8881-65ac-4eec-adc7-dd2ea63dfa11 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:31.172288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6101e2c-9bf5-4a8c-9c81-e3821b228189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:31.175813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6101e2c-9bf5-4a8c-9c81-e3821b228189 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:31.177139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6101e2c-9bf5-4a8c-9c81-e3821b228189 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:31.177644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6101e2c-9bf5-4a8c-9c81-e3821b228189 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:31.178807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6101e2c-9bf5-4a8c-9c81-e3821b228189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:31.183061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6101e2c-9bf5-4a8c-9c81-e3821b228189 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:31.198820 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1339/2698] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:31.199822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6101e2c-9bf5-4a8c-9c81-e3821b228189 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:31.206993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4024451bc4d04ab1b2230e0da780ef32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:31.212287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6101e2c-9bf5-4a8c-9c81-e3821b228189 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:31.219976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18debfb-29f3-45fd-9e5a-e0ed05309176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:31.220850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18debfb-29f3-45fd-9e5a-e0ed05309176 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:31.221321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18debfb-29f3-45fd-9e5a-e0ed05309176 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:31.221839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18debfb-29f3-45fd-9e5a-e0ed05309176 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:31.222583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18debfb-29f3-45fd-9e5a-e0ed05309176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:31.225192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18debfb-29f3-45fd-9e5a-e0ed05309176 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:31.247002 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1340/2699] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:31.250293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18debfb-29f3-45fd-9e5a-e0ed05309176 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:31.260400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4024451bc4d04ab1b2230e0da780ef32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:31.289675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4024451bc4d04ab1b2230e0da780ef32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:31.291499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:31.292322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18debfb-29f3-45fd-9e5a-e0ed05309176 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:31.312026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f8bfa-507a-41f8-a5b1-f314b12f9da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:31.312679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f8bfa-507a-41f8-a5b1-f314b12f9da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:31.314425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f8bfa-507a-41f8-a5b1-f314b12f9da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:31.314425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f8bfa-507a-41f8-a5b1-f314b12f9da3 admin admin] PATH_INFO: `/v3/projects/b8151ebb1b14413381064a8a8d4352c8/users/74a3aef2aae044f088fca1b096db33fb/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:31.317282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4024451bc4d04ab1b2230e0da780ef32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:31.320046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4024451bc4d04ab1b2230e0da780ef32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:31.337787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f8bfa-507a-41f8-a5b1-f314b12f9da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8151ebb1b14413381064a8a8d4352c8, user_id=74a3aef2aae044f088fca1b096db33fb, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:31.339700 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1360/2700] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:30 2022] DELETE /identity/v3/users/4024451bc4d04ab1b2230e0da780ef32 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:54:31.342066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f8bfa-507a-41f8-a5b1-f314b12f9da3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:31.346467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:31.354748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:31.366913 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1341/2701] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:31 2022] PUT /identity/v3/projects/b8151ebb1b14413381064a8a8d4352c8/users/74a3aef2aae044f088fca1b096db33fb/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:31.378405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2c870-90cb-41c2-9b9b-6ede34cce447 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:31.379136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2c870-90cb-41c2-9b9b-6ede34cce447 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:31.379653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2c870-90cb-41c2-9b9b-6ede34cce447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:31.430056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2fe2c870-90cb-41c2-9b9b-6ede34cce447 None None] MFA Rules not processed for user `74a3aef2aae044f088fca1b096db33fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:31.431784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2fe2c870-90cb-41c2-9b9b-6ede34cce447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:31.533559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:31.547024 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1342/2702] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:31.554924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fe409-dc11-436b-9cd7-d8bffb4c5cbf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:31.556889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2c870-90cb-41c2-9b9b-6ede34cce447 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:31.560376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2fe2c870-90cb-41c2-9b9b-6ede34cce447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:31.646228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbb6a3-90d5-42d1-9919-de1f24a10637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:31.646856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbb6a3-90d5-42d1-9919-de1f24a10637 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:31.647124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbb6a3-90d5-42d1-9919-de1f24a10637 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:31.647385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbb6a3-90d5-42d1-9919-de1f24a10637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:31.651548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7fbbb6a3-90d5-42d1-9919-de1f24a10637 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:31.675342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2c870-90cb-41c2-9b9b-6ede34cce447 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:31.683601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f09dfb-99de-4d9a-af20-33bc955ec822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:31.684275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f09dfb-99de-4d9a-af20-33bc955ec822 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:31.685738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f09dfb-99de-4d9a-af20-33bc955ec822 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:31.686020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f09dfb-99de-4d9a-af20-33bc955ec822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:31.698011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f09dfb-99de-4d9a-af20-33bc955ec822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:31.699979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f09dfb-99de-4d9a-af20-33bc955ec822 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:31.712109 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1343/2703] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:31.793732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbbb6a3-90d5-42d1-9919-de1f24a10637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:31.796237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbbb6a3-90d5-42d1-9919-de1f24a10637 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:31.810947 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1361/2704] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:32.009869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f09dfb-99de-4d9a-af20-33bc955ec822 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:32.011929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a9f09dfb-99de-4d9a-af20-33bc955ec822 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:32.157206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f09dfb-99de-4d9a-af20-33bc955ec822 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:32.162463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:32.163809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:32.163809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:32.164034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] PATH_INFO: `/v3/projects/0c63de51873b4e1c93aae1c7980bc2be` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:32.166665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c63de51873b4e1c93aae1c7980bc2be)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:32.168481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:32.250588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbb6a3-90d5-42d1-9919-de1f24a10637 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:32.252573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7fbbb6a3-90d5-42d1-9919-de1f24a10637 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:32.268282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbb6a3-90d5-42d1-9919-de1f24a10637 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:32.280226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c63de51873b4e1c93aae1c7980bc2be', 'request_id': 'req-38ead06d-ae4e-45c6-b723-652e0e86c8e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:32.293760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c63de51873b4e1c93aae1c7980bc2be', 'request_id': 'req-38ead06d-ae4e-45c6-b723-652e0e86c8e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:32.304283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c63de51873b4e1c93aae1c7980bc2be', 'request_id': 'req-38ead06d-ae4e-45c6-b723-652e0e86c8e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:32.318895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:32.343890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] The token cache is being invalidate because project 0c63de51873b4e1c93aae1c7980bc2be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:32.344323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:32.348238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38ead06d-ae4e-45c6-b723-652e0e86c8e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:32.359078 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1344/2705] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:32 2022] DELETE /identity/v3/projects/0c63de51873b4e1c93aae1c7980bc2be => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:32.361295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55515ac4-0c8c-4d4b-bbde-a48f1fef290f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:32.361634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55515ac4-0c8c-4d4b-bbde-a48f1fef290f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:32.362455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55515ac4-0c8c-4d4b-bbde-a48f1fef290f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:32.366760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:32.367815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:32.368349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:32.368793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] PATH_INFO: `/v3/users/74a3aef2aae044f088fca1b096db33fb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:32.370452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a3aef2aae044f088fca1b096db33fb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:32.373775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:32.403881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-55515ac4-0c8c-4d4b-bbde-a48f1fef290f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:32.407821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-55515ac4-0c8c-4d4b-bbde-a48f1fef290f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:32.429297 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1345/2706] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:32.441177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787839e-6cf3-48bc-b114-dc4565431c10 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:32.441397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787839e-6cf3-48bc-b114-dc4565431c10 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:32.441751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787839e-6cf3-48bc-b114-dc4565431c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:32.453189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a3aef2aae044f088fca1b096db33fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:32.457077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:32.473925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a3aef2aae044f088fca1b096db33fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:32.484616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a3aef2aae044f088fca1b096db33fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:32.494659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c787839e-6cf3-48bc-b114-dc4565431c10 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:32.494865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a3aef2aae044f088fca1b096db33fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:32.495542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-859a2d3e-1015-4d15-bf60-b6c5edd264a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a3aef2aae044f088fca1b096db33fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:32.495890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c787839e-6cf3-48bc-b114-dc4565431c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:32.519688 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1346/2707] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:32.525859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787839e-6cf3-48bc-b114-dc4565431c10 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:32.526075 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1362/2708] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:32 2022] DELETE /identity/v3/users/74a3aef2aae044f088fca1b096db33fb => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:32.530368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3400f09-ac0a-4e32-a50b-d3925d5fea6c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:32.532016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3400f09-ac0a-4e32-a50b-d3925d5fea6c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:32.533432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3400f09-ac0a-4e32-a50b-d3925d5fea6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:32.549883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787839e-6cf3-48bc-b114-dc4565431c10 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:32.575408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e3400f09-ac0a-4e32-a50b-d3925d5fea6c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:32.576695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e3400f09-ac0a-4e32-a50b-d3925d5fea6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:32.680963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599ddf3-28d7-4bae-925d-95d6a1bbeb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:32.681922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599ddf3-28d7-4bae-925d-95d6a1bbeb46 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:32.682205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599ddf3-28d7-4bae-925d-95d6a1bbeb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:32.682603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599ddf3-28d7-4bae-925d-95d6a1bbeb46 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:32.683233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599ddf3-28d7-4bae-925d-95d6a1bbeb46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:54:32.683469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599ddf3-28d7-4bae-925d-95d6a1bbeb46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:32.686478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599ddf3-28d7-4bae-925d-95d6a1bbeb46 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:32.720014 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1347/2709] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:54:32.722219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de4529-e568-4978-bc61-9bee78ca62c1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:32.722513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de4529-e568-4978-bc61-9bee78ca62c1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:32.725081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de4529-e568-4978-bc61-9bee78ca62c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:32.743192 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1363/2710] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:32.745045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3400f09-ac0a-4e32-a50b-d3925d5fea6c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:32.767143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3400f09-ac0a-4e32-a50b-d3925d5fea6c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:32.774643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b1ad2-f3a6-448c-9498-ed7eb53aba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:32.775246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b1ad2-f3a6-448c-9498-ed7eb53aba91 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:32.775522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b1ad2-f3a6-448c-9498-ed7eb53aba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:32.775789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b1ad2-f3a6-448c-9498-ed7eb53aba91 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:32.776418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b1ad2-f3a6-448c-9498-ed7eb53aba91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:32.778703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b1ad2-f3a6-448c-9498-ed7eb53aba91 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:32.780662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-91de4529-e568-4978-bc61-9bee78ca62c1 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:32.781963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-91de4529-e568-4978-bc61-9bee78ca62c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:32.816769 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1364/2711] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:32 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:32.817905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b1ad2-f3a6-448c-9498-ed7eb53aba91 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:32.821062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3b5b1ad2-f3a6-448c-9498-ed7eb53aba91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:32.877313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:32.885891 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1348/2712] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:54:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:32.886690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de4529-e568-4978-bc61-9bee78ca62c1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:32.912169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de4529-e568-4978-bc61-9bee78ca62c1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:32.912599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b1ad2-f3a6-448c-9498-ed7eb53aba91 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:32.920125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ba177-600e-418a-b703-e73ac1451e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:32.921807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ba177-600e-418a-b703-e73ac1451e56 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:32.922560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ba177-600e-418a-b703-e73ac1451e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:32.922889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ba177-600e-418a-b703-e73ac1451e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.012805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42b684-9ab0-4cb8-b727-611012912577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:33.013468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42b684-9ab0-4cb8-b727-611012912577 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:33.013918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42b684-9ab0-4cb8-b727-611012912577 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:33.014196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42b684-9ab0-4cb8-b727-611012912577 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.014921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42b684-9ab0-4cb8-b727-611012912577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:33.016833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42b684-9ab0-4cb8-b727-611012912577 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:33.026833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ba177-600e-418a-b703-e73ac1451e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:33.030755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ba177-600e-418a-b703-e73ac1451e56 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:33.053505 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1365/2713] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:33.054361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ba177-600e-418a-b703-e73ac1451e56 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:33.056994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6c9ba177-600e-418a-b703-e73ac1451e56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:33.057228 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1349/2714] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:32 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:33.062017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42b684-9ab0-4cb8-b727-611012912577 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:33.083020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42b684-9ab0-4cb8-b727-611012912577 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:33.088234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51673a79-7b22-4985-ae07-f2c5c8d3bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:33.088838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51673a79-7b22-4985-ae07-f2c5c8d3bd7f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:33.089149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51673a79-7b22-4985-ae07-f2c5c8d3bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:33.089419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51673a79-7b22-4985-ae07-f2c5c8d3bd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.090126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51673a79-7b22-4985-ae07-f2c5c8d3bd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:33.093078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51673a79-7b22-4985-ae07-f2c5c8d3bd7f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:33.110817 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1350/2715] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:33.112381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51673a79-7b22-4985-ae07-f2c5c8d3bd7f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:33.152188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51673a79-7b22-4985-ae07-f2c5c8d3bd7f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:33.164349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3e103-8f13-4876-92b4-cd7d50828ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:33.164963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3e103-8f13-4876-92b4-cd7d50828ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:33.165267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3e103-8f13-4876-92b4-cd7d50828ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:33.165539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3e103-8f13-4876-92b4-cd7d50828ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.166187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3e103-8f13-4876-92b4-cd7d50828ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:33.167810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3e103-8f13-4876-92b4-cd7d50828ae9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:33.202469 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1351/2716] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:33.212632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3e103-8f13-4876-92b4-cd7d50828ae9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:33.220289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ba177-600e-418a-b703-e73ac1451e56 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:33.223769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:33.229779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc339d6-c508-41eb-b207-c091badd90d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:33.230488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc339d6-c508-41eb-b207-c091badd90d6 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:33.231155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc339d6-c508-41eb-b207-c091badd90d6 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:33.232118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc339d6-c508-41eb-b207-c091badd90d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.236719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3e103-8f13-4876-92b4-cd7d50828ae9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:33.260476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fc6e6-7267-4b21-b6e1-bd46eb2ef415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:33.261397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fc6e6-7267-4b21-b6e1-bd46eb2ef415 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:33.261676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fc6e6-7267-4b21-b6e1-bd46eb2ef415 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:33.261948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fc6e6-7267-4b21-b6e1-bd46eb2ef415 admin admin] PATH_INFO: `/v3/projects/457b498aaf514644ac43dbd0aa372d05/users/ae41bed0df8f4df6b242607f758ff782/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.288295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fc6e6-7267-4b21-b6e1-bd46eb2ef415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=457b498aaf514644ac43dbd0aa372d05, user_id=ae41bed0df8f4df6b242607f758ff782, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:33.294702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fc6e6-7267-4b21-b6e1-bd46eb2ef415 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:33.342295 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1352/2717] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:33 2022] PUT /identity/v3/projects/457b498aaf514644ac43dbd0aa372d05/users/ae41bed0df8f4df6b242607f758ff782/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:33.343888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7be0-8896-4262-9378-aa80683b484c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:33.344233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7be0-8896-4262-9378-aa80683b484c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:33.344514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7be0-8896-4262-9378-aa80683b484c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.363534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc339d6-c508-41eb-b207-c091badd90d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:33.370471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc339d6-c508-41eb-b207-c091badd90d6 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:33.389895 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1366/2718] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:54:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:33.418106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6dab7be0-8896-4262-9378-aa80683b484c None None] MFA Rules not processed for user `ae41bed0df8f4df6b242607f758ff782`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:33.419554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6dab7be0-8896-4262-9378-aa80683b484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:33.483891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc339d6-c508-41eb-b207-c091badd90d6 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:33.485749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cdc339d6-c508-41eb-b207-c091badd90d6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:33.524308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc339d6-c508-41eb-b207-c091badd90d6 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:33.568745 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1353/2719] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:33.588799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:33.635267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab7be0-8896-4262-9378-aa80683b484c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:33.641083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6dab7be0-8896-4262-9378-aa80683b484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:33.668112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:33.671009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:33.671303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:33.671577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] PATH_INFO: `/v3/projects/b8151ebb1b14413381064a8a8d4352c8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.672930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8151ebb1b14413381064a8a8d4352c8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:33.676061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:33.764609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab7be0-8896-4262-9378-aa80683b484c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:33.779869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8151ebb1b14413381064a8a8d4352c8', 'request_id': 'req-297f243c-1b70-4147-ba2e-57d15fa145f1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:33.793770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8151ebb1b14413381064a8a8d4352c8', 'request_id': 'req-297f243c-1b70-4147-ba2e-57d15fa145f1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:33.802697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8151ebb1b14413381064a8a8d4352c8', 'request_id': 'req-297f243c-1b70-4147-ba2e-57d15fa145f1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:33.816974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] The token cache is being invalidate because project b8151ebb1b14413381064a8a8d4352c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:33.817766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:33.818884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-297f243c-1b70-4147-ba2e-57d15fa145f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:33.824493 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1367/2720] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:33 2022] DELETE /identity/v3/projects/b8151ebb1b14413381064a8a8d4352c8 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:33.826810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cd1c9-b217-4258-907c-4323ee7557e3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:33.827487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cd1c9-b217-4258-907c-4323ee7557e3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:33.827971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cd1c9-b217-4258-907c-4323ee7557e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.855553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c8900-b2b6-406c-9557-97f0884e2c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:33.855751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f28cd1c9-b217-4258-907c-4323ee7557e3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:33.856176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c8900-b2b6-406c-9557-97f0884e2c3a service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:33.856492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c8900-b2b6-406c-9557-97f0884e2c3a service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:33.856764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c8900-b2b6-406c-9557-97f0884e2c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.857358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f28cd1c9-b217-4258-907c-4323ee7557e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:33.859260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-802c8900-b2b6-406c-9557-97f0884e2c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:33.867980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:33.944784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c8900-b2b6-406c-9557-97f0884e2c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:33.946879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c8900-b2b6-406c-9557-97f0884e2c3a service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:33.960807 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1368/2721] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:33.962295 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1354/2722] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:33.962687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07677b5e-6e9e-4911-8491-72093e3fc80f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:33.962920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07677b5e-6e9e-4911-8491-72093e3fc80f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:33.963187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07677b5e-6e9e-4911-8491-72093e3fc80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:33.990860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-07677b5e-6e9e-4911-8491-72093e3fc80f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:33.992052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-07677b5e-6e9e-4911-8491-72093e3fc80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:34.005510 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1369/2723] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:34.006669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c8900-b2b6-406c-9557-97f0884e2c3a service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:34.018382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c8900-b2b6-406c-9557-97f0884e2c3a service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:34.024916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde57ff5-5db3-46d2-b6e8-6cacb7bd28cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:34.025609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde57ff5-5db3-46d2-b6e8-6cacb7bd28cd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:34.025988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde57ff5-5db3-46d2-b6e8-6cacb7bd28cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:34.026249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde57ff5-5db3-46d2-b6e8-6cacb7bd28cd admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:34.026899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde57ff5-5db3-46d2-b6e8-6cacb7bd28cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:54:34.027389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde57ff5-5db3-46d2-b6e8-6cacb7bd28cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:34.029703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde57ff5-5db3-46d2-b6e8-6cacb7bd28cd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:34.044160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701df307-834f-4174-9b0e-205eacb8ac69 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:34.044491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701df307-834f-4174-9b0e-205eacb8ac69 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:34.044668 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1355/2724] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:54:34.045477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701df307-834f-4174-9b0e-205eacb8ac69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:34.048954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde57ff5-5db3-46d2-b6e8-6cacb7bd28cd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:34.076716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-701df307-834f-4174-9b0e-205eacb8ac69 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:34.078624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-701df307-834f-4174-9b0e-205eacb8ac69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:34.087517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde57ff5-5db3-46d2-b6e8-6cacb7bd28cd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:34.112557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d09e8-bef9-4f8c-947f-4a8b3a15dcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:34.113225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d09e8-bef9-4f8c-947f-4a8b3a15dcd7 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:34.113500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d09e8-bef9-4f8c-947f-4a8b3a15dcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:34.113764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d09e8-bef9-4f8c-947f-4a8b3a15dcd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:34.114388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d09e8-bef9-4f8c-947f-4a8b3a15dcd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:34.116344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d09e8-bef9-4f8c-947f-4a8b3a15dcd7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:34.151953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:34.167776 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1370/2725] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:54:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:34.172383 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1356/2726] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:34 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:34.177025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701df307-834f-4174-9b0e-205eacb8ac69 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:34.201261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701df307-834f-4174-9b0e-205eacb8ac69 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:34.205391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d09e8-bef9-4f8c-947f-4a8b3a15dcd7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:34.208272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-647d09e8-bef9-4f8c-947f-4a8b3a15dcd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:34.307292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fe84f-1dd0-4f63-bc5e-087d742e83d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:34.308078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fe84f-1dd0-4f63-bc5e-087d742e83d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:34.308465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fe84f-1dd0-4f63-bc5e-087d742e83d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:34.308840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fe84f-1dd0-4f63-bc5e-087d742e83d0 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:34.312423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fe84f-1dd0-4f63-bc5e-087d742e83d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:34.312423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fe84f-1dd0-4f63-bc5e-087d742e83d0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:34.335129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d09e8-bef9-4f8c-947f-4a8b3a15dcd7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:34.345344 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1371/2727] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:34 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:54:34.345907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d5662-7e6e-4734-a113-5c53a62e8bbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:34.346703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fe84f-1dd0-4f63-bc5e-087d742e83d0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:34.347275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d5662-7e6e-4734-a113-5c53a62e8bbf service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:34.347861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d5662-7e6e-4734-a113-5c53a62e8bbf service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:34.348323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d5662-7e6e-4734-a113-5c53a62e8bbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:34.359958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fe84f-1dd0-4f63-bc5e-087d742e83d0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:34.365146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abc86f-2228-4f31-aa2b-912016ad5f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:34.365903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abc86f-2228-4f31-aa2b-912016ad5f28 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:34.366996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abc86f-2228-4f31-aa2b-912016ad5f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:34.366996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abc86f-2228-4f31-aa2b-912016ad5f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:34.367699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abc86f-2228-4f31-aa2b-912016ad5f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:34.370113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abc86f-2228-4f31-aa2b-912016ad5f28 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:34.385641 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1372/2728] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:34.387730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abc86f-2228-4f31-aa2b-912016ad5f28 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:34.399382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abc86f-2228-4f31-aa2b-912016ad5f28 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:34.407514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18d680-4690-464a-a716-938e49a5e5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:34.410176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18d680-4690-464a-a716-938e49a5e5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:34.410582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18d680-4690-464a-a716-938e49a5e5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:34.410960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18d680-4690-464a-a716-938e49a5e5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:34.412789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18d680-4690-464a-a716-938e49a5e5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:34.414600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18d680-4690-464a-a716-938e49a5e5e3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:34.432737 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1373/2729] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:34.433376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18d680-4690-464a-a716-938e49a5e5e3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:34.446995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18d680-4690-464a-a716-938e49a5e5e3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:34.457130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4927fc2-febe-4c3d-b841-8d6df24eaadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:34.457928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4927fc2-febe-4c3d-b841-8d6df24eaadf admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:34.458311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4927fc2-febe-4c3d-b841-8d6df24eaadf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:34.458683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4927fc2-febe-4c3d-b841-8d6df24eaadf admin admin] PATH_INFO: `/v3/projects/c456724943014d179f9aab69cb956cc7/users/0d6fa4a9541641a5a36e909dc19b491d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:34.471603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d5662-7e6e-4734-a113-5c53a62e8bbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:34.473253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d5662-7e6e-4734-a113-5c53a62e8bbf service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:34.492474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4927fc2-febe-4c3d-b841-8d6df24eaadf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c456724943014d179f9aab69cb956cc7, user_id=0d6fa4a9541641a5a36e909dc19b491d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:34.495509 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1357/2730] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:54:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 20:54:34.495933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4927fc2-febe-4c3d-b841-8d6df24eaadf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:34.524643 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1374/2731] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:34 2022] PUT /identity/v3/projects/c456724943014d179f9aab69cb956cc7/users/0d6fa4a9541641a5a36e909dc19b491d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:34.535564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07553f7-d55e-48f1-8367-81dfb94dc2ab None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:34.536025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07553f7-d55e-48f1-8367-81dfb94dc2ab None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:34.536458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07553f7-d55e-48f1-8367-81dfb94dc2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:34.571703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:34.604622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d07553f7-d55e-48f1-8367-81dfb94dc2ab None None] MFA Rules not processed for user `0d6fa4a9541641a5a36e909dc19b491d`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:34.606009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d07553f7-d55e-48f1-8367-81dfb94dc2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:34.782574 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1358/2732] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:34.842796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8a6ed-2d3b-4dd3-9c4d-8e538be35297 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:34.843127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8a6ed-2d3b-4dd3-9c4d-8e538be35297 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:34.843591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8a6ed-2d3b-4dd3-9c4d-8e538be35297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:34.849313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07553f7-d55e-48f1-8367-81dfb94dc2ab None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:34.850785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d07553f7-d55e-48f1-8367-81dfb94dc2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:34.897217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-edd8a6ed-2d3b-4dd3-9c4d-8e538be35297 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:34.907494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-edd8a6ed-2d3b-4dd3-9c4d-8e538be35297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:35.040815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07553f7-d55e-48f1-8367-81dfb94dc2ab None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:35.049669 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1375/2733] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:54:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:35.053090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a1b66-996f-4f0c-a033-f4f05cad9128 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:35.054188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a1b66-996f-4f0c-a033-f4f05cad9128 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:35.054657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a1b66-996f-4f0c-a033-f4f05cad9128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:35.060474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554bcd2-3e47-4479-bd5c-464992e29073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:35.061229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554bcd2-3e47-4479-bd5c-464992e29073 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:35.061602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554bcd2-3e47-4479-bd5c-464992e29073 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:35.061970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554bcd2-3e47-4479-bd5c-464992e29073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:35.083136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554bcd2-3e47-4479-bd5c-464992e29073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:35.084900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554bcd2-3e47-4479-bd5c-464992e29073 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:35.099062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f17a1b66-996f-4f0c-a033-f4f05cad9128 None None] MFA Rules not processed for user `c613201f55b94a528c6c36e5b5772799`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:35.100870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f17a1b66-996f-4f0c-a033-f4f05cad9128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:35.109237 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1359/2734] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:54:35.111475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554bcd2-3e47-4479-bd5c-464992e29073 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:35.222820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:35.282291 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1376/2735] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:35.292321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554bcd2-3e47-4479-bd5c-464992e29073 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:35.300429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97060d-0213-4720-9154-2d325aadc174 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:35.300624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97060d-0213-4720-9154-2d325aadc174 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:35.300748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97060d-0213-4720-9154-2d325aadc174 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:35.301005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97060d-0213-4720-9154-2d325aadc174 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:35.310588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97060d-0213-4720-9154-2d325aadc174 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:35.319534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97060d-0213-4720-9154-2d325aadc174 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:35.322155 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1360/2736] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:54:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:35.536444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a1b66-996f-4f0c-a033-f4f05cad9128 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:35.538167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f17a1b66-996f-4f0c-a033-f4f05cad9128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:35.698631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97060d-0213-4720-9154-2d325aadc174 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:35.701402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fb97060d-0213-4720-9154-2d325aadc174 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:35.716944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a1b66-996f-4f0c-a033-f4f05cad9128 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:35.727514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97060d-0213-4720-9154-2d325aadc174 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:35.729188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:35.729933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:35.730213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:35.730479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] PATH_INFO: `/v3/users/3e76b25814474b9db035954b3a437c96` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:35.735316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e76b25814474b9db035954b3a437c96)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:35.741737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:35.748280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:35.749344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:35.749931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:35.750456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] PATH_INFO: `/v3/users/8bbc891924f84d99aac22518f69e7b99` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:35.752721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bbc891924f84d99aac22518f69e7b99)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:35.767100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:35.795334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:35.842701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e76b25814474b9db035954b3a437c96'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:35.869069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e76b25814474b9db035954b3a437c96'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:35.877092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bbc891924f84d99aac22518f69e7b99'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:35.888997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e76b25814474b9db035954b3a437c96'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:35.897300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bbc891924f84d99aac22518f69e7b99'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:35.915073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e76b25814474b9db035954b3a437c96'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:35.917755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e2a60329-bbab-4bf1-8f15-8d6588d27a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e76b25814474b9db035954b3a437c96'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:35.926110 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1377/2737] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:35 2022] DELETE /identity/v3/users/3e76b25814474b9db035954b3a437c96 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:35.931295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36240501-05d8-46e4-a095-5c89a3e5adc0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:35.931805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36240501-05d8-46e4-a095-5c89a3e5adc0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:35.932220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36240501-05d8-46e4-a095-5c89a3e5adc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:35.959979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bbc891924f84d99aac22518f69e7b99'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:35.975372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bbc891924f84d99aac22518f69e7b99'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:35.978256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bbc891924f84d99aac22518f69e7b99'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:35.992413 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1361/2738] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:35 2022] DELETE /identity/v3/users/8bbc891924f84d99aac22518f69e7b99 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:35.999957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-36240501-05d8-46e4-a095-5c89a3e5adc0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:36.003521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-36240501-05d8-46e4-a095-5c89a3e5adc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:36.112472 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1378/2739] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:54:36.129370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:36.133762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:36.227828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00efab-fd44-4cea-b66a-bcd84534fb5f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:36.234011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36621185-62ed-474d-a8e3-e9b6fdee0556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:36.234688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36621185-62ed-474d-a8e3-e9b6fdee0556 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:36.234964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36621185-62ed-474d-a8e3-e9b6fdee0556 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:36.235231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36621185-62ed-474d-a8e3-e9b6fdee0556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:36.258947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36621185-62ed-474d-a8e3-e9b6fdee0556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:36.260559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36621185-62ed-474d-a8e3-e9b6fdee0556 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:36.278550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1362/2740] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:36.598739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36240501-05d8-46e4-a095-5c89a3e5adc0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:36.600157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-36240501-05d8-46e4-a095-5c89a3e5adc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:36.628362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36621185-62ed-474d-a8e3-e9b6fdee0556 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:36.630382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-36621185-62ed-474d-a8e3-e9b6fdee0556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:36.655388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36240501-05d8-46e4-a095-5c89a3e5adc0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:36.660144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36621185-62ed-474d-a8e3-e9b6fdee0556 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:36.674316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:36.675234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:36.675697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:36.676134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] PATH_INFO: `/v3/projects/1c30aa11fd244564be716550bfe5aaa6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:36.679484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c30aa11fd244564be716550bfe5aaa6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:36.681461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:36.682241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:36.683746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:36.687986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:36.688390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] PATH_INFO: `/v3/users/ae41bed0df8f4df6b242607f758ff782` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:36.690403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae41bed0df8f4df6b242607f758ff782)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:36.693890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:36.694788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:36.797010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c30aa11fd244564be716550bfe5aaa6', 'request_id': 'req-220c08ef-edd5-4b10-851d-dd90f3d76051'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:36.809396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c30aa11fd244564be716550bfe5aaa6', 'request_id': 'req-220c08ef-edd5-4b10-851d-dd90f3d76051'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:36.820305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c30aa11fd244564be716550bfe5aaa6', 'request_id': 'req-220c08ef-edd5-4b10-851d-dd90f3d76051'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:36.841492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] The token cache is being invalidate because project 1c30aa11fd244564be716550bfe5aaa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:36.841898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:36.842825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:36.844588 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1379/2741] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:36 2022] DELETE /identity/v3/projects/1c30aa11fd244564be716550bfe5aaa6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:36.846321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:36.850906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:36.856340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae41bed0df8f4df6b242607f758ff782'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:36.873875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae41bed0df8f4df6b242607f758ff782'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:36.906447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae41bed0df8f4df6b242607f758ff782'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:36.922299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae41bed0df8f4df6b242607f758ff782'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:36.923135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae41bed0df8f4df6b242607f758ff782'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:36.939243 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1363/2742] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:36 2022] DELETE /identity/v3/users/ae41bed0df8f4df6b242607f758ff782 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:36.943348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:36.946238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:36.955628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:36.989347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c08ef-edd5-4b10-851d-dd90f3d76051 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:37.098459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6c448-efb4-4892-97b3-ef15a17d7edd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:37.113640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:37.115505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.116689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.117935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] PATH_INFO: `/v3/users/0d6fa4a9541641a5a36e909dc19b491d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.119482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6fa4a9541641a5a36e909dc19b491d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:37.120246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:37.121068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.123542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.123542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] PATH_INFO: `/v3/projects/e5c50f30f706437ea737a3d5d20ea8a1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.124548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:37.125395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5c50f30f706437ea737a3d5d20ea8a1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:37.131691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:37.218751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6fa4a9541641a5a36e909dc19b491d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:37.232242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6fa4a9541641a5a36e909dc19b491d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:37.250206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:37.259919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5c50f30f706437ea737a3d5d20ea8a1', 'request_id': 'req-723d6ef8-90ee-4d4e-bb48-9af1609792cb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:37.260357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6fa4a9541641a5a36e909dc19b491d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:37.272066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5c50f30f706437ea737a3d5d20ea8a1', 'request_id': 'req-723d6ef8-90ee-4d4e-bb48-9af1609792cb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:37.280468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6fa4a9541641a5a36e909dc19b491d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:37.281202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-747b59b4-3c9f-480f-b4bf-548f4343d0fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6fa4a9541641a5a36e909dc19b491d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:37.286103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5c50f30f706437ea737a3d5d20ea8a1', 'request_id': 'req-723d6ef8-90ee-4d4e-bb48-9af1609792cb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:37.293223 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1364/2743] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:36 2022] DELETE /identity/v3/users/0d6fa4a9541641a5a36e909dc19b491d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:37.294024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d47ecb-e1ad-4a01-aa1c-749e8ee19d57 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.294860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d47ecb-e1ad-4a01-aa1c-749e8ee19d57 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.295197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d47ecb-e1ad-4a01-aa1c-749e8ee19d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.311574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] The token cache is being invalidate because project e5c50f30f706437ea737a3d5d20ea8a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:37.311858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:37.312567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-723d6ef8-90ee-4d4e-bb48-9af1609792cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:37.314111 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1380/2744] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:36 2022] DELETE /identity/v3/projects/e5c50f30f706437ea737a3d5d20ea8a1 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:37.315301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef913d-c019-4504-b8de-87b05e138840 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.315571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef913d-c019-4504-b8de-87b05e138840 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.315826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef913d-c019-4504-b8de-87b05e138840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.337136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-33d47ecb-e1ad-4a01-aa1c-749e8ee19d57 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:37.338926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-acef913d-c019-4504-b8de-87b05e138840 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:37.339265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-33d47ecb-e1ad-4a01-aa1c-749e8ee19d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:37.340441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-acef913d-c019-4504-b8de-87b05e138840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:37.451470 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1381/2745] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:37.453126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33922d-43cd-426f-b30f-7d2d0f5f3f20 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.459661 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1365/2746] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:37.461869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33922d-43cd-426f-b30f-7d2d0f5f3f20 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.463169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33922d-43cd-426f-b30f-7d2d0f5f3f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.468924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efcb1a-97d1-468b-8917-207456e7adde None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.469569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efcb1a-97d1-468b-8917-207456e7adde None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.469850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efcb1a-97d1-468b-8917-207456e7adde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.499049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b6efcb1a-97d1-468b-8917-207456e7adde None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:37.500851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b6efcb1a-97d1-468b-8917-207456e7adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:37.505517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7d33922d-43cd-426f-b30f-7d2d0f5f3f20 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:37.507040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7d33922d-43cd-426f-b30f-7d2d0f5f3f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:37.509884 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1366/2747] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:37.511236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efcb1a-97d1-468b-8917-207456e7adde None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:37.513499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b6efcb1a-97d1-468b-8917-207456e7adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:37.524562 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1382/2748] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:37.526294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64808944-d5ed-4e85-b937-370b83adf8f6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.526513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64808944-d5ed-4e85-b937-370b83adf8f6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.526775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64808944-d5ed-4e85-b937-370b83adf8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.551384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:37.573509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-64808944-d5ed-4e85-b937-370b83adf8f6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:37.574725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-64808944-d5ed-4e85-b937-370b83adf8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:37.589436 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1383/2749] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:37.589436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500e4c9-d650-4ae4-9bde-264a50ad48e3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.589436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500e4c9-d650-4ae4-9bde-264a50ad48e3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.589436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500e4c9-d650-4ae4-9bde-264a50ad48e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.610475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8500e4c9-d650-4ae4-9bde-264a50ad48e3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:37.611190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8500e4c9-d650-4ae4-9bde-264a50ad48e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:37.620954 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1384/2750] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:37.622307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500e4c9-d650-4ae4-9bde-264a50ad48e3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:37.623815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8500e4c9-d650-4ae4-9bde-264a50ad48e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:37.638145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500e4c9-d650-4ae4-9bde-264a50ad48e3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:37.643143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ce9d2-46d9-48d2-adfe-f5f0a8b3a1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:37.643867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ce9d2-46d9-48d2-adfe-f5f0a8b3a1cf service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.644237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ce9d2-46d9-48d2-adfe-f5f0a8b3a1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.644595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ce9d2-46d9-48d2-adfe-f5f0a8b3a1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.648728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efcb1a-97d1-468b-8917-207456e7adde None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:37.656864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7732fc7-6c7c-4e7e-ae93-a90f1ac77864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:37.657520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7732fc7-6c7c-4e7e-ae93-a90f1ac77864 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.657797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7732fc7-6c7c-4e7e-ae93-a90f1ac77864 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.658065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7732fc7-6c7c-4e7e-ae93-a90f1ac77864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.660433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ce9d2-46d9-48d2-adfe-f5f0a8b3a1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:37.662011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ce9d2-46d9-48d2-adfe-f5f0a8b3a1cf service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:37.675798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7732fc7-6c7c-4e7e-ae93-a90f1ac77864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:37.677569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7732fc7-6c7c-4e7e-ae93-a90f1ac77864 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:37.693433 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1385/2751] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:37.694626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ce9d2-46d9-48d2-adfe-f5f0a8b3a1cf service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:37.701733 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1367/2752] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:37.702974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7732fc7-6c7c-4e7e-ae93-a90f1ac77864 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:37.715982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ce9d2-46d9-48d2-adfe-f5f0a8b3a1cf service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:37.728344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bd24e-0db4-48a9-ae3a-d9ff7a821a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:37.729247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bd24e-0db4-48a9-ae3a-d9ff7a821a25 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.729702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bd24e-0db4-48a9-ae3a-d9ff7a821a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.730469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7732fc7-6c7c-4e7e-ae93-a90f1ac77864 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:37.730902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bd24e-0db4-48a9-ae3a-d9ff7a821a25 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.732233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bd24e-0db4-48a9-ae3a-d9ff7a821a25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:54:37.734423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bd24e-0db4-48a9-ae3a-d9ff7a821a25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:37.736317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bd24e-0db4-48a9-ae3a-d9ff7a821a25 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:37.737588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ace87-648e-4f8a-9d61-cbf86491a956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:37.738205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ace87-648e-4f8a-9d61-cbf86491a956 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.738580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ace87-648e-4f8a-9d61-cbf86491a956 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.738969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ace87-648e-4f8a-9d61-cbf86491a956 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.740799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ace87-648e-4f8a-9d61-cbf86491a956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:54:37.741255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ace87-648e-4f8a-9d61-cbf86491a956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:37.743190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ace87-648e-4f8a-9d61-cbf86491a956 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:37.760081 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1368/2753] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:54:37.763750 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1386/2754] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:54:37.765089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bd24e-0db4-48a9-ae3a-d9ff7a821a25 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:37.766006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ace87-648e-4f8a-9d61-cbf86491a956 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:37.786794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ace87-648e-4f8a-9d61-cbf86491a956 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:37.787630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bd24e-0db4-48a9-ae3a-d9ff7a821a25 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:37.803550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e16cda-48b1-41b0-ae4d-feae8f5f4f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:37.805541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e16cda-48b1-41b0-ae4d-feae8f5f4f13 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.805872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6cf953-ce85-487d-b0e5-3a4e6dec737c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:37.806482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cf953-ce85-487d-b0e5-3a4e6dec737c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:37.806755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cf953-ce85-487d-b0e5-3a4e6dec737c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.806900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e16cda-48b1-41b0-ae4d-feae8f5f4f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:37.807007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cf953-ce85-487d-b0e5-3a4e6dec737c admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.807446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e16cda-48b1-41b0-ae4d-feae8f5f4f13 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:37.807635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6cf953-ce85-487d-b0e5-3a4e6dec737c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:37.811080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e16cda-48b1-41b0-ae4d-feae8f5f4f13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:37.811238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6cf953-ce85-487d-b0e5-3a4e6dec737c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:37.820872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e16cda-48b1-41b0-ae4d-feae8f5f4f13 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:37.883057 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1369/2755] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:37 2022] POST /identity/v3/projects => generated 391 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:37.884121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6cf953-ce85-487d-b0e5-3a4e6dec737c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:37.896579 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1387/2756] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:37 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:37.901629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e16cda-48b1-41b0-ae4d-feae8f5f4f13 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:37.972184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6cf953-ce85-487d-b0e5-3a4e6dec737c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:38.060269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee36cf-05bc-489c-8a8a-c6db499ae5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:38.060994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee36cf-05bc-489c-8a8a-c6db499ae5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.061401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee36cf-05bc-489c-8a8a-c6db499ae5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.061731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee36cf-05bc-489c-8a8a-c6db499ae5f2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.062430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee36cf-05bc-489c-8a8a-c6db499ae5f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:38.066157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee36cf-05bc-489c-8a8a-c6db499ae5f2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:38.096022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e16cda-48b1-41b0-ae4d-feae8f5f4f13 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:38.096990 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1370/2757] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:37 2022] POST /identity/v3/users => generated 296 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:38.101391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee36cf-05bc-489c-8a8a-c6db499ae5f2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:38.124304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b6d4f-3a93-4ef2-995d-516dd2c319c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:38.126936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b6d4f-3a93-4ef2-995d-516dd2c319c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.128482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b6d4f-3a93-4ef2-995d-516dd2c319c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.128967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b6d4f-3a93-4ef2-995d-516dd2c319c4 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.129966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee36cf-05bc-489c-8a8a-c6db499ae5f2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:38.131676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b6d4f-3a93-4ef2-995d-516dd2c319c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:38.135374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b6d4f-3a93-4ef2-995d-516dd2c319c4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:38.148321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370f52b-46d7-46c8-b8bb-aae7ce5d2d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:38.150436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370f52b-46d7-46c8-b8bb-aae7ce5d2d33 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.150714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370f52b-46d7-46c8-b8bb-aae7ce5d2d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.150985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370f52b-46d7-46c8-b8bb-aae7ce5d2d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.151644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370f52b-46d7-46c8-b8bb-aae7ce5d2d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:38.152867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:38.154524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370f52b-46d7-46c8-b8bb-aae7ce5d2d33 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:38.178916 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1388/2758] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:37 2022] POST /identity/v3/users => generated 288 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:38.182507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b6d4f-3a93-4ef2-995d-516dd2c319c4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:38.196844 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1371/2759] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:54:38.198617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370f52b-46d7-46c8-b8bb-aae7ce5d2d33 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:38.204272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b6d4f-3a93-4ef2-995d-516dd2c319c4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:38.210175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65efc3-0d54-420d-a2b0-f40db2add79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:38.210799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65efc3-0d54-420d-a2b0-f40db2add79e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.211784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65efc3-0d54-420d-a2b0-f40db2add79e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.211784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65efc3-0d54-420d-a2b0-f40db2add79e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.212007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65efc3-0d54-420d-a2b0-f40db2add79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:38.217569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65efc3-0d54-420d-a2b0-f40db2add79e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:38.229242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370f52b-46d7-46c8-b8bb-aae7ce5d2d33 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:38.246012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21fb2b-4b5a-491b-8f62-d056fd08210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:38.246638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21fb2b-4b5a-491b-8f62-d056fd08210e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.248211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21fb2b-4b5a-491b-8f62-d056fd08210e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.250019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21fb2b-4b5a-491b-8f62-d056fd08210e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.250720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21fb2b-4b5a-491b-8f62-d056fd08210e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:38.251203 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1389/2760] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:54:38.253162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21fb2b-4b5a-491b-8f62-d056fd08210e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:38.254214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65efc3-0d54-420d-a2b0-f40db2add79e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:38.261113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:38.278359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65efc3-0d54-420d-a2b0-f40db2add79e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:38.283317 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1372/2761] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:38.286776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39080b61-4835-4a17-a70b-900139732d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:38.287424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39080b61-4835-4a17-a70b-900139732d07 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.287699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39080b61-4835-4a17-a70b-900139732d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.287970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39080b61-4835-4a17-a70b-900139732d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.288620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39080b61-4835-4a17-a70b-900139732d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:38.290401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21fb2b-4b5a-491b-8f62-d056fd08210e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:38.290906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39080b61-4835-4a17-a70b-900139732d07 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:38.311496 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1390/2762] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:38.314482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39080b61-4835-4a17-a70b-900139732d07 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:38.320854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21fb2b-4b5a-491b-8f62-d056fd08210e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:38.335757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e42d5-b358-471d-ace7-68008c5d8916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:38.336592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e42d5-b358-471d-ace7-68008c5d8916 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.336934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e42d5-b358-471d-ace7-68008c5d8916 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.337318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e42d5-b358-471d-ace7-68008c5d8916 admin admin] PATH_INFO: `/v3/projects/421588a6feed49f28f8ab75d089c01dc/users/42766820f2a940a3bf3ff646bd358414/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.349753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39080b61-4835-4a17-a70b-900139732d07 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:38.359069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec2282-ad60-43cf-8011-d7a64e081159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:38.360876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec2282-ad60-43cf-8011-d7a64e081159 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.361412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec2282-ad60-43cf-8011-d7a64e081159 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.361851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec2282-ad60-43cf-8011-d7a64e081159 admin admin] PATH_INFO: `/v3/projects/1deec50c0f374014bb6a24d2cb8be67b/users/f049946797ae43b3bd722e6d20cb8cce/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.374159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e42d5-b358-471d-ace7-68008c5d8916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=421588a6feed49f28f8ab75d089c01dc, user_id=42766820f2a940a3bf3ff646bd358414, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:38.377772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e42d5-b358-471d-ace7-68008c5d8916 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:38.385586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec2282-ad60-43cf-8011-d7a64e081159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1deec50c0f374014bb6a24d2cb8be67b, user_id=f049946797ae43b3bd722e6d20cb8cce, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:38.390402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec2282-ad60-43cf-8011-d7a64e081159 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:38.406504 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1373/2763] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:38 2022] PUT /identity/v3/projects/421588a6feed49f28f8ab75d089c01dc/users/42766820f2a940a3bf3ff646bd358414/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:38.414013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c220e5-6ea7-4bfb-a784-c4c66818f58a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.415847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c220e5-6ea7-4bfb-a784-c4c66818f58a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.416867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c220e5-6ea7-4bfb-a784-c4c66818f58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.419717 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1391/2764] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:38 2022] PUT /identity/v3/projects/1deec50c0f374014bb6a24d2cb8be67b/users/f049946797ae43b3bd722e6d20cb8cce/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:38.432479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf627e-e9ac-4ea7-b95c-311db44a42cf None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.434494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf627e-e9ac-4ea7-b95c-311db44a42cf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.435153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf627e-e9ac-4ea7-b95c-311db44a42cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.459427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-97c220e5-6ea7-4bfb-a784-c4c66818f58a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:38.462259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-97c220e5-6ea7-4bfb-a784-c4c66818f58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:38.489249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7caf627e-e9ac-4ea7-b95c-311db44a42cf None None] MFA Rules not processed for user `f049946797ae43b3bd722e6d20cb8cce`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:38.490978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7caf627e-e9ac-4ea7-b95c-311db44a42cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:38.635958 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1392/2765] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:54:38.635958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf627e-e9ac-4ea7-b95c-311db44a42cf None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:38.635958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7caf627e-e9ac-4ea7-b95c-311db44a42cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:38.656659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf627e-e9ac-4ea7-b95c-311db44a42cf None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:38.657354 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1374/2766] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:38.659145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c220e5-6ea7-4bfb-a784-c4c66818f58a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:38.661256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-97c220e5-6ea7-4bfb-a784-c4c66818f58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:38.665696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:38.666337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.666610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.666878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] PATH_INFO: `/v3/projects/457b498aaf514644ac43dbd0aa372d05` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.668365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] RBAC: Authorizing `identity:delete_project(project_id=457b498aaf514644ac43dbd0aa372d05)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:38.671616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:38.774621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '457b498aaf514644ac43dbd0aa372d05', 'request_id': 'req-fde9bbbd-a474-44c4-89cb-b611f488959c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:38.797668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '457b498aaf514644ac43dbd0aa372d05', 'request_id': 'req-fde9bbbd-a474-44c4-89cb-b611f488959c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:38.818731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c220e5-6ea7-4bfb-a784-c4c66818f58a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:38.820089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '457b498aaf514644ac43dbd0aa372d05', 'request_id': 'req-fde9bbbd-a474-44c4-89cb-b611f488959c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:38.836636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] The token cache is being invalidate because project 457b498aaf514644ac43dbd0aa372d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:38.836927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:38.837948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:38.840435 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1393/2767] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:38 2022] DELETE /identity/v3/projects/457b498aaf514644ac43dbd0aa372d05 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:38.842224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:38.843888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:38.892136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:38.931808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635391b-8992-4bd8-bae4-ad7f4d3ec150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:38.933258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635391b-8992-4bd8-bae4-ad7f4d3ec150 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:38.933962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635391b-8992-4bd8-bae4-ad7f4d3ec150 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:38.934598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635391b-8992-4bd8-bae4-ad7f4d3ec150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:38.937768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a635391b-8992-4bd8-bae4-ad7f4d3ec150 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:38.956354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:38.999850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9bbbd-a474-44c4-89cb-b611f488959c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.006382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d2a19-7cf0-4da6-a9db-5181ff80097c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:39.007276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2a19-7cf0-4da6-a9db-5181ff80097c service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.007566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2a19-7cf0-4da6-a9db-5181ff80097c service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.007817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2a19-7cf0-4da6-a9db-5181ff80097c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.011755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a50d2a19-7cf0-4da6-a9db-5181ff80097c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:39.066323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635391b-8992-4bd8-bae4-ad7f4d3ec150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.068750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635391b-8992-4bd8-bae4-ad7f4d3ec150 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.091690 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1375/2768] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:39.092932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635391b-8992-4bd8-bae4-ad7f4d3ec150 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:39.094338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a635391b-8992-4bd8-bae4-ad7f4d3ec150 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:39.136786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d2a19-7cf0-4da6-a9db-5181ff80097c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.138781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d2a19-7cf0-4da6-a9db-5181ff80097c service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.155099 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1394/2769] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:39.156457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045cadd-2a0c-4d06-a42d-52adbf43e7e7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.156737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045cadd-2a0c-4d06-a42d-52adbf43e7e7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.157004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045cadd-2a0c-4d06-a42d-52adbf43e7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.207329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e045cadd-2a0c-4d06-a42d-52adbf43e7e7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:39.208677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e045cadd-2a0c-4d06-a42d-52adbf43e7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:39.218871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635391b-8992-4bd8-bae4-ad7f4d3ec150 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.226343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:39.227998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.228368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.228729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] PATH_INFO: `/v3/projects/c456724943014d179f9aab69cb956cc7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.229506 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1395/2770] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:39.233350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c456724943014d179f9aab69cb956cc7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.233350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.238453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7038bb6-328b-4d5d-8ae7-bf6cf5f32e5f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.239103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7038bb6-328b-4d5d-8ae7-bf6cf5f32e5f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.240037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7038bb6-328b-4d5d-8ae7-bf6cf5f32e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.269319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e7038bb6-328b-4d5d-8ae7-bf6cf5f32e5f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:39.270935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e7038bb6-328b-4d5d-8ae7-bf6cf5f32e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:39.283999 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1396/2771] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:39.288478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7038bb6-328b-4d5d-8ae7-bf6cf5f32e5f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:39.303994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7038bb6-328b-4d5d-8ae7-bf6cf5f32e5f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.311180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29b55a-128b-43e4-90b2-778245507e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:39.311797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29b55a-128b-43e4-90b2-778245507e34 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.312072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29b55a-128b-43e4-90b2-778245507e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.312345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29b55a-128b-43e4-90b2-778245507e34 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.313066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29b55a-128b-43e4-90b2-778245507e34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:54:39.313414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29b55a-128b-43e4-90b2-778245507e34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.315841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29b55a-128b-43e4-90b2-778245507e34 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.341388 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1397/2772] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:54:39.344041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29b55a-128b-43e4-90b2-778245507e34 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:39.354310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c456724943014d179f9aab69cb956cc7', 'request_id': 'req-6ae4adb0-e803-49fb-affa-cd00acaf69c3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:39.374084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29b55a-128b-43e4-90b2-778245507e34 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.379588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c456724943014d179f9aab69cb956cc7', 'request_id': 'req-6ae4adb0-e803-49fb-affa-cd00acaf69c3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:39.392217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c456724943014d179f9aab69cb956cc7', 'request_id': 'req-6ae4adb0-e803-49fb-affa-cd00acaf69c3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:39.414152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] The token cache is being invalidate because project c456724943014d179f9aab69cb956cc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:39.414581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:39.415712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6ae4adb0-e803-49fb-affa-cd00acaf69c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:39.419719 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1376/2773] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:39 2022] DELETE /identity/v3/projects/c456724943014d179f9aab69cb956cc7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:39.424260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a26a46-c65d-46b1-969c-0fed7f5bf09b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.424899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a26a46-c65d-46b1-969c-0fed7f5bf09b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.425282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a26a46-c65d-46b1-969c-0fed7f5bf09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.456963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be276ddb-fdcb-41c0-80d1-d7b38ca255c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:39.457865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be276ddb-fdcb-41c0-80d1-d7b38ca255c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.458142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be276ddb-fdcb-41c0-80d1-d7b38ca255c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.458406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be276ddb-fdcb-41c0-80d1-d7b38ca255c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.459370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be276ddb-fdcb-41c0-80d1-d7b38ca255c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.461953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-55a26a46-c65d-46b1-969c-0fed7f5bf09b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:39.462476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be276ddb-fdcb-41c0-80d1-d7b38ca255c9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.466606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-55a26a46-c65d-46b1-969c-0fed7f5bf09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:39.488207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1377/2774] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:39.497673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25ce08-c9b9-4aa1-aa3a-e308fa4793b3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.498045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25ce08-c9b9-4aa1-aa3a-e308fa4793b3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.498319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25ce08-c9b9-4aa1-aa3a-e308fa4793b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.528957 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1398/2775] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:39 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:39.533357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be276ddb-fdcb-41c0-80d1-d7b38ca255c9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:39.534205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8c25ce08-c9b9-4aa1-aa3a-e308fa4793b3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:39.536338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-be276ddb-fdcb-41c0-80d1-d7b38ca255c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:39.536957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8c25ce08-c9b9-4aa1-aa3a-e308fa4793b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:39.602151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:39.603664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:39.661308 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1378/2776] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:39.665878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25ce08-c9b9-4aa1-aa3a-e308fa4793b3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:39.674788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be276ddb-fdcb-41c0-80d1-d7b38ca255c9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.678227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25ce08-c9b9-4aa1-aa3a-e308fa4793b3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.683583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f50386-6963-4e8a-8bdc-1be83004833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:39.684182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f50386-6963-4e8a-8bdc-1be83004833b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.684451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f50386-6963-4e8a-8bdc-1be83004833b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.685527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f50386-6963-4e8a-8bdc-1be83004833b admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.686173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f50386-6963-4e8a-8bdc-1be83004833b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:54:39.686464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08afd98-605c-499b-9932-7f4791f6cadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:39.686614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f50386-6963-4e8a-8bdc-1be83004833b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.686844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08afd98-605c-499b-9932-7f4791f6cadc admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.687118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08afd98-605c-499b-9932-7f4791f6cadc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.687384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08afd98-605c-499b-9932-7f4791f6cadc admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.688003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08afd98-605c-499b-9932-7f4791f6cadc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.688142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f50386-6963-4e8a-8bdc-1be83004833b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.690501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08afd98-605c-499b-9932-7f4791f6cadc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.707489 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1379/2777] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:54:39.712663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f50386-6963-4e8a-8bdc-1be83004833b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:39.724667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f50386-6963-4e8a-8bdc-1be83004833b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.730615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792f538-f699-473a-b72c-5e45a8edad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:39.731298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792f538-f699-473a-b72c-5e45a8edad7d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.732772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792f538-f699-473a-b72c-5e45a8edad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.733114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792f538-f699-473a-b72c-5e45a8edad7d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.733749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792f538-f699-473a-b72c-5e45a8edad7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.736018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792f538-f699-473a-b72c-5e45a8edad7d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.741236 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1399/2778] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:39 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:39.743557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08afd98-605c-499b-9932-7f4791f6cadc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:39.776353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08afd98-605c-499b-9932-7f4791f6cadc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.780115 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1380/2779] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:39 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:39.786361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792f538-f699-473a-b72c-5e45a8edad7d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:39.790400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91114b7b-68bf-42db-9ddc-9d9349bcf070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:39.791540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91114b7b-68bf-42db-9ddc-9d9349bcf070 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.792183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91114b7b-68bf-42db-9ddc-9d9349bcf070 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.792526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91114b7b-68bf-42db-9ddc-9d9349bcf070 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.796557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91114b7b-68bf-42db-9ddc-9d9349bcf070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.803535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91114b7b-68bf-42db-9ddc-9d9349bcf070 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.831073 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1400/2780] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:39.837188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91114b7b-68bf-42db-9ddc-9d9349bcf070 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:39.855620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91114b7b-68bf-42db-9ddc-9d9349bcf070 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.912648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792f538-f699-473a-b72c-5e45a8edad7d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.925177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521032d-b8f9-4b1f-bbdb-69773855f1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:39.927832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521032d-b8f9-4b1f-bbdb-69773855f1dd admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.929418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521032d-b8f9-4b1f-bbdb-69773855f1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.929794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521032d-b8f9-4b1f-bbdb-69773855f1dd admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.930804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521032d-b8f9-4b1f-bbdb-69773855f1dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.934285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521032d-b8f9-4b1f-bbdb-69773855f1dd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.967522 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1381/2781] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:39 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:39.971909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521032d-b8f9-4b1f-bbdb-69773855f1dd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:39.977927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076449df-7dcf-4efc-b90d-4c415a5abfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:39.978555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076449df-7dcf-4efc-b90d-4c415a5abfa7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:39.978832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076449df-7dcf-4efc-b90d-4c415a5abfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:39.979099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076449df-7dcf-4efc-b90d-4c415a5abfa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:39.979792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076449df-7dcf-4efc-b90d-4c415a5abfa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:39.982630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076449df-7dcf-4efc-b90d-4c415a5abfa7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:39.996362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521032d-b8f9-4b1f-bbdb-69773855f1dd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:39.998990 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1401/2782] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:40.005184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076449df-7dcf-4efc-b90d-4c415a5abfa7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:40.013020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d5402-f5c2-479b-96e6-1f90548984f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:40.014662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d5402-f5c2-479b-96e6-1f90548984f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:40.015044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d5402-f5c2-479b-96e6-1f90548984f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:40.015373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d5402-f5c2-479b-96e6-1f90548984f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:40.016095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d5402-f5c2-479b-96e6-1f90548984f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:40.018532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d5402-f5c2-479b-96e6-1f90548984f5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:40.027627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076449df-7dcf-4efc-b90d-4c415a5abfa7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:40.035007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a9f73-e3eb-44b4-8aff-be47cbab797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:40.036330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a9f73-e3eb-44b4-8aff-be47cbab797a admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:40.036613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a9f73-e3eb-44b4-8aff-be47cbab797a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:40.037298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a9f73-e3eb-44b4-8aff-be47cbab797a admin admin] PATH_INFO: `/v3/projects/8e65df04cd7f4c6cb91f322aeadc965f/users/cfcea0f8ebea419eb6f74c1091d8fe41/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:40.047567 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1382/2783] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:54:40.051837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d5402-f5c2-479b-96e6-1f90548984f5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:40.067161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a9f73-e3eb-44b4-8aff-be47cbab797a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e65df04cd7f4c6cb91f322aeadc965f, user_id=cfcea0f8ebea419eb6f74c1091d8fe41, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:40.070170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a9f73-e3eb-44b4-8aff-be47cbab797a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:40.077507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d5402-f5c2-479b-96e6-1f90548984f5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:40.084586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9c806-2a2a-49b7-b188-f6c43a5a54cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:40.085364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9c806-2a2a-49b7-b188-f6c43a5a54cf admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:40.085697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9c806-2a2a-49b7-b188-f6c43a5a54cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:40.086085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9c806-2a2a-49b7-b188-f6c43a5a54cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:40.086905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9c806-2a2a-49b7-b188-f6c43a5a54cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:40.088787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9c806-2a2a-49b7-b188-f6c43a5a54cf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:40.097547 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1402/2784] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:40 2022] PUT /identity/v3/projects/8e65df04cd7f4c6cb91f322aeadc965f/users/cfcea0f8ebea419eb6f74c1091d8fe41/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:54:40.102640 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1383/2785] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:40.111718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1d514-05f9-4e40-a96a-3236176873a0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:40.112494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1d514-05f9-4e40-a96a-3236176873a0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:40.112769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1d514-05f9-4e40-a96a-3236176873a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:40.113462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9c806-2a2a-49b7-b188-f6c43a5a54cf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:40.133764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9c806-2a2a-49b7-b188-f6c43a5a54cf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:40.196274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-90d1d514-05f9-4e40-a96a-3236176873a0 None None] MFA Rules not processed for user `cfcea0f8ebea419eb6f74c1091d8fe41`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:40.197840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-90d1d514-05f9-4e40-a96a-3236176873a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:40.277756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95639b55-2bd5-481d-9111-69fddc3e96c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:40.279907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95639b55-2bd5-481d-9111-69fddc3e96c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:40.281195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95639b55-2bd5-481d-9111-69fddc3e96c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:40.281547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95639b55-2bd5-481d-9111-69fddc3e96c8 admin admin] PATH_INFO: `/v3/projects/ca9af3805eba4f69b474a8b2205f0920/users/2bba6c9b3f674959ad79d4a394ed9f2a/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:40.302531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:40.313663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95639b55-2bd5-481d-9111-69fddc3e96c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9af3805eba4f69b474a8b2205f0920, user_id=2bba6c9b3f674959ad79d4a394ed9f2a, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:40.317168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95639b55-2bd5-481d-9111-69fddc3e96c8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:40.356152 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1384/2786] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:40 2022] PUT /identity/v3/projects/ca9af3805eba4f69b474a8b2205f0920/users/2bba6c9b3f674959ad79d4a394ed9f2a/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:54:40.363154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419b97-69e2-4fe5-b7b3-4fd341bd548a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:40.364205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419b97-69e2-4fe5-b7b3-4fd341bd548a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:40.365213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419b97-69e2-4fe5-b7b3-4fd341bd548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:40.404800 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1403/2787] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:40.425124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1d514-05f9-4e40-a96a-3236176873a0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:40.425868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-af419b97-69e2-4fe5-b7b3-4fd341bd548a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:40.427055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-90d1d514-05f9-4e40-a96a-3236176873a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:40.427187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-af419b97-69e2-4fe5-b7b3-4fd341bd548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:40.486910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:40.584062 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1385/2788] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:40.602930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af419b97-69e2-4fe5-b7b3-4fd341bd548a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:40.605310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-af419b97-69e2-4fe5-b7b3-4fd341bd548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:40.639622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1d514-05f9-4e40-a96a-3236176873a0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:40.687096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af419b97-69e2-4fe5-b7b3-4fd341bd548a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:40.694040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be6ebc-e767-4068-a4f5-f1cc648eff4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:40.695184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be6ebc-e767-4068-a4f5-f1cc648eff4b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:40.695643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be6ebc-e767-4068-a4f5-f1cc648eff4b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:40.696073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be6ebc-e767-4068-a4f5-f1cc648eff4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:40.715491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be6ebc-e767-4068-a4f5-f1cc648eff4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:40.718177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be6ebc-e767-4068-a4f5-f1cc648eff4b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:40.736014 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1386/2789] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:40.750436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daa677-1949-423b-bc0d-46421e1a9eda service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:40.751058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa677-1949-423b-bc0d-46421e1a9eda service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:40.751332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa677-1949-423b-bc0d-46421e1a9eda service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:40.751601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa677-1949-423b-bc0d-46421e1a9eda service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:40.881902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daa677-1949-423b-bc0d-46421e1a9eda service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:40.885852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daa677-1949-423b-bc0d-46421e1a9eda service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:40.905936 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1404/2790] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:54:41.167096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be6ebc-e767-4068-a4f5-f1cc648eff4b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:41.170035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-68be6ebc-e767-4068-a4f5-f1cc648eff4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:41.218531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be6ebc-e767-4068-a4f5-f1cc648eff4b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:41.238940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:41.242311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:41.242610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:41.243153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] PATH_INFO: `/v3/users/f049946797ae43b3bd722e6d20cb8cce` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:41.248897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f049946797ae43b3bd722e6d20cb8cce)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:41.251579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:41.268215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6043f-e8ba-41f7-9f45-f22a1bcbe7b2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:41.268627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6043f-e8ba-41f7-9f45-f22a1bcbe7b2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:41.268989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6043f-e8ba-41f7-9f45-f22a1bcbe7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:41.325904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0db6043f-e8ba-41f7-9f45-f22a1bcbe7b2 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:41.327375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0db6043f-e8ba-41f7-9f45-f22a1bcbe7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:41.343509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f049946797ae43b3bd722e6d20cb8cce'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:41.352782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f049946797ae43b3bd722e6d20cb8cce'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:41.368106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f049946797ae43b3bd722e6d20cb8cce'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:41.387918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f049946797ae43b3bd722e6d20cb8cce'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:41.390273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c67d258-072d-4638-a696-b564639fd306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f049946797ae43b3bd722e6d20cb8cce'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:41.403744 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1387/2791] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:41 2022] DELETE /identity/v3/users/f049946797ae43b3bd722e6d20cb8cce => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:41.407199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fece0-73e6-43fe-8766-fa768f510858 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:41.407629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fece0-73e6-43fe-8766-fa768f510858 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:41.408686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fece0-73e6-43fe-8766-fa768f510858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:41.434030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:41.470411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-186fece0-73e6-43fe-8766-fa768f510858 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:41.474298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-186fece0-73e6-43fe-8766-fa768f510858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:41.510866 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1405/2792] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:54:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:54:41.534800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6043f-e8ba-41f7-9f45-f22a1bcbe7b2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:41.536377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0db6043f-e8ba-41f7-9f45-f22a1bcbe7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:41.642787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:41.676704 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1388/2793] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:41.696006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fece0-73e6-43fe-8766-fa768f510858 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:41.697705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-186fece0-73e6-43fe-8766-fa768f510858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:41.758772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6043f-e8ba-41f7-9f45-f22a1bcbe7b2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:41.836955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fece0-73e6-43fe-8766-fa768f510858 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:41.848183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa941c83-dbb9-4d18-b912-6b09ab8319f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:41.848823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa941c83-dbb9-4d18-b912-6b09ab8319f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:41.849129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa941c83-dbb9-4d18-b912-6b09ab8319f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:41.849447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa941c83-dbb9-4d18-b912-6b09ab8319f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:41.873725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa941c83-dbb9-4d18-b912-6b09ab8319f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:41.876728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa941c83-dbb9-4d18-b912-6b09ab8319f8 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:41.901955 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1389/2794] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:54:41.917625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe53a6d-942d-46d5-b683-6d82fdb93e7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:41.918910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe53a6d-942d-46d5-b683-6d82fdb93e7c service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:41.920096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe53a6d-942d-46d5-b683-6d82fdb93e7c service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:41.920530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe53a6d-942d-46d5-b683-6d82fdb93e7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:42.067814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe53a6d-942d-46d5-b683-6d82fdb93e7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:42.070090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe53a6d-942d-46d5-b683-6d82fdb93e7c service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:42.083960 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1406/2795] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:54:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 20:54:43.321579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa941c83-dbb9-4d18-b912-6b09ab8319f8 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:43.324296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa941c83-dbb9-4d18-b912-6b09ab8319f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:43.347895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa941c83-dbb9-4d18-b912-6b09ab8319f8 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:43.392914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:43.395235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:43.400642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:43.403080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] PATH_INFO: `/v3/projects/1deec50c0f374014bb6a24d2cb8be67b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:43.418586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1deec50c0f374014bb6a24d2cb8be67b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:43.420637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:43.552298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1deec50c0f374014bb6a24d2cb8be67b', 'request_id': 'req-076d3fad-2936-4014-8e24-2a8301908413'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:43.599696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1deec50c0f374014bb6a24d2cb8be67b', 'request_id': 'req-076d3fad-2936-4014-8e24-2a8301908413'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:43.610048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1deec50c0f374014bb6a24d2cb8be67b', 'request_id': 'req-076d3fad-2936-4014-8e24-2a8301908413'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:43.633540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] The token cache is being invalidate because project 1deec50c0f374014bb6a24d2cb8be67b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:43.633962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:43.634782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-076d3fad-2936-4014-8e24-2a8301908413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:43.641087 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1390/2796] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:43 2022] DELETE /identity/v3/projects/1deec50c0f374014bb6a24d2cb8be67b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:43.644451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aa5f5-3313-4d64-8123-9faac3170d60 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:43.644976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aa5f5-3313-4d64-8123-9faac3170d60 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:43.645357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aa5f5-3313-4d64-8123-9faac3170d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:43.706376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-976aa5f5-3313-4d64-8123-9faac3170d60 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:43.707946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-976aa5f5-3313-4d64-8123-9faac3170d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:43.913069 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1407/2797] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:43.984283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31884d5a-f8bd-4860-b4ed-ab5d1560be70 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:43.984693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31884d5a-f8bd-4860-b4ed-ab5d1560be70 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:43.985127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31884d5a-f8bd-4860-b4ed-ab5d1560be70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:44.035410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-31884d5a-f8bd-4860-b4ed-ab5d1560be70 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:44.037704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-31884d5a-f8bd-4860-b4ed-ab5d1560be70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:44.056663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976aa5f5-3313-4d64-8123-9faac3170d60 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:44.056882 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1391/2798] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:44.072928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976aa5f5-3313-4d64-8123-9faac3170d60 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:44.082490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20618675-f120-4e4c-9fe2-d4d981b13e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:44.083370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20618675-f120-4e4c-9fe2-d4d981b13e90 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:44.083656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20618675-f120-4e4c-9fe2-d4d981b13e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:44.083927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20618675-f120-4e4c-9fe2-d4d981b13e90 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:44.084627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20618675-f120-4e4c-9fe2-d4d981b13e90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:54:44.084932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20618675-f120-4e4c-9fe2-d4d981b13e90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:44.087641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20618675-f120-4e4c-9fe2-d4d981b13e90 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:44.090459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:44.104031 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1408/2799] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:54:44.109209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31884d5a-f8bd-4860-b4ed-ab5d1560be70 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:44.122957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31884d5a-f8bd-4860-b4ed-ab5d1560be70 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:44.135658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93315ed-558d-45a3-ac06-d75fe5f7b151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:44.136502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93315ed-558d-45a3-ac06-d75fe5f7b151 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:44.136728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93315ed-558d-45a3-ac06-d75fe5f7b151 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:44.137164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93315ed-558d-45a3-ac06-d75fe5f7b151 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:44.137940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93315ed-558d-45a3-ac06-d75fe5f7b151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:44.140372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93315ed-558d-45a3-ac06-d75fe5f7b151 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:44.213281 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1392/2800] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:44 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:44.218177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20618675-f120-4e4c-9fe2-d4d981b13e90 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:44.302979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20618675-f120-4e4c-9fe2-d4d981b13e90 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:44.601951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbb0e9-ca8a-4b01-927f-c62389f3bf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:44.603263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbb0e9-ca8a-4b01-927f-c62389f3bf64 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:44.603544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbb0e9-ca8a-4b01-927f-c62389f3bf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:44.603814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbb0e9-ca8a-4b01-927f-c62389f3bf64 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:44.604446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbb0e9-ca8a-4b01-927f-c62389f3bf64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:44.614052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbb0e9-ca8a-4b01-927f-c62389f3bf64 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:44.687372 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1409/2801] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:44 2022] POST /identity/v3/users => generated 289 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:44.691582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93315ed-558d-45a3-ac06-d75fe5f7b151 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:44.698254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:44.712347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93315ed-558d-45a3-ac06-d75fe5f7b151 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:44.741590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36548022-c8bf-4f40-97e5-a2aa9397713a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:44.743250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36548022-c8bf-4f40-97e5-a2aa9397713a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:44.745660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36548022-c8bf-4f40-97e5-a2aa9397713a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:44.746788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36548022-c8bf-4f40-97e5-a2aa9397713a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:44.748408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36548022-c8bf-4f40-97e5-a2aa9397713a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:44.753601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36548022-c8bf-4f40-97e5-a2aa9397713a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:44.786632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbb0e9-ca8a-4b01-927f-c62389f3bf64 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:44.787358 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1393/2802] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:44.799260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbb0e9-ca8a-4b01-927f-c62389f3bf64 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:44.804366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9466cf6-bc32-47f7-a51f-36698c411aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:44.805181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9466cf6-bc32-47f7-a51f-36698c411aad admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:44.805517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9466cf6-bc32-47f7-a51f-36698c411aad admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:44.805852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9466cf6-bc32-47f7-a51f-36698c411aad admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:44.806564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9466cf6-bc32-47f7-a51f-36698c411aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:44.808243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9466cf6-bc32-47f7-a51f-36698c411aad admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:44.825188 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1410/2803] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:44.828843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36548022-c8bf-4f40-97e5-a2aa9397713a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:44.841099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36548022-c8bf-4f40-97e5-a2aa9397713a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:44.865362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf939455-e479-42da-a4b5-6333e93a2c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:44.865792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939455-e479-42da-a4b5-6333e93a2c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:44.866063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939455-e479-42da-a4b5-6333e93a2c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:44.866336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939455-e479-42da-a4b5-6333e93a2c1a admin admin] PATH_INFO: `/v3/projects/0a0d3787e2a842029da9fb7d0a6715b0/users/de4d5d0008324d7a807a8f8361a60a4e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:44.931297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf939455-e479-42da-a4b5-6333e93a2c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0d3787e2a842029da9fb7d0a6715b0, user_id=de4d5d0008324d7a807a8f8361a60a4e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:44.936155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf939455-e479-42da-a4b5-6333e93a2c1a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:44.987569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935ad37-1a00-4c3e-a7a2-312e659827f7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:44.987860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935ad37-1a00-4c3e-a7a2-312e659827f7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:44.988135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935ad37-1a00-4c3e-a7a2-312e659827f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:44.991155 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1394/2804] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:44 2022] PUT /identity/v3/projects/0a0d3787e2a842029da9fb7d0a6715b0/users/de4d5d0008324d7a807a8f8361a60a4e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:45.029030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3935ad37-1a00-4c3e-a7a2-312e659827f7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:45.030666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3935ad37-1a00-4c3e-a7a2-312e659827f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:45.221313 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1411/2805] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:45.270719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf939455-e479-42da-a4b5-6333e93a2c1a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:45.272920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cf939455-e479-42da-a4b5-6333e93a2c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:45.398045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf939455-e479-42da-a4b5-6333e93a2c1a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:45.408488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a9614-9b10-4409-a987-2b07d2b2dfc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:45.409255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a9614-9b10-4409-a987-2b07d2b2dfc1 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:45.409531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a9614-9b10-4409-a987-2b07d2b2dfc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:45.409797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a9614-9b10-4409-a987-2b07d2b2dfc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:45.437353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a9614-9b10-4409-a987-2b07d2b2dfc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:45.438909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a9614-9b10-4409-a987-2b07d2b2dfc1 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:45.467845 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1395/2806] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:54:48.627732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdda5e-0c0b-41be-bf92-70cae36aeaf1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:48.628289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdda5e-0c0b-41be-bf92-70cae36aeaf1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:48.628289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdda5e-0c0b-41be-bf92-70cae36aeaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:48.710791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d2cdda5e-0c0b-41be-bf92-70cae36aeaf1 None None] MFA Rules not processed for user `42766820f2a940a3bf3ff646bd358414`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:48.715298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d2cdda5e-0c0b-41be-bf92-70cae36aeaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:48.940986 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1412/2807] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:48.952529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc4bc6-2ae5-442b-ae2b-126a9850d4de None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:48.953159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc4bc6-2ae5-442b-ae2b-126a9850d4de None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:48.953450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc4bc6-2ae5-442b-ae2b-126a9850d4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:49.008012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1afc4bc6-2ae5-442b-ae2b-126a9850d4de None None] MFA Rules not processed for user `42766820f2a940a3bf3ff646bd358414`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:49.009755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1afc4bc6-2ae5-442b-ae2b-126a9850d4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:49.046372 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1396/2808] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:49.058841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdda5e-0c0b-41be-bf92-70cae36aeaf1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:49.060999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d2cdda5e-0c0b-41be-bf92-70cae36aeaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:49.067897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:49.244449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdda5e-0c0b-41be-bf92-70cae36aeaf1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:49.262118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5eca2-b6ea-44e3-a1dd-661c42db24ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:49.262118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5eca2-b6ea-44e3-a1dd-661c42db24ce service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:49.262118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5eca2-b6ea-44e3-a1dd-661c42db24ce service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:49.262118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5eca2-b6ea-44e3-a1dd-661c42db24ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:49.278079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5eca2-b6ea-44e3-a1dd-661c42db24ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:49.285840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5eca2-b6ea-44e3-a1dd-661c42db24ce service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:49.313108 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1413/2809] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:49 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:51.305724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb5a45-9def-4db2-b3dd-003dd8ea6944 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:51.306372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb5a45-9def-4db2-b3dd-003dd8ea6944 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:51.306372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb5a45-9def-4db2-b3dd-003dd8ea6944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:51.346430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4cdb5a45-9def-4db2-b3dd-003dd8ea6944 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:51.348184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4cdb5a45-9def-4db2-b3dd-003dd8ea6944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:51.529351 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1397/2810] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:54:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:51.549679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5eca2-b6ea-44e3-a1dd-661c42db24ce service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:51.554724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-93b5eca2-b6ea-44e3-a1dd-661c42db24ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:51.873829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5eca2-b6ea-44e3-a1dd-661c42db24ce service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:51.900999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb91e7f-6484-48e5-8ce8-2800d0a0449f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:51.900999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb91e7f-6484-48e5-8ce8-2800d0a0449f service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:51.905684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb91e7f-6484-48e5-8ce8-2800d0a0449f service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:51.912175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb91e7f-6484-48e5-8ce8-2800d0a0449f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:51.955220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb91e7f-6484-48e5-8ce8-2800d0a0449f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:51.986339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb91e7f-6484-48e5-8ce8-2800d0a0449f service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:52.018048 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1414/2811] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:54:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:52.252952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb5a45-9def-4db2-b3dd-003dd8ea6944 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:52.254518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4cdb5a45-9def-4db2-b3dd-003dd8ea6944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:52.314443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb5a45-9def-4db2-b3dd-003dd8ea6944 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:52.325185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:52.326548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:52.328202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:52.328651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] PATH_INFO: `/v3/users/cfcea0f8ebea419eb6f74c1091d8fe41` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:52.333926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfcea0f8ebea419eb6f74c1091d8fe41)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:52.336147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:52.436019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfcea0f8ebea419eb6f74c1091d8fe41'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:52.446570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfcea0f8ebea419eb6f74c1091d8fe41'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:52.462323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:52.474484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfcea0f8ebea419eb6f74c1091d8fe41'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:52.488406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfcea0f8ebea419eb6f74c1091d8fe41'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:52.490483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfcea0f8ebea419eb6f74c1091d8fe41'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:52.514451 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1398/2812] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:54:52 2022] DELETE /identity/v3/users/cfcea0f8ebea419eb6f74c1091d8fe41 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:52.518842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae048004-0bef-4bb8-8b63-5a8d076c27d8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:52.519190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae048004-0bef-4bb8-8b63-5a8d076c27d8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:52.519522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae048004-0bef-4bb8-8b63-5a8d076c27d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:52.557905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ae048004-0bef-4bb8-8b63-5a8d076c27d8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:52.561946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ae048004-0bef-4bb8-8b63-5a8d076c27d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:52.718255 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1415/2813] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:52.757568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:52.759950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:53.004152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953cd6f-5ae3-40e9-bd9c-f2360cde9d90 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:53.018535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa8fa7-44ee-47a8-bac3-524616abae99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:53.019532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa8fa7-44ee-47a8-bac3-524616abae99 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:53.020162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa8fa7-44ee-47a8-bac3-524616abae99 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:53.020533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa8fa7-44ee-47a8-bac3-524616abae99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:53.046796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa8fa7-44ee-47a8-bac3-524616abae99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:53.051977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa8fa7-44ee-47a8-bac3-524616abae99 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:53.082282 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1399/2814] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:55.318031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae048004-0bef-4bb8-8b63-5a8d076c27d8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:55.319662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ae048004-0bef-4bb8-8b63-5a8d076c27d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:55.339773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae048004-0bef-4bb8-8b63-5a8d076c27d8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:55.362202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:55.363313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:55.363878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:55.364438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] PATH_INFO: `/v3/projects/8e65df04cd7f4c6cb91f322aeadc965f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:55.366783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e65df04cd7f4c6cb91f322aeadc965f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:55.369092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:55.422066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:55.561708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e65df04cd7f4c6cb91f322aeadc965f', 'request_id': 'req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:55.580393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e65df04cd7f4c6cb91f322aeadc965f', 'request_id': 'req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:55.635419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e65df04cd7f4c6cb91f322aeadc965f', 'request_id': 'req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:55.711809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] The token cache is being invalidate because project 8e65df04cd7f4c6cb91f322aeadc965f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:54:55.712108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:55.715013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-278d71fa-5b84-4e5b-9eb4-29d791ede6c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:54:55.720068 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1416/2815] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:55 2022] DELETE /identity/v3/projects/8e65df04cd7f4c6cb91f322aeadc965f => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:55.725018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f400391-8ea0-4bb6-b3b7-a76a368680a7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:55.725489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f400391-8ea0-4bb6-b3b7-a76a368680a7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:55.725788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f400391-8ea0-4bb6-b3b7-a76a368680a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:55.772466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9f400391-8ea0-4bb6-b3b7-a76a368680a7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:55.776220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9f400391-8ea0-4bb6-b3b7-a76a368680a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:55.965829 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1400/2816] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:55.979237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644a0b7-b1dd-4b47-b15d-3ab77d7eca04 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:55.979444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644a0b7-b1dd-4b47-b15d-3ab77d7eca04 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:55.979693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644a0b7-b1dd-4b47-b15d-3ab77d7eca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:56.018202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f644a0b7-b1dd-4b47-b15d-3ab77d7eca04 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:56.019698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f644a0b7-b1dd-4b47-b15d-3ab77d7eca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:56.045386 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1417/2817] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:54:56.055529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f400391-8ea0-4bb6-b3b7-a76a368680a7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:56.070066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f400391-8ea0-4bb6-b3b7-a76a368680a7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:56.107938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df78f6c-fad4-407d-bfd3-796edb5f2eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:56.110704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df78f6c-fad4-407d-bfd3-796edb5f2eaa admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:56.112207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df78f6c-fad4-407d-bfd3-796edb5f2eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:56.112995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df78f6c-fad4-407d-bfd3-796edb5f2eaa admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:56.131213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df78f6c-fad4-407d-bfd3-796edb5f2eaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:54:56.132140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df78f6c-fad4-407d-bfd3-796edb5f2eaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:56.138702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df78f6c-fad4-407d-bfd3-796edb5f2eaa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:56.176980 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1401/2818] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:54:56.179851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644a0b7-b1dd-4b47-b15d-3ab77d7eca04 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:56.201925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644a0b7-b1dd-4b47-b15d-3ab77d7eca04 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:56.206754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ece003-372b-4a20-adb7-d00d25ed5ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:56.207358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece003-372b-4a20-adb7-d00d25ed5ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:56.210171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece003-372b-4a20-adb7-d00d25ed5ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:56.210171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece003-372b-4a20-adb7-d00d25ed5ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:56.210692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ece003-372b-4a20-adb7-d00d25ed5ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:56.212793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ece003-372b-4a20-adb7-d00d25ed5ea5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:56.284888 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1418/2819] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:54:56 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:56.290551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df78f6c-fad4-407d-bfd3-796edb5f2eaa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:56.333814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df78f6c-fad4-407d-bfd3-796edb5f2eaa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:56.532738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1f79d-3009-43e4-83ae-698878b8a02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:56.534198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1f79d-3009-43e4-83ae-698878b8a02e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:56.537126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1f79d-3009-43e4-83ae-698878b8a02e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:56.537126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1f79d-3009-43e4-83ae-698878b8a02e admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:56.539456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1f79d-3009-43e4-83ae-698878b8a02e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:56.545568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1f79d-3009-43e4-83ae-698878b8a02e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:56.606472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:56.617162 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1402/2820] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:54:56 2022] POST /identity/v3/users => generated 283 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:54:56.620746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ece003-372b-4a20-adb7-d00d25ed5ea5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:56.634310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ece003-372b-4a20-adb7-d00d25ed5ea5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:56.652275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f84d1-08b2-4b58-8dc6-d5801c30d0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:56.652507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f84d1-08b2-4b58-8dc6-d5801c30d0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:56.655268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f84d1-08b2-4b58-8dc6-d5801c30d0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:56.656415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f84d1-08b2-4b58-8dc6-d5801c30d0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:56.658403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f84d1-08b2-4b58-8dc6-d5801c30d0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:56.660993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f84d1-08b2-4b58-8dc6-d5801c30d0d4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:56.709577 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1419/2821] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:54:56.711208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1f79d-3009-43e4-83ae-698878b8a02e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:56.728782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1f79d-3009-43e4-83ae-698878b8a02e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:56.735301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682cfa3-1ae7-4bfa-b9e7-ea42635d7674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:56.735913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682cfa3-1ae7-4bfa-b9e7-ea42635d7674 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:56.736164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682cfa3-1ae7-4bfa-b9e7-ea42635d7674 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:56.736439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682cfa3-1ae7-4bfa-b9e7-ea42635d7674 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:56.737802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682cfa3-1ae7-4bfa-b9e7-ea42635d7674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:56.739447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682cfa3-1ae7-4bfa-b9e7-ea42635d7674 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:56.757673 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1403/2822] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:54:56.760135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f84d1-08b2-4b58-8dc6-d5801c30d0d4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:56.782012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f84d1-08b2-4b58-8dc6-d5801c30d0d4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:56.788601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b6b86-43cd-4102-9b6a-2dfe42a77526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:56.789437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b6b86-43cd-4102-9b6a-2dfe42a77526 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:56.789776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b6b86-43cd-4102-9b6a-2dfe42a77526 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:56.791162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b6b86-43cd-4102-9b6a-2dfe42a77526 admin admin] PATH_INFO: `/v3/projects/b62c28804ebd4f1f9ac47aa541500110/users/5d93a680570f4d23b1f6df534bc082b2/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:56.815137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b6b86-43cd-4102-9b6a-2dfe42a77526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62c28804ebd4f1f9ac47aa541500110, user_id=5d93a680570f4d23b1f6df534bc082b2, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:56.818820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b6b86-43cd-4102-9b6a-2dfe42a77526 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:56.845116 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1420/2823] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:54:56 2022] PUT /identity/v3/projects/b62c28804ebd4f1f9ac47aa541500110/users/5d93a680570f4d23b1f6df534bc082b2/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:54:56.847844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893f40e-5e02-4eda-a378-b035c36001ed None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:56.848962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893f40e-5e02-4eda-a378-b035c36001ed None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:56.849482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893f40e-5e02-4eda-a378-b035c36001ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:56.884189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3893f40e-5e02-4eda-a378-b035c36001ed None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:56.886376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3893f40e-5e02-4eda-a378-b035c36001ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:56.993410 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1404/2824] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:54:57.001854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b6b86-43cd-4102-9b6a-2dfe42a77526 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:54:57.003711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fc7b6b86-43cd-4102-9b6a-2dfe42a77526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:54:57.161018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b6b86-43cd-4102-9b6a-2dfe42a77526 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:54:57.166666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c3d11-75ed-461e-97bf-26ead7b155e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:54:57.167594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c3d11-75ed-461e-97bf-26ead7b155e6 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:57.167976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c3d11-75ed-461e-97bf-26ead7b155e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:57.168383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c3d11-75ed-461e-97bf-26ead7b155e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:57.205950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c3d11-75ed-461e-97bf-26ead7b155e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:54:57.218428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c3d11-75ed-461e-97bf-26ead7b155e6 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:54:57.250335 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1421/2825] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:54:59.897890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f90c1-4993-4929-9729-e120c69c76c3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:54:59.899161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f90c1-4993-4929-9729-e120c69c76c3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:54:59.899710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f90c1-4993-4929-9729-e120c69c76c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:54:59.969566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9a8f90c1-4993-4929-9729-e120c69c76c3 None None] MFA Rules not processed for user `2bba6c9b3f674959ad79d4a394ed9f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:54:59.972601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9a8f90c1-4993-4929-9729-e120c69c76c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:00.011075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3a695-62e1-4424-a4af-baf7f477d8ba None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:00.015398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3a695-62e1-4424-a4af-baf7f477d8ba None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:00.015755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3a695-62e1-4424-a4af-baf7f477d8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:00.077277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-23c3a695-62e1-4424-a4af-baf7f477d8ba None None] MFA Rules not processed for user `180d9b7141ff4792ab7ab19e21a3d7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:00.079008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-23c3a695-62e1-4424-a4af-baf7f477d8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:00.155629 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1405/2826] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:54:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:00.157923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38eb50a-f486-4048-b305-b443e8a3fa3f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:00.158563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38eb50a-f486-4048-b305-b443e8a3fa3f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:00.159217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38eb50a-f486-4048-b305-b443e8a3fa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:00.205449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f38eb50a-f486-4048-b305-b443e8a3fa3f None None] MFA Rules not processed for user `2bba6c9b3f674959ad79d4a394ed9f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:00.206810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f38eb50a-f486-4048-b305-b443e8a3fa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:00.208477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:00.223119 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1406/2827] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:00.232797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38eb50a-f486-4048-b305-b443e8a3fa3f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:00.234480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f38eb50a-f486-4048-b305-b443e8a3fa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:00.269572 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1422/2828] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:00.424698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38eb50a-f486-4048-b305-b443e8a3fa3f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:00.476917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10862bf-7b07-41b1-8926-8dc531b9a169 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:00.477741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10862bf-7b07-41b1-8926-8dc531b9a169 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:00.482820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10862bf-7b07-41b1-8926-8dc531b9a169 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:00.484315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10862bf-7b07-41b1-8926-8dc531b9a169 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:00.508719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10862bf-7b07-41b1-8926-8dc531b9a169 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:00.511873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10862bf-7b07-41b1-8926-8dc531b9a169 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:00.517124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:00.554140 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1407/2829] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:01.147456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3a695-62e1-4424-a4af-baf7f477d8ba None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:01.162767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3a695-62e1-4424-a4af-baf7f477d8ba None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:01.174124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db2489-a252-4f16-ba18-7bb8974e2272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:01.175072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db2489-a252-4f16-ba18-7bb8974e2272 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:01.175915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db2489-a252-4f16-ba18-7bb8974e2272 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:01.176563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db2489-a252-4f16-ba18-7bb8974e2272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:01.180417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d5db2489-a252-4f16-ba18-7bb8974e2272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:01.208347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db2489-a252-4f16-ba18-7bb8974e2272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:01.208347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db2489-a252-4f16-ba18-7bb8974e2272 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:01.244087 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1423/2830] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:03.500475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07096a4a-926f-417d-9c18-6b135336c6bd None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:03.500928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07096a4a-926f-417d-9c18-6b135336c6bd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:03.501055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07096a4a-926f-417d-9c18-6b135336c6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:03.610816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-07096a4a-926f-417d-9c18-6b135336c6bd None None] MFA Rules not processed for user `de4d5d0008324d7a807a8f8361a60a4e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:03.624944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-07096a4a-926f-417d-9c18-6b135336c6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:03.815085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1408/2831] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:03.823103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db2489-a252-4f16-ba18-7bb8974e2272 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:03.825049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d5db2489-a252-4f16-ba18-7bb8974e2272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:03.881274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db2489-a252-4f16-ba18-7bb8974e2272 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:03.909346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db12de-3c02-4666-b238-1a8749597bde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:03.909346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db12de-3c02-4666-b238-1a8749597bde service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:03.909346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db12de-3c02-4666-b238-1a8749597bde service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:03.909346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db12de-3c02-4666-b238-1a8749597bde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:03.931888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db12de-3c02-4666-b238-1a8749597bde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:03.938240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db12de-3c02-4666-b238-1a8749597bde service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:03.976581 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1424/2832] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:08.922489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eafc7-e51b-4893-8b52-b0e3baf3e69e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:08.923001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eafc7-e51b-4893-8b52-b0e3baf3e69e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:08.923001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eafc7-e51b-4893-8b52-b0e3baf3e69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:08.994895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-540eafc7-e51b-4893-8b52-b0e3baf3e69e None None] MFA Rules not processed for user `5d93a680570f4d23b1f6df534bc082b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:08.996414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-540eafc7-e51b-4893-8b52-b0e3baf3e69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:09.163763 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1409/2833] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:09.177134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9fb48-29d1-4670-9233-00178ad41da5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:09.177749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9fb48-29d1-4670-9233-00178ad41da5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:09.177749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9fb48-29d1-4670-9233-00178ad41da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:09.269227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-95e9fb48-29d1-4670-9233-00178ad41da5 None None] MFA Rules not processed for user `5d93a680570f4d23b1f6df534bc082b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:09.270627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-95e9fb48-29d1-4670-9233-00178ad41da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:09.297981 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1425/2834] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:09.308608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eafc7-e51b-4893-8b52-b0e3baf3e69e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:09.333008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eafc7-e51b-4893-8b52-b0e3baf3e69e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:09.345301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fba38d-f0ff-4a7f-9f8b-49742af526ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:09.346285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fba38d-f0ff-4a7f-9f8b-49742af526ab service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:09.346723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fba38d-f0ff-4a7f-9f8b-49742af526ab service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:09.347295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fba38d-f0ff-4a7f-9f8b-49742af526ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:09.372819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fba38d-f0ff-4a7f-9f8b-49742af526ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:09.374934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fba38d-f0ff-4a7f-9f8b-49742af526ab service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:09.400879 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1410/2835] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:10.682813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9fb48-29d1-4670-9233-00178ad41da5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:10.709524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9fb48-29d1-4670-9233-00178ad41da5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:10.725698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cf391-596d-4fd2-9899-efe9528d9cb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:10.726770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cf391-596d-4fd2-9899-efe9528d9cb6 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:10.727274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cf391-596d-4fd2-9899-efe9528d9cb6 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:10.727791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cf391-596d-4fd2-9899-efe9528d9cb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:10.734529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d66cf391-596d-4fd2-9899-efe9528d9cb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:10.943827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66cf391-596d-4fd2-9899-efe9528d9cb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:10.949869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66cf391-596d-4fd2-9899-efe9528d9cb6 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:11.031090 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1426/2836] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:11.207840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61e6aa-806d-4f23-bff5-f045063c19a3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:11.208388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61e6aa-806d-4f23-bff5-f045063c19a3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:11.208638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61e6aa-806d-4f23-bff5-f045063c19a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:11.252165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ff61e6aa-806d-4f23-bff5-f045063c19a3 None None] MFA Rules not processed for user `de4d5d0008324d7a807a8f8361a60a4e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:11.254684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ff61e6aa-806d-4f23-bff5-f045063c19a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:11.290306 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1411/2837] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:14.740672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f252c83-8561-4669-a226-8526a7c18afb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:14.741142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f252c83-8561-4669-a226-8526a7c18afb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:14.741142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f252c83-8561-4669-a226-8526a7c18afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:14.773500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0f252c83-8561-4669-a226-8526a7c18afb None None] MFA Rules not processed for user `ec8dcd7615cc43b3866034d5af34a9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:14.775464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0f252c83-8561-4669-a226-8526a7c18afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:14.900153 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1427/2838] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:21.249969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=74 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:24.889234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=76 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:29.490750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61e6aa-806d-4f23-bff5-f045063c19a3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:29.493486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ff61e6aa-806d-4f23-bff5-f045063c19a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:29.515484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61e6aa-806d-4f23-bff5-f045063c19a3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:29.524309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:29.525311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:29.525710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:29.526103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] PATH_INFO: `/v3/users/42766820f2a940a3bf3ff646bd358414` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:29.527620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42766820f2a940a3bf3ff646bd358414)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:29.530990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:29.600751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42766820f2a940a3bf3ff646bd358414'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:55:29.611001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42766820f2a940a3bf3ff646bd358414'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:55:29.632341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42766820f2a940a3bf3ff646bd358414'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:55:29.649722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42766820f2a940a3bf3ff646bd358414'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:55:29.651919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-773d147c-cdfe-42cc-9c5a-80ee200c7126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42766820f2a940a3bf3ff646bd358414'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:55:29.666281 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1412/2839] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:55:29 2022] DELETE /identity/v3/users/42766820f2a940a3bf3ff646bd358414 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:55:30.043210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f252c83-8561-4669-a226-8526a7c18afb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:30.063583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f252c83-8561-4669-a226-8526a7c18afb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:30.198870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:30.199945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:30.200330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:30.200761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] PATH_INFO: `/v3/projects/421588a6feed49f28f8ab75d089c01dc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:30.209393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=421588a6feed49f28f8ab75d089c01dc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:30.211828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:30.295598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '421588a6feed49f28f8ab75d089c01dc', 'request_id': 'req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:55:30.305309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '421588a6feed49f28f8ab75d089c01dc', 'request_id': 'req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:55:30.318621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '421588a6feed49f28f8ab75d089c01dc', 'request_id': 'req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:55:30.337717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] The token cache is being invalidate because project 421588a6feed49f28f8ab75d089c01dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:55:30.338263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:55:30.339220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5345c17c-4bd5-41f4-99b8-8c5e5c7889e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:55:30.342094 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1428/2840] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:30 2022] DELETE /identity/v3/projects/421588a6feed49f28f8ab75d089c01dc => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:55:30.349437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221acffc-7781-40ef-8d92-08045f6ecaf7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:30.357974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221acffc-7781-40ef-8d92-08045f6ecaf7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:30.358517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221acffc-7781-40ef-8d92-08045f6ecaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:30.384807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-221acffc-7781-40ef-8d92-08045f6ecaf7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:30.386386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-221acffc-7781-40ef-8d92-08045f6ecaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:30.459877 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1413/2841] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:30.470166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2433f10-ebca-4ab6-ae68-864ea1064cc8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:30.474492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2433f10-ebca-4ab6-ae68-864ea1064cc8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:30.474721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2433f10-ebca-4ab6-ae68-864ea1064cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:30.499248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b2433f10-ebca-4ab6-ae68-864ea1064cc8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:30.502293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b2433f10-ebca-4ab6-ae68-864ea1064cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:30.514142 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1429/2842] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:30.518081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221acffc-7781-40ef-8d92-08045f6ecaf7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:30.534074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221acffc-7781-40ef-8d92-08045f6ecaf7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:30.569871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07062a-30ee-42ad-95ed-5f86dd5bbb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:30.571976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07062a-30ee-42ad-95ed-5f86dd5bbb35 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:30.572359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07062a-30ee-42ad-95ed-5f86dd5bbb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:30.572737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07062a-30ee-42ad-95ed-5f86dd5bbb35 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:30.574656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07062a-30ee-42ad-95ed-5f86dd5bbb35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:55:30.575069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07062a-30ee-42ad-95ed-5f86dd5bbb35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:30.577151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07062a-30ee-42ad-95ed-5f86dd5bbb35 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:30.597975 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1414/2843] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:55:30.599634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2433f10-ebca-4ab6-ae68-864ea1064cc8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:30.611101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2433f10-ebca-4ab6-ae68-864ea1064cc8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:30.619005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64d71b-4da4-4292-b3a8-85116209d21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:30.619738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64d71b-4da4-4292-b3a8-85116209d21f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:30.620227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64d71b-4da4-4292-b3a8-85116209d21f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:30.620608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64d71b-4da4-4292-b3a8-85116209d21f admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:30.621376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64d71b-4da4-4292-b3a8-85116209d21f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:30.624088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64d71b-4da4-4292-b3a8-85116209d21f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:30.706529 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1430/2844] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:55:30 2022] POST /identity/v3/projects => generated 397 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:55:30.711081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07062a-30ee-42ad-95ed-5f86dd5bbb35 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:30.730809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07062a-30ee-42ad-95ed-5f86dd5bbb35 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:30.854028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375ff4a-acf2-4e69-9dcd-8868aac18c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:30.854930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375ff4a-acf2-4e69-9dcd-8868aac18c4a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:30.858299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375ff4a-acf2-4e69-9dcd-8868aac18c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:30.858578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375ff4a-acf2-4e69-9dcd-8868aac18c4a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:30.859825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375ff4a-acf2-4e69-9dcd-8868aac18c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:30.863828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375ff4a-acf2-4e69-9dcd-8868aac18c4a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:30.885356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:30.920249 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1415/2845] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:55:30 2022] POST /identity/v3/users => generated 299 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:55:30.927444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64d71b-4da4-4292-b3a8-85116209d21f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:30.964050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64d71b-4da4-4292-b3a8-85116209d21f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:30.981913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88768698-f33c-4930-8ab4-278316cfe1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:30.984314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88768698-f33c-4930-8ab4-278316cfe1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:30.984600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88768698-f33c-4930-8ab4-278316cfe1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:30.984853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88768698-f33c-4930-8ab4-278316cfe1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:30.988187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88768698-f33c-4930-8ab4-278316cfe1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:30.994321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88768698-f33c-4930-8ab4-278316cfe1d2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:31.008075 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1431/2846] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:55:31.012115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375ff4a-acf2-4e69-9dcd-8868aac18c4a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:31.023834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375ff4a-acf2-4e69-9dcd-8868aac18c4a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:31.030861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a1fb7-3d1b-4b0d-a996-2ea0fcab109b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:31.031619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a1fb7-3d1b-4b0d-a996-2ea0fcab109b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:31.031955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a1fb7-3d1b-4b0d-a996-2ea0fcab109b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:31.032509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a1fb7-3d1b-4b0d-a996-2ea0fcab109b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:31.033254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a1fb7-3d1b-4b0d-a996-2ea0fcab109b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:31.035668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a1fb7-3d1b-4b0d-a996-2ea0fcab109b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:31.050407 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1416/2847] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:55:31.055612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88768698-f33c-4930-8ab4-278316cfe1d2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:31.071387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88768698-f33c-4930-8ab4-278316cfe1d2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:31.078717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5d8fd-53dc-4155-ae72-59bb02e07d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:31.079701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5d8fd-53dc-4155-ae72-59bb02e07d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:31.080220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5d8fd-53dc-4155-ae72-59bb02e07d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:31.085291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5d8fd-53dc-4155-ae72-59bb02e07d6a admin admin] PATH_INFO: `/v3/projects/678b3cd3dda745fda1201db024f2d746/users/5f0724ab706d407fb067d4ed444e10a4/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:31.106364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5d8fd-53dc-4155-ae72-59bb02e07d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=678b3cd3dda745fda1201db024f2d746, user_id=5f0724ab706d407fb067d4ed444e10a4, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:31.112017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5d8fd-53dc-4155-ae72-59bb02e07d6a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:31.133604 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1432/2848] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:55:31 2022] PUT /identity/v3/projects/678b3cd3dda745fda1201db024f2d746/users/5f0724ab706d407fb067d4ed444e10a4/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:55:31.140238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532614a-54a6-4475-a306-a57ed8d7b054 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:31.143431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532614a-54a6-4475-a306-a57ed8d7b054 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:31.143748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532614a-54a6-4475-a306-a57ed8d7b054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:31.178302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a532614a-54a6-4475-a306-a57ed8d7b054 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:31.180084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a532614a-54a6-4475-a306-a57ed8d7b054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:31.350205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1417/2849] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:31.361827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5d8fd-53dc-4155-ae72-59bb02e07d6a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:31.364168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1ac5d8fd-53dc-4155-ae72-59bb02e07d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:31.442479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:31.488087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5d8fd-53dc-4155-ae72-59bb02e07d6a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:31.492407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba517b0-35c2-4ca3-ab88-162be436c324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:31.493234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba517b0-35c2-4ca3-ab88-162be436c324 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:31.493508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba517b0-35c2-4ca3-ab88-162be436c324 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:31.493771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba517b0-35c2-4ca3-ab88-162be436c324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:31.506652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba517b0-35c2-4ca3-ab88-162be436c324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:31.508226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba517b0-35c2-4ca3-ab88-162be436c324 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:31.520731 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1433/2850] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:32.022121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4c784-f138-4ee1-981d-57e77718d9d8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:32.023215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4c784-f138-4ee1-981d-57e77718d9d8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:32.023685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4c784-f138-4ee1-981d-57e77718d9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:32.047691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-42e4c784-f138-4ee1-981d-57e77718d9d8 None None] MFA Rules not processed for user `2bba6c9b3f674959ad79d4a394ed9f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:32.050708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-42e4c784-f138-4ee1-981d-57e77718d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:32.126404 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1418/2851] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:35.154560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9956c0f-6e4d-49f7-a8dc-8827cb537b93 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:35.156216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9956c0f-6e4d-49f7-a8dc-8827cb537b93 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:35.156572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9956c0f-6e4d-49f7-a8dc-8827cb537b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:35.195025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c9956c0f-6e4d-49f7-a8dc-8827cb537b93 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:35.196439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c9956c0f-6e4d-49f7-a8dc-8827cb537b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:35.312288 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1434/2852] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:55:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:35.337136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4c784-f138-4ee1-981d-57e77718d9d8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:35.340952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-42e4c784-f138-4ee1-981d-57e77718d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:35.510211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4c784-f138-4ee1-981d-57e77718d9d8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:35.536175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab9cb0-ada6-4fda-aeca-6884a7a8ca4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:35.538538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab9cb0-ada6-4fda-aeca-6884a7a8ca4d service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:35.538822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab9cb0-ada6-4fda-aeca-6884a7a8ca4d service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:35.539095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab9cb0-ada6-4fda-aeca-6884a7a8ca4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:35.566675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab9cb0-ada6-4fda-aeca-6884a7a8ca4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:35.568881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab9cb0-ada6-4fda-aeca-6884a7a8ca4d service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:35.592000 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1419/2853] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:55:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:55:39.641728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e673ba-ee47-44b4-97ca-cf1011cacd95 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:39.642741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e673ba-ee47-44b4-97ca-cf1011cacd95 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:39.643212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e673ba-ee47-44b4-97ca-cf1011cacd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:39.731391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-80e673ba-ee47-44b4-97ca-cf1011cacd95 None None] MFA Rules not processed for user `5f0724ab706d407fb067d4ed444e10a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:39.735506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-80e673ba-ee47-44b4-97ca-cf1011cacd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:39.975503 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1435/2854] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:39.993674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202011a-49c8-4164-92ff-a48ba350c6f7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:39.994214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202011a-49c8-4164-92ff-a48ba350c6f7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:39.994453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202011a-49c8-4164-92ff-a48ba350c6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:40.058059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5202011a-49c8-4164-92ff-a48ba350c6f7 None None] MFA Rules not processed for user `5f0724ab706d407fb067d4ed444e10a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:40.064160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5202011a-49c8-4164-92ff-a48ba350c6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:40.098694 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1420/2855] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:40.714790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e673ba-ee47-44b4-97ca-cf1011cacd95 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:40.718905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-80e673ba-ee47-44b4-97ca-cf1011cacd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:40.972456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:40.986619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e673ba-ee47-44b4-97ca-cf1011cacd95 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:40.996839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb6eb0-a114-4fca-a37c-94088033694a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:40.998800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb6eb0-a114-4fca-a37c-94088033694a service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:40.999096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb6eb0-a114-4fca-a37c-94088033694a service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:40.999414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb6eb0-a114-4fca-a37c-94088033694a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:41.011370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb6eb0-a114-4fca-a37c-94088033694a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:41.013003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb6eb0-a114-4fca-a37c-94088033694a service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:41.037869 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1436/2856] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:40 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:42.729352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202011a-49c8-4164-92ff-a48ba350c6f7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:42.731048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5202011a-49c8-4164-92ff-a48ba350c6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:42.741911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:42.882574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202011a-49c8-4164-92ff-a48ba350c6f7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:42.888970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee68e0-7570-41a5-896c-34eef12e55d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:42.889634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee68e0-7570-41a5-896c-34eef12e55d9 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:42.889914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee68e0-7570-41a5-896c-34eef12e55d9 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:42.890187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee68e0-7570-41a5-896c-34eef12e55d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:42.893011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b5ee68e0-7570-41a5-896c-34eef12e55d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:42.917730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee68e0-7570-41a5-896c-34eef12e55d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:42.921237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee68e0-7570-41a5-896c-34eef12e55d9 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:42.936583 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1421/2857] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:49.471110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb6eb0-a114-4fca-a37c-94088033694a service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:49.488368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb6eb0-a114-4fca-a37c-94088033694a service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:49.501145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f095c-ba4a-4e38-a66f-97f3c8500721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:49.502166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f095c-ba4a-4e38-a66f-97f3c8500721 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:49.502681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f095c-ba4a-4e38-a66f-97f3c8500721 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:49.503185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f095c-ba4a-4e38-a66f-97f3c8500721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:49.506472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-db4f095c-ba4a-4e38-a66f-97f3c8500721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:49.593619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f095c-ba4a-4e38-a66f-97f3c8500721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:49.596940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f095c-ba4a-4e38-a66f-97f3c8500721 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:49.610273 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1437/2858] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:50.006606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee68e0-7570-41a5-896c-34eef12e55d9 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:50.008134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b5ee68e0-7570-41a5-896c-34eef12e55d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:50.028234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee68e0-7570-41a5-896c-34eef12e55d9 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:50.050084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb866963-ec14-445f-936a-f3ea1279533e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:50.051056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb866963-ec14-445f-936a-f3ea1279533e service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:50.051336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb866963-ec14-445f-936a-f3ea1279533e service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:50.051658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb866963-ec14-445f-936a-f3ea1279533e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:50.056103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bb866963-ec14-445f-936a-f3ea1279533e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:50.078839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb866963-ec14-445f-936a-f3ea1279533e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:50.081276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb866963-ec14-445f-936a-f3ea1279533e service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:50.097472 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1422/2859] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:53.728766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f095c-ba4a-4e38-a66f-97f3c8500721 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:53.741912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f095c-ba4a-4e38-a66f-97f3c8500721 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:53.755285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45eb4e-30a5-4e19-8245-bf340ebec8f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:53.755946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45eb4e-30a5-4e19-8245-bf340ebec8f7 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:53.756217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45eb4e-30a5-4e19-8245-bf340ebec8f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:53.756485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45eb4e-30a5-4e19-8245-bf340ebec8f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:53.758506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0c45eb4e-30a5-4e19-8245-bf340ebec8f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:53.782460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45eb4e-30a5-4e19-8245-bf340ebec8f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:53.784180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45eb4e-30a5-4e19-8245-bf340ebec8f7 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:53.842844 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1438/2860] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:55:53.850013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb866963-ec14-445f-936a-f3ea1279533e service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:53.861424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb866963-ec14-445f-936a-f3ea1279533e service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:53.866272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e3b92-db8d-4e52-b1e1-c6e912718bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:53.867003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e3b92-db8d-4e52-b1e1-c6e912718bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:53.867389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e3b92-db8d-4e52-b1e1-c6e912718bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:53.867718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e3b92-db8d-4e52-b1e1-c6e912718bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:53.869738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-327e3b92-db8d-4e52-b1e1-c6e912718bf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:53.886123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e3b92-db8d-4e52-b1e1-c6e912718bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:53.888451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e3b92-db8d-4e52-b1e1-c6e912718bf9 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:53.901653 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1423/2861] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:55:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:55:56.358141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca2b86-9c34-408e-a72a-63af2b427287 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:56.358625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca2b86-9c34-408e-a72a-63af2b427287 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:56.358729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca2b86-9c34-408e-a72a-63af2b427287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:56.378947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-11ca2b86-9c34-408e-a72a-63af2b427287 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:56.380154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-11ca2b86-9c34-408e-a72a-63af2b427287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:56.387516 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1439/2862] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:56.400949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e3b92-db8d-4e52-b1e1-c6e912718bf9 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:55:56.415726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e3b92-db8d-4e52-b1e1-c6e912718bf9 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:55:56.422185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d1d0f-538c-49ea-8650-5a2e4c6924ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:55:56.423447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d1d0f-538c-49ea-8650-5a2e4c6924ee service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:56.423887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d1d0f-538c-49ea-8650-5a2e4c6924ee service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:56.424317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d1d0f-538c-49ea-8650-5a2e4c6924ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:56.433931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d1d0f-538c-49ea-8650-5a2e4c6924ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:55:56.436935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d1d0f-538c-49ea-8650-5a2e4c6924ee service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:55:56.458268 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1424/2863] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:55:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:55:57.535622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b62d8-58ba-4993-9ab3-26f231f4f765 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:57.536225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b62d8-58ba-4993-9ab3-26f231f4f765 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:57.536225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b62d8-58ba-4993-9ab3-26f231f4f765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:57.571185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e81b62d8-58ba-4993-9ab3-26f231f4f765 None None] MFA Rules not processed for user `5d93a680570f4d23b1f6df534bc082b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:57.575407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e81b62d8-58ba-4993-9ab3-26f231f4f765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:57.707177 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1440/2864] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:57.962423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c2196-09a9-4fd6-aee8-023866d2e1d1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:57.963498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c2196-09a9-4fd6-aee8-023866d2e1d1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:57.964464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c2196-09a9-4fd6-aee8-023866d2e1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:58.044429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a11c2196-09a9-4fd6-aee8-023866d2e1d1 None None] MFA Rules not processed for user `5f0724ab706d407fb067d4ed444e10a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:58.049100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a11c2196-09a9-4fd6-aee8-023866d2e1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:58.057761 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1425/2865] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:55:59.931484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fab5f-cf80-4366-a079-2128f9ed7be4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:55:59.931985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fab5f-cf80-4366-a079-2128f9ed7be4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:55:59.932112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fab5f-cf80-4366-a079-2128f9ed7be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:55:59.961518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-376fab5f-cf80-4366-a079-2128f9ed7be4 None None] MFA Rules not processed for user `5f0724ab706d407fb067d4ed444e10a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:55:59.962786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-376fab5f-cf80-4366-a079-2128f9ed7be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:55:59.971614 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1441/2866] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:55:59 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:00.397429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88bad7-94df-40dd-90fb-3c2e9c11ccc1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:00.397965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88bad7-94df-40dd-90fb-3c2e9c11ccc1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:00.397965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88bad7-94df-40dd-90fb-3c2e9c11ccc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:00.437001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9e88bad7-94df-40dd-90fb-3c2e9c11ccc1 None None] MFA Rules not processed for user `5f0724ab706d407fb067d4ed444e10a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:00.439291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e88bad7-94df-40dd-90fb-3c2e9c11ccc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:00.466162 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1426/2867] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:00 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:00.906414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb414a7e-0b5f-4d24-82ba-3fa79b3cc314 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:00.906791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb414a7e-0b5f-4d24-82ba-3fa79b3cc314 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:00.907569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb414a7e-0b5f-4d24-82ba-3fa79b3cc314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:00.947054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bb414a7e-0b5f-4d24-82ba-3fa79b3cc314 None None] MFA Rules not processed for user `5f0724ab706d407fb067d4ed444e10a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:00.949421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bb414a7e-0b5f-4d24-82ba-3fa79b3cc314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:00.969370 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1442/2868] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:00 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:01.097879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a152a2-5fd9-475a-a4b7-84f900090bea None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:01.098160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a152a2-5fd9-475a-a4b7-84f900090bea None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:01.098709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a152a2-5fd9-475a-a4b7-84f900090bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:01.123969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c0a152a2-5fd9-475a-a4b7-84f900090bea None None] MFA Rules not processed for user `5f0724ab706d407fb067d4ed444e10a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:01.125703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c0a152a2-5fd9-475a-a4b7-84f900090bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:01.139841 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1427/2869] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:01.648135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb414a7e-0b5f-4d24-82ba-3fa79b3cc314 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:01.680357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb414a7e-0b5f-4d24-82ba-3fa79b3cc314 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:01.695254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f086b-c880-4c6e-b677-6a99d57c3894 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:01.696043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f086b-c880-4c6e-b677-6a99d57c3894 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:01.696711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f086b-c880-4c6e-b677-6a99d57c3894 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:01.697336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f086b-c880-4c6e-b677-6a99d57c3894 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:01.699618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b14f086b-c880-4c6e-b677-6a99d57c3894 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:01.719551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f086b-c880-4c6e-b677-6a99d57c3894 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:01.722057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f086b-c880-4c6e-b677-6a99d57c3894 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:01.741706 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1443/2870] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:56:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:56:10.680696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c22709-14a3-4777-ac93-c0a107cb1682 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:10.681838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c22709-14a3-4777-ac93-c0a107cb1682 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:10.682377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c22709-14a3-4777-ac93-c0a107cb1682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:10.725702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-30c22709-14a3-4777-ac93-c0a107cb1682 None None] MFA Rules not processed for user `2bba6c9b3f674959ad79d4a394ed9f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:10.726990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-30c22709-14a3-4777-ac93-c0a107cb1682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:10.740915 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1428/2871] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:10.873534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96e848-33f0-4d5c-87d7-7f29a51bfb65 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:10.873534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96e848-33f0-4d5c-87d7-7f29a51bfb65 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:10.874623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96e848-33f0-4d5c-87d7-7f29a51bfb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:10.894379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c22709-14a3-4777-ac93-c0a107cb1682 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:10.895947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-30c22709-14a3-4777-ac93-c0a107cb1682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:10.908343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9b96e848-33f0-4d5c-87d7-7f29a51bfb65 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:10.912423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9b96e848-33f0-4d5c-87d7-7f29a51bfb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:10.920539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c22709-14a3-4777-ac93-c0a107cb1682 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:10.924816 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1444/2872] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:56:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:10.927586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:10.928556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:10.928834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:10.929264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] PATH_INFO: `/v3/users/ec8dcd7615cc43b3866034d5af34a9a1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:10.930825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec8dcd7615cc43b3866034d5af34a9a1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:10.933120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:10.945840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96e848-33f0-4d5c-87d7-7f29a51bfb65 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:10.971569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96e848-33f0-4d5c-87d7-7f29a51bfb65 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:10.992956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cf3cd-2af6-40ea-b33c-5dc957633a14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:10.993611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cf3cd-2af6-40ea-b33c-5dc957633a14 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:10.994154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cf3cd-2af6-40ea-b33c-5dc957633a14 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:10.994626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cf3cd-2af6-40ea-b33c-5dc957633a14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:11.017748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:11.022718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cf3cd-2af6-40ea-b33c-5dc957633a14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:11.024042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec8dcd7615cc43b3866034d5af34a9a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.024814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cf3cd-2af6-40ea-b33c-5dc957633a14 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:11.039834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec8dcd7615cc43b3866034d5af34a9a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.044349 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1445/2873] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:56:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:11.053137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:11.062622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec8dcd7615cc43b3866034d5af34a9a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.071352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec8dcd7615cc43b3866034d5af34a9a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.072211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec8dcd7615cc43b3866034d5af34a9a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.080410 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1429/2874] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:56:10 2022] DELETE /identity/v3/users/ec8dcd7615cc43b3866034d5af34a9a1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:56:11.290158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cf3cd-2af6-40ea-b33c-5dc957633a14 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:11.308979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cf3cd-2af6-40ea-b33c-5dc957633a14 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:11.410687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:11.411441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:11.411787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:11.412123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] PATH_INFO: `/v3/projects/e08b85b4f6cf47a88fe24ead8e6d45e9` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:11.414316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e08b85b4f6cf47a88fe24ead8e6d45e9)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:11.422673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:11.549836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'request_id': 'req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.564792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'request_id': 'req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.581109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'request_id': 'req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.610492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] The token cache is being invalidate because project e08b85b4f6cf47a88fe24ead8e6d45e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:56:11.611883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.614723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.621201 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1446/2875] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:56:11 2022] DELETE /identity/v3/projects/e08b85b4f6cf47a88fe24ead8e6d45e9 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:11.623328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:11.625452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:11.643643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f510af-5496-4c22-b2cc-a60e99a56af6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:11.761196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:11.762815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:11.763098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:11.763599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] PATH_INFO: `/v3/users/9749157277104da9bc3ffeef563adeb4` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:11.765216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9749157277104da9bc3ffeef563adeb4)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:11.767564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:11.851163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9749157277104da9bc3ffeef563adeb4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.870827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9749157277104da9bc3ffeef563adeb4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.893026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9749157277104da9bc3ffeef563adeb4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.910157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9749157277104da9bc3ffeef563adeb4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.913208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9749157277104da9bc3ffeef563adeb4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:11.925023 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1430/2876] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:56:11 2022] DELETE /identity/v3/users/9749157277104da9bc3ffeef563adeb4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:12.830491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:12.833139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:12.867947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee5f49-4640-44e5-b1c9-a1ab505b5e19 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:12.959059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:12.960011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:12.960491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:12.960758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] PATH_INFO: `/v3/users/5f0724ab706d407fb067d4ed444e10a4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:12.962216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0724ab706d407fb067d4ed444e10a4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:12.964771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:13.004418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:13.031029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0724ab706d407fb067d4ed444e10a4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.054418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0724ab706d407fb067d4ed444e10a4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.066195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0724ab706d407fb067d4ed444e10a4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.079472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0724ab706d407fb067d4ed444e10a4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.080586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0724ab706d407fb067d4ed444e10a4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.090167 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1447/2877] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:56:12 2022] DELETE /identity/v3/users/5f0724ab706d407fb067d4ed444e10a4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:13.299370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:13.300908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:13.325809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7334d41-2f11-4787-9cb3-e6c2bacce5d4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:13.432516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:13.434159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:13.469437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:13.470161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:13.470652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:13.471123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] PATH_INFO: `/v3/projects/971ec797f8d94d0989654cfa2d952a80` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:13.472600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=971ec797f8d94d0989654cfa2d952a80)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:13.475326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:13.479881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:13.559373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '971ec797f8d94d0989654cfa2d952a80', 'request_id': 'req-71c93a08-c04c-4a9e-b926-4b7919b64218'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.570768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '971ec797f8d94d0989654cfa2d952a80', 'request_id': 'req-71c93a08-c04c-4a9e-b926-4b7919b64218'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.579512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '971ec797f8d94d0989654cfa2d952a80', 'request_id': 'req-71c93a08-c04c-4a9e-b926-4b7919b64218'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.584193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c9c0d-314a-48a6-b354-b7090b17cf77 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:13.593415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] The token cache is being invalidate because project 971ec797f8d94d0989654cfa2d952a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:56:13.593747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.594638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-71c93a08-c04c-4a9e-b926-4b7919b64218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.596898 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1431/2878] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:56:13 2022] DELETE /identity/v3/projects/971ec797f8d94d0989654cfa2d952a80 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:13.602181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db913-2566-4d61-8d7c-55bf39a1f658 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:13.602709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db913-2566-4d61-8d7c-55bf39a1f658 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:13.603320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db913-2566-4d61-8d7c-55bf39a1f658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:13.635614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7a4db913-2566-4d61-8d7c-55bf39a1f658 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:13.640175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7a4db913-2566-4d61-8d7c-55bf39a1f658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:13.715833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:13.719192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:13.719632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:13.720028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] PATH_INFO: `/v3/projects/678b3cd3dda745fda1201db024f2d746` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:13.721358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] RBAC: Authorizing `identity:delete_project(project_id=678b3cd3dda745fda1201db024f2d746)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:13.724351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:13.764714 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1432/2879] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:13.776214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b61e41-8ba4-4eb9-82bf-c0e1c7db6732 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:13.776214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b61e41-8ba4-4eb9-82bf-c0e1c7db6732 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:13.776214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b61e41-8ba4-4eb9-82bf-c0e1c7db6732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:13.803903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f4b61e41-8ba4-4eb9-82bf-c0e1c7db6732 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:13.805350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f4b61e41-8ba4-4eb9-82bf-c0e1c7db6732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:13.823665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678b3cd3dda745fda1201db024f2d746', 'request_id': 'req-3fc83562-9026-46f7-81e8-0010aab4a50e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.828048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:13.841877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678b3cd3dda745fda1201db024f2d746', 'request_id': 'req-3fc83562-9026-46f7-81e8-0010aab4a50e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.852598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678b3cd3dda745fda1201db024f2d746', 'request_id': 'req-3fc83562-9026-46f7-81e8-0010aab4a50e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.876491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] The token cache is being invalidate because project 678b3cd3dda745fda1201db024f2d746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:56:13.877584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.878916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3fc83562-9026-46f7-81e8-0010aab4a50e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:13.883309 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1448/2880] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:56:13 2022] DELETE /identity/v3/projects/678b3cd3dda745fda1201db024f2d746 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:13.890132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aee47-0fa8-45c3-9ed4-82a353ecb330 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:13.890415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aee47-0fa8-45c3-9ed4-82a353ecb330 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:13.890688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aee47-0fa8-45c3-9ed4-82a353ecb330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:13.924515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ba5aee47-0fa8-45c3-9ed4-82a353ecb330 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:13.924854 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1433/2881] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:13.926004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ba5aee47-0fa8-45c3-9ed4-82a353ecb330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:13.929054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b61e41-8ba4-4eb9-82bf-c0e1c7db6732 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:13.936739 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1449/2882] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:56:13.942162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b61e41-8ba4-4eb9-82bf-c0e1c7db6732 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:13.945570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163ac35-2856-4dfd-85fa-0bd0ce9180c1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:13.946098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163ac35-2856-4dfd-85fa-0bd0ce9180c1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:13.946814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163ac35-2856-4dfd-85fa-0bd0ce9180c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:13.948620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5546a69-4569-4a87-bda3-f2ee69dc4c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:13.949534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546a69-4569-4a87-bda3-f2ee69dc4c40 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:13.949923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546a69-4569-4a87-bda3-f2ee69dc4c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:13.950255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546a69-4569-4a87-bda3-f2ee69dc4c40 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:13.952339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5546a69-4569-4a87-bda3-f2ee69dc4c40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:56:13.952717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5546a69-4569-4a87-bda3-f2ee69dc4c40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:13.956105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5546a69-4569-4a87-bda3-f2ee69dc4c40 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:13.974269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3163ac35-2856-4dfd-85fa-0bd0ce9180c1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:13.975980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3163ac35-2856-4dfd-85fa-0bd0ce9180c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:13.991587 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1434/2883] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:56:13.995486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5546a69-4569-4a87-bda3-f2ee69dc4c40 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.003601 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1450/2884] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:56:14.018951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163ac35-2856-4dfd-85fa-0bd0ce9180c1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.036728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5546a69-4569-4a87-bda3-f2ee69dc4c40 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.039860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163ac35-2856-4dfd-85fa-0bd0ce9180c1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.049249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0309bea-6155-4ad1-9b47-a4a642214238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.049946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a16456-5820-4c16-88e1-0392edf86d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.050677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309bea-6155-4ad1-9b47-a4a642214238 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.050957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a16456-5820-4c16-88e1-0392edf86d5d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.051235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a16456-5820-4c16-88e1-0392edf86d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.051644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a16456-5820-4c16-88e1-0392edf86d5d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.051907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309bea-6155-4ad1-9b47-a4a642214238 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.052274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a16456-5820-4c16-88e1-0392edf86d5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.052636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309bea-6155-4ad1-9b47-a4a642214238 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.053542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0309bea-6155-4ad1-9b47-a4a642214238 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:56:14.054083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0309bea-6155-4ad1-9b47-a4a642214238 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.054769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a16456-5820-4c16-88e1-0392edf86d5d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.056301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0309bea-6155-4ad1-9b47-a4a642214238 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.076038 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1451/2885] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:56:14.077682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0309bea-6155-4ad1-9b47-a4a642214238 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.093922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0309bea-6155-4ad1-9b47-a4a642214238 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.103771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178406d-2840-47fb-afeb-b60923ffc1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.104316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:14.105621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178406d-2840-47fb-afeb-b60923ffc1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.106159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178406d-2840-47fb-afeb-b60923ffc1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.107695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178406d-2840-47fb-afeb-b60923ffc1f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.108786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178406d-2840-47fb-afeb-b60923ffc1f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.111641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178406d-2840-47fb-afeb-b60923ffc1f6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.125098 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1435/2886] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:56:13 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:56:14.126928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a16456-5820-4c16-88e1-0392edf86d5d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.150237 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1452/2887] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:56:14 2022] POST /identity/v3/projects => generated 367 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:56:14.155085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178406d-2840-47fb-afeb-b60923ffc1f6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.233375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a16456-5820-4c16-88e1-0392edf86d5d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.269288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178406d-2840-47fb-afeb-b60923ffc1f6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.280283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97353d38-8324-49e5-9c9a-3c02d23a89df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.281365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97353d38-8324-49e5-9c9a-3c02d23a89df admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.281365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97353d38-8324-49e5-9c9a-3c02d23a89df admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.281365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97353d38-8324-49e5-9c9a-3c02d23a89df admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.281859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97353d38-8324-49e5-9c9a-3c02d23a89df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.284341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97353d38-8324-49e5-9c9a-3c02d23a89df admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.327227 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1453/2888] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:56:14 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:56:14.331316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97353d38-8324-49e5-9c9a-3c02d23a89df admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.355737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97353d38-8324-49e5-9c9a-3c02d23a89df admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.368479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8181f59-0fe1-4bdc-b646-a8b79858778c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.369627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8181f59-0fe1-4bdc-b646-a8b79858778c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.369915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8181f59-0fe1-4bdc-b646-a8b79858778c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.370180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8181f59-0fe1-4bdc-b646-a8b79858778c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.371808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8181f59-0fe1-4bdc-b646-a8b79858778c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.373229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8181f59-0fe1-4bdc-b646-a8b79858778c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.391362 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1454/2889] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:56:14.392805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8181f59-0fe1-4bdc-b646-a8b79858778c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.412718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57603c-f889-4947-bc3e-7186b49239d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.415990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57603c-f889-4947-bc3e-7186b49239d2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.416729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57603c-f889-4947-bc3e-7186b49239d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.417784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57603c-f889-4947-bc3e-7186b49239d2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.418890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57603c-f889-4947-bc3e-7186b49239d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.425548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57603c-f889-4947-bc3e-7186b49239d2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.429188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8181f59-0fe1-4bdc-b646-a8b79858778c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.438021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeddfe-090a-44e8-a205-c1019da486b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.438740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeddfe-090a-44e8-a205-c1019da486b9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.439897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeddfe-090a-44e8-a205-c1019da486b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.440458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeddfe-090a-44e8-a205-c1019da486b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.442410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeddfe-090a-44e8-a205-c1019da486b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.444722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeddfe-090a-44e8-a205-c1019da486b9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.468207 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1455/2890] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:56:14.469416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeddfe-090a-44e8-a205-c1019da486b9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.485861 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1436/2891] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:56:14 2022] POST /identity/v3/users => generated 294 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:56:14.487125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeddfe-090a-44e8-a205-c1019da486b9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.490474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57603c-f889-4947-bc3e-7186b49239d2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.494747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4934b-1b93-4085-b69b-dc8a5b1a6419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.496001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4934b-1b93-4085-b69b-dc8a5b1a6419 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.496412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4934b-1b93-4085-b69b-dc8a5b1a6419 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.496772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4934b-1b93-4085-b69b-dc8a5b1a6419 admin admin] PATH_INFO: `/v3/projects/44a01230c0fc49a6b60d553d085f3808/users/6adaa12c2ca9469299994300bc856ac5/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.514968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57603c-f889-4947-bc3e-7186b49239d2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.521489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4934b-1b93-4085-b69b-dc8a5b1a6419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a01230c0fc49a6b60d553d085f3808, user_id=6adaa12c2ca9469299994300bc856ac5, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.526308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4934b-1b93-4085-b69b-dc8a5b1a6419 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.532793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bc85f-253b-41cf-b76c-bfa39a16a526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.533827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bc85f-253b-41cf-b76c-bfa39a16a526 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.534142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bc85f-253b-41cf-b76c-bfa39a16a526 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.534362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bc85f-253b-41cf-b76c-bfa39a16a526 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.535018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bc85f-253b-41cf-b76c-bfa39a16a526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.536971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bc85f-253b-41cf-b76c-bfa39a16a526 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.551575 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1456/2892] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:56:14 2022] PUT /identity/v3/projects/44a01230c0fc49a6b60d553d085f3808/users/6adaa12c2ca9469299994300bc856ac5/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:56:14.557030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de878c-c9e5-46ba-8f6b-82cb0ef15725 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.557784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de878c-c9e5-46ba-8f6b-82cb0ef15725 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.558317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de878c-c9e5-46ba-8f6b-82cb0ef15725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.563883 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1437/2893] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:56:14.570334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bc85f-253b-41cf-b76c-bfa39a16a526 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.587916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bc85f-253b-41cf-b76c-bfa39a16a526 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.591731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-99de878c-c9e5-46ba-8f6b-82cb0ef15725 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:14.593349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-99de878c-c9e5-46ba-8f6b-82cb0ef15725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:14.637320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:14.715731 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1457/2894] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:56:14.725719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084fba2-c042-478c-957a-f138f5728cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.726346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084fba2-c042-478c-957a-f138f5728cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.726619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084fba2-c042-478c-957a-f138f5728cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.726883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084fba2-c042-478c-957a-f138f5728cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.727525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084fba2-c042-478c-957a-f138f5728cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.729983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084fba2-c042-478c-957a-f138f5728cf0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.751548 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1438/2895] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:56:14.752844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de878c-c9e5-46ba-8f6b-82cb0ef15725 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.762592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084fba2-c042-478c-957a-f138f5728cf0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:14.766222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6084fba2-c042-478c-957a-f138f5728cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:14.767990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de878c-c9e5-46ba-8f6b-82cb0ef15725 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.781144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4ad0d-a83f-4877-b595-dd2ce30e0f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:14.781881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ad0d-a83f-4877-b595-dd2ce30e0f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.782182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ad0d-a83f-4877-b595-dd2ce30e0f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.782448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ad0d-a83f-4877-b595-dd2ce30e0f7b admin admin] PATH_INFO: `/v3/projects/2e3da14141a04d7881dc4253f2ac75af/users/95c5e013a10d4693947db0b06a2f42dd/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.815718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:14.829716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4ad0d-a83f-4877-b595-dd2ce30e0f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3da14141a04d7881dc4253f2ac75af, user_id=95c5e013a10d4693947db0b06a2f42dd, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:14.832517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4ad0d-a83f-4877-b595-dd2ce30e0f7b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:14.867099 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1458/2896] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:56:14 2022] PUT /identity/v3/projects/2e3da14141a04d7881dc4253f2ac75af/users/95c5e013a10d4693947db0b06a2f42dd/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:14.870297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c27a1-306d-4fa8-80f8-fa976fd9f76f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:14.870457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c27a1-306d-4fa8-80f8-fa976fd9f76f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:14.872115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c27a1-306d-4fa8-80f8-fa976fd9f76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:14.892464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084fba2-c042-478c-957a-f138f5728cf0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:14.934172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7a5c27a1-306d-4fa8-80f8-fa976fd9f76f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:14.938134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7a5c27a1-306d-4fa8-80f8-fa976fd9f76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:15.012544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443d4dc-7089-4dcc-9dfd-e4f35a975328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:15.017165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443d4dc-7089-4dcc-9dfd-e4f35a975328 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:15.018453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443d4dc-7089-4dcc-9dfd-e4f35a975328 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:15.018794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443d4dc-7089-4dcc-9dfd-e4f35a975328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:15.084341 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1459/2897] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:15.108844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c27a1-306d-4fa8-80f8-fa976fd9f76f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:15.123162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c27a1-306d-4fa8-80f8-fa976fd9f76f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:15.127702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f169a-a43a-442c-94c7-7636e3798081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:15.128469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f169a-a43a-442c-94c7-7636e3798081 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:15.128765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f169a-a43a-442c-94c7-7636e3798081 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:15.129067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f169a-a43a-442c-94c7-7636e3798081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:15.132642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443d4dc-7089-4dcc-9dfd-e4f35a975328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:15.135634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443d4dc-7089-4dcc-9dfd-e4f35a975328 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:15.141474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f169a-a43a-442c-94c7-7636e3798081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:15.142930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f169a-a43a-442c-94c7-7636e3798081 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:15.152530 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1439/2898] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:56:15.159765 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1460/2899] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:56:17.135523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75193af-7a57-4644-be54-c05065b88548 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:17.136099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75193af-7a57-4644-be54-c05065b88548 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:17.136221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75193af-7a57-4644-be54-c05065b88548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:17.166153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d75193af-7a57-4644-be54-c05065b88548 None None] MFA Rules not processed for user `2bba6c9b3f674959ad79d4a394ed9f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:17.167846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d75193af-7a57-4644-be54-c05065b88548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:17.247291 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1440/2900] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:25.150935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=54,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:26.092710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2223de-1d27-46dd-bf1c-73283feab704 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:26.092889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2223de-1d27-46dd-bf1c-73283feab704 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:26.093193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2223de-1d27-46dd-bf1c-73283feab704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:26.134239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6e2223de-1d27-46dd-bf1c-73283feab704 None None] MFA Rules not processed for user `6adaa12c2ca9469299994300bc856ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:26.135720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e2223de-1d27-46dd-bf1c-73283feab704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:26.277580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869447ce-3302-4b4f-97d6-362535264273 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:26.278088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869447ce-3302-4b4f-97d6-362535264273 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:26.278194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869447ce-3302-4b4f-97d6-362535264273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:26.285374 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1461/2901] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:26.314260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-869447ce-3302-4b4f-97d6-362535264273 None None] MFA Rules not processed for user `6adaa12c2ca9469299994300bc856ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:26.315520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-869447ce-3302-4b4f-97d6-362535264273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:26.333207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1441/2902] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:26.831926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2223de-1d27-46dd-bf1c-73283feab704 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:26.834366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e2223de-1d27-46dd-bf1c-73283feab704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:26.987930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2223de-1d27-46dd-bf1c-73283feab704 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:26.998947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415d4a6-c93e-4ec7-9357-319e28da1e6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:26.999936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415d4a6-c93e-4ec7-9357-319e28da1e6c service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:27.000770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415d4a6-c93e-4ec7-9357-319e28da1e6c service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:27.001278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415d4a6-c93e-4ec7-9357-319e28da1e6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:27.015074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415d4a6-c93e-4ec7-9357-319e28da1e6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:27.018907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415d4a6-c93e-4ec7-9357-319e28da1e6c service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:27.031704 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1462/2903] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:56:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:56:28.955001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd49c7-38ee-419f-ad9d-9fac10c6185c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:28.955755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd49c7-38ee-419f-ad9d-9fac10c6185c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:28.956185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd49c7-38ee-419f-ad9d-9fac10c6185c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:29.027059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-90fd49c7-38ee-419f-ad9d-9fac10c6185c None None] MFA Rules not processed for user `95c5e013a10d4693947db0b06a2f42dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:29.039888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-90fd49c7-38ee-419f-ad9d-9fac10c6185c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:29.076816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:29.207562 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1442/2904] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:29.458421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415d4a6-c93e-4ec7-9357-319e28da1e6c service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:29.460013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d415d4a6-c93e-4ec7-9357-319e28da1e6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:29.612647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:29.617790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415d4a6-c93e-4ec7-9357-319e28da1e6c service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:29.628123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf160b-a168-4369-a0ce-848a054bad27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:29.628748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf160b-a168-4369-a0ce-848a054bad27 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:29.629020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf160b-a168-4369-a0ce-848a054bad27 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:29.629347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf160b-a168-4369-a0ce-848a054bad27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:29.634270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-05bf160b-a168-4369-a0ce-848a054bad27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:29.736890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf160b-a168-4369-a0ce-848a054bad27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:29.738416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf160b-a168-4369-a0ce-848a054bad27 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:29.769334 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1463/2905] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:56:29 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:56:30.641099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd49c7-38ee-419f-ad9d-9fac10c6185c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:30.667508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd49c7-38ee-419f-ad9d-9fac10c6185c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:30.675673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b8b9f-f69f-4dcf-9da7-f90c7540e2c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:30.676619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b8b9f-f69f-4dcf-9da7-f90c7540e2c9 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:30.677308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b8b9f-f69f-4dcf-9da7-f90c7540e2c9 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:30.677775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b8b9f-f69f-4dcf-9da7-f90c7540e2c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:30.694741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b8b9f-f69f-4dcf-9da7-f90c7540e2c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:30.696937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b8b9f-f69f-4dcf-9da7-f90c7540e2c9 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:30.735392 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1443/2906] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:56:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:56:33.838092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf160b-a168-4369-a0ce-848a054bad27 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:33.840225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-05bf160b-a168-4369-a0ce-848a054bad27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:33.858811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf160b-a168-4369-a0ce-848a054bad27 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:33.864074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:33.864681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:33.864955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:33.865382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] PATH_INFO: `/v3/users/999bc03f52824573bb3c8bdc6a1179cf` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:33.866733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=999bc03f52824573bb3c8bdc6a1179cf)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:33.870624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:33.929326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999bc03f52824573bb3c8bdc6a1179cf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:33.948919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999bc03f52824573bb3c8bdc6a1179cf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:33.958207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999bc03f52824573bb3c8bdc6a1179cf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:33.965482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999bc03f52824573bb3c8bdc6a1179cf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:33.966140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999bc03f52824573bb3c8bdc6a1179cf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:33.972781 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1464/2907] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:56:33 2022] DELETE /identity/v3/users/999bc03f52824573bb3c8bdc6a1179cf => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:34.184726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b8b9f-f69f-4dcf-9da7-f90c7540e2c9 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:34.199217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b8b9f-f69f-4dcf-9da7-f90c7540e2c9 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:34.305952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:34.307016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:34.307460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:34.307756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] PATH_INFO: `/v3/projects/e67382792a7d4ccf8100febea9052e1b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:34.309095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e67382792a7d4ccf8100febea9052e1b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:34.311161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:34.581410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e67382792a7d4ccf8100febea9052e1b', 'request_id': 'req-bd324d58-2149-4965-8fde-c1bada75ff12'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:34.601889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e67382792a7d4ccf8100febea9052e1b', 'request_id': 'req-bd324d58-2149-4965-8fde-c1bada75ff12'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:34.611124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e67382792a7d4ccf8100febea9052e1b', 'request_id': 'req-bd324d58-2149-4965-8fde-c1bada75ff12'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:34.624380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] The token cache is being invalidate because project e67382792a7d4ccf8100febea9052e1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:56:34.624735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:34.625713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:34.627360 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1444/2908] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:56:34 2022] DELETE /identity/v3/projects/e67382792a7d4ccf8100febea9052e1b => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:34.631370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:34.634604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:34.650708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671d390-0e39-47c4-a93f-8fecedd86483 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:34.729325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:34.730043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:34.730392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:34.730668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] PATH_INFO: `/v3/users/6435555fee024871b05e04d4a0a7cd4f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:34.732746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6435555fee024871b05e04d4a0a7cd4f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:34.734762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:34.754009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:34.797389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6435555fee024871b05e04d4a0a7cd4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:34.812315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6435555fee024871b05e04d4a0a7cd4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:34.826293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6435555fee024871b05e04d4a0a7cd4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:34.834640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6435555fee024871b05e04d4a0a7cd4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:34.835396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b260cb39-a04a-4f38-9a99-233a9f198384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6435555fee024871b05e04d4a0a7cd4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:34.842386 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1465/2909] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:56:34 2022] DELETE /identity/v3/users/6435555fee024871b05e04d4a0a7cd4f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:35.049079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:35.050684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:35.072221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd324d58-2149-4965-8fde-c1bada75ff12 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:35.107834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:35.138914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:35.139571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.139841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.140106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] PATH_INFO: `/v3/projects/7544347d450845ef8157b63b71c83959` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.141262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7544347d450845ef8157b63b71c83959)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:35.143139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:35.199455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7544347d450845ef8157b63b71c83959', 'request_id': 'req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:35.211007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7544347d450845ef8157b63b71c83959', 'request_id': 'req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:35.218869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7544347d450845ef8157b63b71c83959', 'request_id': 'req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:35.230129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] The token cache is being invalidate because project 7544347d450845ef8157b63b71c83959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:56:35.230421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:35.231065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fbacc8c-7c8f-435a-98ae-056bd57a4bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:56:35.233201 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1445/2910] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:56:35 2022] DELETE /identity/v3/projects/7544347d450845ef8157b63b71c83959 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:35.238548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3a34e-bee5-47b7-99e0-a2d5189363dc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.238847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3a34e-bee5-47b7-99e0-a2d5189363dc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.239104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3a34e-bee5-47b7-99e0-a2d5189363dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.282437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-38a3a34e-bee5-47b7-99e0-a2d5189363dc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:35.283969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-38a3a34e-bee5-47b7-99e0-a2d5189363dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:35.351365 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1466/2911] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:56:35.360985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e233-dde8-494d-9495-c85877327a9e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.361572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e233-dde8-494d-9495-c85877327a9e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.362151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e233-dde8-494d-9495-c85877327a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.384380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3f73e233-dde8-494d-9495-c85877327a9e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:35.385703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3f73e233-dde8-494d-9495-c85877327a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:35.401972 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1446/2912] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:35.403317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3a34e-bee5-47b7-99e0-a2d5189363dc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:35.414613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3a34e-bee5-47b7-99e0-a2d5189363dc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:35.420308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95691975-fe74-48d5-8713-fb62b37ab3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:35.420914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95691975-fe74-48d5-8713-fb62b37ab3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.421403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95691975-fe74-48d5-8713-fb62b37ab3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.422083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95691975-fe74-48d5-8713-fb62b37ab3e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.422416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95691975-fe74-48d5-8713-fb62b37ab3e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:56:35.422599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95691975-fe74-48d5-8713-fb62b37ab3e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:35.424214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95691975-fe74-48d5-8713-fb62b37ab3e9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:35.442770 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1467/2913] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:56:35.444467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73e233-dde8-494d-9495-c85877327a9e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:35.454633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73e233-dde8-494d-9495-c85877327a9e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:35.459361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb8747-7c7b-4877-9255-be9a9aca9593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:35.459950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb8747-7c7b-4877-9255-be9a9aca9593 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.460221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb8747-7c7b-4877-9255-be9a9aca9593 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.460483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb8747-7c7b-4877-9255-be9a9aca9593 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.461152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb8747-7c7b-4877-9255-be9a9aca9593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:35.463259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb8747-7c7b-4877-9255-be9a9aca9593 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:35.492035 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1447/2914] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:56:35 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:56:35.496483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95691975-fe74-48d5-8713-fb62b37ab3e9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:35.506600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95691975-fe74-48d5-8713-fb62b37ab3e9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:35.572634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491dc68-1855-4090-a36f-1d270d0bf75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:35.573266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491dc68-1855-4090-a36f-1d270d0bf75a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.573544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491dc68-1855-4090-a36f-1d270d0bf75a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.573815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491dc68-1855-4090-a36f-1d270d0bf75a admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.574429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491dc68-1855-4090-a36f-1d270d0bf75a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:35.577052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491dc68-1855-4090-a36f-1d270d0bf75a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:35.590124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:35.621072 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1468/2915] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:56:35 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:56:35.621453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb8747-7c7b-4877-9255-be9a9aca9593 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:35.621453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb8747-7c7b-4877-9255-be9a9aca9593 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:35.626813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd8ff4-d7e9-4a85-8d4f-c28709b730cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:35.627563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd8ff4-d7e9-4a85-8d4f-c28709b730cf admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.627903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd8ff4-d7e9-4a85-8d4f-c28709b730cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.628233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd8ff4-d7e9-4a85-8d4f-c28709b730cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.628966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd8ff4-d7e9-4a85-8d4f-c28709b730cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:35.630708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd8ff4-d7e9-4a85-8d4f-c28709b730cf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:35.643335 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1448/2916] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:56:35.672991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491dc68-1855-4090-a36f-1d270d0bf75a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:35.683774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491dc68-1855-4090-a36f-1d270d0bf75a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:35.690659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d10cf-daac-466e-8f21-459386989326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:35.691369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d10cf-daac-466e-8f21-459386989326 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.691615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d10cf-daac-466e-8f21-459386989326 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.691888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d10cf-daac-466e-8f21-459386989326 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.692581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d10cf-daac-466e-8f21-459386989326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:35.694238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d10cf-daac-466e-8f21-459386989326 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:35.710612 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1469/2917] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:56:35.714946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd8ff4-d7e9-4a85-8d4f-c28709b730cf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:35.725164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd8ff4-d7e9-4a85-8d4f-c28709b730cf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:35.730570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc06ec-73b4-4b50-b74a-4b97ee8fedf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:35.731026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc06ec-73b4-4b50-b74a-4b97ee8fedf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.731299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc06ec-73b4-4b50-b74a-4b97ee8fedf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.731567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc06ec-73b4-4b50-b74a-4b97ee8fedf8 admin admin] PATH_INFO: `/v3/projects/77dd9d0796b845a4b66069ed09b88f12/users/1fe50f300f3d4fa295033618d73faa7e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.747258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc06ec-73b4-4b50-b74a-4b97ee8fedf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77dd9d0796b845a4b66069ed09b88f12, user_id=1fe50f300f3d4fa295033618d73faa7e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:35.750517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc06ec-73b4-4b50-b74a-4b97ee8fedf8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:35.765283 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1449/2918] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:56:35 2022] PUT /identity/v3/projects/77dd9d0796b845a4b66069ed09b88f12/users/1fe50f300f3d4fa295033618d73faa7e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:56:35.770911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779489b7-f622-4f1a-9c74-c73cc0fb8abd None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.771289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779489b7-f622-4f1a-9c74-c73cc0fb8abd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.771658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779489b7-f622-4f1a-9c74-c73cc0fb8abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.793069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-779489b7-f622-4f1a-9c74-c73cc0fb8abd None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:35.796270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-779489b7-f622-4f1a-9c74-c73cc0fb8abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:35.855995 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1470/2919] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:35.866744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc06ec-73b4-4b50-b74a-4b97ee8fedf8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:35.868725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7bdc06ec-73b4-4b50-b74a-4b97ee8fedf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:35.928188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc06ec-73b4-4b50-b74a-4b97ee8fedf8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:35.931958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce383cc4-3159-4831-bf7a-d7b0581e1fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:35.932565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce383cc4-3159-4831-bf7a-d7b0581e1fcb service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:35.932830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce383cc4-3159-4831-bf7a-d7b0581e1fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:35.933190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce383cc4-3159-4831-bf7a-d7b0581e1fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:35.943252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce383cc4-3159-4831-bf7a-d7b0581e1fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:35.945002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce383cc4-3159-4831-bf7a-d7b0581e1fcb service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:35.951222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:35.959353 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1450/2920] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:56:41.694941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e23db-6f16-4714-b478-72c501c69faa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:41.696101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e23db-6f16-4714-b478-72c501c69faa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:41.696506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e23db-6f16-4714-b478-72c501c69faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:41.726254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ba4e23db-6f16-4714-b478-72c501c69faa None None] MFA Rules not processed for user `1fe50f300f3d4fa295033618d73faa7e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:41.727488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ba4e23db-6f16-4714-b478-72c501c69faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:41.825316 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1471/2921] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:43.820464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce383cc4-3159-4831-bf7a-d7b0581e1fcb service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:43.822662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ce383cc4-3159-4831-bf7a-d7b0581e1fcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:43.912527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce383cc4-3159-4831-bf7a-d7b0581e1fcb service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:43.920747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ab6bd-468b-42f0-95cb-70700abf9c00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:43.921374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ab6bd-468b-42f0-95cb-70700abf9c00 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:43.921675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ab6bd-468b-42f0-95cb-70700abf9c00 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:43.921919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ab6bd-468b-42f0-95cb-70700abf9c00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:43.930647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ab6bd-468b-42f0-95cb-70700abf9c00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:43.932229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ab6bd-468b-42f0-95cb-70700abf9c00 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:43.942952 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1451/2922] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:56:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:56:49.793086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a644e-8e0a-461c-897c-74015d7080d0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:49.793593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a644e-8e0a-461c-897c-74015d7080d0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:49.793593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a644e-8e0a-461c-897c-74015d7080d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:49.813637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3c6a644e-8e0a-461c-897c-74015d7080d0 None None] MFA Rules not processed for user `6adaa12c2ca9469299994300bc856ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:49.814825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3c6a644e-8e0a-461c-897c-74015d7080d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:49.876617 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1472/2923] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:50.077480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acdfdd-dec0-4bf9-b9bc-6a0853e6d0b5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:50.078196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acdfdd-dec0-4bf9-b9bc-6a0853e6d0b5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:50.078196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acdfdd-dec0-4bf9-b9bc-6a0853e6d0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:50.110362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-32acdfdd-dec0-4bf9-b9bc-6a0853e6d0b5 None None] MFA Rules not processed for user `2bba6c9b3f674959ad79d4a394ed9f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:50.112421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-32acdfdd-dec0-4bf9-b9bc-6a0853e6d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:50.172771 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1452/2924] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:56:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:56:50.349801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a644e-8e0a-461c-897c-74015d7080d0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:50.351236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3c6a644e-8e0a-461c-897c-74015d7080d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:50.363955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a644e-8e0a-461c-897c-74015d7080d0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:50.367609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d19dc-e7c7-46d3-b8b9-019065fc66bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:50.368200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d19dc-e7c7-46d3-b8b9-019065fc66bb service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:50.368468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d19dc-e7c7-46d3-b8b9-019065fc66bb service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:50.368731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d19dc-e7c7-46d3-b8b9-019065fc66bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:50.371239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1f8d19dc-e7c7-46d3-b8b9-019065fc66bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:50.384763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d19dc-e7c7-46d3-b8b9-019065fc66bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:50.386245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d19dc-e7c7-46d3-b8b9-019065fc66bb service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:50.396834 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1473/2925] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:56:56.954378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acdfdd-dec0-4bf9-b9bc-6a0853e6d0b5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:56.955852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-32acdfdd-dec0-4bf9-b9bc-6a0853e6d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:57.030622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acdfdd-dec0-4bf9-b9bc-6a0853e6d0b5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:57.036049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd93d-cfa6-439b-8c62-1e9ee9946285 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:57.036648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd93d-cfa6-439b-8c62-1e9ee9946285 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:57.036915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd93d-cfa6-439b-8c62-1e9ee9946285 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:57.037205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd93d-cfa6-439b-8c62-1e9ee9946285 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:57.038974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bd6bd93d-cfa6-439b-8c62-1e9ee9946285 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:57.051573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bd93d-cfa6-439b-8c62-1e9ee9946285 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:57.053169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bd93d-cfa6-439b-8c62-1e9ee9946285 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:57.076471 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1453/2926] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:56:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:56:58.048185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0070d-9987-4d47-9440-944d5a41b60d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:58.048708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0070d-9987-4d47-9440-944d5a41b60d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:58.048863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0070d-9987-4d47-9440-944d5a41b60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:58.066483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:58.073583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f4f0070d-9987-4d47-9440-944d5a41b60d None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:56:58.075208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f4f0070d-9987-4d47-9440-944d5a41b60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:58.155857 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1474/2927] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:56:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:56:58.207825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd93d-cfa6-439b-8c62-1e9ee9946285 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:56:58.211412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bd6bd93d-cfa6-439b-8c62-1e9ee9946285 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:56:58.238014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:58.312933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd93d-cfa6-439b-8c62-1e9ee9946285 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:56:58.318632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16c338-79b1-43fa-9631-d4bb3e3359b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:56:58.319394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16c338-79b1-43fa-9631-d4bb3e3359b5 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:56:58.319843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16c338-79b1-43fa-9631-d4bb3e3359b5 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:56:58.320876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16c338-79b1-43fa-9631-d4bb3e3359b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:56:58.332508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16c338-79b1-43fa-9631-d4bb3e3359b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:56:58.335337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16c338-79b1-43fa-9631-d4bb3e3359b5 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:56:58.346450 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1454/2928] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:56:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:57:01.554186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0070d-9987-4d47-9440-944d5a41b60d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:01.571236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0070d-9987-4d47-9440-944d5a41b60d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:01.577539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063289ae-676f-400f-9bc0-9b3ca0b533d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:01.578751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063289ae-676f-400f-9bc0-9b3ca0b533d0 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:01.579237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063289ae-676f-400f-9bc0-9b3ca0b533d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:01.580365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063289ae-676f-400f-9bc0-9b3ca0b533d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:01.582649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-063289ae-676f-400f-9bc0-9b3ca0b533d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:01.598857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063289ae-676f-400f-9bc0-9b3ca0b533d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:01.600677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063289ae-676f-400f-9bc0-9b3ca0b533d0 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:01.635857 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1475/2929] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:03.083837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16c338-79b1-43fa-9631-d4bb3e3359b5 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:03.085985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4e16c338-79b1-43fa-9631-d4bb3e3359b5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:03.106326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16c338-79b1-43fa-9631-d4bb3e3359b5 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:03.111394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:03.112326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:03.112751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:03.113236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] PATH_INFO: `/v3/users/c613201f55b94a528c6c36e5b5772799` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:03.114500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c613201f55b94a528c6c36e5b5772799)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:03.116849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:03.176389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c613201f55b94a528c6c36e5b5772799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:03.186125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c613201f55b94a528c6c36e5b5772799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:03.202476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c613201f55b94a528c6c36e5b5772799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:03.212128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c613201f55b94a528c6c36e5b5772799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:03.212914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c613201f55b94a528c6c36e5b5772799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:03.220827 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1455/2930] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:03 2022] DELETE /identity/v3/users/c613201f55b94a528c6c36e5b5772799 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:03.451423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063289ae-676f-400f-9bc0-9b3ca0b533d0 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:03.470138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063289ae-676f-400f-9bc0-9b3ca0b533d0 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:03.598426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:03.599219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:03.600724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:03.601175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] PATH_INFO: `/v3/projects/fe14ede74b2140269cd7dfc32fe14f9c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:03.602607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe14ede74b2140269cd7dfc32fe14f9c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:03.604798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:03.686430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe14ede74b2140269cd7dfc32fe14f9c', 'request_id': 'req-10d0fcba-0c28-477a-94ea-b69e507ddef2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:03.706315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe14ede74b2140269cd7dfc32fe14f9c', 'request_id': 'req-10d0fcba-0c28-477a-94ea-b69e507ddef2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:03.721419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe14ede74b2140269cd7dfc32fe14f9c', 'request_id': 'req-10d0fcba-0c28-477a-94ea-b69e507ddef2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:03.750010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] The token cache is being invalidate because project fe14ede74b2140269cd7dfc32fe14f9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:03.750380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:03.753901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:03.755864 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1476/2931] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:03 2022] DELETE /identity/v3/projects/fe14ede74b2140269cd7dfc32fe14f9c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:03.760420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:03.764304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:03.789303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79a18f-581d-47a2-a1d1-ae86f8aba3e6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:03.914628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:03.916209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:03.916590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:03.917247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] PATH_INFO: `/v3/users/92d7ca8b7e1a4ca6acc797364290c607` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:03.918579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=92d7ca8b7e1a4ca6acc797364290c607)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:03.921332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:04.001026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:04.097901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92d7ca8b7e1a4ca6acc797364290c607'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:04.161577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92d7ca8b7e1a4ca6acc797364290c607'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:04.256832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92d7ca8b7e1a4ca6acc797364290c607'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:04.281128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92d7ca8b7e1a4ca6acc797364290c607'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:04.282207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4acf1ad7-a247-4269-a2b2-0bbf71366f4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92d7ca8b7e1a4ca6acc797364290c607'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:04.300517 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1456/2932] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:03 2022] DELETE /identity/v3/users/92d7ca8b7e1a4ca6acc797364290c607 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:05.499874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:05.501673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:05.549351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0fcba-0c28-477a-94ea-b69e507ddef2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:05.617153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:05.690639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:05.691276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:05.691547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:05.691960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] PATH_INFO: `/v3/projects/1e95d86475074ada9ee35335e52b62dd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:05.693401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e95d86475074ada9ee35335e52b62dd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:05.695502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:05.807901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e95d86475074ada9ee35335e52b62dd', 'request_id': 'req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:05.829724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e95d86475074ada9ee35335e52b62dd', 'request_id': 'req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:05.845261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e95d86475074ada9ee35335e52b62dd', 'request_id': 'req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:05.879590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] The token cache is being invalidate because project 1e95d86475074ada9ee35335e52b62dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:05.880498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:05.881542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949df2d9-0bc3-4bd1-83c7-fe0c98bf251a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:05.887489 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1477/2933] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:05 2022] DELETE /identity/v3/projects/1e95d86475074ada9ee35335e52b62dd => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:05.889587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5a17e-df0f-4afd-8d6c-4290c8ef6896 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:05.890047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5a17e-df0f-4afd-8d6c-4290c8ef6896 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:05.890139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5a17e-df0f-4afd-8d6c-4290c8ef6896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:05.944287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-13f5a17e-df0f-4afd-8d6c-4290c8ef6896 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:05.946005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-13f5a17e-df0f-4afd-8d6c-4290c8ef6896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:06.076691 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1457/2934] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:06.085412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95069916-4aaa-4d68-b412-32933e56dece None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:06.085689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95069916-4aaa-4d68-b412-32933e56dece None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:06.086203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95069916-4aaa-4d68-b412-32933e56dece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:06.132568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-95069916-4aaa-4d68-b412-32933e56dece None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:06.133890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-95069916-4aaa-4d68-b412-32933e56dece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:06.148491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1478/2935] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:06.152744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5a17e-df0f-4afd-8d6c-4290c8ef6896 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:06.177152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5a17e-df0f-4afd-8d6c-4290c8ef6896 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:06.184432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22c03d-d6b8-4eab-99eb-3118241f27a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:06.185114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22c03d-d6b8-4eab-99eb-3118241f27a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:06.185382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22c03d-d6b8-4eab-99eb-3118241f27a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:06.185652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22c03d-d6b8-4eab-99eb-3118241f27a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:06.186399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22c03d-d6b8-4eab-99eb-3118241f27a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:57:06.186700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22c03d-d6b8-4eab-99eb-3118241f27a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:06.188297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22c03d-d6b8-4eab-99eb-3118241f27a6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:06.211233 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1458/2936] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:57:06.212049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95069916-4aaa-4d68-b412-32933e56dece None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:06.231293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95069916-4aaa-4d68-b412-32933e56dece None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:06.244308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ce169-4262-4416-bb0d-f8f158bbd97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:06.244925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ce169-4262-4416-bb0d-f8f158bbd97a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:06.245222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ce169-4262-4416-bb0d-f8f158bbd97a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:06.245494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ce169-4262-4416-bb0d-f8f158bbd97a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:06.246141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ce169-4262-4416-bb0d-f8f158bbd97a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:06.248158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ce169-4262-4416-bb0d-f8f158bbd97a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:06.309291 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1479/2937] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:06 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:57:06.313715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22c03d-d6b8-4eab-99eb-3118241f27a6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:06.332862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22c03d-d6b8-4eab-99eb-3118241f27a6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:06.439179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102611a-5efd-4519-8868-37d97273c504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:06.440037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102611a-5efd-4519-8868-37d97273c504 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:06.440991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102611a-5efd-4519-8868-37d97273c504 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:06.442751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102611a-5efd-4519-8868-37d97273c504 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:06.443470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102611a-5efd-4519-8868-37d97273c504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:06.445609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102611a-5efd-4519-8868-37d97273c504 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:06.485145 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1459/2938] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:06 2022] POST /identity/v3/users => generated 296 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:06.488073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ce169-4262-4416-bb0d-f8f158bbd97a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:06.501604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ce169-4262-4416-bb0d-f8f158bbd97a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:06.515606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bc037-dd1d-439d-b8cb-77b87367c5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:06.516125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bc037-dd1d-439d-b8cb-77b87367c5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:06.516400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bc037-dd1d-439d-b8cb-77b87367c5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:06.516667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bc037-dd1d-439d-b8cb-77b87367c5c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:06.517590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664bc037-dd1d-439d-b8cb-77b87367c5c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:06.520768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664bc037-dd1d-439d-b8cb-77b87367c5c7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:06.548252 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1480/2939] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:06.552266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102611a-5efd-4519-8868-37d97273c504 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:06.563940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102611a-5efd-4519-8868-37d97273c504 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:06.570099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a785f8-d0a0-438a-a9a4-4d52cf0500ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:06.570743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a785f8-d0a0-438a-a9a4-4d52cf0500ad admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:06.571187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a785f8-d0a0-438a-a9a4-4d52cf0500ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:06.571839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a785f8-d0a0-438a-a9a4-4d52cf0500ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:06.575343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a785f8-d0a0-438a-a9a4-4d52cf0500ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:06.585118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a785f8-d0a0-438a-a9a4-4d52cf0500ad admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:06.608070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bc037-dd1d-439d-b8cb-77b87367c5c7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:06.613979 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1460/2940] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:06.624778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bc037-dd1d-439d-b8cb-77b87367c5c7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:06.630537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30247f8-f15a-48c4-a12b-83053874a0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:06.631428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30247f8-f15a-48c4-a12b-83053874a0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:06.631882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30247f8-f15a-48c4-a12b-83053874a0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:06.632347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30247f8-f15a-48c4-a12b-83053874a0b7 admin admin] PATH_INFO: `/v3/projects/ba05e0be960d4f33a04730891f16a44e/users/065721429ecc424caf193efe5aae8c99/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:06.658037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30247f8-f15a-48c4-a12b-83053874a0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba05e0be960d4f33a04730891f16a44e, user_id=065721429ecc424caf193efe5aae8c99, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:06.661785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30247f8-f15a-48c4-a12b-83053874a0b7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:06.698252 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1481/2941] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:06 2022] PUT /identity/v3/projects/ba05e0be960d4f33a04730891f16a44e/users/065721429ecc424caf193efe5aae8c99/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:06.711941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c203d-11f8-4567-9e79-8dccfd3d90bd None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:06.712346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c203d-11f8-4567-9e79-8dccfd3d90bd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:06.712719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c203d-11f8-4567-9e79-8dccfd3d90bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:06.727593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:06.751376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-276c203d-11f8-4567-9e79-8dccfd3d90bd None None] MFA Rules not processed for user `065721429ecc424caf193efe5aae8c99`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:06.752763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-276c203d-11f8-4567-9e79-8dccfd3d90bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:06.897367 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1461/2942] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:06.901815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30247f8-f15a-48c4-a12b-83053874a0b7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:06.999858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:07.009672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30247f8-f15a-48c4-a12b-83053874a0b7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:07.019106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89566d81-b695-41d9-a2a3-4dd62054cecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:07.019991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566d81-b695-41d9-a2a3-4dd62054cecf admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:07.020934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566d81-b695-41d9-a2a3-4dd62054cecf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:07.021405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566d81-b695-41d9-a2a3-4dd62054cecf admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:07.022223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89566d81-b695-41d9-a2a3-4dd62054cecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:07.025815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89566d81-b695-41d9-a2a3-4dd62054cecf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:07.068279 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1482/2943] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:06 2022] POST /identity/v3/projects => generated 389 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:07.072564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c203d-11f8-4567-9e79-8dccfd3d90bd None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:07.104824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c203d-11f8-4567-9e79-8dccfd3d90bd None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:07.238223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176190dd-bc61-493c-bd2b-76a7875a6993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:07.244207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176190dd-bc61-493c-bd2b-76a7875a6993 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:07.244207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176190dd-bc61-493c-bd2b-76a7875a6993 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:07.245059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176190dd-bc61-493c-bd2b-76a7875a6993 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:07.253860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176190dd-bc61-493c-bd2b-76a7875a6993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:07.258953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176190dd-bc61-493c-bd2b-76a7875a6993 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:07.309515 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1462/2944] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:07 2022] POST /identity/v3/users => generated 301 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:07.313333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89566d81-b695-41d9-a2a3-4dd62054cecf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:07.339959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89566d81-b695-41d9-a2a3-4dd62054cecf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:07.375438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f3177-9420-449c-924c-7490c3f36ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:07.382489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f3177-9420-449c-924c-7490c3f36ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:07.383013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f3177-9420-449c-924c-7490c3f36ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:07.383348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f3177-9420-449c-924c-7490c3f36ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:07.387892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f3177-9420-449c-924c-7490c3f36ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:07.400325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f3177-9420-449c-924c-7490c3f36ba5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:07.447014 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1483/2945] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:07.447696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176190dd-bc61-493c-bd2b-76a7875a6993 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:07.474417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176190dd-bc61-493c-bd2b-76a7875a6993 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:07.482602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f663e-77da-43fd-a570-460b033c7f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:07.483872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f663e-77da-43fd-a570-460b033c7f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:07.484388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f663e-77da-43fd-a570-460b033c7f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:07.484887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f663e-77da-43fd-a570-460b033c7f1d admin admin] PATH_INFO: `/v3/projects/b5a39a1482644346921682dfcfa3f990/users/af16718ffcfa4424889c9d618914befc/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:07.545974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f663e-77da-43fd-a570-460b033c7f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a39a1482644346921682dfcfa3f990, user_id=af16718ffcfa4424889c9d618914befc, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:07.549859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f663e-77da-43fd-a570-460b033c7f1d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:07.580589 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1463/2946] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:07 2022] PUT /identity/v3/projects/b5a39a1482644346921682dfcfa3f990/users/af16718ffcfa4424889c9d618914befc/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:57:07.584257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f3177-9420-449c-924c-7490c3f36ba5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:07.599415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f3177-9420-449c-924c-7490c3f36ba5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:07.730719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584aae0-99d2-4508-b133-43f8b87eb34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:07.731345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584aae0-99d2-4508-b133-43f8b87eb34e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:07.731620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584aae0-99d2-4508-b133-43f8b87eb34e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:07.731942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584aae0-99d2-4508-b133-43f8b87eb34e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:07.732627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584aae0-99d2-4508-b133-43f8b87eb34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:07.747992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584aae0-99d2-4508-b133-43f8b87eb34e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:07.765280 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1484/2947] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:07.765486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f663e-77da-43fd-a570-460b033c7f1d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:07.784261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f663e-77da-43fd-a570-460b033c7f1d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:07.803718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76af694-4a98-462c-8e43-eb49bd78d41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:07.804481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76af694-4a98-462c-8e43-eb49bd78d41e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:07.804775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76af694-4a98-462c-8e43-eb49bd78d41e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:07.805060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76af694-4a98-462c-8e43-eb49bd78d41e admin admin] PATH_INFO: `/v3/projects/b5a39a1482644346921682dfcfa3f990/users/af16718ffcfa4424889c9d618914befc/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:07.817711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76af694-4a98-462c-8e43-eb49bd78d41e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a39a1482644346921682dfcfa3f990, user_id=af16718ffcfa4424889c9d618914befc, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:07.820370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76af694-4a98-462c-8e43-eb49bd78d41e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:07.875967 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1464/2948] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:07 2022] PUT /identity/v3/projects/b5a39a1482644346921682dfcfa3f990/users/af16718ffcfa4424889c9d618914befc/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:07.886475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b08e2-4345-48ea-924e-79fcc9f79626 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:07.887333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b08e2-4345-48ea-924e-79fcc9f79626 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:07.888633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b08e2-4345-48ea-924e-79fcc9f79626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:07.992055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-556b08e2-4345-48ea-924e-79fcc9f79626 None None] MFA Rules not processed for user `af16718ffcfa4424889c9d618914befc`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:07.992055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-556b08e2-4345-48ea-924e-79fcc9f79626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:08.161827 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1485/2949] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:09.213390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76af694-4a98-462c-8e43-eb49bd78d41e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:09.215335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c76af694-4a98-462c-8e43-eb49bd78d41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:09.319202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:09.378572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76af694-4a98-462c-8e43-eb49bd78d41e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:09.418283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b471c20-557b-4af0-80f9-65a61bd2cf12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:09.420603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b471c20-557b-4af0-80f9-65a61bd2cf12 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:09.421076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b471c20-557b-4af0-80f9-65a61bd2cf12 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:09.421802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b471c20-557b-4af0-80f9-65a61bd2cf12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:09.547208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b471c20-557b-4af0-80f9-65a61bd2cf12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:09.548806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b471c20-557b-4af0-80f9-65a61bd2cf12 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:09.577373 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1465/2950] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:11.435856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78d02a-ef6c-4c80-a661-e6e3de715321 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:11.436405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78d02a-ef6c-4c80-a661-e6e3de715321 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:11.436708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78d02a-ef6c-4c80-a661-e6e3de715321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:11.498105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5d78d02a-ef6c-4c80-a661-e6e3de715321 None None] MFA Rules not processed for user `6adaa12c2ca9469299994300bc856ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:11.499689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5d78d02a-ef6c-4c80-a661-e6e3de715321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:11.625651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:11.648313 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1486/2951] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:15.249670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b471c20-557b-4af0-80f9-65a61bd2cf12 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:15.280595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b471c20-557b-4af0-80f9-65a61bd2cf12 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:15.326818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362edb5e-3633-49ce-8f0e-ca649652891f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:15.330309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362edb5e-3633-49ce-8f0e-ca649652891f service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:15.330426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362edb5e-3633-49ce-8f0e-ca649652891f service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:15.330606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362edb5e-3633-49ce-8f0e-ca649652891f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:15.384851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362edb5e-3633-49ce-8f0e-ca649652891f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:15.393243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362edb5e-3633-49ce-8f0e-ca649652891f service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:15.464051 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1466/2952] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:15 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:18.442328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78d02a-ef6c-4c80-a661-e6e3de715321 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:18.444849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5d78d02a-ef6c-4c80-a661-e6e3de715321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:18.682262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78d02a-ef6c-4c80-a661-e6e3de715321 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:18.700312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:18.703251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:18.703647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:18.705606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] PATH_INFO: `/v3/users/95c5e013a10d4693947db0b06a2f42dd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:18.708779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c5e013a10d4693947db0b06a2f42dd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:18.711017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:18.835885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c5e013a10d4693947db0b06a2f42dd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:18.866070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c5e013a10d4693947db0b06a2f42dd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:18.884292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c5e013a10d4693947db0b06a2f42dd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:18.896817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c5e013a10d4693947db0b06a2f42dd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:18.897805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c5e013a10d4693947db0b06a2f42dd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:18.912240 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1487/2953] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:18 2022] DELETE /identity/v3/users/95c5e013a10d4693947db0b06a2f42dd => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:18.965836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362edb5e-3633-49ce-8f0e-ca649652891f service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:18.968701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-362edb5e-3633-49ce-8f0e-ca649652891f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:18.995368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362edb5e-3633-49ce-8f0e-ca649652891f service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:19.171540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:19.174102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:19.176897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:19.178151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] PATH_INFO: `/v3/users/1fe50f300f3d4fa295033618d73faa7e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:19.183868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fe50f300f3d4fa295033618d73faa7e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:19.189967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:19.308596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fe50f300f3d4fa295033618d73faa7e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:19.328521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fe50f300f3d4fa295033618d73faa7e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:19.359252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fe50f300f3d4fa295033618d73faa7e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:19.363538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:19.384183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fe50f300f3d4fa295033618d73faa7e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:19.384864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-81b296db-ab3d-4040-834f-c8073c11c1b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fe50f300f3d4fa295033618d73faa7e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:19.399813 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1467/2954] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:18 2022] DELETE /identity/v3/users/1fe50f300f3d4fa295033618d73faa7e => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:19.646930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:19.648758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:19.692254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e5df0-7cce-4e19-88fd-aa6639d3a37b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:19.884370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:19.886940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:19.887419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:19.889007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] PATH_INFO: `/v3/projects/2e3da14141a04d7881dc4253f2ac75af` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:19.891640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e3da14141a04d7881dc4253f2ac75af)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:19.897849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:19.926574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:19.996670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e3da14141a04d7881dc4253f2ac75af', 'request_id': 'req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:20.014772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e3da14141a04d7881dc4253f2ac75af', 'request_id': 'req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:20.034758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e3da14141a04d7881dc4253f2ac75af', 'request_id': 'req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:20.055030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] The token cache is being invalidate because project 2e3da14141a04d7881dc4253f2ac75af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:20.055513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:20.056307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fd806cf-8c58-4aa9-9cdd-17e079c807eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:20.060907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1488/2955] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:19 2022] DELETE /identity/v3/projects/2e3da14141a04d7881dc4253f2ac75af => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:20.068125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bed75-3bd4-49db-a1a3-d4afcb740bda None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:20.068593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bed75-3bd4-49db-a1a3-d4afcb740bda None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:20.068973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bed75-3bd4-49db-a1a3-d4afcb740bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:20.106301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-292bed75-3bd4-49db-a1a3-d4afcb740bda None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:20.108465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-292bed75-3bd4-49db-a1a3-d4afcb740bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:20.346556 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1468/2956] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:20.355003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ad21f-5dd0-4d6c-9a43-e5f09863d0f5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:20.355611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ad21f-5dd0-4d6c-9a43-e5f09863d0f5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:20.356005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ad21f-5dd0-4d6c-9a43-e5f09863d0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:20.443567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-274ad21f-5dd0-4d6c-9a43-e5f09863d0f5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:20.444805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-274ad21f-5dd0-4d6c-9a43-e5f09863d0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:20.508536 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1489/2957] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:57:20.511084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bed75-3bd4-49db-a1a3-d4afcb740bda None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:20.521692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bed75-3bd4-49db-a1a3-d4afcb740bda None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:20.527334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c2122-8e87-4537-9efc-ae3b3a43badc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:20.528080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2122-8e87-4537-9efc-ae3b3a43badc admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:20.528359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2122-8e87-4537-9efc-ae3b3a43badc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:20.528660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2122-8e87-4537-9efc-ae3b3a43badc admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:20.529645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c2122-8e87-4537-9efc-ae3b3a43badc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:57:20.530131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c2122-8e87-4537-9efc-ae3b3a43badc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:20.531747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c2122-8e87-4537-9efc-ae3b3a43badc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:20.546871 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1469/2958] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:57:20.551683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ad21f-5dd0-4d6c-9a43-e5f09863d0f5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:20.571111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ad21f-5dd0-4d6c-9a43-e5f09863d0f5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:20.578625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3365c69-a845-4a63-9389-fe72ba4f8285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:20.579245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3365c69-a845-4a63-9389-fe72ba4f8285 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:20.579520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3365c69-a845-4a63-9389-fe72ba4f8285 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:20.579904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3365c69-a845-4a63-9389-fe72ba4f8285 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:20.581028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3365c69-a845-4a63-9389-fe72ba4f8285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:20.584109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3365c69-a845-4a63-9389-fe72ba4f8285 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:20.640663 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1490/2959] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:20 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:20.644500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c2122-8e87-4537-9efc-ae3b3a43badc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:20.681542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c2122-8e87-4537-9efc-ae3b3a43badc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:20.814439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b7025-f0b0-4e3b-aaba-88ccf69f29b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:20.815624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7025-f0b0-4e3b-aaba-88ccf69f29b9 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:20.816117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7025-f0b0-4e3b-aaba-88ccf69f29b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:20.826410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7025-f0b0-4e3b-aaba-88ccf69f29b9 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:20.828378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b7025-f0b0-4e3b-aaba-88ccf69f29b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:20.830686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b7025-f0b0-4e3b-aaba-88ccf69f29b9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:20.883728 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1470/2960] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:20 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:57:20.884945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3365c69-a845-4a63-9389-fe72ba4f8285 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:20.899562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b7025-f0b0-4e3b-aaba-88ccf69f29b9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:20.901554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b00b7025-f0b0-4e3b-aaba-88ccf69f29b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:20.901691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3365c69-a845-4a63-9389-fe72ba4f8285 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:20.916016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0415081-4675-4355-98fd-e0b0223f283d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:20.916628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0415081-4675-4355-98fd-e0b0223f283d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:20.916903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0415081-4675-4355-98fd-e0b0223f283d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:20.917207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0415081-4675-4355-98fd-e0b0223f283d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:20.917855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0415081-4675-4355-98fd-e0b0223f283d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:20.920268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0415081-4675-4355-98fd-e0b0223f283d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:20.926309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b7025-f0b0-4e3b-aaba-88ccf69f29b9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:20.941584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:20.942512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:20.942963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:20.943402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] PATH_INFO: `/v3/projects/77dd9d0796b845a4b66069ed09b88f12` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:20.948350 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1491/2961] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:20.948510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77dd9d0796b845a4b66069ed09b88f12)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:20.951619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0415081-4675-4355-98fd-e0b0223f283d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:20.951830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:20.965829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0415081-4675-4355-98fd-e0b0223f283d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:20.972828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7103a-bf3d-4a61-972c-514b6ebdda1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:20.973480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7103a-bf3d-4a61-972c-514b6ebdda1e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:20.973758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7103a-bf3d-4a61-972c-514b6ebdda1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:20.974043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7103a-bf3d-4a61-972c-514b6ebdda1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:20.974787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7103a-bf3d-4a61-972c-514b6ebdda1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:20.977478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7103a-bf3d-4a61-972c-514b6ebdda1e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:21.040865 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1492/2962] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:21.042095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7103a-bf3d-4a61-972c-514b6ebdda1e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:21.055937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7103a-bf3d-4a61-972c-514b6ebdda1e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:21.068460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77dd9d0796b845a4b66069ed09b88f12', 'request_id': 'req-4ff29386-2815-476b-b6b4-aac265eebe61'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:21.087235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77dd9d0796b845a4b66069ed09b88f12', 'request_id': 'req-4ff29386-2815-476b-b6b4-aac265eebe61'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:21.098459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77dd9d0796b845a4b66069ed09b88f12', 'request_id': 'req-4ff29386-2815-476b-b6b4-aac265eebe61'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:21.102830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=5,keystone:DELETE=7,keystone:SELECT=88 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:21.126199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] The token cache is being invalidate because project 77dd9d0796b845a4b66069ed09b88f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:21.126849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:21.127284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ff29386-2815-476b-b6b4-aac265eebe61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:21.130931 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1471/2963] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:20 2022] DELETE /identity/v3/projects/77dd9d0796b845a4b66069ed09b88f12 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:57:21.136330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d0d11-c53c-42c7-826c-d3e64e037303 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.136673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d0d11-c53c-42c7-826c-d3e64e037303 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.136998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d0d11-c53c-42c7-826c-d3e64e037303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:21.176597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91a2c5-7bbb-45d1-b3ff-98d49ced1248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:21.177293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91a2c5-7bbb-45d1-b3ff-98d49ced1248 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.178186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91a2c5-7bbb-45d1-b3ff-98d49ced1248 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.178524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91a2c5-7bbb-45d1-b3ff-98d49ced1248 admin admin] PATH_INFO: `/v3/projects/2e241b6abf12428ba07a6f052bc00c70/users/353e4db513a94d96bd82b85742624fc2/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:21.181688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ce3d0d11-c53c-42c7-826c-d3e64e037303 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:21.184224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ce3d0d11-c53c-42c7-826c-d3e64e037303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:21.209366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91a2c5-7bbb-45d1-b3ff-98d49ced1248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e241b6abf12428ba07a6f052bc00c70, user_id=353e4db513a94d96bd82b85742624fc2, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:21.212330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91a2c5-7bbb-45d1-b3ff-98d49ced1248 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:21.215726 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1472/2964] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:21.222931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff5745-7474-45d0-89a1-205b389ae8b0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.223288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff5745-7474-45d0-89a1-205b389ae8b0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.223724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff5745-7474-45d0-89a1-205b389ae8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:21.243082 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1493/2965] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:21 2022] PUT /identity/v3/projects/2e241b6abf12428ba07a6f052bc00c70/users/353e4db513a94d96bd82b85742624fc2/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:21.244829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c580a-1c78-4b35-9d9d-04107e099918 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.245285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c580a-1c78-4b35-9d9d-04107e099918 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.245696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c580a-1c78-4b35-9d9d-04107e099918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:21.268023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2cff5745-7474-45d0-89a1-205b389ae8b0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:21.269867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2cff5745-7474-45d0-89a1-205b389ae8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:21.292437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f03c580a-1c78-4b35-9d9d-04107e099918 None None] MFA Rules not processed for user `353e4db513a94d96bd82b85742624fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:21.296150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f03c580a-1c78-4b35-9d9d-04107e099918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:21.399005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:21.440073 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1473/2966] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:21.443925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff5745-7474-45d0-89a1-205b389ae8b0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:21.463454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff5745-7474-45d0-89a1-205b389ae8b0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:21.471573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8ff73-0313-49d0-9db8-1c67ccf851db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:21.472178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8ff73-0313-49d0-9db8-1c67ccf851db admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.472487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8ff73-0313-49d0-9db8-1c67ccf851db admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.472695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8ff73-0313-49d0-9db8-1c67ccf851db admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:21.473958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8ff73-0313-49d0-9db8-1c67ccf851db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:57:21.474148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8ff73-0313-49d0-9db8-1c67ccf851db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:21.475780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8ff73-0313-49d0-9db8-1c67ccf851db admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:21.492007 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1474/2967] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:57:21.496404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8ff73-0313-49d0-9db8-1c67ccf851db admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:21.500661 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1494/2968] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:21.515071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8ff73-0313-49d0-9db8-1c67ccf851db admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:21.520771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8eeed-bbd3-4324-870c-08a3f65a20be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:21.521400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8eeed-bbd3-4324-870c-08a3f65a20be admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.521676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8eeed-bbd3-4324-870c-08a3f65a20be admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.521942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8eeed-bbd3-4324-870c-08a3f65a20be admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:21.522619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8eeed-bbd3-4324-870c-08a3f65a20be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:21.525951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8eeed-bbd3-4324-870c-08a3f65a20be admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:21.578359 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1475/2969] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:21 2022] POST /identity/v3/projects => generated 409 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:21.580871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c580a-1c78-4b35-9d9d-04107e099918 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:21.600857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c580a-1c78-4b35-9d9d-04107e099918 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:21.745180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c144cc-64fa-4c8b-a5cc-e0f3e3a8dbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:21.751090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c144cc-64fa-4c8b-a5cc-e0f3e3a8dbf9 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.751539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c144cc-64fa-4c8b-a5cc-e0f3e3a8dbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.751820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c144cc-64fa-4c8b-a5cc-e0f3e3a8dbf9 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:21.753359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c144cc-64fa-4c8b-a5cc-e0f3e3a8dbf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:21.755465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c144cc-64fa-4c8b-a5cc-e0f3e3a8dbf9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:21.788902 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1495/2970] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:21 2022] POST /identity/v3/users => generated 305 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:21.797224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8eeed-bbd3-4324-870c-08a3f65a20be admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:21.819952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8eeed-bbd3-4324-870c-08a3f65a20be admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:21.829485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb62c7-ec73-4a24-80c2-c895e3caa647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:21.830725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb62c7-ec73-4a24-80c2-c895e3caa647 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.831296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb62c7-ec73-4a24-80c2-c895e3caa647 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.831819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb62c7-ec73-4a24-80c2-c895e3caa647 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:21.832883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb62c7-ec73-4a24-80c2-c895e3caa647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:21.834866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb62c7-ec73-4a24-80c2-c895e3caa647 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:21.855156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c144cc-64fa-4c8b-a5cc-e0f3e3a8dbf9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:21.855520 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1476/2971] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:21.868710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c144cc-64fa-4c8b-a5cc-e0f3e3a8dbf9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:21.875172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbe437-0d8a-4afa-945e-5c383d0350fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:21.875979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbe437-0d8a-4afa-945e-5c383d0350fd admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.876539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbe437-0d8a-4afa-945e-5c383d0350fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.876938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbe437-0d8a-4afa-945e-5c383d0350fd admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:21.877835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbe437-0d8a-4afa-945e-5c383d0350fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:21.879796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbe437-0d8a-4afa-945e-5c383d0350fd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:21.897473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb62c7-ec73-4a24-80c2-c895e3caa647 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:21.906785 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1496/2972] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:21.911357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb62c7-ec73-4a24-80c2-c895e3caa647 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:21.917058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b46989-4801-4e6c-9777-8e82a954142d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:21.917941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b46989-4801-4e6c-9777-8e82a954142d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.918279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b46989-4801-4e6c-9777-8e82a954142d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.918565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b46989-4801-4e6c-9777-8e82a954142d admin admin] PATH_INFO: `/v3/projects/11bdafc438884621944db399da5528f8/users/78d679a10c9e4bf2bc35786235aeaf2a/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:21.937062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b46989-4801-4e6c-9777-8e82a954142d admin admin] RBAC: Authorizing `identity:create_grant(project_id=11bdafc438884621944db399da5528f8, user_id=78d679a10c9e4bf2bc35786235aeaf2a, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:21.949412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b46989-4801-4e6c-9777-8e82a954142d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:21.969148 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1477/2973] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:21 2022] PUT /identity/v3/projects/11bdafc438884621944db399da5528f8/users/78d679a10c9e4bf2bc35786235aeaf2a/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:21.971806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9e0c8-9bce-4616-816a-dac862f95581 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:21.972091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9e0c8-9bce-4616-816a-dac862f95581 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:21.972362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9e0c8-9bce-4616-816a-dac862f95581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:22.000795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fde9e0c8-9bce-4616-816a-dac862f95581 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:22.002317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fde9e0c8-9bce-4616-816a-dac862f95581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:22.097727 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1497/2974] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:22.109759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b46989-4801-4e6c-9777-8e82a954142d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:22.111502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-83b46989-4801-4e6c-9777-8e82a954142d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:22.194822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b46989-4801-4e6c-9777-8e82a954142d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:22.205847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3317e6-6c57-4bb1-af01-a9041c8aaef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:22.206546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3317e6-6c57-4bb1-af01-a9041c8aaef7 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:22.206892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3317e6-6c57-4bb1-af01-a9041c8aaef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:22.207429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3317e6-6c57-4bb1-af01-a9041c8aaef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:22.218335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3317e6-6c57-4bb1-af01-a9041c8aaef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:22.219871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3317e6-6c57-4bb1-af01-a9041c8aaef7 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:22.231757 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1478/2975] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:22.830931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9e0c8-9bce-4616-816a-dac862f95581 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:22.832385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fde9e0c8-9bce-4616-816a-dac862f95581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:22.847252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9e0c8-9bce-4616-816a-dac862f95581 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:22.853879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:22.854380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:22.854655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:22.854977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] PATH_INFO: `/v3/users/180d9b7141ff4792ab7ab19e21a3d7ac` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:22.856226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=180d9b7141ff4792ab7ab19e21a3d7ac)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:22.859726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:22.933565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:22.942753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '180d9b7141ff4792ab7ab19e21a3d7ac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:22.963116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '180d9b7141ff4792ab7ab19e21a3d7ac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:22.976144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '180d9b7141ff4792ab7ab19e21a3d7ac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:22.989993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '180d9b7141ff4792ab7ab19e21a3d7ac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:22.991541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '180d9b7141ff4792ab7ab19e21a3d7ac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.002545 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1498/2976] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:22 2022] DELETE /identity/v3/users/180d9b7141ff4792ab7ab19e21a3d7ac => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:23.217932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3317e6-6c57-4bb1-af01-a9041c8aaef7 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:23.229107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3317e6-6c57-4bb1-af01-a9041c8aaef7 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:23.243651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:23.308965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:23.310554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:23.311402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:23.311680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] PATH_INFO: `/v3/projects/f400a79cb647438f968a895142c5410d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:23.313013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f400a79cb647438f968a895142c5410d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:23.315076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:23.381280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f400a79cb647438f968a895142c5410d', 'request_id': 'req-5a85bcdc-2d97-4162-8b63-1485616d2471'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.390306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f400a79cb647438f968a895142c5410d', 'request_id': 'req-5a85bcdc-2d97-4162-8b63-1485616d2471'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.397622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f400a79cb647438f968a895142c5410d', 'request_id': 'req-5a85bcdc-2d97-4162-8b63-1485616d2471'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.409075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] The token cache is being invalidate because project f400a79cb647438f968a895142c5410d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:23.409249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.409960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.411600 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1479/2977] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:23 2022] DELETE /identity/v3/projects/f400a79cb647438f968a895142c5410d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:23.415596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:23.421353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:23.437843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792b6ed-9055-4dfc-88b4-46145ea59e94 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:23.525847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:23.526714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:23.527283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:23.528114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] PATH_INFO: `/v3/users/c2394c816d8c47b887c085027a81b7a9` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:23.529788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2394c816d8c47b887c085027a81b7a9)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:23.533918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:23.589729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2394c816d8c47b887c085027a81b7a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.608088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2394c816d8c47b887c085027a81b7a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.620326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2394c816d8c47b887c085027a81b7a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.632761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2394c816d8c47b887c085027a81b7a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.634342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94579258-b446-47ff-adc7-790c55d0a58d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2394c816d8c47b887c085027a81b7a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:23.651612 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1499/2978] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:23 2022] DELETE /identity/v3/users/c2394c816d8c47b887c085027a81b7a9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:23.882588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:23.883604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:23.901494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85bcdc-2d97-4162-8b63-1485616d2471 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:23.994699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:23.995376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:23.995680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:23.995946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] PATH_INFO: `/v3/projects/4a5d6aa09fb0449ea9078a380c02ac87` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:23.997175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5d6aa09fb0449ea9078a380c02ac87)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:23.999778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:24.091641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5d6aa09fb0449ea9078a380c02ac87', 'request_id': 'req-0c721d5a-856f-43a6-828e-d89c8333ad81'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:24.094451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:24.105974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5d6aa09fb0449ea9078a380c02ac87', 'request_id': 'req-0c721d5a-856f-43a6-828e-d89c8333ad81'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:24.115980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5d6aa09fb0449ea9078a380c02ac87', 'request_id': 'req-0c721d5a-856f-43a6-828e-d89c8333ad81'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:24.129320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] The token cache is being invalidate because project 4a5d6aa09fb0449ea9078a380c02ac87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:24.129664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:24.130319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c721d5a-856f-43a6-828e-d89c8333ad81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:24.132039 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1480/2979] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:23 2022] DELETE /identity/v3/projects/4a5d6aa09fb0449ea9078a380c02ac87 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:24.137480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff958-9713-4269-9577-02b6179f26fe None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:24.138589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff958-9713-4269-9577-02b6179f26fe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:24.139171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff958-9713-4269-9577-02b6179f26fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:24.167766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ac2ff958-9713-4269-9577-02b6179f26fe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:24.171463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac2ff958-9713-4269-9577-02b6179f26fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:24.223634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:24.265287 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1500/2980] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:57:24.271880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fadfa-2fb0-4f2a-8789-163cfef557dc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:24.272425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fadfa-2fb0-4f2a-8789-163cfef557dc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:24.272425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fadfa-2fb0-4f2a-8789-163cfef557dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:24.293116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e71fadfa-2fb0-4f2a-8789-163cfef557dc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:24.295143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e71fadfa-2fb0-4f2a-8789-163cfef557dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:24.304750 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1481/2981] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:24.311872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ff958-9713-4269-9577-02b6179f26fe None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:24.324543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ff958-9713-4269-9577-02b6179f26fe None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:24.337690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ab149-a7a1-46d9-b831-9df45f0f1507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:24.338326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ab149-a7a1-46d9-b831-9df45f0f1507 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:24.338602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ab149-a7a1-46d9-b831-9df45f0f1507 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:24.339356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ab149-a7a1-46d9-b831-9df45f0f1507 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:24.339535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ab149-a7a1-46d9-b831-9df45f0f1507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:57:24.339842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ab149-a7a1-46d9-b831-9df45f0f1507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:24.342686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ab149-a7a1-46d9-b831-9df45f0f1507 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:24.359831 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1501/2982] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:57:24.364507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fadfa-2fb0-4f2a-8789-163cfef557dc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:24.380321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fadfa-2fb0-4f2a-8789-163cfef557dc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:24.385854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0e6c5-65c5-4bd6-896f-937e431dcdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:24.386532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0e6c5-65c5-4bd6-896f-937e431dcdb3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:24.386810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0e6c5-65c5-4bd6-896f-937e431dcdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:24.387077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0e6c5-65c5-4bd6-896f-937e431dcdb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:24.387719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0e6c5-65c5-4bd6-896f-937e431dcdb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:24.390878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0e6c5-65c5-4bd6-896f-937e431dcdb3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:24.439875 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1482/2983] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:24 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:24.444075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ab149-a7a1-46d9-b831-9df45f0f1507 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:24.463702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ab149-a7a1-46d9-b831-9df45f0f1507 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:24.539597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4889b-2775-4f73-801a-3195facdf32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:24.540078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4889b-2775-4f73-801a-3195facdf32e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:24.540643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4889b-2775-4f73-801a-3195facdf32e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:24.540728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4889b-2775-4f73-801a-3195facdf32e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:24.541466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4889b-2775-4f73-801a-3195facdf32e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:24.543657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4889b-2775-4f73-801a-3195facdf32e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:24.571968 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1502/2984] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:24 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:24.576402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0e6c5-65c5-4bd6-896f-937e431dcdb3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:24.593272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0e6c5-65c5-4bd6-896f-937e431dcdb3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:24.600194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08c743-a859-47ad-bd6c-9be09a69ea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:24.614991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08c743-a859-47ad-bd6c-9be09a69ea81 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:24.615284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08c743-a859-47ad-bd6c-9be09a69ea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:24.615560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08c743-a859-47ad-bd6c-9be09a69ea81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:24.616835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08c743-a859-47ad-bd6c-9be09a69ea81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:24.618701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08c743-a859-47ad-bd6c-9be09a69ea81 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:24.633356 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1483/2985] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:24.636925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4889b-2775-4f73-801a-3195facdf32e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:24.649847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4889b-2775-4f73-801a-3195facdf32e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:24.656053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c45ac-899c-40d5-84cc-294ab67e25a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:24.657204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c45ac-899c-40d5-84cc-294ab67e25a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:24.658323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c45ac-899c-40d5-84cc-294ab67e25a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:24.658863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c45ac-899c-40d5-84cc-294ab67e25a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:24.659780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c45ac-899c-40d5-84cc-294ab67e25a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:24.661596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c45ac-899c-40d5-84cc-294ab67e25a4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:24.678046 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1503/2986] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:24.681946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08c743-a859-47ad-bd6c-9be09a69ea81 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:24.697846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08c743-a859-47ad-bd6c-9be09a69ea81 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:24.705078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e96ac-01cd-423f-a540-0540da2aef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:24.706806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e96ac-01cd-423f-a540-0540da2aef0a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:24.707274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e96ac-01cd-423f-a540-0540da2aef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:24.707958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e96ac-01cd-423f-a540-0540da2aef0a admin admin] PATH_INFO: `/v3/projects/c44841db4e33412e805da46ba4262fe7/users/500cba6141d34c4081d38a7b1be1775e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:24.737117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e96ac-01cd-423f-a540-0540da2aef0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c44841db4e33412e805da46ba4262fe7, user_id=500cba6141d34c4081d38a7b1be1775e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:24.740512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e96ac-01cd-423f-a540-0540da2aef0a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:24.775142 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1484/2987] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:24 2022] PUT /identity/v3/projects/c44841db4e33412e805da46ba4262fe7/users/500cba6141d34c4081d38a7b1be1775e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:24.776616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704bdf9-79b6-4b2f-b3dc-03ebdd4ad380 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:24.776947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704bdf9-79b6-4b2f-b3dc-03ebdd4ad380 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:24.777252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704bdf9-79b6-4b2f-b3dc-03ebdd4ad380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:24.832543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a704bdf9-79b6-4b2f-b3dc-03ebdd4ad380 None None] MFA Rules not processed for user `500cba6141d34c4081d38a7b1be1775e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:24.834134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a704bdf9-79b6-4b2f-b3dc-03ebdd4ad380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:24.906151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e96ac-01cd-423f-a540-0540da2aef0a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:24.909062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-023e96ac-01cd-423f-a540-0540da2aef0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:24.995095 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1504/2988] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:24.999541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704bdf9-79b6-4b2f-b3dc-03ebdd4ad380 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:25.044688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:25.078841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e96ac-01cd-423f-a540-0540da2aef0a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:25.093835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d30b3-e746-4ef0-bf62-39ef2fd10706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:25.094044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d30b3-e746-4ef0-bf62-39ef2fd10706 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:25.094308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d30b3-e746-4ef0-bf62-39ef2fd10706 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:25.094575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d30b3-e746-4ef0-bf62-39ef2fd10706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:25.097506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704bdf9-79b6-4b2f-b3dc-03ebdd4ad380 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:25.098704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-da2d30b3-e746-4ef0-bf62-39ef2fd10706 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:25.108618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf15f7-705a-496f-8709-efdb064ceb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:25.109522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf15f7-705a-496f-8709-efdb064ceb17 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:25.110009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf15f7-705a-496f-8709-efdb064ceb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:25.110281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf15f7-705a-496f-8709-efdb064ceb17 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:25.110944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf15f7-705a-496f-8709-efdb064ceb17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:25.115561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf15f7-705a-496f-8709-efdb064ceb17 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:25.178242 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1505/2989] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:24 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:57:25.179216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf15f7-705a-496f-8709-efdb064ceb17 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:25.211018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d30b3-e746-4ef0-bf62-39ef2fd10706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:25.212894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d30b3-e746-4ef0-bf62-39ef2fd10706 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:25.226273 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1485/2990] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:25.299124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf15f7-705a-496f-8709-efdb064ceb17 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:25.306939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa70836-e326-4f0a-bf1f-57087415899f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:25.308041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa70836-e326-4f0a-bf1f-57087415899f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:25.308509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa70836-e326-4f0a-bf1f-57087415899f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:25.309285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa70836-e326-4f0a-bf1f-57087415899f admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:25.311964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa70836-e326-4f0a-bf1f-57087415899f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:25.314270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa70836-e326-4f0a-bf1f-57087415899f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:25.355413 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1506/2991] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:25 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:25.360336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d30b3-e746-4ef0-bf62-39ef2fd10706 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:25.376227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d30b3-e746-4ef0-bf62-39ef2fd10706 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:25.384054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2f323-ed8a-48cd-b670-1ebf8545b48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:25.384674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2f323-ed8a-48cd-b670-1ebf8545b48c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:25.384943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2f323-ed8a-48cd-b670-1ebf8545b48c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:25.385290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2f323-ed8a-48cd-b670-1ebf8545b48c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:25.385932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2f323-ed8a-48cd-b670-1ebf8545b48c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:25.388809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2f323-ed8a-48cd-b670-1ebf8545b48c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:25.402562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:25.409497 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1486/2992] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:25.414011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa70836-e326-4f0a-bf1f-57087415899f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:25.425845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa70836-e326-4f0a-bf1f-57087415899f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:25.430949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d835a-1b23-41c2-892d-b651a856c21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:25.431692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d835a-1b23-41c2-892d-b651a856c21e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:25.432027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d835a-1b23-41c2-892d-b651a856c21e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:25.432361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d835a-1b23-41c2-892d-b651a856c21e admin admin] PATH_INFO: `/v3/projects/af0b8f89094f42fb9ae9829807d9f6d4/users/623abfad80794a058a678b8b50009e3e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:25.449404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d835a-1b23-41c2-892d-b651a856c21e admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0b8f89094f42fb9ae9829807d9f6d4, user_id=623abfad80794a058a678b8b50009e3e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:25.452830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d835a-1b23-41c2-892d-b651a856c21e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:25.467837 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1507/2993] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:25 2022] PUT /identity/v3/projects/af0b8f89094f42fb9ae9829807d9f6d4/users/623abfad80794a058a678b8b50009e3e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:57:25.471794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2f323-ed8a-48cd-b670-1ebf8545b48c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:25.482992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2f323-ed8a-48cd-b670-1ebf8545b48c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:25.564386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ac34-81b0-4521-b956-25c3e86631a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:25.565468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ac34-81b0-4521-b956-25c3e86631a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:25.565938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ac34-81b0-4521-b956-25c3e86631a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:25.566348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ac34-81b0-4521-b956-25c3e86631a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:25.567184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5ac34-81b0-4521-b956-25c3e86631a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:25.569018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5ac34-81b0-4521-b956-25c3e86631a3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:25.581459 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1487/2994] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:25.585326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d835a-1b23-41c2-892d-b651a856c21e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:25.599349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d835a-1b23-41c2-892d-b651a856c21e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:25.604622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac659e0-c369-4e38-a488-d62a9f76e928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:25.606318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac659e0-c369-4e38-a488-d62a9f76e928 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:25.606318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac659e0-c369-4e38-a488-d62a9f76e928 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:25.606318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac659e0-c369-4e38-a488-d62a9f76e928 admin admin] PATH_INFO: `/v3/projects/af0b8f89094f42fb9ae9829807d9f6d4/users/623abfad80794a058a678b8b50009e3e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:25.609304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac659e0-c369-4e38-a488-d62a9f76e928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0b8f89094f42fb9ae9829807d9f6d4, user_id=623abfad80794a058a678b8b50009e3e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:25.612089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac659e0-c369-4e38-a488-d62a9f76e928 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:25.636981 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1508/2995] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:25 2022] PUT /identity/v3/projects/af0b8f89094f42fb9ae9829807d9f6d4/users/623abfad80794a058a678b8b50009e3e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:25.638820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516c34d-cbfc-47b4-ac2b-c65527952067 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:25.639419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516c34d-cbfc-47b4-ac2b-c65527952067 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:25.639828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516c34d-cbfc-47b4-ac2b-c65527952067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:25.680368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a516c34d-cbfc-47b4-ac2b-c65527952067 None None] MFA Rules not processed for user `623abfad80794a058a678b8b50009e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:25.681726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a516c34d-cbfc-47b4-ac2b-c65527952067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:25.788791 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1488/2996] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:25.801520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5e180-96a5-4227-80a1-2f58d3c3e4b4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:25.802359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5e180-96a5-4227-80a1-2f58d3c3e4b4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:25.802359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5e180-96a5-4227-80a1-2f58d3c3e4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:25.834759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-abc5e180-96a5-4227-80a1-2f58d3c3e4b4 None None] MFA Rules not processed for user `500cba6141d34c4081d38a7b1be1775e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:25.836595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-abc5e180-96a5-4227-80a1-2f58d3c3e4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:25.902622 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1509/2997] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:26.668180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516c34d-cbfc-47b4-ac2b-c65527952067 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:26.755444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516c34d-cbfc-47b4-ac2b-c65527952067 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:26.763548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6dbfe-27ae-4cdc-b589-4287c8608eef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:26.763548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6dbfe-27ae-4cdc-b589-4287c8608eef service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:26.763917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6dbfe-27ae-4cdc-b589-4287c8608eef service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:26.764095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6dbfe-27ae-4cdc-b589-4287c8608eef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:26.774424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6dbfe-27ae-4cdc-b589-4287c8608eef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:26.776827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6dbfe-27ae-4cdc-b589-4287c8608eef service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:26.787812 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1489/2998] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:27.570402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa3d8f-4eb2-4400-bc6a-3c3d750456d4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:27.570863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa3d8f-4eb2-4400-bc6a-3c3d750456d4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:27.570950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa3d8f-4eb2-4400-bc6a-3c3d750456d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:27.598397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bafa3d8f-4eb2-4400-bc6a-3c3d750456d4 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:27.599877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bafa3d8f-4eb2-4400-bc6a-3c3d750456d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:27.616322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:27.768562 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1510/2999] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:27.786638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6dbfe-27ae-4cdc-b589-4287c8608eef service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:27.789929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a1e6dbfe-27ae-4cdc-b589-4287c8608eef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:27.804691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:27.922231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6dbfe-27ae-4cdc-b589-4287c8608eef service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:27.928372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20808890-e049-4084-891f-45d3d5665c6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:27.929131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20808890-e049-4084-891f-45d3d5665c6c service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:27.929523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20808890-e049-4084-891f-45d3d5665c6c service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:27.930060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20808890-e049-4084-891f-45d3d5665c6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:27.951801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20808890-e049-4084-891f-45d3d5665c6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:27.955877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20808890-e049-4084-891f-45d3d5665c6c service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:27.968031 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1490/3000] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:57:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:57:30.213275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3c808-5b71-45a7-8943-c069ad39398a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:30.213761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3c808-5b71-45a7-8943-c069ad39398a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:30.213855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3c808-5b71-45a7-8943-c069ad39398a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:30.296790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b0a3c808-5b71-45a7-8943-c069ad39398a None None] MFA Rules not processed for user `78d679a10c9e4bf2bc35786235aeaf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:30.302859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b0a3c808-5b71-45a7-8943-c069ad39398a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:30.394875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4127d-db9e-45ef-a42d-67d4faf9449b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:30.395887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4127d-db9e-45ef-a42d-67d4faf9449b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:30.396382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4127d-db9e-45ef-a42d-67d4faf9449b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:30.445444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-13a4127d-db9e-45ef-a42d-67d4faf9449b None None] MFA Rules not processed for user `5d93a680570f4d23b1f6df534bc082b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:30.450725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-13a4127d-db9e-45ef-a42d-67d4faf9449b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:30.519171 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1511/3001] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:30.520907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab42d91-2f5f-4cfb-b5bd-937f308e0dd1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:30.521277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab42d91-2f5f-4cfb-b5bd-937f308e0dd1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:30.522200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab42d91-2f5f-4cfb-b5bd-937f308e0dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:30.579668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-eab42d91-2f5f-4cfb-b5bd-937f308e0dd1 None None] MFA Rules not processed for user `78d679a10c9e4bf2bc35786235aeaf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:30.581269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eab42d91-2f5f-4cfb-b5bd-937f308e0dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:30.606327 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1512/3002] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:30.645500 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1491/3003] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:31.349256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab42d91-2f5f-4cfb-b5bd-937f308e0dd1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:31.365511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab42d91-2f5f-4cfb-b5bd-937f308e0dd1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:31.372300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d89e68-1ce4-4afb-a73f-00e7ee719220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:31.373117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d89e68-1ce4-4afb-a73f-00e7ee719220 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:31.373574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d89e68-1ce4-4afb-a73f-00e7ee719220 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:31.373849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d89e68-1ce4-4afb-a73f-00e7ee719220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:31.388582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d89e68-1ce4-4afb-a73f-00e7ee719220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:31.390215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d89e68-1ce4-4afb-a73f-00e7ee719220 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:31.407879 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1513/3004] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:31 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:31.892192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4127d-db9e-45ef-a42d-67d4faf9449b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:31.898916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-13a4127d-db9e-45ef-a42d-67d4faf9449b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:32.071710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4127d-db9e-45ef-a42d-67d4faf9449b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:32.079488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:32.081175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:32.081738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:32.082517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] PATH_INFO: `/v3/users/de4d5d0008324d7a807a8f8361a60a4e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:32.083925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=de4d5d0008324d7a807a8f8361a60a4e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:32.087578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:32.184549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:32.190719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de4d5d0008324d7a807a8f8361a60a4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:32.200038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de4d5d0008324d7a807a8f8361a60a4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:32.223124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de4d5d0008324d7a807a8f8361a60a4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:32.232820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de4d5d0008324d7a807a8f8361a60a4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:32.233556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-023a5c27-6053-4e11-98ad-e2ae9b5c7a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de4d5d0008324d7a807a8f8361a60a4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:32.245452 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1492/3005] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:31 2022] DELETE /identity/v3/users/de4d5d0008324d7a807a8f8361a60a4e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:32.932937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d89e68-1ce4-4afb-a73f-00e7ee719220 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:32.952505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d89e68-1ce4-4afb-a73f-00e7ee719220 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:33.066583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:33.068481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:33.069688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:33.070548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] PATH_INFO: `/v3/projects/0a0d3787e2a842029da9fb7d0a6715b0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:33.072162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a0d3787e2a842029da9fb7d0a6715b0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:33.075468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:33.135002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:33.151301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a0d3787e2a842029da9fb7d0a6715b0', 'request_id': 'req-3ad47b96-7dbe-408e-994e-fbaa5acdd541'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:33.163003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a0d3787e2a842029da9fb7d0a6715b0', 'request_id': 'req-3ad47b96-7dbe-408e-994e-fbaa5acdd541'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:33.175222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a0d3787e2a842029da9fb7d0a6715b0', 'request_id': 'req-3ad47b96-7dbe-408e-994e-fbaa5acdd541'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:33.189533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] The token cache is being invalidate because project 0a0d3787e2a842029da9fb7d0a6715b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:33.202949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:33.203839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ad47b96-7dbe-408e-994e-fbaa5acdd541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:33.206042 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1514/3006] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:32 2022] DELETE /identity/v3/projects/0a0d3787e2a842029da9fb7d0a6715b0 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:33.210963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1ae8e-f1f3-4e3d-898a-2d731fd42637 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:33.212562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1ae8e-f1f3-4e3d-898a-2d731fd42637 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:33.212802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1ae8e-f1f3-4e3d-898a-2d731fd42637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:33.252020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-37f1ae8e-f1f3-4e3d-898a-2d731fd42637 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:33.256080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-37f1ae8e-f1f3-4e3d-898a-2d731fd42637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:33.356976 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1493/3007] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:33.365555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339f87-acc7-4537-899d-f9f8533c0633 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:33.365961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339f87-acc7-4537-899d-f9f8533c0633 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:33.367029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339f87-acc7-4537-899d-f9f8533c0633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:33.395640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-80339f87-acc7-4537-899d-f9f8533c0633 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:33.397141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-80339f87-acc7-4537-899d-f9f8533c0633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:33.410429 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1515/3008] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:33.413067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1ae8e-f1f3-4e3d-898a-2d731fd42637 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:33.426732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1ae8e-f1f3-4e3d-898a-2d731fd42637 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:33.431774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9f32b-6d41-4fa9-b307-663f068e2576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:33.432729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9f32b-6d41-4fa9-b307-663f068e2576 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:33.433013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9f32b-6d41-4fa9-b307-663f068e2576 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:33.433308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9f32b-6d41-4fa9-b307-663f068e2576 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:33.433952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9f32b-6d41-4fa9-b307-663f068e2576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:57:33.434295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9f32b-6d41-4fa9-b307-663f068e2576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:33.437883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9f32b-6d41-4fa9-b307-663f068e2576 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:33.459294 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1494/3009] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:57:33.463238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80339f87-acc7-4537-899d-f9f8533c0633 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:33.475829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80339f87-acc7-4537-899d-f9f8533c0633 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:33.489481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45878360-5682-4a7d-ba90-fad2bab7afc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:33.490093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45878360-5682-4a7d-ba90-fad2bab7afc0 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:33.490428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45878360-5682-4a7d-ba90-fad2bab7afc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:33.490766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45878360-5682-4a7d-ba90-fad2bab7afc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:33.491454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45878360-5682-4a7d-ba90-fad2bab7afc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:33.495426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45878360-5682-4a7d-ba90-fad2bab7afc0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:33.552062 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1516/3010] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:33 2022] POST /identity/v3/projects => generated 399 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:33.553692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9f32b-6d41-4fa9-b307-663f068e2576 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:33.566524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9f32b-6d41-4fa9-b307-663f068e2576 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:33.730281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fc7d2-8143-48fa-a1ff-b63d3d270405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:33.730949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fc7d2-8143-48fa-a1ff-b63d3d270405 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:33.731227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fc7d2-8143-48fa-a1ff-b63d3d270405 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:33.731798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fc7d2-8143-48fa-a1ff-b63d3d270405 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:33.732161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fc7d2-8143-48fa-a1ff-b63d3d270405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:33.734678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fc7d2-8143-48fa-a1ff-b63d3d270405 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:33.771379 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1495/3011] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:33 2022] POST /identity/v3/users => generated 300 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:57:33.775104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45878360-5682-4a7d-ba90-fad2bab7afc0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:33.789201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45878360-5682-4a7d-ba90-fad2bab7afc0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:33.797883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813456b7-387c-42bc-8fc7-e9c11b71ff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:33.798829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813456b7-387c-42bc-8fc7-e9c11b71ff9c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:33.799288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813456b7-387c-42bc-8fc7-e9c11b71ff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:33.799736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813456b7-387c-42bc-8fc7-e9c11b71ff9c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:33.800834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813456b7-387c-42bc-8fc7-e9c11b71ff9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:33.802938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813456b7-387c-42bc-8fc7-e9c11b71ff9c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:33.816642 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1517/3012] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:57:33.820058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fc7d2-8143-48fa-a1ff-b63d3d270405 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:33.836699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fc7d2-8143-48fa-a1ff-b63d3d270405 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:33.845596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d45a5-dd50-409e-8860-dc4e572746b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:33.846403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d45a5-dd50-409e-8860-dc4e572746b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:33.846686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d45a5-dd50-409e-8860-dc4e572746b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:33.846952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d45a5-dd50-409e-8860-dc4e572746b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:33.847756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d45a5-dd50-409e-8860-dc4e572746b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:33.850245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d45a5-dd50-409e-8860-dc4e572746b8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:33.865840 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1496/3013] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:33.870076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813456b7-387c-42bc-8fc7-e9c11b71ff9c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:33.882101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813456b7-387c-42bc-8fc7-e9c11b71ff9c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:33.888686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dddee-8e02-4b7a-90b5-290c24ed9626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:33.890428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dddee-8e02-4b7a-90b5-290c24ed9626 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:33.890759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dddee-8e02-4b7a-90b5-290c24ed9626 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:33.891047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dddee-8e02-4b7a-90b5-290c24ed9626 admin admin] PATH_INFO: `/v3/projects/974bd50eb06e453e9bb20218b9635555/users/79e244a9ce2343dbb903a4722da86eaf/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:33.915213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dddee-8e02-4b7a-90b5-290c24ed9626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974bd50eb06e453e9bb20218b9635555, user_id=79e244a9ce2343dbb903a4722da86eaf, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:33.918177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dddee-8e02-4b7a-90b5-290c24ed9626 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:33.946143 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1518/3014] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:33 2022] PUT /identity/v3/projects/974bd50eb06e453e9bb20218b9635555/users/79e244a9ce2343dbb903a4722da86eaf/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:33.953520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898acd8-1c98-4e19-8cfa-c92ab88b47a6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:33.953805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898acd8-1c98-4e19-8cfa-c92ab88b47a6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:33.954074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898acd8-1c98-4e19-8cfa-c92ab88b47a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:33.978222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1898acd8-1c98-4e19-8cfa-c92ab88b47a6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:33.979646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1898acd8-1c98-4e19-8cfa-c92ab88b47a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:34.006143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:34.093210 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1497/3015] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:34.104023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dddee-8e02-4b7a-90b5-290c24ed9626 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:34.106957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af8dddee-8e02-4b7a-90b5-290c24ed9626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:34.193945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dddee-8e02-4b7a-90b5-290c24ed9626 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:34.206105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a078-f745-4e9e-bdca-b5c651fbe2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:34.207000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a078-f745-4e9e-bdca-b5c651fbe2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:34.207000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a078-f745-4e9e-bdca-b5c651fbe2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:34.207273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a078-f745-4e9e-bdca-b5c651fbe2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:34.221778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a078-f745-4e9e-bdca-b5c651fbe2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:34.224508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a078-f745-4e9e-bdca-b5c651fbe2f9 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:34.238986 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1519/3016] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:35.175553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898acd8-1c98-4e19-8cfa-c92ab88b47a6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:35.177475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1898acd8-1c98-4e19-8cfa-c92ab88b47a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:35.248743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898acd8-1c98-4e19-8cfa-c92ab88b47a6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:35.259803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:35.260579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:35.260864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:35.262527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] PATH_INFO: `/v3/users/065721429ecc424caf193efe5aae8c99` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:35.264375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=065721429ecc424caf193efe5aae8c99)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:35.269285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:35.341959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '065721429ecc424caf193efe5aae8c99'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:35.350772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '065721429ecc424caf193efe5aae8c99'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:35.366830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '065721429ecc424caf193efe5aae8c99'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:35.383788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '065721429ecc424caf193efe5aae8c99'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:35.384859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '065721429ecc424caf193efe5aae8c99'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:35.404698 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1498/3017] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:35 2022] DELETE /identity/v3/users/065721429ecc424caf193efe5aae8c99 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:35.409880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1471fc8-a0e8-43d8-b862-4ad45c267fc8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:35.410435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1471fc8-a0e8-43d8-b862-4ad45c267fc8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:35.410653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1471fc8-a0e8-43d8-b862-4ad45c267fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:35.439272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d1471fc8-a0e8-43d8-b862-4ad45c267fc8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:35.440755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d1471fc8-a0e8-43d8-b862-4ad45c267fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:35.556398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:35.572033 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1520/3018] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:35.587282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:35.589384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:35.696909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df3d0-ba09-4b8e-93cd-767aab88e672 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:35.703560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97e898-5069-4c90-b0c4-8bd3fe862d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:35.704382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97e898-5069-4c90-b0c4-8bd3fe862d9e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:35.704667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97e898-5069-4c90-b0c4-8bd3fe862d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:35.704927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97e898-5069-4c90-b0c4-8bd3fe862d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:35.716454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97e898-5069-4c90-b0c4-8bd3fe862d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:35.718026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97e898-5069-4c90-b0c4-8bd3fe862d9e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:35.741404 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1499/3019] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:36.011587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1471fc8-a0e8-43d8-b862-4ad45c267fc8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:36.015301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d1471fc8-a0e8-43d8-b862-4ad45c267fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:36.031265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1471fc8-a0e8-43d8-b862-4ad45c267fc8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:36.037098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:36.038300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:36.038812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:36.039252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] PATH_INFO: `/v3/projects/ba05e0be960d4f33a04730891f16a44e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:36.041736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba05e0be960d4f33a04730891f16a44e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:36.047619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:36.169981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba05e0be960d4f33a04730891f16a44e', 'request_id': 'req-9f234953-a253-443c-8312-0fd67b2546c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:36.202897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba05e0be960d4f33a04730891f16a44e', 'request_id': 'req-9f234953-a253-443c-8312-0fd67b2546c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:36.211542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba05e0be960d4f33a04730891f16a44e', 'request_id': 'req-9f234953-a253-443c-8312-0fd67b2546c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:36.227206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] The token cache is being invalidate because project ba05e0be960d4f33a04730891f16a44e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:36.227670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:36.229070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:36.233906 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1521/3020] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:36 2022] DELETE /identity/v3/projects/ba05e0be960d4f33a04730891f16a44e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:36.238403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97e898-5069-4c90-b0c4-8bd3fe862d9e service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:36.258966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97e898-5069-4c90-b0c4-8bd3fe862d9e service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:36.348791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:36.380249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:36.381765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:36.383795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:36.384157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] PATH_INFO: `/v3/users/af16718ffcfa4424889c9d618914befc` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:36.387321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af16718ffcfa4424889c9d618914befc)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:36.391214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:36.485995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af16718ffcfa4424889c9d618914befc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:36.495550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af16718ffcfa4424889c9d618914befc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:36.512620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af16718ffcfa4424889c9d618914befc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:36.524994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af16718ffcfa4424889c9d618914befc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:36.526641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd63dd02-e4f3-41e5-9740-a2cb00f1f2a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af16718ffcfa4424889c9d618914befc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:36.551197 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1500/3021] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:36 2022] DELETE /identity/v3/users/af16718ffcfa4424889c9d618914befc => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:36.976525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:36.978337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:37.005365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f234953-a253-443c-8312-0fd67b2546c6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:37.095827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:37.097333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:37.097777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:37.098202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] PATH_INFO: `/v3/projects/b5a39a1482644346921682dfcfa3f990` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:37.099884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5a39a1482644346921682dfcfa3f990)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:37.102222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:37.173615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5a39a1482644346921682dfcfa3f990', 'request_id': 'req-16ce7513-2018-4636-aca7-a059a0280b9b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:37.187566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5a39a1482644346921682dfcfa3f990', 'request_id': 'req-16ce7513-2018-4636-aca7-a059a0280b9b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:37.198869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5a39a1482644346921682dfcfa3f990', 'request_id': 'req-16ce7513-2018-4636-aca7-a059a0280b9b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:37.213653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] The token cache is being invalidate because project b5a39a1482644346921682dfcfa3f990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:37.214004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:37.214922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16ce7513-2018-4636-aca7-a059a0280b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:37.217359 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1522/3022] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:36 2022] DELETE /identity/v3/projects/b5a39a1482644346921682dfcfa3f990 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:37.222666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec82d6c-cb57-48e3-9384-bd8ae7405b6b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:37.222965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec82d6c-cb57-48e3-9384-bd8ae7405b6b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:37.223949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec82d6c-cb57-48e3-9384-bd8ae7405b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:37.285163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1ec82d6c-cb57-48e3-9384-bd8ae7405b6b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:37.293117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1ec82d6c-cb57-48e3-9384-bd8ae7405b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:37.440547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210dce4-f2e7-4a39-8743-a2100b052edb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:37.440825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210dce4-f2e7-4a39-8743-a2100b052edb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:37.441119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210dce4-f2e7-4a39-8743-a2100b052edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:37.445719 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1501/3023] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:37.470286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c210dce4-f2e7-4a39-8743-a2100b052edb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:37.471738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c210dce4-f2e7-4a39-8743-a2100b052edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:37.499540 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1523/3024] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:57:37.500328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec82d6c-cb57-48e3-9384-bd8ae7405b6b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:37.514863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec82d6c-cb57-48e3-9384-bd8ae7405b6b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:37.520262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071e9a8-7849-4872-bd30-3c048717bfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:37.521526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071e9a8-7849-4872-bd30-3c048717bfd1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:37.521965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071e9a8-7849-4872-bd30-3c048717bfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:37.522912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071e9a8-7849-4872-bd30-3c048717bfd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:37.523762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071e9a8-7849-4872-bd30-3c048717bfd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:57:37.524213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071e9a8-7849-4872-bd30-3c048717bfd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:37.526083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071e9a8-7849-4872-bd30-3c048717bfd1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:37.548433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1502/3025] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:57:37.549397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210dce4-f2e7-4a39-8743-a2100b052edb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:37.561006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210dce4-f2e7-4a39-8743-a2100b052edb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:37.568205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab758f64-a4ae-446f-9ce8-ce12909c6309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:37.569020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab758f64-a4ae-446f-9ce8-ce12909c6309 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:37.569486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab758f64-a4ae-446f-9ce8-ce12909c6309 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:37.569897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab758f64-a4ae-446f-9ce8-ce12909c6309 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:37.570681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab758f64-a4ae-446f-9ce8-ce12909c6309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:37.572823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab758f64-a4ae-446f-9ce8-ce12909c6309 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:37.580935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:37.634701 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1524/3026] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:37 2022] POST /identity/v3/projects => generated 393 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:37.637146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071e9a8-7849-4872-bd30-3c048717bfd1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:37.654560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071e9a8-7849-4872-bd30-3c048717bfd1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:37.726267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:37.769202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552931f3-ee1d-4fb9-9716-27aefc104b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:37.769202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552931f3-ee1d-4fb9-9716-27aefc104b34 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:37.769202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552931f3-ee1d-4fb9-9716-27aefc104b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:37.769202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552931f3-ee1d-4fb9-9716-27aefc104b34 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:37.770952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552931f3-ee1d-4fb9-9716-27aefc104b34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:37.773848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552931f3-ee1d-4fb9-9716-27aefc104b34 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:37.837553 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1503/3027] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:37 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:37.842129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab758f64-a4ae-446f-9ce8-ce12909c6309 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:37.857993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab758f64-a4ae-446f-9ce8-ce12909c6309 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:37.865166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fe833-eeef-4052-acd5-3e9a15a4db3f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:37.865586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fe833-eeef-4052-acd5-3e9a15a4db3f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:37.865955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fe833-eeef-4052-acd5-3e9a15a4db3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:37.886899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee971af2-e9e6-4581-912a-2c0b521117f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:37.888396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee971af2-e9e6-4581-912a-2c0b521117f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:37.888802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee971af2-e9e6-4581-912a-2c0b521117f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:37.893098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee971af2-e9e6-4581-912a-2c0b521117f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:37.894255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee971af2-e9e6-4581-912a-2c0b521117f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:37.896388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee971af2-e9e6-4581-912a-2c0b521117f3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:37.919003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e32fe833-eeef-4052-acd5-3e9a15a4db3f None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:37.922650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e32fe833-eeef-4052-acd5-3e9a15a4db3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:37.926201 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1525/3028] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:37.932008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee971af2-e9e6-4581-912a-2c0b521117f3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:37.960343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee971af2-e9e6-4581-912a-2c0b521117f3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:37.979745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6db2ef-c919-4156-bf31-a684d773d1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:37.980612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6db2ef-c919-4156-bf31-a684d773d1bd admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:37.981021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6db2ef-c919-4156-bf31-a684d773d1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:37.981480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6db2ef-c919-4156-bf31-a684d773d1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:37.983734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6db2ef-c919-4156-bf31-a684d773d1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:37.990316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6db2ef-c919-4156-bf31-a684d773d1bd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:38.022239 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1526/3029] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:38.026528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6db2ef-c919-4156-bf31-a684d773d1bd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:38.072780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6db2ef-c919-4156-bf31-a684d773d1bd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:38.088992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249f3c4-e447-47ee-8774-43a775599b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:38.090370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249f3c4-e447-47ee-8774-43a775599b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:38.093226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249f3c4-e447-47ee-8774-43a775599b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:38.093765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249f3c4-e447-47ee-8774-43a775599b8f admin admin] PATH_INFO: `/v3/projects/c12effac060d4c23a3a3f0b4019a9bf5/users/d2fb28a271f343b6b21960f82c213406/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:38.144234 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1504/3030] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:38.145408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fe833-eeef-4052-acd5-3e9a15a4db3f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:38.147397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249f3c4-e447-47ee-8774-43a775599b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12effac060d4c23a3a3f0b4019a9bf5, user_id=d2fb28a271f343b6b21960f82c213406, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:38.147884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e32fe833-eeef-4052-acd5-3e9a15a4db3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:38.151799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249f3c4-e447-47ee-8774-43a775599b8f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:38.172716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fe833-eeef-4052-acd5-3e9a15a4db3f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:38.187345 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1527/3031] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:38 2022] PUT /identity/v3/projects/c12effac060d4c23a3a3f0b4019a9bf5/users/d2fb28a271f343b6b21960f82c213406/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:38.194848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249f3c4-e447-47ee-8774-43a775599b8f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:38.206839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a249f3c4-e447-47ee-8774-43a775599b8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:38.326938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:38.329671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:38.329947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:38.330900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] PATH_INFO: `/v3/users/353e4db513a94d96bd82b85742624fc2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:38.333600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=353e4db513a94d96bd82b85742624fc2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:38.337455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:38.418495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249f3c4-e447-47ee-8774-43a775599b8f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:38.428450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91396e77-a614-4f1e-b15e-45a3219491de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:38.429473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91396e77-a614-4f1e-b15e-45a3219491de service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:38.429889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91396e77-a614-4f1e-b15e-45a3219491de service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:38.430457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91396e77-a614-4f1e-b15e-45a3219491de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:38.485406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353e4db513a94d96bd82b85742624fc2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:38.513621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353e4db513a94d96bd82b85742624fc2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:38.545709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353e4db513a94d96bd82b85742624fc2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:38.564485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353e4db513a94d96bd82b85742624fc2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:38.565365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-95ea2a44-a8ef-4e8b-a66d-873b91d30b1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353e4db513a94d96bd82b85742624fc2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:38.590389 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1505/3032] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:38 2022] DELETE /identity/v3/users/353e4db513a94d96bd82b85742624fc2 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:38.594057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c3565-7774-4a32-9db4-b6a29485c6cc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:38.594554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c3565-7774-4a32-9db4-b6a29485c6cc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:38.595181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c3565-7774-4a32-9db4-b6a29485c6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:38.604334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:38.614699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91396e77-a614-4f1e-b15e-45a3219491de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:38.617517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91396e77-a614-4f1e-b15e-45a3219491de service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:38.630987 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1528/3033] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:57:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:57:38.632468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032e997-03a3-4f4e-a0fd-6794b286b69c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:38.632816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032e997-03a3-4f4e-a0fd-6794b286b69c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:38.632976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032e997-03a3-4f4e-a0fd-6794b286b69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:38.667184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e032e997-03a3-4f4e-a0fd-6794b286b69c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:38.668541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e032e997-03a3-4f4e-a0fd-6794b286b69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:38.722115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-de5c3565-7774-4a32-9db4-b6a29485c6cc None None] MFA Rules not processed for user `d2fb28a271f343b6b21960f82c213406`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:38.729837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de5c3565-7774-4a32-9db4-b6a29485c6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:38.734144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:38.786903 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1529/3034] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:38.824288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032e997-03a3-4f4e-a0fd-6794b286b69c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:38.826898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e032e997-03a3-4f4e-a0fd-6794b286b69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:38.915007 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1506/3035] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:38.919792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c3565-7774-4a32-9db4-b6a29485c6cc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:38.921244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de5c3565-7774-4a32-9db4-b6a29485c6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:38.948553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c3565-7774-4a32-9db4-b6a29485c6cc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:38.959856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b3ca0-36fa-4ecc-8568-503e5727bb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:38.960878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3ca0-36fa-4ecc-8568-503e5727bb0b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:38.961387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3ca0-36fa-4ecc-8568-503e5727bb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:38.961822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3ca0-36fa-4ecc-8568-503e5727bb0b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:38.963225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b3ca0-36fa-4ecc-8568-503e5727bb0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:38.966513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b3ca0-36fa-4ecc-8568-503e5727bb0b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:39.002283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032e997-03a3-4f4e-a0fd-6794b286b69c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:39.014041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b924bd6-e036-414c-86cc-9b700c7e44b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:39.016799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b924bd6-e036-414c-86cc-9b700c7e44b3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:39.017119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b924bd6-e036-414c-86cc-9b700c7e44b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:39.017492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b924bd6-e036-414c-86cc-9b700c7e44b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:39.045107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b924bd6-e036-414c-86cc-9b700c7e44b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:39.047809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b924bd6-e036-414c-86cc-9b700c7e44b3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:39.064694 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1507/3036] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:38 2022] POST /identity/v3/projects => generated 395 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:39.065138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b3ca0-36fa-4ecc-8568-503e5727bb0b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:39.082055 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1530/3037] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:57:39.206592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b3ca0-36fa-4ecc-8568-503e5727bb0b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:39.219710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e5f94-1838-45ef-b8bd-8b377e7aa0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:39.220327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e5f94-1838-45ef-b8bd-8b377e7aa0ef admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:39.220596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e5f94-1838-45ef-b8bd-8b377e7aa0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:39.220858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e5f94-1838-45ef-b8bd-8b377e7aa0ef admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:39.221516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e5f94-1838-45ef-b8bd-8b377e7aa0ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:39.223422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e5f94-1838-45ef-b8bd-8b377e7aa0ef admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:39.285937 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1508/3038] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:39 2022] POST /identity/v3/users => generated 304 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:39.290069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b924bd6-e036-414c-86cc-9b700c7e44b3 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:39.310962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b924bd6-e036-414c-86cc-9b700c7e44b3 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:39.338996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbac26-5e7b-40ec-92ae-6ac74c2039cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:39.349459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbac26-5e7b-40ec-92ae-6ac74c2039cf admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:39.349970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbac26-5e7b-40ec-92ae-6ac74c2039cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:39.350526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbac26-5e7b-40ec-92ae-6ac74c2039cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:39.351958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbac26-5e7b-40ec-92ae-6ac74c2039cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:39.353795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbac26-5e7b-40ec-92ae-6ac74c2039cf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:39.390754 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1531/3039] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:39.395373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e5f94-1838-45ef-b8bd-8b377e7aa0ef admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:39.415018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e5f94-1838-45ef-b8bd-8b377e7aa0ef admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:39.430304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6056517-48c5-4cb7-89aa-099725f5ddb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:39.431517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6056517-48c5-4cb7-89aa-099725f5ddb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:39.431517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6056517-48c5-4cb7-89aa-099725f5ddb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:39.431517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6056517-48c5-4cb7-89aa-099725f5ddb6 admin admin] PATH_INFO: `/v3/projects/de81a425b2714eabb2bc8924ddd45229/users/a7d0c0608e644d118d03fd76f5370d09/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:39.473701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6056517-48c5-4cb7-89aa-099725f5ddb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de81a425b2714eabb2bc8924ddd45229, user_id=a7d0c0608e644d118d03fd76f5370d09, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:39.476338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6056517-48c5-4cb7-89aa-099725f5ddb6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:39.512146 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1509/3040] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:39 2022] PUT /identity/v3/projects/de81a425b2714eabb2bc8924ddd45229/users/a7d0c0608e644d118d03fd76f5370d09/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:57:39.516756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbac26-5e7b-40ec-92ae-6ac74c2039cf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:39.531507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbac26-5e7b-40ec-92ae-6ac74c2039cf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:39.666977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848b2d4-2807-4c93-97c1-4c9e2891c835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:39.667703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848b2d4-2807-4c93-97c1-4c9e2891c835 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:39.668010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848b2d4-2807-4c93-97c1-4c9e2891c835 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:39.668460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848b2d4-2807-4c93-97c1-4c9e2891c835 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:39.669841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848b2d4-2807-4c93-97c1-4c9e2891c835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:39.671497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848b2d4-2807-4c93-97c1-4c9e2891c835 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:39.703576 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1532/3041] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:39.708035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6056517-48c5-4cb7-89aa-099725f5ddb6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:39.722926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6056517-48c5-4cb7-89aa-099725f5ddb6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:39.736670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fd17d-47b4-472b-8fc8-d27066de97bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:39.737540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fd17d-47b4-472b-8fc8-d27066de97bb admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:39.737807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fd17d-47b4-472b-8fc8-d27066de97bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:39.738075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fd17d-47b4-472b-8fc8-d27066de97bb admin admin] PATH_INFO: `/v3/projects/de81a425b2714eabb2bc8924ddd45229/users/a7d0c0608e644d118d03fd76f5370d09/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:39.750812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fd17d-47b4-472b-8fc8-d27066de97bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=de81a425b2714eabb2bc8924ddd45229, user_id=a7d0c0608e644d118d03fd76f5370d09, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:39.753482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fd17d-47b4-472b-8fc8-d27066de97bb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:39.780641 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1510/3042] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:39 2022] PUT /identity/v3/projects/de81a425b2714eabb2bc8924ddd45229/users/a7d0c0608e644d118d03fd76f5370d09/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:39.791064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3468adf-bf80-4581-b8d8-9327e2392d8d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:39.791340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3468adf-bf80-4581-b8d8-9327e2392d8d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:39.791610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3468adf-bf80-4581-b8d8-9327e2392d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:39.852421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f3468adf-bf80-4581-b8d8-9327e2392d8d None None] MFA Rules not processed for user `a7d0c0608e644d118d03fd76f5370d09`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:39.853734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f3468adf-bf80-4581-b8d8-9327e2392d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:39.933899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:39.948110 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1533/3043] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:39.963315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbf616-f662-4ee6-b197-667abeeaee5d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:39.964800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbf616-f662-4ee6-b197-667abeeaee5d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:39.965346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbf616-f662-4ee6-b197-667abeeaee5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:40.006148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a2bbf616-f662-4ee6-b197-667abeeaee5d None None] MFA Rules not processed for user `d2fb28a271f343b6b21960f82c213406`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:40.007476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a2bbf616-f662-4ee6-b197-667abeeaee5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:40.050375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:40.149762 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1511/3044] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:42.317960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3468adf-bf80-4581-b8d8-9327e2392d8d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:42.319416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f3468adf-bf80-4581-b8d8-9327e2392d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:42.502097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3468adf-bf80-4581-b8d8-9327e2392d8d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:42.529213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:42.530108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:42.530382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:42.530684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] PATH_INFO: `/v3/projects/2e241b6abf12428ba07a6f052bc00c70` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:42.540933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e241b6abf12428ba07a6f052bc00c70)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:42.542946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:42.666955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e241b6abf12428ba07a6f052bc00c70', 'request_id': 'req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:42.668847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbf616-f662-4ee6-b197-667abeeaee5d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:42.671965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a2bbf616-f662-4ee6-b197-667abeeaee5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:42.681350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e241b6abf12428ba07a6f052bc00c70', 'request_id': 'req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:42.693590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e241b6abf12428ba07a6f052bc00c70', 'request_id': 'req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:42.714934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] The token cache is being invalidate because project 2e241b6abf12428ba07a6f052bc00c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:42.715280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:42.737929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f79fe4b-3b80-4977-b47f-c14a1cf657ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:42.744610 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1534/3045] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:42 2022] DELETE /identity/v3/projects/2e241b6abf12428ba07a6f052bc00c70 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:42.751984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff9813-46e1-476e-a66b-3525d7d42501 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:42.752381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff9813-46e1-476e-a66b-3525d7d42501 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:42.752654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff9813-46e1-476e-a66b-3525d7d42501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:42.806695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-62ff9813-46e1-476e-a66b-3525d7d42501 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:42.809169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-62ff9813-46e1-476e-a66b-3525d7d42501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:42.864508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbf616-f662-4ee6-b197-667abeeaee5d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:42.983599 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1535/3046] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:57:43.000583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784c1e5-a574-42bc-b6b6-8f417493e9d5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:43.001297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784c1e5-a574-42bc-b6b6-8f417493e9d5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:43.002076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784c1e5-a574-42bc-b6b6-8f417493e9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:43.031062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7784c1e5-a574-42bc-b6b6-8f417493e9d5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:43.032858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7784c1e5-a574-42bc-b6b6-8f417493e9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:43.044772 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1536/3047] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:43.051593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784c1e5-a574-42bc-b6b6-8f417493e9d5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:43.084186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971ad50-2b46-4b07-a340-0d7bd9af9916 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:43.085088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971ad50-2b46-4b07-a340-0d7bd9af9916 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:43.085234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971ad50-2b46-4b07-a340-0d7bd9af9916 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:43.085637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971ad50-2b46-4b07-a340-0d7bd9af9916 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:43.093343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784c1e5-a574-42bc-b6b6-8f417493e9d5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:43.093502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7971ad50-2b46-4b07-a340-0d7bd9af9916 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:43.114537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc61986-4956-42c0-960b-313fe76a7f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:43.115239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc61986-4956-42c0-960b-313fe76a7f5f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:43.115961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc61986-4956-42c0-960b-313fe76a7f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:43.115961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc61986-4956-42c0-960b-313fe76a7f5f admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:43.116761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc61986-4956-42c0-960b-313fe76a7f5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:57:43.117068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc61986-4956-42c0-960b-313fe76a7f5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:43.119017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc61986-4956-42c0-960b-313fe76a7f5f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:43.161610 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1537/3048] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:57:43.167497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc61986-4956-42c0-960b-313fe76a7f5f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:43.179110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:43.192310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc61986-4956-42c0-960b-313fe76a7f5f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:43.214411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682c9c2-072b-4e5a-b2eb-4ab47fc25335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:43.215043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682c9c2-072b-4e5a-b2eb-4ab47fc25335 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:43.215380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682c9c2-072b-4e5a-b2eb-4ab47fc25335 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:43.215712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682c9c2-072b-4e5a-b2eb-4ab47fc25335 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:43.220046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682c9c2-072b-4e5a-b2eb-4ab47fc25335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:43.232765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682c9c2-072b-4e5a-b2eb-4ab47fc25335 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:43.299209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971ad50-2b46-4b07-a340-0d7bd9af9916 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:43.301222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971ad50-2b46-4b07-a340-0d7bd9af9916 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:43.313438 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1538/3049] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:43 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:43.317568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682c9c2-072b-4e5a-b2eb-4ab47fc25335 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:43.325993 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1512/3050] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:42 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:43.497732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682c9c2-072b-4e5a-b2eb-4ab47fc25335 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:43.558074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427268a7-4fe4-4480-93ce-12e3d115cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:43.559971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427268a7-4fe4-4480-93ce-12e3d115cd8c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:43.561451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427268a7-4fe4-4480-93ce-12e3d115cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:43.563272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427268a7-4fe4-4480-93ce-12e3d115cd8c admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:43.563941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427268a7-4fe4-4480-93ce-12e3d115cd8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:43.570333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427268a7-4fe4-4480-93ce-12e3d115cd8c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:43.660015 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1539/3051] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:43 2022] POST /identity/v3/users => generated 290 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:57:43.665264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971ad50-2b46-4b07-a340-0d7bd9af9916 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:43.671025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:43.685249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971ad50-2b46-4b07-a340-0d7bd9af9916 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:43.699318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12477dc-412e-4bbd-a649-fa10e37c8c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:43.699951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12477dc-412e-4bbd-a649-fa10e37c8c66 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:43.700747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12477dc-412e-4bbd-a649-fa10e37c8c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:43.701014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12477dc-412e-4bbd-a649-fa10e37c8c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:43.701737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12477dc-412e-4bbd-a649-fa10e37c8c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:43.703449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12477dc-412e-4bbd-a649-fa10e37c8c66 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:43.735642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2b3f9-a423-4ad8-b33a-129bb049f999 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:43.735972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2b3f9-a423-4ad8-b33a-129bb049f999 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:43.736185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2b3f9-a423-4ad8-b33a-129bb049f999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:43.770721 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1513/3052] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:43.776141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12477dc-412e-4bbd-a649-fa10e37c8c66 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:43.840082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12477dc-412e-4bbd-a649-fa10e37c8c66 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:43.845665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d2e2b3f9-a423-4ad8-b33a-129bb049f999 None None] MFA Rules not processed for user `79e244a9ce2343dbb903a4722da86eaf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:43.851604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d2e2b3f9-a423-4ad8-b33a-129bb049f999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:43.854577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f58d57-250b-41ad-abb9-de4ca39aba1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:43.855199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f58d57-250b-41ad-abb9-de4ca39aba1a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:43.855735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f58d57-250b-41ad-abb9-de4ca39aba1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:43.855735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f58d57-250b-41ad-abb9-de4ca39aba1a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:43.859774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f58d57-250b-41ad-abb9-de4ca39aba1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:43.864923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f58d57-250b-41ad-abb9-de4ca39aba1a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:43.896109 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1514/3053] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:43.898937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f58d57-250b-41ad-abb9-de4ca39aba1a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:43.936911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f58d57-250b-41ad-abb9-de4ca39aba1a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:43.954896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c518cf-5c99-49d0-90ba-b579f864549f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:43.956989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c518cf-5c99-49d0-90ba-b579f864549f admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:43.957155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c518cf-5c99-49d0-90ba-b579f864549f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:43.957345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c518cf-5c99-49d0-90ba-b579f864549f admin admin] PATH_INFO: `/v3/projects/82da41101c6149398c50e738633abcc9/users/83a2646b4fda43e49efe6f75bf377ee7/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:44.013353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c518cf-5c99-49d0-90ba-b579f864549f admin admin] RBAC: Authorizing `identity:create_grant(project_id=82da41101c6149398c50e738633abcc9, user_id=83a2646b4fda43e49efe6f75bf377ee7, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:44.019128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c518cf-5c99-49d0-90ba-b579f864549f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:44.055984 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1515/3054] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:43 2022] PUT /identity/v3/projects/82da41101c6149398c50e738633abcc9/users/83a2646b4fda43e49efe6f75bf377ee7/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:44.071471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f060b0-8776-48a3-a116-27afa25386b5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:44.071778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f060b0-8776-48a3-a116-27afa25386b5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:44.072983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f060b0-8776-48a3-a116-27afa25386b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:44.101790 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1540/3055] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:57:44.104858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115caa9f-282c-4fb7-a38c-c99e9fce511e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:44.107383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115caa9f-282c-4fb7-a38c-c99e9fce511e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:44.109996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115caa9f-282c-4fb7-a38c-c99e9fce511e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:44.140646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-115caa9f-282c-4fb7-a38c-c99e9fce511e None None] MFA Rules not processed for user `79e244a9ce2343dbb903a4722da86eaf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:44.142238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-115caa9f-282c-4fb7-a38c-c99e9fce511e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:44.157263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-62f060b0-8776-48a3-a116-27afa25386b5 None None] MFA Rules not processed for user `83a2646b4fda43e49efe6f75bf377ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:44.160016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-62f060b0-8776-48a3-a116-27afa25386b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:44.329486 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1516/3056] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:44.356012 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1541/3057] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:44 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:47.266534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f060b0-8776-48a3-a116-27afa25386b5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:47.268753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-62f060b0-8776-48a3-a116-27afa25386b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:47.432513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f060b0-8776-48a3-a116-27afa25386b5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:47.439365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c246-0b45-442f-80b5-ed349172e5c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:47.440252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c246-0b45-442f-80b5-ed349172e5c1 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:47.440589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c246-0b45-442f-80b5-ed349172e5c1 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:47.440928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c246-0b45-442f-80b5-ed349172e5c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:47.481192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370c246-0b45-442f-80b5-ed349172e5c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:47.483532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370c246-0b45-442f-80b5-ed349172e5c1 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:47.499676 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1517/3058] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:47.586871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115caa9f-282c-4fb7-a38c-c99e9fce511e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:47.594005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c246-0b45-442f-80b5-ed349172e5c1 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:47.607818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c246-0b45-442f-80b5-ed349172e5c1 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:47.609265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:47.624010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333115d-053b-4d68-b183-fe7b9a964f78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:47.624602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333115d-053b-4d68-b183-fe7b9a964f78 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:47.624602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333115d-053b-4d68-b183-fe7b9a964f78 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:47.624864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333115d-053b-4d68-b183-fe7b9a964f78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:47.652014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333115d-053b-4d68-b183-fe7b9a964f78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:47.655431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333115d-053b-4d68-b183-fe7b9a964f78 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:47.678248 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1518/3059] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:47 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:47.751651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115caa9f-282c-4fb7-a38c-c99e9fce511e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:47.757371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:47.758194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:47.758476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:47.758873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] PATH_INFO: `/v3/users/83a2646b4fda43e49efe6f75bf377ee7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:47.763004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=83a2646b4fda43e49efe6f75bf377ee7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:47.767657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:47.867126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83a2646b4fda43e49efe6f75bf377ee7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:47.885442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83a2646b4fda43e49efe6f75bf377ee7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:47.944520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83a2646b4fda43e49efe6f75bf377ee7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:47.967358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83a2646b4fda43e49efe6f75bf377ee7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:47.971899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83a2646b4fda43e49efe6f75bf377ee7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:48.023156 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1542/3060] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:57:47 2022] DELETE /identity/v3/users/83a2646b4fda43e49efe6f75bf377ee7 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:48.027709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86967a3-9e53-4b32-9db2-803501ae0f9c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:48.027856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86967a3-9e53-4b32-9db2-803501ae0f9c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:48.028082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86967a3-9e53-4b32-9db2-803501ae0f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:48.076727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c86967a3-9e53-4b32-9db2-803501ae0f9c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:48.078018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c86967a3-9e53-4b32-9db2-803501ae0f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:48.097809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:48.181950 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1519/3061] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:48.215659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:48.217349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:48.324367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65bbc2-0cb5-425b-bbfb-d29d9151a6ad admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:48.328654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d11b7-76e6-414d-b3aa-4c70895de0df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:48.329444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d11b7-76e6-414d-b3aa-4c70895de0df service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:48.329792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d11b7-76e6-414d-b3aa-4c70895de0df service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:48.330799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d11b7-76e6-414d-b3aa-4c70895de0df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:48.342897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d11b7-76e6-414d-b3aa-4c70895de0df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:48.344396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d11b7-76e6-414d-b3aa-4c70895de0df service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:48.358514 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1543/3062] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:48.973832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86967a3-9e53-4b32-9db2-803501ae0f9c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:48.976089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c86967a3-9e53-4b32-9db2-803501ae0f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:49.019110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86967a3-9e53-4b32-9db2-803501ae0f9c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:49.041724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:49.049437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:49.049850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:49.050162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] PATH_INFO: `/v3/projects/82da41101c6149398c50e738633abcc9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:49.052653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82da41101c6149398c50e738633abcc9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:49.056531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:49.213821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82da41101c6149398c50e738633abcc9', 'request_id': 'req-596e3564-b173-4c44-97fc-3d07678e74c5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:49.230891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82da41101c6149398c50e738633abcc9', 'request_id': 'req-596e3564-b173-4c44-97fc-3d07678e74c5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:49.259166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82da41101c6149398c50e738633abcc9', 'request_id': 'req-596e3564-b173-4c44-97fc-3d07678e74c5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:49.287702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] The token cache is being invalidate because project 82da41101c6149398c50e738633abcc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:57:49.288001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:49.288782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-596e3564-b173-4c44-97fc-3d07678e74c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:57:49.290813 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1520/3063] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:48 2022] DELETE /identity/v3/projects/82da41101c6149398c50e738633abcc9 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:49.295950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66e865-4272-4313-9558-320d77123845 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:49.296432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66e865-4272-4313-9558-320d77123845 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:49.296599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66e865-4272-4313-9558-320d77123845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:49.336196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4e66e865-4272-4313-9558-320d77123845 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:49.338330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4e66e865-4272-4313-9558-320d77123845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:49.387550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:49.470683 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1544/3064] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:49.478869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4b39-4bdd-4861-aee0-b263d5284a86 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:49.479142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4b39-4bdd-4861-aee0-b263d5284a86 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:49.479410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4b39-4bdd-4861-aee0-b263d5284a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:49.522356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ce4e4b39-4bdd-4861-aee0-b263d5284a86 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:49.524485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ce4e4b39-4bdd-4861-aee0-b263d5284a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:49.564908 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1521/3065] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:49.569799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66e865-4272-4313-9558-320d77123845 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:49.582604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66e865-4272-4313-9558-320d77123845 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:49.641487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b8f35-a822-452d-8c23-9974e3c99930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:49.645958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b8f35-a822-452d-8c23-9974e3c99930 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:49.646271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b8f35-a822-452d-8c23-9974e3c99930 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:49.646547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b8f35-a822-452d-8c23-9974e3c99930 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:49.651297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b8f35-a822-452d-8c23-9974e3c99930 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:57:49.651669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b8f35-a822-452d-8c23-9974e3c99930 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:49.657768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b8f35-a822-452d-8c23-9974e3c99930 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:49.689628 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1545/3066] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:57:49.693500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e4b39-4bdd-4861-aee0-b263d5284a86 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:49.707258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e4b39-4bdd-4861-aee0-b263d5284a86 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:49.712881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43eae4a-3844-4821-9576-7e9f0df2aca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:49.713582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43eae4a-3844-4821-9576-7e9f0df2aca5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:49.713925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43eae4a-3844-4821-9576-7e9f0df2aca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:49.714630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43eae4a-3844-4821-9576-7e9f0df2aca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:49.715338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43eae4a-3844-4821-9576-7e9f0df2aca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:49.720833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43eae4a-3844-4821-9576-7e9f0df2aca5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:49.815323 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1522/3067] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:57:49 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:49.819877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b8f35-a822-452d-8c23-9974e3c99930 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:49.847176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b8f35-a822-452d-8c23-9974e3c99930 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:50.029098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cda43d-a780-4ff5-b5c0-4c5bbc3f032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:50.029889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda43d-a780-4ff5-b5c0-4c5bbc3f032d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:50.030169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda43d-a780-4ff5-b5c0-4c5bbc3f032d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:50.030435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda43d-a780-4ff5-b5c0-4c5bbc3f032d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:50.031069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cda43d-a780-4ff5-b5c0-4c5bbc3f032d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:50.033092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cda43d-a780-4ff5-b5c0-4c5bbc3f032d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:50.078521 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1546/3068] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:57:49 2022] POST /identity/v3/users => generated 284 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:57:50.082458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43eae4a-3844-4821-9576-7e9f0df2aca5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:50.095867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43eae4a-3844-4821-9576-7e9f0df2aca5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:50.104864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28951a37-0d7e-4e15-b022-0e7e43ce0189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:50.106459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951a37-0d7e-4e15-b022-0e7e43ce0189 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:50.107456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951a37-0d7e-4e15-b022-0e7e43ce0189 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:50.107905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951a37-0d7e-4e15-b022-0e7e43ce0189 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:50.108733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28951a37-0d7e-4e15-b022-0e7e43ce0189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:50.113338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28951a37-0d7e-4e15-b022-0e7e43ce0189 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:50.129963 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1523/3069] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:50.133705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cda43d-a780-4ff5-b5c0-4c5bbc3f032d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:50.155089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cda43d-a780-4ff5-b5c0-4c5bbc3f032d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:50.178774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06db252-ceb8-4179-9798-e67a321822dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:50.179471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06db252-ceb8-4179-9798-e67a321822dc admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:50.179750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06db252-ceb8-4179-9798-e67a321822dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:50.180052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06db252-ceb8-4179-9798-e67a321822dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:50.180739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06db252-ceb8-4179-9798-e67a321822dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:50.182545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06db252-ceb8-4179-9798-e67a321822dc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:50.248667 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1547/3070] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:57:50.252870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28951a37-0d7e-4e15-b022-0e7e43ce0189 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:50.277605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28951a37-0d7e-4e15-b022-0e7e43ce0189 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:50.285644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca60b9b-6c2b-4de7-9763-5d3cbed4279f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:50.286639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca60b9b-6c2b-4de7-9763-5d3cbed4279f admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:50.287395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca60b9b-6c2b-4de7-9763-5d3cbed4279f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:50.287894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca60b9b-6c2b-4de7-9763-5d3cbed4279f admin admin] PATH_INFO: `/v3/projects/7e45e841218949e688b3deee07fd25ed/users/a8d437aff8aa486ca43edb2605a57b4e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:50.342314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca60b9b-6c2b-4de7-9763-5d3cbed4279f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e45e841218949e688b3deee07fd25ed, user_id=a8d437aff8aa486ca43edb2605a57b4e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:50.345030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca60b9b-6c2b-4de7-9763-5d3cbed4279f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:50.367078 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1524/3071] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:57:50 2022] PUT /identity/v3/projects/7e45e841218949e688b3deee07fd25ed/users/a8d437aff8aa486ca43edb2605a57b4e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:57:50.368773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8d104-a75f-48ea-9f1c-00e0e4fafff4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:50.369172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8d104-a75f-48ea-9f1c-00e0e4fafff4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:50.369322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8d104-a75f-48ea-9f1c-00e0e4fafff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:50.397546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-56a8d104-a75f-48ea-9f1c-00e0e4fafff4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:50.398952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-56a8d104-a75f-48ea-9f1c-00e0e4fafff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:50.488625 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1548/3072] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:57:50.501307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca60b9b-6c2b-4de7-9763-5d3cbed4279f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:57:50.503681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6ca60b9b-6c2b-4de7-9763-5d3cbed4279f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:50.580321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:50.603776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca60b9b-6c2b-4de7-9763-5d3cbed4279f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:57:50.609965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2dc59c-fc51-42f1-982d-7a41c8b32fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:57:50.611121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2dc59c-fc51-42f1-982d-7a41c8b32fbe service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:50.611599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2dc59c-fc51-42f1-982d-7a41c8b32fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:50.611940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2dc59c-fc51-42f1-982d-7a41c8b32fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:50.623755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2dc59c-fc51-42f1-982d-7a41c8b32fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:57:50.625647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2dc59c-fc51-42f1-982d-7a41c8b32fbe service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:57:50.642054 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1525/3073] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:52.593267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2bc0-806f-4c7d-8c0c-d50d4a62f5bb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:57:52.593802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2bc0-806f-4c7d-8c0c-d50d4a62f5bb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:57:52.593922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2bc0-806f-4c7d-8c0c-d50d4a62f5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:57:52.649751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d54f2bc0-806f-4c7d-8c0c-d50d4a62f5bb None None] MFA Rules not processed for user `6adaa12c2ca9469299994300bc856ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:57:52.651453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d54f2bc0-806f-4c7d-8c0c-d50d4a62f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:57:52.693608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:52.753820 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1549/3074] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:57:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:00.635575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=9 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:02.729361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:02.869957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb2c32-a5dd-4971-9fe8-9e2355498024 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:02.870489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb2c32-a5dd-4971-9fe8-9e2355498024 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:02.870714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb2c32-a5dd-4971-9fe8-9e2355498024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:02.912021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-dfdb2c32-a5dd-4971-9fe8-9e2355498024 None None] MFA Rules not processed for user `a8d437aff8aa486ca43edb2605a57b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:02.913736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dfdb2c32-a5dd-4971-9fe8-9e2355498024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:03.005355 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1526/3075] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:58:04.905487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f2bc0-806f-4c7d-8c0c-d50d4a62f5bb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:04.907040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d54f2bc0-806f-4c7d-8c0c-d50d4a62f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:04.984049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f2bc0-806f-4c7d-8c0c-d50d4a62f5bb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:04.989950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999a83c-c0fc-42e6-baf1-650dd3249cda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:04.990563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999a83c-c0fc-42e6-baf1-650dd3249cda service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:04.990836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999a83c-c0fc-42e6-baf1-650dd3249cda service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:04.991102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999a83c-c0fc-42e6-baf1-650dd3249cda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:04.992857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5999a83c-c0fc-42e6-baf1-650dd3249cda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:05.060283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999a83c-c0fc-42e6-baf1-650dd3249cda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:05.062658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999a83c-c0fc-42e6-baf1-650dd3249cda service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:05.077401 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1550/3076] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:05.345931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb2c32-a5dd-4971-9fe8-9e2355498024 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:05.359251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb2c32-a5dd-4971-9fe8-9e2355498024 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:05.365627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51e7e5-9216-4083-b983-fcac3a269eeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:05.366231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51e7e5-9216-4083-b983-fcac3a269eeb service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:05.366503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51e7e5-9216-4083-b983-fcac3a269eeb service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:05.366766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51e7e5-9216-4083-b983-fcac3a269eeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:05.375499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51e7e5-9216-4083-b983-fcac3a269eeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:05.377153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51e7e5-9216-4083-b983-fcac3a269eeb service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:05.386868 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1527/3077] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:58:07.080665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e1120-c4a5-4ffd-b662-ad10710253d4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:07.081278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e1120-c4a5-4ffd-b662-ad10710253d4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:07.082104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e1120-c4a5-4ffd-b662-ad10710253d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:07.104311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6a8e1120-c4a5-4ffd-b662-ad10710253d4 None None] MFA Rules not processed for user `a8d437aff8aa486ca43edb2605a57b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:07.105830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6a8e1120-c4a5-4ffd-b662-ad10710253d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:07.114148 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1551/3078] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:08.416541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51e7e5-9216-4083-b983-fcac3a269eeb service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:08.418031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0e51e7e5-9216-4083-b983-fcac3a269eeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:08.431165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51e7e5-9216-4083-b983-fcac3a269eeb service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:08.436118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36222f-c173-4486-b1ca-c87034c807a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:08.436962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36222f-c173-4486-b1ca-c87034c807a8 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:08.437281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36222f-c173-4486-b1ca-c87034c807a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:08.437622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36222f-c173-4486-b1ca-c87034c807a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:08.439297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cd36222f-c173-4486-b1ca-c87034c807a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:08.455163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36222f-c173-4486-b1ca-c87034c807a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:08.456598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36222f-c173-4486-b1ca-c87034c807a8 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:08.469265 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1528/3079] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:08.614169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60e7e4-50e7-4d5a-baf2-028c2ff819e5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:08.614668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60e7e4-50e7-4d5a-baf2-028c2ff819e5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:08.614764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60e7e4-50e7-4d5a-baf2-028c2ff819e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:08.634887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3e60e7e4-50e7-4d5a-baf2-028c2ff819e5 None None] MFA Rules not processed for user `500cba6141d34c4081d38a7b1be1775e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:08.636368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3e60e7e4-50e7-4d5a-baf2-028c2ff819e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:08.702174 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1552/3080] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:08.851854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faf3c1-3ef0-4000-b662-893e00f66729 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:08.852137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faf3c1-3ef0-4000-b662-893e00f66729 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:08.852403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faf3c1-3ef0-4000-b662-893e00f66729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:08.901887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-03faf3c1-3ef0-4000-b662-893e00f66729 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:08.903393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-03faf3c1-3ef0-4000-b662-893e00f66729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:08.971833 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1529/3081] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:58:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:08.986301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60e7e4-50e7-4d5a-baf2-028c2ff819e5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:08.987731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3e60e7e4-50e7-4d5a-baf2-028c2ff819e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:09.021873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:09.081346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60e7e4-50e7-4d5a-baf2-028c2ff819e5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:09.086404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5cda3-621b-4e62-821d-538d2c917d31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:09.086999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5cda3-621b-4e62-821d-538d2c917d31 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:09.087275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5cda3-621b-4e62-821d-538d2c917d31 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:09.087543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5cda3-621b-4e62-821d-538d2c917d31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:09.095632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5cda3-621b-4e62-821d-538d2c917d31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:09.097246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5cda3-621b-4e62-821d-538d2c917d31 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:09.105282 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1553/3082] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:58:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:09.940308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03faf3c1-3ef0-4000-b662-893e00f66729 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:09.941782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-03faf3c1-3ef0-4000-b662-893e00f66729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:09.986599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:10.025340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03faf3c1-3ef0-4000-b662-893e00f66729 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:10.031432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41482a-6478-4b17-a68c-aec444c31684 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:10.032132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41482a-6478-4b17-a68c-aec444c31684 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:10.032406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41482a-6478-4b17-a68c-aec444c31684 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:10.032694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41482a-6478-4b17-a68c-aec444c31684 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:10.034486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6a41482a-6478-4b17-a68c-aec444c31684 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:10.046718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41482a-6478-4b17-a68c-aec444c31684 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:10.048271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41482a-6478-4b17-a68c-aec444c31684 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:10.058107 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1530/3083] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:10.617942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5cda3-621b-4e62-821d-538d2c917d31 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:10.628395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5cda3-621b-4e62-821d-538d2c917d31 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:10.632558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57056f0-05eb-4ff9-9884-27df0ed7ca09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:10.633186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57056f0-05eb-4ff9-9884-27df0ed7ca09 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:10.633460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57056f0-05eb-4ff9-9884-27df0ed7ca09 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:10.633722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57056f0-05eb-4ff9-9884-27df0ed7ca09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:10.635498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d57056f0-05eb-4ff9-9884-27df0ed7ca09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:10.646016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57056f0-05eb-4ff9-9884-27df0ed7ca09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:10.648055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57056f0-05eb-4ff9-9884-27df0ed7ca09 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:10.657151 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1554/3084] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:11.269101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41482a-6478-4b17-a68c-aec444c31684 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:11.271645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6a41482a-6478-4b17-a68c-aec444c31684 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:11.287826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41482a-6478-4b17-a68c-aec444c31684 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:11.293291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65741c4-0be0-42f8-a6d3-aba3ad42a2c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:11.294141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65741c4-0be0-42f8-a6d3-aba3ad42a2c5 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:11.294418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65741c4-0be0-42f8-a6d3-aba3ad42a2c5 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:11.294684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65741c4-0be0-42f8-a6d3-aba3ad42a2c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:11.296646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f65741c4-0be0-42f8-a6d3-aba3ad42a2c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:11.308761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65741c4-0be0-42f8-a6d3-aba3ad42a2c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:11.310545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65741c4-0be0-42f8-a6d3-aba3ad42a2c5 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:11.323466 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1531/3085] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:14.375958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316990e8-3a2a-4d99-abfe-2b163796d834 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:14.376469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316990e8-3a2a-4d99-abfe-2b163796d834 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:14.376469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316990e8-3a2a-4d99-abfe-2b163796d834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:14.399363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-316990e8-3a2a-4d99-abfe-2b163796d834 None None] MFA Rules not processed for user `2bba6c9b3f674959ad79d4a394ed9f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:14.400587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-316990e8-3a2a-4d99-abfe-2b163796d834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:14.498589 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1555/3086] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:21.317317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=22 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:23.970684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaba835-0dbd-4e2d-9e2e-044b333ebd80 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:23.971628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaba835-0dbd-4e2d-9e2e-044b333ebd80 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:23.972210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaba835-0dbd-4e2d-9e2e-044b333ebd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:23.998810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3eaba835-0dbd-4e2d-9e2e-044b333ebd80 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:24.000297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3eaba835-0dbd-4e2d-9e2e-044b333ebd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:24.009877 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1532/3087] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:58:24.024190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316990e8-3a2a-4d99-abfe-2b163796d834 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:24.042511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316990e8-3a2a-4d99-abfe-2b163796d834 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:24.049567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46194f64-2a77-4d9d-aa78-9ca4de48bb3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:24.051279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46194f64-2a77-4d9d-aa78-9ca4de48bb3c service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:24.051740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46194f64-2a77-4d9d-aa78-9ca4de48bb3c service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:24.053027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46194f64-2a77-4d9d-aa78-9ca4de48bb3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:24.065351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46194f64-2a77-4d9d-aa78-9ca4de48bb3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:24.068017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46194f64-2a77-4d9d-aa78-9ca4de48bb3c service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:24.088494 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1556/3088] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:58:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:24.987769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaba835-0dbd-4e2d-9e2e-044b333ebd80 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:24.988417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3eaba835-0dbd-4e2d-9e2e-044b333ebd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:25.006177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaba835-0dbd-4e2d-9e2e-044b333ebd80 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:25.015250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:25.016154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:25.016504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:25.016845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] PATH_INFO: `/v3/users/d2fb28a271f343b6b21960f82c213406` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:25.018094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2fb28a271f343b6b21960f82c213406)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:25.020077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:25.116688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2fb28a271f343b6b21960f82c213406'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:25.126215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2fb28a271f343b6b21960f82c213406'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:25.144427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2fb28a271f343b6b21960f82c213406'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:25.157187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2fb28a271f343b6b21960f82c213406'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:25.157890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2fb28a271f343b6b21960f82c213406'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:25.168146 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1533/3089] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:58:24 2022] DELETE /identity/v3/users/d2fb28a271f343b6b21960f82c213406 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:25.170565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989844-6d18-4f47-986f-5f024f024b55 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:25.171368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989844-6d18-4f47-986f-5f024f024b55 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:25.171783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989844-6d18-4f47-986f-5f024f024b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:25.207303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9d989844-6d18-4f47-986f-5f024f024b55 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:25.208689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9d989844-6d18-4f47-986f-5f024f024b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:25.304060 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1557/3090] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:25.316891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:25.318364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:25.417307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147be979-8a01-4351-96e4-a0973428319b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:25.423497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140463cb-c54f-4080-a3f3-e001fb436097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:25.424129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140463cb-c54f-4080-a3f3-e001fb436097 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:25.424402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140463cb-c54f-4080-a3f3-e001fb436097 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:25.424671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140463cb-c54f-4080-a3f3-e001fb436097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:25.434730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140463cb-c54f-4080-a3f3-e001fb436097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:25.436146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140463cb-c54f-4080-a3f3-e001fb436097 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:25.448555 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1534/3091] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:25.750109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d989844-6d18-4f47-986f-5f024f024b55 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:25.752283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9d989844-6d18-4f47-986f-5f024f024b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:25.772060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d989844-6d18-4f47-986f-5f024f024b55 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:25.780315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:25.781092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:25.781642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:25.782105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] PATH_INFO: `/v3/projects/c12effac060d4c23a3a3f0b4019a9bf5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:25.783948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12effac060d4c23a3a3f0b4019a9bf5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:25.786298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:25.827494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:25.895952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12effac060d4c23a3a3f0b4019a9bf5', 'request_id': 'req-c340b221-958f-4e67-8374-b9e424bbbc6d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:25.911910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12effac060d4c23a3a3f0b4019a9bf5', 'request_id': 'req-c340b221-958f-4e67-8374-b9e424bbbc6d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:25.922103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12effac060d4c23a3a3f0b4019a9bf5', 'request_id': 'req-c340b221-958f-4e67-8374-b9e424bbbc6d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:25.937779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] The token cache is being invalidate because project c12effac060d4c23a3a3f0b4019a9bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:58:25.938227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:25.939440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:25.941880 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1558/3092] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:25 2022] DELETE /identity/v3/projects/c12effac060d4c23a3a3f0b4019a9bf5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:25.945529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140463cb-c54f-4080-a3f3-e001fb436097 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:25.958786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140463cb-c54f-4080-a3f3-e001fb436097 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:26.051049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:26.051980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:26.052264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:26.052535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] PATH_INFO: `/v3/users/a7d0c0608e644d118d03fd76f5370d09` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:26.054628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d0c0608e644d118d03fd76f5370d09)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:26.056559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:26.077896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:26.145264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d0c0608e644d118d03fd76f5370d09'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:26.159362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d0c0608e644d118d03fd76f5370d09'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:26.194014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d0c0608e644d118d03fd76f5370d09'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:26.221640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d0c0608e644d118d03fd76f5370d09'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:26.222390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7d3a6d77-3e33-4e74-8d58-af5ec9be59cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d0c0608e644d118d03fd76f5370d09'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:26.234293 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1535/3093] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:58:25 2022] DELETE /identity/v3/users/a7d0c0608e644d118d03fd76f5370d09 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:26.637111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:26.642703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:26.667042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340b221-958f-4e67-8374-b9e424bbbc6d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:26.752335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:26.753927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:26.754260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:26.754780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] PATH_INFO: `/v3/projects/de81a425b2714eabb2bc8924ddd45229` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:26.755992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de81a425b2714eabb2bc8924ddd45229)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:26.758863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:26.844301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de81a425b2714eabb2bc8924ddd45229', 'request_id': 'req-aee5fc23-d070-472a-8706-3b7f1db50943'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:26.860980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de81a425b2714eabb2bc8924ddd45229', 'request_id': 'req-aee5fc23-d070-472a-8706-3b7f1db50943'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:26.875086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de81a425b2714eabb2bc8924ddd45229', 'request_id': 'req-aee5fc23-d070-472a-8706-3b7f1db50943'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:26.890124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] The token cache is being invalidate because project de81a425b2714eabb2bc8924ddd45229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:58:26.894400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:26.896113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aee5fc23-d070-472a-8706-3b7f1db50943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:26.897947 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1559/3094] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:26 2022] DELETE /identity/v3/projects/de81a425b2714eabb2bc8924ddd45229 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:26.903010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622d5f2-71ab-4d96-a300-6b7587674b7e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:26.903295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622d5f2-71ab-4d96-a300-6b7587674b7e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:26.903561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622d5f2-71ab-4d96-a300-6b7587674b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:26.938970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9622d5f2-71ab-4d96-a300-6b7587674b7e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:26.940392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9622d5f2-71ab-4d96-a300-6b7587674b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:27.023882 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1536/3095] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:27.025636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4a28d-80e4-40ad-bf15-d6a37e1f297f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:27.025972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4a28d-80e4-40ad-bf15-d6a37e1f297f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:27.026247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4a28d-80e4-40ad-bf15-d6a37e1f297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:27.056046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bbd4a28d-80e4-40ad-bf15-d6a37e1f297f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:27.057215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bbd4a28d-80e4-40ad-bf15-d6a37e1f297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:27.069301 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1560/3096] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:27.075146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622d5f2-71ab-4d96-a300-6b7587674b7e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:27.088091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622d5f2-71ab-4d96-a300-6b7587674b7e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:27.098492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a46e4d-cd1b-4757-8194-a5f68fa1769a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:27.099203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a46e4d-cd1b-4757-8194-a5f68fa1769a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:27.099546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a46e4d-cd1b-4757-8194-a5f68fa1769a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:27.099886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a46e4d-cd1b-4757-8194-a5f68fa1769a admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:27.100626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a46e4d-cd1b-4757-8194-a5f68fa1769a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:58:27.100994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a46e4d-cd1b-4757-8194-a5f68fa1769a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:27.103478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a46e4d-cd1b-4757-8194-a5f68fa1769a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:27.125699 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1537/3097] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:58:27.126436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4a28d-80e4-40ad-bf15-d6a37e1f297f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:27.136202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4a28d-80e4-40ad-bf15-d6a37e1f297f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:27.143100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46895a90-093d-4e35-9a75-993f59a77914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:27.143993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46895a90-093d-4e35-9a75-993f59a77914 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:27.144525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46895a90-093d-4e35-9a75-993f59a77914 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:27.145112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46895a90-093d-4e35-9a75-993f59a77914 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:27.146410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46895a90-093d-4e35-9a75-993f59a77914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:27.150032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46895a90-093d-4e35-9a75-993f59a77914 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:27.184426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:27.208690 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1561/3098] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:58:27 2022] POST /identity/v3/projects => generated 403 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:58:27.213117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a46e4d-cd1b-4757-8194-a5f68fa1769a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:27.228650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a46e4d-cd1b-4757-8194-a5f68fa1769a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:27.377782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91c8af-3087-47ac-9ae3-72ef58412e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:27.379302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c8af-3087-47ac-9ae3-72ef58412e7e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:27.379582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c8af-3087-47ac-9ae3-72ef58412e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:27.379873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c8af-3087-47ac-9ae3-72ef58412e7e admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:27.380501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91c8af-3087-47ac-9ae3-72ef58412e7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:27.386671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91c8af-3087-47ac-9ae3-72ef58412e7e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:27.437550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1538/3099] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:58:27 2022] POST /identity/v3/users => generated 302 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:58:27.441849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46895a90-093d-4e35-9a75-993f59a77914 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:27.456834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46895a90-093d-4e35-9a75-993f59a77914 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:27.463384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448d0dc-52e4-4a91-b0a3-77c95f7dc57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:27.464854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448d0dc-52e4-4a91-b0a3-77c95f7dc57c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:27.465342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448d0dc-52e4-4a91-b0a3-77c95f7dc57c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:27.465775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448d0dc-52e4-4a91-b0a3-77c95f7dc57c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:27.467199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448d0dc-52e4-4a91-b0a3-77c95f7dc57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:27.469256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448d0dc-52e4-4a91-b0a3-77c95f7dc57c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:27.486401 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1562/3100] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:58:27.490307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91c8af-3087-47ac-9ae3-72ef58412e7e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:27.497264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:27.503628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91c8af-3087-47ac-9ae3-72ef58412e7e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:27.517174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6359a2d-b75b-47c2-9f8e-361949644eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:27.517862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6359a2d-b75b-47c2-9f8e-361949644eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:27.518147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6359a2d-b75b-47c2-9f8e-361949644eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:27.519042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6359a2d-b75b-47c2-9f8e-361949644eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:27.521087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6359a2d-b75b-47c2-9f8e-361949644eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:27.527199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6359a2d-b75b-47c2-9f8e-361949644eb2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:27.551856 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1539/3101] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:58:27.556700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448d0dc-52e4-4a91-b0a3-77c95f7dc57c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:27.567840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448d0dc-52e4-4a91-b0a3-77c95f7dc57c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:27.573246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918913b-f8b2-44db-a12e-99865a96d687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:27.573831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918913b-f8b2-44db-a12e-99865a96d687 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:27.574104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918913b-f8b2-44db-a12e-99865a96d687 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:27.574371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918913b-f8b2-44db-a12e-99865a96d687 admin admin] PATH_INFO: `/v3/projects/bccddd29eb9d4df692e1599e63daffc5/users/eb0650ccb0ab4d17b795db9e18a6a160/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:27.597652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918913b-f8b2-44db-a12e-99865a96d687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccddd29eb9d4df692e1599e63daffc5, user_id=eb0650ccb0ab4d17b795db9e18a6a160, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:27.602599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918913b-f8b2-44db-a12e-99865a96d687 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:27.621554 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1563/3102] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:58:27 2022] PUT /identity/v3/projects/bccddd29eb9d4df692e1599e63daffc5/users/eb0650ccb0ab4d17b795db9e18a6a160/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:27.627238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703935a-2fd4-4122-94cf-a140c697da9a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:27.627889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703935a-2fd4-4122-94cf-a140c697da9a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:27.628342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703935a-2fd4-4122-94cf-a140c697da9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:27.659380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1703935a-2fd4-4122-94cf-a140c697da9a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:27.661378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1703935a-2fd4-4122-94cf-a140c697da9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:27.766163 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1540/3103] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:58:27.775628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918913b-f8b2-44db-a12e-99865a96d687 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:27.778713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b918913b-f8b2-44db-a12e-99865a96d687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:27.864301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918913b-f8b2-44db-a12e-99865a96d687 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:27.877147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d8394-07d0-4e19-b244-2da3734c7795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:27.878209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d8394-07d0-4e19-b244-2da3734c7795 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:27.878327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d8394-07d0-4e19-b244-2da3734c7795 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:27.878553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d8394-07d0-4e19-b244-2da3734c7795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:27.904120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d8394-07d0-4e19-b244-2da3734c7795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:27.904120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d8394-07d0-4e19-b244-2da3734c7795 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:27.916616 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1564/3104] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:29.879741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703935a-2fd4-4122-94cf-a140c697da9a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:29.881258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1703935a-2fd4-4122-94cf-a140c697da9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:29.901009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703935a-2fd4-4122-94cf-a140c697da9a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:29.907539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb205440-c396-4c7a-a5b6-3191d32a446e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:29.908673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb205440-c396-4c7a-a5b6-3191d32a446e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:29.909300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb205440-c396-4c7a-a5b6-3191d32a446e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:29.909853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb205440-c396-4c7a-a5b6-3191d32a446e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:29.912125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eb205440-c396-4c7a-a5b6-3191d32a446e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:29.931626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb205440-c396-4c7a-a5b6-3191d32a446e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:29.937723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb205440-c396-4c7a-a5b6-3191d32a446e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:29.950550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1541/3105] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:33.077377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d8394-07d0-4e19-b244-2da3734c7795 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:33.079257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2f5d8394-07d0-4e19-b244-2da3734c7795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:33.248389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d8394-07d0-4e19-b244-2da3734c7795 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:33.254467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143159e-04f7-4bc4-93f7-e6b5a048099a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:33.255085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143159e-04f7-4bc4-93f7-e6b5a048099a service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:33.255360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143159e-04f7-4bc4-93f7-e6b5a048099a service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:33.255633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143159e-04f7-4bc4-93f7-e6b5a048099a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:33.259574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5143159e-04f7-4bc4-93f7-e6b5a048099a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:33.285009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143159e-04f7-4bc4-93f7-e6b5a048099a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:33.286937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143159e-04f7-4bc4-93f7-e6b5a048099a service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:33.306854 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1565/3106] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:58:34.284753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb205440-c396-4c7a-a5b6-3191d32a446e service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:34.304314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb205440-c396-4c7a-a5b6-3191d32a446e service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:34.314663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9179d-282a-4c61-af14-5c0a3e3bdbe5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:34.315667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9179d-282a-4c61-af14-5c0a3e3bdbe5 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:34.316163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9179d-282a-4c61-af14-5c0a3e3bdbe5 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:34.316755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9179d-282a-4c61-af14-5c0a3e3bdbe5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:34.318957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cab9179d-282a-4c61-af14-5c0a3e3bdbe5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:34.337807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9179d-282a-4c61-af14-5c0a3e3bdbe5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:34.343390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9179d-282a-4c61-af14-5c0a3e3bdbe5 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:34.364859 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1542/3107] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:35.299672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8c68-1514-4f28-8ee7-2c389adc6f19 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:35.299672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8c68-1514-4f28-8ee7-2c389adc6f19 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:35.300930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8c68-1514-4f28-8ee7-2c389adc6f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:35.357511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-653a8c68-1514-4f28-8ee7-2c389adc6f19 None None] MFA Rules not processed for user `6adaa12c2ca9469299994300bc856ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:35.359097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-653a8c68-1514-4f28-8ee7-2c389adc6f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:35.436129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:35.451507 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1566/3108] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:40.020268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9179d-282a-4c61-af14-5c0a3e3bdbe5 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:40.046730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9179d-282a-4c61-af14-5c0a3e3bdbe5 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:40.079177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a7042-8836-43f7-855a-369b87c51c33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:40.080264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a7042-8836-43f7-855a-369b87c51c33 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:40.081511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a7042-8836-43f7-855a-369b87c51c33 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:40.082079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a7042-8836-43f7-855a-369b87c51c33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:40.084615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b65a7042-8836-43f7-855a-369b87c51c33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:40.100991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a7042-8836-43f7-855a-369b87c51c33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:40.109805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a7042-8836-43f7-855a-369b87c51c33 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:40.134888 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1543/3109] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:41.254157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582e7e-5048-4782-940c-747047dbd726 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:41.254666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582e7e-5048-4782-940c-747047dbd726 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:41.254767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582e7e-5048-4782-940c-747047dbd726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:41.292055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c0582e7e-5048-4782-940c-747047dbd726 None None] MFA Rules not processed for user `eb0650ccb0ab4d17b795db9e18a6a160`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:41.293749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c0582e7e-5048-4782-940c-747047dbd726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:41.441544 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1567/3110] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:41 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:41.617326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a7042-8836-43f7-855a-369b87c51c33 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:41.634525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a7042-8836-43f7-855a-369b87c51c33 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:41.643763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7985067-36b5-42f3-a30a-eaa39a046da2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:41.645712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7985067-36b5-42f3-a30a-eaa39a046da2 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:41.646246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7985067-36b5-42f3-a30a-eaa39a046da2 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:41.646692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7985067-36b5-42f3-a30a-eaa39a046da2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:41.659803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7985067-36b5-42f3-a30a-eaa39a046da2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:41.662235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7985067-36b5-42f3-a30a-eaa39a046da2 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:41.677136 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1544/3111] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:41 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:42.868512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1389389-1931-4ed0-a5b9-1dfd2df52f00 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:42.869077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1389389-1931-4ed0-a5b9-1dfd2df52f00 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:42.869821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1389389-1931-4ed0-a5b9-1dfd2df52f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:42.897317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b1389389-1931-4ed0-a5b9-1dfd2df52f00 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:42.899790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b1389389-1931-4ed0-a5b9-1dfd2df52f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:43.009695 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1568/3112] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:43.017977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7985067-36b5-42f3-a30a-eaa39a046da2 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:43.020264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c7985067-36b5-42f3-a30a-eaa39a046da2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:43.195716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7985067-36b5-42f3-a30a-eaa39a046da2 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:43.219325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d8533-e2c8-4d0b-982e-422fc5f0b882 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:43.220175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d8533-e2c8-4d0b-982e-422fc5f0b882 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:43.220509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d8533-e2c8-4d0b-982e-422fc5f0b882 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:43.221138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d8533-e2c8-4d0b-982e-422fc5f0b882 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:43.235711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d8533-e2c8-4d0b-982e-422fc5f0b882 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:43.239984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d8533-e2c8-4d0b-982e-422fc5f0b882 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:43.261105 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1545/3113] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:58:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:46.237387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26313b-2aa2-4f8d-948f-79c770417ed7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:46.238619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26313b-2aa2-4f8d-948f-79c770417ed7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:46.240312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26313b-2aa2-4f8d-948f-79c770417ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:46.301246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fe26313b-2aa2-4f8d-948f-79c770417ed7 None None] MFA Rules not processed for user `5d93a680570f4d23b1f6df534bc082b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:46.302495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fe26313b-2aa2-4f8d-948f-79c770417ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:46.459648 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1569/3114] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:46.875254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d8533-e2c8-4d0b-982e-422fc5f0b882 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:46.876778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b63d8533-e2c8-4d0b-982e-422fc5f0b882 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:46.900336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d8533-e2c8-4d0b-982e-422fc5f0b882 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:46.906194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:46.907226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:46.908101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:46.908781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] PATH_INFO: `/v3/users/79e244a9ce2343dbb903a4722da86eaf` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:46.910855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e244a9ce2343dbb903a4722da86eaf)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:46.913574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:46.959703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:47.014645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e244a9ce2343dbb903a4722da86eaf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.031331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e244a9ce2343dbb903a4722da86eaf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.057853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e244a9ce2343dbb903a4722da86eaf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.073318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e244a9ce2343dbb903a4722da86eaf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.074303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e244a9ce2343dbb903a4722da86eaf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.083292 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1546/3115] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:58:46 2022] DELETE /identity/v3/users/79e244a9ce2343dbb903a4722da86eaf => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:47.166731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26313b-2aa2-4f8d-948f-79c770417ed7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:47.169268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fe26313b-2aa2-4f8d-948f-79c770417ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:47.208608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26313b-2aa2-4f8d-948f-79c770417ed7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:47.283791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:47.362785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:47.363982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:47.364489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:47.364977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] PATH_INFO: `/v3/users/78d679a10c9e4bf2bc35786235aeaf2a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:47.366459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] RBAC: Authorizing `identity:delete_user(user_id=78d679a10c9e4bf2bc35786235aeaf2a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:47.368842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:47.417941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:47.421226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:47.449515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78d679a10c9e4bf2bc35786235aeaf2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.455186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf9561-c58f-420b-86b0-e347ca4028ef admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:47.470624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78d679a10c9e4bf2bc35786235aeaf2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.474710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:47.475816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:47.476198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:47.477197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] PATH_INFO: `/v3/projects/974bd50eb06e453e9bb20218b9635555` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:47.478812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] RBAC: Authorizing `identity:delete_project(project_id=974bd50eb06e453e9bb20218b9635555)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:47.481291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:47.489190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78d679a10c9e4bf2bc35786235aeaf2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.509490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78d679a10c9e4bf2bc35786235aeaf2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.511825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8cd74e49-26e8-4f27-a38f-728aa56e893d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78d679a10c9e4bf2bc35786235aeaf2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.526065 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1570/3116] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:58:47 2022] DELETE /identity/v3/users/78d679a10c9e4bf2bc35786235aeaf2a => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:47.588204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '974bd50eb06e453e9bb20218b9635555', 'request_id': 'req-d25c584b-bc51-41fe-86a1-3454bd44568d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.606422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '974bd50eb06e453e9bb20218b9635555', 'request_id': 'req-d25c584b-bc51-41fe-86a1-3454bd44568d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.615395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '974bd50eb06e453e9bb20218b9635555', 'request_id': 'req-d25c584b-bc51-41fe-86a1-3454bd44568d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.628541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] The token cache is being invalidate because project 974bd50eb06e453e9bb20218b9635555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:58:47.629583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.630081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d25c584b-bc51-41fe-86a1-3454bd44568d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.632060 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1547/3117] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:47 2022] DELETE /identity/v3/projects/974bd50eb06e453e9bb20218b9635555 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:58:47.638357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc6141-ba2a-428c-88f4-692fd8b11da4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:47.638357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc6141-ba2a-428c-88f4-692fd8b11da4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:47.638357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc6141-ba2a-428c-88f4-692fd8b11da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:47.660400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-abfc6141-ba2a-428c-88f4-692fd8b11da4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:47.661631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-abfc6141-ba2a-428c-88f4-692fd8b11da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:47.745094 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1571/3118] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:47.749809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ec5d-3de8-40ed-bda4-ccb0963cf53a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:47.750300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ec5d-3de8-40ed-bda4-ccb0963cf53a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:47.752046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ec5d-3de8-40ed-bda4-ccb0963cf53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:47.773005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc6141-ba2a-428c-88f4-692fd8b11da4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:47.776051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-abfc6141-ba2a-428c-88f4-692fd8b11da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:47.794819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9aa6ec5d-3de8-40ed-bda4-ccb0963cf53a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:47.795193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc6141-ba2a-428c-88f4-692fd8b11da4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:47.796484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9aa6ec5d-3de8-40ed-bda4-ccb0963cf53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:47.803691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:47.804430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:47.804916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:47.805279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] PATH_INFO: `/v3/projects/11bdafc438884621944db399da5528f8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:47.806541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=11bdafc438884621944db399da5528f8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:47.810181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:47.810353 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1548/3119] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:58:47.817148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6ec5d-3de8-40ed-bda4-ccb0963cf53a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:47.844830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6ec5d-3de8-40ed-bda4-ccb0963cf53a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:47.853982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55115db3-3eac-460e-ab45-c68321b8a640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:47.854782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115db3-3eac-460e-ab45-c68321b8a640 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:47.854873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115db3-3eac-460e-ab45-c68321b8a640 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:47.855142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115db3-3eac-460e-ab45-c68321b8a640 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:47.856210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55115db3-3eac-460e-ab45-c68321b8a640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:58:47.856498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55115db3-3eac-460e-ab45-c68321b8a640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:47.859199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55115db3-3eac-460e-ab45-c68321b8a640 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:47.882982 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1549/3120] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:58:47.883903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55115db3-3eac-460e-ab45-c68321b8a640 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:47.892754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:47.894727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11bdafc438884621944db399da5528f8', 'request_id': 'req-64476e67-8586-4d99-9199-438bf378ee1d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.902297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55115db3-3eac-460e-ab45-c68321b8a640 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:47.915140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11bdafc438884621944db399da5528f8', 'request_id': 'req-64476e67-8586-4d99-9199-438bf378ee1d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.929156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11bdafc438884621944db399da5528f8', 'request_id': 'req-64476e67-8586-4d99-9199-438bf378ee1d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.952787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] The token cache is being invalidate because project 11bdafc438884621944db399da5528f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:58:47.952991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.954731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64476e67-8586-4d99-9199-438bf378ee1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:47.956789 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1572/3121] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:47 2022] DELETE /identity/v3/projects/11bdafc438884621944db399da5528f8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:47.963673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51e75d-3cc5-4c19-90a7-19a11ca9c7ae None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:47.964202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51e75d-3cc5-4c19-90a7-19a11ca9c7ae None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:47.964768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51e75d-3cc5-4c19-90a7-19a11ca9c7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:47.989080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacd096-3b47-4c82-af67-e825e2df96c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:47.990118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacd096-3b47-4c82-af67-e825e2df96c6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:47.990599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacd096-3b47-4c82-af67-e825e2df96c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:47.991076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacd096-3b47-4c82-af67-e825e2df96c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:47.991903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baacd096-3b47-4c82-af67-e825e2df96c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:47.994064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baacd096-3b47-4c82-af67-e825e2df96c6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.003606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8a51e75d-3cc5-4c19-90a7-19a11ca9c7ae None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:48.005121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a51e75d-3cc5-4c19-90a7-19a11ca9c7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:48.015690 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1573/3122] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:58:48.022816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:48.024192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525c7ec-5414-4db1-ab95-fee6ad51f216 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.024521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525c7ec-5414-4db1-ab95-fee6ad51f216 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.024898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525c7ec-5414-4db1-ab95-fee6ad51f216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.047235 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1550/3123] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:58:47 2022] POST /identity/v3/projects => generated 391 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:58:48.052210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacd096-3b47-4c82-af67-e825e2df96c6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.056209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-baacd096-3b47-4c82-af67-e825e2df96c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:48.058979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4525c7ec-5414-4db1-ab95-fee6ad51f216 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:48.061451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4525c7ec-5414-4db1-ab95-fee6ad51f216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:48.152333 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1574/3124] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:48.155156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525c7ec-5414-4db1-ab95-fee6ad51f216 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.165844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525c7ec-5414-4db1-ab95-fee6ad51f216 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.166703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacd096-3b47-4c82-af67-e825e2df96c6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.172407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8482edd-0399-4e59-8fcc-3ca53cd81c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.173523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8482edd-0399-4e59-8fcc-3ca53cd81c8d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.174124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8482edd-0399-4e59-8fcc-3ca53cd81c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.174772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8482edd-0399-4e59-8fcc-3ca53cd81c8d admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.175690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8482edd-0399-4e59-8fcc-3ca53cd81c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.178202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16db12e-3c21-4152-af07-32caf1db0fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.178914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8482edd-0399-4e59-8fcc-3ca53cd81c8d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.179171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16db12e-3c21-4152-af07-32caf1db0fac admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.179512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16db12e-3c21-4152-af07-32caf1db0fac admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.179845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16db12e-3c21-4152-af07-32caf1db0fac admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.180784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16db12e-3c21-4152-af07-32caf1db0fac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:58:48.181174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16db12e-3c21-4152-af07-32caf1db0fac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.185144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16db12e-3c21-4152-af07-32caf1db0fac admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.217967 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1575/3125] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:58:48.219109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16db12e-3c21-4152-af07-32caf1db0fac admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.226000 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1551/3126] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:58:48 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:58:48.228186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8482edd-0399-4e59-8fcc-3ca53cd81c8d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.237202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16db12e-3c21-4152-af07-32caf1db0fac admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.242556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620f4d-73f0-444d-b156-40963c921db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.243018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8482edd-0399-4e59-8fcc-3ca53cd81c8d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.243507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620f4d-73f0-444d-b156-40963c921db5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.243793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620f4d-73f0-444d-b156-40963c921db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.244152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620f4d-73f0-444d-b156-40963c921db5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.245019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3620f4d-73f0-444d-b156-40963c921db5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.247884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3620f4d-73f0-444d-b156-40963c921db5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.248155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1a2b8-7fff-461e-aeb6-f682c2aad634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.248969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1a2b8-7fff-461e-aeb6-f682c2aad634 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.249514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1a2b8-7fff-461e-aeb6-f682c2aad634 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.250035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1a2b8-7fff-461e-aeb6-f682c2aad634 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.250703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1a2b8-7fff-461e-aeb6-f682c2aad634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.254217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1a2b8-7fff-461e-aeb6-f682c2aad634 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.271375 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1552/3127] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:58:48.274773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1a2b8-7fff-461e-aeb6-f682c2aad634 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.288977 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1576/3128] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:58:48 2022] POST /identity/v3/projects => generated 391 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:58:48.291926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1a2b8-7fff-461e-aeb6-f682c2aad634 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.293117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620f4d-73f0-444d-b156-40963c921db5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.364396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a884c-b9fe-4ad0-93b5-ab2eedf53850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.365016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a884c-b9fe-4ad0-93b5-ab2eedf53850 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.365320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a884c-b9fe-4ad0-93b5-ab2eedf53850 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.365585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a884c-b9fe-4ad0-93b5-ab2eedf53850 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.366216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a884c-b9fe-4ad0-93b5-ab2eedf53850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.367787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a884c-b9fe-4ad0-93b5-ab2eedf53850 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.378865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620f4d-73f0-444d-b156-40963c921db5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.380556 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1553/3129] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:58:48.385372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a884c-b9fe-4ad0-93b5-ab2eedf53850 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.386051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96c84a-f546-49ee-82df-dea3d3c8ace3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.386732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96c84a-f546-49ee-82df-dea3d3c8ace3 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.387077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96c84a-f546-49ee-82df-dea3d3c8ace3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.387411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96c84a-f546-49ee-82df-dea3d3c8ace3 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.388347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96c84a-f546-49ee-82df-dea3d3c8ace3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.391616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96c84a-f546-49ee-82df-dea3d3c8ace3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.397645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a884c-b9fe-4ad0-93b5-ab2eedf53850 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.403293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e522426-69a1-4f6d-8b9d-65cf872cca18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.404478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e522426-69a1-4f6d-8b9d-65cf872cca18 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.404752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e522426-69a1-4f6d-8b9d-65cf872cca18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.405011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e522426-69a1-4f6d-8b9d-65cf872cca18 admin admin] PATH_INFO: `/v3/projects/8d37935df706485388d6137dff1faba5/users/6c8c93cfed5140b183cc0e3a559b1ce7/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.419408 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1577/3130] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:58:48 2022] POST /identity/v3/users => generated 296 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:58:48.426316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96c84a-f546-49ee-82df-dea3d3c8ace3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.432182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e522426-69a1-4f6d-8b9d-65cf872cca18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d37935df706485388d6137dff1faba5, user_id=6c8c93cfed5140b183cc0e3a559b1ce7, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.435481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e522426-69a1-4f6d-8b9d-65cf872cca18 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.445811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96c84a-f546-49ee-82df-dea3d3c8ace3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.464641 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1554/3131] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:58:48 2022] PUT /identity/v3/projects/8d37935df706485388d6137dff1faba5/users/6c8c93cfed5140b183cc0e3a559b1ce7/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:48.467078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6be4f7-ce16-4202-9e17-14fc9d496c65 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.467379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6be4f7-ce16-4202-9e17-14fc9d496c65 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.467618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6be4f7-ce16-4202-9e17-14fc9d496c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.473890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:48.493189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7d6be4f7-ce16-4202-9e17-14fc9d496c65 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:48.494435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7d6be4f7-ce16-4202-9e17-14fc9d496c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:48.524045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476ec42-b310-4c2f-a7dd-8d97cf4db3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.524844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476ec42-b310-4c2f-a7dd-8d97cf4db3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.525150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476ec42-b310-4c2f-a7dd-8d97cf4db3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.525418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476ec42-b310-4c2f-a7dd-8d97cf4db3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.526112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476ec42-b310-4c2f-a7dd-8d97cf4db3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.527738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476ec42-b310-4c2f-a7dd-8d97cf4db3c2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.540418 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1578/3132] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:58:48.544183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476ec42-b310-4c2f-a7dd-8d97cf4db3c2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.546615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:48.556803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476ec42-b310-4c2f-a7dd-8d97cf4db3c2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.568626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05ec6b-47a8-4952-8305-decd79074d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.569660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05ec6b-47a8-4952-8305-decd79074d39 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.574945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05ec6b-47a8-4952-8305-decd79074d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.575511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05ec6b-47a8-4952-8305-decd79074d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.576535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05ec6b-47a8-4952-8305-decd79074d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.578890 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1555/3133] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:48.586122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05ec6b-47a8-4952-8305-decd79074d39 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.588150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6be4f7-ce16-4202-9e17-14fc9d496c65 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.589844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7d6be4f7-ce16-4202-9e17-14fc9d496c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:48.602189 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1579/3134] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:58:48.608019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05ec6b-47a8-4952-8305-decd79074d39 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.631296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05ec6b-47a8-4952-8305-decd79074d39 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.646153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc023079-8d36-4191-8047-57da92ed1a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.647787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc023079-8d36-4191-8047-57da92ed1a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.648072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc023079-8d36-4191-8047-57da92ed1a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.648352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc023079-8d36-4191-8047-57da92ed1a0c admin admin] PATH_INFO: `/v3/projects/884bc7642fc14986a1f105a44322716c/users/a9d5bc114a3347f58daee9ea6a440dea/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.668732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc023079-8d36-4191-8047-57da92ed1a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=884bc7642fc14986a1f105a44322716c, user_id=a9d5bc114a3347f58daee9ea6a440dea, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.677784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc023079-8d36-4191-8047-57da92ed1a0c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.694305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6be4f7-ce16-4202-9e17-14fc9d496c65 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.705796 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1580/3135] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:58:48 2022] PUT /identity/v3/projects/884bc7642fc14986a1f105a44322716c/users/a9d5bc114a3347f58daee9ea6a440dea/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:48.711009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16740c70-88ac-4634-bade-d3cd3e505156 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.711233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16740c70-88ac-4634-bade-d3cd3e505156 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.711762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16740c70-88ac-4634-bade-d3cd3e505156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.762465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-16740c70-88ac-4634-bade-d3cd3e505156 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:48.764892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-16740c70-88ac-4634-bade-d3cd3e505156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:48.808498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca8f52-5f1f-4806-bd95-6c3b32bc326a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.809903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca8f52-5f1f-4806-bd95-6c3b32bc326a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.810524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca8f52-5f1f-4806-bd95-6c3b32bc326a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.811015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca8f52-5f1f-4806-bd95-6c3b32bc326a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.861795 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1581/3136] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:58:48.872826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16740c70-88ac-4634-bade-d3cd3e505156 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:48.885749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16740c70-88ac-4634-bade-d3cd3e505156 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:48.891320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523c09d-db52-4b61-a68d-148670d00b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:48.892583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523c09d-db52-4b61-a68d-148670d00b28 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:48.893003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523c09d-db52-4b61-a68d-148670d00b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:48.893444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523c09d-db52-4b61-a68d-148670d00b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:48.904904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca8f52-5f1f-4806-bd95-6c3b32bc326a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.907365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca8f52-5f1f-4806-bd95-6c3b32bc326a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.911386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523c09d-db52-4b61-a68d-148670d00b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:48.912918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523c09d-db52-4b61-a68d-148670d00b28 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:48.930055 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1582/3137] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:48.930660 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1556/3138] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:52.738555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523c09d-db52-4b61-a68d-148670d00b28 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:52.750341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4523c09d-db52-4b61-a68d-148670d00b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:52.770968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523c09d-db52-4b61-a68d-148670d00b28 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:52.784088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:52.784733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:52.785005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:52.785308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] PATH_INFO: `/v3/users/eb0650ccb0ab4d17b795db9e18a6a160` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:52.792089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0650ccb0ab4d17b795db9e18a6a160)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:52.794452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:52.894935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0650ccb0ab4d17b795db9e18a6a160'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:52.920568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0650ccb0ab4d17b795db9e18a6a160'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:52.927045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:52.942963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0650ccb0ab4d17b795db9e18a6a160'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:52.964425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0650ccb0ab4d17b795db9e18a6a160'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:52.965429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43899ce8-4c6a-46cf-9c15-f43f53983661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0650ccb0ab4d17b795db9e18a6a160'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:52.974885 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1583/3139] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:58:52 2022] DELETE /identity/v3/users/eb0650ccb0ab4d17b795db9e18a6a160 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:53.961526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca8f52-5f1f-4806-bd95-6c3b32bc326a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:53.986823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca8f52-5f1f-4806-bd95-6c3b32bc326a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:54.117448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:54.146253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:54.146911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:54.147256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:54.147429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] PATH_INFO: `/v3/projects/bccddd29eb9d4df692e1599e63daffc5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:54.151042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccddd29eb9d4df692e1599e63daffc5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:54.153237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:54.277939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccddd29eb9d4df692e1599e63daffc5', 'request_id': 'req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:54.293505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccddd29eb9d4df692e1599e63daffc5', 'request_id': 'req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:54.316941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccddd29eb9d4df692e1599e63daffc5', 'request_id': 'req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:54.343715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] The token cache is being invalidate because project bccddd29eb9d4df692e1599e63daffc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:58:54.344299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:54.345380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ba4e7502-8b4f-4fe6-ac1a-7a4f62d81a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:54.362657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0f9d0-90ff-4551-99dc-5c08088af9b8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:54.363534 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1557/3140] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:53 2022] DELETE /identity/v3/projects/bccddd29eb9d4df692e1599e63daffc5 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:54.363736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0f9d0-90ff-4551-99dc-5c08088af9b8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:54.364047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0f9d0-90ff-4551-99dc-5c08088af9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:54.406212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e0d0f9d0-90ff-4551-99dc-5c08088af9b8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:54.407442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e0d0f9d0-90ff-4551-99dc-5c08088af9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:54.541181 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1584/3141] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:58:54.551804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943706de-9d0d-43bd-a6ed-e75a3173cf23 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:54.552266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943706de-9d0d-43bd-a6ed-e75a3173cf23 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:54.552642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943706de-9d0d-43bd-a6ed-e75a3173cf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:54.605724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-943706de-9d0d-43bd-a6ed-e75a3173cf23 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:54.607264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-943706de-9d0d-43bd-a6ed-e75a3173cf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:54.639101 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1558/3142] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:54.643550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0f9d0-90ff-4551-99dc-5c08088af9b8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:54.662068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0f9d0-90ff-4551-99dc-5c08088af9b8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:54.673831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a76a45-1f0d-4ee7-bcb2-a65406ccac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:54.674457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a76a45-1f0d-4ee7-bcb2-a65406ccac04 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:54.674904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a76a45-1f0d-4ee7-bcb2-a65406ccac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:54.679388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a76a45-1f0d-4ee7-bcb2-a65406ccac04 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:54.680161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a76a45-1f0d-4ee7-bcb2-a65406ccac04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:58:54.680465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a76a45-1f0d-4ee7-bcb2-a65406ccac04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:54.682371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a76a45-1f0d-4ee7-bcb2-a65406ccac04 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:54.710030 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1585/3143] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:58:54.714761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943706de-9d0d-43bd-a6ed-e75a3173cf23 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:54.726764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943706de-9d0d-43bd-a6ed-e75a3173cf23 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:54.735289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd940-eac5-44dd-bfa5-ffd1b861f1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:54.735878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd940-eac5-44dd-bfa5-ffd1b861f1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:54.736505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd940-eac5-44dd-bfa5-ffd1b861f1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:54.736861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd940-eac5-44dd-bfa5-ffd1b861f1e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:54.737627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edd940-eac5-44dd-bfa5-ffd1b861f1e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:54.739965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edd940-eac5-44dd-bfa5-ffd1b861f1e6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:54.823546 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1559/3144] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:58:54 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:58:54.830073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a76a45-1f0d-4ee7-bcb2-a65406ccac04 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:54.843929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a76a45-1f0d-4ee7-bcb2-a65406ccac04 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:54.948777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d5854-166a-410e-b0cb-697fcfc4b93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:54.949463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5854-166a-410e-b0cb-697fcfc4b93e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:54.953783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5854-166a-410e-b0cb-697fcfc4b93e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:54.954086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5854-166a-410e-b0cb-697fcfc4b93e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:54.954742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d5854-166a-410e-b0cb-697fcfc4b93e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:54.957859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d5854-166a-410e-b0cb-697fcfc4b93e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:54.992028 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1586/3145] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:58:54 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:58:54.996174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd940-eac5-44dd-bfa5-ffd1b861f1e6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:55.009981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd940-eac5-44dd-bfa5-ffd1b861f1e6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:55.016014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb46f75-5da5-4dce-ba2c-34de4aa737aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:55.016771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb46f75-5da5-4dce-ba2c-34de4aa737aa admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:55.017195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb46f75-5da5-4dce-ba2c-34de4aa737aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:55.017553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb46f75-5da5-4dce-ba2c-34de4aa737aa admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:55.018358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb46f75-5da5-4dce-ba2c-34de4aa737aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:55.020079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb46f75-5da5-4dce-ba2c-34de4aa737aa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:55.033946 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1560/3146] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:58:55.037993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d5854-166a-410e-b0cb-697fcfc4b93e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:55.051982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d5854-166a-410e-b0cb-697fcfc4b93e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:55.060482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fb4f3-ce35-460f-84c4-746f9b0a3ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:55.061428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fb4f3-ce35-460f-84c4-746f9b0a3ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:55.061813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fb4f3-ce35-460f-84c4-746f9b0a3ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:55.062242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fb4f3-ce35-460f-84c4-746f9b0a3ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:55.063002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fb4f3-ce35-460f-84c4-746f9b0a3ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:55.065264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fb4f3-ce35-460f-84c4-746f9b0a3ed5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:55.088554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb46f75-5da5-4dce-ba2c-34de4aa737aa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:55.089540 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1587/3147] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:58:55.101559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb46f75-5da5-4dce-ba2c-34de4aa737aa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:55.106659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1e9d7-8376-4a4e-a435-1c0413f04ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:55.107822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1e9d7-8376-4a4e-a435-1c0413f04ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:55.108091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1e9d7-8376-4a4e-a435-1c0413f04ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:55.108450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1e9d7-8376-4a4e-a435-1c0413f04ecd admin admin] PATH_INFO: `/v3/projects/57ac50a5098942e58c8ab72ff09d5159/users/83d85ef4ebdb452a93f24682e2bbd46c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:55.125966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1e9d7-8376-4a4e-a435-1c0413f04ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ac50a5098942e58c8ab72ff09d5159, user_id=83d85ef4ebdb452a93f24682e2bbd46c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:55.129459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1e9d7-8376-4a4e-a435-1c0413f04ecd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:55.148388 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1561/3148] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:58:55 2022] PUT /identity/v3/projects/57ac50a5098942e58c8ab72ff09d5159/users/83d85ef4ebdb452a93f24682e2bbd46c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:58:55.159772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822271c-b66b-49e8-9578-c281e91f0305 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:55.160058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822271c-b66b-49e8-9578-c281e91f0305 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:55.160331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822271c-b66b-49e8-9578-c281e91f0305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:55.214766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c822271c-b66b-49e8-9578-c281e91f0305 None None] MFA Rules not processed for user `83d85ef4ebdb452a93f24682e2bbd46c`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:55.216620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c822271c-b66b-49e8-9578-c281e91f0305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:55.274926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:55.350837 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1588/3149] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:55.363586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b55f6-3433-481f-ad5b-a40484fa027e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:55.363915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b55f6-3433-481f-ad5b-a40484fa027e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:55.364131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b55f6-3433-481f-ad5b-a40484fa027e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:55.404759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ad9b55f6-3433-481f-ad5b-a40484fa027e None None] MFA Rules not processed for user `83d85ef4ebdb452a93f24682e2bbd46c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:55.406143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ad9b55f6-3433-481f-ad5b-a40484fa027e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:55.425400 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1562/3150] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:57.061888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822271c-b66b-49e8-9578-c281e91f0305 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:57.063626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c822271c-b66b-49e8-9578-c281e91f0305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:57.287579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822271c-b66b-49e8-9578-c281e91f0305 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:57.293958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7631244-0c71-4d55-81ff-cee38df87cb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:57.295994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7631244-0c71-4d55-81ff-cee38df87cb6 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:57.296133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7631244-0c71-4d55-81ff-cee38df87cb6 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:57.296404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7631244-0c71-4d55-81ff-cee38df87cb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:57.306666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7631244-0c71-4d55-81ff-cee38df87cb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:57.308708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7631244-0c71-4d55-81ff-cee38df87cb6 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:57.339288 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1589/3151] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:58:57.699851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76ca93-6bd5-4f6d-b716-1bce791ca017 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:57.700823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76ca93-6bd5-4f6d-b716-1bce791ca017 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:57.700996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76ca93-6bd5-4f6d-b716-1bce791ca017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:57.750423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4d76ca93-6bd5-4f6d-b716-1bce791ca017 None None] MFA Rules not processed for user `6c8c93cfed5140b183cc0e3a559b1ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:57.754966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4d76ca93-6bd5-4f6d-b716-1bce791ca017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:57.858798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:57.864883 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1563/3152] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:58:57.874431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918494e-7853-43c7-b516-5f2e3ffc2567 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:57.875006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918494e-7853-43c7-b516-5f2e3ffc2567 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:57.875623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918494e-7853-43c7-b516-5f2e3ffc2567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:57.897595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e918494e-7853-43c7-b516-5f2e3ffc2567 None None] MFA Rules not processed for user `6c8c93cfed5140b183cc0e3a559b1ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:58:57.898986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e918494e-7853-43c7-b516-5f2e3ffc2567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:57.907957 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1590/3153] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:58:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:58:59.497466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76ca93-6bd5-4f6d-b716-1bce791ca017 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:59.499174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4d76ca93-6bd5-4f6d-b716-1bce791ca017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:58:59.590426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918494e-7853-43c7-b516-5f2e3ffc2567 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:58:59.624439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918494e-7853-43c7-b516-5f2e3ffc2567 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:59.647464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9b19f-a2e3-4815-a330-109b5ed612d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:59.648579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9b19f-a2e3-4815-a330-109b5ed612d6 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:59.649160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9b19f-a2e3-4815-a330-109b5ed612d6 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:59.649632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9b19f-a2e3-4815-a330-109b5ed612d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:59.694210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76ca93-6bd5-4f6d-b716-1bce791ca017 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:58:59.700792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9b19f-a2e3-4815-a330-109b5ed612d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:59.705656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:58:59.706420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:58:59.706751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:58:59.706930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] PATH_INFO: `/v3/users/500cba6141d34c4081d38a7b1be1775e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:58:59.709866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] RBAC: Authorizing `identity:delete_user(user_id=500cba6141d34c4081d38a7b1be1775e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:58:59.711443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:59.711653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9b19f-a2e3-4815-a330-109b5ed612d6 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:58:59.745965 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1591/3154] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:58:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:58:59.914945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500cba6141d34c4081d38a7b1be1775e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:59.942319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500cba6141d34c4081d38a7b1be1775e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:59.987177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500cba6141d34c4081d38a7b1be1775e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:59.999131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500cba6141d34c4081d38a7b1be1775e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:58:59.999980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500cba6141d34c4081d38a7b1be1775e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:00.015954 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1564/3155] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:58:59 2022] DELETE /identity/v3/users/500cba6141d34c4081d38a7b1be1775e => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:00.016146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de140b0-8bc1-42e4-b6a8-30197d7ecd57 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:00.016722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de140b0-8bc1-42e4-b6a8-30197d7ecd57 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:00.017178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de140b0-8bc1-42e4-b6a8-30197d7ecd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:00.051139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3de140b0-8bc1-42e4-b6a8-30197d7ecd57 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:00.053649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3de140b0-8bc1-42e4-b6a8-30197d7ecd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:00.189083 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1592/3156] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:00.206118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:00.207806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:00.345992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20669dac-c2b2-4838-9378-05958900372b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:00.352438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ac5b1-1884-475d-8db2-d72abe7a6076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:00.353208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ac5b1-1884-475d-8db2-d72abe7a6076 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:00.355331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ac5b1-1884-475d-8db2-d72abe7a6076 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:00.355780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ac5b1-1884-475d-8db2-d72abe7a6076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:00.383666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ac5b1-1884-475d-8db2-d72abe7a6076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:00.386159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ac5b1-1884-475d-8db2-d72abe7a6076 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:00.402911 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1565/3157] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:00.904770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1318ed-2e84-4662-9c4f-b6407bfe577d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:00.912789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1318ed-2e84-4662-9c4f-b6407bfe577d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:00.913116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1318ed-2e84-4662-9c4f-b6407bfe577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:00.944014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ac5b1-1884-475d-8db2-d72abe7a6076 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:00.945713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-341ac5b1-1884-475d-8db2-d72abe7a6076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:00.974989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ac5b1-1884-475d-8db2-d72abe7a6076 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:00.994817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1f1318ed-2e84-4662-9c4f-b6407bfe577d None None] MFA Rules not processed for user `a9d5bc114a3347f58daee9ea6a440dea`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:00.996125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1f1318ed-2e84-4662-9c4f-b6407bfe577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:01.000761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:01.001852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:01.002306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:01.002760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] PATH_INFO: `/v3/projects/c44841db4e33412e805da46ba4262fe7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:01.004289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] RBAC: Authorizing `identity:delete_project(project_id=c44841db4e33412e805da46ba4262fe7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:01.013406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:01.043320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:01.094814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:01.132803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c44841db4e33412e805da46ba4262fe7', 'request_id': 'req-35d5b768-6f93-4a62-b8fa-11b22c34a7be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:01.150823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c44841db4e33412e805da46ba4262fe7', 'request_id': 'req-35d5b768-6f93-4a62-b8fa-11b22c34a7be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:01.160545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c44841db4e33412e805da46ba4262fe7', 'request_id': 'req-35d5b768-6f93-4a62-b8fa-11b22c34a7be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:01.177259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] The token cache is being invalidate because project c44841db4e33412e805da46ba4262fe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:59:01.178222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:01.178222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:01.178930 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1593/3158] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:01.180528 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1566/3159] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:00 2022] DELETE /identity/v3/projects/c44841db4e33412e805da46ba4262fe7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:01.186411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1318ed-2e84-4662-9c4f-b6407bfe577d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:01.207208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1318ed-2e84-4662-9c4f-b6407bfe577d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:01.315115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:01.315906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:01.316265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:01.316600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] PATH_INFO: `/v3/users/623abfad80794a058a678b8b50009e3e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:01.317933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=623abfad80794a058a678b8b50009e3e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:01.319955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:01.391183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '623abfad80794a058a678b8b50009e3e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:01.410193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '623abfad80794a058a678b8b50009e3e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:01.421373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '623abfad80794a058a678b8b50009e3e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:01.433770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '623abfad80794a058a678b8b50009e3e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:01.434607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-233695fb-2cc5-487a-9e9c-43931fcbc888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '623abfad80794a058a678b8b50009e3e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:01.447011 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1594/3160] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:59:01 2022] DELETE /identity/v3/users/623abfad80794a058a678b8b50009e3e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:02.290832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:02.292833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:02.315840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5b768-6f93-4a62-b8fa-11b22c34a7be admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:02.443112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:02.444546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:02.445295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:02.446207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] PATH_INFO: `/v3/projects/af0b8f89094f42fb9ae9829807d9f6d4` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:02.447755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0b8f89094f42fb9ae9829807d9f6d4)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:02.451778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:02.550876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0b8f89094f42fb9ae9829807d9f6d4', 'request_id': 'req-d919d4fe-6be2-487b-b1bf-f0f22fd069df'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:02.570330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0b8f89094f42fb9ae9829807d9f6d4', 'request_id': 'req-d919d4fe-6be2-487b-b1bf-f0f22fd069df'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:02.583860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0b8f89094f42fb9ae9829807d9f6d4', 'request_id': 'req-d919d4fe-6be2-487b-b1bf-f0f22fd069df'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:02.600727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] The token cache is being invalidate because project af0b8f89094f42fb9ae9829807d9f6d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:59:02.600888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:02.601811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:02.608902 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1567/3161] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:02 2022] DELETE /identity/v3/projects/af0b8f89094f42fb9ae9829807d9f6d4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:02.610322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1f46-09fa-4b7c-b95b-c13a1aaf1673 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:02.610835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1f46-09fa-4b7c-b95b-c13a1aaf1673 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:02.610835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1f46-09fa-4b7c-b95b-c13a1aaf1673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:02.641935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d4af1f46-09fa-4b7c-b95b-c13a1aaf1673 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:02.644232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4af1f46-09fa-4b7c-b95b-c13a1aaf1673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:02.704794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:02.706435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:02.761218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:02.811709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:02.816489 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1595/3162] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:02.820977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa235b-d393-4478-8863-9af7e3e59c3a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:02.821404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa235b-d393-4478-8863-9af7e3e59c3a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:02.824235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa235b-d393-4478-8863-9af7e3e59c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:02.874404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1afa235b-d393-4478-8863-9af7e3e59c3a None None] MFA Rules not processed for user `6adaa12c2ca9469299994300bc856ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:02.875627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1afa235b-d393-4478-8863-9af7e3e59c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:03.017294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919d4fe-6be2-487b-b1bf-f0f22fd069df admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:03.043242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e22e58-8530-48d7-8552-2ee3f40475be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:03.044432 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1596/3163] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:03.044554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e22e58-8530-48d7-8552-2ee3f40475be service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:03.044810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e22e58-8530-48d7-8552-2ee3f40475be service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:03.045205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e22e58-8530-48d7-8552-2ee3f40475be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:03.045684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11646772-9b1c-47aa-b2aa-09e981fcf53a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:03.045999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11646772-9b1c-47aa-b2aa-09e981fcf53a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:03.046267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11646772-9b1c-47aa-b2aa-09e981fcf53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:03.050231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e3e22e58-8530-48d7-8552-2ee3f40475be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:03.111596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-11646772-9b1c-47aa-b2aa-09e981fcf53a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:03.115014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-11646772-9b1c-47aa-b2aa-09e981fcf53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:03.134581 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1597/3164] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:03.139552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11646772-9b1c-47aa-b2aa-09e981fcf53a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:03.153949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11646772-9b1c-47aa-b2aa-09e981fcf53a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:03.160002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721cc85-e0af-47d6-a121-831199d1e586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:03.160757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721cc85-e0af-47d6-a121-831199d1e586 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:03.161173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721cc85-e0af-47d6-a121-831199d1e586 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:03.161534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721cc85-e0af-47d6-a121-831199d1e586 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:03.162561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721cc85-e0af-47d6-a121-831199d1e586 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:59:03.162956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721cc85-e0af-47d6-a121-831199d1e586 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:03.165450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721cc85-e0af-47d6-a121-831199d1e586 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:03.194935 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1598/3165] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 20:59:03.194935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721cc85-e0af-47d6-a121-831199d1e586 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:03.223748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721cc85-e0af-47d6-a121-831199d1e586 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:03.223960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e22e58-8530-48d7-8552-2ee3f40475be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:03.227060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e22e58-8530-48d7-8552-2ee3f40475be service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:03.240697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba94cb-f515-4024-902c-5589cbcda5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:03.242158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba94cb-f515-4024-902c-5589cbcda5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:03.242583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba94cb-f515-4024-902c-5589cbcda5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:03.243027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba94cb-f515-4024-902c-5589cbcda5f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:03.243812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ba94cb-f515-4024-902c-5589cbcda5f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:03.254385 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1568/3166] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:03.261984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ba94cb-f515-4024-902c-5589cbcda5f3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:03.329098 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1599/3167] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:59:03 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:59:03.333227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e22e58-8530-48d7-8552-2ee3f40475be service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:03.347540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e22e58-8530-48d7-8552-2ee3f40475be service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:03.441562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0090da4-79a7-41b5-bdaf-7201aa0dbfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:03.442475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0090da4-79a7-41b5-bdaf-7201aa0dbfda admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:03.442909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0090da4-79a7-41b5-bdaf-7201aa0dbfda admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:03.443274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0090da4-79a7-41b5-bdaf-7201aa0dbfda admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:03.444264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0090da4-79a7-41b5-bdaf-7201aa0dbfda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:03.447136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0090da4-79a7-41b5-bdaf-7201aa0dbfda admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:03.490083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba94cb-f515-4024-902c-5589cbcda5f3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:03.500007 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1569/3168] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:59:03 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:59:03.503623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba94cb-f515-4024-902c-5589cbcda5f3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:03.511656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28472608-3887-4e26-8241-5b53dd47bd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:03.512524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28472608-3887-4e26-8241-5b53dd47bd37 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:03.512981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28472608-3887-4e26-8241-5b53dd47bd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:03.513385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28472608-3887-4e26-8241-5b53dd47bd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:03.514257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28472608-3887-4e26-8241-5b53dd47bd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:03.516014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28472608-3887-4e26-8241-5b53dd47bd37 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:03.533027 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1600/3169] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:03.537253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0090da4-79a7-41b5-bdaf-7201aa0dbfda admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:03.552811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0090da4-79a7-41b5-bdaf-7201aa0dbfda admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:03.559405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09004b-a986-4670-971e-f909ea62003b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:03.569446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09004b-a986-4670-971e-f909ea62003b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:03.569446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09004b-a986-4670-971e-f909ea62003b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:03.570176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09004b-a986-4670-971e-f909ea62003b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:03.570988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09004b-a986-4670-971e-f909ea62003b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:03.573194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09004b-a986-4670-971e-f909ea62003b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:03.593328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28472608-3887-4e26-8241-5b53dd47bd37 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:03.600494 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1570/3170] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:03.605686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28472608-3887-4e26-8241-5b53dd47bd37 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:03.624149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed1e5b-f707-411d-bf07-9cfa157b29ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:03.625568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed1e5b-f707-411d-bf07-9cfa157b29ae admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:03.625950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed1e5b-f707-411d-bf07-9cfa157b29ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:03.626746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed1e5b-f707-411d-bf07-9cfa157b29ae admin admin] PATH_INFO: `/v3/projects/9ea11fd1a5a64f4a9641d7ed07f8e94f/users/42276cf2f2894712ac91f87e4ce1f6ca/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:03.651721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed1e5b-f707-411d-bf07-9cfa157b29ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea11fd1a5a64f4a9641d7ed07f8e94f, user_id=42276cf2f2894712ac91f87e4ce1f6ca, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:03.659895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed1e5b-f707-411d-bf07-9cfa157b29ae admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:03.697371 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1601/3171] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:59:03 2022] PUT /identity/v3/projects/9ea11fd1a5a64f4a9641d7ed07f8e94f/users/42276cf2f2894712ac91f87e4ce1f6ca/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:03.709531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ca3c7-a14a-4695-847f-4b59998fffa9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:03.709911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ca3c7-a14a-4695-847f-4b59998fffa9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:03.710362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ca3c7-a14a-4695-847f-4b59998fffa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:03.753638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-393ca3c7-a14a-4695-847f-4b59998fffa9 None None] MFA Rules not processed for user `42276cf2f2894712ac91f87e4ce1f6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:03.755885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-393ca3c7-a14a-4695-847f-4b59998fffa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:03.863807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:03.928486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed1e5b-f707-411d-bf07-9cfa157b29ae admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:03.929482 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1571/3172] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:04.062233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed1e5b-f707-411d-bf07-9cfa157b29ae admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:04.074228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84a017-bf64-473c-a031-0d2553391255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:04.074895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84a017-bf64-473c-a031-0d2553391255 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:04.075658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84a017-bf64-473c-a031-0d2553391255 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:04.076000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84a017-bf64-473c-a031-0d2553391255 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:04.076754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84a017-bf64-473c-a031-0d2553391255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:04.078907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84a017-bf64-473c-a031-0d2553391255 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:04.131132 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1602/3173] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:59:03 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:59:04.135668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ca3c7-a14a-4695-847f-4b59998fffa9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:04.151666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ca3c7-a14a-4695-847f-4b59998fffa9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:04.250754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86e307-33c9-4011-9980-68fd69b5a838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:04.251479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86e307-33c9-4011-9980-68fd69b5a838 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:04.251853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86e307-33c9-4011-9980-68fd69b5a838 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:04.252244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86e307-33c9-4011-9980-68fd69b5a838 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:04.252979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86e307-33c9-4011-9980-68fd69b5a838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:04.255055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86e307-33c9-4011-9980-68fd69b5a838 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:04.289117 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1572/3174] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:59:04 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:59:04.292666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84a017-bf64-473c-a031-0d2553391255 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:04.308561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84a017-bf64-473c-a031-0d2553391255 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:04.315915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e25351-c539-4203-99c7-05f7e25db4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:04.317638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e25351-c539-4203-99c7-05f7e25db4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:04.318301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e25351-c539-4203-99c7-05f7e25db4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:04.319072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e25351-c539-4203-99c7-05f7e25db4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:04.320816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e25351-c539-4203-99c7-05f7e25db4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:04.322969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e25351-c539-4203-99c7-05f7e25db4a4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:04.327297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:04.346502 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1603/3175] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 20:59:04.347745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86e307-33c9-4011-9980-68fd69b5a838 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:04.366099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86e307-33c9-4011-9980-68fd69b5a838 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:04.378150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ab68d-f3cd-40ad-96bc-593033566477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:04.380612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab68d-f3cd-40ad-96bc-593033566477 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:04.380898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab68d-f3cd-40ad-96bc-593033566477 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:04.381316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab68d-f3cd-40ad-96bc-593033566477 admin admin] PATH_INFO: `/v3/projects/e34ada1829574d1d8f301572a287d2c2/users/c053667200af4dc9a8f8bcf00f624c3f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:04.423019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ab68d-f3cd-40ad-96bc-593033566477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e34ada1829574d1d8f301572a287d2c2, user_id=c053667200af4dc9a8f8bcf00f624c3f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:04.425981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ab68d-f3cd-40ad-96bc-593033566477 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:04.448696 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1573/3176] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:59:04 2022] PUT /identity/v3/projects/e34ada1829574d1d8f301572a287d2c2/users/c053667200af4dc9a8f8bcf00f624c3f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:04.453348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e25351-c539-4203-99c7-05f7e25db4a4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:04.465519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e25351-c539-4203-99c7-05f7e25db4a4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:04.537353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b795e-b3ce-4c11-9a26-987f828c4e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:04.537967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b795e-b3ce-4c11-9a26-987f828c4e80 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:04.538249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b795e-b3ce-4c11-9a26-987f828c4e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:04.538514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b795e-b3ce-4c11-9a26-987f828c4e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:04.539167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b795e-b3ce-4c11-9a26-987f828c4e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:04.540772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b795e-b3ce-4c11-9a26-987f828c4e80 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:04.554370 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1604/3177] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:04.558192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ab68d-f3cd-40ad-96bc-593033566477 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:04.570837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ab68d-f3cd-40ad-96bc-593033566477 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:04.576559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b2b20-7808-4bac-8a40-761fce0c4315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:04.577372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b2b20-7808-4bac-8a40-761fce0c4315 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:04.577716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b2b20-7808-4bac-8a40-761fce0c4315 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:04.578178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b2b20-7808-4bac-8a40-761fce0c4315 admin admin] PATH_INFO: `/v3/projects/e34ada1829574d1d8f301572a287d2c2/users/c053667200af4dc9a8f8bcf00f624c3f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:04.580562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b2b20-7808-4bac-8a40-761fce0c4315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e34ada1829574d1d8f301572a287d2c2, user_id=c053667200af4dc9a8f8bcf00f624c3f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:04.583349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b2b20-7808-4bac-8a40-761fce0c4315 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:04.598212 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1574/3178] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:59:04 2022] PUT /identity/v3/projects/e34ada1829574d1d8f301572a287d2c2/users/c053667200af4dc9a8f8bcf00f624c3f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:04.609973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cec6f9-c121-4e75-93fe-d91b88a3eee6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:04.610265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cec6f9-c121-4e75-93fe-d91b88a3eee6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:04.610522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cec6f9-c121-4e75-93fe-d91b88a3eee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:04.656912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-83cec6f9-c121-4e75-93fe-d91b88a3eee6 None None] MFA Rules not processed for user `c053667200af4dc9a8f8bcf00f624c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:04.659637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83cec6f9-c121-4e75-93fe-d91b88a3eee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:04.810961 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1605/3179] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:04.811199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966e8a-1150-4840-965c-2893d7863190 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:04.811678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966e8a-1150-4840-965c-2893d7863190 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:04.811964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966e8a-1150-4840-965c-2893d7863190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:04.852241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e6966e8a-1150-4840-965c-2893d7863190 None None] MFA Rules not processed for user `42276cf2f2894712ac91f87e4ce1f6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:04.855991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e6966e8a-1150-4840-965c-2893d7863190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:04.969354 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1575/3180] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:59:08.590240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cec6f9-c121-4e75-93fe-d91b88a3eee6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:08.682507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cec6f9-c121-4e75-93fe-d91b88a3eee6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:08.688234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a5688-54a9-4b77-9083-e4aa42373948 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:08.688913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a5688-54a9-4b77-9083-e4aa42373948 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:08.689461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a5688-54a9-4b77-9083-e4aa42373948 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:08.689917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a5688-54a9-4b77-9083-e4aa42373948 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:08.698806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a5688-54a9-4b77-9083-e4aa42373948 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:08.700468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a5688-54a9-4b77-9083-e4aa42373948 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:08.713972 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1606/3181] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:14.527816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966e8a-1150-4840-965c-2893d7863190 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:14.534134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e6966e8a-1150-4840-965c-2893d7863190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:14.595446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:14.638001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966e8a-1150-4840-965c-2893d7863190 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:14.644059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:14.645092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:14.645244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:14.645588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] PATH_INFO: `/v3/users/a9d5bc114a3347f58daee9ea6a440dea` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:14.646862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9d5bc114a3347f58daee9ea6a440dea)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:14.649064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:14.709469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d5bc114a3347f58daee9ea6a440dea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:14.717388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d5bc114a3347f58daee9ea6a440dea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:14.730089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d5bc114a3347f58daee9ea6a440dea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:14.743683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d5bc114a3347f58daee9ea6a440dea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:14.744458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-880f6cf7-0d52-4075-b48f-9655ee6703c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d5bc114a3347f58daee9ea6a440dea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:14.754479 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1576/3182] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:59:14 2022] DELETE /identity/v3/users/a9d5bc114a3347f58daee9ea6a440dea => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:14.936362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a5688-54a9-4b77-9083-e4aa42373948 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:14.944522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:14.949778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a5688-54a9-4b77-9083-e4aa42373948 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:15.020191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:15.020821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:15.021125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:15.021397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] PATH_INFO: `/v3/projects/884bc7642fc14986a1f105a44322716c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:15.022517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=884bc7642fc14986a1f105a44322716c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:15.024430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:15.081863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884bc7642fc14986a1f105a44322716c', 'request_id': 'req-2f9918b2-3b91-4f24-a63e-0497c2d974d6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:15.091618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884bc7642fc14986a1f105a44322716c', 'request_id': 'req-2f9918b2-3b91-4f24-a63e-0497c2d974d6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:15.099157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884bc7642fc14986a1f105a44322716c', 'request_id': 'req-2f9918b2-3b91-4f24-a63e-0497c2d974d6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:15.111396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] The token cache is being invalidate because project 884bc7642fc14986a1f105a44322716c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:59:15.111929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:15.112643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f9918b2-3b91-4f24-a63e-0497c2d974d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:15.114112 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1607/3183] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:14 2022] DELETE /identity/v3/projects/884bc7642fc14986a1f105a44322716c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:15.119169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527868c-6e86-4b5b-aa4e-b4ad1714ae8c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:15.119560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527868c-6e86-4b5b-aa4e-b4ad1714ae8c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:15.119952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527868c-6e86-4b5b-aa4e-b4ad1714ae8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:15.143759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5527868c-6e86-4b5b-aa4e-b4ad1714ae8c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:15.144985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5527868c-6e86-4b5b-aa4e-b4ad1714ae8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:15.210787 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1577/3184] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:15.218838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147e824-30e6-4400-9f20-f63e669c64eb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:15.219124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147e824-30e6-4400-9f20-f63e669c64eb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:15.219428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147e824-30e6-4400-9f20-f63e669c64eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:15.241306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f147e824-30e6-4400-9f20-f63e669c64eb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:15.242793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f147e824-30e6-4400-9f20-f63e669c64eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:15.251271 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1608/3185] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:15.255522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527868c-6e86-4b5b-aa4e-b4ad1714ae8c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:15.265898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527868c-6e86-4b5b-aa4e-b4ad1714ae8c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:15.270527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0b88e-7924-4ab1-b460-efc0307150ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:15.271116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0b88e-7924-4ab1-b460-efc0307150ae admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:15.271387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0b88e-7924-4ab1-b460-efc0307150ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:15.271652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0b88e-7924-4ab1-b460-efc0307150ae admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:15.272288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0b88e-7924-4ab1-b460-efc0307150ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:59:15.272583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0b88e-7924-4ab1-b460-efc0307150ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:15.274206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0b88e-7924-4ab1-b460-efc0307150ae admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:15.287127 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1578/3186] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:59:15.291623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147e824-30e6-4400-9f20-f63e669c64eb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:15.302643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147e824-30e6-4400-9f20-f63e669c64eb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:15.317375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e8483-d007-4cdf-abd2-8104c683d04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:15.318002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e8483-d007-4cdf-abd2-8104c683d04a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:15.318280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e8483-d007-4cdf-abd2-8104c683d04a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:15.318547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e8483-d007-4cdf-abd2-8104c683d04a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:15.319621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e8483-d007-4cdf-abd2-8104c683d04a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:15.321474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e8483-d007-4cdf-abd2-8104c683d04a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:15.355926 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1609/3187] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:59:15 2022] POST /identity/v3/projects => generated 397 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:59:15.357279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0b88e-7924-4ab1-b460-efc0307150ae admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:15.377495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0b88e-7924-4ab1-b460-efc0307150ae admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:15.422652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:15.444191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591e7b9-8c06-4620-a8d0-9872d8576f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:15.444805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591e7b9-8c06-4620-a8d0-9872d8576f2a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:15.445102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591e7b9-8c06-4620-a8d0-9872d8576f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:15.445379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591e7b9-8c06-4620-a8d0-9872d8576f2a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:15.446013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591e7b9-8c06-4620-a8d0-9872d8576f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:15.448089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591e7b9-8c06-4620-a8d0-9872d8576f2a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:15.472055 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1579/3188] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:59:15 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:59:15.476458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e8483-d007-4cdf-abd2-8104c683d04a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:15.511084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e8483-d007-4cdf-abd2-8104c683d04a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:15.525192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70653578-846e-468a-b662-632514316591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:15.525813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70653578-846e-468a-b662-632514316591 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:15.526089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70653578-846e-468a-b662-632514316591 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:15.526358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70653578-846e-468a-b662-632514316591 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:15.527003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70653578-846e-468a-b662-632514316591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:15.528872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70653578-846e-468a-b662-632514316591 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:15.541229 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1610/3189] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:15.545525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591e7b9-8c06-4620-a8d0-9872d8576f2a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:15.556320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591e7b9-8c06-4620-a8d0-9872d8576f2a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:15.562851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54327-ef5c-432c-adc5-440c6892f4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:15.563485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54327-ef5c-432c-adc5-440c6892f4fc admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:15.563760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54327-ef5c-432c-adc5-440c6892f4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:15.564025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54327-ef5c-432c-adc5-440c6892f4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:15.564686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f54327-ef5c-432c-adc5-440c6892f4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:15.566321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f54327-ef5c-432c-adc5-440c6892f4fc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:15.579401 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1580/3190] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:15.583618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70653578-846e-468a-b662-632514316591 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:15.594202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70653578-846e-468a-b662-632514316591 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:15.599266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c651c0-e863-408f-bafa-8b725c4b3c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:15.600040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c651c0-e863-408f-bafa-8b725c4b3c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:15.600399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c651c0-e863-408f-bafa-8b725c4b3c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:15.601242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c651c0-e863-408f-bafa-8b725c4b3c1d admin admin] PATH_INFO: `/v3/projects/ac52ed6c42444cecbd7212dae59bee9f/users/d07d1b1c31b04feb91f36fbf7fdc1b24/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:15.618398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c651c0-e863-408f-bafa-8b725c4b3c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac52ed6c42444cecbd7212dae59bee9f, user_id=d07d1b1c31b04feb91f36fbf7fdc1b24, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:15.621022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c651c0-e863-408f-bafa-8b725c4b3c1d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:15.636127 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1611/3191] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:59:15 2022] PUT /identity/v3/projects/ac52ed6c42444cecbd7212dae59bee9f/users/d07d1b1c31b04feb91f36fbf7fdc1b24/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:15.648371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be381c-5867-4906-b2b3-307b2b82540d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:15.648751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be381c-5867-4906-b2b3-307b2b82540d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:15.648929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be381c-5867-4906-b2b3-307b2b82540d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:15.689240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d3be381c-5867-4906-b2b3-307b2b82540d None None] MFA Rules not processed for user `d07d1b1c31b04feb91f36fbf7fdc1b24`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:15.690796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d3be381c-5867-4906-b2b3-307b2b82540d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:15.783108 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1581/3192] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:16.463735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c651c0-e863-408f-bafa-8b725c4b3c1d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:16.470510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-55c651c0-e863-408f-bafa-8b725c4b3c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:16.507481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:16.549312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c651c0-e863-408f-bafa-8b725c4b3c1d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:16.554738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729c36b-d1ab-4fa9-80f5-6a11bd41a197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:16.555353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729c36b-d1ab-4fa9-80f5-6a11bd41a197 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:16.555622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729c36b-d1ab-4fa9-80f5-6a11bd41a197 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:16.555887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729c36b-d1ab-4fa9-80f5-6a11bd41a197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:16.563539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729c36b-d1ab-4fa9-80f5-6a11bd41a197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:16.565150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729c36b-d1ab-4fa9-80f5-6a11bd41a197 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:16.573885 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1612/3193] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:16 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:21.757196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be381c-5867-4906-b2b3-307b2b82540d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:21.834552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be381c-5867-4906-b2b3-307b2b82540d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:21.841832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:21.842899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:21.843171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:21.843514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] PATH_INFO: `/v3/users/d07d1b1c31b04feb91f36fbf7fdc1b24` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:21.844755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d07d1b1c31b04feb91f36fbf7fdc1b24)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:21.846724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:21.928729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd07d1b1c31b04feb91f36fbf7fdc1b24'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:21.934696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:21.938994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd07d1b1c31b04feb91f36fbf7fdc1b24'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:21.965422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd07d1b1c31b04feb91f36fbf7fdc1b24'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:21.978502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd07d1b1c31b04feb91f36fbf7fdc1b24'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:21.979176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd07d1b1c31b04feb91f36fbf7fdc1b24'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:21.987754 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1582/3194] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:59:21 2022] DELETE /identity/v3/users/d07d1b1c31b04feb91f36fbf7fdc1b24 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:21.991996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6189b3a-062f-4711-a902-b08ce19f38f9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:21.992507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6189b3a-062f-4711-a902-b08ce19f38f9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:21.992507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6189b3a-062f-4711-a902-b08ce19f38f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:22.014048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c6189b3a-062f-4711-a902-b08ce19f38f9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:22.015276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c6189b3a-062f-4711-a902-b08ce19f38f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:22.079864 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1613/3195] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:22.091947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:22.093429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:22.161989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0627fd-6b0e-4061-b502-f51bac8f9f78 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:22.165893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30f679-2063-4f6d-a3b9-318a744944ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:22.166462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30f679-2063-4f6d-a3b9-318a744944ef service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:22.166738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30f679-2063-4f6d-a3b9-318a744944ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:22.167005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30f679-2063-4f6d-a3b9-318a744944ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:22.188014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30f679-2063-4f6d-a3b9-318a744944ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:22.189488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30f679-2063-4f6d-a3b9-318a744944ef service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:22.220795 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1583/3196] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:22.634039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4f79-fd61-4df0-8d24-ebbd5b3cf0c3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:22.634440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4f79-fd61-4df0-8d24-ebbd5b3cf0c3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:22.634801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4f79-fd61-4df0-8d24-ebbd5b3cf0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:22.657662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8f9f4f79-fd61-4df0-8d24-ebbd5b3cf0c3 None None] MFA Rules not processed for user `2bba6c9b3f674959ad79d4a394ed9f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:22.659394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8f9f4f79-fd61-4df0-8d24-ebbd5b3cf0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:22.749672 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1614/3197] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:22.845862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ace24a-8361-4578-b528-52a35e05efd9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:22.846856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ace24a-8361-4578-b528-52a35e05efd9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:22.847129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ace24a-8361-4578-b528-52a35e05efd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:22.872483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e0ace24a-8361-4578-b528-52a35e05efd9 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:22.874023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e0ace24a-8361-4578-b528-52a35e05efd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:22.940766 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1584/3198] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:59:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:22.971451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f4f79-fd61-4df0-8d24-ebbd5b3cf0c3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:22.974866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8f9f4f79-fd61-4df0-8d24-ebbd5b3cf0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:23.104941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:23.112713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f4f79-fd61-4df0-8d24-ebbd5b3cf0c3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:23.120225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f7ca9-0ec3-40b9-b6df-d895e5ee2480 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:23.123488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f7ca9-0ec3-40b9-b6df-d895e5ee2480 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:23.123768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f7ca9-0ec3-40b9-b6df-d895e5ee2480 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:23.124041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f7ca9-0ec3-40b9-b6df-d895e5ee2480 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:23.135598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f7ca9-0ec3-40b9-b6df-d895e5ee2480 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:23.137290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f7ca9-0ec3-40b9-b6df-d895e5ee2480 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:23.176150 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1615/3199] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:59:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:23.310495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ace24a-8361-4578-b528-52a35e05efd9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:23.317633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e0ace24a-8361-4578-b528-52a35e05efd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:23.342979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ace24a-8361-4578-b528-52a35e05efd9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:23.354192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:23.355279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:23.355772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:23.357111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] PATH_INFO: `/v3/projects/ac52ed6c42444cecbd7212dae59bee9f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:23.358600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac52ed6c42444cecbd7212dae59bee9f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:23.361314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:23.469333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac52ed6c42444cecbd7212dae59bee9f', 'request_id': 'req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:23.485107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac52ed6c42444cecbd7212dae59bee9f', 'request_id': 'req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:23.494204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac52ed6c42444cecbd7212dae59bee9f', 'request_id': 'req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:23.508667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:23.515456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] The token cache is being invalidate because project ac52ed6c42444cecbd7212dae59bee9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:59:23.517682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:23.519225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:23.524223 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1585/3200] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:23 2022] DELETE /identity/v3/projects/ac52ed6c42444cecbd7212dae59bee9f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:23.527387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2a055-350f-4b23-85c7-ed7ff75b5af1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:23.527757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2a055-350f-4b23-85c7-ed7ff75b5af1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:23.528988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2a055-350f-4b23-85c7-ed7ff75b5af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:23.567910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3cd2a055-350f-4b23-85c7-ed7ff75b5af1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:23.569604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3cd2a055-350f-4b23-85c7-ed7ff75b5af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:23.615884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:23.620084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:23.724944 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1616/3201] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:23.733721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b59993-a8ee-4f91-a5f0-0b19e92b5d89 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:23.734002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b59993-a8ee-4f91-a5f0-0b19e92b5d89 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:23.734410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b59993-a8ee-4f91-a5f0-0b19e92b5d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:23.740810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7176-2bc6-4ee6-bf32-c21b5775c1dc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:23.745549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6e044-129b-4d53-8f8b-b1a70f80e2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:23.746651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e044-129b-4d53-8f8b-b1a70f80e2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:23.747455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e044-129b-4d53-8f8b-b1a70f80e2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:23.747722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e044-129b-4d53-8f8b-b1a70f80e2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:23.750800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-31a6e044-129b-4d53-8f8b-b1a70f80e2c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:23.774510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6e044-129b-4d53-8f8b-b1a70f80e2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:23.775769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-33b59993-a8ee-4f91-a5f0-0b19e92b5d89 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:23.776107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6e044-129b-4d53-8f8b-b1a70f80e2c7 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:23.779474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-33b59993-a8ee-4f91-a5f0-0b19e92b5d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:23.793297 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1586/3202] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 20:59:23.799006 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1617/3203] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:59:23.803288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6e044-129b-4d53-8f8b-b1a70f80e2c7 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:23.825281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6e044-129b-4d53-8f8b-b1a70f80e2c7 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:23.830603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7a1d4-981b-4398-82ec-7c74e0ecaea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:23.831591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7a1d4-981b-4398-82ec-7c74e0ecaea9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:23.832053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7a1d4-981b-4398-82ec-7c74e0ecaea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:23.832531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7a1d4-981b-4398-82ec-7c74e0ecaea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:23.833466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7a1d4-981b-4398-82ec-7c74e0ecaea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:59:23.833946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7a1d4-981b-4398-82ec-7c74e0ecaea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:23.835825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7a1d4-981b-4398-82ec-7c74e0ecaea9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:23.862179 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1587/3204] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:59:23.862965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b59993-a8ee-4f91-a5f0-0b19e92b5d89 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:23.874153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b59993-a8ee-4f91-a5f0-0b19e92b5d89 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:23.882099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e25efc-575a-4b28-a509-0f29fd5660a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:23.883845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e25efc-575a-4b28-a509-0f29fd5660a4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:23.884622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e25efc-575a-4b28-a509-0f29fd5660a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:23.884901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e25efc-575a-4b28-a509-0f29fd5660a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:23.885575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e25efc-575a-4b28-a509-0f29fd5660a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:23.888953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e25efc-575a-4b28-a509-0f29fd5660a4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:23.972418 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1618/3205] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:59:23 2022] POST /identity/v3/projects => generated 401 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:59:23.976732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7a1d4-981b-4398-82ec-7c74e0ecaea9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:23.991841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7a1d4-981b-4398-82ec-7c74e0ecaea9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:24.096134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac81a3-8449-4d1a-9126-576ef9660a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:24.096902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac81a3-8449-4d1a-9126-576ef9660a5f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:24.097376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac81a3-8449-4d1a-9126-576ef9660a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:24.097646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac81a3-8449-4d1a-9126-576ef9660a5f admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:24.099126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac81a3-8449-4d1a-9126-576ef9660a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:24.101472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac81a3-8449-4d1a-9126-576ef9660a5f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:24.149067 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1588/3206] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:59:23 2022] POST /identity/v3/users => generated 301 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:59:24.154088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e25efc-575a-4b28-a509-0f29fd5660a4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:24.172462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e25efc-575a-4b28-a509-0f29fd5660a4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:24.182615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faee61a-b771-44c3-9172-cb5e98f2f4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:24.183311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faee61a-b771-44c3-9172-cb5e98f2f4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:24.183597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faee61a-b771-44c3-9172-cb5e98f2f4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:24.186761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faee61a-b771-44c3-9172-cb5e98f2f4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:24.187699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faee61a-b771-44c3-9172-cb5e98f2f4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:24.189568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faee61a-b771-44c3-9172-cb5e98f2f4e4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:24.221243 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1619/3207] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:24.225203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac81a3-8449-4d1a-9126-576ef9660a5f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:24.240469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac81a3-8449-4d1a-9126-576ef9660a5f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:24.266859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5813592-52c7-4393-bb56-a6b7cd9c61ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:24.267742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5813592-52c7-4393-bb56-a6b7cd9c61ac admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:24.268211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5813592-52c7-4393-bb56-a6b7cd9c61ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:24.269745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5813592-52c7-4393-bb56-a6b7cd9c61ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:24.271266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5813592-52c7-4393-bb56-a6b7cd9c61ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:24.274199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5813592-52c7-4393-bb56-a6b7cd9c61ac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:24.292755 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1589/3208] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:24.298027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faee61a-b771-44c3-9172-cb5e98f2f4e4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:24.322702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faee61a-b771-44c3-9172-cb5e98f2f4e4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:24.344234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851f7de-0d4e-467f-b258-5be5d675e793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:24.344681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851f7de-0d4e-467f-b258-5be5d675e793 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:24.345076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851f7de-0d4e-467f-b258-5be5d675e793 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:24.345345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851f7de-0d4e-467f-b258-5be5d675e793 admin admin] PATH_INFO: `/v3/projects/c8800104fcdd4b7b8eed6332dd249c80/users/37534369e7d644d78af7e3a93ab25462/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:24.382998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851f7de-0d4e-467f-b258-5be5d675e793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8800104fcdd4b7b8eed6332dd249c80, user_id=37534369e7d644d78af7e3a93ab25462, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:24.389926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851f7de-0d4e-467f-b258-5be5d675e793 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:24.427769 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1620/3209] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:59:24 2022] PUT /identity/v3/projects/c8800104fcdd4b7b8eed6332dd249c80/users/37534369e7d644d78af7e3a93ab25462/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:24.430220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430645d8-02ff-4a64-811a-6aa1b2930f9a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:24.430827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430645d8-02ff-4a64-811a-6aa1b2930f9a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:24.431218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430645d8-02ff-4a64-811a-6aa1b2930f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:24.529959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-430645d8-02ff-4a64-811a-6aa1b2930f9a None None] MFA Rules not processed for user `37534369e7d644d78af7e3a93ab25462`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:24.535874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-430645d8-02ff-4a64-811a-6aa1b2930f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:24.676905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:24.690321 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1590/3210] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:24 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:25.879369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851f7de-0d4e-467f-b258-5be5d675e793 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:25.881542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c851f7de-0d4e-467f-b258-5be5d675e793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:25.963405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851f7de-0d4e-467f-b258-5be5d675e793 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:25.970677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6deaf-53d7-4fcb-b0e9-19ace2a93502 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:25.971546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6deaf-53d7-4fcb-b0e9-19ace2a93502 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:25.971967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6deaf-53d7-4fcb-b0e9-19ace2a93502 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:25.972383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6deaf-53d7-4fcb-b0e9-19ace2a93502 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:25.983133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6deaf-53d7-4fcb-b0e9-19ace2a93502 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:25.985406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6deaf-53d7-4fcb-b0e9-19ace2a93502 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:26.040499 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1621/3211] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:25 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:26.084742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430645d8-02ff-4a64-811a-6aa1b2930f9a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:26.278772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430645d8-02ff-4a64-811a-6aa1b2930f9a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:26.294166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:26.295085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:26.295508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:26.295952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] PATH_INFO: `/v3/users/37534369e7d644d78af7e3a93ab25462` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:26.301079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37534369e7d644d78af7e3a93ab25462)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:26.303356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:26.411270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37534369e7d644d78af7e3a93ab25462'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:26.424920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37534369e7d644d78af7e3a93ab25462'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:26.462615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37534369e7d644d78af7e3a93ab25462'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:26.478233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37534369e7d644d78af7e3a93ab25462'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:26.479031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37534369e7d644d78af7e3a93ab25462'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:26.490772 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1591/3212] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:59:26 2022] DELETE /identity/v3/users/37534369e7d644d78af7e3a93ab25462 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:26.496132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad828e5-c055-41da-a5b5-bbe26377cc49 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:26.496132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad828e5-c055-41da-a5b5-bbe26377cc49 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:26.496132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad828e5-c055-41da-a5b5-bbe26377cc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:26.540502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9ad828e5-c055-41da-a5b5-bbe26377cc49 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:26.543366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9ad828e5-c055-41da-a5b5-bbe26377cc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:26.562613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:26.893381 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1622/3213] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:26.906458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:26.910910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:27.069528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b31dfd-0f80-4b24-919b-525642c46144 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:27.077777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7629af-0ba8-473d-8375-09113d1d8648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:27.078397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7629af-0ba8-473d-8375-09113d1d8648 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:27.078665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7629af-0ba8-473d-8375-09113d1d8648 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:27.079074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7629af-0ba8-473d-8375-09113d1d8648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:27.100581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7629af-0ba8-473d-8375-09113d1d8648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:27.102154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7629af-0ba8-473d-8375-09113d1d8648 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:27.127114 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1592/3214] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:27.768228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad828e5-c055-41da-a5b5-bbe26377cc49 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:27.769375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9ad828e5-c055-41da-a5b5-bbe26377cc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:27.794463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad828e5-c055-41da-a5b5-bbe26377cc49 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:27.812447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:27.813118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:27.813400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:27.813713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] PATH_INFO: `/v3/projects/c8800104fcdd4b7b8eed6332dd249c80` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:27.830692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8800104fcdd4b7b8eed6332dd249c80)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:27.841778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:27.955213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8800104fcdd4b7b8eed6332dd249c80', 'request_id': 'req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:27.974961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8800104fcdd4b7b8eed6332dd249c80', 'request_id': 'req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:27.985097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8800104fcdd4b7b8eed6332dd249c80', 'request_id': 'req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:28.004025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] The token cache is being invalidate because project c8800104fcdd4b7b8eed6332dd249c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:59:28.004500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:28.005212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d8eb9dc0-2e6e-424d-b0cf-9689ebb60676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:28.007251 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1623/3215] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:27 2022] DELETE /identity/v3/projects/c8800104fcdd4b7b8eed6332dd249c80 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:28.012775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462ef8d-9bed-4e23-a52d-da2cbac770c6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:28.014049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462ef8d-9bed-4e23-a52d-da2cbac770c6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:28.014461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462ef8d-9bed-4e23-a52d-da2cbac770c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:28.050875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a462ef8d-9bed-4e23-a52d-da2cbac770c6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:28.052462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a462ef8d-9bed-4e23-a52d-da2cbac770c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:28.082698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:28.188878 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1593/3216] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:28.198764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9cb8f-f1ae-461c-a202-bebe92b735d1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:28.198948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9cb8f-f1ae-461c-a202-bebe92b735d1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:28.199211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9cb8f-f1ae-461c-a202-bebe92b735d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:28.230646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e0b9cb8f-f1ae-461c-a202-bebe92b735d1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:28.233374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e0b9cb8f-f1ae-461c-a202-bebe92b735d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:28.245356 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1624/3217] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:28.249515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462ef8d-9bed-4e23-a52d-da2cbac770c6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:28.261611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462ef8d-9bed-4e23-a52d-da2cbac770c6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:28.271319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6492f5-54be-4616-9571-52fd151d7d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:28.271319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6492f5-54be-4616-9571-52fd151d7d01 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:28.271319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6492f5-54be-4616-9571-52fd151d7d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:28.271319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6492f5-54be-4616-9571-52fd151d7d01 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:28.271743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6492f5-54be-4616-9571-52fd151d7d01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:59:28.271743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6492f5-54be-4616-9571-52fd151d7d01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:28.271743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6492f5-54be-4616-9571-52fd151d7d01 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:28.290174 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1594/3218] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:59:28.292831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9cb8f-f1ae-461c-a202-bebe92b735d1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:28.304914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9cb8f-f1ae-461c-a202-bebe92b735d1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:28.318517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb780d-2aaa-4252-b88c-c1e3d2cfd218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:28.319245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb780d-2aaa-4252-b88c-c1e3d2cfd218 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:28.319520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb780d-2aaa-4252-b88c-c1e3d2cfd218 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:28.319781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb780d-2aaa-4252-b88c-c1e3d2cfd218 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:28.320425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb780d-2aaa-4252-b88c-c1e3d2cfd218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:28.322809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb780d-2aaa-4252-b88c-c1e3d2cfd218 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:28.484472 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1625/3219] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:59:28 2022] POST /identity/v3/projects => generated 397 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:59:28.488984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6492f5-54be-4616-9571-52fd151d7d01 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:28.521676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6492f5-54be-4616-9571-52fd151d7d01 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:28.674539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1ace6-8ca7-4150-ae99-b54833cd255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:28.674661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1ace6-8ca7-4150-ae99-b54833cd255d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:28.674900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1ace6-8ca7-4150-ae99-b54833cd255d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:28.675169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1ace6-8ca7-4150-ae99-b54833cd255d admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:28.676395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1ace6-8ca7-4150-ae99-b54833cd255d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:28.683239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1ace6-8ca7-4150-ae99-b54833cd255d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:28.735155 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1595/3220] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:59:28 2022] POST /identity/v3/users => generated 299 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:59:28.737619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb780d-2aaa-4252-b88c-c1e3d2cfd218 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:28.751403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb780d-2aaa-4252-b88c-c1e3d2cfd218 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:28.765113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb0e90-09e1-4a4f-83c8-40e06d4b029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:28.765735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb0e90-09e1-4a4f-83c8-40e06d4b029e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:28.766008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb0e90-09e1-4a4f-83c8-40e06d4b029e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:28.766277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb0e90-09e1-4a4f-83c8-40e06d4b029e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:28.766920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb0e90-09e1-4a4f-83c8-40e06d4b029e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:28.769570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb0e90-09e1-4a4f-83c8-40e06d4b029e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:28.796919 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1626/3221] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:28.801287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1ace6-8ca7-4150-ae99-b54833cd255d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:28.818352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1ace6-8ca7-4150-ae99-b54833cd255d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:28.870243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34099c11-aed0-4d77-a46c-ff17369704f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:28.870243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34099c11-aed0-4d77-a46c-ff17369704f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:28.870243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34099c11-aed0-4d77-a46c-ff17369704f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:28.870243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34099c11-aed0-4d77-a46c-ff17369704f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:28.875296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34099c11-aed0-4d77-a46c-ff17369704f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:28.875296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34099c11-aed0-4d77-a46c-ff17369704f7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:28.894459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb0e90-09e1-4a4f-83c8-40e06d4b029e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:28.894899 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1596/3222] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:28.906466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb0e90-09e1-4a4f-83c8-40e06d4b029e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:28.911528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dcd67-5bf1-4f22-85b4-ad58e83af689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:28.912304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dcd67-5bf1-4f22-85b4-ad58e83af689 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:28.912647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dcd67-5bf1-4f22-85b4-ad58e83af689 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:28.912986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dcd67-5bf1-4f22-85b4-ad58e83af689 admin admin] PATH_INFO: `/v3/projects/39d4d95d057544d2a6aa821f38be4e9a/users/598073dd466a4417a1a7e31ba2dc95d2/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:28.943503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50dcd67-5bf1-4f22-85b4-ad58e83af689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d4d95d057544d2a6aa821f38be4e9a, user_id=598073dd466a4417a1a7e31ba2dc95d2, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:28.947859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50dcd67-5bf1-4f22-85b4-ad58e83af689 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:28.970551 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1627/3223] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:59:28 2022] PUT /identity/v3/projects/39d4d95d057544d2a6aa821f38be4e9a/users/598073dd466a4417a1a7e31ba2dc95d2/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:28.975781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d820e-532e-4793-a347-fd8bc53326ce None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:28.976064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d820e-532e-4793-a347-fd8bc53326ce None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:28.976336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d820e-532e-4793-a347-fd8bc53326ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:29.021732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b21d820e-532e-4793-a347-fd8bc53326ce None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:29.023558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b21d820e-532e-4793-a347-fd8bc53326ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:29.171889 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1597/3224] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:29.193675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dcd67-5bf1-4f22-85b4-ad58e83af689 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:29.195526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a50dcd67-5bf1-4f22-85b4-ad58e83af689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:29.292419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:29.351665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dcd67-5bf1-4f22-85b4-ad58e83af689 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:29.363360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9154a-14a9-4e49-8e90-bfaeb41de96c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:29.363635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9154a-14a9-4e49-8e90-bfaeb41de96c service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:29.364003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9154a-14a9-4e49-8e90-bfaeb41de96c service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:29.364466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9154a-14a9-4e49-8e90-bfaeb41de96c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:29.381912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9154a-14a9-4e49-8e90-bfaeb41de96c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:29.384209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9154a-14a9-4e49-8e90-bfaeb41de96c service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:29.413416 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1628/3225] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:30.557166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2f00d-d546-4b59-a090-284a6212ce56 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:30.558271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2f00d-d546-4b59-a090-284a6212ce56 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:30.558754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2f00d-d546-4b59-a090-284a6212ce56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:30.600204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ebd2f00d-d546-4b59-a090-284a6212ce56 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:30.605190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ebd2f00d-d546-4b59-a090-284a6212ce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:30.660044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:30.751133 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1598/3226] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:59:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 20:59:30.757679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9154a-14a9-4e49-8e90-bfaeb41de96c service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:30.759287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a6e9154a-14a9-4e49-8e90-bfaeb41de96c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:30.936874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9154a-14a9-4e49-8e90-bfaeb41de96c service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:30.949770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3096f7-5573-419e-8b15-0d3d46a265a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:30.950725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3096f7-5573-419e-8b15-0d3d46a265a7 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:30.951228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3096f7-5573-419e-8b15-0d3d46a265a7 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:30.951735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3096f7-5573-419e-8b15-0d3d46a265a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:30.971449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3096f7-5573-419e-8b15-0d3d46a265a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:30.973740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3096f7-5573-419e-8b15-0d3d46a265a7 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:30.993077 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1629/3227] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:59:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:31.818875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2f00d-d546-4b59-a090-284a6212ce56 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:31.820488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ebd2f00d-d546-4b59-a090-284a6212ce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:31.862331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2f00d-d546-4b59-a090-284a6212ce56 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:31.893423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:31.896167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:31.899012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:31.901001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] PATH_INFO: `/v3/users/a8d437aff8aa486ca43edb2605a57b4e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:31.909016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8d437aff8aa486ca43edb2605a57b4e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:31.926245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:32.059641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8d437aff8aa486ca43edb2605a57b4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:32.077522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8d437aff8aa486ca43edb2605a57b4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:32.100297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8d437aff8aa486ca43edb2605a57b4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:32.128354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8d437aff8aa486ca43edb2605a57b4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:32.129224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cf965e03-c0fb-407d-b933-3fee433cd946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8d437aff8aa486ca43edb2605a57b4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:32.140499 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1599/3228] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:59:31 2022] DELETE /identity/v3/users/a8d437aff8aa486ca43edb2605a57b4e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:33.928640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3096f7-5573-419e-8b15-0d3d46a265a7 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:33.957400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3096f7-5573-419e-8b15-0d3d46a265a7 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:34.162093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:34.165347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:34.166825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:34.167229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] PATH_INFO: `/v3/projects/7e45e841218949e688b3deee07fd25ed` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:34.172668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e45e841218949e688b3deee07fd25ed)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:34.179006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:34.310172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e45e841218949e688b3deee07fd25ed', 'request_id': 'req-ef52cfe7-059d-462d-b2ec-999ba87a5f67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:34.334550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e45e841218949e688b3deee07fd25ed', 'request_id': 'req-ef52cfe7-059d-462d-b2ec-999ba87a5f67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:34.349973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e45e841218949e688b3deee07fd25ed', 'request_id': 'req-ef52cfe7-059d-462d-b2ec-999ba87a5f67'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:34.361121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:34.380885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] The token cache is being invalidate because project 7e45e841218949e688b3deee07fd25ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:59:34.381609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:34.382271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ef52cfe7-059d-462d-b2ec-999ba87a5f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:34.384071 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1630/3229] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:33 2022] DELETE /identity/v3/projects/7e45e841218949e688b3deee07fd25ed => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:34.390046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b4cba-7fe9-4ff7-9382-a5d9ae5cd605 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:34.391210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b4cba-7fe9-4ff7-9382-a5d9ae5cd605 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:34.391650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b4cba-7fe9-4ff7-9382-a5d9ae5cd605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:34.443043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a06b4cba-7fe9-4ff7-9382-a5d9ae5cd605 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:34.444610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a06b4cba-7fe9-4ff7-9382-a5d9ae5cd605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:34.643694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45894cf3-1dba-4921-afe7-51735e80965c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:34.645832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45894cf3-1dba-4921-afe7-51735e80965c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:34.646289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45894cf3-1dba-4921-afe7-51735e80965c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:34.649839 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1600/3230] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:34.698057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-45894cf3-1dba-4921-afe7-51735e80965c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:34.699708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-45894cf3-1dba-4921-afe7-51735e80965c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:34.743792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b4cba-7fe9-4ff7-9382-a5d9ae5cd605 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:34.744303 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1631/3231] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:34.769709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b4cba-7fe9-4ff7-9382-a5d9ae5cd605 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:34.787441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcf3da-2ba6-4bbb-9531-50e046785562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:34.788065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcf3da-2ba6-4bbb-9531-50e046785562 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:34.788338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcf3da-2ba6-4bbb-9531-50e046785562 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:34.788606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcf3da-2ba6-4bbb-9531-50e046785562 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:34.789285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcf3da-2ba6-4bbb-9531-50e046785562 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:59:34.789583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcf3da-2ba6-4bbb-9531-50e046785562 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:34.791248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcf3da-2ba6-4bbb-9531-50e046785562 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:34.829991 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1601/3232] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:59:34.837967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45894cf3-1dba-4921-afe7-51735e80965c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:34.848894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45894cf3-1dba-4921-afe7-51735e80965c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:34.883678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4132f92-0f9b-40d4-a936-702ee9d2e14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:34.884297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132f92-0f9b-40d4-a936-702ee9d2e14d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:34.884573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132f92-0f9b-40d4-a936-702ee9d2e14d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:34.884839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132f92-0f9b-40d4-a936-702ee9d2e14d admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:34.885515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4132f92-0f9b-40d4-a936-702ee9d2e14d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:34.887584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4132f92-0f9b-40d4-a936-702ee9d2e14d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:34.937435 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1632/3233] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:59:34 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:59:34.941788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcf3da-2ba6-4bbb-9531-50e046785562 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:34.959562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcf3da-2ba6-4bbb-9531-50e046785562 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:35.056133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:35.150194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a53f2-a918-4046-b2c2-9ddb41957732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:35.150194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a53f2-a918-4046-b2c2-9ddb41957732 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:35.150194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a53f2-a918-4046-b2c2-9ddb41957732 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:35.158002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a53f2-a918-4046-b2c2-9ddb41957732 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:35.158002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a53f2-a918-4046-b2c2-9ddb41957732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:35.159255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a53f2-a918-4046-b2c2-9ddb41957732 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:35.255167 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1602/3234] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:59:34 2022] POST /identity/v3/users => generated 290 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:59:35.256202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4132f92-0f9b-40d4-a936-702ee9d2e14d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:35.293264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4132f92-0f9b-40d4-a936-702ee9d2e14d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:35.333403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9848c-1a6b-498c-a25d-28fc5598e804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:35.334743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9848c-1a6b-498c-a25d-28fc5598e804 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:35.335089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9848c-1a6b-498c-a25d-28fc5598e804 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:35.335277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9848c-1a6b-498c-a25d-28fc5598e804 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:35.350280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9848c-1a6b-498c-a25d-28fc5598e804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:35.350280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9848c-1a6b-498c-a25d-28fc5598e804 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:35.375510 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1633/3235] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:35.379997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a53f2-a918-4046-b2c2-9ddb41957732 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:35.403793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a53f2-a918-4046-b2c2-9ddb41957732 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:35.422126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250a5a5-1208-47e5-88d1-701ffe37c777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:35.422758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250a5a5-1208-47e5-88d1-701ffe37c777 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:35.423034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250a5a5-1208-47e5-88d1-701ffe37c777 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:35.423301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250a5a5-1208-47e5-88d1-701ffe37c777 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:35.423946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250a5a5-1208-47e5-88d1-701ffe37c777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:35.430227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250a5a5-1208-47e5-88d1-701ffe37c777 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:35.458787 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1603/3236] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:35.462979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9848c-1a6b-498c-a25d-28fc5598e804 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:35.485096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9848c-1a6b-498c-a25d-28fc5598e804 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:35.505098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52472767-259e-433b-8e08-36c608e2b71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:35.515279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472767-259e-433b-8e08-36c608e2b71c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:35.523791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472767-259e-433b-8e08-36c608e2b71c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:35.524123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472767-259e-433b-8e08-36c608e2b71c admin admin] PATH_INFO: `/v3/projects/6b955a7e7fb04b569773c0453624911a/users/6925000aee9d499a85c6dc1282d09993/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:35.578043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52472767-259e-433b-8e08-36c608e2b71c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b955a7e7fb04b569773c0453624911a, user_id=6925000aee9d499a85c6dc1282d09993, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:35.580961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52472767-259e-433b-8e08-36c608e2b71c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:35.625977 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1634/3237] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:59:35 2022] PUT /identity/v3/projects/6b955a7e7fb04b569773c0453624911a/users/6925000aee9d499a85c6dc1282d09993/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:35.628578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a314996-5bc1-492a-8970-249311c27f8a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:35.628578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a314996-5bc1-492a-8970-249311c27f8a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:35.628578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a314996-5bc1-492a-8970-249311c27f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:35.684697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9a314996-5bc1-492a-8970-249311c27f8a None None] MFA Rules not processed for user `6925000aee9d499a85c6dc1282d09993`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:35.684697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9a314996-5bc1-492a-8970-249311c27f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:35.845915 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1604/3238] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:35.857755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427febca-18ca-4561-94ee-6ab1e103ff1e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:35.858031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427febca-18ca-4561-94ee-6ab1e103ff1e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:35.858299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427febca-18ca-4561-94ee-6ab1e103ff1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:35.930142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-427febca-18ca-4561-94ee-6ab1e103ff1e None None] MFA Rules not processed for user `6925000aee9d499a85c6dc1282d09993`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:35.931764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-427febca-18ca-4561-94ee-6ab1e103ff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:35.958090 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1635/3239] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:39.509455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a2140-d61a-4614-88c7-b513ad07b7ec None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:39.510011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a2140-d61a-4614-88c7-b513ad07b7ec None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:39.510011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a2140-d61a-4614-88c7-b513ad07b7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:39.593876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4f9a2140-d61a-4614-88c7-b513ad07b7ec None None] MFA Rules not processed for user `598073dd466a4417a1a7e31ba2dc95d2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:39.595065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4f9a2140-d61a-4614-88c7-b513ad07b7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:39.748998 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1605/3240] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:42.301161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427febca-18ca-4561-94ee-6ab1e103ff1e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:42.302763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-427febca-18ca-4561-94ee-6ab1e103ff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:42.414029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427febca-18ca-4561-94ee-6ab1e103ff1e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:42.420683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfd4e6-3449-45c0-85c1-abd9ad3a9957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:42.421363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfd4e6-3449-45c0-85c1-abd9ad3a9957 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:42.421611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfd4e6-3449-45c0-85c1-abd9ad3a9957 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:42.421870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfd4e6-3449-45c0-85c1-abd9ad3a9957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:42.430883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfd4e6-3449-45c0-85c1-abd9ad3a9957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:42.432730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfd4e6-3449-45c0-85c1-abd9ad3a9957 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:42.444361 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1636/3241] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:42.664645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a2140-d61a-4614-88c7-b513ad07b7ec None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:42.698360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a2140-d61a-4614-88c7-b513ad07b7ec None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:42.726953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13c5e2-bed2-4c56-ab78-af89921b356e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:42.727795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13c5e2-bed2-4c56-ab78-af89921b356e service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:42.728176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13c5e2-bed2-4c56-ab78-af89921b356e service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:42.728548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13c5e2-bed2-4c56-ab78-af89921b356e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:42.750315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13c5e2-bed2-4c56-ab78-af89921b356e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:42.752618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13c5e2-bed2-4c56-ab78-af89921b356e service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:42.770800 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1606/3242] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:42 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:42.908420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728c7c3-85cf-4438-b4fe-75cabdebab87 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:42.908977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728c7c3-85cf-4438-b4fe-75cabdebab87 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:42.909375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728c7c3-85cf-4438-b4fe-75cabdebab87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:42.941774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c728c7c3-85cf-4438-b4fe-75cabdebab87 None None] MFA Rules not processed for user `5d93a680570f4d23b1f6df534bc082b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:42.943210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c728c7c3-85cf-4438-b4fe-75cabdebab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:43.013504 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1637/3243] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:49.086432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1b298-8c91-4444-aff9-9d1fdbe10358 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:49.087040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1b298-8c91-4444-aff9-9d1fdbe10358 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:49.087040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1b298-8c91-4444-aff9-9d1fdbe10358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:49.110479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f5c1b298-8c91-4444-aff9-9d1fdbe10358 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:49.112817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f5c1b298-8c91-4444-aff9-9d1fdbe10358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:49.127761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:49.245655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728c7c3-85cf-4438-b4fe-75cabdebab87 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:49.249163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c728c7c3-85cf-4438-b4fe-75cabdebab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:49.258855 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1607/3244] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 20:59:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:49.278063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:49.336911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728c7c3-85cf-4438-b4fe-75cabdebab87 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:49.343627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b444bad-93e0-45be-b3f9-49dfde1183d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:49.344500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b444bad-93e0-45be-b3f9-49dfde1183d8 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:49.344911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b444bad-93e0-45be-b3f9-49dfde1183d8 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:49.345379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b444bad-93e0-45be-b3f9-49dfde1183d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:49.355562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b444bad-93e0-45be-b3f9-49dfde1183d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:49.357457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b444bad-93e0-45be-b3f9-49dfde1183d8 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:49.375086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1638/3245] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:59:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 20:59:54.335056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1b298-8c91-4444-aff9-9d1fdbe10358 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:54.337707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f5c1b298-8c91-4444-aff9-9d1fdbe10358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:54.455345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1b298-8c91-4444-aff9-9d1fdbe10358 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:54.461926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:54.462559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:54.462812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:54.463091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] PATH_INFO: `/v3/users/598073dd466a4417a1a7e31ba2dc95d2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:54.464204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=598073dd466a4417a1a7e31ba2dc95d2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:54.467056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:54.571169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '598073dd466a4417a1a7e31ba2dc95d2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:54.590505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '598073dd466a4417a1a7e31ba2dc95d2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:54.621990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '598073dd466a4417a1a7e31ba2dc95d2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:54.635946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '598073dd466a4417a1a7e31ba2dc95d2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:54.636686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '598073dd466a4417a1a7e31ba2dc95d2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:54.646565 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1608/3246] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:59:54 2022] DELETE /identity/v3/users/598073dd466a4417a1a7e31ba2dc95d2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:54.746091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42565-104c-4f3b-8ce8-a4b7893e3094 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:54.747251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42565-104c-4f3b-8ce8-a4b7893e3094 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:54.747527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42565-104c-4f3b-8ce8-a4b7893e3094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:54.774092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6cc42565-104c-4f3b-8ce8-a4b7893e3094 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:54.775725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6cc42565-104c-4f3b-8ce8-a4b7893e3094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:54.857711 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1639/3247] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 20:59:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:54.877959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:54.879621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:54.909914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc42565-104c-4f3b-8ce8-a4b7893e3094 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:54.915349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6cc42565-104c-4f3b-8ce8-a4b7893e3094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:54.991676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:55.036167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bbd4c-659d-41fe-b3a5-6607195ea7d8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:55.042364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:55.043237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.043584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.043889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] PATH_INFO: `/v3/projects/39d4d95d057544d2a6aa821f38be4e9a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.045050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] RBAC: Authorizing `identity:delete_project(project_id=39d4d95d057544d2a6aa821f38be4e9a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:55.047132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:55.052078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc42565-104c-4f3b-8ce8-a4b7893e3094 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:55.063999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bb95a-6176-4650-8e9e-e03f04e1e56b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:55.064778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bb95a-6176-4650-8e9e-e03f04e1e56b service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.065134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bb95a-6176-4650-8e9e-e03f04e1e56b service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.065463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bb95a-6176-4650-8e9e-e03f04e1e56b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.081892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bb95a-6176-4650-8e9e-e03f04e1e56b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:55.083945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bb95a-6176-4650-8e9e-e03f04e1e56b service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:55.094262 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1640/3248] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 20:59:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:55.107105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39d4d95d057544d2a6aa821f38be4e9a', 'request_id': 'req-adbb85f2-fc43-4511-9dc0-16075500645b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:55.126448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39d4d95d057544d2a6aa821f38be4e9a', 'request_id': 'req-adbb85f2-fc43-4511-9dc0-16075500645b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:55.137371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39d4d95d057544d2a6aa821f38be4e9a', 'request_id': 'req-adbb85f2-fc43-4511-9dc0-16075500645b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:55.147962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] The token cache is being invalidate because project 39d4d95d057544d2a6aa821f38be4e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 20:59:55.148310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:55.149163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-adbb85f2-fc43-4511-9dc0-16075500645b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 20:59:55.151226 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1609/3249] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:54 2022] DELETE /identity/v3/projects/39d4d95d057544d2a6aa821f38be4e9a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 20:59:55.157423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f87c41-22f6-448e-855c-9b0e2adf7027 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.157793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f87c41-22f6-448e-855c-9b0e2adf7027 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.158142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f87c41-22f6-448e-855c-9b0e2adf7027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.182253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-70f87c41-22f6-448e-855c-9b0e2adf7027 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:55.183525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-70f87c41-22f6-448e-855c-9b0e2adf7027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:55.241229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:55.280197 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1641/3250] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:55.288374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642baa9-bf4d-4031-86bc-b39dec7da20b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.288824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642baa9-bf4d-4031-86bc-b39dec7da20b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.289261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642baa9-bf4d-4031-86bc-b39dec7da20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.368877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7642baa9-bf4d-4031-86bc-b39dec7da20b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:55.377109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7642baa9-bf4d-4031-86bc-b39dec7da20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:55.387628 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1610/3251] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:55.392078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f87c41-22f6-448e-855c-9b0e2adf7027 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:55.403186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f87c41-22f6-448e-855c-9b0e2adf7027 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:55.415086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b414d-37df-49e3-9436-4492cd5960e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:55.415713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b414d-37df-49e3-9436-4492cd5960e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.415988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b414d-37df-49e3-9436-4492cd5960e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.416257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b414d-37df-49e3-9436-4492cd5960e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.416914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b414d-37df-49e3-9436-4492cd5960e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 20:59:55.417295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b414d-37df-49e3-9436-4492cd5960e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:55.419431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b414d-37df-49e3-9436-4492cd5960e9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:55.435299 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1642/3252] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 20:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 20:59:55.440799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642baa9-bf4d-4031-86bc-b39dec7da20b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:55.451434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642baa9-bf4d-4031-86bc-b39dec7da20b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:55.458233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeed7d-6c88-41a7-8163-9ae6bc4e0aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:55.459344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeed7d-6c88-41a7-8163-9ae6bc4e0aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.459954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeed7d-6c88-41a7-8163-9ae6bc4e0aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.460510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeed7d-6c88-41a7-8163-9ae6bc4e0aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.461475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eeed7d-6c88-41a7-8163-9ae6bc4e0aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:55.464686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eeed7d-6c88-41a7-8163-9ae6bc4e0aa0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:55.551797 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1611/3253] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 20:59:55 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 20:59:55.559802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b414d-37df-49e3-9436-4492cd5960e9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:55.590581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b414d-37df-49e3-9436-4492cd5960e9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:55.693261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbcb62-94d0-494e-84b0-a4c2c5973862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:55.693935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbcb62-94d0-494e-84b0-a4c2c5973862 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.694216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbcb62-94d0-494e-84b0-a4c2c5973862 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.694483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbcb62-94d0-494e-84b0-a4c2c5973862 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.695107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbcb62-94d0-494e-84b0-a4c2c5973862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:55.697080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbcb62-94d0-494e-84b0-a4c2c5973862 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:55.752534 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1643/3254] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 20:59:55 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 20:59:55.755446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeed7d-6c88-41a7-8163-9ae6bc4e0aa0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:55.772638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeed7d-6c88-41a7-8163-9ae6bc4e0aa0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:55.779954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edabaa5-00dd-4cb1-8c42-4bea4ebdb869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:55.781067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edabaa5-00dd-4cb1-8c42-4bea4ebdb869 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.781601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edabaa5-00dd-4cb1-8c42-4bea4ebdb869 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.782047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edabaa5-00dd-4cb1-8c42-4bea4ebdb869 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.782945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edabaa5-00dd-4cb1-8c42-4bea4ebdb869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:55.785283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edabaa5-00dd-4cb1-8c42-4bea4ebdb869 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:55.832548 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1612/3255] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:55.836802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbcb62-94d0-494e-84b0-a4c2c5973862 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:55.855142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbcb62-94d0-494e-84b0-a4c2c5973862 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:55.862092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e18ab-5289-44ca-ba44-480de58e2ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:55.863163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e18ab-5289-44ca-ba44-480de58e2ffc admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.863599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e18ab-5289-44ca-ba44-480de58e2ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.864019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e18ab-5289-44ca-ba44-480de58e2ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.864827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e18ab-5289-44ca-ba44-480de58e2ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:55.867493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e18ab-5289-44ca-ba44-480de58e2ffc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:55.884697 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1644/3256] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 20:59:55.885821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edabaa5-00dd-4cb1-8c42-4bea4ebdb869 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:55.897825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edabaa5-00dd-4cb1-8c42-4bea4ebdb869 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:55.904227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b6073-41ea-45f4-9df3-eb25affb1e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:55.904897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b6073-41ea-45f4-9df3-eb25affb1e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.905300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b6073-41ea-45f4-9df3-eb25affb1e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.905712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b6073-41ea-45f4-9df3-eb25affb1e73 admin admin] PATH_INFO: `/v3/projects/12a874b752564923ad6a4747c7a44d61/users/84c19569dc6d49f4a29c814ef921e144/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.929135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b6073-41ea-45f4-9df3-eb25affb1e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a874b752564923ad6a4747c7a44d61, user_id=84c19569dc6d49f4a29c814ef921e144, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:55.932263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b6073-41ea-45f4-9df3-eb25affb1e73 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:55.958475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba19b-3b1b-4b90-b046-6d4a41e2e9f7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:55.958760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba19b-3b1b-4b90-b046-6d4a41e2e9f7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:55.959030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba19b-3b1b-4b90-b046-6d4a41e2e9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:55.962796 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1613/3257] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 20:59:55 2022] PUT /identity/v3/projects/12a874b752564923ad6a4747c7a44d61/users/84c19569dc6d49f4a29c814ef921e144/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 20:59:55.984994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-70eba19b-3b1b-4b90-b046-6d4a41e2e9f7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 20:59:55.986372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-70eba19b-3b1b-4b90-b046-6d4a41e2e9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:56.062628 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1645/3258] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 20:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 20:59:56.073463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b6073-41ea-45f4-9df3-eb25affb1e73 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:56.075902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-864b6073-41ea-45f4-9df3-eb25affb1e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:56.178794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b6073-41ea-45f4-9df3-eb25affb1e73 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:56.183088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2926107-ad5a-4818-909e-157929faa305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:56.183978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2926107-ad5a-4818-909e-157929faa305 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:56.184394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2926107-ad5a-4818-909e-157929faa305 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:56.184791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2926107-ad5a-4818-909e-157929faa305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:56.201153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2926107-ad5a-4818-909e-157929faa305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:56.202593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2926107-ad5a-4818-909e-157929faa305 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:56.214845 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1614/3259] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 20:59:59.072983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eba19b-3b1b-4b90-b046-6d4a41e2e9f7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 20:59:59.075537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-70eba19b-3b1b-4b90-b046-6d4a41e2e9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:59.145599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:59.185199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eba19b-3b1b-4b90-b046-6d4a41e2e9f7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 20:59:59.192482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd41587-5f3c-484d-a6e4-d70e2144fee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 20:59:59.193248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd41587-5f3c-484d-a6e4-d70e2144fee3 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 20:59:59.193582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd41587-5f3c-484d-a6e4-d70e2144fee3 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 20:59:59.193911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd41587-5f3c-484d-a6e4-d70e2144fee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 20:59:59.195900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6bd41587-5f3c-484d-a6e4-d70e2144fee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 20:59:59.293738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd41587-5f3c-484d-a6e4-d70e2144fee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 20:59:59.295876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd41587-5f3c-484d-a6e4-d70e2144fee3 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 20:59:59.323407 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1646/3260] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 20:59:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:00.865242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2926107-ad5a-4818-909e-157929faa305 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:00.887820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2926107-ad5a-4818-909e-157929faa305 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:00.898965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77239bab-847a-423b-bea5-a71a24eaf2ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:00.900346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77239bab-847a-423b-bea5-a71a24eaf2ab service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:00.900834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77239bab-847a-423b-bea5-a71a24eaf2ab service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:00.901440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77239bab-847a-423b-bea5-a71a24eaf2ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:00.904802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-77239bab-847a-423b-bea5-a71a24eaf2ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:00.915812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:01.003336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77239bab-847a-423b-bea5-a71a24eaf2ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:01.005319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77239bab-847a-423b-bea5-a71a24eaf2ab service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:01.019550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1615/3261] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:02.714282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73e40e-d701-4840-b9dd-0294d44c36de None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:02.715481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73e40e-d701-4840-b9dd-0294d44c36de None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:02.715481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73e40e-d701-4840-b9dd-0294d44c36de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:02.768915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0d73e40e-d701-4840-b9dd-0294d44c36de None None] MFA Rules not processed for user `84c19569dc6d49f4a29c814ef921e144`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:02.770181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0d73e40e-d701-4840-b9dd-0294d44c36de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:02.853775 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1647/3262] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:02.859470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77239bab-847a-423b-bea5-a71a24eaf2ab service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:02.882011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77239bab-847a-423b-bea5-a71a24eaf2ab service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:02.891076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398b81c-676e-4f84-b46e-f27ad0898293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:02.891396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398b81c-676e-4f84-b46e-f27ad0898293 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:02.891718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398b81c-676e-4f84-b46e-f27ad0898293 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:02.892093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398b81c-676e-4f84-b46e-f27ad0898293 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:02.892955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398b81c-676e-4f84-b46e-f27ad0898293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:02.895214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398b81c-676e-4f84-b46e-f27ad0898293 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:02.957840 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1616/3263] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:00:02 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:00:02.961993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73e40e-d701-4840-b9dd-0294d44c36de None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:02.974722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73e40e-d701-4840-b9dd-0294d44c36de None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:03.073361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748943aa-6eb7-456a-94cf-7746fb804c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:03.074765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748943aa-6eb7-456a-94cf-7746fb804c12 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:03.075143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748943aa-6eb7-456a-94cf-7746fb804c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:03.075513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748943aa-6eb7-456a-94cf-7746fb804c12 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:03.076342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748943aa-6eb7-456a-94cf-7746fb804c12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:03.078654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748943aa-6eb7-456a-94cf-7746fb804c12 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:03.111047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:03.130494 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1648/3264] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:00:02 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:00:03.134652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398b81c-676e-4f84-b46e-f27ad0898293 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:03.159341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398b81c-676e-4f84-b46e-f27ad0898293 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:03.169214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2eaf9-5db0-4d77-b8d3-be2fa009a19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:03.171044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2eaf9-5db0-4d77-b8d3-be2fa009a19c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:03.171820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2eaf9-5db0-4d77-b8d3-be2fa009a19c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:03.171820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2eaf9-5db0-4d77-b8d3-be2fa009a19c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:03.172251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2eaf9-5db0-4d77-b8d3-be2fa009a19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:03.173894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2eaf9-5db0-4d77-b8d3-be2fa009a19c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:03.199003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748943aa-6eb7-456a-94cf-7746fb804c12 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:03.216578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748943aa-6eb7-456a-94cf-7746fb804c12 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:03.223045 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1617/3265] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:00:03.238457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e78fd-96b3-4e96-b1c7-7cfa58b98cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:03.240364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e78fd-96b3-4e96-b1c7-7cfa58b98cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:03.240972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e78fd-96b3-4e96-b1c7-7cfa58b98cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:03.242578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e78fd-96b3-4e96-b1c7-7cfa58b98cf9 admin admin] PATH_INFO: `/v3/projects/562fbd30b3b547ad84cef4c396b0b68e/users/03fc75503c7b426cba42614c38545fd3/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:03.312237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e78fd-96b3-4e96-b1c7-7cfa58b98cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=562fbd30b3b547ad84cef4c396b0b68e, user_id=03fc75503c7b426cba42614c38545fd3, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:03.315351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e78fd-96b3-4e96-b1c7-7cfa58b98cf9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:03.339291 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1649/3266] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:00:03 2022] PUT /identity/v3/projects/562fbd30b3b547ad84cef4c396b0b68e/users/03fc75503c7b426cba42614c38545fd3/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:03.344065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2eaf9-5db0-4d77-b8d3-be2fa009a19c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:03.361758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2eaf9-5db0-4d77-b8d3-be2fa009a19c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:03.471739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d888f-59e2-408a-8d23-2e82d9037f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:03.472842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d888f-59e2-408a-8d23-2e82d9037f7a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:03.473344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d888f-59e2-408a-8d23-2e82d9037f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:03.473760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d888f-59e2-408a-8d23-2e82d9037f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:03.474597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d888f-59e2-408a-8d23-2e82d9037f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:03.476561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d888f-59e2-408a-8d23-2e82d9037f7a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:03.501863 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1618/3267] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:03.506251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e78fd-96b3-4e96-b1c7-7cfa58b98cf9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:03.525442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e78fd-96b3-4e96-b1c7-7cfa58b98cf9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:03.538113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aef620-d4a6-4f33-b360-2d75c886d9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:03.539444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aef620-d4a6-4f33-b360-2d75c886d9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:03.539444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aef620-d4a6-4f33-b360-2d75c886d9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:03.539645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aef620-d4a6-4f33-b360-2d75c886d9bc admin admin] PATH_INFO: `/v3/projects/562fbd30b3b547ad84cef4c396b0b68e/users/03fc75503c7b426cba42614c38545fd3/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:03.555619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aef620-d4a6-4f33-b360-2d75c886d9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=562fbd30b3b547ad84cef4c396b0b68e, user_id=03fc75503c7b426cba42614c38545fd3, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:03.558338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aef620-d4a6-4f33-b360-2d75c886d9bc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:03.599234 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1650/3268] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:00:03 2022] PUT /identity/v3/projects/562fbd30b3b547ad84cef4c396b0b68e/users/03fc75503c7b426cba42614c38545fd3/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:00:05.690841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d888f-59e2-408a-8d23-2e82d9037f7a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:05.692914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-999d888f-59e2-408a-8d23-2e82d9037f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:05.716424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d888f-59e2-408a-8d23-2e82d9037f7a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:05.750039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9375b-e728-4e4e-8ba3-285c54b29964 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:05.750895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9375b-e728-4e4e-8ba3-285c54b29964 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:05.751386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9375b-e728-4e4e-8ba3-285c54b29964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:05.791026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dfc9375b-e728-4e4e-8ba3-285c54b29964 None None] MFA Rules not processed for user `6adaa12c2ca9469299994300bc856ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:05.794573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dfc9375b-e728-4e4e-8ba3-285c54b29964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:05.845412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:05.846116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:05.846389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:05.848859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] PATH_INFO: `/v3/users/6925000aee9d499a85c6dc1282d09993` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:05.850247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6925000aee9d499a85c6dc1282d09993)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:05.852198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:05.884840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:05.924065 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1651/3269] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 15 21:00:05.959337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6925000aee9d499a85c6dc1282d09993'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:05.970017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6925000aee9d499a85c6dc1282d09993'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:05.989800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6925000aee9d499a85c6dc1282d09993'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:06.003001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6925000aee9d499a85c6dc1282d09993'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:06.003877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6925000aee9d499a85c6dc1282d09993'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:06.013188 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1619/3270] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:00:05 2022] DELETE /identity/v3/users/6925000aee9d499a85c6dc1282d09993 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:06.018605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de309ea-4e6b-4760-b77b-341740a0d5ab None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:06.018896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de309ea-4e6b-4760-b77b-341740a0d5ab None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:06.019159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de309ea-4e6b-4760-b77b-341740a0d5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:06.046652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7de309ea-4e6b-4760-b77b-341740a0d5ab None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:06.048585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7de309ea-4e6b-4760-b77b-341740a0d5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:06.153410 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1652/3271] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:06.207705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:06.209281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:06.316946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f570b5-5ca3-4a71-9823-d5e67d85b763 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:06.323566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f25901-40cf-48ca-8405-1f2def799338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:06.325268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f25901-40cf-48ca-8405-1f2def799338 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:06.326015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f25901-40cf-48ca-8405-1f2def799338 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:06.326522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f25901-40cf-48ca-8405-1f2def799338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:06.367412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f25901-40cf-48ca-8405-1f2def799338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:06.370014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f25901-40cf-48ca-8405-1f2def799338 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:06.391717 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1620/3272] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:07.858837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de309ea-4e6b-4760-b77b-341740a0d5ab None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:07.861452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7de309ea-4e6b-4760-b77b-341740a0d5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:07.889597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de309ea-4e6b-4760-b77b-341740a0d5ab None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:07.907864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:07.909214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:07.909651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:07.910103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] PATH_INFO: `/v3/projects/6b955a7e7fb04b569773c0453624911a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:07.912909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b955a7e7fb04b569773c0453624911a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:07.926632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:07.955082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c49eb0-108e-43e9-a5d9-e566214cf526 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:07.955647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c49eb0-108e-43e9-a5d9-e566214cf526 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:07.955861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c49eb0-108e-43e9-a5d9-e566214cf526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:08.007759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-11c49eb0-108e-43e9-a5d9-e566214cf526 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:08.009899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-11c49eb0-108e-43e9-a5d9-e566214cf526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:08.026351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b955a7e7fb04b569773c0453624911a', 'request_id': 'req-41a7974c-ae37-4d6d-beb8-05694fd53a14'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:08.045626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b955a7e7fb04b569773c0453624911a', 'request_id': 'req-41a7974c-ae37-4d6d-beb8-05694fd53a14'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:08.065697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b955a7e7fb04b569773c0453624911a', 'request_id': 'req-41a7974c-ae37-4d6d-beb8-05694fd53a14'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:08.067632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:08.089211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] The token cache is being invalidate because project 6b955a7e7fb04b569773c0453624911a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:00:08.090274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:08.091531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-41a7974c-ae37-4d6d-beb8-05694fd53a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:08.096896 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1653/3273] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:07 2022] DELETE /identity/v3/projects/6b955a7e7fb04b569773c0453624911a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:08.104126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc09ef-f574-47e9-bb79-fc48fb25c7e0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:08.104673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc09ef-f574-47e9-bb79-fc48fb25c7e0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:08.105155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc09ef-f574-47e9-bb79-fc48fb25c7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:08.148815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b5bc09ef-f574-47e9-bb79-fc48fb25c7e0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:08.150095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b5bc09ef-f574-47e9-bb79-fc48fb25c7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:08.176840 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1621/3274] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:00:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:08.225915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c49eb0-108e-43e9-a5d9-e566214cf526 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:08.228271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-11c49eb0-108e-43e9-a5d9-e566214cf526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:08.258628 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1654/3275] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:08.268362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1863c24-3a69-48d0-b757-3f455d0667b6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:08.268709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1863c24-3a69-48d0-b757-3f455d0667b6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:08.268989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1863c24-3a69-48d0-b757-3f455d0667b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:08.299152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a1863c24-3a69-48d0-b757-3f455d0667b6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:08.300594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a1863c24-3a69-48d0-b757-3f455d0667b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:08.327909 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1655/3276] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:00:08.328087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:08.329310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1863c24-3a69-48d0-b757-3f455d0667b6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:08.339845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1863c24-3a69-48d0-b757-3f455d0667b6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:08.367251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99fb54-e9b9-432b-8d40-4655becba907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:08.367872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99fb54-e9b9-432b-8d40-4655becba907 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:08.368147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99fb54-e9b9-432b-8d40-4655becba907 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:08.368417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99fb54-e9b9-432b-8d40-4655becba907 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:08.369093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99fb54-e9b9-432b-8d40-4655becba907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:00:08.369398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99fb54-e9b9-432b-8d40-4655becba907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:08.371543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99fb54-e9b9-432b-8d40-4655becba907 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:08.384802 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1656/3277] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:00:08.389587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99fb54-e9b9-432b-8d40-4655becba907 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:08.403093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99fb54-e9b9-432b-8d40-4655becba907 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:08.409966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafb3bd-5a89-40f5-8581-2e657a56d28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:08.412007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafb3bd-5a89-40f5-8581-2e657a56d28b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:08.412400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafb3bd-5a89-40f5-8581-2e657a56d28b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:08.412776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafb3bd-5a89-40f5-8581-2e657a56d28b admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:08.416400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafb3bd-5a89-40f5-8581-2e657a56d28b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:08.419559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafb3bd-5a89-40f5-8581-2e657a56d28b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:08.424426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c49eb0-108e-43e9-a5d9-e566214cf526 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:08.469645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b07b8-b116-4e4e-8798-28edca8a77ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:08.473513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b07b8-b116-4e4e-8798-28edca8a77ea service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:08.474866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b07b8-b116-4e4e-8798-28edca8a77ea service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:08.475691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b07b8-b116-4e4e-8798-28edca8a77ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:08.495057 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1657/3278] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:00:08 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:00:08.498334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafb3bd-5a89-40f5-8581-2e657a56d28b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:08.511422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b07b8-b116-4e4e-8798-28edca8a77ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:08.517391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b07b8-b116-4e4e-8798-28edca8a77ea service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:08.540403 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1622/3279] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:00:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:00:08.675848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafb3bd-5a89-40f5-8581-2e657a56d28b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:08.697979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199864db-8893-4cf3-ab04-43c137149060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:08.701394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199864db-8893-4cf3-ab04-43c137149060 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:08.702817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199864db-8893-4cf3-ab04-43c137149060 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:08.703309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199864db-8893-4cf3-ab04-43c137149060 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:08.704791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199864db-8893-4cf3-ab04-43c137149060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:08.721605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199864db-8893-4cf3-ab04-43c137149060 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:08.797520 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1658/3280] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:00:08 2022] POST /identity/v3/users => generated 294 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:00:08.801966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b07b8-b116-4e4e-8798-28edca8a77ea service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:08.828262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b07b8-b116-4e4e-8798-28edca8a77ea service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:08.849257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec419e-d92e-4f4b-b426-96d99deb4660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:08.850130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec419e-d92e-4f4b-b426-96d99deb4660 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:08.850636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec419e-d92e-4f4b-b426-96d99deb4660 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:08.851117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec419e-d92e-4f4b-b426-96d99deb4660 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:08.852251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec419e-d92e-4f4b-b426-96d99deb4660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:08.855147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec419e-d92e-4f4b-b426-96d99deb4660 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:08.895856 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1623/3281] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:08.897449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199864db-8893-4cf3-ab04-43c137149060 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:08.911115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199864db-8893-4cf3-ab04-43c137149060 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:08.926854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451b67a-75cc-4826-80ad-742eba6c1a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:08.927668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451b67a-75cc-4826-80ad-742eba6c1a07 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:08.928071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451b67a-75cc-4826-80ad-742eba6c1a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:08.928446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451b67a-75cc-4826-80ad-742eba6c1a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:08.930539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451b67a-75cc-4826-80ad-742eba6c1a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:08.936410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451b67a-75cc-4826-80ad-742eba6c1a07 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:08.965292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec419e-d92e-4f4b-b426-96d99deb4660 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:08.976028 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1659/3282] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:08.978399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec419e-d92e-4f4b-b426-96d99deb4660 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:08.997910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd8b96-1f0d-4538-9515-e2db7f6fcdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:08.999069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd8b96-1f0d-4538-9515-e2db7f6fcdf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:08.999453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd8b96-1f0d-4538-9515-e2db7f6fcdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:08.999833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd8b96-1f0d-4538-9515-e2db7f6fcdf1 admin admin] PATH_INFO: `/v3/projects/c04b2976577d42799a800d21545d1969/users/fb8dc371c65f4bbb92d559c7ed78964c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:09.047297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd8b96-1f0d-4538-9515-e2db7f6fcdf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04b2976577d42799a800d21545d1969, user_id=fb8dc371c65f4bbb92d559c7ed78964c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:09.054258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd8b96-1f0d-4538-9515-e2db7f6fcdf1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:09.094310 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1624/3283] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:00:08 2022] PUT /identity/v3/projects/c04b2976577d42799a800d21545d1969/users/fb8dc371c65f4bbb92d559c7ed78964c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:00:09.099748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd10c0b-11a5-459b-b55d-7f0ff81f1bd9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:09.100457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd10c0b-11a5-459b-b55d-7f0ff81f1bd9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:09.100928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd10c0b-11a5-459b-b55d-7f0ff81f1bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:09.201134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bfd10c0b-11a5-459b-b55d-7f0ff81f1bd9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:09.207923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bfd10c0b-11a5-459b-b55d-7f0ff81f1bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:09.296280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:09.445390 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1660/3284] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:09.510473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd8b96-1f0d-4538-9515-e2db7f6fcdf1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:09.512494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-34dd8b96-1f0d-4538-9515-e2db7f6fcdf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:09.720912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd8b96-1f0d-4538-9515-e2db7f6fcdf1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:09.732053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee613663-d752-4033-8d0d-642ecb46fd7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:09.732053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee613663-d752-4033-8d0d-642ecb46fd7d service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:09.732053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee613663-d752-4033-8d0d-642ecb46fd7d service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:09.732500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee613663-d752-4033-8d0d-642ecb46fd7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:09.778304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee613663-d752-4033-8d0d-642ecb46fd7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:09.785289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee613663-d752-4033-8d0d-642ecb46fd7d service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:09.823674 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1625/3285] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:10.223783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd10c0b-11a5-459b-b55d-7f0ff81f1bd9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:10.225262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bfd10c0b-11a5-459b-b55d-7f0ff81f1bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:10.549210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd10c0b-11a5-459b-b55d-7f0ff81f1bd9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:10.559536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c469c46-2509-4b12-a4db-d1b467b8e791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:10.563236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c469c46-2509-4b12-a4db-d1b467b8e791 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:10.565466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c469c46-2509-4b12-a4db-d1b467b8e791 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:10.566189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c469c46-2509-4b12-a4db-d1b467b8e791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:10.570674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4c469c46-2509-4b12-a4db-d1b467b8e791 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:10.742593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c469c46-2509-4b12-a4db-d1b467b8e791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:10.756684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c469c46-2509-4b12-a4db-d1b467b8e791 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:10.815625 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1661/3286] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:11.878995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee613663-d752-4033-8d0d-642ecb46fd7d service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:11.881435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ee613663-d752-4033-8d0d-642ecb46fd7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:11.932114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee613663-d752-4033-8d0d-642ecb46fd7d service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:11.958338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:11.959038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:11.959313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:11.962312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] PATH_INFO: `/v3/users/6c8c93cfed5140b183cc0e3a559b1ce7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:11.972147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c8c93cfed5140b183cc0e3a559b1ce7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:11.974364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:12.056258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c8c93cfed5140b183cc0e3a559b1ce7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:12.066031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c8c93cfed5140b183cc0e3a559b1ce7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:12.086209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c8c93cfed5140b183cc0e3a559b1ce7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:12.114906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c8c93cfed5140b183cc0e3a559b1ce7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:12.116350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c96311-8e4b-4bf0-a68a-e29246eb19f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c8c93cfed5140b183cc0e3a559b1ce7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:12.136177 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1626/3287] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:00:11 2022] DELETE /identity/v3/users/6c8c93cfed5140b183cc0e3a559b1ce7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:12.779377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c469c46-2509-4b12-a4db-d1b467b8e791 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:12.799702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c469c46-2509-4b12-a4db-d1b467b8e791 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:12.935962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:12.937988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:12.938791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:12.940011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] PATH_INFO: `/v3/projects/8d37935df706485388d6137dff1faba5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:12.945838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d37935df706485388d6137dff1faba5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:12.948166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:12.965302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:13.062909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d37935df706485388d6137dff1faba5', 'request_id': 'req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:13.076795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d37935df706485388d6137dff1faba5', 'request_id': 'req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:13.090899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d37935df706485388d6137dff1faba5', 'request_id': 'req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:13.129379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] The token cache is being invalidate because project 8d37935df706485388d6137dff1faba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:00:13.137387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:13.140576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f260437-b9f0-4ce7-a576-f1e6efbe97ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:13.150202 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1662/3288] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:12 2022] DELETE /identity/v3/projects/8d37935df706485388d6137dff1faba5 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:13.150918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eeee63-d0c3-47fd-9e3c-d6fa8d8f1954 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:13.151404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eeee63-d0c3-47fd-9e3c-d6fa8d8f1954 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:13.151517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eeee63-d0c3-47fd-9e3c-d6fa8d8f1954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:13.217488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:13.272535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-66eeee63-d0c3-47fd-9e3c-d6fa8d8f1954 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:13.277002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-66eeee63-d0c3-47fd-9e3c-d6fa8d8f1954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:13.443243 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1627/3289] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:13.457378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad4de-89e4-484d-a410-7b7606e079ad None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:13.457688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad4de-89e4-484d-a410-7b7606e079ad None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:13.457950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad4de-89e4-484d-a410-7b7606e079ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:13.502961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a98ad4de-89e4-484d-a410-7b7606e079ad None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:13.504225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a98ad4de-89e4-484d-a410-7b7606e079ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:13.544999 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1663/3290] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:13.549426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eeee63-d0c3-47fd-9e3c-d6fa8d8f1954 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:13.569335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eeee63-d0c3-47fd-9e3c-d6fa8d8f1954 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:13.586997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d8164-c810-42f2-af18-304fcc49104d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:13.587706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d8164-c810-42f2-af18-304fcc49104d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:13.588053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d8164-c810-42f2-af18-304fcc49104d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:13.588393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d8164-c810-42f2-af18-304fcc49104d admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:13.591085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d8164-c810-42f2-af18-304fcc49104d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:00:13.591657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d8164-c810-42f2-af18-304fcc49104d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:13.593349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d8164-c810-42f2-af18-304fcc49104d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:13.623963 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1628/3291] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:00:13.628183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ad4de-89e4-484d-a410-7b7606e079ad None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:13.641225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ad4de-89e4-484d-a410-7b7606e079ad None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:13.660929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0f4e7-745c-4e2e-86a8-7d1135a5edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:13.663271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0f4e7-745c-4e2e-86a8-7d1135a5edae admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:13.664042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0f4e7-745c-4e2e-86a8-7d1135a5edae admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:13.664042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0f4e7-745c-4e2e-86a8-7d1135a5edae admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:13.665505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0f4e7-745c-4e2e-86a8-7d1135a5edae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:13.671048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0f4e7-745c-4e2e-86a8-7d1135a5edae admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:13.735597 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1664/3292] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:00:13 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:00:13.739665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d8164-c810-42f2-af18-304fcc49104d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:13.754941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d8164-c810-42f2-af18-304fcc49104d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:13.931289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1317e-13cd-4fae-8cec-5360d9737cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:13.932065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1317e-13cd-4fae-8cec-5360d9737cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:13.932350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1317e-13cd-4fae-8cec-5360d9737cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:13.932619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1317e-13cd-4fae-8cec-5360d9737cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:13.933862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1317e-13cd-4fae-8cec-5360d9737cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:13.935823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1317e-13cd-4fae-8cec-5360d9737cb0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:13.990954 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1629/3293] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:00:13 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:00:13.995228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0f4e7-745c-4e2e-86a8-7d1135a5edae admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:14.014024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0f4e7-745c-4e2e-86a8-7d1135a5edae admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:14.035012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca990d-641a-4e32-a38c-01b3c9fdea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:14.038717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca990d-641a-4e32-a38c-01b3c9fdea6a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:14.040072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca990d-641a-4e32-a38c-01b3c9fdea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:14.042416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca990d-641a-4e32-a38c-01b3c9fdea6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:14.044086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca990d-641a-4e32-a38c-01b3c9fdea6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:14.057079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca990d-641a-4e32-a38c-01b3c9fdea6a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:14.099929 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1665/3294] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:14.103742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1317e-13cd-4fae-8cec-5360d9737cb0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:14.120470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1317e-13cd-4fae-8cec-5360d9737cb0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:14.132304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07224b-ba1f-44e6-a899-664b27b4cfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:14.132945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07224b-ba1f-44e6-a899-664b27b4cfda admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:14.135431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07224b-ba1f-44e6-a899-664b27b4cfda admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:14.135431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07224b-ba1f-44e6-a899-664b27b4cfda admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:14.135431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07224b-ba1f-44e6-a899-664b27b4cfda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:14.136043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07224b-ba1f-44e6-a899-664b27b4cfda admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:14.157624 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1630/3295] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:14.162675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca990d-641a-4e32-a38c-01b3c9fdea6a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:14.178698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca990d-641a-4e32-a38c-01b3c9fdea6a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:14.213509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de767be-4e62-4116-85d3-3aa990978fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:14.213509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de767be-4e62-4116-85d3-3aa990978fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:14.213509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de767be-4e62-4116-85d3-3aa990978fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:14.213509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de767be-4e62-4116-85d3-3aa990978fcf admin admin] PATH_INFO: `/v3/projects/9bc93093ddf44aada9598a14b5417a2d/users/4e1353ffe42243af82b590bc98e96a42/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:14.251471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de767be-4e62-4116-85d3-3aa990978fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc93093ddf44aada9598a14b5417a2d, user_id=4e1353ffe42243af82b590bc98e96a42, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:14.257732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de767be-4e62-4116-85d3-3aa990978fcf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:14.295557 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1666/3296] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:00:14 2022] PUT /identity/v3/projects/9bc93093ddf44aada9598a14b5417a2d/users/4e1353ffe42243af82b590bc98e96a42/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:00:14.299939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d65e9-ef11-4fcb-b16b-71c5d830cef5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:14.300723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d65e9-ef11-4fcb-b16b-71c5d830cef5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:14.301144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d65e9-ef11-4fcb-b16b-71c5d830cef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:14.420575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-618d65e9-ef11-4fcb-b16b-71c5d830cef5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:14.421853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-618d65e9-ef11-4fcb-b16b-71c5d830cef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:14.632122 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1631/3297] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:14.672566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de767be-4e62-4116-85d3-3aa990978fcf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:14.674803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9de767be-4e62-4116-85d3-3aa990978fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:14.801102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de767be-4e62-4116-85d3-3aa990978fcf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:14.816900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38426fb9-271c-483a-8ee0-be6d7967d0b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:14.817838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38426fb9-271c-483a-8ee0-be6d7967d0b8 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:14.818299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38426fb9-271c-483a-8ee0-be6d7967d0b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:14.818721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38426fb9-271c-483a-8ee0-be6d7967d0b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:14.836270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38426fb9-271c-483a-8ee0-be6d7967d0b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:14.837985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38426fb9-271c-483a-8ee0-be6d7967d0b8 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:14.867355 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1667/3298] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:18.469077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40564cac-1523-42d8-8300-75ecc184fc84 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:18.469769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40564cac-1523-42d8-8300-75ecc184fc84 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:18.469951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40564cac-1523-42d8-8300-75ecc184fc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:18.535223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:18.540014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-40564cac-1523-42d8-8300-75ecc184fc84 None None] MFA Rules not processed for user `03fc75503c7b426cba42614c38545fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:18.542069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-40564cac-1523-42d8-8300-75ecc184fc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:18.719184 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1632/3299] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:18.740719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38426fb9-271c-483a-8ee0-be6d7967d0b8 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:18.751772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-38426fb9-271c-483a-8ee0-be6d7967d0b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:18.862004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:18.908790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38426fb9-271c-483a-8ee0-be6d7967d0b8 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:18.914318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86616cb8-c425-4eb4-9d6f-f3512c0a7057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:18.915201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86616cb8-c425-4eb4-9d6f-f3512c0a7057 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:18.915589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86616cb8-c425-4eb4-9d6f-f3512c0a7057 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:18.916577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86616cb8-c425-4eb4-9d6f-f3512c0a7057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:18.919376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-86616cb8-c425-4eb4-9d6f-f3512c0a7057 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:19.114455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86616cb8-c425-4eb4-9d6f-f3512c0a7057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:19.114455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86616cb8-c425-4eb4-9d6f-f3512c0a7057 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:19.155757 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1668/3300] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:20.939325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6558-2388-411a-9c55-409088f68ac9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:20.940242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6558-2388-411a-9c55-409088f68ac9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:20.940751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6558-2388-411a-9c55-409088f68ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:20.966331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7a5f6558-2388-411a-9c55-409088f68ac9 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:20.976913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7a5f6558-2388-411a-9c55-409088f68ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:21.129217 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1633/3301] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:00:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:21.180742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86616cb8-c425-4eb4-9d6f-f3512c0a7057 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:21.183122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-86616cb8-c425-4eb4-9d6f-f3512c0a7057 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:21.394096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86616cb8-c425-4eb4-9d6f-f3512c0a7057 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:21.405857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80c8d6-731c-49da-8478-5575531924c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:21.406666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80c8d6-731c-49da-8478-5575531924c4 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:21.407039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80c8d6-731c-49da-8478-5575531924c4 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:21.407401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80c8d6-731c-49da-8478-5575531924c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:21.429230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80c8d6-731c-49da-8478-5575531924c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:21.431005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80c8d6-731c-49da-8478-5575531924c4 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:21.453800 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1669/3302] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:00:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:24.946942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb05fe-8273-424c-9228-00bef66c7c96 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:24.947839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb05fe-8273-424c-9228-00bef66c7c96 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:24.948330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb05fe-8273-424c-9228-00bef66c7c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:25.083674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-aabb05fe-8273-424c-9228-00bef66c7c96 None None] MFA Rules not processed for user `5d93a680570f4d23b1f6df534bc082b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:25.085467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aabb05fe-8273-424c-9228-00bef66c7c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:25.266578 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1634/3303] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:28.231478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24e4dd-5df8-4c20-b97b-4227d6c38685 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:28.231908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24e4dd-5df8-4c20-b97b-4227d6c38685 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:28.232002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24e4dd-5df8-4c20-b97b-4227d6c38685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:28.289216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fa24e4dd-5df8-4c20-b97b-4227d6c38685 None None] MFA Rules not processed for user `4e1353ffe42243af82b590bc98e96a42`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:28.290457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fa24e4dd-5df8-4c20-b97b-4227d6c38685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:28.480516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8215c-0a84-4368-81eb-986da12f79a3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:28.481577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8215c-0a84-4368-81eb-986da12f79a3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:28.482129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8215c-0a84-4368-81eb-986da12f79a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:28.491574 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1670/3304] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:28.515493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-82e8215c-0a84-4368-81eb-986da12f79a3 None None] MFA Rules not processed for user `4e1353ffe42243af82b590bc98e96a42`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:28.517404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-82e8215c-0a84-4368-81eb-986da12f79a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:28.594490 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1635/3305] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:28.996230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b81e5-66f0-4a92-af56-ddd9448b387c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:28.996520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b81e5-66f0-4a92-af56-ddd9448b387c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:28.997984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b81e5-66f0-4a92-af56-ddd9448b387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:29.047547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:29.131290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-103b81e5-66f0-4a92-af56-ddd9448b387c None None] MFA Rules not processed for user `fb8dc371c65f4bbb92d559c7ed78964c`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:29.133443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-103b81e5-66f0-4a92-af56-ddd9448b387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:29.405022 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1671/3306] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:29.776344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8215c-0a84-4368-81eb-986da12f79a3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:29.800156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8215c-0a84-4368-81eb-986da12f79a3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:29.812514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871784fe-c84d-496a-bce9-04a65ef99f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:29.816091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871784fe-c84d-496a-bce9-04a65ef99f56 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:29.816642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871784fe-c84d-496a-bce9-04a65ef99f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:29.817143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871784fe-c84d-496a-bce9-04a65ef99f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:29.832533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-871784fe-c84d-496a-bce9-04a65ef99f56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:29.876906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871784fe-c84d-496a-bce9-04a65ef99f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:29.882566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871784fe-c84d-496a-bce9-04a65ef99f56 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:29.923405 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1636/3307] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:30.726372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b81e5-66f0-4a92-af56-ddd9448b387c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:30.727875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-103b81e5-66f0-4a92-af56-ddd9448b387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:30.914707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b81e5-66f0-4a92-af56-ddd9448b387c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:30.928759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1db133-3d78-479d-ac8c-e214cea2eb4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:30.929783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db133-3d78-479d-ac8c-e214cea2eb4a service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:30.931409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db133-3d78-479d-ac8c-e214cea2eb4a service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:30.931967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db133-3d78-479d-ac8c-e214cea2eb4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:30.983073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1db133-3d78-479d-ac8c-e214cea2eb4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:30.989518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1db133-3d78-479d-ac8c-e214cea2eb4a service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:31.042616 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1672/3308] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:33.743712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871784fe-c84d-496a-bce9-04a65ef99f56 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:33.767435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871784fe-c84d-496a-bce9-04a65ef99f56 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:33.788521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee3f55-8fd2-44fc-b358-9e49a1e53d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:33.789539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee3f55-8fd2-44fc-b358-9e49a1e53d06 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:33.790951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee3f55-8fd2-44fc-b358-9e49a1e53d06 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:33.791387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee3f55-8fd2-44fc-b358-9e49a1e53d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:33.823163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee3f55-8fd2-44fc-b358-9e49a1e53d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:33.828272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee3f55-8fd2-44fc-b358-9e49a1e53d06 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:33.842567 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1637/3309] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:37.429341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a08e9-decb-46e9-b1d2-d7bca0de12d2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:37.429867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a08e9-decb-46e9-b1d2-d7bca0de12d2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:37.429867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a08e9-decb-46e9-b1d2-d7bca0de12d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:37.499359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e99a08e9-decb-46e9-b1d2-d7bca0de12d2 None None] MFA Rules not processed for user `fb8dc371c65f4bbb92d559c7ed78964c`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:37.504990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e99a08e9-decb-46e9-b1d2-d7bca0de12d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:37.543754 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1673/3310] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:43.835957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:43.857904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee3f55-8fd2-44fc-b358-9e49a1e53d06 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:43.859579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-50ee3f55-8fd2-44fc-b358-9e49a1e53d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:43.912710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a08e9-decb-46e9-b1d2-d7bca0de12d2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:43.914638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e99a08e9-decb-46e9-b1d2-d7bca0de12d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:43.943379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a08e9-decb-46e9-b1d2-d7bca0de12d2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:43.961781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:43.962570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:43.965653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:43.965938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] PATH_INFO: `/v3/users/2bba6c9b3f674959ad79d4a394ed9f2a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:43.969198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bba6c9b3f674959ad79d4a394ed9f2a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:43.971357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:44.052427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee3f55-8fd2-44fc-b358-9e49a1e53d06 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:44.064164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34b791-4459-4c9f-b777-d66fe6ef68b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:44.064848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34b791-4459-4c9f-b777-d66fe6ef68b0 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:44.065160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34b791-4459-4c9f-b777-d66fe6ef68b0 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:44.065629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34b791-4459-4c9f-b777-d66fe6ef68b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:44.067818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7e34b791-4459-4c9f-b777-d66fe6ef68b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:44.101974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bba6c9b3f674959ad79d4a394ed9f2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.104380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34b791-4459-4c9f-b777-d66fe6ef68b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:44.107082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34b791-4459-4c9f-b777-d66fe6ef68b0 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:44.127464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bba6c9b3f674959ad79d4a394ed9f2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.134337 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1638/3311] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:44.139510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:44.147418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bba6c9b3f674959ad79d4a394ed9f2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.159563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bba6c9b3f674959ad79d4a394ed9f2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.160294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bba6c9b3f674959ad79d4a394ed9f2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.171549 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1674/3312] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:00:43 2022] DELETE /identity/v3/users/2bba6c9b3f674959ad79d4a394ed9f2a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:44.402749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34b791-4459-4c9f-b777-d66fe6ef68b0 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:44.429463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34b791-4459-4c9f-b777-d66fe6ef68b0 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:44.497629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:44.498422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:44.498704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:44.498999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] PATH_INFO: `/v3/projects/ca9af3805eba4f69b474a8b2205f0920` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:44.500160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9af3805eba4f69b474a8b2205f0920)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:44.503044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:44.571886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9af3805eba4f69b474a8b2205f0920', 'request_id': 'req-2d445788-c90a-4ac0-854a-6a19e9130de8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.584676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9af3805eba4f69b474a8b2205f0920', 'request_id': 'req-2d445788-c90a-4ac0-854a-6a19e9130de8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.601447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:44.602398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9af3805eba4f69b474a8b2205f0920', 'request_id': 'req-2d445788-c90a-4ac0-854a-6a19e9130de8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.603340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:44.624482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] The token cache is being invalidate because project ca9af3805eba4f69b474a8b2205f0920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:00:44.627122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.629020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2d445788-c90a-4ac0-854a-6a19e9130de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.636459 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1639/3313] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:44 2022] DELETE /identity/v3/projects/ca9af3805eba4f69b474a8b2205f0920 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:44.637584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a137040-8b20-45dc-8421-c7006e0718f8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:44.643854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e488b97-72d4-42eb-a936-df8c5eee618f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:44.644138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e488b97-72d4-42eb-a936-df8c5eee618f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:44.644787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e488b97-72d4-42eb-a936-df8c5eee618f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:44.686335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4e488b97-72d4-42eb-a936-df8c5eee618f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:44.688028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4e488b97-72d4-42eb-a936-df8c5eee618f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:44.707501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:44.777574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:44.778379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:44.778650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:44.778914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] PATH_INFO: `/v3/users/42276cf2f2894712ac91f87e4ce1f6ca` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:44.780248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] RBAC: Authorizing `identity:delete_user(user_id=42276cf2f2894712ac91f87e4ce1f6ca)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:44.782313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:44.827178 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1640/3314] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:44.836262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44076031-1f21-4a4a-a09b-d46cae9daa46 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:44.836962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44076031-1f21-4a4a-a09b-d46cae9daa46 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:44.836962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44076031-1f21-4a4a-a09b-d46cae9daa46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:44.862985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42276cf2f2894712ac91f87e4ce1f6ca'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.868783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-44076031-1f21-4a4a-a09b-d46cae9daa46 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:44.871279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-44076031-1f21-4a4a-a09b-d46cae9daa46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:44.879392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42276cf2f2894712ac91f87e4ce1f6ca'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.886711 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1641/3315] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:44.888534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44076031-1f21-4a4a-a09b-d46cae9daa46 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:44.890012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42276cf2f2894712ac91f87e4ce1f6ca'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.902200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42276cf2f2894712ac91f87e4ce1f6ca'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.903360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c5ddfacd-b34f-4500-8691-a4f5dece269c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42276cf2f2894712ac91f87e4ce1f6ca'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:44.903995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44076031-1f21-4a4a-a09b-d46cae9daa46 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:44.910896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675340d5-53a6-4c0a-bbce-52e9bd9b2c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:44.911501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675340d5-53a6-4c0a-bbce-52e9bd9b2c82 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:44.911774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675340d5-53a6-4c0a-bbce-52e9bd9b2c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:44.912085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675340d5-53a6-4c0a-bbce-52e9bd9b2c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:44.912735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675340d5-53a6-4c0a-bbce-52e9bd9b2c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:00:44.913029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675340d5-53a6-4c0a-bbce-52e9bd9b2c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:44.914488 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1675/3316] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:00:44 2022] DELETE /identity/v3/users/42276cf2f2894712ac91f87e4ce1f6ca => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:00:44.917882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675340d5-53a6-4c0a-bbce-52e9bd9b2c82 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:44.921254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4f71b-501b-4796-88b2-384bb71e0428 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:44.921628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4f71b-501b-4796-88b2-384bb71e0428 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:44.921986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4f71b-501b-4796-88b2-384bb71e0428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:44.936376 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1642/3317] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:00:44.940545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675340d5-53a6-4c0a-bbce-52e9bd9b2c82 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:44.943040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-675340d5-53a6-4c0a-bbce-52e9bd9b2c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:44.947760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8ac4f71b-501b-4796-88b2-384bb71e0428 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:44.949667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8ac4f71b-501b-4796-88b2-384bb71e0428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:44.957735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675340d5-53a6-4c0a-bbce-52e9bd9b2c82 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:45.025939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907ca6a-9402-4590-b449-f273cb013fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:45.027339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907ca6a-9402-4590-b449-f273cb013fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:45.027596 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1676/3318] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:45.031010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907ca6a-9402-4590-b449-f273cb013fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:45.031310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907ca6a-9402-4590-b449-f273cb013fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:45.031961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907ca6a-9402-4590-b449-f273cb013fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:45.033926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907ca6a-9402-4590-b449-f273cb013fb1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:45.037833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4f71b-501b-4796-88b2-384bb71e0428 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:45.040152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8ac4f71b-501b-4796-88b2-384bb71e0428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:45.053898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:45.066197 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1643/3319] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:00:44 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:00:45.071362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907ca6a-9402-4590-b449-f273cb013fb1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:45.121742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4f71b-501b-4796-88b2-384bb71e0428 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:45.154902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907ca6a-9402-4590-b449-f273cb013fb1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:45.160822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d94e5a-651d-4a85-a2b8-e4bb53194bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:45.161464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d94e5a-651d-4a85-a2b8-e4bb53194bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:45.161876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d94e5a-651d-4a85-a2b8-e4bb53194bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:45.162148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d94e5a-651d-4a85-a2b8-e4bb53194bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:45.162942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d94e5a-651d-4a85-a2b8-e4bb53194bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:45.165023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d94e5a-651d-4a85-a2b8-e4bb53194bc9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:45.178303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec744583-1972-4b2d-aade-4489477107f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:45.178907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec744583-1972-4b2d-aade-4489477107f0 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:45.179176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec744583-1972-4b2d-aade-4489477107f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:45.179637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec744583-1972-4b2d-aade-4489477107f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:45.195646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec744583-1972-4b2d-aade-4489477107f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:45.197672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec744583-1972-4b2d-aade-4489477107f0 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:45.200112 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1644/3320] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:00:45 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:00:45.201293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d94e5a-651d-4a85-a2b8-e4bb53194bc9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:45.211669 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1677/3321] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:00:45.216690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d94e5a-651d-4a85-a2b8-e4bb53194bc9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:45.222425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c005e47-d710-44ab-972b-310ca1c18adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:45.223576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c005e47-d710-44ab-972b-310ca1c18adb admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:45.224004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c005e47-d710-44ab-972b-310ca1c18adb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:45.224435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c005e47-d710-44ab-972b-310ca1c18adb admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:45.225685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c005e47-d710-44ab-972b-310ca1c18adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:45.227661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c005e47-d710-44ab-972b-310ca1c18adb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:45.244301 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1645/3322] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:45.245962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec744583-1972-4b2d-aade-4489477107f0 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:45.257856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec744583-1972-4b2d-aade-4489477107f0 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:45.263265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566d9a3-42a6-431d-9440-94755ea3c78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:45.263942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566d9a3-42a6-431d-9440-94755ea3c78c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:45.264408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566d9a3-42a6-431d-9440-94755ea3c78c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:45.264539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566d9a3-42a6-431d-9440-94755ea3c78c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:45.265822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566d9a3-42a6-431d-9440-94755ea3c78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:45.267373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566d9a3-42a6-431d-9440-94755ea3c78c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:45.282416 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1678/3323] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:00:45.286708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c005e47-d710-44ab-972b-310ca1c18adb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:45.294492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:45.299337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c005e47-d710-44ab-972b-310ca1c18adb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:45.305817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0591d0-6868-47ef-826a-b43f884d7fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:45.306597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0591d0-6868-47ef-826a-b43f884d7fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:45.307751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0591d0-6868-47ef-826a-b43f884d7fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:45.308036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0591d0-6868-47ef-826a-b43f884d7fcb admin admin] PATH_INFO: `/v3/projects/90ca0b4db2b64900aaa1a4e05dec8a6f/users/f00da962dc434ae18a1ab29228846688/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:45.335970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0591d0-6868-47ef-826a-b43f884d7fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ca0b4db2b64900aaa1a4e05dec8a6f, user_id=f00da962dc434ae18a1ab29228846688, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:45.339930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0591d0-6868-47ef-826a-b43f884d7fcb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:45.373005 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1646/3324] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:00:45 2022] PUT /identity/v3/projects/90ca0b4db2b64900aaa1a4e05dec8a6f/users/f00da962dc434ae18a1ab29228846688/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:45.374532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d7e9-2efb-4667-867d-83003a2cbb89 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:45.374801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d7e9-2efb-4667-867d-83003a2cbb89 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:45.375134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d7e9-2efb-4667-867d-83003a2cbb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:45.422777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1f47d7e9-2efb-4667-867d-83003a2cbb89 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:45.424138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1f47d7e9-2efb-4667-867d-83003a2cbb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:45.497153 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1679/3325] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:45.559783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0591d0-6868-47ef-826a-b43f884d7fcb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:45.563124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2a0591d0-6868-47ef-826a-b43f884d7fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:45.650550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47d7e9-2efb-4667-867d-83003a2cbb89 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:45.652623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1f47d7e9-2efb-4667-867d-83003a2cbb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:45.674590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47d7e9-2efb-4667-867d-83003a2cbb89 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:45.684656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0591d0-6868-47ef-826a-b43f884d7fcb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:45.687902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:45.688552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:45.688848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:45.690915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] PATH_INFO: `/v3/projects/9ea11fd1a5a64f4a9641d7ed07f8e94f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:45.692900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f939dcb-46c4-4e4a-9f0f-aeeab0dcfe60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:45.694428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939dcb-46c4-4e4a-9f0f-aeeab0dcfe60 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:45.694661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939dcb-46c4-4e4a-9f0f-aeeab0dcfe60 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:45.694934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939dcb-46c4-4e4a-9f0f-aeeab0dcfe60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:45.695564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea11fd1a5a64f4a9641d7ed07f8e94f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:45.698406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:45.722050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f939dcb-46c4-4e4a-9f0f-aeeab0dcfe60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:45.723558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f939dcb-46c4-4e4a-9f0f-aeeab0dcfe60 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:45.737972 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1647/3326] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:45.780815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'request_id': 'req-d74d4b22-5370-47a0-a2ca-4a34b335743d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:45.782707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:45.791337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'request_id': 'req-d74d4b22-5370-47a0-a2ca-4a34b335743d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:45.800087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'request_id': 'req-d74d4b22-5370-47a0-a2ca-4a34b335743d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:45.816820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] The token cache is being invalidate because project 9ea11fd1a5a64f4a9641d7ed07f8e94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:00:45.817151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:45.818006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:45.819893 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1680/3327] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:45 2022] DELETE /identity/v3/projects/9ea11fd1a5a64f4a9641d7ed07f8e94f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:45.823845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f939dcb-46c4-4e4a-9f0f-aeeab0dcfe60 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:45.833862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f939dcb-46c4-4e4a-9f0f-aeeab0dcfe60 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:45.913281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:45.914683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:45.915124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:45.915398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] PATH_INFO: `/v3/users/c053667200af4dc9a8f8bcf00f624c3f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:45.916573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c053667200af4dc9a8f8bcf00f624c3f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:45.919404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:45.970732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c053667200af4dc9a8f8bcf00f624c3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:45.978172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c053667200af4dc9a8f8bcf00f624c3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:45.992997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c053667200af4dc9a8f8bcf00f624c3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:46.002060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c053667200af4dc9a8f8bcf00f624c3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:46.002800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e31641bf-f2fa-475f-9685-89e7df4c9024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c053667200af4dc9a8f8bcf00f624c3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:46.009246 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1648/3328] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:00:45 2022] DELETE /identity/v3/users/c053667200af4dc9a8f8bcf00f624c3f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:46.351043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:46.352600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:46.386317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d4b22-5370-47a0-a2ca-4a34b335743d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:46.471983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:46.472614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:46.473230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:46.473422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] PATH_INFO: `/v3/projects/e34ada1829574d1d8f301572a287d2c2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:46.474879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e34ada1829574d1d8f301572a287d2c2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:46.476754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:46.549903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e34ada1829574d1d8f301572a287d2c2', 'request_id': 'req-1ce023b1-0705-4771-83b9-e58f2b9d164e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:46.559815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e34ada1829574d1d8f301572a287d2c2', 'request_id': 'req-1ce023b1-0705-4771-83b9-e58f2b9d164e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:46.569456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e34ada1829574d1d8f301572a287d2c2', 'request_id': 'req-1ce023b1-0705-4771-83b9-e58f2b9d164e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:46.585225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] The token cache is being invalidate because project e34ada1829574d1d8f301572a287d2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:00:46.585785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:46.586444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1ce023b1-0705-4771-83b9-e58f2b9d164e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:00:46.587921 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1681/3329] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:46 2022] DELETE /identity/v3/projects/e34ada1829574d1d8f301572a287d2c2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:46.594461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328f78f-6cb4-42b6-b9f1-d71d163193a4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:46.594988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328f78f-6cb4-42b6-b9f1-d71d163193a4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:46.595508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328f78f-6cb4-42b6-b9f1-d71d163193a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:46.607859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:46.630611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5328f78f-6cb4-42b6-b9f1-d71d163193a4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:46.631842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5328f78f-6cb4-42b6-b9f1-d71d163193a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:46.695316 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1649/3330] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:46.704608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf7ea-7dc9-4598-b856-a56b4b519705 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:46.705527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf7ea-7dc9-4598-b856-a56b4b519705 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:46.705527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf7ea-7dc9-4598-b856-a56b4b519705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:46.740055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0bbbf7ea-7dc9-4598-b856-a56b4b519705 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:46.741401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0bbbf7ea-7dc9-4598-b856-a56b4b519705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:46.757104 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1682/3331] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:46.761452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328f78f-6cb4-42b6-b9f1-d71d163193a4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:46.779011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328f78f-6cb4-42b6-b9f1-d71d163193a4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:46.785515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5066e-21a2-42d8-9250-46ca149f4bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:46.786166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5066e-21a2-42d8-9250-46ca149f4bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:46.786441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5066e-21a2-42d8-9250-46ca149f4bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:46.787147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5066e-21a2-42d8-9250-46ca149f4bb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:46.787366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5066e-21a2-42d8-9250-46ca149f4bb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:00:46.787667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5066e-21a2-42d8-9250-46ca149f4bb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:46.789650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5066e-21a2-42d8-9250-46ca149f4bb6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:46.806750 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1650/3332] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:00:46.812389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbf7ea-7dc9-4598-b856-a56b4b519705 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:46.823636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbf7ea-7dc9-4598-b856-a56b4b519705 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:46.828505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb185e8-83a6-428a-be58-536603cf9e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:46.829152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb185e8-83a6-428a-be58-536603cf9e05 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:46.829425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb185e8-83a6-428a-be58-536603cf9e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:46.829724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb185e8-83a6-428a-be58-536603cf9e05 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:46.830354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb185e8-83a6-428a-be58-536603cf9e05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:46.832307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb185e8-83a6-428a-be58-536603cf9e05 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:46.867134 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1683/3333] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:00:46 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:00:46.872076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5066e-21a2-42d8-9250-46ca149f4bb6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:46.886131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5066e-21a2-42d8-9250-46ca149f4bb6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:46.982907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4dcf9-74f4-44d3-9c5e-a7970f57d962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:46.984589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4dcf9-74f4-44d3-9c5e-a7970f57d962 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:46.985008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4dcf9-74f4-44d3-9c5e-a7970f57d962 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:46.985485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4dcf9-74f4-44d3-9c5e-a7970f57d962 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:46.986327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4dcf9-74f4-44d3-9c5e-a7970f57d962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:46.988849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4dcf9-74f4-44d3-9c5e-a7970f57d962 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:47.026668 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1651/3334] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:00:46 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:00:47.027854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb185e8-83a6-428a-be58-536603cf9e05 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:47.042647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb185e8-83a6-428a-be58-536603cf9e05 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:47.055997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575994e3-f541-4eae-99bf-8361c8dbe528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:47.056979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575994e3-f541-4eae-99bf-8361c8dbe528 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:47.057354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575994e3-f541-4eae-99bf-8361c8dbe528 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:47.058141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575994e3-f541-4eae-99bf-8361c8dbe528 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:47.059559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575994e3-f541-4eae-99bf-8361c8dbe528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:47.061842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575994e3-f541-4eae-99bf-8361c8dbe528 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:47.075563 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1684/3335] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:47.079964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4dcf9-74f4-44d3-9c5e-a7970f57d962 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:47.092348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4dcf9-74f4-44d3-9c5e-a7970f57d962 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:47.101783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cee44-f237-4f08-824f-8f0be43e2b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:47.102494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cee44-f237-4f08-824f-8f0be43e2b30 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:47.102771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cee44-f237-4f08-824f-8f0be43e2b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:47.103089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cee44-f237-4f08-824f-8f0be43e2b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:47.103897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cee44-f237-4f08-824f-8f0be43e2b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:47.105627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cee44-f237-4f08-824f-8f0be43e2b30 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:47.118303 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1652/3336] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:47.124786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575994e3-f541-4eae-99bf-8361c8dbe528 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:47.137069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575994e3-f541-4eae-99bf-8361c8dbe528 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:47.142312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0890f-a44a-4e79-9a4e-84dcd651b24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:47.143145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0890f-a44a-4e79-9a4e-84dcd651b24b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:47.143599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0890f-a44a-4e79-9a4e-84dcd651b24b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:47.143880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0890f-a44a-4e79-9a4e-84dcd651b24b admin admin] PATH_INFO: `/v3/projects/e91f96ee17114009a55fe2bf388a7aa7/users/573c02d502744876babbdb4fbb4fb9e6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:47.161971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0890f-a44a-4e79-9a4e-84dcd651b24b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91f96ee17114009a55fe2bf388a7aa7, user_id=573c02d502744876babbdb4fbb4fb9e6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:47.164853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0890f-a44a-4e79-9a4e-84dcd651b24b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:47.181541 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1685/3337] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:00:47 2022] PUT /identity/v3/projects/e91f96ee17114009a55fe2bf388a7aa7/users/573c02d502744876babbdb4fbb4fb9e6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:47.193682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da44dab-fdb2-48ab-8865-cefc4fe5246a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:47.194248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da44dab-fdb2-48ab-8865-cefc4fe5246a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:47.194684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da44dab-fdb2-48ab-8865-cefc4fe5246a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:47.228373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5da44dab-fdb2-48ab-8865-cefc4fe5246a None None] MFA Rules not processed for user `573c02d502744876babbdb4fbb4fb9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:47.229744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5da44dab-fdb2-48ab-8865-cefc4fe5246a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:47.304075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:47.316532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0890f-a44a-4e79-9a4e-84dcd651b24b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:47.317247 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1653/3338] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:47.325692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:47.402019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0890f-a44a-4e79-9a4e-84dcd651b24b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:47.407385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f90506-f695-4af3-8f35-880548b84e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:47.408048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f90506-f695-4af3-8f35-880548b84e83 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:47.408439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f90506-f695-4af3-8f35-880548b84e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:47.408773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f90506-f695-4af3-8f35-880548b84e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:47.409707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f90506-f695-4af3-8f35-880548b84e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:47.411759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f90506-f695-4af3-8f35-880548b84e83 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:47.455008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da44dab-fdb2-48ab-8865-cefc4fe5246a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:47.465560 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1686/3339] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:00:47 2022] POST /identity/v3/projects => generated 389 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:00:47.471901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da44dab-fdb2-48ab-8865-cefc4fe5246a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:47.563639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e843b-c81d-4fac-a2f3-3f290873d087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:47.564006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e843b-c81d-4fac-a2f3-3f290873d087 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:47.564281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e843b-c81d-4fac-a2f3-3f290873d087 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:47.564552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e843b-c81d-4fac-a2f3-3f290873d087 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:47.576825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e843b-c81d-4fac-a2f3-3f290873d087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:47.580937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e843b-c81d-4fac-a2f3-3f290873d087 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:47.621370 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1654/3340] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:00:47 2022] POST /identity/v3/users => generated 301 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:00:47.625577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f90506-f695-4af3-8f35-880548b84e83 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:47.643164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f90506-f695-4af3-8f35-880548b84e83 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:47.676800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f1a4b-c6d8-45f0-b311-2df4605f2a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:47.680356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f1a4b-c6d8-45f0-b311-2df4605f2a01 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:47.685057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f1a4b-c6d8-45f0-b311-2df4605f2a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:47.686729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f1a4b-c6d8-45f0-b311-2df4605f2a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:47.687623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f1a4b-c6d8-45f0-b311-2df4605f2a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:47.689312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f1a4b-c6d8-45f0-b311-2df4605f2a01 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:47.718741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e843b-c81d-4fac-a2f3-3f290873d087 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:47.726224 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1687/3341] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:47.738481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e843b-c81d-4fac-a2f3-3f290873d087 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:47.749603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b8173-8290-4d9d-9cdb-2faa073ffa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:47.751133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b8173-8290-4d9d-9cdb-2faa073ffa02 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:47.754291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b8173-8290-4d9d-9cdb-2faa073ffa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:47.754786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b8173-8290-4d9d-9cdb-2faa073ffa02 admin admin] PATH_INFO: `/v3/projects/a484be1f1a70400ea15a4560a26f7dbe/users/e0fd11023fb64e60bdeeeb473be1e5ef/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:47.777205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b8173-8290-4d9d-9cdb-2faa073ffa02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a484be1f1a70400ea15a4560a26f7dbe, user_id=e0fd11023fb64e60bdeeeb473be1e5ef, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:47.781679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b8173-8290-4d9d-9cdb-2faa073ffa02 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:47.813707 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1655/3342] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:00:47 2022] PUT /identity/v3/projects/a484be1f1a70400ea15a4560a26f7dbe/users/e0fd11023fb64e60bdeeeb473be1e5ef/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:00:47.815086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f1a4b-c6d8-45f0-b311-2df4605f2a01 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:47.841839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f1a4b-c6d8-45f0-b311-2df4605f2a01 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:47.928717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c6c5d-e15f-4ad4-9b37-79232f6c14cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:47.930448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c6c5d-e15f-4ad4-9b37-79232f6c14cd admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:47.930966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c6c5d-e15f-4ad4-9b37-79232f6c14cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:47.931229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c6c5d-e15f-4ad4-9b37-79232f6c14cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:47.932120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c6c5d-e15f-4ad4-9b37-79232f6c14cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:47.934869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c6c5d-e15f-4ad4-9b37-79232f6c14cd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:47.947639 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1688/3343] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:47.953603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b8173-8290-4d9d-9cdb-2faa073ffa02 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:47.967235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b8173-8290-4d9d-9cdb-2faa073ffa02 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:47.975499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba186d5b-fbbd-4fa2-b95e-fe430595c184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:47.976367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba186d5b-fbbd-4fa2-b95e-fe430595c184 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:47.977546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba186d5b-fbbd-4fa2-b95e-fe430595c184 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:47.978405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba186d5b-fbbd-4fa2-b95e-fe430595c184 admin admin] PATH_INFO: `/v3/projects/a484be1f1a70400ea15a4560a26f7dbe/users/e0fd11023fb64e60bdeeeb473be1e5ef/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:47.981797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba186d5b-fbbd-4fa2-b95e-fe430595c184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a484be1f1a70400ea15a4560a26f7dbe, user_id=e0fd11023fb64e60bdeeeb473be1e5ef, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:47.987916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba186d5b-fbbd-4fa2-b95e-fe430595c184 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:48.020366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f69f85-3bf5-4baa-ad56-427f1f5c8cc1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:48.020879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f69f85-3bf5-4baa-ad56-427f1f5c8cc1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:48.021294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f69f85-3bf5-4baa-ad56-427f1f5c8cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:48.021616 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1656/3344] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:00:47 2022] PUT /identity/v3/projects/a484be1f1a70400ea15a4560a26f7dbe/users/e0fd11023fb64e60bdeeeb473be1e5ef/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:48.071816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-33f69f85-3bf5-4baa-ad56-427f1f5c8cc1 None None] MFA Rules not processed for user `e0fd11023fb64e60bdeeeb473be1e5ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:48.075600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-33f69f85-3bf5-4baa-ad56-427f1f5c8cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:48.293029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:48.312070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307a7fd-e8cc-4b20-802b-5dd9bd125deb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:48.312385 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1689/3345] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:48 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:48.312754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307a7fd-e8cc-4b20-802b-5dd9bd125deb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:48.313142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307a7fd-e8cc-4b20-802b-5dd9bd125deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:48.341629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b307a7fd-e8cc-4b20-802b-5dd9bd125deb None None] MFA Rules not processed for user `573c02d502744876babbdb4fbb4fb9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:48.344650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b307a7fd-e8cc-4b20-802b-5dd9bd125deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:48.453137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1657/3346] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:50.187985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dcf90-73d1-42f7-975a-074043d1c857 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:50.189432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dcf90-73d1-42f7-975a-074043d1c857 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:50.189732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dcf90-73d1-42f7-975a-074043d1c857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:50.218626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0c5dcf90-73d1-42f7-975a-074043d1c857 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:50.230468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0c5dcf90-73d1-42f7-975a-074043d1c857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:50.324656 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1690/3347] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:00:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:50.338122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307a7fd-e8cc-4b20-802b-5dd9bd125deb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:50.340523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b307a7fd-e8cc-4b20-802b-5dd9bd125deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:50.427881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:50.456701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307a7fd-e8cc-4b20-802b-5dd9bd125deb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:50.466227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ba4c2-85df-4965-98f7-8e14a57595ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:50.466859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ba4c2-85df-4965-98f7-8e14a57595ac service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:50.467134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ba4c2-85df-4965-98f7-8e14a57595ac service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:50.469268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ba4c2-85df-4965-98f7-8e14a57595ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:50.483937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ba4c2-85df-4965-98f7-8e14a57595ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:50.498008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ba4c2-85df-4965-98f7-8e14a57595ac service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:50.535647 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1658/3348] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:00:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:50.933386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051d2eb-b76a-4255-839a-83756ce52166 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:50.933603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051d2eb-b76a-4255-839a-83756ce52166 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:50.933982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051d2eb-b76a-4255-839a-83756ce52166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:50.981236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e051d2eb-b76a-4255-839a-83756ce52166 None None] MFA Rules not processed for user `84c19569dc6d49f4a29c814ef921e144`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:50.982615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e051d2eb-b76a-4255-839a-83756ce52166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:51.021664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ba4c2-85df-4965-98f7-8e14a57595ac service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:51.023277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-276ba4c2-85df-4965-98f7-8e14a57595ac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:51.163305 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1691/3349] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:51.165198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504022c2-f6f6-4fd3-bb1a-be985c4b10f9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:51.166602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504022c2-f6f6-4fd3-bb1a-be985c4b10f9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:51.167538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504022c2-f6f6-4fd3-bb1a-be985c4b10f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:51.224705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-504022c2-f6f6-4fd3-bb1a-be985c4b10f9 None None] MFA Rules not processed for user `84c19569dc6d49f4a29c814ef921e144`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:51.227197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-504022c2-f6f6-4fd3-bb1a-be985c4b10f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:51.247201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ba4c2-85df-4965-98f7-8e14a57595ac service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:51.256219 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1692/3350] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:51.259655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63195034-4d8d-43c4-8d78-5c4225ee3f99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:51.260395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63195034-4d8d-43c4-8d78-5c4225ee3f99 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:51.261448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63195034-4d8d-43c4-8d78-5c4225ee3f99 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:51.261982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63195034-4d8d-43c4-8d78-5c4225ee3f99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:51.272097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63195034-4d8d-43c4-8d78-5c4225ee3f99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:51.274134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63195034-4d8d-43c4-8d78-5c4225ee3f99 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:51.285204 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1659/3351] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:53.902812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504022c2-f6f6-4fd3-bb1a-be985c4b10f9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:53.905382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-504022c2-f6f6-4fd3-bb1a-be985c4b10f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:54.078225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504022c2-f6f6-4fd3-bb1a-be985c4b10f9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:54.092723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e608a-37ab-46b6-9d84-a3f0bdffa1f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:54.093851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e608a-37ab-46b6-9d84-a3f0bdffa1f0 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:54.094106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e608a-37ab-46b6-9d84-a3f0bdffa1f0 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:54.094596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e608a-37ab-46b6-9d84-a3f0bdffa1f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:54.103313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-db0e608a-37ab-46b6-9d84-a3f0bdffa1f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:54.148854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e608a-37ab-46b6-9d84-a3f0bdffa1f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:54.157227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e608a-37ab-46b6-9d84-a3f0bdffa1f0 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:54.209682 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1693/3352] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:00:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:00:54.288124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac756b-0cb2-4c87-9d6d-2a9462148609 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:54.288765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac756b-0cb2-4c87-9d6d-2a9462148609 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:54.288765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac756b-0cb2-4c87-9d6d-2a9462148609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:54.351671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-55ac756b-0cb2-4c87-9d6d-2a9462148609 None None] MFA Rules not processed for user `f00da962dc434ae18a1ab29228846688`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:00:54.352933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-55ac756b-0cb2-4c87-9d6d-2a9462148609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:54.504747 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1660/3353] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:00:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:00:54.509340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e608a-37ab-46b6-9d84-a3f0bdffa1f0 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:54.511351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-db0e608a-37ab-46b6-9d84-a3f0bdffa1f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:00:54.606569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:54.693129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e608a-37ab-46b6-9d84-a3f0bdffa1f0 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:54.731465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806dc3-dc5a-446b-8651-74a1f62b9bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:54.734658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806dc3-dc5a-446b-8651-74a1f62b9bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:54.734936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806dc3-dc5a-446b-8651-74a1f62b9bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:54.735219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806dc3-dc5a-446b-8651-74a1f62b9bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:54.735888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd806dc3-dc5a-446b-8651-74a1f62b9bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:54.740931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd806dc3-dc5a-446b-8651-74a1f62b9bf3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:54.824015 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1694/3354] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:00:54 2022] POST /identity/v3/projects => generated 379 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:00:54.827877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac756b-0cb2-4c87-9d6d-2a9462148609 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:54.841207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac756b-0cb2-4c87-9d6d-2a9462148609 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:54.957525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661badb-b145-412e-8047-19e9625870e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:54.958548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661badb-b145-412e-8047-19e9625870e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:54.959704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661badb-b145-412e-8047-19e9625870e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:54.960178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661badb-b145-412e-8047-19e9625870e4 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:54.961360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661badb-b145-412e-8047-19e9625870e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:54.964502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661badb-b145-412e-8047-19e9625870e4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:54.974816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:55.002765 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1661/3355] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:00:54 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:00:55.004003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806dc3-dc5a-446b-8651-74a1f62b9bf3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:55.018932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806dc3-dc5a-446b-8651-74a1f62b9bf3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:55.036398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862a5c4-5b18-455e-835e-10dd4082be5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:55.037029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862a5c4-5b18-455e-835e-10dd4082be5c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:55.037342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862a5c4-5b18-455e-835e-10dd4082be5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:55.037724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862a5c4-5b18-455e-835e-10dd4082be5c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:55.038492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862a5c4-5b18-455e-835e-10dd4082be5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:55.040261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862a5c4-5b18-455e-835e-10dd4082be5c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:55.063641 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1695/3356] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:55.071394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661badb-b145-412e-8047-19e9625870e4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:55.105211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661badb-b145-412e-8047-19e9625870e4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:55.132322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2a808-63df-45ca-9f7c-6cab6e704d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:55.139821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2a808-63df-45ca-9f7c-6cab6e704d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:55.140215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2a808-63df-45ca-9f7c-6cab6e704d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:55.140501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2a808-63df-45ca-9f7c-6cab6e704d4f admin admin] PATH_INFO: `/v3/projects/d3c58c35393d47688321147f21e69d4a/users/3d212845339449069a8ecd805d1d8eef/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:55.174211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2a808-63df-45ca-9f7c-6cab6e704d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c58c35393d47688321147f21e69d4a, user_id=3d212845339449069a8ecd805d1d8eef, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:55.177679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2a808-63df-45ca-9f7c-6cab6e704d4f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:55.223154 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1662/3357] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:00:55 2022] PUT /identity/v3/projects/d3c58c35393d47688321147f21e69d4a/users/3d212845339449069a8ecd805d1d8eef/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:00:55.224443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862a5c4-5b18-455e-835e-10dd4082be5c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:55.245882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862a5c4-5b18-455e-835e-10dd4082be5c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:55.410679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c0755-6233-408a-ba8c-a8ca445ecbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:55.411294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c0755-6233-408a-ba8c-a8ca445ecbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:55.411567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c0755-6233-408a-ba8c-a8ca445ecbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:55.420467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c0755-6233-408a-ba8c-a8ca445ecbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:55.421213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c0755-6233-408a-ba8c-a8ca445ecbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:55.422848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c0755-6233-408a-ba8c-a8ca445ecbd2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:55.449873 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1696/3358] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:00:55.451020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2a808-63df-45ca-9f7c-6cab6e704d4f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:00:55.465333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2a808-63df-45ca-9f7c-6cab6e704d4f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:00:55.475582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de405c-d9a8-4f59-a09e-c182ebcb4019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:00:55.479774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de405c-d9a8-4f59-a09e-c182ebcb4019 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:00:55.480888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de405c-d9a8-4f59-a09e-c182ebcb4019 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:00:55.481273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de405c-d9a8-4f59-a09e-c182ebcb4019 admin admin] PATH_INFO: `/v3/projects/d3c58c35393d47688321147f21e69d4a/users/3d212845339449069a8ecd805d1d8eef/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:00:55.485721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de405c-d9a8-4f59-a09e-c182ebcb4019 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c58c35393d47688321147f21e69d4a, user_id=3d212845339449069a8ecd805d1d8eef, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:00:55.491166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de405c-d9a8-4f59-a09e-c182ebcb4019 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:00:55.528231 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1663/3359] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:00:55 2022] PUT /identity/v3/projects/d3c58c35393d47688321147f21e69d4a/users/3d212845339449069a8ecd805d1d8eef/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:01:00.541458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c0755-6233-408a-ba8c-a8ca445ecbd2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:00.544907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-338c0755-6233-408a-ba8c-a8ca445ecbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:00.880445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c0755-6233-408a-ba8c-a8ca445ecbd2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:01.031619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d81e1d-9d2c-4d6d-9cc4-d0df06062a91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:01.032242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d81e1d-9d2c-4d6d-9cc4-d0df06062a91 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:01.032517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d81e1d-9d2c-4d6d-9cc4-d0df06062a91 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:01.032789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d81e1d-9d2c-4d6d-9cc4-d0df06062a91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:01.040415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-31d81e1d-9d2c-4d6d-9cc4-d0df06062a91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:01.090389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:01.283265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d81e1d-9d2c-4d6d-9cc4-d0df06062a91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:01.285825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d81e1d-9d2c-4d6d-9cc4-d0df06062a91 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:01.310057 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1697/3360] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:01:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 769 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:01:01.451101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5509301-3808-462d-99ea-cf254b16541b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:01.452006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5509301-3808-462d-99ea-cf254b16541b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:01.452464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5509301-3808-462d-99ea-cf254b16541b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:01.513515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b5509301-3808-462d-99ea-cf254b16541b None None] MFA Rules not processed for user `3d212845339449069a8ecd805d1d8eef`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:01.514741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b5509301-3808-462d-99ea-cf254b16541b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:01.653368 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1664/3361] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:01.665263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d81e1d-9d2c-4d6d-9cc4-d0df06062a91 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:01.667235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-31d81e1d-9d2c-4d6d-9cc4-d0df06062a91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:01.794326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d81e1d-9d2c-4d6d-9cc4-d0df06062a91 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:01.800401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42609496-faab-42b7-9d91-3daa6200e168 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:01.801709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42609496-faab-42b7-9d91-3daa6200e168 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:01.801993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42609496-faab-42b7-9d91-3daa6200e168 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:01.802268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42609496-faab-42b7-9d91-3daa6200e168 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:01.964688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42609496-faab-42b7-9d91-3daa6200e168 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:01.966069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42609496-faab-42b7-9d91-3daa6200e168 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:01.996610 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1698/3362] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:01:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:01:02.787233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ac5a8-72fb-4ec8-b683-bff5791b5007 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:02.787691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ac5a8-72fb-4ec8-b683-bff5791b5007 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:02.787929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ac5a8-72fb-4ec8-b683-bff5791b5007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:02.824041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-534ac5a8-72fb-4ec8-b683-bff5791b5007 None None] MFA Rules not processed for user `6adaa12c2ca9469299994300bc856ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:02.831800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-534ac5a8-72fb-4ec8-b683-bff5791b5007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:02.959518 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1665/3363] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:10.424167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42609496-faab-42b7-9d91-3daa6200e168 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:10.425737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-42609496-faab-42b7-9d91-3daa6200e168 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:10.579504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:10.587971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42609496-faab-42b7-9d91-3daa6200e168 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:10.598625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:10.599450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:10.599735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:10.600057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] PATH_INFO: `/v3/users/83d85ef4ebdb452a93f24682e2bbd46c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:10.602239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d85ef4ebdb452a93f24682e2bbd46c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:10.605138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:10.683458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2881c7-a1c7-4974-a870-c37e1942a713 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:10.683993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2881c7-a1c7-4974-a870-c37e1942a713 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:10.683993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2881c7-a1c7-4974-a870-c37e1942a713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:10.719015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6a2881c7-a1c7-4974-a870-c37e1942a713 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:10.720175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6a2881c7-a1c7-4974-a870-c37e1942a713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:10.758466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d85ef4ebdb452a93f24682e2bbd46c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:01:10.785310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d85ef4ebdb452a93f24682e2bbd46c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:01:10.810112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d85ef4ebdb452a93f24682e2bbd46c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:01:10.823313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d85ef4ebdb452a93f24682e2bbd46c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:01:10.824089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-892fa224-0413-421a-9445-aa63c87f691a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d85ef4ebdb452a93f24682e2bbd46c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:01:10.830844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:10.844058 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1699/3364] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:01:10 2022] DELETE /identity/v3/users/83d85ef4ebdb452a93f24682e2bbd46c => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:01:10.848295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcaff3-bace-4bf1-a9ae-2c34a7ce4e94 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:10.848622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcaff3-bace-4bf1-a9ae-2c34a7ce4e94 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:10.848897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcaff3-bace-4bf1-a9ae-2c34a7ce4e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:10.869137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1666/3365] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:01:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:01:10.891299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2881c7-a1c7-4974-a870-c37e1942a713 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:10.892752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6a2881c7-a1c7-4974-a870-c37e1942a713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:10.901460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2afcaff3-bace-4bf1-a9ae-2c34a7ce4e94 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:10.908246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2afcaff3-bace-4bf1-a9ae-2c34a7ce4e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:11.045497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2881c7-a1c7-4974-a870-c37e1942a713 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:11.067808 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1700/3366] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:11.096415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcaff3-bace-4bf1-a9ae-2c34a7ce4e94 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:11.098703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2afcaff3-bace-4bf1-a9ae-2c34a7ce4e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:11.209088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4592b00-fecf-4427-9e2a-98518c7e3fce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:11.209959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4592b00-fecf-4427-9e2a-98518c7e3fce service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:11.210394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4592b00-fecf-4427-9e2a-98518c7e3fce service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:11.210817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4592b00-fecf-4427-9e2a-98518c7e3fce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:11.262885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcaff3-bace-4bf1-a9ae-2c34a7ce4e94 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:11.268957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe388f9-13f8-4b4d-8804-6c57d58c488e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:11.269725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe388f9-13f8-4b4d-8804-6c57d58c488e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:11.270121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe388f9-13f8-4b4d-8804-6c57d58c488e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:11.270488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe388f9-13f8-4b4d-8804-6c57d58c488e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:11.289192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe388f9-13f8-4b4d-8804-6c57d58c488e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:11.290880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe388f9-13f8-4b4d-8804-6c57d58c488e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:11.311757 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1701/3367] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:01:11.365395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4592b00-fecf-4427-9e2a-98518c7e3fce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:11.367090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4592b00-fecf-4427-9e2a-98518c7e3fce service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:11.393277 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1667/3368] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:01:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:01:12.045885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe388f9-13f8-4b4d-8804-6c57d58c488e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:12.047765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7fe388f9-13f8-4b4d-8804-6c57d58c488e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:12.072264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe388f9-13f8-4b4d-8804-6c57d58c488e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:12.089018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:12.089852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:12.090234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:12.090770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] PATH_INFO: `/v3/projects/57ac50a5098942e58c8ab72ff09d5159` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:12.092388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ac50a5098942e58c8ab72ff09d5159)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:12.113733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:12.169589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:12.327742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ac50a5098942e58c8ab72ff09d5159', 'request_id': 'req-5dc19136-b599-42c5-b519-c5c2dc8e1179'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:01:12.356213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ac50a5098942e58c8ab72ff09d5159', 'request_id': 'req-5dc19136-b599-42c5-b519-c5c2dc8e1179'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:01:12.374412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ac50a5098942e58c8ab72ff09d5159', 'request_id': 'req-5dc19136-b599-42c5-b519-c5c2dc8e1179'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:01:12.403374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] The token cache is being invalidate because project 57ac50a5098942e58c8ab72ff09d5159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:01:12.403668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:01:12.406768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5dc19136-b599-42c5-b519-c5c2dc8e1179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:01:12.422154 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1702/3369] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:01:12 2022] DELETE /identity/v3/projects/57ac50a5098942e58c8ab72ff09d5159 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:01:12.431417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84d751-7e07-4910-8899-2fc5fbb91c1f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:12.432577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84d751-7e07-4910-8899-2fc5fbb91c1f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:12.432577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84d751-7e07-4910-8899-2fc5fbb91c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:12.468375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ca84d751-7e07-4910-8899-2fc5fbb91c1f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:12.470754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ca84d751-7e07-4910-8899-2fc5fbb91c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:12.532006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:12.651652 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1668/3370] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:12.653158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751a5e2-459b-4ab9-af9d-3c1b9066ddb4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:12.653480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751a5e2-459b-4ab9-af9d-3c1b9066ddb4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:12.653732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751a5e2-459b-4ab9-af9d-3c1b9066ddb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:12.707525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2751a5e2-459b-4ab9-af9d-3c1b9066ddb4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:12.708796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2751a5e2-459b-4ab9-af9d-3c1b9066ddb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:12.735716 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1703/3371] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:12.736865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84d751-7e07-4910-8899-2fc5fbb91c1f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:12.750700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84d751-7e07-4910-8899-2fc5fbb91c1f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:12.777694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2ac76-20d2-4707-aaf9-0344c9968bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:12.778762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2ac76-20d2-4707-aaf9-0344c9968bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:12.781240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2ac76-20d2-4707-aaf9-0344c9968bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:12.784118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2ac76-20d2-4707-aaf9-0344c9968bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:12.785080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2ac76-20d2-4707-aaf9-0344c9968bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:01:12.792653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2ac76-20d2-4707-aaf9-0344c9968bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:12.794911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2ac76-20d2-4707-aaf9-0344c9968bf4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:12.852133 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1669/3372] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:01:12.858039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751a5e2-459b-4ab9-af9d-3c1b9066ddb4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:12.871937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751a5e2-459b-4ab9-af9d-3c1b9066ddb4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:12.887484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a75e8f-2729-4c23-98d9-baaa492e6e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:12.888100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a75e8f-2729-4c23-98d9-baaa492e6e0b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:12.888378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a75e8f-2729-4c23-98d9-baaa492e6e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:12.888648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a75e8f-2729-4c23-98d9-baaa492e6e0b admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:12.889644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a75e8f-2729-4c23-98d9-baaa492e6e0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:12.891914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a75e8f-2729-4c23-98d9-baaa492e6e0b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:12.962942 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1704/3373] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:01:12 2022] POST /identity/v3/projects => generated 403 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:01:12.967757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2ac76-20d2-4707-aaf9-0344c9968bf4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:12.988377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2ac76-20d2-4707-aaf9-0344c9968bf4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:13.188225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee92ba-0513-4d1e-a1d1-53b8ccd6af82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:13.189836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee92ba-0513-4d1e-a1d1-53b8ccd6af82 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:13.190233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee92ba-0513-4d1e-a1d1-53b8ccd6af82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:13.198111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee92ba-0513-4d1e-a1d1-53b8ccd6af82 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:13.199226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee92ba-0513-4d1e-a1d1-53b8ccd6af82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:13.209403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee92ba-0513-4d1e-a1d1-53b8ccd6af82 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:13.295682 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1670/3374] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:01:12 2022] POST /identity/v3/users => generated 302 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:01:13.300293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a75e8f-2729-4c23-98d9-baaa492e6e0b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:13.328523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a75e8f-2729-4c23-98d9-baaa492e6e0b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:13.347398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392b31f-ed6c-4f4f-b15a-849be1a05433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:13.348405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392b31f-ed6c-4f4f-b15a-849be1a05433 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:13.348897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392b31f-ed6c-4f4f-b15a-849be1a05433 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:13.349468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392b31f-ed6c-4f4f-b15a-849be1a05433 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:13.350694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392b31f-ed6c-4f4f-b15a-849be1a05433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:13.353556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392b31f-ed6c-4f4f-b15a-849be1a05433 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:13.393367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee92ba-0513-4d1e-a1d1-53b8ccd6af82 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:13.402086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1705/3375] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:01:13.409854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee92ba-0513-4d1e-a1d1-53b8ccd6af82 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:13.432573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750797c0-d151-4000-a2bd-1330ec77423e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:13.438295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750797c0-d151-4000-a2bd-1330ec77423e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:13.441886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750797c0-d151-4000-a2bd-1330ec77423e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:13.442821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750797c0-d151-4000-a2bd-1330ec77423e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:13.446226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750797c0-d151-4000-a2bd-1330ec77423e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:13.452947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750797c0-d151-4000-a2bd-1330ec77423e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:13.503858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392b31f-ed6c-4f4f-b15a-849be1a05433 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:13.516083 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1671/3376] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:01:13.532463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392b31f-ed6c-4f4f-b15a-849be1a05433 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:13.566574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694cee-54d5-483d-86b7-ac0c15f175a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:13.567926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694cee-54d5-483d-86b7-ac0c15f175a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:13.568443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694cee-54d5-483d-86b7-ac0c15f175a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:13.576288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694cee-54d5-483d-86b7-ac0c15f175a1 admin admin] PATH_INFO: `/v3/projects/453ad8b708754a7ea5788abe2cc688d7/users/5d625eb28f0946f5905d4e99543fa014/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:13.626637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef694cee-54d5-483d-86b7-ac0c15f175a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=453ad8b708754a7ea5788abe2cc688d7, user_id=5d625eb28f0946f5905d4e99543fa014, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:13.633396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef694cee-54d5-483d-86b7-ac0c15f175a1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:13.664085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1706/3377] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:01:13 2022] PUT /identity/v3/projects/453ad8b708754a7ea5788abe2cc688d7/users/5d625eb28f0946f5905d4e99543fa014/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:01:13.669786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45b510-1309-42af-b337-2d5e1347d0b1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:13.670389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45b510-1309-42af-b337-2d5e1347d0b1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:13.670970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45b510-1309-42af-b337-2d5e1347d0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:13.709138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3c45b510-1309-42af-b337-2d5e1347d0b1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:13.710829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3c45b510-1309-42af-b337-2d5e1347d0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:13.871833 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1672/3378] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:13.882424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694cee-54d5-483d-86b7-ac0c15f175a1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:13.884447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ef694cee-54d5-483d-86b7-ac0c15f175a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:14.009164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694cee-54d5-483d-86b7-ac0c15f175a1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:14.016878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b6a8a-3452-4a33-b9ab-9d1a27109f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:14.017625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b6a8a-3452-4a33-b9ab-9d1a27109f2c service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:14.018000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b6a8a-3452-4a33-b9ab-9d1a27109f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:14.022017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b6a8a-3452-4a33-b9ab-9d1a27109f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:14.038140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b6a8a-3452-4a33-b9ab-9d1a27109f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:14.041419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b6a8a-3452-4a33-b9ab-9d1a27109f2c service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:14.064867 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1707/3379] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:01:22.058830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613df98-c492-4a0d-a9f3-eb9eda0af130 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:22.059361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613df98-c492-4a0d-a9f3-eb9eda0af130 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:22.059602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613df98-c492-4a0d-a9f3-eb9eda0af130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:22.075143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b6a8a-3452-4a33-b9ab-9d1a27109f2c service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:22.077979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2b0b6a8a-3452-4a33-b9ab-9d1a27109f2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:22.107392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b613df98-c492-4a0d-a9f3-eb9eda0af130 None None] MFA Rules not processed for user `4e1353ffe42243af82b590bc98e96a42`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:22.110384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b613df98-c492-4a0d-a9f3-eb9eda0af130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:22.195835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:22.231306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:22.269163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b6a8a-3452-4a33-b9ab-9d1a27109f2c service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:22.276721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a0ff5-4c4f-4f6f-8fe6-a39722ce03f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:22.277692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a0ff5-4c4f-4f6f-8fe6-a39722ce03f1 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:22.278531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a0ff5-4c4f-4f6f-8fe6-a39722ce03f1 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:22.278998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a0ff5-4c4f-4f6f-8fe6-a39722ce03f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:22.285368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a31a0ff5-4c4f-4f6f-8fe6-a39722ce03f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:22.288861 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1673/3380] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:22.377872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a0ff5-4c4f-4f6f-8fe6-a39722ce03f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:22.380836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a0ff5-4c4f-4f6f-8fe6-a39722ce03f1 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:22.392822 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1708/3381] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:01:22 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:01:22.432838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bec73-8362-4807-a87a-83b662e52707 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:22.432838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bec73-8362-4807-a87a-83b662e52707 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:22.432838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bec73-8362-4807-a87a-83b662e52707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:22.457370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ed6bec73-8362-4807-a87a-83b662e52707 None None] MFA Rules not processed for user `573c02d502744876babbdb4fbb4fb9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:22.460004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ed6bec73-8362-4807-a87a-83b662e52707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:22.530085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1674/3382] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:24.069370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c00e1-1d2e-4887-a8e5-8da04a28c4d8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:24.070041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c00e1-1d2e-4887-a8e5-8da04a28c4d8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:24.070041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c00e1-1d2e-4887-a8e5-8da04a28c4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:24.096041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-823c00e1-1d2e-4887-a8e5-8da04a28c4d8 None None] MFA Rules not processed for user `f00da962dc434ae18a1ab29228846688`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:24.097412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-823c00e1-1d2e-4887-a8e5-8da04a28c4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:24.237362 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1709/3383] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:24.239214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a9c8b-5d69-40dd-86b6-618638169921 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:24.239694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a9c8b-5d69-40dd-86b6-618638169921 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:24.239794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a9c8b-5d69-40dd-86b6-618638169921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:24.279516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-757a9c8b-5d69-40dd-86b6-618638169921 None None] MFA Rules not processed for user `f00da962dc434ae18a1ab29228846688`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:24.281387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-757a9c8b-5d69-40dd-86b6-618638169921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:24.298888 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1675/3384] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:27.466224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c00e1-1d2e-4887-a8e5-8da04a28c4d8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:27.490228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c00e1-1d2e-4887-a8e5-8da04a28c4d8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:27.513621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a0ea8-c387-4a46-9fa3-d0c6d7ed8c95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:27.514255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a0ea8-c387-4a46-9fa3-d0c6d7ed8c95 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:27.514531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a0ea8-c387-4a46-9fa3-d0c6d7ed8c95 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:27.514801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a0ea8-c387-4a46-9fa3-d0c6d7ed8c95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:27.521441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-368a0ea8-c387-4a46-9fa3-d0c6d7ed8c95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:27.704849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a0ea8-c387-4a46-9fa3-d0c6d7ed8c95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:27.706825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a0ea8-c387-4a46-9fa3-d0c6d7ed8c95 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:27.721834 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1710/3385] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:01:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:01:29.541917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de8994-fe71-433a-9cff-68aa58f06b44 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:29.542413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de8994-fe71-433a-9cff-68aa58f06b44 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:29.542500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de8994-fe71-433a-9cff-68aa58f06b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:29.586238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-21de8994-fe71-433a-9cff-68aa58f06b44 None None] MFA Rules not processed for user `5d625eb28f0946f5905d4e99543fa014`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:29.587848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-21de8994-fe71-433a-9cff-68aa58f06b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:29.685425 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1676/3386] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:01:29 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:30.657668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f533a4d-670c-46f1-8e06-d08603ba3994 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:30.658166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f533a4d-670c-46f1-8e06-d08603ba3994 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:30.658271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f533a4d-670c-46f1-8e06-d08603ba3994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:30.682279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1f533a4d-670c-46f1-8e06-d08603ba3994 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:01:30.683857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1f533a4d-670c-46f1-8e06-d08603ba3994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:30.700374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:30.756217 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1711/3387] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:01:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:01:30.772929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de8994-fe71-433a-9cff-68aa58f06b44 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:30.774789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-21de8994-fe71-433a-9cff-68aa58f06b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:01:30.859438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de8994-fe71-433a-9cff-68aa58f06b44 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:30.865153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abd41d-e189-4b4c-b337-6daddcf79917 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:30.865911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abd41d-e189-4b4c-b337-6daddcf79917 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:30.866287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abd41d-e189-4b4c-b337-6daddcf79917 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:30.866654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abd41d-e189-4b4c-b337-6daddcf79917 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:30.877718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abd41d-e189-4b4c-b337-6daddcf79917 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:30.879456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abd41d-e189-4b4c-b337-6daddcf79917 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:30.889660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:30.895569 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1677/3388] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:01:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:01:39.781922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f533a4d-670c-46f1-8e06-d08603ba3994 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:01:39.796821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f533a4d-670c-46f1-8e06-d08603ba3994 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:01:39.803021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618fd3c-3e88-45f4-9f20-ac248ed735d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:01:39.803618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618fd3c-3e88-45f4-9f20-ac248ed735d9 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:01:39.803889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618fd3c-3e88-45f4-9f20-ac248ed735d9 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:01:39.804152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618fd3c-3e88-45f4-9f20-ac248ed735d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:01:39.812733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618fd3c-3e88-45f4-9f20-ac248ed735d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:01:39.814348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618fd3c-3e88-45f4-9f20-ac248ed735d9 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:01:39.824060 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1712/3389] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:01:39 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:01:49.819455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=24 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:12.865358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07670f6a-d9d8-45f4-bde5-8ec8c2cfff77 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:12.865897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07670f6a-d9d8-45f4-bde5-8ec8c2cfff77 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:12.865986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07670f6a-d9d8-45f4-bde5-8ec8c2cfff77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:12.923504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-07670f6a-d9d8-45f4-bde5-8ec8c2cfff77 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:12.927762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-07670f6a-d9d8-45f4-bde5-8ec8c2cfff77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:12.952397 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1678/3390] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:02:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:12.974097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618fd3c-3e88-45f4-9f20-ac248ed735d9 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:12.996283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618fd3c-3e88-45f4-9f20-ac248ed735d9 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:13.013092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45058159-5418-4972-9dd1-940022b273fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:13.013941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45058159-5418-4972-9dd1-940022b273fa service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:13.014463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45058159-5418-4972-9dd1-940022b273fa service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:13.019432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45058159-5418-4972-9dd1-940022b273fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:13.036560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45058159-5418-4972-9dd1-940022b273fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:13.041906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45058159-5418-4972-9dd1-940022b273fa service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:13.054037 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1713/3391] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:02:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:14.153713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07670f6a-d9d8-45f4-bde5-8ec8c2cfff77 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:14.169574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07670f6a-d9d8-45f4-bde5-8ec8c2cfff77 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:14.179379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b4ffd-c4ee-4259-a90e-5ed9982f7a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:14.180566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b4ffd-c4ee-4259-a90e-5ed9982f7a8f service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:14.181567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b4ffd-c4ee-4259-a90e-5ed9982f7a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:14.183756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b4ffd-c4ee-4259-a90e-5ed9982f7a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:14.187686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-752b4ffd-c4ee-4259-a90e-5ed9982f7a8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:14.227312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b4ffd-c4ee-4259-a90e-5ed9982f7a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:14.230735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b4ffd-c4ee-4259-a90e-5ed9982f7a8f service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:14.245945 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1679/3392] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:14.388087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45058159-5418-4972-9dd1-940022b273fa service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:14.390596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-45058159-5418-4972-9dd1-940022b273fa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:14.536841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b4ffd-c4ee-4259-a90e-5ed9982f7a8f service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:14.539212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-752b4ffd-c4ee-4259-a90e-5ed9982f7a8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:14.569309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b4ffd-c4ee-4259-a90e-5ed9982f7a8f service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:14.585517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:14.589637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:14.591514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:14.591954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] PATH_INFO: `/v3/users/4e1353ffe42243af82b590bc98e96a42` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:14.594530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e1353ffe42243af82b590bc98e96a42)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:14.599612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:14.603584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45058159-5418-4972-9dd1-940022b273fa service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:14.622263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdb0af-9d11-4a49-a746-073b8594f191 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:14.625306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdb0af-9d11-4a49-a746-073b8594f191 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:14.626456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdb0af-9d11-4a49-a746-073b8594f191 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:14.627015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdb0af-9d11-4a49-a746-073b8594f191 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:14.640022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-89fdb0af-9d11-4a49-a746-073b8594f191 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:14.684746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdb0af-9d11-4a49-a746-073b8594f191 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:14.686383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdb0af-9d11-4a49-a746-073b8594f191 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:14.717524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e1353ffe42243af82b590bc98e96a42'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:14.720517 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1714/3393] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:14.736064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e1353ffe42243af82b590bc98e96a42'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:14.794009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e1353ffe42243af82b590bc98e96a42'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:14.814801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e1353ffe42243af82b590bc98e96a42'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:14.815931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38040f20-a8f1-48e2-ae31-cc169749aae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e1353ffe42243af82b590bc98e96a42'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:14.826493 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1680/3394] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:02:14 2022] DELETE /identity/v3/users/4e1353ffe42243af82b590bc98e96a42 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:02:15.833622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdb0af-9d11-4a49-a746-073b8594f191 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:15.853665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdb0af-9d11-4a49-a746-073b8594f191 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:15.996440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:15.997205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:15.997551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:15.997882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] PATH_INFO: `/v3/projects/9bc93093ddf44aada9598a14b5417a2d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:15.999575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc93093ddf44aada9598a14b5417a2d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:16.001759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:16.118394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc93093ddf44aada9598a14b5417a2d', 'request_id': 'req-adcb5559-f2b5-49cb-afc1-482a9414544d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:16.138597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc93093ddf44aada9598a14b5417a2d', 'request_id': 'req-adcb5559-f2b5-49cb-afc1-482a9414544d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:16.151210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc93093ddf44aada9598a14b5417a2d', 'request_id': 'req-adcb5559-f2b5-49cb-afc1-482a9414544d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:16.191272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] The token cache is being invalidate because project 9bc93093ddf44aada9598a14b5417a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:02:16.191699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:16.198009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-adcb5559-f2b5-49cb-afc1-482a9414544d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:16.204714 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1715/3395] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:02:15 2022] DELETE /identity/v3/projects/9bc93093ddf44aada9598a14b5417a2d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:16.206572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d35a97-252f-416f-9670-9ab746158ae0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:16.207458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d35a97-252f-416f-9670-9ab746158ae0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:16.207895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d35a97-252f-416f-9670-9ab746158ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:16.249737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-70d35a97-252f-416f-9670-9ab746158ae0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:16.253081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-70d35a97-252f-416f-9670-9ab746158ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:16.377329 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1681/3396] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:02:16.385003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef898b-8977-416b-aa8a-353f6819beea None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:16.385443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef898b-8977-416b-aa8a-353f6819beea None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:16.385828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef898b-8977-416b-aa8a-353f6819beea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:16.415381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:16.424854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5fef898b-8977-416b-aa8a-353f6819beea None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:16.430461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5fef898b-8977-416b-aa8a-353f6819beea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:16.455655 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1716/3397] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:02:16.459937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d35a97-252f-416f-9670-9ab746158ae0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:16.484189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d35a97-252f-416f-9670-9ab746158ae0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:16.496032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52304a3-72f6-4e81-871e-f13729f17a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:16.496500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52304a3-72f6-4e81-871e-f13729f17a22 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:16.496778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52304a3-72f6-4e81-871e-f13729f17a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:16.501109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52304a3-72f6-4e81-871e-f13729f17a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:16.502949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52304a3-72f6-4e81-871e-f13729f17a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:02:16.503152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52304a3-72f6-4e81-871e-f13729f17a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:16.504705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52304a3-72f6-4e81-871e-f13729f17a22 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:16.577047 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1682/3398] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:02:16.581443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef898b-8977-416b-aa8a-353f6819beea None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:16.612344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef898b-8977-416b-aa8a-353f6819beea None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:16.636415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72b7cb-28e6-43c9-a127-2849bd996f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:16.638901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72b7cb-28e6-43c9-a127-2849bd996f16 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:16.640808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72b7cb-28e6-43c9-a127-2849bd996f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:16.641320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72b7cb-28e6-43c9-a127-2849bd996f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:16.642661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72b7cb-28e6-43c9-a127-2849bd996f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:16.646697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72b7cb-28e6-43c9-a127-2849bd996f16 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:16.733907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1717/3399] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:02:16 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:02:16.734850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52304a3-72f6-4e81-871e-f13729f17a22 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:16.748771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52304a3-72f6-4e81-871e-f13729f17a22 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:16.899654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:16.955910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359050e6-db17-4b83-a7ef-5750dda932a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:16.955910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359050e6-db17-4b83-a7ef-5750dda932a3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:16.955910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359050e6-db17-4b83-a7ef-5750dda932a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:16.955910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359050e6-db17-4b83-a7ef-5750dda932a3 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:16.958608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359050e6-db17-4b83-a7ef-5750dda932a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:16.963019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359050e6-db17-4b83-a7ef-5750dda932a3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:17.015674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72b7cb-28e6-43c9-a127-2849bd996f16 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:17.023356 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1683/3400] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:02:16 2022] POST /identity/v3/users => generated 285 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:02:17.029258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72b7cb-28e6-43c9-a127-2849bd996f16 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:17.036098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a16a2-d7a9-4120-b764-2087eb4e0e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:17.036865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a16a2-d7a9-4120-b764-2087eb4e0e1d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:17.037278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a16a2-d7a9-4120-b764-2087eb4e0e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:17.037647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a16a2-d7a9-4120-b764-2087eb4e0e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:17.038413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a16a2-d7a9-4120-b764-2087eb4e0e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:17.041155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a16a2-d7a9-4120-b764-2087eb4e0e1d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:17.062102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359050e6-db17-4b83-a7ef-5750dda932a3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:17.073221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1718/3401] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:17.080050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359050e6-db17-4b83-a7ef-5750dda932a3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:17.089655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ad208-341f-4b28-b7b6-46ba782f6e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:17.090480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ad208-341f-4b28-b7b6-46ba782f6e70 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:17.090861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ad208-341f-4b28-b7b6-46ba782f6e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:17.091231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ad208-341f-4b28-b7b6-46ba782f6e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:17.093184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ad208-341f-4b28-b7b6-46ba782f6e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:17.096750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ad208-341f-4b28-b7b6-46ba782f6e70 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:17.112783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a16a2-d7a9-4120-b764-2087eb4e0e1d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:17.113457 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1684/3402] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:17.127649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a16a2-d7a9-4120-b764-2087eb4e0e1d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:17.136605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c9a3b-ebc6-47a7-b19b-e37e8e01959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:17.137830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9a3b-ebc6-47a7-b19b-e37e8e01959d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:17.138329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9a3b-ebc6-47a7-b19b-e37e8e01959d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:17.139272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9a3b-ebc6-47a7-b19b-e37e8e01959d admin admin] PATH_INFO: `/v3/projects/26a4431be325443c85b0ecef861d42ff/users/c50ab103943e4e3bacdc2777892c3bfe/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:17.168355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c9a3b-ebc6-47a7-b19b-e37e8e01959d admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a4431be325443c85b0ecef861d42ff, user_id=c50ab103943e4e3bacdc2777892c3bfe, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:17.172772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c9a3b-ebc6-47a7-b19b-e37e8e01959d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:17.219988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb38cb-6f23-4275-a7a9-b3ea54e92ebf None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:17.220871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb38cb-6f23-4275-a7a9-b3ea54e92ebf None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:17.221454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb38cb-6f23-4275-a7a9-b3ea54e92ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:17.224491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1719/3403] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:02:17 2022] PUT /identity/v3/projects/26a4431be325443c85b0ecef861d42ff/users/c50ab103943e4e3bacdc2777892c3bfe/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:17.274036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-effb38cb-6f23-4275-a7a9-b3ea54e92ebf None None] MFA Rules not processed for user `c50ab103943e4e3bacdc2777892c3bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:17.275724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-effb38cb-6f23-4275-a7a9-b3ea54e92ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:17.437866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3419fe3-1249-46f5-874e-7742cb9b1eb8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:17.438251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3419fe3-1249-46f5-874e-7742cb9b1eb8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:17.438790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3419fe3-1249-46f5-874e-7742cb9b1eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:17.442911 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1685/3404] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:17.481001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b3419fe3-1249-46f5-874e-7742cb9b1eb8 None None] MFA Rules not processed for user `c50ab103943e4e3bacdc2777892c3bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:17.484174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b3419fe3-1249-46f5-874e-7742cb9b1eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:17.507004 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1720/3405] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:18.264538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb38cb-6f23-4275-a7a9-b3ea54e92ebf None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:18.266270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-effb38cb-6f23-4275-a7a9-b3ea54e92ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:18.367325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb38cb-6f23-4275-a7a9-b3ea54e92ebf None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:18.379880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:18.380938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:18.381553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:18.382209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] PATH_INFO: `/v3/users/573c02d502744876babbdb4fbb4fb9e6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:18.384584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=573c02d502744876babbdb4fbb4fb9e6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:18.387336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:18.484720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573c02d502744876babbdb4fbb4fb9e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:18.488009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:18.496476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573c02d502744876babbdb4fbb4fb9e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:18.520888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573c02d502744876babbdb4fbb4fb9e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:18.533628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573c02d502744876babbdb4fbb4fb9e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:18.534634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573c02d502744876babbdb4fbb4fb9e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:18.547559 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1686/3406] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:02:18 2022] DELETE /identity/v3/users/573c02d502744876babbdb4fbb4fb9e6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:18.549116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1c127-d0f4-4c05-a676-a61eb42db73f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:18.549600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1c127-d0f4-4c05-a676-a61eb42db73f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:18.549707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1c127-d0f4-4c05-a676-a61eb42db73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:18.582526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fae1c127-d0f4-4c05-a676-a61eb42db73f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:18.584307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fae1c127-d0f4-4c05-a676-a61eb42db73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:18.680025 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1721/3407] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:18.692223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:18.694948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:18.840912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c94db-5d52-4a13-afee-41ad9db39c4a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:18.861888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16eb99-ac8f-4ead-bc5a-c9de9b87f871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:18.863062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16eb99-ac8f-4ead-bc5a-c9de9b87f871 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:18.863729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16eb99-ac8f-4ead-bc5a-c9de9b87f871 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:18.864596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16eb99-ac8f-4ead-bc5a-c9de9b87f871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:18.893770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16eb99-ac8f-4ead-bc5a-c9de9b87f871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:18.895538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16eb99-ac8f-4ead-bc5a-c9de9b87f871 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:18.919842 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1687/3408] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:19.424008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1c127-d0f4-4c05-a676-a61eb42db73f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:19.425749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fae1c127-d0f4-4c05-a676-a61eb42db73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:19.447869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1c127-d0f4-4c05-a676-a61eb42db73f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:19.462582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:19.463798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:19.464270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:19.464717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] PATH_INFO: `/v3/projects/e91f96ee17114009a55fe2bf388a7aa7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:19.466763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e91f96ee17114009a55fe2bf388a7aa7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:19.473308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:19.610426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e91f96ee17114009a55fe2bf388a7aa7', 'request_id': 'req-8dc2c02c-d1c6-4fea-bff9-46b32e964824'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:19.630661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e91f96ee17114009a55fe2bf388a7aa7', 'request_id': 'req-8dc2c02c-d1c6-4fea-bff9-46b32e964824'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:19.639584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e91f96ee17114009a55fe2bf388a7aa7', 'request_id': 'req-8dc2c02c-d1c6-4fea-bff9-46b32e964824'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:19.685909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] The token cache is being invalidate because project e91f96ee17114009a55fe2bf388a7aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:02:19.686507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:19.697506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:19.702394 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1722/3409] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:02:19 2022] DELETE /identity/v3/projects/e91f96ee17114009a55fe2bf388a7aa7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:19.704873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16eb99-ac8f-4ead-bc5a-c9de9b87f871 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:19.721137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16eb99-ac8f-4ead-bc5a-c9de9b87f871 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:19.844029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:19.845283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:19.845878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:19.846444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] PATH_INFO: `/v3/users/e0fd11023fb64e60bdeeeb473be1e5ef` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:19.850667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0fd11023fb64e60bdeeeb473be1e5ef)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:19.857265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:19.967111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0fd11023fb64e60bdeeeb473be1e5ef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:19.979909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0fd11023fb64e60bdeeeb473be1e5ef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:19.997430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:19.999239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0fd11023fb64e60bdeeeb473be1e5ef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:20.018290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0fd11023fb64e60bdeeeb473be1e5ef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:20.021151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0fd11023fb64e60bdeeeb473be1e5ef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:20.040385 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1688/3410] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:02:19 2022] DELETE /identity/v3/users/e0fd11023fb64e60bdeeeb473be1e5ef => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:20.755402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:20.756790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:20.814042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:20.943341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c02c-d1c6-4fea-bff9-46b32e964824 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:21.131156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debb8f5-90ee-4935-8d0b-6b6adcd092f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:21.131911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb8f5-90ee-4935-8d0b-6b6adcd092f7 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:21.132797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb8f5-90ee-4935-8d0b-6b6adcd092f7 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:21.133335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb8f5-90ee-4935-8d0b-6b6adcd092f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:21.136539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6debb8f5-90ee-4935-8d0b-6b6adcd092f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:21.289223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debb8f5-90ee-4935-8d0b-6b6adcd092f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:21.291462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debb8f5-90ee-4935-8d0b-6b6adcd092f7 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:21.325490 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1723/3411] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:21.329342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:21.331424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:21.446756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6aea5-87a4-436d-9a93-58f923b37257 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:21.453562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:21.454950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:21.455329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:21.455700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] PATH_INFO: `/v3/projects/a484be1f1a70400ea15a4560a26f7dbe` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:21.456966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a484be1f1a70400ea15a4560a26f7dbe)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:21.459089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:21.583048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a484be1f1a70400ea15a4560a26f7dbe', 'request_id': 'req-2564b338-4938-424b-864a-2b58248e098b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:21.621836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a484be1f1a70400ea15a4560a26f7dbe', 'request_id': 'req-2564b338-4938-424b-864a-2b58248e098b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:21.647458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a484be1f1a70400ea15a4560a26f7dbe', 'request_id': 'req-2564b338-4938-424b-864a-2b58248e098b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:21.699338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] The token cache is being invalidate because project a484be1f1a70400ea15a4560a26f7dbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:02:21.699669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:21.708126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2564b338-4938-424b-864a-2b58248e098b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:21.718380 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1689/3412] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:02:21 2022] DELETE /identity/v3/projects/a484be1f1a70400ea15a4560a26f7dbe => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:02:21.724808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e1a16-6675-4308-8fb6-33a2f491d739 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:21.725540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e1a16-6675-4308-8fb6-33a2f491d739 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:21.726030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e1a16-6675-4308-8fb6-33a2f491d739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:21.766630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8b0e1a16-6675-4308-8fb6-33a2f491d739 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:21.768668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8b0e1a16-6675-4308-8fb6-33a2f491d739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:21.918666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:22.079796 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1724/3413] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:22.088268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81ebef-8a20-41b9-98d7-d4129250c37b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:22.088841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81ebef-8a20-41b9-98d7-d4129250c37b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:22.089233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81ebef-8a20-41b9-98d7-d4129250c37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:22.141303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fa81ebef-8a20-41b9-98d7-d4129250c37b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:22.145959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa81ebef-8a20-41b9-98d7-d4129250c37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:22.185830 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1690/3414] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:22.187155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e1a16-6675-4308-8fb6-33a2f491d739 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:22.207217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e1a16-6675-4308-8fb6-33a2f491d739 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:22.238222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a8886-87cd-426d-b887-5166ee1ebb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:22.238924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a8886-87cd-426d-b887-5166ee1ebb8b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:22.240052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a8886-87cd-426d-b887-5166ee1ebb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:22.241237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a8886-87cd-426d-b887-5166ee1ebb8b admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:22.247637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a8886-87cd-426d-b887-5166ee1ebb8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:02:22.248898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a8886-87cd-426d-b887-5166ee1ebb8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:22.261901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a8886-87cd-426d-b887-5166ee1ebb8b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:22.332225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81ebef-8a20-41b9-98d7-d4129250c37b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:22.340210 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1725/3415] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:02:22.361864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81ebef-8a20-41b9-98d7-d4129250c37b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:22.384989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fbd34-2ca1-4a57-ac7a-6ea701b463eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:22.385837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fbd34-2ca1-4a57-ac7a-6ea701b463eb admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:22.386247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fbd34-2ca1-4a57-ac7a-6ea701b463eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:22.386658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fbd34-2ca1-4a57-ac7a-6ea701b463eb admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:22.387431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fbd34-2ca1-4a57-ac7a-6ea701b463eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:22.389575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fbd34-2ca1-4a57-ac7a-6ea701b463eb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:22.547443 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1691/3416] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:02:22 2022] POST /identity/v3/projects => generated 377 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:02:22.548258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a8886-87cd-426d-b887-5166ee1ebb8b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:22.565615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a8886-87cd-426d-b887-5166ee1ebb8b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:22.720909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd4541-f7b6-4fae-9f5f-42e90c6eba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:22.721579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd4541-f7b6-4fae-9f5f-42e90c6eba65 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:22.723011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd4541-f7b6-4fae-9f5f-42e90c6eba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:22.723403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd4541-f7b6-4fae-9f5f-42e90c6eba65 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:22.725793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd4541-f7b6-4fae-9f5f-42e90c6eba65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:22.745263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd4541-f7b6-4fae-9f5f-42e90c6eba65 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:22.814728 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1726/3417] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:02:22 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:02:22.815955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fbd34-2ca1-4a57-ac7a-6ea701b463eb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:22.857334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fbd34-2ca1-4a57-ac7a-6ea701b463eb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:22.881941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd4541-f7b6-4fae-9f5f-42e90c6eba65 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:22.884746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-affd4541-f7b6-4fae-9f5f-42e90c6eba65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:22.896269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b99568b-88c3-4577-b6df-11fd825f432f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:22.897361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99568b-88c3-4577-b6df-11fd825f432f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:22.897896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99568b-88c3-4577-b6df-11fd825f432f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:22.898580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99568b-88c3-4577-b6df-11fd825f432f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:22.900818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b99568b-88c3-4577-b6df-11fd825f432f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:22.911954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b99568b-88c3-4577-b6df-11fd825f432f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:22.935518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd4541-f7b6-4fae-9f5f-42e90c6eba65 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:22.959301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:22.960228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:22.963999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:22.964483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] PATH_INFO: `/v3/users/5d93a680570f4d23b1f6df534bc082b2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:22.965137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1692/3418] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:22.966218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b99568b-88c3-4577-b6df-11fd825f432f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:22.968131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d93a680570f4d23b1f6df534bc082b2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:22.997725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:23.021341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b99568b-88c3-4577-b6df-11fd825f432f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:23.044033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b37d42-9217-46ae-9a19-aa1efaf68c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:23.045583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b37d42-9217-46ae-9a19-aa1efaf68c45 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:23.046189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b37d42-9217-46ae-9a19-aa1efaf68c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:23.046464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b37d42-9217-46ae-9a19-aa1efaf68c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:23.047793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b37d42-9217-46ae-9a19-aa1efaf68c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:23.053682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b37d42-9217-46ae-9a19-aa1efaf68c45 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:23.101971 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1693/3419] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:23.106298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b37d42-9217-46ae-9a19-aa1efaf68c45 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:23.144774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b37d42-9217-46ae-9a19-aa1efaf68c45 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:23.157467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d93a680570f4d23b1f6df534bc082b2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:23.159954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf12d4-38be-46d3-993c-dc981e8fe28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:23.166763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf12d4-38be-46d3-993c-dc981e8fe28c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:23.168489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf12d4-38be-46d3-993c-dc981e8fe28c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:23.168938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf12d4-38be-46d3-993c-dc981e8fe28c admin admin] PATH_INFO: `/v3/projects/5430106b642d476ebfe6866f130b3a56/users/0db8e2fbd08c4b658e95dd345cae7fb7/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:23.176511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:23.212313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d93a680570f4d23b1f6df534bc082b2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:23.227942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf12d4-38be-46d3-993c-dc981e8fe28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5430106b642d476ebfe6866f130b3a56, user_id=0db8e2fbd08c4b658e95dd345cae7fb7, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:23.234303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf12d4-38be-46d3-993c-dc981e8fe28c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:23.239734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d93a680570f4d23b1f6df534bc082b2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:23.260192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d93a680570f4d23b1f6df534bc082b2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:23.263651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d93a680570f4d23b1f6df534bc082b2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:23.279669 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1694/3420] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:02:23 2022] PUT /identity/v3/projects/5430106b642d476ebfe6866f130b3a56/users/0db8e2fbd08c4b658e95dd345cae7fb7/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:23.293420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a294b6-c75a-4def-9d5d-bb898df11f00 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:23.293700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a294b6-c75a-4def-9d5d-bb898df11f00 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:23.293969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a294b6-c75a-4def-9d5d-bb898df11f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:23.297271 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1727/3421] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:02:22 2022] DELETE /identity/v3/users/5d93a680570f4d23b1f6df534bc082b2 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:23.356411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f8a294b6-c75a-4def-9d5d-bb898df11f00 None None] MFA Rules not processed for user `0db8e2fbd08c4b658e95dd345cae7fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:23.357787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f8a294b6-c75a-4def-9d5d-bb898df11f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:23.609283 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1695/3422] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:23.838650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:23.841364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:23.848876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:24.057390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e551027-4524-4560-bd1f-77134a47f988 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:24.068901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:24.070791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:24.071574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:24.072822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] PATH_INFO: `/v3/projects/b62c28804ebd4f1f9ac47aa541500110` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:24.076333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62c28804ebd4f1f9ac47aa541500110)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:24.082273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:24.222721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62c28804ebd4f1f9ac47aa541500110', 'request_id': 'req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:24.241963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62c28804ebd4f1f9ac47aa541500110', 'request_id': 'req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:24.259020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62c28804ebd4f1f9ac47aa541500110', 'request_id': 'req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:24.302020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] The token cache is being invalidate because project b62c28804ebd4f1f9ac47aa541500110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:02:24.302461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:24.303301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:24.305731 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1728/3423] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:02:23 2022] DELETE /identity/v3/projects/b62c28804ebd4f1f9ac47aa541500110 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:24.310906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e68a18-1714-4973-9ace-3e8010e0e4af None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:24.311120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e68a18-1714-4973-9ace-3e8010e0e4af None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:24.311393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e68a18-1714-4973-9ace-3e8010e0e4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:24.355855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:24.357324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:24.378413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b4e68a18-1714-4973-9ace-3e8010e0e4af None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:24.379628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b4e68a18-1714-4973-9ace-3e8010e0e4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:24.536974 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1696/3424] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:24.544080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bbd43-e05e-49c4-84f2-020ec0f205f3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:24.544592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bbd43-e05e-49c4-84f2-020ec0f205f3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:24.545611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bbd43-e05e-49c4-84f2-020ec0f205f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:24.580932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c5fe0-9354-4c4c-8413-7b2f6077fc2e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:24.587048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-815bbd43-e05e-49c4-84f2-020ec0f205f3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:24.589156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-815bbd43-e05e-49c4-84f2-020ec0f205f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:24.595576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b220e-c24c-4b20-b67a-c86958ae575f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:24.596580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b220e-c24c-4b20-b67a-c86958ae575f service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:24.597064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b220e-c24c-4b20-b67a-c86958ae575f service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:24.597731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b220e-c24c-4b20-b67a-c86958ae575f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:24.600816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ec0b220e-c24c-4b20-b67a-c86958ae575f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:24.608655 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1697/3425] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:24.612895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bbd43-e05e-49c4-84f2-020ec0f205f3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:24.634565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bbd43-e05e-49c4-84f2-020ec0f205f3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:24.636286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:24.649414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b8eb8-3dbe-4009-b990-909756b48ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:24.650428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8eb8-3dbe-4009-b990-909756b48ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:24.650900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8eb8-3dbe-4009-b990-909756b48ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:24.651626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8eb8-3dbe-4009-b990-909756b48ca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:24.652555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b8eb8-3dbe-4009-b990-909756b48ca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:02:24.653663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b8eb8-3dbe-4009-b990-909756b48ca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:24.657948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b8eb8-3dbe-4009-b990-909756b48ca8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:24.689021 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1698/3426] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:02:24.692512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b8eb8-3dbe-4009-b990-909756b48ca8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:24.721005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b8eb8-3dbe-4009-b990-909756b48ca8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:24.732073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b6733-3817-40ec-9907-e0e2589f5f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:24.733331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b6733-3817-40ec-9907-e0e2589f5f6b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:24.734888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b6733-3817-40ec-9907-e0e2589f5f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:24.735708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b6733-3817-40ec-9907-e0e2589f5f6b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:24.739627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b6733-3817-40ec-9907-e0e2589f5f6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:24.742272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b6733-3817-40ec-9907-e0e2589f5f6b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:24.750516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b220e-c24c-4b20-b67a-c86958ae575f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:24.753332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b220e-c24c-4b20-b67a-c86958ae575f service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:24.786017 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1729/3427] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:02:24.844145 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1699/3428] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:02:24 2022] POST /identity/v3/projects => generated 397 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:02:24.848875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b220e-c24c-4b20-b67a-c86958ae575f service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:24.870251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b220e-c24c-4b20-b67a-c86958ae575f service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:25.064326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a5248-662a-43f8-8dc2-e009d48ffc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:25.065175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a5248-662a-43f8-8dc2-e009d48ffc65 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:25.065700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a5248-662a-43f8-8dc2-e009d48ffc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:25.071743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a5248-662a-43f8-8dc2-e009d48ffc65 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:25.072685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a5248-662a-43f8-8dc2-e009d48ffc65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:25.074797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a5248-662a-43f8-8dc2-e009d48ffc65 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:25.116287 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1730/3429] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:02:24 2022] POST /identity/v3/users => generated 299 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:02:25.116954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b6733-3817-40ec-9907-e0e2589f5f6b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:25.132614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b6733-3817-40ec-9907-e0e2589f5f6b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:25.135839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59791bf9-8065-40c7-8110-7443c502d90e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:25.136264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59791bf9-8065-40c7-8110-7443c502d90e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:25.136637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59791bf9-8065-40c7-8110-7443c502d90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:25.158058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7655bf-9033-4100-aa75-d89c81c80db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:25.158685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7655bf-9033-4100-aa75-d89c81c80db5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:25.158959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7655bf-9033-4100-aa75-d89c81c80db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:25.159229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7655bf-9033-4100-aa75-d89c81c80db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:25.159868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7655bf-9033-4100-aa75-d89c81c80db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:25.161487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7655bf-9033-4100-aa75-d89c81c80db5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:25.217439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-59791bf9-8065-40c7-8110-7443c502d90e None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:25.220879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-59791bf9-8065-40c7-8110-7443c502d90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:25.221618 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1700/3430] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:25.222569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7655bf-9033-4100-aa75-d89c81c80db5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:25.246063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7655bf-9033-4100-aa75-d89c81c80db5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:25.259754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88778e-45dd-499e-8dd7-80b3a8cc90e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:25.260887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88778e-45dd-499e-8dd7-80b3a8cc90e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:25.260887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88778e-45dd-499e-8dd7-80b3a8cc90e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:25.262324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88778e-45dd-499e-8dd7-80b3a8cc90e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:25.262624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88778e-45dd-499e-8dd7-80b3a8cc90e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:25.264341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88778e-45dd-499e-8dd7-80b3a8cc90e6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:25.291845 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1701/3431] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:25.293664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88778e-45dd-499e-8dd7-80b3a8cc90e6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:25.323099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88778e-45dd-499e-8dd7-80b3a8cc90e6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:25.347117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576fb14-463f-4de0-b10b-a3ba4e8e2f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:25.347962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576fb14-463f-4de0-b10b-a3ba4e8e2f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:25.348240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576fb14-463f-4de0-b10b-a3ba4e8e2f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:25.348814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576fb14-463f-4de0-b10b-a3ba4e8e2f91 admin admin] PATH_INFO: `/v3/projects/994259bbbe124b4e9ff0542ca402b9f7/users/a8af9544fe68497c8557248575f184ed/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:25.372529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:25.389017 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1731/3432] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:02:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:02:25.395447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576fb14-463f-4de0-b10b-a3ba4e8e2f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=994259bbbe124b4e9ff0542ca402b9f7, user_id=a8af9544fe68497c8557248575f184ed, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:25.400283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576fb14-463f-4de0-b10b-a3ba4e8e2f91 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:25.406384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59791bf9-8065-40c7-8110-7443c502d90e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:25.408774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-59791bf9-8065-40c7-8110-7443c502d90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:25.478474 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1702/3433] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:02:25 2022] PUT /identity/v3/projects/994259bbbe124b4e9ff0542ca402b9f7/users/a8af9544fe68497c8557248575f184ed/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:25.483260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1e9d1-a31f-4aa4-b45d-75fa7a215a72 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:25.483732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1e9d1-a31f-4aa4-b45d-75fa7a215a72 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:25.484160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1e9d1-a31f-4aa4-b45d-75fa7a215a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:25.536400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-40d1e9d1-a31f-4aa4-b45d-75fa7a215a72 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:25.538095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-40d1e9d1-a31f-4aa4-b45d-75fa7a215a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:25.617029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:25.639917 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1703/3434] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:25.653268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1e9d1-a31f-4aa4-b45d-75fa7a215a72 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:25.659797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-40d1e9d1-a31f-4aa4-b45d-75fa7a215a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:25.684779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59791bf9-8065-40c7-8110-7443c502d90e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:25.799771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1e9d1-a31f-4aa4-b45d-75fa7a215a72 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:25.811584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad945862-90ef-47a7-96ea-6ff4289c9b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:25.813640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad945862-90ef-47a7-96ea-6ff4289c9b58 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:25.814416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad945862-90ef-47a7-96ea-6ff4289c9b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:25.816085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad945862-90ef-47a7-96ea-6ff4289c9b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:25.833187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad945862-90ef-47a7-96ea-6ff4289c9b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:25.835712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad945862-90ef-47a7-96ea-6ff4289c9b58 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:25.844629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f30d27-6eee-48b0-9866-9147dedb0f3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:25.845238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f30d27-6eee-48b0-9866-9147dedb0f3f service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:25.845512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f30d27-6eee-48b0-9866-9147dedb0f3f service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:25.845839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f30d27-6eee-48b0-9866-9147dedb0f3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:25.865199 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1704/3435] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:02:26.003064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f30d27-6eee-48b0-9866-9147dedb0f3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:26.005052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f30d27-6eee-48b0-9866-9147dedb0f3f service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:26.031985 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1732/3436] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:02:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:26.369958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c65dae-c6df-4cb7-be3d-a6d4b3351f2e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:26.370395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c65dae-c6df-4cb7-be3d-a6d4b3351f2e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:26.370774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c65dae-c6df-4cb7-be3d-a6d4b3351f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:26.423222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f7c65dae-c6df-4cb7-be3d-a6d4b3351f2e None None] MFA Rules not processed for user `0db8e2fbd08c4b658e95dd345cae7fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:26.427007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f7c65dae-c6df-4cb7-be3d-a6d4b3351f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:26.618693 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1705/3437] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:27.861559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21efe96-7832-449f-8466-b4d6d7d6590e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:27.862997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21efe96-7832-449f-8466-b4d6d7d6590e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:27.863532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21efe96-7832-449f-8466-b4d6d7d6590e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:27.902253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a21efe96-7832-449f-8466-b4d6d7d6590e None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:27.904012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a21efe96-7832-449f-8466-b4d6d7d6590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:27.928548 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1733/3438] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:02:27.941597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c65dae-c6df-4cb7-be3d-a6d4b3351f2e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:27.965518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c65dae-c6df-4cb7-be3d-a6d4b3351f2e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:27.977809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596e8cd-313d-4b20-9ee7-03c49b875497 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:28.011699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596e8cd-313d-4b20-9ee7-03c49b875497 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:28.012178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596e8cd-313d-4b20-9ee7-03c49b875497 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:28.012700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596e8cd-313d-4b20-9ee7-03c49b875497 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:28.059139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596e8cd-313d-4b20-9ee7-03c49b875497 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:28.064527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596e8cd-313d-4b20-9ee7-03c49b875497 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:28.119662 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1706/3439] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:02:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:28.370172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21efe96-7832-449f-8466-b4d6d7d6590e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:28.372264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a21efe96-7832-449f-8466-b4d6d7d6590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:28.399847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21efe96-7832-449f-8466-b4d6d7d6590e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:28.413606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:28.415170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:28.415582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:28.416031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] PATH_INFO: `/v3/users/0db8e2fbd08c4b658e95dd345cae7fb7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:28.418509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db8e2fbd08c4b658e95dd345cae7fb7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:28.421823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:28.515048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596e8cd-313d-4b20-9ee7-03c49b875497 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:28.516913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d596e8cd-313d-4b20-9ee7-03c49b875497 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:28.522102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db8e2fbd08c4b658e95dd345cae7fb7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:28.547777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db8e2fbd08c4b658e95dd345cae7fb7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:28.549120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596e8cd-313d-4b20-9ee7-03c49b875497 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:28.562578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db8e2fbd08c4b658e95dd345cae7fb7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:28.562826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:28.563602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:28.563856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:28.565199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] PATH_INFO: `/v3/users/5d625eb28f0946f5905d4e99543fa014` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:28.569481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d625eb28f0946f5905d4e99543fa014)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:28.569739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:28.574823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:28.580417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db8e2fbd08c4b658e95dd345cae7fb7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:28.582060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d68f23f8-8e51-4410-94f3-09ddd815b7f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db8e2fbd08c4b658e95dd345cae7fb7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:28.602281 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1734/3440] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:02:28 2022] DELETE /identity/v3/users/0db8e2fbd08c4b658e95dd345cae7fb7 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:28.606354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337f4c3-a0b1-4c2e-af7b-24e43b8952a6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:28.606748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337f4c3-a0b1-4c2e-af7b-24e43b8952a6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:28.607264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337f4c3-a0b1-4c2e-af7b-24e43b8952a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:28.620935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:28.675361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b337f4c3-a0b1-4c2e-af7b-24e43b8952a6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:28.678777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b337f4c3-a0b1-4c2e-af7b-24e43b8952a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:28.753495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d625eb28f0946f5905d4e99543fa014'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:28.767005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d625eb28f0946f5905d4e99543fa014'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:28.800949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d625eb28f0946f5905d4e99543fa014'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:28.845526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d625eb28f0946f5905d4e99543fa014'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:28.850631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d625eb28f0946f5905d4e99543fa014'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:28.855255 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1735/3441] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:02:28.868959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337f4c3-a0b1-4c2e-af7b-24e43b8952a6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:28.870515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b337f4c3-a0b1-4c2e-af7b-24e43b8952a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:28.897005 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1707/3442] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:02:28 2022] DELETE /identity/v3/users/5d625eb28f0946f5905d4e99543fa014 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:29.008745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337f4c3-a0b1-4c2e-af7b-24e43b8952a6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:29.116527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b0b79-ba50-4e9b-a7f3-2b015159223f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:29.117911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b0b79-ba50-4e9b-a7f3-2b015159223f service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:29.118187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b0b79-ba50-4e9b-a7f3-2b015159223f service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:29.118457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b0b79-ba50-4e9b-a7f3-2b015159223f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:29.125634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-127b0b79-ba50-4e9b-a7f3-2b015159223f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:29.288839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b0b79-ba50-4e9b-a7f3-2b015159223f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:29.291999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b0b79-ba50-4e9b-a7f3-2b015159223f service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:29.298490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:29.306679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:29.313857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:29.342017 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1736/3443] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:29.344669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d9a04-2c99-40c0-a2fb-385eda25aa90 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:29.353553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:29.354749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:29.355488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:29.355968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] PATH_INFO: `/v3/projects/453ad8b708754a7ea5788abe2cc688d7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:29.362804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=453ad8b708754a7ea5788abe2cc688d7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:29.366128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:29.482305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453ad8b708754a7ea5788abe2cc688d7', 'request_id': 'req-fdf23bb3-0b99-4f73-a813-0c64415ff271'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:29.510080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453ad8b708754a7ea5788abe2cc688d7', 'request_id': 'req-fdf23bb3-0b99-4f73-a813-0c64415ff271'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:29.528454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453ad8b708754a7ea5788abe2cc688d7', 'request_id': 'req-fdf23bb3-0b99-4f73-a813-0c64415ff271'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:29.569395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] The token cache is being invalidate because project 453ad8b708754a7ea5788abe2cc688d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:02:29.571934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:29.573957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fdf23bb3-0b99-4f73-a813-0c64415ff271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:29.578703 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1708/3444] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:02:29 2022] DELETE /identity/v3/projects/453ad8b708754a7ea5788abe2cc688d7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:02:29.584379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee350254-a5fa-4034-86c4-163dc59f2960 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:29.584719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee350254-a5fa-4034-86c4-163dc59f2960 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:29.585243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee350254-a5fa-4034-86c4-163dc59f2960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:29.629278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ee350254-a5fa-4034-86c4-163dc59f2960 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:29.630526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ee350254-a5fa-4034-86c4-163dc59f2960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:29.750221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1737/3445] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:29.759516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0d1a3-4d82-4db7-adb9-7fc643e32f2a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:29.759727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0d1a3-4d82-4db7-adb9-7fc643e32f2a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:29.760029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0d1a3-4d82-4db7-adb9-7fc643e32f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:29.840332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-09f0d1a3-4d82-4db7-adb9-7fc643e32f2a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:29.843579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-09f0d1a3-4d82-4db7-adb9-7fc643e32f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:29.876801 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1709/3446] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:29.880062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee350254-a5fa-4034-86c4-163dc59f2960 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:29.911772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee350254-a5fa-4034-86c4-163dc59f2960 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:29.933057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1712b36-39c0-4a16-b788-ad3bd825fac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:29.934028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1712b36-39c0-4a16-b788-ad3bd825fac6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:29.934516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1712b36-39c0-4a16-b788-ad3bd825fac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:29.934993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1712b36-39c0-4a16-b788-ad3bd825fac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:29.935876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1712b36-39c0-4a16-b788-ad3bd825fac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:02:29.936662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1712b36-39c0-4a16-b788-ad3bd825fac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:29.939748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1712b36-39c0-4a16-b788-ad3bd825fac6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:29.980337 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1738/3447] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:02:29.984874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0d1a3-4d82-4db7-adb9-7fc643e32f2a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:30.000418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0d1a3-4d82-4db7-adb9-7fc643e32f2a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:30.016474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf24a91-1a8e-46c2-89de-4453106d5a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:30.019016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf24a91-1a8e-46c2-89de-4453106d5a3c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:30.022626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf24a91-1a8e-46c2-89de-4453106d5a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:30.022927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf24a91-1a8e-46c2-89de-4453106d5a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:30.026368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf24a91-1a8e-46c2-89de-4453106d5a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:30.033965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf24a91-1a8e-46c2-89de-4453106d5a3c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:30.125554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1712b36-39c0-4a16-b788-ad3bd825fac6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:30.127518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c1712b36-39c0-4a16-b788-ad3bd825fac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:30.134989 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1710/3448] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:02:29 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:02:30.141774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf24a91-1a8e-46c2-89de-4453106d5a3c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:30.167650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1712b36-39c0-4a16-b788-ad3bd825fac6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:30.250065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:30.279769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf24a91-1a8e-46c2-89de-4453106d5a3c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:30.301638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47ed5c-b857-422d-886f-d23c24a71761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:30.303164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47ed5c-b857-422d-886f-d23c24a71761 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:30.303726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47ed5c-b857-422d-886f-d23c24a71761 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:30.304205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47ed5c-b857-422d-886f-d23c24a71761 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:30.306857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47ed5c-b857-422d-886f-d23c24a71761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:30.316043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47ed5c-b857-422d-886f-d23c24a71761 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:30.339856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:30.342114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:30.342395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:30.342818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] PATH_INFO: `/v3/projects/5430106b642d476ebfe6866f130b3a56` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:30.344442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5430106b642d476ebfe6866f130b3a56)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:30.368418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:30.376926 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1711/3449] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:02:30 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:02:30.385174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47ed5c-b857-422d-886f-d23c24a71761 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:30.421644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47ed5c-b857-422d-886f-d23c24a71761 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:30.432034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0c8ac-409b-43f3-a757-e90628a5c0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:30.432647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0c8ac-409b-43f3-a757-e90628a5c0ab admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:30.432921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0c8ac-409b-43f3-a757-e90628a5c0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:30.433217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0c8ac-409b-43f3-a757-e90628a5c0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:30.433858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0c8ac-409b-43f3-a757-e90628a5c0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:30.435452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0c8ac-409b-43f3-a757-e90628a5c0ab admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:30.472072 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1712/3450] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:30.473161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0c8ac-409b-43f3-a757-e90628a5c0ab admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:30.495626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0c8ac-409b-43f3-a757-e90628a5c0ab admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:30.516490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5430106b642d476ebfe6866f130b3a56', 'request_id': 'req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:30.537357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5430106b642d476ebfe6866f130b3a56', 'request_id': 'req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:30.550893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5430106b642d476ebfe6866f130b3a56', 'request_id': 'req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:30.575476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] The token cache is being invalidate because project 5430106b642d476ebfe6866f130b3a56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:02:30.575476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:30.576821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-57a1a66b-73ee-4deb-a87e-dd89dd4cd0c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:30.582460 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1739/3451] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:02:30 2022] DELETE /identity/v3/projects/5430106b642d476ebfe6866f130b3a56 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:30.583845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6be7a-613a-4117-bb44-d4fa4130fa0b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:30.584990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6be7a-613a-4117-bb44-d4fa4130fa0b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:30.585324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6be7a-613a-4117-bb44-d4fa4130fa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:30.622849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-16e6be7a-613a-4117-bb44-d4fa4130fa0b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:30.627198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-16e6be7a-613a-4117-bb44-d4fa4130fa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:30.639089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3e46a-4456-47a8-b59c-a9bb1fe00e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:30.639841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3e46a-4456-47a8-b59c-a9bb1fe00e38 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:30.640208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3e46a-4456-47a8-b59c-a9bb1fe00e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:30.641144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3e46a-4456-47a8-b59c-a9bb1fe00e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:30.642602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3e46a-4456-47a8-b59c-a9bb1fe00e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:30.644434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3e46a-4456-47a8-b59c-a9bb1fe00e38 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:30.653168 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1740/3452] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:30.654655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68788d-4720-4fdd-8898-3ad7abb31d27 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:30.654937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68788d-4720-4fdd-8898-3ad7abb31d27 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:30.655204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68788d-4720-4fdd-8898-3ad7abb31d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:30.671262 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1713/3453] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:30.675140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3e46a-4456-47a8-b59c-a9bb1fe00e38 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:30.678663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-14b3e46a-4456-47a8-b59c-a9bb1fe00e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:30.701956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ee68788d-4720-4fdd-8898-3ad7abb31d27 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:30.703182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ee68788d-4720-4fdd-8898-3ad7abb31d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:30.729720 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1741/3454] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:30.730419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68788d-4720-4fdd-8898-3ad7abb31d27 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:30.732011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3e46a-4456-47a8-b59c-a9bb1fe00e38 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:30.734983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:30.743766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547902f-e0f0-4a1d-9d42-e37e4927839e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:30.750968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547902f-e0f0-4a1d-9d42-e37e4927839e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:30.751102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68788d-4720-4fdd-8898-3ad7abb31d27 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:30.751647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547902f-e0f0-4a1d-9d42-e37e4927839e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:30.753406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547902f-e0f0-4a1d-9d42-e37e4927839e admin admin] PATH_INFO: `/v3/projects/da46bd0ec0384f9398a890dcf12deb7a/users/6efa496c103449e486364cdef764054d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:30.763556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934f504-6579-4a36-9367-7f31eda2203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:30.765807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934f504-6579-4a36-9367-7f31eda2203b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:30.766181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934f504-6579-4a36-9367-7f31eda2203b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:30.767919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934f504-6579-4a36-9367-7f31eda2203b admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:30.769400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934f504-6579-4a36-9367-7f31eda2203b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:02:30.770614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934f504-6579-4a36-9367-7f31eda2203b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:30.776182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934f504-6579-4a36-9367-7f31eda2203b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:30.799077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547902f-e0f0-4a1d-9d42-e37e4927839e admin admin] RBAC: Authorizing `identity:create_grant(project_id=da46bd0ec0384f9398a890dcf12deb7a, user_id=6efa496c103449e486364cdef764054d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:30.805049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547902f-e0f0-4a1d-9d42-e37e4927839e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:30.809724 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1742/3455] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:02:30.815033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934f504-6579-4a36-9367-7f31eda2203b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:30.847012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934f504-6579-4a36-9367-7f31eda2203b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:30.847923 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1714/3456] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:02:30 2022] PUT /identity/v3/projects/da46bd0ec0384f9398a890dcf12deb7a/users/6efa496c103449e486364cdef764054d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:02:30.853922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18246e9b-be54-4fa0-9bcd-a667be6c8b0a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:30.854690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18246e9b-be54-4fa0-9bcd-a667be6c8b0a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:30.854690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18246e9b-be54-4fa0-9bcd-a667be6c8b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:30.910246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-18246e9b-be54-4fa0-9bcd-a667be6c8b0a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:30.917744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-18246e9b-be54-4fa0-9bcd-a667be6c8b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:31.069724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1260f9-d6c7-4729-a3f9-13b92e346794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:31.073843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1260f9-d6c7-4729-a3f9-13b92e346794 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:31.074225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1260f9-d6c7-4729-a3f9-13b92e346794 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:31.075056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1260f9-d6c7-4729-a3f9-13b92e346794 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:31.075771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1260f9-d6c7-4729-a3f9-13b92e346794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:31.081465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1260f9-d6c7-4729-a3f9-13b92e346794 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:31.142553 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1743/3457] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:02:30 2022] POST /identity/v3/projects => generated 395 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:02:31.144211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1260f9-d6c7-4729-a3f9-13b92e346794 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:31.157278 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1715/3458] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:31.169085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18246e9b-be54-4fa0-9bcd-a667be6c8b0a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:31.170705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-18246e9b-be54-4fa0-9bcd-a667be6c8b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:31.271275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:31.283410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1260f9-d6c7-4729-a3f9-13b92e346794 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:31.295948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c276a2-75d2-458e-957c-cb207ac0415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:31.296566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c276a2-75d2-458e-957c-cb207ac0415d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:31.296846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c276a2-75d2-458e-957c-cb207ac0415d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:31.297160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c276a2-75d2-458e-957c-cb207ac0415d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:31.297790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c276a2-75d2-458e-957c-cb207ac0415d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:31.299802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c276a2-75d2-458e-957c-cb207ac0415d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:31.307153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18246e9b-be54-4fa0-9bcd-a667be6c8b0a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:31.350942 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1744/3459] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:02:31 2022] POST /identity/v3/users => generated 298 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:02:31.355740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c276a2-75d2-458e-957c-cb207ac0415d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:31.377575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c276a2-75d2-458e-957c-cb207ac0415d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:31.392011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca980f24-0f7e-4e96-84f1-27f49bef4ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:31.393820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca980f24-0f7e-4e96-84f1-27f49bef4ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:31.394100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca980f24-0f7e-4e96-84f1-27f49bef4ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:31.394366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca980f24-0f7e-4e96-84f1-27f49bef4ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:31.395014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca980f24-0f7e-4e96-84f1-27f49bef4ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:31.399342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca980f24-0f7e-4e96-84f1-27f49bef4ba3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:31.422016 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1745/3460] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:02:31.425367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca980f24-0f7e-4e96-84f1-27f49bef4ba3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:31.425967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bb524-60c2-4f7a-a9f3-331045e049a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:31.427296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb524-60c2-4f7a-a9f3-331045e049a8 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:31.428254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb524-60c2-4f7a-a9f3-331045e049a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:31.429207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb524-60c2-4f7a-a9f3-331045e049a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:31.450590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca980f24-0f7e-4e96-84f1-27f49bef4ba3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:31.456411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2bb524-60c2-4f7a-a9f3-331045e049a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:31.459056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2bb524-60c2-4f7a-a9f3-331045e049a8 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:31.465840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1ff51-51f9-4711-ae02-9e29cb01950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:31.466644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1ff51-51f9-4711-ae02-9e29cb01950b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:31.467005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1ff51-51f9-4711-ae02-9e29cb01950b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:31.467434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1ff51-51f9-4711-ae02-9e29cb01950b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:31.468195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1ff51-51f9-4711-ae02-9e29cb01950b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:31.469916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1ff51-51f9-4711-ae02-9e29cb01950b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:31.494687 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1716/3461] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:31.495636 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1746/3462] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:31.499900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bb524-60c2-4f7a-a9f3-331045e049a8 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:31.515002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bb524-60c2-4f7a-a9f3-331045e049a8 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:31.527970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cddaa2-289c-4107-ac9e-e8d145984628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:31.529232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cddaa2-289c-4107-ac9e-e8d145984628 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:31.529765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cddaa2-289c-4107-ac9e-e8d145984628 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:31.530263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cddaa2-289c-4107-ac9e-e8d145984628 admin admin] PATH_INFO: `/v3/projects/f278bb578eca4b5895634257d5ac832e/users/f2295a9d9b034667957b8e1d3c4901ab/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:31.575077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cddaa2-289c-4107-ac9e-e8d145984628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f278bb578eca4b5895634257d5ac832e, user_id=f2295a9d9b034667957b8e1d3c4901ab, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:31.581551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cddaa2-289c-4107-ac9e-e8d145984628 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:31.627954 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1717/3463] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:02:31 2022] PUT /identity/v3/projects/f278bb578eca4b5895634257d5ac832e/users/f2295a9d9b034667957b8e1d3c4901ab/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:31.635391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2efb9f-45da-4792-82a9-f6383538e13b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:31.636665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2efb9f-45da-4792-82a9-f6383538e13b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:31.637192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2efb9f-45da-4792-82a9-f6383538e13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:31.695965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1f2efb9f-45da-4792-82a9-f6383538e13b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:31.701267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1f2efb9f-45da-4792-82a9-f6383538e13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:31.797067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:31.857620 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1747/3464] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:31.916098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cddaa2-289c-4107-ac9e-e8d145984628 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:31.919126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-41cddaa2-289c-4107-ac9e-e8d145984628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:32.027465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cddaa2-289c-4107-ac9e-e8d145984628 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:32.036972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89192dc-c59c-4a18-8ac7-e64ab4618ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:32.038275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89192dc-c59c-4a18-8ac7-e64ab4618ceb service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:32.038815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89192dc-c59c-4a18-8ac7-e64ab4618ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:32.039351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89192dc-c59c-4a18-8ac7-e64ab4618ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:32.070067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89192dc-c59c-4a18-8ac7-e64ab4618ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:32.073278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89192dc-c59c-4a18-8ac7-e64ab4618ceb service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:32.100224 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1718/3465] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:41.832244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=8 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:41.857108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27206-0b53-4527-97e7-8a75231e1f4e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:41.857410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27206-0b53-4527-97e7-8a75231e1f4e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:41.857665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27206-0b53-4527-97e7-8a75231e1f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:41.931792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7fd27206-0b53-4527-97e7-8a75231e1f4e None None] MFA Rules not processed for user `a8af9544fe68497c8557248575f184ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:41.933455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7fd27206-0b53-4527-97e7-8a75231e1f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:42.086610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=64,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:42.110829 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1748/3466] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:42.119461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f47cf-952a-4d4c-a316-413de42e2a05 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:42.119786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f47cf-952a-4d4c-a316-413de42e2a05 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:42.120008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f47cf-952a-4d4c-a316-413de42e2a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:42.155933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-280f47cf-952a-4d4c-a316-413de42e2a05 None None] MFA Rules not processed for user `a8af9544fe68497c8557248575f184ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:42.157783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-280f47cf-952a-4d4c-a316-413de42e2a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:42.177820 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1719/3467] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:42.994805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd27206-0b53-4527-97e7-8a75231e1f4e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:43.081404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd27206-0b53-4527-97e7-8a75231e1f4e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:43.100959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf7d3d-16a6-4c7d-8324-a2fe47a27f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:43.104407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf7d3d-16a6-4c7d-8324-a2fe47a27f78 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:43.106070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf7d3d-16a6-4c7d-8324-a2fe47a27f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:43.106406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf7d3d-16a6-4c7d-8324-a2fe47a27f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:43.110328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-44cf7d3d-16a6-4c7d-8324-a2fe47a27f78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:43.138896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf7d3d-16a6-4c7d-8324-a2fe47a27f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:43.144095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf7d3d-16a6-4c7d-8324-a2fe47a27f78 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:43.204416 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1749/3468] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:43.444177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f47cf-952a-4d4c-a316-413de42e2a05 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:43.448409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-280f47cf-952a-4d4c-a316-413de42e2a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:43.680894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f47cf-952a-4d4c-a316-413de42e2a05 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:43.712337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaace4f-3438-4ba1-9b32-619858acda67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:43.712584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaace4f-3438-4ba1-9b32-619858acda67 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:43.713106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaace4f-3438-4ba1-9b32-619858acda67 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:43.713549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaace4f-3438-4ba1-9b32-619858acda67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:43.731314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaace4f-3438-4ba1-9b32-619858acda67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:43.733972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaace4f-3438-4ba1-9b32-619858acda67 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:43.764435 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1720/3469] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:43 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:45.421145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e96ac-b6ec-4183-98c6-99ef022e9198 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:45.421145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e96ac-b6ec-4183-98c6-99ef022e9198 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:45.421709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e96ac-b6ec-4183-98c6-99ef022e9198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:45.468215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ae2e96ac-b6ec-4183-98c6-99ef022e9198 None None] MFA Rules not processed for user `6efa496c103449e486364cdef764054d`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:45.474472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ae2e96ac-b6ec-4183-98c6-99ef022e9198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:45.650673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbe90-345d-4e4a-bb3f-abc0520eab78 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:45.651236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbe90-345d-4e4a-bb3f-abc0520eab78 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:45.651236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbe90-345d-4e4a-bb3f-abc0520eab78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:45.663421 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1750/3470] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:45.708726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2e2cbe90-345d-4e4a-bb3f-abc0520eab78 None None] MFA Rules not processed for user `6efa496c103449e486364cdef764054d`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:45.712084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2e2cbe90-345d-4e4a-bb3f-abc0520eab78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:45.742513 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1721/3471] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:46.024319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd03fc1-81cd-420a-838a-a6340162f1f0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:46.024712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd03fc1-81cd-420a-838a-a6340162f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:46.025110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd03fc1-81cd-420a-838a-a6340162f1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:46.090100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bdd03fc1-81cd-420a-838a-a6340162f1f0 None None] MFA Rules not processed for user `f2295a9d9b034667957b8e1d3c4901ab`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:46.091984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bdd03fc1-81cd-420a-838a-a6340162f1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:46.189235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:46.258934 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1751/3472] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:46 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:46.269078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e167033-759f-4ac0-9c50-e39ec75000a8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:46.269874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e167033-759f-4ac0-9c50-e39ec75000a8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:46.270360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e167033-759f-4ac0-9c50-e39ec75000a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:46.310089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6e167033-759f-4ac0-9c50-e39ec75000a8 None None] MFA Rules not processed for user `f2295a9d9b034667957b8e1d3c4901ab`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:46.316740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e167033-759f-4ac0-9c50-e39ec75000a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:46.338292 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1722/3473] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:46 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:46.650720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd03fc1-81cd-420a-838a-a6340162f1f0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:46.705619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd03fc1-81cd-420a-838a-a6340162f1f0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:46.725578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12292595-160c-4cf8-b73c-ff0d31051fa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:46.727520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292595-160c-4cf8-b73c-ff0d31051fa2 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:46.732217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292595-160c-4cf8-b73c-ff0d31051fa2 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:46.733786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292595-160c-4cf8-b73c-ff0d31051fa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:46.749973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12292595-160c-4cf8-b73c-ff0d31051fa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:46.752117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12292595-160c-4cf8-b73c-ff0d31051fa2 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:46.765231 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1752/3474] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:49.254699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e167033-759f-4ac0-9c50-e39ec75000a8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:49.287447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e167033-759f-4ac0-9c50-e39ec75000a8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:49.315333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21ad8f-0d52-4774-916f-b461cad58d5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:49.316414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21ad8f-0d52-4774-916f-b461cad58d5f service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:49.318127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21ad8f-0d52-4774-916f-b461cad58d5f service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:49.320677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21ad8f-0d52-4774-916f-b461cad58d5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:49.356976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21ad8f-0d52-4774-916f-b461cad58d5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:49.370592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21ad8f-0d52-4774-916f-b461cad58d5f service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:49.399275 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1723/3475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:49 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:53.683336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12292595-160c-4cf8-b73c-ff0d31051fa2 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:53.689406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-12292595-160c-4cf8-b73c-ff0d31051fa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:53.895054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21ad8f-0d52-4774-916f-b461cad58d5f service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:53.898560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de21ad8f-0d52-4774-916f-b461cad58d5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:53.904974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12292595-160c-4cf8-b73c-ff0d31051fa2 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:53.928194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21ad8f-0d52-4774-916f-b461cad58d5f service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:53.930316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a097f-c950-4838-88fb-374fe67bd4d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:53.932126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a097f-c950-4838-88fb-374fe67bd4d8 service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:53.935950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:53.936940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:53.937969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:53.938090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a097f-c950-4838-88fb-374fe67bd4d8 service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:53.938282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a097f-c950-4838-88fb-374fe67bd4d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:53.939137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] PATH_INFO: `/v3/users/6adaa12c2ca9469299994300bc856ac5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:53.943172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6adaa12c2ca9469299994300bc856ac5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:53.945658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:53.949399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f4a097f-c950-4838-88fb-374fe67bd4d8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:54.022135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a097f-c950-4838-88fb-374fe67bd4d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:54.027746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a097f-c950-4838-88fb-374fe67bd4d8 service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:54.111680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6adaa12c2ca9469299994300bc856ac5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:54.112091 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1753/3476] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:54.124008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6adaa12c2ca9469299994300bc856ac5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:54.174620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6adaa12c2ca9469299994300bc856ac5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:54.190406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6adaa12c2ca9469299994300bc856ac5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:54.191468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6adaa12c2ca9469299994300bc856ac5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:54.220199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a097f-c950-4838-88fb-374fe67bd4d8 service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:54.250116 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1724/3477] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:02:53 2022] DELETE /identity/v3/users/6adaa12c2ca9469299994300bc856ac5 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:54.265577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a097f-c950-4838-88fb-374fe67bd4d8 service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:54.433860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dab71c-2b51-4dae-b789-f3956c69f9e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:54.434611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dab71c-2b51-4dae-b789-f3956c69f9e1 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:54.434878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dab71c-2b51-4dae-b789-f3956c69f9e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:54.437014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dab71c-2b51-4dae-b789-f3956c69f9e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:54.444310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83dab71c-2b51-4dae-b789-f3956c69f9e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:54.501442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dab71c-2b51-4dae-b789-f3956c69f9e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:54.502916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dab71c-2b51-4dae-b789-f3956c69f9e1 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:54.519577 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1754/3478] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:54.695543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:54.698115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:54.738235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:54.803884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5a16-ac98-4160-9adb-3436e8cdb61c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:54.810817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:54.811545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:54.811888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:54.812231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] PATH_INFO: `/v3/projects/44a01230c0fc49a6b60d553d085f3808` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:54.813999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a01230c0fc49a6b60d553d085f3808)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:54.817147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:54.847714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dab71c-2b51-4dae-b789-f3956c69f9e1 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:54.849938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83dab71c-2b51-4dae-b789-f3956c69f9e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:54.873006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dab71c-2b51-4dae-b789-f3956c69f9e1 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:54.883445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16c007-599c-4cfd-9754-02fe64a51875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:54.884051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16c007-599c-4cfd-9754-02fe64a51875 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:54.884518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16c007-599c-4cfd-9754-02fe64a51875 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:54.884991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16c007-599c-4cfd-9754-02fe64a51875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:54.892810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2e16c007-599c-4cfd-9754-02fe64a51875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:54.927348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a01230c0fc49a6b60d553d085f3808', 'request_id': 'req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:54.939630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a01230c0fc49a6b60d553d085f3808', 'request_id': 'req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:54.968465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a01230c0fc49a6b60d553d085f3808', 'request_id': 'req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:54.994650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] The token cache is being invalidate because project 44a01230c0fc49a6b60d553d085f3808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:02:54.995672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:54.998239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-327b1cf5-f9ba-4773-8ebf-b6ca7c23fad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:02:55.002248 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1725/3479] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:02:54 2022] DELETE /identity/v3/projects/44a01230c0fc49a6b60d553d085f3808 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:02:55.007560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38057d78-f54f-42cc-90c0-72f5fdd08847 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:55.008192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38057d78-f54f-42cc-90c0-72f5fdd08847 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:55.008693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38057d78-f54f-42cc-90c0-72f5fdd08847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:55.025164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:55.079905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16c007-599c-4cfd-9754-02fe64a51875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:55.082746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16c007-599c-4cfd-9754-02fe64a51875 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:55.095825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-38057d78-f54f-42cc-90c0-72f5fdd08847 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:55.097194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-38057d78-f54f-42cc-90c0-72f5fdd08847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:55.099349 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1755/3480] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:02:55.175850 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1726/3481] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:55.184115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ebd30-5020-4030-af67-82d3bbef98f9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:55.184687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ebd30-5020-4030-af67-82d3bbef98f9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:55.186166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ebd30-5020-4030-af67-82d3bbef98f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:55.215544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6b9ebd30-5020-4030-af67-82d3bbef98f9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:55.217740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6b9ebd30-5020-4030-af67-82d3bbef98f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:55.227862 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1756/3482] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:55.239207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38057d78-f54f-42cc-90c0-72f5fdd08847 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:55.259909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38057d78-f54f-42cc-90c0-72f5fdd08847 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:55.265778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1ce63-6a49-4b3e-b128-e291c2615aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:55.266529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1ce63-6a49-4b3e-b128-e291c2615aef admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:55.266896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1ce63-6a49-4b3e-b128-e291c2615aef admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:55.267172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1ce63-6a49-4b3e-b128-e291c2615aef admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:55.268167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1ce63-6a49-4b3e-b128-e291c2615aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:02:55.268533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1ce63-6a49-4b3e-b128-e291c2615aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:55.273432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1ce63-6a49-4b3e-b128-e291c2615aef admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:55.299321 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1727/3483] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:02:55.304416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ebd30-5020-4030-af67-82d3bbef98f9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:55.320549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ebd30-5020-4030-af67-82d3bbef98f9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:55.327554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2d5ae-1b3f-431e-99c5-6e7534fcb100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:55.328242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2d5ae-1b3f-431e-99c5-6e7534fcb100 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:55.329133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2d5ae-1b3f-431e-99c5-6e7534fcb100 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:55.329420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2d5ae-1b3f-431e-99c5-6e7534fcb100 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:55.330219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2d5ae-1b3f-431e-99c5-6e7534fcb100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:55.336187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2d5ae-1b3f-431e-99c5-6e7534fcb100 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:55.386442 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1757/3484] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:02:55 2022] POST /identity/v3/projects => generated 369 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:02:55.390691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1ce63-6a49-4b3e-b128-e291c2615aef admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:55.413587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1ce63-6a49-4b3e-b128-e291c2615aef admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:55.446939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:55.499437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72a76-1f41-4fb3-abf8-5130e7182fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:55.500130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72a76-1f41-4fb3-abf8-5130e7182fed admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:55.500459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72a76-1f41-4fb3-abf8-5130e7182fed admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:55.500773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72a76-1f41-4fb3-abf8-5130e7182fed admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:55.502764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b72a76-1f41-4fb3-abf8-5130e7182fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:55.506279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b72a76-1f41-4fb3-abf8-5130e7182fed admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:55.552419 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1728/3485] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:02:55 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:02:55.557608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2d5ae-1b3f-431e-99c5-6e7534fcb100 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:55.574165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2d5ae-1b3f-431e-99c5-6e7534fcb100 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:55.581206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f28a4-73fe-4a46-88ff-1cb391693a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:55.582444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f28a4-73fe-4a46-88ff-1cb391693a03 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:55.583171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f28a4-73fe-4a46-88ff-1cb391693a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:55.583260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f28a4-73fe-4a46-88ff-1cb391693a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:55.584380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f28a4-73fe-4a46-88ff-1cb391693a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:55.586984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f28a4-73fe-4a46-88ff-1cb391693a03 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:55.599148 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1758/3486] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:55.603940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72a76-1f41-4fb3-abf8-5130e7182fed admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:55.616043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72a76-1f41-4fb3-abf8-5130e7182fed admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:55.623436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cf3fb-5892-479c-960f-9b6ea7cd20ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:55.624714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cf3fb-5892-479c-960f-9b6ea7cd20ee admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:55.624998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cf3fb-5892-479c-960f-9b6ea7cd20ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:55.625370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cf3fb-5892-479c-960f-9b6ea7cd20ee admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:55.626260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cf3fb-5892-479c-960f-9b6ea7cd20ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:55.627864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cf3fb-5892-479c-960f-9b6ea7cd20ee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:55.642736 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1729/3487] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:02:55.647750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f28a4-73fe-4a46-88ff-1cb391693a03 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:55.660531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f28a4-73fe-4a46-88ff-1cb391693a03 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:55.668559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af2b7a-fbb6-4b39-9258-ce3b0a79bfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:55.669889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af2b7a-fbb6-4b39-9258-ce3b0a79bfd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:55.670262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af2b7a-fbb6-4b39-9258-ce3b0a79bfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:55.670540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af2b7a-fbb6-4b39-9258-ce3b0a79bfd4 admin admin] PATH_INFO: `/v3/projects/642b98a4621b4d84b3e1e97137a68d33/users/c7ceb51003c24728b665e7f641e3634b/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:55.691555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af2b7a-fbb6-4b39-9258-ce3b0a79bfd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642b98a4621b4d84b3e1e97137a68d33, user_id=c7ceb51003c24728b665e7f641e3634b, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:55.695654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af2b7a-fbb6-4b39-9258-ce3b0a79bfd4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:55.717482 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1759/3488] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:02:55 2022] PUT /identity/v3/projects/642b98a4621b4d84b3e1e97137a68d33/users/c7ceb51003c24728b665e7f641e3634b/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:02:55.729492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd316e-8623-4b88-ba9f-bffbe0b8e298 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:55.729914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd316e-8623-4b88-ba9f-bffbe0b8e298 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:55.730046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd316e-8623-4b88-ba9f-bffbe0b8e298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:55.778760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a0dd316e-8623-4b88-ba9f-bffbe0b8e298 None None] MFA Rules not processed for user `c7ceb51003c24728b665e7f641e3634b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:55.780301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a0dd316e-8623-4b88-ba9f-bffbe0b8e298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:55.874877 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1730/3489] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:55.876855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d2d87-24fd-4711-9dd1-0a06e0131807 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:55.877396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d2d87-24fd-4711-9dd1-0a06e0131807 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:55.877947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d2d87-24fd-4711-9dd1-0a06e0131807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:55.907615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e33d2d87-24fd-4711-9dd1-0a06e0131807 None None] MFA Rules not processed for user `c7ceb51003c24728b665e7f641e3634b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:02:55.908855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e33d2d87-24fd-4711-9dd1-0a06e0131807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:55.921933 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1760/3490] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:02:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:02:57.607025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd316e-8623-4b88-ba9f-bffbe0b8e298 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:02:57.609136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a0dd316e-8623-4b88-ba9f-bffbe0b8e298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:02:57.719029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd316e-8623-4b88-ba9f-bffbe0b8e298 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:02:57.725472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2e8b6-6306-4983-8f0f-b0b459860319 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:02:57.726987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2e8b6-6306-4983-8f0f-b0b459860319 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:02:57.727297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2e8b6-6306-4983-8f0f-b0b459860319 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:02:57.727490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2e8b6-6306-4983-8f0f-b0b459860319 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:02:57.739839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2e8b6-6306-4983-8f0f-b0b459860319 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:02:57.743345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2e8b6-6306-4983-8f0f-b0b459860319 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:02:57.769067 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1731/3491] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:02:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:03.952097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58462a-a317-4d8b-a63f-a38fb7fd0772 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:03.952667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58462a-a317-4d8b-a63f-a38fb7fd0772 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:03.952667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58462a-a317-4d8b-a63f-a38fb7fd0772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:03.978723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6e58462a-a317-4d8b-a63f-a38fb7fd0772 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:03.980060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e58462a-a317-4d8b-a63f-a38fb7fd0772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:04.053194 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1761/3492] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:03:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:04.067520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2e8b6-6306-4983-8f0f-b0b459860319 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:04.070064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6cc2e8b6-6306-4983-8f0f-b0b459860319 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:04.094725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:04.157282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2e8b6-6306-4983-8f0f-b0b459860319 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:04.165549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31adfc5-d502-4422-9f0d-413a7fbeb875 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:04.166222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31adfc5-d502-4422-9f0d-413a7fbeb875 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:04.166635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31adfc5-d502-4422-9f0d-413a7fbeb875 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:04.166907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31adfc5-d502-4422-9f0d-413a7fbeb875 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:04.176875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31adfc5-d502-4422-9f0d-413a7fbeb875 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:04.179356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31adfc5-d502-4422-9f0d-413a7fbeb875 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:04.189428 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1732/3493] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:03:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:03:08.146122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58462a-a317-4d8b-a63f-a38fb7fd0772 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:08.148671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e58462a-a317-4d8b-a63f-a38fb7fd0772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:08.277177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58462a-a317-4d8b-a63f-a38fb7fd0772 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:08.299384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa0513-509f-4c1c-a3b5-53a8e38e6bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:08.300396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0513-509f-4c1c-a3b5-53a8e38e6bdf service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:08.300912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0513-509f-4c1c-a3b5-53a8e38e6bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:08.301579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0513-509f-4c1c-a3b5-53a8e38e6bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:08.305139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83fa0513-509f-4c1c-a3b5-53a8e38e6bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:08.330677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa0513-509f-4c1c-a3b5-53a8e38e6bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:08.334709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa0513-509f-4c1c-a3b5-53a8e38e6bdf service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:08.363188 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1762/3494] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:14.185216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=23 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:14.924451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31adfc5-d502-4422-9f0d-413a7fbeb875 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:14.937581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31adfc5-d502-4422-9f0d-413a7fbeb875 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:14.942259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad6011-2e89-48be-8f1f-3b1663228d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:14.943011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6011-2e89-48be-8f1f-3b1663228d31 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:14.943225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6011-2e89-48be-8f1f-3b1663228d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:14.943482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6011-2e89-48be-8f1f-3b1663228d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:14.946148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-45ad6011-2e89-48be-8f1f-3b1663228d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:14.958070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad6011-2e89-48be-8f1f-3b1663228d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:14.959511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad6011-2e89-48be-8f1f-3b1663228d31 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:14.971520 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1733/3495] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:16.196180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c10dea-df6c-4632-b4e3-2dc2a4f35f28 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:16.196753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c10dea-df6c-4632-b4e3-2dc2a4f35f28 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:16.197063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c10dea-df6c-4632-b4e3-2dc2a4f35f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:16.221876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-01c10dea-df6c-4632-b4e3-2dc2a4f35f28 None None] MFA Rules not processed for user `f2295a9d9b034667957b8e1d3c4901ab`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:16.223353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-01c10dea-df6c-4632-b4e3-2dc2a4f35f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:16.229235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:16.301076 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1763/3496] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:16 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:17.489271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad6011-2e89-48be-8f1f-3b1663228d31 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:17.492151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-45ad6011-2e89-48be-8f1f-3b1663228d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:17.573508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad6011-2e89-48be-8f1f-3b1663228d31 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:17.578827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be472274-7192-4b8f-a6d4-d9f3cf1da377 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:17.579390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472274-7192-4b8f-a6d4-d9f3cf1da377 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:17.579693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472274-7192-4b8f-a6d4-d9f3cf1da377 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:17.579922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472274-7192-4b8f-a6d4-d9f3cf1da377 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:17.581732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-be472274-7192-4b8f-a6d4-d9f3cf1da377 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:17.595086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be472274-7192-4b8f-a6d4-d9f3cf1da377 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:17.596824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be472274-7192-4b8f-a6d4-d9f3cf1da377 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:17.608909 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1734/3497] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:18.277105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c10dea-df6c-4632-b4e3-2dc2a4f35f28 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:18.278768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-01c10dea-df6c-4632-b4e3-2dc2a4f35f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:18.384386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c10dea-df6c-4632-b4e3-2dc2a4f35f28 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:18.396073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:18.397006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:18.398128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:18.398594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] PATH_INFO: `/v3/users/c50ab103943e4e3bacdc2777892c3bfe` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:18.400669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c50ab103943e4e3bacdc2777892c3bfe)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:18.404997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:18.498909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c50ab103943e4e3bacdc2777892c3bfe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:18.516979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c50ab103943e4e3bacdc2777892c3bfe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:18.536798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c50ab103943e4e3bacdc2777892c3bfe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:18.549165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c50ab103943e4e3bacdc2777892c3bfe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:18.550379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c50ab103943e4e3bacdc2777892c3bfe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:18.562887 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1764/3498] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:03:18 2022] DELETE /identity/v3/users/c50ab103943e4e3bacdc2777892c3bfe => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:03:18.564565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb78ef8-7fc5-4365-b62d-a454075adb54 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:18.565109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb78ef8-7fc5-4365-b62d-a454075adb54 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:18.565109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb78ef8-7fc5-4365-b62d-a454075adb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:18.590235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7bb78ef8-7fc5-4365-b62d-a454075adb54 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:18.592023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7bb78ef8-7fc5-4365-b62d-a454075adb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:18.676256 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1735/3499] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:18.686915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:18.688433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:18.735661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:18.766155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa126cb-397f-4c13-9f79-bdf3816dd87b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:18.770890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee5fde-8bb3-43a3-8705-24d25937f93e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:18.771626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee5fde-8bb3-43a3-8705-24d25937f93e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:18.771902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee5fde-8bb3-43a3-8705-24d25937f93e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:18.772745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee5fde-8bb3-43a3-8705-24d25937f93e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:18.792375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee5fde-8bb3-43a3-8705-24d25937f93e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:18.794053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee5fde-8bb3-43a3-8705-24d25937f93e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:18.807882 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1765/3500] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:19.257719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb78ef8-7fc5-4365-b62d-a454075adb54 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:19.259501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7bb78ef8-7fc5-4365-b62d-a454075adb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:19.280646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb78ef8-7fc5-4365-b62d-a454075adb54 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:19.311752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:19.314571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:19.314868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:19.317943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] PATH_INFO: `/v3/projects/26a4431be325443c85b0ecef861d42ff` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:19.323434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a4431be325443c85b0ecef861d42ff)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:19.329953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:19.408966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a4431be325443c85b0ecef861d42ff', 'request_id': 'req-461282d7-c264-4760-973e-a33c59d0fa85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:19.422086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a4431be325443c85b0ecef861d42ff', 'request_id': 'req-461282d7-c264-4760-973e-a33c59d0fa85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:19.431868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a4431be325443c85b0ecef861d42ff', 'request_id': 'req-461282d7-c264-4760-973e-a33c59d0fa85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:19.446848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:19.454140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] The token cache is being invalidate because project 26a4431be325443c85b0ecef861d42ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:03:19.455304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:19.456789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-461282d7-c264-4760-973e-a33c59d0fa85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:19.466404 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1736/3501] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:03:19 2022] DELETE /identity/v3/projects/26a4431be325443c85b0ecef861d42ff => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:03:19.472272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a968dc-0515-4a49-a00d-68f75ee950c7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:19.474802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a968dc-0515-4a49-a00d-68f75ee950c7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:19.475346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a968dc-0515-4a49-a00d-68f75ee950c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:19.508156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-02a968dc-0515-4a49-a00d-68f75ee950c7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:19.509928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-02a968dc-0515-4a49-a00d-68f75ee950c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:19.599750 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1766/3502] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:19.609554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d8541-267d-4446-8ad4-7cb8b5218acd None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:19.610559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d8541-267d-4446-8ad4-7cb8b5218acd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:19.611053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d8541-267d-4446-8ad4-7cb8b5218acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:19.639826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-185d8541-267d-4446-8ad4-7cb8b5218acd None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:19.641264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-185d8541-267d-4446-8ad4-7cb8b5218acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:19.653675 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1737/3503] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:19.658211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a968dc-0515-4a49-a00d-68f75ee950c7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:19.676440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a968dc-0515-4a49-a00d-68f75ee950c7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:19.682865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb0384-44a5-45e3-a71b-edcb77758c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:19.683479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb0384-44a5-45e3-a71b-edcb77758c8a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:19.684020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb0384-44a5-45e3-a71b-edcb77758c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:19.684338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb0384-44a5-45e3-a71b-edcb77758c8a admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:19.685077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb0384-44a5-45e3-a71b-edcb77758c8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:03:19.685734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb0384-44a5-45e3-a71b-edcb77758c8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:19.687413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb0384-44a5-45e3-a71b-edcb77758c8a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:19.706011 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1767/3504] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:03:19.711185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d8541-267d-4446-8ad4-7cb8b5218acd None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:19.724858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d8541-267d-4446-8ad4-7cb8b5218acd None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:19.734234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05dc45-1426-4868-b875-65b892922cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:19.734995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05dc45-1426-4868-b875-65b892922cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:19.735273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05dc45-1426-4868-b875-65b892922cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:19.735564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05dc45-1426-4868-b875-65b892922cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:19.736944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05dc45-1426-4868-b875-65b892922cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:19.739047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05dc45-1426-4868-b875-65b892922cf0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:19.779013 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1738/3505] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:03:19 2022] POST /identity/v3/projects => generated 403 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:03:19.781204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb0384-44a5-45e3-a71b-edcb77758c8a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:19.799461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb0384-44a5-45e3-a71b-edcb77758c8a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:19.919366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e481c-76d9-486b-a8c5-99a446d5c977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:19.920093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e481c-76d9-486b-a8c5-99a446d5c977 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:19.920378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e481c-76d9-486b-a8c5-99a446d5c977 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:19.920715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e481c-76d9-486b-a8c5-99a446d5c977 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:19.937910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e481c-76d9-486b-a8c5-99a446d5c977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:19.939953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e481c-76d9-486b-a8c5-99a446d5c977 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:19.982343 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1768/3506] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:03:19 2022] POST /identity/v3/users => generated 302 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:03:19.986650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05dc45-1426-4868-b875-65b892922cf0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:20.004550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05dc45-1426-4868-b875-65b892922cf0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:20.010802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54027b-2d05-43e6-afbc-ac09e7e28e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:20.011825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54027b-2d05-43e6-afbc-ac09e7e28e9c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:20.012177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54027b-2d05-43e6-afbc-ac09e7e28e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:20.012585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54027b-2d05-43e6-afbc-ac09e7e28e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:20.013575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54027b-2d05-43e6-afbc-ac09e7e28e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:20.015572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54027b-2d05-43e6-afbc-ac09e7e28e9c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:20.033437 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1739/3507] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:03:20.035978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e481c-76d9-486b-a8c5-99a446d5c977 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:20.054108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e481c-76d9-486b-a8c5-99a446d5c977 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:20.067129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb46fde-f2f4-4f77-a483-eab38bbbbab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:20.068614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb46fde-f2f4-4f77-a483-eab38bbbbab6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:20.068614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb46fde-f2f4-4f77-a483-eab38bbbbab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:20.069416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb46fde-f2f4-4f77-a483-eab38bbbbab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:20.070721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb46fde-f2f4-4f77-a483-eab38bbbbab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:20.073323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb46fde-f2f4-4f77-a483-eab38bbbbab6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:20.097440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54027b-2d05-43e6-afbc-ac09e7e28e9c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:20.097748 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1769/3508] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:03:20.113136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54027b-2d05-43e6-afbc-ac09e7e28e9c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:20.122484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c7c8a-39d1-4441-9e2a-098d81558431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:20.124451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c7c8a-39d1-4441-9e2a-098d81558431 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:20.124990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c7c8a-39d1-4441-9e2a-098d81558431 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:20.126265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c7c8a-39d1-4441-9e2a-098d81558431 admin admin] PATH_INFO: `/v3/projects/c6d5585dc2254ed2aed773305d209049/users/14c805d8b3e944e8ad9f6150322b96ce/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:20.174373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c7c8a-39d1-4441-9e2a-098d81558431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d5585dc2254ed2aed773305d209049, user_id=14c805d8b3e944e8ad9f6150322b96ce, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:20.177823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c7c8a-39d1-4441-9e2a-098d81558431 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:20.197574 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1740/3509] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:03:20 2022] PUT /identity/v3/projects/c6d5585dc2254ed2aed773305d209049/users/14c805d8b3e944e8ad9f6150322b96ce/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:03:20.207326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fb59d-cc96-4a94-ab57-5ddc75a9c357 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:20.208473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fb59d-cc96-4a94-ab57-5ddc75a9c357 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:20.209006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fb59d-cc96-4a94-ab57-5ddc75a9c357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:20.267155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5a4fb59d-cc96-4a94-ab57-5ddc75a9c357 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:20.268691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5a4fb59d-cc96-4a94-ab57-5ddc75a9c357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:20.336102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:20.388014 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1770/3510] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:20.399656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c7c8a-39d1-4441-9e2a-098d81558431 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:20.409745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e25c7c8a-39d1-4441-9e2a-098d81558431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:20.563333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c7c8a-39d1-4441-9e2a-098d81558431 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:20.573773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec1e66-1900-4115-ad3f-0548f3f382be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:20.574703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1e66-1900-4115-ad3f-0548f3f382be service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:20.575565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1e66-1900-4115-ad3f-0548f3f382be service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:20.575777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1e66-1900-4115-ad3f-0548f3f382be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:20.588644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec1e66-1900-4115-ad3f-0548f3f382be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:20.591684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec1e66-1900-4115-ad3f-0548f3f382be service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:20.608703 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1741/3511] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:26.449062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae147a2-eb4a-4ec2-8ee3-cdd4a0fd14fe None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:26.449062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae147a2-eb4a-4ec2-8ee3-cdd4a0fd14fe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:26.449062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae147a2-eb4a-4ec2-8ee3-cdd4a0fd14fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:26.502468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1ae147a2-eb4a-4ec2-8ee3-cdd4a0fd14fe None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:26.503717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1ae147a2-eb4a-4ec2-8ee3-cdd4a0fd14fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:26.577602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fbb66-1c3f-4311-b889-4bb31cbcc1d7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:26.578227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fbb66-1c3f-4311-b889-4bb31cbcc1d7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:26.578227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fbb66-1c3f-4311-b889-4bb31cbcc1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:26.606196 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1771/3512] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:03:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:26.629067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-638fbb66-1c3f-4311-b889-4bb31cbcc1d7 None None] MFA Rules not processed for user `14c805d8b3e944e8ad9f6150322b96ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:26.630445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-638fbb66-1c3f-4311-b889-4bb31cbcc1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:26.643190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae147a2-eb4a-4ec2-8ee3-cdd4a0fd14fe None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:26.646366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1ae147a2-eb4a-4ec2-8ee3-cdd4a0fd14fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:26.762346 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1742/3513] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:26.768122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fbb66-1c3f-4311-b889-4bb31cbcc1d7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:26.777373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae147a2-eb4a-4ec2-8ee3-cdd4a0fd14fe None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:26.785873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbafcc1-94a5-4caf-a390-6683b37cfd34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:26.786886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbafcc1-94a5-4caf-a390-6683b37cfd34 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:26.787226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbafcc1-94a5-4caf-a390-6683b37cfd34 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:26.787846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbafcc1-94a5-4caf-a390-6683b37cfd34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:26.798373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fbb66-1c3f-4311-b889-4bb31cbcc1d7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:26.801300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbafcc1-94a5-4caf-a390-6683b37cfd34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:26.805177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85412097-321d-41a2-99ef-04d091e7587c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:26.805804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85412097-321d-41a2-99ef-04d091e7587c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:26.806077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85412097-321d-41a2-99ef-04d091e7587c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:26.806338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85412097-321d-41a2-99ef-04d091e7587c admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:26.806461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbafcc1-94a5-4caf-a390-6683b37cfd34 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:26.807305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85412097-321d-41a2-99ef-04d091e7587c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:26.809838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85412097-321d-41a2-99ef-04d091e7587c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:26.821558 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1772/3514] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:03:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:03:26.894796 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1743/3515] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:03:26 2022] POST /identity/v3/projects => generated 403 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:03:26.898678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbafcc1-94a5-4caf-a390-6683b37cfd34 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:26.935339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbafcc1-94a5-4caf-a390-6683b37cfd34 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:27.129214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7e88-9ad5-409c-b872-e8aa48d88bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:27.130737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7e88-9ad5-409c-b872-e8aa48d88bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:27.133079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7e88-9ad5-409c-b872-e8aa48d88bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:27.133381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7e88-9ad5-409c-b872-e8aa48d88bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:27.135990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a7e88-9ad5-409c-b872-e8aa48d88bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:27.145877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a7e88-9ad5-409c-b872-e8aa48d88bf7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:27.220637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85412097-321d-41a2-99ef-04d091e7587c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:27.221402 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1773/3516] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:03:26 2022] POST /identity/v3/users => generated 302 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:03:27.245126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85412097-321d-41a2-99ef-04d091e7587c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:27.267536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d78766-b2c6-4f62-b4f1-549a2bd60e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:27.268717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78766-b2c6-4f62-b4f1-549a2bd60e29 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:27.269241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78766-b2c6-4f62-b4f1-549a2bd60e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:27.270120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78766-b2c6-4f62-b4f1-549a2bd60e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:27.271086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d78766-b2c6-4f62-b4f1-549a2bd60e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:27.278181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d78766-b2c6-4f62-b4f1-549a2bd60e29 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:27.280852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:27.320299 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1744/3517] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:03:27.323642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7e88-9ad5-409c-b872-e8aa48d88bf7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:27.338760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7e88-9ad5-409c-b872-e8aa48d88bf7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:27.351712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc468a-1b3b-4a89-b354-6f5407944bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:27.353472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc468a-1b3b-4a89-b354-6f5407944bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:27.355294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc468a-1b3b-4a89-b354-6f5407944bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:27.356039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc468a-1b3b-4a89-b354-6f5407944bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:27.357164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc468a-1b3b-4a89-b354-6f5407944bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:27.362747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc468a-1b3b-4a89-b354-6f5407944bc1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:27.368664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:27.393935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d78766-b2c6-4f62-b4f1-549a2bd60e29 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:27.394598 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1774/3518] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:03:27.410082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d78766-b2c6-4f62-b4f1-549a2bd60e29 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:27.418295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9724-35c5-4aeb-bef1-f12cd8509ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:27.419331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9724-35c5-4aeb-bef1-f12cd8509ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:27.419989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9724-35c5-4aeb-bef1-f12cd8509ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:27.420462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9724-35c5-4aeb-bef1-f12cd8509ceb admin admin] PATH_INFO: `/v3/projects/42cfcf1dcb864013ac16376c2419d05f/users/7d02831cb7e24fc69671dd999f4746fb/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:27.457194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9724-35c5-4aeb-bef1-f12cd8509ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=42cfcf1dcb864013ac16376c2419d05f, user_id=7d02831cb7e24fc69671dd999f4746fb, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:27.467696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9724-35c5-4aeb-bef1-f12cd8509ceb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:27.515759 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1745/3519] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:03:27 2022] PUT /identity/v3/projects/42cfcf1dcb864013ac16376c2419d05f/users/7d02831cb7e24fc69671dd999f4746fb/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:03:34.402163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc468a-1b3b-4a89-b354-6f5407944bc1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:34.404377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-26dc468a-1b3b-4a89-b354-6f5407944bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:34.432878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275aaedf-aa99-4710-9a26-59e9d95c80ca None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:34.434082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275aaedf-aa99-4710-9a26-59e9d95c80ca None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:34.434676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275aaedf-aa99-4710-9a26-59e9d95c80ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:34.482264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-275aaedf-aa99-4710-9a26-59e9d95c80ca None None] MFA Rules not processed for user `a8af9544fe68497c8557248575f184ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:34.483967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-275aaedf-aa99-4710-9a26-59e9d95c80ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:34.639197 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1746/3520] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:34.667859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc468a-1b3b-4a89-b354-6f5407944bc1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:34.800458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd27d1-ea5a-4b1b-9379-a7ad51fb1ae0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:34.801700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd27d1-ea5a-4b1b-9379-a7ad51fb1ae0 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:34.802443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd27d1-ea5a-4b1b-9379-a7ad51fb1ae0 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:34.803978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd27d1-ea5a-4b1b-9379-a7ad51fb1ae0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:34.808973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-15dd27d1-ea5a-4b1b-9379-a7ad51fb1ae0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:34.836777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd27d1-ea5a-4b1b-9379-a7ad51fb1ae0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:34.838774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd27d1-ea5a-4b1b-9379-a7ad51fb1ae0 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:34.866975 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1775/3521] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:35.045911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275aaedf-aa99-4710-9a26-59e9d95c80ca None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:35.047931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-275aaedf-aa99-4710-9a26-59e9d95c80ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:35.173992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275aaedf-aa99-4710-9a26-59e9d95c80ca None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:35.179665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb838eba-816c-4a5f-9958-52707ae7e612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:35.180790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb838eba-816c-4a5f-9958-52707ae7e612 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:35.181180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb838eba-816c-4a5f-9958-52707ae7e612 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:35.181512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb838eba-816c-4a5f-9958-52707ae7e612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:35.183609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cb838eba-816c-4a5f-9958-52707ae7e612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:35.217903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb838eba-816c-4a5f-9958-52707ae7e612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:35.220397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb838eba-816c-4a5f-9958-52707ae7e612 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:35.237010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd27d1-ea5a-4b1b-9379-a7ad51fb1ae0 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:35.239186 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1747/3522] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:35.241553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-15dd27d1-ea5a-4b1b-9379-a7ad51fb1ae0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:35.459674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd27d1-ea5a-4b1b-9379-a7ad51fb1ae0 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:35.464427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb838eba-816c-4a5f-9958-52707ae7e612 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:35.466388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cb838eba-816c-4a5f-9958-52707ae7e612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:35.483536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c64da-8572-4533-b532-9ee29a592297 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:35.484310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c64da-8572-4533-b532-9ee29a592297 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:35.497644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c64da-8572-4533-b532-9ee29a592297 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:35.498256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c64da-8572-4533-b532-9ee29a592297 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:35.501957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-913c64da-8572-4533-b532-9ee29a592297 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:35.555912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c64da-8572-4533-b532-9ee29a592297 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:35.558305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c64da-8572-4533-b532-9ee29a592297 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:35.598573 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1776/3523] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:03:35.773350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb838eba-816c-4a5f-9958-52707ae7e612 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:35.781971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:35.783287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:35.783625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:35.783867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] PATH_INFO: `/v3/users/6efa496c103449e486364cdef764054d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:35.785772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6efa496c103449e486364cdef764054d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:35.789217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:35.832856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:35.909397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6efa496c103449e486364cdef764054d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:35.922715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6efa496c103449e486364cdef764054d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:35.939874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6efa496c103449e486364cdef764054d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:35.954365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6efa496c103449e486364cdef764054d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:35.955703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf27889-a6e4-4ab2-9ed9-92287d835c7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6efa496c103449e486364cdef764054d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:35.968735 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1748/3524] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:03:35 2022] DELETE /identity/v3/users/6efa496c103449e486364cdef764054d => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:03:36.288968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c64da-8572-4533-b532-9ee29a592297 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:36.304378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c64da-8572-4533-b532-9ee29a592297 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:36.316006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:36.418828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:36.420423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:36.420919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:36.421467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] PATH_INFO: `/v3/projects/da46bd0ec0384f9398a890dcf12deb7a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:36.423645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] RBAC: Authorizing `identity:delete_project(project_id=da46bd0ec0384f9398a890dcf12deb7a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:36.426925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:36.534089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da46bd0ec0384f9398a890dcf12deb7a', 'request_id': 'req-979285e5-f6c5-49c7-808b-b1e95e4a649c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:36.564640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da46bd0ec0384f9398a890dcf12deb7a', 'request_id': 'req-979285e5-f6c5-49c7-808b-b1e95e4a649c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:36.595225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da46bd0ec0384f9398a890dcf12deb7a', 'request_id': 'req-979285e5-f6c5-49c7-808b-b1e95e4a649c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:36.626509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] The token cache is being invalidate because project da46bd0ec0384f9398a890dcf12deb7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:03:36.626884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:36.628468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-979285e5-f6c5-49c7-808b-b1e95e4a649c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:36.637109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4e187-b636-4a0d-abb4-05b91da3217b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:36.637963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4e187-b636-4a0d-abb4-05b91da3217b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:36.638421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4e187-b636-4a0d-abb4-05b91da3217b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:36.641271 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1777/3525] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:03:36 2022] DELETE /identity/v3/projects/da46bd0ec0384f9398a890dcf12deb7a => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:03:36.686376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-64a4e187-b636-4a0d-abb4-05b91da3217b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:36.689403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-64a4e187-b636-4a0d-abb4-05b91da3217b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:36.954788 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1749/3526] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:36.955473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09f14f-0fff-4825-8555-cfc8edfc62f7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:36.955986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09f14f-0fff-4825-8555-cfc8edfc62f7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:36.957514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09f14f-0fff-4825-8555-cfc8edfc62f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:36.998514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9b09f14f-0fff-4825-8555-cfc8edfc62f7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:37.001399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9b09f14f-0fff-4825-8555-cfc8edfc62f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:37.014835 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1778/3527] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:37.019208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4e187-b636-4a0d-abb4-05b91da3217b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:37.047658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4e187-b636-4a0d-abb4-05b91da3217b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:37.061559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605c0d5-f52e-45c4-818a-87da47ecfd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:37.062331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605c0d5-f52e-45c4-818a-87da47ecfd33 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:37.062600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605c0d5-f52e-45c4-818a-87da47ecfd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:37.062866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605c0d5-f52e-45c4-818a-87da47ecfd33 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:37.063856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605c0d5-f52e-45c4-818a-87da47ecfd33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:03:37.064197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605c0d5-f52e-45c4-818a-87da47ecfd33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:37.074341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605c0d5-f52e-45c4-818a-87da47ecfd33 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:37.121511 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1750/3528] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:03:37.122854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09f14f-0fff-4825-8555-cfc8edfc62f7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:37.164616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09f14f-0fff-4825-8555-cfc8edfc62f7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:37.187578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031bb7d-d45c-4c08-bd03-3a8d44b812ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:37.188280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031bb7d-d45c-4c08-bd03-3a8d44b812ce admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:37.190080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031bb7d-d45c-4c08-bd03-3a8d44b812ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:37.190903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031bb7d-d45c-4c08-bd03-3a8d44b812ce admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:37.192144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031bb7d-d45c-4c08-bd03-3a8d44b812ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:37.203667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031bb7d-d45c-4c08-bd03-3a8d44b812ce admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:37.271584 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1779/3529] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:03:37 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:03:37.274882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605c0d5-f52e-45c4-818a-87da47ecfd33 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:37.295997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605c0d5-f52e-45c4-818a-87da47ecfd33 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:37.399473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3579a7-0091-4c01-9cc8-9af57f2715a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:37.401077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3579a7-0091-4c01-9cc8-9af57f2715a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:37.401077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3579a7-0091-4c01-9cc8-9af57f2715a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:37.401077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3579a7-0091-4c01-9cc8-9af57f2715a9 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:37.402262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3579a7-0091-4c01-9cc8-9af57f2715a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:37.404203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3579a7-0091-4c01-9cc8-9af57f2715a9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:37.451128 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1751/3530] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:03:37 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:03:37.456981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031bb7d-d45c-4c08-bd03-3a8d44b812ce admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:37.489999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031bb7d-d45c-4c08-bd03-3a8d44b812ce admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:37.507594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b170e-17f2-4bcd-808c-61df1bb4e763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:37.508561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b170e-17f2-4bcd-808c-61df1bb4e763 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:37.509093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b170e-17f2-4bcd-808c-61df1bb4e763 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:37.509626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b170e-17f2-4bcd-808c-61df1bb4e763 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:37.510634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b170e-17f2-4bcd-808c-61df1bb4e763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:37.512487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b170e-17f2-4bcd-808c-61df1bb4e763 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:37.538977 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1780/3531] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:03:37.541140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3579a7-0091-4c01-9cc8-9af57f2715a9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:37.557136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3579a7-0091-4c01-9cc8-9af57f2715a9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:37.575012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fa1d5-dc45-4759-8ae5-4960ed88a647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:37.578205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fa1d5-dc45-4759-8ae5-4960ed88a647 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:37.579117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fa1d5-dc45-4759-8ae5-4960ed88a647 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:37.580888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fa1d5-dc45-4759-8ae5-4960ed88a647 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:37.584255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fa1d5-dc45-4759-8ae5-4960ed88a647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:37.587207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fa1d5-dc45-4759-8ae5-4960ed88a647 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:37.608733 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1752/3532] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:03:37.613541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b170e-17f2-4bcd-808c-61df1bb4e763 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:37.638000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b170e-17f2-4bcd-808c-61df1bb4e763 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:37.648098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be56db-e02c-4a38-ae28-5846e3fe4faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:37.649070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be56db-e02c-4a38-ae28-5846e3fe4faa admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:37.649620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be56db-e02c-4a38-ae28-5846e3fe4faa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:37.650188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be56db-e02c-4a38-ae28-5846e3fe4faa admin admin] PATH_INFO: `/v3/projects/f20ae5c8217d412289d006ab9e01e7f4/users/124511010d044f8fb07dc09e7d754303/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:37.688981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be56db-e02c-4a38-ae28-5846e3fe4faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20ae5c8217d412289d006ab9e01e7f4, user_id=124511010d044f8fb07dc09e7d754303, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:37.694558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be56db-e02c-4a38-ae28-5846e3fe4faa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:37.745756 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1781/3533] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:03:37 2022] PUT /identity/v3/projects/f20ae5c8217d412289d006ab9e01e7f4/users/124511010d044f8fb07dc09e7d754303/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:03:37.751252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e068e-8ecd-4cf5-8405-bff37c353df1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:37.754367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e068e-8ecd-4cf5-8405-bff37c353df1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:37.754980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e068e-8ecd-4cf5-8405-bff37c353df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:37.769814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:37.799035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-506e068e-8ecd-4cf5-8405-bff37c353df1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:37.800596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-506e068e-8ecd-4cf5-8405-bff37c353df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:37.916880 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1753/3534] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:37.941327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681e77e-70fd-4044-9bf8-c9ebfff05767 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:37.942701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681e77e-70fd-4044-9bf8-c9ebfff05767 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:37.942701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681e77e-70fd-4044-9bf8-c9ebfff05767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:37.949692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e068e-8ecd-4cf5-8405-bff37c353df1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:37.952070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-506e068e-8ecd-4cf5-8405-bff37c353df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:38.019744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8681e77e-70fd-4044-9bf8-c9ebfff05767 None None] MFA Rules not processed for user `7d02831cb7e24fc69671dd999f4746fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:38.023928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8681e77e-70fd-4044-9bf8-c9ebfff05767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:38.060311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e068e-8ecd-4cf5-8405-bff37c353df1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:38.074601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a4be7-71be-43e6-8721-63b8f274051a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:38.076149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4be7-71be-43e6-8721-63b8f274051a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:38.076810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4be7-71be-43e6-8721-63b8f274051a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:38.077222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4be7-71be-43e6-8721-63b8f274051a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:38.094418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a4be7-71be-43e6-8721-63b8f274051a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:38.096720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a4be7-71be-43e6-8721-63b8f274051a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:38.129085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1754/3535] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:38.135000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:38.264788 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1782/3536] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:38.276505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5ad26-6c80-4e2a-90d2-6feca505f6b7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:38.276839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5ad26-6c80-4e2a-90d2-6feca505f6b7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:38.277195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5ad26-6c80-4e2a-90d2-6feca505f6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:38.319735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9ab5ad26-6c80-4e2a-90d2-6feca505f6b7 None None] MFA Rules not processed for user `14c805d8b3e944e8ad9f6150322b96ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:38.321816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9ab5ad26-6c80-4e2a-90d2-6feca505f6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:38.465905 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1755/3537] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:38.810984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681e77e-70fd-4044-9bf8-c9ebfff05767 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:38.813625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8681e77e-70fd-4044-9bf8-c9ebfff05767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:38.924217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681e77e-70fd-4044-9bf8-c9ebfff05767 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:38.932564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0588e-1d6e-41bb-a656-7bbcadf475f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:38.933223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0588e-1d6e-41bb-a656-7bbcadf475f3 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:38.933686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0588e-1d6e-41bb-a656-7bbcadf475f3 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:38.934235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0588e-1d6e-41bb-a656-7bbcadf475f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:38.936468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dfc0588e-1d6e-41bb-a656-7bbcadf475f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:38.951554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0588e-1d6e-41bb-a656-7bbcadf475f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:38.953284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0588e-1d6e-41bb-a656-7bbcadf475f3 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:38.964786 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1783/3538] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:38 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:39.179372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26094f86-6427-425d-bfff-fa21a0149421 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:39.180268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26094f86-6427-425d-bfff-fa21a0149421 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:39.180536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26094f86-6427-425d-bfff-fa21a0149421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:39.220698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-26094f86-6427-425d-bfff-fa21a0149421 None None] MFA Rules not processed for user `124511010d044f8fb07dc09e7d754303`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:39.222359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-26094f86-6427-425d-bfff-fa21a0149421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:39.280924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:39.321552 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1756/3539] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:39.331688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cf11f-5566-4d6e-affc-b6fbecfbd812 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:39.332022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cf11f-5566-4d6e-affc-b6fbecfbd812 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:39.332301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cf11f-5566-4d6e-affc-b6fbecfbd812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:39.362677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c81cf11f-5566-4d6e-affc-b6fbecfbd812 None None] MFA Rules not processed for user `124511010d044f8fb07dc09e7d754303`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:39.364053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c81cf11f-5566-4d6e-affc-b6fbecfbd812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:39.374255 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1784/3540] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:39.383947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656dcd0-1067-4307-a8ef-8a19da2ca262 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:39.384229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656dcd0-1067-4307-a8ef-8a19da2ca262 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:39.384494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656dcd0-1067-4307-a8ef-8a19da2ca262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:39.407346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1656dcd0-1067-4307-a8ef-8a19da2ca262 None None] MFA Rules not processed for user `124511010d044f8fb07dc09e7d754303`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:39.408815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1656dcd0-1067-4307-a8ef-8a19da2ca262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:39.423347 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1757/3541] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:45.242415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cf11f-5566-4d6e-affc-b6fbecfbd812 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:45.263702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cf11f-5566-4d6e-affc-b6fbecfbd812 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:45.281794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294b0bd-333d-4764-8899-2d2a9c54705e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:45.282774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294b0bd-333d-4764-8899-2d2a9c54705e service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:45.283076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294b0bd-333d-4764-8899-2d2a9c54705e service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:45.283428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294b0bd-333d-4764-8899-2d2a9c54705e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:45.299820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294b0bd-333d-4764-8899-2d2a9c54705e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:45.305779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294b0bd-333d-4764-8899-2d2a9c54705e service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:45.323219 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1785/3542] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:46.580350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a07c-9085-4069-a7a1-3db6da348436 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:46.580944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a07c-9085-4069-a7a1-3db6da348436 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:46.581779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a07c-9085-4069-a7a1-3db6da348436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:46.614695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c589a07c-9085-4069-a7a1-3db6da348436 None None] MFA Rules not processed for user `124511010d044f8fb07dc09e7d754303`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:46.617468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c589a07c-9085-4069-a7a1-3db6da348436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:46.633774 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1758/3543] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:46.640636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294b0bd-333d-4764-8899-2d2a9c54705e service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:46.642629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0294b0bd-333d-4764-8899-2d2a9c54705e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:46.659459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294b0bd-333d-4764-8899-2d2a9c54705e service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:46.667397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:46.668561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:46.668837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:46.669237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] PATH_INFO: `/v3/users/c7ceb51003c24728b665e7f641e3634b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:46.671343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ceb51003c24728b665e7f641e3634b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:46.674287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:46.755085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ceb51003c24728b665e7f641e3634b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:46.782602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ceb51003c24728b665e7f641e3634b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:46.795634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ceb51003c24728b665e7f641e3634b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:46.808973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ceb51003c24728b665e7f641e3634b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:46.809866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ceb51003c24728b665e7f641e3634b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:46.821117 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1786/3544] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:03:46 2022] DELETE /identity/v3/users/c7ceb51003c24728b665e7f641e3634b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:03:46.824474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b3f0-fc33-4479-86d4-40dfa7565380 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:46.824884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b3f0-fc33-4479-86d4-40dfa7565380 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:46.825231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b3f0-fc33-4479-86d4-40dfa7565380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:46.850915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3138b3f0-fc33-4479-86d4-40dfa7565380 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:46.853016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3138b3f0-fc33-4479-86d4-40dfa7565380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:46.947242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:46.947242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:46.962862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:46.981309 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1759/3545] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:03:46.996622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b3f0-fc33-4479-86d4-40dfa7565380 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:46.999405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3138b3f0-fc33-4479-86d4-40dfa7565380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:47.090870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b3f0-fc33-4479-86d4-40dfa7565380 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:47.094958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71111345-5a83-443b-afc6-ad81a6b5d4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:47.095602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71111345-5a83-443b-afc6-ad81a6b5d4aa service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:47.095898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71111345-5a83-443b-afc6-ad81a6b5d4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:47.096259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71111345-5a83-443b-afc6-ad81a6b5d4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:47.110411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71111345-5a83-443b-afc6-ad81a6b5d4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:47.111877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71111345-5a83-443b-afc6-ad81a6b5d4aa service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:47.114726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8bc9f-8cd2-46a6-8dd5-7685502d9e6e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:47.123462 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1760/3546] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:03:47.126075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae5326-4489-4c47-b6b1-72edff6ed123 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:47.126661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae5326-4489-4c47-b6b1-72edff6ed123 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:47.127024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae5326-4489-4c47-b6b1-72edff6ed123 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:47.127215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae5326-4489-4c47-b6b1-72edff6ed123 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:47.129581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7dae5326-4489-4c47-b6b1-72edff6ed123 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:47.148648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae5326-4489-4c47-b6b1-72edff6ed123 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:47.152550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae5326-4489-4c47-b6b1-72edff6ed123 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:47.175452 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1787/3547] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:47.488075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71111345-5a83-443b-afc6-ad81a6b5d4aa service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:47.490887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-71111345-5a83-443b-afc6-ad81a6b5d4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:47.541805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71111345-5a83-443b-afc6-ad81a6b5d4aa service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:47.547312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:47.547920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:47.548258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:47.548535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] PATH_INFO: `/v3/projects/642b98a4621b4d84b3e1e97137a68d33` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:47.553865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=642b98a4621b4d84b3e1e97137a68d33)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:47.555881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:47.628347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642b98a4621b4d84b3e1e97137a68d33', 'request_id': 'req-c94d49ce-d34e-41a3-95e6-f4218631ed32'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:47.644813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642b98a4621b4d84b3e1e97137a68d33', 'request_id': 'req-c94d49ce-d34e-41a3-95e6-f4218631ed32'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:47.658062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642b98a4621b4d84b3e1e97137a68d33', 'request_id': 'req-c94d49ce-d34e-41a3-95e6-f4218631ed32'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:47.678917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:47.696776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] The token cache is being invalidate because project 642b98a4621b4d84b3e1e97137a68d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:03:47.697797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:47.701434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c94d49ce-d34e-41a3-95e6-f4218631ed32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:03:47.703574 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1761/3548] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:03:47 2022] DELETE /identity/v3/projects/642b98a4621b4d84b3e1e97137a68d33 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:03:47.709388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd453-ae37-4a31-bb87-eba001f3690a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:47.710283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd453-ae37-4a31-bb87-eba001f3690a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:47.710688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd453-ae37-4a31-bb87-eba001f3690a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:47.736514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-518fd453-ae37-4a31-bb87-eba001f3690a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:47.737749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-518fd453-ae37-4a31-bb87-eba001f3690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:47.816599 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1788/3549] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:03:47.826362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8308d00-d5aa-4e67-a96d-86b83b700a2e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:47.827085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8308d00-d5aa-4e67-a96d-86b83b700a2e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:47.827681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8308d00-d5aa-4e67-a96d-86b83b700a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:47.874235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d8308d00-d5aa-4e67-a96d-86b83b700a2e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:47.875771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8308d00-d5aa-4e67-a96d-86b83b700a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:47.889137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1762/3550] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:47.892240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fd453-ae37-4a31-bb87-eba001f3690a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:47.906647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fd453-ae37-4a31-bb87-eba001f3690a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:47.913908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b37940-b322-4c9d-8f84-d402d04ff9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:47.914907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b37940-b322-4c9d-8f84-d402d04ff9aa admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:47.915092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b37940-b322-4c9d-8f84-d402d04ff9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:47.915352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b37940-b322-4c9d-8f84-d402d04ff9aa admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:47.916262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b37940-b322-4c9d-8f84-d402d04ff9aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:03:47.916746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b37940-b322-4c9d-8f84-d402d04ff9aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:47.918761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b37940-b322-4c9d-8f84-d402d04ff9aa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:47.936149 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1789/3551] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:03:47.941897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8308d00-d5aa-4e67-a96d-86b83b700a2e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:47.954399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8308d00-d5aa-4e67-a96d-86b83b700a2e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:47.961095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069003e-6f2e-4077-9422-480338b4d0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:47.961928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069003e-6f2e-4077-9422-480338b4d0c8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:47.962079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069003e-6f2e-4077-9422-480338b4d0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:47.962262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069003e-6f2e-4077-9422-480338b4d0c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:47.962885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069003e-6f2e-4077-9422-480338b4d0c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:47.965094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069003e-6f2e-4077-9422-480338b4d0c8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:48.001946 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1763/3552] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:03:47 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:03:48.006790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b37940-b322-4c9d-8f84-d402d04ff9aa admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:48.023245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b37940-b322-4c9d-8f84-d402d04ff9aa admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:48.122627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41312b2f-cf0d-4ecf-8307-cafe47c4bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:48.123263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41312b2f-cf0d-4ecf-8307-cafe47c4bd7f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:48.123547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41312b2f-cf0d-4ecf-8307-cafe47c4bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:48.123807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41312b2f-cf0d-4ecf-8307-cafe47c4bd7f admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:48.125730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41312b2f-cf0d-4ecf-8307-cafe47c4bd7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:48.129795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41312b2f-cf0d-4ecf-8307-cafe47c4bd7f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:48.196328 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1790/3553] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:03:48 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:03:48.200363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069003e-6f2e-4077-9422-480338b4d0c8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:48.225612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069003e-6f2e-4077-9422-480338b4d0c8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:48.232421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c4e8d-6c71-4064-bc4d-f16b1ec22213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:48.233794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4e8d-6c71-4064-bc4d-f16b1ec22213 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:48.233794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4e8d-6c71-4064-bc4d-f16b1ec22213 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:48.233794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4e8d-6c71-4064-bc4d-f16b1ec22213 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:48.237077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c4e8d-6c71-4064-bc4d-f16b1ec22213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:48.237383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c4e8d-6c71-4064-bc4d-f16b1ec22213 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:48.260048 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1764/3554] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:03:48.264368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41312b2f-cf0d-4ecf-8307-cafe47c4bd7f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:48.280795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41312b2f-cf0d-4ecf-8307-cafe47c4bd7f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:48.293638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c85bcb-49d2-4b16-b73e-d16d6d702edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:48.301754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c85bcb-49d2-4b16-b73e-d16d6d702edf admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:48.302022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c85bcb-49d2-4b16-b73e-d16d6d702edf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:48.302371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c85bcb-49d2-4b16-b73e-d16d6d702edf admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:48.306104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c85bcb-49d2-4b16-b73e-d16d6d702edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:48.312837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c85bcb-49d2-4b16-b73e-d16d6d702edf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:48.314957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:48.348100 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1791/3555] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:03:48.348807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c4e8d-6c71-4064-bc4d-f16b1ec22213 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:48.366399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c4e8d-6c71-4064-bc4d-f16b1ec22213 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:48.375976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e5b47-0414-402f-8a75-e78f76c3e1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:48.376978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e5b47-0414-402f-8a75-e78f76c3e1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:48.377425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e5b47-0414-402f-8a75-e78f76c3e1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:48.377734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e5b47-0414-402f-8a75-e78f76c3e1bf admin admin] PATH_INFO: `/v3/projects/172114b9a46e4df8ae8076ed582f0b5f/users/90e8754a100240a498e422de07c76bbf/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:48.400840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e5b47-0414-402f-8a75-e78f76c3e1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=172114b9a46e4df8ae8076ed582f0b5f, user_id=90e8754a100240a498e422de07c76bbf, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:48.404348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e5b47-0414-402f-8a75-e78f76c3e1bf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:48.428735 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1765/3556] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:03:48 2022] PUT /identity/v3/projects/172114b9a46e4df8ae8076ed582f0b5f/users/90e8754a100240a498e422de07c76bbf/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:03:48.434058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f524cdc-b742-43bc-9acd-aff6a17d238a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:48.434353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f524cdc-b742-43bc-9acd-aff6a17d238a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:48.436336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f524cdc-b742-43bc-9acd-aff6a17d238a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:48.483015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9f524cdc-b742-43bc-9acd-aff6a17d238a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:48.484234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9f524cdc-b742-43bc-9acd-aff6a17d238a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:48.562412 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1792/3557] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:48.575426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e5b47-0414-402f-8a75-e78f76c3e1bf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:48.578610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c13e5b47-0414-402f-8a75-e78f76c3e1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:48.677462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e5b47-0414-402f-8a75-e78f76c3e1bf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:48.682476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b59cd-d0a8-431f-a78a-7411ca2bb02a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:48.683739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b59cd-d0a8-431f-a78a-7411ca2bb02a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:48.684277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b59cd-d0a8-431f-a78a-7411ca2bb02a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:48.684727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b59cd-d0a8-431f-a78a-7411ca2bb02a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:48.701639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b59cd-d0a8-431f-a78a-7411ca2bb02a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:48.703455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b59cd-d0a8-431f-a78a-7411ca2bb02a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:48.720593 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1766/3558] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:03:54.933917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce06c8-4c7a-456c-afee-131593d8cc70 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:54.934470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce06c8-4c7a-456c-afee-131593d8cc70 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:54.934470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce06c8-4c7a-456c-afee-131593d8cc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:54.996992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a6ce06c8-4c7a-456c-afee-131593d8cc70 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:54.999173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a6ce06c8-4c7a-456c-afee-131593d8cc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:55.177869 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1793/3559] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:55.189366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079eeaf-7e6a-444f-abf3-722bc31814b8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:55.191306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079eeaf-7e6a-444f-abf3-722bc31814b8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:55.191788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079eeaf-7e6a-444f-abf3-722bc31814b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:55.223979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4079eeaf-7e6a-444f-abf3-722bc31814b8 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:55.225417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4079eeaf-7e6a-444f-abf3-722bc31814b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:55.251947 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1767/3560] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:55.766237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa09ab7-3814-4432-a206-81521893dc5a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:55.769601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa09ab7-3814-4432-a206-81521893dc5a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:55.769829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa09ab7-3814-4432-a206-81521893dc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:55.811679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bfa09ab7-3814-4432-a206-81521893dc5a None None] MFA Rules not processed for user `124511010d044f8fb07dc09e7d754303`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:03:55.812915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bfa09ab7-3814-4432-a206-81521893dc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:55.913818 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1794/3561] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:03:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:03:58.379191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079eeaf-7e6a-444f-abf3-722bc31814b8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:03:58.381493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4079eeaf-7e6a-444f-abf3-722bc31814b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:03:58.496299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079eeaf-7e6a-444f-abf3-722bc31814b8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:03:58.504731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bce92-9f3f-4f55-9c28-593708a30ed6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:03:58.505687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bce92-9f3f-4f55-9c28-593708a30ed6 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:03:58.506174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bce92-9f3f-4f55-9c28-593708a30ed6 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:03:58.506690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bce92-9f3f-4f55-9c28-593708a30ed6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:03:58.516556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bce92-9f3f-4f55-9c28-593708a30ed6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:03:58.519136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bce92-9f3f-4f55-9c28-593708a30ed6 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:03:58.532198 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1768/3562] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:03:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:04:03.386904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a650a5-614a-4757-8ec4-aeac906586e0 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:03.387558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a650a5-614a-4757-8ec4-aeac906586e0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:03.387664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a650a5-614a-4757-8ec4-aeac906586e0 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:03.390143 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1795/3563] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 21:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:04:03.396599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eaeab-b4fa-4fbe-945f-cef1f8d539b3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:03.397201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eaeab-b4fa-4fbe-945f-cef1f8d539b3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:03.397201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eaeab-b4fa-4fbe-945f-cef1f8d539b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:03.423490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:03.444016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c39eaeab-b4fa-4fbe-945f-cef1f8d539b3 None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:03.444016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c39eaeab-b4fa-4fbe-945f-cef1f8d539b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:03.451280 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1769/3564] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:04:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:03.469706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a650a5-614a-4757-8ec4-aeac906586e0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:03.483112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a650a5-614a-4757-8ec4-aeac906586e0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:03.487546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fcab1-4a55-461e-bfbe-d0ffeff476f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:03.488323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fcab1-4a55-461e-bfbe-d0ffeff476f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:03.488735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fcab1-4a55-461e-bfbe-d0ffeff476f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:03.489063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fcab1-4a55-461e-bfbe-d0ffeff476f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:03.499188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fcab1-4a55-461e-bfbe-d0ffeff476f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:03.500768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fcab1-4a55-461e-bfbe-d0ffeff476f6 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:03.511061 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1796/3565] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:04:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:04:12.674134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325bf10-0b28-4c55-9011-e12d04cf90a1 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:12.675095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325bf10-0b28-4c55-9011-e12d04cf90a1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:12.675552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325bf10-0b28-4c55-9011-e12d04cf90a1 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:12.677624 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1770/3566] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 21:04:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:04:12.684698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04eca3-49dd-4aa0-8352-1b58b3096b72 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:12.685266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04eca3-49dd-4aa0-8352-1b58b3096b72 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:12.685266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04eca3-49dd-4aa0-8352-1b58b3096b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:12.707437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2a04eca3-49dd-4aa0-8352-1b58b3096b72 None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:12.709378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2a04eca3-49dd-4aa0-8352-1b58b3096b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:12.716523 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1797/3567] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:12.731421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325bf10-0b28-4c55-9011-e12d04cf90a1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:12.734364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9325bf10-0b28-4c55-9011-e12d04cf90a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:12.749832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325bf10-0b28-4c55-9011-e12d04cf90a1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:12.753416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3b323-5509-459b-888f-5c5f74c03ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:12.754081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3b323-5509-459b-888f-5c5f74c03ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:12.754416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3b323-5509-459b-888f-5c5f74c03ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:12.754951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3b323-5509-459b-888f-5c5f74c03ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:12.763246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3b323-5509-459b-888f-5c5f74c03ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:12.764794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3b323-5509-459b-888f-5c5f74c03ed4 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:12.773581 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1771/3568] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:04:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:04:14.638271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04eca3-49dd-4aa0-8352-1b58b3096b72 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:14.665313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04eca3-49dd-4aa0-8352-1b58b3096b72 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:14.684584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56abfdb-ffd5-4dd0-91ff-62793fcbc08b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:14.685234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56abfdb-ffd5-4dd0-91ff-62793fcbc08b service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:14.685558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56abfdb-ffd5-4dd0-91ff-62793fcbc08b service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:14.685820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56abfdb-ffd5-4dd0-91ff-62793fcbc08b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:14.690928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b56abfdb-ffd5-4dd0-91ff-62793fcbc08b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:14.704564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:14.782915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56abfdb-ffd5-4dd0-91ff-62793fcbc08b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:14.784606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56abfdb-ffd5-4dd0-91ff-62793fcbc08b service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:14.798018 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1798/3569] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:04:14 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:04:19.817686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8766e9c-9b32-4f56-96e1-1efa9e86f077 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:19.818328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8766e9c-9b32-4f56-96e1-1efa9e86f077 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:19.818328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8766e9c-9b32-4f56-96e1-1efa9e86f077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:19.837947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d8766e9c-9b32-4f56-96e1-1efa9e86f077 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:19.839158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8766e9c-9b32-4f56-96e1-1efa9e86f077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:19.846556 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1772/3570] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:24.790473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=23 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:26.279020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0401f38-1075-4812-bd41-13164aa7dd10 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:26.279595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0401f38-1075-4812-bd41-13164aa7dd10 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:26.279595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0401f38-1075-4812-bd41-13164aa7dd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:26.318907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b0401f38-1075-4812-bd41-13164aa7dd10 None None] MFA Rules not processed for user `a8af9544fe68497c8557248575f184ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:26.321345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b0401f38-1075-4812-bd41-13164aa7dd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:26.401093 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1799/3571] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:04:26.975398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ac2cc-0a41-485c-a8b5-bf83510d9b80 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:26.976021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ac2cc-0a41-485c-a8b5-bf83510d9b80 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:26.976149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ac2cc-0a41-485c-a8b5-bf83510d9b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:26.996806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-767ac2cc-0a41-485c-a8b5-bf83510d9b80 None None] MFA Rules not processed for user `fb8dc371c65f4bbb92d559c7ed78964c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:26.998074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-767ac2cc-0a41-485c-a8b5-bf83510d9b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:27.059743 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1773/3572] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:28.081360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0401f38-1075-4812-bd41-13164aa7dd10 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:28.092996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0401f38-1075-4812-bd41-13164aa7dd10 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:28.096749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f83a99-fcf8-44e9-a570-d90f23b1607e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:28.097515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f83a99-fcf8-44e9-a570-d90f23b1607e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:28.097917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f83a99-fcf8-44e9-a570-d90f23b1607e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:28.098189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f83a99-fcf8-44e9-a570-d90f23b1607e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:28.099916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-97f83a99-fcf8-44e9-a570-d90f23b1607e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:28.110891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f83a99-fcf8-44e9-a570-d90f23b1607e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:28.111993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f83a99-fcf8-44e9-a570-d90f23b1607e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:28.123458 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1800/3573] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:04:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:04:37.052434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=37 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:38.119622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=31 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:41.013914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ac2cc-0a41-485c-a8b5-bf83510d9b80 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:41.016179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-767ac2cc-0a41-485c-a8b5-bf83510d9b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:41.044544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ac2cc-0a41-485c-a8b5-bf83510d9b80 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:41.051374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:41.052491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:41.052928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:41.053401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] PATH_INFO: `/v3/users/f2295a9d9b034667957b8e1d3c4901ab` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:41.054730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2295a9d9b034667957b8e1d3c4901ab)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:41.056968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:41.128533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2295a9d9b034667957b8e1d3c4901ab'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:41.137536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2295a9d9b034667957b8e1d3c4901ab'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:41.152109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2295a9d9b034667957b8e1d3c4901ab'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:41.161229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2295a9d9b034667957b8e1d3c4901ab'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:41.161982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1202a4d9-ef60-440a-8250-1e9ed75a73c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2295a9d9b034667957b8e1d3c4901ab'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:41.168206 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1774/3574] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:04:41 2022] DELETE /identity/v3/users/f2295a9d9b034667957b8e1d3c4901ab => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:41.350935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f83a99-fcf8-44e9-a570-d90f23b1607e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:41.364441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f83a99-fcf8-44e9-a570-d90f23b1607e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:41.434581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:41.435211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:41.435484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:41.435880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] PATH_INFO: `/v3/projects/f278bb578eca4b5895634257d5ac832e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:41.437022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f278bb578eca4b5895634257d5ac832e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:41.438958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:41.490484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f278bb578eca4b5895634257d5ac832e', 'request_id': 'req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:41.498746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f278bb578eca4b5895634257d5ac832e', 'request_id': 'req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:41.505832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f278bb578eca4b5895634257d5ac832e', 'request_id': 'req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:41.518165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] The token cache is being invalidate because project f278bb578eca4b5895634257d5ac832e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:04:41.518392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:41.519231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca8970ba-ccdf-41bc-b0fa-cb7e33833ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:41.520905 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1801/3575] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:04:41 2022] DELETE /identity/v3/projects/f278bb578eca4b5895634257d5ac832e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:41.525938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e6a4-9141-4519-94b4-cd301bfa34fc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:41.526225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e6a4-9141-4519-94b4-cd301bfa34fc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:41.526600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e6a4-9141-4519-94b4-cd301bfa34fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:41.550889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e889e6a4-9141-4519-94b4-cd301bfa34fc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:41.552322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e889e6a4-9141-4519-94b4-cd301bfa34fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:41.621176 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1775/3576] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:41.631148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05648355-4108-4f43-878e-a2b7367556b1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:41.631694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05648355-4108-4f43-878e-a2b7367556b1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:41.632198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05648355-4108-4f43-878e-a2b7367556b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:41.666290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-05648355-4108-4f43-878e-a2b7367556b1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:41.667504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-05648355-4108-4f43-878e-a2b7367556b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:41.683048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e6a4-9141-4519-94b4-cd301bfa34fc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:41.694689 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1802/3577] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:41.696172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e6a4-9141-4519-94b4-cd301bfa34fc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:41.702783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7f10c-8aa1-4e02-8039-3448921174e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:41.703751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7f10c-8aa1-4e02-8039-3448921174e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:41.704316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7f10c-8aa1-4e02-8039-3448921174e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:41.704737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7f10c-8aa1-4e02-8039-3448921174e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:41.705602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7f10c-8aa1-4e02-8039-3448921174e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:04:41.706130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7f10c-8aa1-4e02-8039-3448921174e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:41.709837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7f10c-8aa1-4e02-8039-3448921174e2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:41.748922 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1776/3578] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:04:41.752867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05648355-4108-4f43-878e-a2b7367556b1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:41.763599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05648355-4108-4f43-878e-a2b7367556b1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:41.769855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f40fe3-8f2b-4fbf-a8b2-b33013167822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:41.770831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f40fe3-8f2b-4fbf-a8b2-b33013167822 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:41.771116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f40fe3-8f2b-4fbf-a8b2-b33013167822 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:41.771381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f40fe3-8f2b-4fbf-a8b2-b33013167822 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:41.772027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f40fe3-8f2b-4fbf-a8b2-b33013167822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:41.773995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f40fe3-8f2b-4fbf-a8b2-b33013167822 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:41.818846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7f10c-8aa1-4e02-8039-3448921174e2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:41.825214 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1803/3579] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:04:41 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:04:41.832141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7f10c-8aa1-4e02-8039-3448921174e2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:41.905480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab8670-5bf0-475c-abe9-a326fbd70ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:41.906384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab8670-5bf0-475c-abe9-a326fbd70ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:41.907002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab8670-5bf0-475c-abe9-a326fbd70ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:41.907347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab8670-5bf0-475c-abe9-a326fbd70ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:41.908072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab8670-5bf0-475c-abe9-a326fbd70ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:41.910365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab8670-5bf0-475c-abe9-a326fbd70ae9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:41.920787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:41.944458 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1777/3580] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:04:41 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:04:41.945736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f40fe3-8f2b-4fbf-a8b2-b33013167822 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:41.967946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f40fe3-8f2b-4fbf-a8b2-b33013167822 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:41.976244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944a5dd-2f5b-4395-943f-1bab5c526f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:41.976862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944a5dd-2f5b-4395-943f-1bab5c526f7c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:41.977171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944a5dd-2f5b-4395-943f-1bab5c526f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:41.977438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944a5dd-2f5b-4395-943f-1bab5c526f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:41.978254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944a5dd-2f5b-4395-943f-1bab5c526f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:41.980259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944a5dd-2f5b-4395-943f-1bab5c526f7c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:41.992135 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1804/3581] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:04:41.996380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab8670-5bf0-475c-abe9-a326fbd70ae9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:42.008314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab8670-5bf0-475c-abe9-a326fbd70ae9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:42.015315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeec867-e525-4703-aff7-36796b45fc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:42.016014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeec867-e525-4703-aff7-36796b45fc9b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:42.016183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeec867-e525-4703-aff7-36796b45fc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:42.016686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeec867-e525-4703-aff7-36796b45fc9b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:42.017722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeec867-e525-4703-aff7-36796b45fc9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:42.020460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeec867-e525-4703-aff7-36796b45fc9b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:42.039778 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1778/3582] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:04:42.040668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944a5dd-2f5b-4395-943f-1bab5c526f7c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:42.052042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944a5dd-2f5b-4395-943f-1bab5c526f7c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:42.059992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b90415-55c6-40cd-995e-c6fece7f1e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:42.060822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b90415-55c6-40cd-995e-c6fece7f1e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:42.061187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b90415-55c6-40cd-995e-c6fece7f1e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:42.061517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b90415-55c6-40cd-995e-c6fece7f1e4a admin admin] PATH_INFO: `/v3/projects/b6a0bed8192d413da2cc4d413b5bc569/users/c7deeb24e8834437a8410f8cc0c29f6d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:42.082958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b90415-55c6-40cd-995e-c6fece7f1e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a0bed8192d413da2cc4d413b5bc569, user_id=c7deeb24e8834437a8410f8cc0c29f6d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:42.085430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b90415-55c6-40cd-995e-c6fece7f1e4a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:42.101219 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1805/3583] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:04:42 2022] PUT /identity/v3/projects/b6a0bed8192d413da2cc4d413b5bc569/users/c7deeb24e8834437a8410f8cc0c29f6d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:42.106251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa09f4e-1283-4044-a916-5b71bdcee237 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:42.106533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa09f4e-1283-4044-a916-5b71bdcee237 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:42.106803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa09f4e-1283-4044-a916-5b71bdcee237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:42.127001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0fa09f4e-1283-4044-a916-5b71bdcee237 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:42.128212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0fa09f4e-1283-4044-a916-5b71bdcee237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:42.156902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b90415-55c6-40cd-995e-c6fece7f1e4a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:42.162146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b8b90415-55c6-40cd-995e-c6fece7f1e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:42.207729 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1779/3584] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:42.209154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:42.214942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa09f4e-1283-4044-a916-5b71bdcee237 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:42.216471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0fa09f4e-1283-4044-a916-5b71bdcee237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:42.245466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b90415-55c6-40cd-995e-c6fece7f1e4a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:42.250585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:42.251641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:42.251922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:42.252310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] PATH_INFO: `/v3/users/14c805d8b3e944e8ad9f6150322b96ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:42.255627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c805d8b3e944e8ad9f6150322b96ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:42.258705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:42.290076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa09f4e-1283-4044-a916-5b71bdcee237 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:42.294148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123eda8-bfe6-4632-ac0f-9fee91f1c2e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:42.294918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123eda8-bfe6-4632-ac0f-9fee91f1c2e1 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:42.295247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123eda8-bfe6-4632-ac0f-9fee91f1c2e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:42.295507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123eda8-bfe6-4632-ac0f-9fee91f1c2e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:42.306734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123eda8-bfe6-4632-ac0f-9fee91f1c2e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:42.308299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123eda8-bfe6-4632-ac0f-9fee91f1c2e1 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:42.319431 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1780/3585] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:04:42.321637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c805d8b3e944e8ad9f6150322b96ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:42.339453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c805d8b3e944e8ad9f6150322b96ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:42.352971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c805d8b3e944e8ad9f6150322b96ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:42.362619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c805d8b3e944e8ad9f6150322b96ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:42.364198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c805d8b3e944e8ad9f6150322b96ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:42.372721 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1806/3586] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:04:42 2022] DELETE /identity/v3/users/14c805d8b3e944e8ad9f6150322b96ce => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:42.687706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123eda8-bfe6-4632-ac0f-9fee91f1c2e1 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:42.708095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123eda8-bfe6-4632-ac0f-9fee91f1c2e1 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:42.782377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:42.783009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:42.783278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:42.783552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] PATH_INFO: `/v3/projects/c6d5585dc2254ed2aed773305d209049` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:42.785210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d5585dc2254ed2aed773305d209049)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:42.787360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:42.821530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=3,keystone:SELECT=96,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:42.871371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d5585dc2254ed2aed773305d209049', 'request_id': 'req-68986474-d0c8-4d71-ab38-65397f4aeafe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:42.882984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d5585dc2254ed2aed773305d209049', 'request_id': 'req-68986474-d0c8-4d71-ab38-65397f4aeafe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:42.912108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d5585dc2254ed2aed773305d209049', 'request_id': 'req-68986474-d0c8-4d71-ab38-65397f4aeafe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:42.928189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] The token cache is being invalidate because project c6d5585dc2254ed2aed773305d209049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:04:42.928537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:42.929186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:42.931859 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1781/3587] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:04:42 2022] DELETE /identity/v3/projects/c6d5585dc2254ed2aed773305d209049 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:42.935711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:42.937712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:42.971277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe71f1-77b7-4f63-8f5f-dccd3b98c4ca admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:43.098035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:43.098498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:43.098842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:43.099177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] PATH_INFO: `/v3/users/7d02831cb7e24fc69671dd999f4746fb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:43.100744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d02831cb7e24fc69671dd999f4746fb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:43.103020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:43.183882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d02831cb7e24fc69671dd999f4746fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:43.185070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:43.201468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d02831cb7e24fc69671dd999f4746fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:43.219149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d02831cb7e24fc69671dd999f4746fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:43.234032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d02831cb7e24fc69671dd999f4746fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:43.234932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b494247d-82de-466f-ae88-4c39eb24b8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d02831cb7e24fc69671dd999f4746fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:43.247206 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1807/3588] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:04:42 2022] DELETE /identity/v3/users/7d02831cb7e24fc69671dd999f4746fb => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:43.461321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:43.463790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:43.481682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68986474-d0c8-4d71-ab38-65397f4aeafe admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:43.556124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:43.556763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:43.557067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:43.557332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] PATH_INFO: `/v3/projects/42cfcf1dcb864013ac16376c2419d05f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:43.558450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42cfcf1dcb864013ac16376c2419d05f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:43.560381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:43.613911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42cfcf1dcb864013ac16376c2419d05f', 'request_id': 'req-7e6d3977-7fff-45a7-9017-9025ec03c1f6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:43.625535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42cfcf1dcb864013ac16376c2419d05f', 'request_id': 'req-7e6d3977-7fff-45a7-9017-9025ec03c1f6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:43.638679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42cfcf1dcb864013ac16376c2419d05f', 'request_id': 'req-7e6d3977-7fff-45a7-9017-9025ec03c1f6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:43.652724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] The token cache is being invalidate because project 42cfcf1dcb864013ac16376c2419d05f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:04:43.652997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:43.654179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e6d3977-7fff-45a7-9017-9025ec03c1f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:43.656435 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1782/3589] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:04:43 2022] DELETE /identity/v3/projects/42cfcf1dcb864013ac16376c2419d05f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:43.663401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9864b-cd4b-41ce-b787-bfec5bb41f3b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:43.665935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9864b-cd4b-41ce-b787-bfec5bb41f3b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:43.666347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9864b-cd4b-41ce-b787-bfec5bb41f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:43.699130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-42d9864b-cd4b-41ce-b787-bfec5bb41f3b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:43.705802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-42d9864b-cd4b-41ce-b787-bfec5bb41f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:43.822573 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1808/3590] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:43.835086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e509e-763f-41a9-a824-5fdcd1d7879e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:43.835362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e509e-763f-41a9-a824-5fdcd1d7879e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:43.835857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e509e-763f-41a9-a824-5fdcd1d7879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:43.866177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-451e509e-763f-41a9-a824-5fdcd1d7879e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:43.867538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-451e509e-763f-41a9-a824-5fdcd1d7879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:43.878703 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1783/3591] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:43.882895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9864b-cd4b-41ce-b787-bfec5bb41f3b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:43.895260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9864b-cd4b-41ce-b787-bfec5bb41f3b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:43.899851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c007756-2232-4c9b-bba2-c71bc482908b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:43.900444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c007756-2232-4c9b-bba2-c71bc482908b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:43.900906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c007756-2232-4c9b-bba2-c71bc482908b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:43.901210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c007756-2232-4c9b-bba2-c71bc482908b admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:43.902000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c007756-2232-4c9b-bba2-c71bc482908b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:04:43.902292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c007756-2232-4c9b-bba2-c71bc482908b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:43.904056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c007756-2232-4c9b-bba2-c71bc482908b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:43.918211 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1809/3592] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:04:43.922688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e509e-763f-41a9-a824-5fdcd1d7879e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:43.933087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e509e-763f-41a9-a824-5fdcd1d7879e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:43.938405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e9b66-a9ba-441e-b093-236b213fa010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:43.939139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e9b66-a9ba-441e-b093-236b213fa010 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:43.939597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e9b66-a9ba-441e-b093-236b213fa010 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:43.940185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e9b66-a9ba-441e-b093-236b213fa010 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:43.941720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e9b66-a9ba-441e-b093-236b213fa010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:43.943546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e9b66-a9ba-441e-b093-236b213fa010 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:43.979872 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1784/3593] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:04:43 2022] POST /identity/v3/projects => generated 361 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:04:43.984564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c007756-2232-4c9b-bba2-c71bc482908b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:43.998994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c007756-2232-4c9b-bba2-c71bc482908b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:44.090904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0723b00-bc05-4d52-a815-9aa088656226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:44.091558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0723b00-bc05-4d52-a815-9aa088656226 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:44.091836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0723b00-bc05-4d52-a815-9aa088656226 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:44.092102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0723b00-bc05-4d52-a815-9aa088656226 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:44.092770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0723b00-bc05-4d52-a815-9aa088656226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:44.094829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0723b00-bc05-4d52-a815-9aa088656226 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:44.125364 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1810/3594] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:04:43 2022] POST /identity/v3/users => generated 281 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:04:44.128956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e9b66-a9ba-441e-b093-236b213fa010 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:44.143417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e9b66-a9ba-441e-b093-236b213fa010 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:44.152341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209396eb-5cbd-4173-9ec7-dbd97c59ab69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:44.153388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209396eb-5cbd-4173-9ec7-dbd97c59ab69 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:44.153843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209396eb-5cbd-4173-9ec7-dbd97c59ab69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:44.154266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209396eb-5cbd-4173-9ec7-dbd97c59ab69 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:44.155795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209396eb-5cbd-4173-9ec7-dbd97c59ab69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:44.158411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209396eb-5cbd-4173-9ec7-dbd97c59ab69 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:44.181970 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1785/3595] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:04:44.182634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0723b00-bc05-4d52-a815-9aa088656226 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:44.204518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0723b00-bc05-4d52-a815-9aa088656226 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:44.225631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0810ce9-8d7c-4c01-a766-705ce3e0f221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:44.226890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810ce9-8d7c-4c01-a766-705ce3e0f221 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:44.227218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810ce9-8d7c-4c01-a766-705ce3e0f221 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:44.228243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810ce9-8d7c-4c01-a766-705ce3e0f221 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:44.229654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0810ce9-8d7c-4c01-a766-705ce3e0f221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:44.236367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0810ce9-8d7c-4c01-a766-705ce3e0f221 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:44.264345 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1811/3596] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:04:44.266835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209396eb-5cbd-4173-9ec7-dbd97c59ab69 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:44.272901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:44.280746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209396eb-5cbd-4173-9ec7-dbd97c59ab69 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:44.289992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359976d4-097e-4254-a72a-67abe002dd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:44.290948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359976d4-097e-4254-a72a-67abe002dd15 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:44.291713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359976d4-097e-4254-a72a-67abe002dd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:44.291988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359976d4-097e-4254-a72a-67abe002dd15 admin admin] PATH_INFO: `/v3/projects/a12cc8a234bb4e3995970f81f531919a/users/e89cfcc06ee24c45bc6c6555bbba3f0f/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:44.309732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359976d4-097e-4254-a72a-67abe002dd15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12cc8a234bb4e3995970f81f531919a, user_id=e89cfcc06ee24c45bc6c6555bbba3f0f, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:44.312818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359976d4-097e-4254-a72a-67abe002dd15 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:44.341113 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1786/3597] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:04:44 2022] PUT /identity/v3/projects/a12cc8a234bb4e3995970f81f531919a/users/e89cfcc06ee24c45bc6c6555bbba3f0f/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:44.346884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdad59-0cad-48c6-b095-d1e97f65194d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:44.347788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdad59-0cad-48c6-b095-d1e97f65194d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:44.348204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdad59-0cad-48c6-b095-d1e97f65194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:44.389747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e5bdad59-0cad-48c6-b095-d1e97f65194d None None] MFA Rules not processed for user `e89cfcc06ee24c45bc6c6555bbba3f0f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:44.391263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e5bdad59-0cad-48c6-b095-d1e97f65194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:44.416093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:44.494392 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1812/3598] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:44.896556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28548-1bd3-4a71-b38c-472cf2c162b0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:44.897171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28548-1bd3-4a71-b38c-472cf2c162b0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:44.897351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28548-1bd3-4a71-b38c-472cf2c162b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:44.923048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b8b28548-1bd3-4a71-b38c-472cf2c162b0 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:44.924583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b8b28548-1bd3-4a71-b38c-472cf2c162b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:45.013083 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1787/3599] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:45.524138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdad59-0cad-48c6-b095-d1e97f65194d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:45.616826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdad59-0cad-48c6-b095-d1e97f65194d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:45.621659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:45.622457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:45.622740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:45.623013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] PATH_INFO: `/v3/users/e89cfcc06ee24c45bc6c6555bbba3f0f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:45.624329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e89cfcc06ee24c45bc6c6555bbba3f0f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:45.626368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:45.695045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e89cfcc06ee24c45bc6c6555bbba3f0f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:45.712272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e89cfcc06ee24c45bc6c6555bbba3f0f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:45.741004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e89cfcc06ee24c45bc6c6555bbba3f0f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:45.751553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e89cfcc06ee24c45bc6c6555bbba3f0f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:45.751721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e89cfcc06ee24c45bc6c6555bbba3f0f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:45.760125 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1813/3600] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:04:45 2022] DELETE /identity/v3/users/e89cfcc06ee24c45bc6c6555bbba3f0f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:45.764932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a97d8-b0b9-4e57-8d82-eeafe48081db None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:45.765173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a97d8-b0b9-4e57-8d82-eeafe48081db None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:45.765404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a97d8-b0b9-4e57-8d82-eeafe48081db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:45.807666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-630a97d8-b0b9-4e57-8d82-eeafe48081db None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:45.809256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-630a97d8-b0b9-4e57-8d82-eeafe48081db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:45.953868 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1788/3601] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:45.980376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:45.981985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:46.049702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:46.085731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eba84c-23c9-4b3a-8817-003c2b41965a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:46.109228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130ccfb-abaf-489f-b845-2cef307ae5db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:46.109775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130ccfb-abaf-489f-b845-2cef307ae5db service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:46.110062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130ccfb-abaf-489f-b845-2cef307ae5db service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:46.110337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130ccfb-abaf-489f-b845-2cef307ae5db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:46.127193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130ccfb-abaf-489f-b845-2cef307ae5db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:46.130017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130ccfb-abaf-489f-b845-2cef307ae5db service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:46.153060 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1814/3602] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:04:47.158442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a97d8-b0b9-4e57-8d82-eeafe48081db None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:47.161286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-630a97d8-b0b9-4e57-8d82-eeafe48081db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:47.193063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a97d8-b0b9-4e57-8d82-eeafe48081db None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:47.242176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:47.242841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:47.243111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:47.243377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] PATH_INFO: `/v3/projects/a12cc8a234bb4e3995970f81f531919a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:47.251337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12cc8a234bb4e3995970f81f531919a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:47.253796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:47.364925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12cc8a234bb4e3995970f81f531919a', 'request_id': 'req-2293b2a7-4890-4e26-86e4-2203c05ecca2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:47.382148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12cc8a234bb4e3995970f81f531919a', 'request_id': 'req-2293b2a7-4890-4e26-86e4-2203c05ecca2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:47.393566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12cc8a234bb4e3995970f81f531919a', 'request_id': 'req-2293b2a7-4890-4e26-86e4-2203c05ecca2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:47.410335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] The token cache is being invalidate because project a12cc8a234bb4e3995970f81f531919a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:04:47.410681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:47.412479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2293b2a7-4890-4e26-86e4-2203c05ecca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:04:47.414138 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1789/3603] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:04:47 2022] DELETE /identity/v3/projects/a12cc8a234bb4e3995970f81f531919a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:47.419936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd322c3b-db61-45fc-9357-ae29a111cf80 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:47.420516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd322c3b-db61-45fc-9357-ae29a111cf80 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:47.420884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd322c3b-db61-45fc-9357-ae29a111cf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:47.468093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fd322c3b-db61-45fc-9357-ae29a111cf80 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:47.469313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fd322c3b-db61-45fc-9357-ae29a111cf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:47.613751 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1815/3604] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:47.624102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4b3f6-72c3-4c01-8bc0-8fa0c27ac92d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:47.624102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4b3f6-72c3-4c01-8bc0-8fa0c27ac92d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:47.624370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4b3f6-72c3-4c01-8bc0-8fa0c27ac92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:47.646166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:47.666260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-44d4b3f6-72c3-4c01-8bc0-8fa0c27ac92d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:47.667512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-44d4b3f6-72c3-4c01-8bc0-8fa0c27ac92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:47.690146 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1790/3605] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:47.691693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd322c3b-db61-45fc-9357-ae29a111cf80 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:47.715278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd322c3b-db61-45fc-9357-ae29a111cf80 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:47.728365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f7fc1-7287-401f-8ea7-55733dce4972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:47.728982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f7fc1-7287-401f-8ea7-55733dce4972 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:47.729280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f7fc1-7287-401f-8ea7-55733dce4972 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:47.729542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f7fc1-7287-401f-8ea7-55733dce4972 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:47.730189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f7fc1-7287-401f-8ea7-55733dce4972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:04:47.730588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f7fc1-7287-401f-8ea7-55733dce4972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:47.732075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f7fc1-7287-401f-8ea7-55733dce4972 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:47.770564 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1816/3606] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:04:47.775464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4b3f6-72c3-4c01-8bc0-8fa0c27ac92d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:47.790136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4b3f6-72c3-4c01-8bc0-8fa0c27ac92d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:47.806707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47124dcc-3029-4f1d-925c-6499c1470d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:47.807055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47124dcc-3029-4f1d-925c-6499c1470d3f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:47.807329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47124dcc-3029-4f1d-925c-6499c1470d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:47.807590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47124dcc-3029-4f1d-925c-6499c1470d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:47.808231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47124dcc-3029-4f1d-925c-6499c1470d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:47.810160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47124dcc-3029-4f1d-925c-6499c1470d3f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:47.914721 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1791/3607] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:04:47 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:04:47.915869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f7fc1-7287-401f-8ea7-55733dce4972 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:47.926626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f7fc1-7287-401f-8ea7-55733dce4972 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:48.019636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcf524-bea4-4077-a682-261b9040d2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:48.020635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcf524-bea4-4077-a682-261b9040d2af admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:48.021612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcf524-bea4-4077-a682-261b9040d2af admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:48.022201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcf524-bea4-4077-a682-261b9040d2af admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:48.023741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcf524-bea4-4077-a682-261b9040d2af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:48.028922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcf524-bea4-4077-a682-261b9040d2af admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:48.064996 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1817/3608] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:04:47 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:04:48.069264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47124dcc-3029-4f1d-925c-6499c1470d3f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:48.095385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47124dcc-3029-4f1d-925c-6499c1470d3f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:48.106022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2e831-6fed-4adb-94d6-0e3108fe085f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:48.107024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2e831-6fed-4adb-94d6-0e3108fe085f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:48.108070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2e831-6fed-4adb-94d6-0e3108fe085f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:48.108451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2e831-6fed-4adb-94d6-0e3108fe085f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:48.109411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2e831-6fed-4adb-94d6-0e3108fe085f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:48.112098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2e831-6fed-4adb-94d6-0e3108fe085f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:48.133346 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1792/3609] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:04:48.134590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcf524-bea4-4077-a682-261b9040d2af admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:48.146746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcf524-bea4-4077-a682-261b9040d2af admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:48.156597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc4015-0692-46d6-864c-aa3b2d1904bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:48.157278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc4015-0692-46d6-864c-aa3b2d1904bd admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:48.157928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc4015-0692-46d6-864c-aa3b2d1904bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:48.158209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc4015-0692-46d6-864c-aa3b2d1904bd admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:48.158863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc4015-0692-46d6-864c-aa3b2d1904bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:48.162321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc4015-0692-46d6-864c-aa3b2d1904bd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:48.182206 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1818/3610] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:04:48.183436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2e831-6fed-4adb-94d6-0e3108fe085f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:48.196231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2e831-6fed-4adb-94d6-0e3108fe085f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:48.204920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a09d98-e443-4620-95ab-16f9df7256c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:48.205646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a09d98-e443-4620-95ab-16f9df7256c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:48.209515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a09d98-e443-4620-95ab-16f9df7256c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:48.209928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a09d98-e443-4620-95ab-16f9df7256c0 admin admin] PATH_INFO: `/v3/projects/cfe55a7ec21b4e6ebbf53684a480e535/users/2f4e4fc2f97f4c1a832c6348a97fd155/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:48.234626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a09d98-e443-4620-95ab-16f9df7256c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe55a7ec21b4e6ebbf53684a480e535, user_id=2f4e4fc2f97f4c1a832c6348a97fd155, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:48.237735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a09d98-e443-4620-95ab-16f9df7256c0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:48.261110 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1793/3611] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:04:48 2022] PUT /identity/v3/projects/cfe55a7ec21b4e6ebbf53684a480e535/users/2f4e4fc2f97f4c1a832c6348a97fd155/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:04:48.266307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442df06-8f7f-4fe4-899c-f8fe2454e770 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:48.266839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442df06-8f7f-4fe4-899c-f8fe2454e770 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:48.267316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442df06-8f7f-4fe4-899c-f8fe2454e770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:48.304754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7442df06-8f7f-4fe4-899c-f8fe2454e770 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:48.306438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7442df06-8f7f-4fe4-899c-f8fe2454e770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:48.393227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:48.422497 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1819/3612] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:48.446368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a09d98-e443-4620-95ab-16f9df7256c0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:48.448083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-09a09d98-e443-4620-95ab-16f9df7256c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:48.457126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5e0d-9246-4b20-8a42-007eb4154fe6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:48.457954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5e0d-9246-4b20-8a42-007eb4154fe6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:48.458427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5e0d-9246-4b20-8a42-007eb4154fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:48.506362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1c1e5e0d-9246-4b20-8a42-007eb4154fe6 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:48.507615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1c1e5e0d-9246-4b20-8a42-007eb4154fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:48.577311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a09d98-e443-4620-95ab-16f9df7256c0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:48.582349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48252d6d-50e5-4459-9f7f-c31e32719a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:48.583214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48252d6d-50e5-4459-9f7f-c31e32719a99 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:48.583790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48252d6d-50e5-4459-9f7f-c31e32719a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:48.584077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48252d6d-50e5-4459-9f7f-c31e32719a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:48.598980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48252d6d-50e5-4459-9f7f-c31e32719a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:48.600592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48252d6d-50e5-4459-9f7f-c31e32719a99 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:48.606359 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1820/3613] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:04:48.617399 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1794/3614] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:04:48.975607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91249e08-19c1-47f2-9503-96ec9799b4f7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:48.975911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91249e08-19c1-47f2-9503-96ec9799b4f7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:48.976461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91249e08-19c1-47f2-9503-96ec9799b4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:49.008543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-91249e08-19c1-47f2-9503-96ec9799b4f7 None None] MFA Rules not processed for user `c7deeb24e8834437a8410f8cc0c29f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:49.009922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-91249e08-19c1-47f2-9503-96ec9799b4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:49.169604 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1821/3615] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:49.770896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48252d6d-50e5-4459-9f7f-c31e32719a99 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:49.772395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-48252d6d-50e5-4459-9f7f-c31e32719a99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:49.939453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48252d6d-50e5-4459-9f7f-c31e32719a99 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:49.951315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69679b-16b4-4383-81eb-98769100f3b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:49.952471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69679b-16b4-4383-81eb-98769100f3b9 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:49.952753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69679b-16b4-4383-81eb-98769100f3b9 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:49.955151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69679b-16b4-4383-81eb-98769100f3b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:49.975872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69679b-16b4-4383-81eb-98769100f3b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:49.977518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69679b-16b4-4383-81eb-98769100f3b9 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:49.999038 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1795/3616] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:04:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:04:50.947674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633b039-3486-43d9-a61b-a5d6a6bd4b9e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:50.948549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633b039-3486-43d9-a61b-a5d6a6bd4b9e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:50.949025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633b039-3486-43d9-a61b-a5d6a6bd4b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:50.982651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1633b039-3486-43d9-a61b-a5d6a6bd4b9e None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:50.983901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1633b039-3486-43d9-a61b-a5d6a6bd4b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:51.000603 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1822/3617] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:54.562967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653cffa-c416-4112-a928-31183894e03e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:54.563567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653cffa-c416-4112-a928-31183894e03e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:54.563686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653cffa-c416-4112-a928-31183894e03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:54.606971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1653cffa-c416-4112-a928-31183894e03e None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:54.611104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1653cffa-c416-4112-a928-31183894e03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:54.657204 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1796/3618] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:04:56.780256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766eed9f-3f8a-4d88-accc-575d47645b15 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:56.781119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766eed9f-3f8a-4d88-accc-575d47645b15 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:56.781119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766eed9f-3f8a-4d88-accc-575d47645b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:56.863900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-766eed9f-3f8a-4d88-accc-575d47645b15 None None] MFA Rules not processed for user `2f4e4fc2f97f4c1a832c6348a97fd155`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:56.865431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-766eed9f-3f8a-4d88-accc-575d47645b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:57.000978 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1823/3619] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:04:59.269806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653cffa-c416-4112-a928-31183894e03e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:04:59.288012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653cffa-c416-4112-a928-31183894e03e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:04:59.295685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27749f34-7c41-467b-82fb-13f2760bdb32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:04:59.296306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27749f34-7c41-467b-82fb-13f2760bdb32 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:59.296584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27749f34-7c41-467b-82fb-13f2760bdb32 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:59.296853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27749f34-7c41-467b-82fb-13f2760bdb32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:59.308024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27749f34-7c41-467b-82fb-13f2760bdb32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:04:59.309673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27749f34-7c41-467b-82fb-13f2760bdb32 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:04:59.321341 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1797/3620] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:04:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:04:59.847269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7adf1-49cb-47ab-abd6-507e41c6ddfd None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:04:59.848608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7adf1-49cb-47ab-abd6-507e41c6ddfd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:04:59.849401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7adf1-49cb-47ab-abd6-507e41c6ddfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:04:59.866655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:59.874198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7cb7adf1-49cb-47ab-abd6-507e41c6ddfd None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:04:59.876240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7cb7adf1-49cb-47ab-abd6-507e41c6ddfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:04:59.887293 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1824/3621] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:01.206855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602714e-8e3b-40d0-8d3d-4ff2c01c7409 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:01.229969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602714e-8e3b-40d0-8d3d-4ff2c01c7409 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:01.229969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602714e-8e3b-40d0-8d3d-4ff2c01c7409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:01.254700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e602714e-8e3b-40d0-8d3d-4ff2c01c7409 None None] MFA Rules not processed for user `2f4e4fc2f97f4c1a832c6348a97fd155`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:01.256254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e602714e-8e3b-40d0-8d3d-4ff2c01c7409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:01.265526 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1798/3622] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:04.090846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd8036-257a-474f-b950-222a77e7bfa5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:04.091365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd8036-257a-474f-b950-222a77e7bfa5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:04.091482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd8036-257a-474f-b950-222a77e7bfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:04.127556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a2dd8036-257a-474f-b950-222a77e7bfa5 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:04.130970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a2dd8036-257a-474f-b950-222a77e7bfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:04.151680 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1825/3623] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:06.524360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602714e-8e3b-40d0-8d3d-4ff2c01c7409 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:06.529254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e602714e-8e3b-40d0-8d3d-4ff2c01c7409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:06.555211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:06.568351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602714e-8e3b-40d0-8d3d-4ff2c01c7409 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:06.598788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:06.598788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:06.598788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:06.598788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] PATH_INFO: `/v3/users/a8af9544fe68497c8557248575f184ed` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:06.599147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8af9544fe68497c8557248575f184ed)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:06.599147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:06.666103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8af9544fe68497c8557248575f184ed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:06.678179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8af9544fe68497c8557248575f184ed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:06.711669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8af9544fe68497c8557248575f184ed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:06.723823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8af9544fe68497c8557248575f184ed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:06.725570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8af9544fe68497c8557248575f184ed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:06.734095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1799/3624] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:05:06 2022] DELETE /identity/v3/users/a8af9544fe68497c8557248575f184ed => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:07.008911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981bdb5-0f0e-453e-9516-60984376d906 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.009488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981bdb5-0f0e-453e-9516-60984376d906 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.009602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981bdb5-0f0e-453e-9516-60984376d906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.033254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9981bdb5-0f0e-453e-9516-60984376d906 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:07.035276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9981bdb5-0f0e-453e-9516-60984376d906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:07.110794 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1826/3625] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:07.203511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:07.205537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:07.289534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88088ec-e9be-4823-b9cb-a4d9eecba92e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:07.298445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:07.299107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.299930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.300236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] PATH_INFO: `/v3/projects/994259bbbe124b4e9ff0542ca402b9f7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.302377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=994259bbbe124b4e9ff0542ca402b9f7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:07.304486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:07.361389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '994259bbbe124b4e9ff0542ca402b9f7', 'request_id': 'req-198a3d2d-63de-4fd8-b7e4-d064a40cd543'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:07.370751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '994259bbbe124b4e9ff0542ca402b9f7', 'request_id': 'req-198a3d2d-63de-4fd8-b7e4-d064a40cd543'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:07.378010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '994259bbbe124b4e9ff0542ca402b9f7', 'request_id': 'req-198a3d2d-63de-4fd8-b7e4-d064a40cd543'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:07.390164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] The token cache is being invalidate because project 994259bbbe124b4e9ff0542ca402b9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:05:07.390698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:07.391692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-198a3d2d-63de-4fd8-b7e4-d064a40cd543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:07.394300 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1800/3626] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:05:07 2022] DELETE /identity/v3/projects/994259bbbe124b4e9ff0542ca402b9f7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:07.399582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c22b9-9def-401f-b09b-2a5a4fe15341 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.399927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c22b9-9def-401f-b09b-2a5a4fe15341 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.400197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c22b9-9def-401f-b09b-2a5a4fe15341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.424081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7b7c22b9-9def-401f-b09b-2a5a4fe15341 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:07.426058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7b7c22b9-9def-401f-b09b-2a5a4fe15341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:07.507607 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1827/3627] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:07.517061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314afb9-69dc-4474-8e14-f87d02987615 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.517469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314afb9-69dc-4474-8e14-f87d02987615 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.517836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314afb9-69dc-4474-8e14-f87d02987615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.535415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:07.542047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a314afb9-69dc-4474-8e14-f87d02987615 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:07.544014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a314afb9-69dc-4474-8e14-f87d02987615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:07.555102 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1801/3628] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:07.558974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c22b9-9def-401f-b09b-2a5a4fe15341 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:07.571166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c22b9-9def-401f-b09b-2a5a4fe15341 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:07.577342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2cebe-45c2-4d69-8e2c-6cc5fc69d4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:07.578160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2cebe-45c2-4d69-8e2c-6cc5fc69d4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.578516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2cebe-45c2-4d69-8e2c-6cc5fc69d4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.578822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2cebe-45c2-4d69-8e2c-6cc5fc69d4a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.579500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2cebe-45c2-4d69-8e2c-6cc5fc69d4a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:05:07.580015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2cebe-45c2-4d69-8e2c-6cc5fc69d4a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:07.582693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2cebe-45c2-4d69-8e2c-6cc5fc69d4a2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:07.599731 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1828/3629] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:05:07.604961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314afb9-69dc-4474-8e14-f87d02987615 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:07.616750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314afb9-69dc-4474-8e14-f87d02987615 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:07.622901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dfa9b-8261-4b61-9742-230cbdaaaa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:07.623834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfa9b-8261-4b61-9742-230cbdaaaa95 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.624256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfa9b-8261-4b61-9742-230cbdaaaa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.624606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfa9b-8261-4b61-9742-230cbdaaaa95 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.625366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dfa9b-8261-4b61-9742-230cbdaaaa95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:07.628154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dfa9b-8261-4b61-9742-230cbdaaaa95 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:07.659768 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1802/3630] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:05:07 2022] POST /identity/v3/projects => generated 395 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:05:07.664478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2cebe-45c2-4d69-8e2c-6cc5fc69d4a2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:07.675149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2cebe-45c2-4d69-8e2c-6cc5fc69d4a2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:07.756699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e815cc9-de93-4a1d-8394-d0d1d41e6c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:07.757196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e815cc9-de93-4a1d-8394-d0d1d41e6c14 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.757611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e815cc9-de93-4a1d-8394-d0d1d41e6c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.757876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e815cc9-de93-4a1d-8394-d0d1d41e6c14 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.758505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e815cc9-de93-4a1d-8394-d0d1d41e6c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:07.760447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e815cc9-de93-4a1d-8394-d0d1d41e6c14 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:07.795090 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1829/3631] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:05:07 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:05:07.800042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dfa9b-8261-4b61-9742-230cbdaaaa95 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:07.812318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dfa9b-8261-4b61-9742-230cbdaaaa95 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:07.817782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262488f-7555-46ef-afce-39c87d8bec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:07.818383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262488f-7555-46ef-afce-39c87d8bec24 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.818652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262488f-7555-46ef-afce-39c87d8bec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.818915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262488f-7555-46ef-afce-39c87d8bec24 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.819537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262488f-7555-46ef-afce-39c87d8bec24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:07.821248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262488f-7555-46ef-afce-39c87d8bec24 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:07.833573 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1803/3632] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:05:07.837683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e815cc9-de93-4a1d-8394-d0d1d41e6c14 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:07.850312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e815cc9-de93-4a1d-8394-d0d1d41e6c14 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:07.855908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b10755-e210-41e8-962f-0859ac1ae384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:07.856581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b10755-e210-41e8-962f-0859ac1ae384 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.856919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b10755-e210-41e8-962f-0859ac1ae384 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.857283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b10755-e210-41e8-962f-0859ac1ae384 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.857982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b10755-e210-41e8-962f-0859ac1ae384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:07.859712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b10755-e210-41e8-962f-0859ac1ae384 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:07.872201 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1830/3633] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:05:07.876393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262488f-7555-46ef-afce-39c87d8bec24 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:07.887489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262488f-7555-46ef-afce-39c87d8bec24 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:07.892197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c918186-8d6d-4829-94d6-49b35ee7dcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:07.892856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c918186-8d6d-4829-94d6-49b35ee7dcdd admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.893189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c918186-8d6d-4829-94d6-49b35ee7dcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.893460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c918186-8d6d-4829-94d6-49b35ee7dcdd admin admin] PATH_INFO: `/v3/projects/99b33358601542a1801dc998487a6867/users/9471c89b05814869897c5b0bbff4835f/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.909015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c918186-8d6d-4829-94d6-49b35ee7dcdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b33358601542a1801dc998487a6867, user_id=9471c89b05814869897c5b0bbff4835f, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:07.911711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c918186-8d6d-4829-94d6-49b35ee7dcdd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:07.927327 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1804/3634] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:05:07 2022] PUT /identity/v3/projects/99b33358601542a1801dc998487a6867/users/9471c89b05814869897c5b0bbff4835f/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:07.937562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ceebf-ae8c-47ff-adf5-23987ba406d3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:07.938016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ceebf-ae8c-47ff-adf5-23987ba406d3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:07.938528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ceebf-ae8c-47ff-adf5-23987ba406d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:07.948097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:07.971547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0f2ceebf-ae8c-47ff-adf5-23987ba406d3 None None] MFA Rules not processed for user `9471c89b05814869897c5b0bbff4835f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:07.972773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0f2ceebf-ae8c-47ff-adf5-23987ba406d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:08.043643 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1831/3635] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:09.052616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c918186-8d6d-4829-94d6-49b35ee7dcdd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:09.054599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2c918186-8d6d-4829-94d6-49b35ee7dcdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:09.138556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c918186-8d6d-4829-94d6-49b35ee7dcdd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:09.145416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330ef9c-856b-4185-9efb-027a30b5a33d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:09.145906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330ef9c-856b-4185-9efb-027a30b5a33d service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:09.146178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330ef9c-856b-4185-9efb-027a30b5a33d service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:09.146439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330ef9c-856b-4185-9efb-027a30b5a33d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:09.154783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330ef9c-856b-4185-9efb-027a30b5a33d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:09.156591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330ef9c-856b-4185-9efb-027a30b5a33d service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:09.166509 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1805/3636] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:09 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:09.238677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb788db1-6789-4c58-a5d0-b692cec28626 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:09.239312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb788db1-6789-4c58-a5d0-b692cec28626 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:09.239312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb788db1-6789-4c58-a5d0-b692cec28626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:09.262492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fb788db1-6789-4c58-a5d0-b692cec28626 None None] MFA Rules not processed for user `9471c89b05814869897c5b0bbff4835f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:09.263873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fb788db1-6789-4c58-a5d0-b692cec28626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:09.272727 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1832/3637] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:09.704572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c09c-67fc-4480-a209-13a6ab80db72 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:09.704994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c09c-67fc-4480-a209-13a6ab80db72 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:09.705295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c09c-67fc-4480-a209-13a6ab80db72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:09.728482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8767c09c-67fc-4480-a209-13a6ab80db72 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:09.729557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8767c09c-67fc-4480-a209-13a6ab80db72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:09.805104 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1806/3638] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:10.119496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb788db1-6789-4c58-a5d0-b692cec28626 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:10.205166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb788db1-6789-4c58-a5d0-b692cec28626 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:10.214848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:10.215476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:10.215878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:10.216185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] PATH_INFO: `/v3/users/9471c89b05814869897c5b0bbff4835f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:10.217630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9471c89b05814869897c5b0bbff4835f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:10.219591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:10.285278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9471c89b05814869897c5b0bbff4835f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:10.301346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9471c89b05814869897c5b0bbff4835f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:10.310800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9471c89b05814869897c5b0bbff4835f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:10.320405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9471c89b05814869897c5b0bbff4835f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:10.321149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9471c89b05814869897c5b0bbff4835f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:10.327679 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1833/3639] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:05:10 2022] DELETE /identity/v3/users/9471c89b05814869897c5b0bbff4835f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:10.334018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a86283-053d-4d21-b6d1-ec33f3728a4d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:10.334563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a86283-053d-4d21-b6d1-ec33f3728a4d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:10.334563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a86283-053d-4d21-b6d1-ec33f3728a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:10.357597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a5a86283-053d-4d21-b6d1-ec33f3728a4d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:10.359171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a5a86283-053d-4d21-b6d1-ec33f3728a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:10.394539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:10.434737 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1807/3640] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:10.446496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:10.448259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:10.460417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:10.568396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afc6fb-3dcc-4331-b6d1-5ef558157488 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:10.573505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdf5dd-563b-44fa-871e-e120833b00da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:10.573827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdf5dd-563b-44fa-871e-e120833b00da service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:10.574197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdf5dd-563b-44fa-871e-e120833b00da service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:10.574511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdf5dd-563b-44fa-871e-e120833b00da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:10.590612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdf5dd-563b-44fa-871e-e120833b00da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:10.592499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdf5dd-563b-44fa-871e-e120833b00da service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:10.606687 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1834/3641] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:11.032197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a86283-053d-4d21-b6d1-ec33f3728a4d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:11.034064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a5a86283-053d-4d21-b6d1-ec33f3728a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:11.048168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a86283-053d-4d21-b6d1-ec33f3728a4d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:11.054534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:11.055548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.055833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.056242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] PATH_INFO: `/v3/projects/99b33358601542a1801dc998487a6867` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.057545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b33358601542a1801dc998487a6867)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:11.059549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:11.122222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b33358601542a1801dc998487a6867', 'request_id': 'req-02f88ed4-f382-4b41-8173-abb2f32a321a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:11.134393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b33358601542a1801dc998487a6867', 'request_id': 'req-02f88ed4-f382-4b41-8173-abb2f32a321a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:11.143912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b33358601542a1801dc998487a6867', 'request_id': 'req-02f88ed4-f382-4b41-8173-abb2f32a321a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:11.162277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] The token cache is being invalidate because project 99b33358601542a1801dc998487a6867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:05:11.162595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:11.163554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02f88ed4-f382-4b41-8173-abb2f32a321a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:11.165369 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1808/3642] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:05:11 2022] DELETE /identity/v3/projects/99b33358601542a1801dc998487a6867 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:11.170838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313d192-3898-4c8a-93d1-a80d83451abd None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.171797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313d192-3898-4c8a-93d1-a80d83451abd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.172795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313d192-3898-4c8a-93d1-a80d83451abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.209067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3313d192-3898-4c8a-93d1-a80d83451abd None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:11.210397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3313d192-3898-4c8a-93d1-a80d83451abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:11.290211 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1835/3643] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:11.297848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b11c7-21cd-4d80-9e7e-b55c0bcce216 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.298131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b11c7-21cd-4d80-9e7e-b55c0bcce216 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.298431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b11c7-21cd-4d80-9e7e-b55c0bcce216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.318426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4b2b11c7-21cd-4d80-9e7e-b55c0bcce216 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:11.319623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4b2b11c7-21cd-4d80-9e7e-b55c0bcce216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:11.328328 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1809/3644] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:11.332573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313d192-3898-4c8a-93d1-a80d83451abd None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:11.342679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313d192-3898-4c8a-93d1-a80d83451abd None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:11.347890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891e5ac-5517-4e99-84f7-a21c5ef9603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:11.348494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891e5ac-5517-4e99-84f7-a21c5ef9603e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.348769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891e5ac-5517-4e99-84f7-a21c5ef9603e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.349086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891e5ac-5517-4e99-84f7-a21c5ef9603e admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.349730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891e5ac-5517-4e99-84f7-a21c5ef9603e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:05:11.350028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891e5ac-5517-4e99-84f7-a21c5ef9603e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:11.351631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891e5ac-5517-4e99-84f7-a21c5ef9603e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:11.366887 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1836/3645] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:05:11.371640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b11c7-21cd-4d80-9e7e-b55c0bcce216 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:11.383611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b11c7-21cd-4d80-9e7e-b55c0bcce216 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:11.389014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2be0db-57fe-4ac7-a89d-fa98d0f251d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:11.390711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2be0db-57fe-4ac7-a89d-fa98d0f251d5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.391195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2be0db-57fe-4ac7-a89d-fa98d0f251d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.391616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2be0db-57fe-4ac7-a89d-fa98d0f251d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.392464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2be0db-57fe-4ac7-a89d-fa98d0f251d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:11.394871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2be0db-57fe-4ac7-a89d-fa98d0f251d5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:11.434393 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1810/3646] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:05:11 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:05:11.439041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891e5ac-5517-4e99-84f7-a21c5ef9603e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:11.454072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891e5ac-5517-4e99-84f7-a21c5ef9603e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:11.532407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0141202-6adb-4ecd-9380-14d91b901e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:11.533009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0141202-6adb-4ecd-9380-14d91b901e54 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.533320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0141202-6adb-4ecd-9380-14d91b901e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.533593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0141202-6adb-4ecd-9380-14d91b901e54 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.535080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0141202-6adb-4ecd-9380-14d91b901e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:11.536863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0141202-6adb-4ecd-9380-14d91b901e54 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:11.560047 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1837/3647] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:05:11 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:05:11.564131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2be0db-57fe-4ac7-a89d-fa98d0f251d5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:11.575097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2be0db-57fe-4ac7-a89d-fa98d0f251d5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:11.579818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ee3f2-af56-40a0-94af-24647d7f1477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:11.580365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ee3f2-af56-40a0-94af-24647d7f1477 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.583035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ee3f2-af56-40a0-94af-24647d7f1477 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.583339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ee3f2-af56-40a0-94af-24647d7f1477 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.583977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ee3f2-af56-40a0-94af-24647d7f1477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:11.585591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ee3f2-af56-40a0-94af-24647d7f1477 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:11.595996 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1811/3648] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:05:11.600688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0141202-6adb-4ecd-9380-14d91b901e54 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:11.628886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0141202-6adb-4ecd-9380-14d91b901e54 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:11.634488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed9c60-d86e-4596-955c-05f270fcb892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:11.635094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed9c60-d86e-4596-955c-05f270fcb892 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.635371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed9c60-d86e-4596-955c-05f270fcb892 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.635638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed9c60-d86e-4596-955c-05f270fcb892 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.636275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed9c60-d86e-4596-955c-05f270fcb892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:11.637922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed9c60-d86e-4596-955c-05f270fcb892 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:11.649745 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1838/3649] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:05:11.653956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ee3f2-af56-40a0-94af-24647d7f1477 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:11.664924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ee3f2-af56-40a0-94af-24647d7f1477 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:11.669585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a656b8-3a8b-414d-a71e-ef1cd95a8a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:11.670251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a656b8-3a8b-414d-a71e-ef1cd95a8a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.670527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a656b8-3a8b-414d-a71e-ef1cd95a8a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.670792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a656b8-3a8b-414d-a71e-ef1cd95a8a22 admin admin] PATH_INFO: `/v3/projects/87739141ec764f6ca347224117dc45c2/users/86129e282ee8469eba032a84e9a55096/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.688030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a656b8-3a8b-414d-a71e-ef1cd95a8a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87739141ec764f6ca347224117dc45c2, user_id=86129e282ee8469eba032a84e9a55096, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:11.691031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a656b8-3a8b-414d-a71e-ef1cd95a8a22 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:11.704720 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1812/3650] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:05:11 2022] PUT /identity/v3/projects/87739141ec764f6ca347224117dc45c2/users/86129e282ee8469eba032a84e9a55096/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:11.710419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625a8a9-a718-4bc0-8fc7-e765ed19023b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.710705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625a8a9-a718-4bc0-8fc7-e765ed19023b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.711004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625a8a9-a718-4bc0-8fc7-e765ed19023b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.731091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2625a8a9-a718-4bc0-8fc7-e765ed19023b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:11.732301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2625a8a9-a718-4bc0-8fc7-e765ed19023b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:11.743591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:11.804301 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1839/3651] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:11.820684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a656b8-3a8b-414d-a71e-ef1cd95a8a22 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:11.820684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-06a656b8-3a8b-414d-a71e-ef1cd95a8a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:11.880184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a656b8-3a8b-414d-a71e-ef1cd95a8a22 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:11.883979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385bd2d-54e7-450c-a4c2-dba1aeb6e9ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:11.884581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385bd2d-54e7-450c-a4c2-dba1aeb6e9ae service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:11.884876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385bd2d-54e7-450c-a4c2-dba1aeb6e9ae service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:11.885197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385bd2d-54e7-450c-a4c2-dba1aeb6e9ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:11.892509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:11.895785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385bd2d-54e7-450c-a4c2-dba1aeb6e9ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:11.897713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385bd2d-54e7-450c-a4c2-dba1aeb6e9ae service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:11.910540 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1813/3652] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:12.009109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757c784-6808-44db-8675-2fa535a9b780 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:12.009109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757c784-6808-44db-8675-2fa535a9b780 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:12.009507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757c784-6808-44db-8675-2fa535a9b780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:12.032304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4757c784-6808-44db-8675-2fa535a9b780 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:12.033894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4757c784-6808-44db-8675-2fa535a9b780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:12.107197 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1840/3653] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:14.824990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a0dc8-4aba-4faf-860a-d4817284f478 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:14.825493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a0dc8-4aba-4faf-860a-d4817284f478 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:14.825655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a0dc8-4aba-4faf-860a-d4817284f478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:14.879459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c87a0dc8-4aba-4faf-860a-d4817284f478 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:14.883209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c87a0dc8-4aba-4faf-860a-d4817284f478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:14.913315 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1814/3654] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:17.467865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fda6-eb1f-4b1a-aaf0-4ea4ff4679e6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:17.468856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fda6-eb1f-4b1a-aaf0-4ea4ff4679e6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:17.469374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fda6-eb1f-4b1a-aaf0-4ea4ff4679e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:17.505910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1c12fda6-eb1f-4b1a-aaf0-4ea4ff4679e6 None None] MFA Rules not processed for user `86129e282ee8469eba032a84e9a55096`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:17.507324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1c12fda6-eb1f-4b1a-aaf0-4ea4ff4679e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:17.621647 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1841/3655] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:18.061350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a0dc8-4aba-4faf-860a-d4817284f478 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:18.064486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c87a0dc8-4aba-4faf-860a-d4817284f478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:18.238422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a0dc8-4aba-4faf-860a-d4817284f478 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:18.250951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f14bb-0c57-4652-9f82-74c282bb4909 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:18.252713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f14bb-0c57-4652-9f82-74c282bb4909 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:18.254495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f14bb-0c57-4652-9f82-74c282bb4909 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:18.255093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f14bb-0c57-4652-9f82-74c282bb4909 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:18.278747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f14bb-0c57-4652-9f82-74c282bb4909 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:18.280499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f14bb-0c57-4652-9f82-74c282bb4909 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:18.313720 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1815/3656] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:20.784707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12fda6-eb1f-4b1a-aaf0-4ea4ff4679e6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:20.812396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12fda6-eb1f-4b1a-aaf0-4ea4ff4679e6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:20.844979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f67362-3157-4ad4-abf1-178b1a587355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:20.846192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f67362-3157-4ad4-abf1-178b1a587355 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:20.849631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f67362-3157-4ad4-abf1-178b1a587355 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:20.850188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f67362-3157-4ad4-abf1-178b1a587355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:20.858425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-39f67362-3157-4ad4-abf1-178b1a587355 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:20.900947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f67362-3157-4ad4-abf1-178b1a587355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:20.905483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f67362-3157-4ad4-abf1-178b1a587355 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:20.938246 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1842/3657] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:21.606514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687eb4f-8060-4038-a3a3-45424868e9e9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:21.607154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687eb4f-8060-4038-a3a3-45424868e9e9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:21.607154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687eb4f-8060-4038-a3a3-45424868e9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:21.638600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0687eb4f-8060-4038-a3a3-45424868e9e9 None None] MFA Rules not processed for user `86129e282ee8469eba032a84e9a55096`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:21.640397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0687eb4f-8060-4038-a3a3-45424868e9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:21.658804 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1816/3658] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:23.936282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f67362-3157-4ad4-abf1-178b1a587355 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:23.938719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-39f67362-3157-4ad4-abf1-178b1a587355 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:23.972035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f67362-3157-4ad4-abf1-178b1a587355 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:23.989717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:23.991324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:23.993734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:23.994239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] PATH_INFO: `/v3/users/c7deeb24e8834437a8410f8cc0c29f6d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:23.997737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7deeb24e8834437a8410f8cc0c29f6d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:24.002011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:24.087545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:24.121500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7deeb24e8834437a8410f8cc0c29f6d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:24.150410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7deeb24e8834437a8410f8cc0c29f6d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:24.163288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7deeb24e8834437a8410f8cc0c29f6d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:24.184400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7deeb24e8834437a8410f8cc0c29f6d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:24.185314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-241203f3-e984-40e5-928b-8016ce2eb47e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7deeb24e8834437a8410f8cc0c29f6d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:24.197117 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1843/3659] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:05:23 2022] DELETE /identity/v3/users/c7deeb24e8834437a8410f8cc0c29f6d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:24.475806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687eb4f-8060-4038-a3a3-45424868e9e9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:24.503337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687eb4f-8060-4038-a3a3-45424868e9e9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:24.629733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:24.630296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:24.630571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:24.630962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] PATH_INFO: `/v3/projects/b6a0bed8192d413da2cc4d413b5bc569` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:24.632103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a0bed8192d413da2cc4d413b5bc569)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:24.634145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:24.709285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a0bed8192d413da2cc4d413b5bc569', 'request_id': 'req-5bfe8516-c04b-407c-a916-5b31f45567a6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:24.717270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a0bed8192d413da2cc4d413b5bc569', 'request_id': 'req-5bfe8516-c04b-407c-a916-5b31f45567a6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:24.724434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a0bed8192d413da2cc4d413b5bc569', 'request_id': 'req-5bfe8516-c04b-407c-a916-5b31f45567a6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:24.730783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:24.735978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] The token cache is being invalidate because project b6a0bed8192d413da2cc4d413b5bc569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:05:24.736307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:24.737786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5bfe8516-c04b-407c-a916-5b31f45567a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:24.739992 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1817/3660] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:05:24 2022] DELETE /identity/v3/projects/b6a0bed8192d413da2cc4d413b5bc569 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:24.748299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fdbdf-f127-4725-b9a5-f0aaaf50b326 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:24.748577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fdbdf-f127-4725-b9a5-f0aaaf50b326 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:24.748851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fdbdf-f127-4725-b9a5-f0aaaf50b326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:24.769725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-622fdbdf-f127-4725-b9a5-f0aaaf50b326 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:24.770951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-622fdbdf-f127-4725-b9a5-f0aaaf50b326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:24.833209 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1844/3661] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:24.914865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763b95c-3ae6-49ec-a7e9-ccecf5c91273 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:24.915150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763b95c-3ae6-49ec-a7e9-ccecf5c91273 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:24.915422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763b95c-3ae6-49ec-a7e9-ccecf5c91273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:24.935495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7763b95c-3ae6-49ec-a7e9-ccecf5c91273 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:24.936708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7763b95c-3ae6-49ec-a7e9-ccecf5c91273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:24.945519 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1818/3662] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:24.949963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fdbdf-f127-4725-b9a5-f0aaaf50b326 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:24.961267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fdbdf-f127-4725-b9a5-f0aaaf50b326 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:24.966179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a49cee-553a-42c9-b812-78e5d8ccaee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:24.966771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a49cee-553a-42c9-b812-78e5d8ccaee6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:24.967045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a49cee-553a-42c9-b812-78e5d8ccaee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:24.967311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a49cee-553a-42c9-b812-78e5d8ccaee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:24.967974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a49cee-553a-42c9-b812-78e5d8ccaee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:05:24.968270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a49cee-553a-42c9-b812-78e5d8ccaee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:24.969913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a49cee-553a-42c9-b812-78e5d8ccaee6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:24.982327 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1845/3663] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:05:24.986541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763b95c-3ae6-49ec-a7e9-ccecf5c91273 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:24.996088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763b95c-3ae6-49ec-a7e9-ccecf5c91273 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:25.000806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef3e04-19f4-418d-ac47-4bb6aad0687a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:25.001522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3e04-19f4-418d-ac47-4bb6aad0687a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:25.002391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3e04-19f4-418d-ac47-4bb6aad0687a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:25.002633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3e04-19f4-418d-ac47-4bb6aad0687a admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:25.003599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef3e04-19f4-418d-ac47-4bb6aad0687a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:25.005537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef3e04-19f4-418d-ac47-4bb6aad0687a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:25.039404 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1819/3664] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:05:24 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:05:25.043703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a49cee-553a-42c9-b812-78e5d8ccaee6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:25.054715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a49cee-553a-42c9-b812-78e5d8ccaee6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:25.115864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b17b30-1d64-41a5-b701-f2e9e95089b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:25.116463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b17b30-1d64-41a5-b701-f2e9e95089b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:25.116737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b17b30-1d64-41a5-b701-f2e9e95089b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:25.117001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b17b30-1d64-41a5-b701-f2e9e95089b8 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:25.117643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b17b30-1d64-41a5-b701-f2e9e95089b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:25.119560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b17b30-1d64-41a5-b701-f2e9e95089b8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:25.144540 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1846/3665] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:05:25 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:05:25.148716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef3e04-19f4-418d-ac47-4bb6aad0687a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:25.160023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef3e04-19f4-418d-ac47-4bb6aad0687a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:25.164593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada312d-9cc4-4d7a-9a06-92ad18946e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:25.165232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada312d-9cc4-4d7a-9a06-92ad18946e64 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:25.165504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada312d-9cc4-4d7a-9a06-92ad18946e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:25.165766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada312d-9cc4-4d7a-9a06-92ad18946e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:25.166385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada312d-9cc4-4d7a-9a06-92ad18946e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:25.167956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada312d-9cc4-4d7a-9a06-92ad18946e64 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:25.179113 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1820/3666] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:05:25.183185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b17b30-1d64-41a5-b701-f2e9e95089b8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:25.195057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b17b30-1d64-41a5-b701-f2e9e95089b8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:25.200357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c9873-0c7a-4796-936f-cf96a027b2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:25.201113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c9873-0c7a-4796-936f-cf96a027b2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:25.201593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c9873-0c7a-4796-936f-cf96a027b2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:25.201867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c9873-0c7a-4796-936f-cf96a027b2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:25.202503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c9873-0c7a-4796-936f-cf96a027b2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:25.204102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c9873-0c7a-4796-936f-cf96a027b2f9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:25.216261 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1847/3667] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:05:25.220604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada312d-9cc4-4d7a-9a06-92ad18946e64 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:25.232077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada312d-9cc4-4d7a-9a06-92ad18946e64 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:25.237259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123f846-9e2b-403b-9bfc-9a4702f516bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:25.238011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123f846-9e2b-403b-9bfc-9a4702f516bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:25.238285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123f846-9e2b-403b-9bfc-9a4702f516bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:25.238547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123f846-9e2b-403b-9bfc-9a4702f516bf admin admin] PATH_INFO: `/v3/projects/b2c267ee2a9d4191a6e9de6e7bdee449/users/be3724ed7cfa41cabc1c1a5e8fb1f218/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:25.254274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123f846-9e2b-403b-9bfc-9a4702f516bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c267ee2a9d4191a6e9de6e7bdee449, user_id=be3724ed7cfa41cabc1c1a5e8fb1f218, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:25.256891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123f846-9e2b-403b-9bfc-9a4702f516bf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:25.271415 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1821/3668] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:05:25 2022] PUT /identity/v3/projects/b2c267ee2a9d4191a6e9de6e7bdee449/users/be3724ed7cfa41cabc1c1a5e8fb1f218/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:05:25.276831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d6974-44e5-4419-a373-5d98b89c623c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:25.277152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d6974-44e5-4419-a373-5d98b89c623c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:25.277425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d6974-44e5-4419-a373-5d98b89c623c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:25.296667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-798d6974-44e5-4419-a373-5d98b89c623c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:25.297909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-798d6974-44e5-4419-a373-5d98b89c623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:25.307993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:25.369514 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1848/3669] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:25.378872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123f846-9e2b-403b-9bfc-9a4702f516bf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:25.380598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7123f846-9e2b-403b-9bfc-9a4702f516bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:25.438959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123f846-9e2b-403b-9bfc-9a4702f516bf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:25.442677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c3201-865f-4ed5-ace4-d07f1dff5298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:25.443391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3201-865f-4ed5-ace4-d07f1dff5298 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:25.443522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3201-865f-4ed5-ace4-d07f1dff5298 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:25.443825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3201-865f-4ed5-ace4-d07f1dff5298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:25.453090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c3201-865f-4ed5-ace4-d07f1dff5298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:25.454500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c3201-865f-4ed5-ace4-d07f1dff5298 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:25.464214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1822/3670] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:26.182437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196ccc9-d94b-4817-92c4-3ea75d217149 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:26.182927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196ccc9-d94b-4817-92c4-3ea75d217149 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:26.183557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196ccc9-d94b-4817-92c4-3ea75d217149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:26.234598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c196ccc9-d94b-4817-92c4-3ea75d217149 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:26.235935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c196ccc9-d94b-4817-92c4-3ea75d217149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:26.328686 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1849/3671] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:28.250311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb13d-2cb1-4ad4-85a4-fceaee40bdf8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:28.251509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb13d-2cb1-4ad4-85a4-fceaee40bdf8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:28.252315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb13d-2cb1-4ad4-85a4-fceaee40bdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:28.291432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-05abb13d-2cb1-4ad4-85a4-fceaee40bdf8 None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:28.293769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-05abb13d-2cb1-4ad4-85a4-fceaee40bdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:28.309962 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1823/3672] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:31.560418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196ccc9-d94b-4817-92c4-3ea75d217149 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:31.562113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c196ccc9-d94b-4817-92c4-3ea75d217149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:31.592193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196ccc9-d94b-4817-92c4-3ea75d217149 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:31.602745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:31.603551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:31.604035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:31.605226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] PATH_INFO: `/v3/users/124511010d044f8fb07dc09e7d754303` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:31.607251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] RBAC: Authorizing `identity:delete_user(user_id=124511010d044f8fb07dc09e7d754303)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:31.613327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:31.696768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124511010d044f8fb07dc09e7d754303'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:31.715690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124511010d044f8fb07dc09e7d754303'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:31.728299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124511010d044f8fb07dc09e7d754303'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:31.738878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124511010d044f8fb07dc09e7d754303'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:31.740283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8c1ef822-d6e8-4212-b3d5-a99ab863068a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124511010d044f8fb07dc09e7d754303'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:31.748843 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1850/3673] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:05:31 2022] DELETE /identity/v3/users/124511010d044f8fb07dc09e7d754303 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:31.972955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abb13d-2cb1-4ad4-85a4-fceaee40bdf8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:31.992495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abb13d-2cb1-4ad4-85a4-fceaee40bdf8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:32.087247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:32.088054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.088520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.088790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] PATH_INFO: `/v3/projects/f20ae5c8217d412289d006ab9e01e7f4` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:32.089947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20ae5c8217d412289d006ab9e01e7f4)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:32.092541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:32.145622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:32.163782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20ae5c8217d412289d006ab9e01e7f4', 'request_id': 'req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:32.174080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20ae5c8217d412289d006ab9e01e7f4', 'request_id': 'req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:32.182986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20ae5c8217d412289d006ab9e01e7f4', 'request_id': 'req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:32.196846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] The token cache is being invalidate because project f20ae5c8217d412289d006ab9e01e7f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:05:32.197515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:32.198680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6ea018b-541b-4c52-b23f-bcc9141d6d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:05:32.207208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c5e12-c486-4872-803d-bca91fa2d2c6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.207541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c5e12-c486-4872-803d-bca91fa2d2c6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.207757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c5e12-c486-4872-803d-bca91fa2d2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:32.208056 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1824/3674] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:05:31 2022] DELETE /identity/v3/projects/f20ae5c8217d412289d006ab9e01e7f4 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:32.232601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-182c5e12-c486-4872-803d-bca91fa2d2c6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:32.233901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-182c5e12-c486-4872-803d-bca91fa2d2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:32.273553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:32.321016 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1851/3675] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:05:32.321663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b39e1-5542-4155-8ddf-95dc1c665d9b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.322062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b39e1-5542-4155-8ddf-95dc1c665d9b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.322432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b39e1-5542-4155-8ddf-95dc1c665d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:32.344212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b60b39e1-5542-4155-8ddf-95dc1c665d9b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:32.346445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b60b39e1-5542-4155-8ddf-95dc1c665d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:32.354889 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1825/3676] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:32.359235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c5e12-c486-4872-803d-bca91fa2d2c6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:32.370491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c5e12-c486-4872-803d-bca91fa2d2c6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:32.377216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62ab2f-8b61-4393-882e-8049c24957cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:32.378412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62ab2f-8b61-4393-882e-8049c24957cc admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.378694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62ab2f-8b61-4393-882e-8049c24957cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.378963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62ab2f-8b61-4393-882e-8049c24957cc admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:32.379832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62ab2f-8b61-4393-882e-8049c24957cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:05:32.379961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62ab2f-8b61-4393-882e-8049c24957cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:32.381557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62ab2f-8b61-4393-882e-8049c24957cc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:32.395920 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1852/3677] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:05:32.400723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b39e1-5542-4155-8ddf-95dc1c665d9b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:32.412754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b39e1-5542-4155-8ddf-95dc1c665d9b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:32.418947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bc76a-a358-4cde-bc66-2daf8547c9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:32.419897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bc76a-a358-4cde-bc66-2daf8547c9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.420176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bc76a-a358-4cde-bc66-2daf8547c9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.420443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bc76a-a358-4cde-bc66-2daf8547c9e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:32.421197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bc76a-a358-4cde-bc66-2daf8547c9e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:32.423099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bc76a-a358-4cde-bc66-2daf8547c9e3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:32.458820 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1826/3678] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:05:32 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:05:32.460152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62ab2f-8b61-4393-882e-8049c24957cc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:32.471974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62ab2f-8b61-4393-882e-8049c24957cc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:32.543870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389530b4-6ead-45f7-af82-ed7abd69795d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:32.544127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389530b4-6ead-45f7-af82-ed7abd69795d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.544391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389530b4-6ead-45f7-af82-ed7abd69795d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.544679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389530b4-6ead-45f7-af82-ed7abd69795d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:32.546617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389530b4-6ead-45f7-af82-ed7abd69795d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:32.547501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389530b4-6ead-45f7-af82-ed7abd69795d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:32.572723 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1853/3679] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:05:32 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:05:32.576479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bc76a-a358-4cde-bc66-2daf8547c9e3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:32.589320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bc76a-a358-4cde-bc66-2daf8547c9e3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:32.594583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6b295-a91c-4752-b4db-278b8b147847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:32.595188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6b295-a91c-4752-b4db-278b8b147847 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.595556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6b295-a91c-4752-b4db-278b8b147847 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.595806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6b295-a91c-4752-b4db-278b8b147847 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:32.596585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6b295-a91c-4752-b4db-278b8b147847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:32.598237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6b295-a91c-4752-b4db-278b8b147847 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:32.608868 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1827/3680] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:05:32.613929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389530b4-6ead-45f7-af82-ed7abd69795d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:32.625661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389530b4-6ead-45f7-af82-ed7abd69795d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:32.640411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cf834-76ce-40f4-9f46-2b7efc8af271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:32.641021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cf834-76ce-40f4-9f46-2b7efc8af271 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.641322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cf834-76ce-40f4-9f46-2b7efc8af271 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.641595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cf834-76ce-40f4-9f46-2b7efc8af271 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:32.642763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cf834-76ce-40f4-9f46-2b7efc8af271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:32.644756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cf834-76ce-40f4-9f46-2b7efc8af271 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:32.660634 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1854/3681] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:05:32.661479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6b295-a91c-4752-b4db-278b8b147847 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:32.684092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6b295-a91c-4752-b4db-278b8b147847 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:32.691541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee23f7-a488-4593-b798-ec9f52192a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:32.692204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee23f7-a488-4593-b798-ec9f52192a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.692482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee23f7-a488-4593-b798-ec9f52192a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.692761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee23f7-a488-4593-b798-ec9f52192a88 admin admin] PATH_INFO: `/v3/projects/38b9244bbab846b7baf42985cebc72c2/users/1aadd87b97c247cb95321664ba0c32a9/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:32.722171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee23f7-a488-4593-b798-ec9f52192a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b9244bbab846b7baf42985cebc72c2, user_id=1aadd87b97c247cb95321664ba0c32a9, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:32.729606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee23f7-a488-4593-b798-ec9f52192a88 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:32.748326 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1828/3682] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:05:32 2022] PUT /identity/v3/projects/38b9244bbab846b7baf42985cebc72c2/users/1aadd87b97c247cb95321664ba0c32a9/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:05:32.753472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750a96b-0d6b-4617-a645-915d89c52de7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.754142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750a96b-0d6b-4617-a645-915d89c52de7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.754590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750a96b-0d6b-4617-a645-915d89c52de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:32.786081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f750a96b-0d6b-4617-a645-915d89c52de7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:32.788099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f750a96b-0d6b-4617-a645-915d89c52de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:32.864542 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1855/3683] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:32.877203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee23f7-a488-4593-b798-ec9f52192a88 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:32.879254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8ee23f7-a488-4593-b798-ec9f52192a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:32.965928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee23f7-a488-4593-b798-ec9f52192a88 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:32.977785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7055e3-b4d5-4a57-b397-c66c6abcda2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:32.979415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7055e3-b4d5-4a57-b397-c66c6abcda2f service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:32.979856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7055e3-b4d5-4a57-b397-c66c6abcda2f service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:32.980308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7055e3-b4d5-4a57-b397-c66c6abcda2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:33.043710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7055e3-b4d5-4a57-b397-c66c6abcda2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:33.046372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7055e3-b4d5-4a57-b397-c66c6abcda2f service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:33.076611 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1829/3684] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:34.850681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87836d0-4d6b-42d0-8bc4-c0ea42f9120e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:34.851532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87836d0-4d6b-42d0-8bc4-c0ea42f9120e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:34.852024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87836d0-4d6b-42d0-8bc4-c0ea42f9120e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:34.916182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d87836d0-4d6b-42d0-8bc4-c0ea42f9120e None None] MFA Rules not processed for user `be3724ed7cfa41cabc1c1a5e8fb1f218`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:34.918386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d87836d0-4d6b-42d0-8bc4-c0ea42f9120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:34.964803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:35.034510 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1856/3685] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:35.165389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7055e3-b4d5-4a57-b397-c66c6abcda2f service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:35.169252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5d7055e3-b4d5-4a57-b397-c66c6abcda2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:35.296395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:35.329459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7055e3-b4d5-4a57-b397-c66c6abcda2f service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:35.346816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f6609-fa1f-4850-93fe-555aa14b4cb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:35.347833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f6609-fa1f-4850-93fe-555aa14b4cb7 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:35.348829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f6609-fa1f-4850-93fe-555aa14b4cb7 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:35.349217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f6609-fa1f-4850-93fe-555aa14b4cb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:35.369029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f6609-fa1f-4850-93fe-555aa14b4cb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:35.374361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f6609-fa1f-4850-93fe-555aa14b4cb7 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:35.395769 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1830/3686] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:38.328765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffa158-699e-43ba-a275-c245cdf205b6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:38.330093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffa158-699e-43ba-a275-c245cdf205b6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:38.330560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffa158-699e-43ba-a275-c245cdf205b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:38.363330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-82ffa158-699e-43ba-a275-c245cdf205b6 None None] MFA Rules not processed for user `be3724ed7cfa41cabc1c1a5e8fb1f218`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:38.366686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-82ffa158-699e-43ba-a275-c245cdf205b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:38.389514 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1857/3687] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:39.973621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56a547-44c3-4c72-a348-b3e56de41e5f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:39.974130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56a547-44c3-4c72-a348-b3e56de41e5f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:39.974535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56a547-44c3-4c72-a348-b3e56de41e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:40.034875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0b56a547-44c3-4c72-a348-b3e56de41e5f None None] MFA Rules not processed for user `1aadd87b97c247cb95321664ba0c32a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:40.037220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0b56a547-44c3-4c72-a348-b3e56de41e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:40.172016 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1831/3688] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:40.458901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffa158-699e-43ba-a275-c245cdf205b6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:40.477792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffa158-699e-43ba-a275-c245cdf205b6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:40.485101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2433eb-f29d-433b-ab58-cdd2b1be3ca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:40.485878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2433eb-f29d-433b-ab58-cdd2b1be3ca8 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:40.486224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2433eb-f29d-433b-ab58-cdd2b1be3ca8 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:40.486867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2433eb-f29d-433b-ab58-cdd2b1be3ca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:40.502298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2433eb-f29d-433b-ab58-cdd2b1be3ca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:40.504124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2433eb-f29d-433b-ab58-cdd2b1be3ca8 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:40.525365 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1858/3689] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:42.789709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3131afb-d236-4f4f-87d2-7f096e21cb34 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:42.790227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3131afb-d236-4f4f-87d2-7f096e21cb34 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:42.790360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3131afb-d236-4f4f-87d2-7f096e21cb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:42.834357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d3131afb-d236-4f4f-87d2-7f096e21cb34 None None] MFA Rules not processed for user `1aadd87b97c247cb95321664ba0c32a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:42.836621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d3131afb-d236-4f4f-87d2-7f096e21cb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:42.860742 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1832/3690] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:05:46.431767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8801f-7f33-4012-af12-aba8140e092d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:46.432341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8801f-7f33-4012-af12-aba8140e092d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:46.432544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8801f-7f33-4012-af12-aba8140e092d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:46.453270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-00d8801f-7f33-4012-af12-aba8140e092d None None] MFA Rules not processed for user `90e8754a100240a498e422de07c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:05:46.455186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-00d8801f-7f33-4012-af12-aba8140e092d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:46.515437 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1859/3691] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:05:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:05:51.750076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3131afb-d236-4f4f-87d2-7f096e21cb34 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:51.752111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d3131afb-d236-4f4f-87d2-7f096e21cb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:51.885512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3131afb-d236-4f4f-87d2-7f096e21cb34 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:51.892400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd687867-c528-4271-bbe8-266e39cb73bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:51.893295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd687867-c528-4271-bbe8-266e39cb73bf service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:51.893761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd687867-c528-4271-bbe8-266e39cb73bf service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:51.894341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd687867-c528-4271-bbe8-266e39cb73bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:51.897444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd687867-c528-4271-bbe8-266e39cb73bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:51.911076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd687867-c528-4271-bbe8-266e39cb73bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:51.912827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd687867-c528-4271-bbe8-266e39cb73bf service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:51.925991 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1833/3692] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:54.857451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8801f-7f33-4012-af12-aba8140e092d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:05:54.887437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8801f-7f33-4012-af12-aba8140e092d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:05:54.897309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6fe1c-137a-46d6-aae3-42db246dc43e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:05:54.898100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6fe1c-137a-46d6-aae3-42db246dc43e service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:05:54.898826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6fe1c-137a-46d6-aae3-42db246dc43e service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:05:54.899191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6fe1c-137a-46d6-aae3-42db246dc43e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:05:54.901658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-38c6fe1c-137a-46d6-aae3-42db246dc43e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:05:54.918303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6fe1c-137a-46d6-aae3-42db246dc43e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:05:54.920014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6fe1c-137a-46d6-aae3-42db246dc43e service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:05:54.931660 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1860/3693] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:05:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:06:01.919618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=83 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:03.716060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd687867-c528-4271-bbe8-266e39cb73bf service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:03.729071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd687867-c528-4271-bbe8-266e39cb73bf service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:03.734477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1d4e7-a907-49f6-acd9-ccdf94c8ab37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:03.735055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d4e7-a907-49f6-acd9-ccdf94c8ab37 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:03.735327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d4e7-a907-49f6-acd9-ccdf94c8ab37 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:03.735591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d4e7-a907-49f6-acd9-ccdf94c8ab37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:03.738200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-41a1d4e7-a907-49f6-acd9-ccdf94c8ab37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:03.795743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1d4e7-a907-49f6-acd9-ccdf94c8ab37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:03.797403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1d4e7-a907-49f6-acd9-ccdf94c8ab37 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:03.807218 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1834/3694] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:06:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:06:04.926441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=54 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:05.292585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6fe1c-137a-46d6-aae3-42db246dc43e service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:05.306063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6fe1c-137a-46d6-aae3-42db246dc43e service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:05.311553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164e1c1-6929-4b47-9f5e-afd6fbe6edbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:05.312144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164e1c1-6929-4b47-9f5e-afd6fbe6edbd service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:05.313336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164e1c1-6929-4b47-9f5e-afd6fbe6edbd service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:05.313336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164e1c1-6929-4b47-9f5e-afd6fbe6edbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:05.314553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9164e1c1-6929-4b47-9f5e-afd6fbe6edbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:05.379206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164e1c1-6929-4b47-9f5e-afd6fbe6edbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:05.381143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164e1c1-6929-4b47-9f5e-afd6fbe6edbd service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:05.390708 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1861/3695] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:06:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:06:08.790811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16af850-112f-458f-97cd-f34caf063190 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:08.791377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16af850-112f-458f-97cd-f34caf063190 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:08.791477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16af850-112f-458f-97cd-f34caf063190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:08.811801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c16af850-112f-458f-97cd-f34caf063190 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:06:08.813255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c16af850-112f-458f-97cd-f34caf063190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:08.880828 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1835/3696] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:06:08.937570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164e1c1-6929-4b47-9f5e-afd6fbe6edbd service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:08.939134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9164e1c1-6929-4b47-9f5e-afd6fbe6edbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:09.067127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164e1c1-6929-4b47-9f5e-afd6fbe6edbd service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:09.074302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ce06a-bd9b-4db2-8aa7-ce72a0da8e21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:09.074525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ce06a-bd9b-4db2-8aa7-ce72a0da8e21 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:09.074780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ce06a-bd9b-4db2-8aa7-ce72a0da8e21 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:09.075160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ce06a-bd9b-4db2-8aa7-ce72a0da8e21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:09.086326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ce06a-bd9b-4db2-8aa7-ce72a0da8e21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:09.088001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ce06a-bd9b-4db2-8aa7-ce72a0da8e21 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:09.100506 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1862/3697] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:06:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:06:15.629558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16af850-112f-458f-97cd-f34caf063190 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:15.664635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16af850-112f-458f-97cd-f34caf063190 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:15.673480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b319e-8481-47c4-b68e-4b781e27bca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:15.675956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b319e-8481-47c4-b68e-4b781e27bca8 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:15.676655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b319e-8481-47c4-b68e-4b781e27bca8 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:15.677197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b319e-8481-47c4-b68e-4b781e27bca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:15.679588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7e2b319e-8481-47c4-b68e-4b781e27bca8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:15.769262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b319e-8481-47c4-b68e-4b781e27bca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:15.770972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b319e-8481-47c4-b68e-4b781e27bca8 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:15.783413 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1836/3698] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:06:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:06:19.095275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=60 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:25.776363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=81 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:29.373001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ce06a-bd9b-4db2-8aa7-ce72a0da8e21 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:29.374784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-155ce06a-bd9b-4db2-8aa7-ce72a0da8e21 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:29.502709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ce06a-bd9b-4db2-8aa7-ce72a0da8e21 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:29.510648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74bee2-a7c2-48c6-97ca-ecd20047b67c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:29.511963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74bee2-a7c2-48c6-97ca-ecd20047b67c service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:29.512422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74bee2-a7c2-48c6-97ca-ecd20047b67c service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:29.512852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74bee2-a7c2-48c6-97ca-ecd20047b67c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:29.515607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cb74bee2-a7c2-48c6-97ca-ecd20047b67c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:29.574359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74bee2-a7c2-48c6-97ca-ecd20047b67c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:29.578182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74bee2-a7c2-48c6-97ca-ecd20047b67c service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:29.633223 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1863/3699] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:06:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:06:29.739651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b319e-8481-47c4-b68e-4b781e27bca8 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:29.752097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b319e-8481-47c4-b68e-4b781e27bca8 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:29.757114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2ea4d-cac0-45e9-81b4-14ba35d41b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:29.757795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2ea4d-cac0-45e9-81b4-14ba35d41b54 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:29.758190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2ea4d-cac0-45e9-81b4-14ba35d41b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:29.758595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2ea4d-cac0-45e9-81b4-14ba35d41b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:29.771247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-34b2ea4d-cac0-45e9-81b4-14ba35d41b54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:29.795014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2ea4d-cac0-45e9-81b4-14ba35d41b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:29.796898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2ea4d-cac0-45e9-81b4-14ba35d41b54 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:29.815835 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1837/3700] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:06:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:06:29.923780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74bee2-a7c2-48c6-97ca-ecd20047b67c service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:29.938297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74bee2-a7c2-48c6-97ca-ecd20047b67c service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:29.943541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed169c0-2fa3-4855-82d5-f0b2064ae411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:29.944437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed169c0-2fa3-4855-82d5-f0b2064ae411 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:29.945142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed169c0-2fa3-4855-82d5-f0b2064ae411 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:29.945690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed169c0-2fa3-4855-82d5-f0b2064ae411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:29.948196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fed169c0-2fa3-4855-82d5-f0b2064ae411 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:29.966670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed169c0-2fa3-4855-82d5-f0b2064ae411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:29.969130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed169c0-2fa3-4855-82d5-f0b2064ae411 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:29.985306 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1864/3701] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:06:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:06:30.077627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71173-af4e-4c52-9317-c54ad0b859d3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:30.078290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71173-af4e-4c52-9317-c54ad0b859d3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:30.079865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71173-af4e-4c52-9317-c54ad0b859d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:30.111968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b9d71173-af4e-4c52-9317-c54ad0b859d3 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:06:30.113997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b9d71173-af4e-4c52-9317-c54ad0b859d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:30.126334 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1838/3702] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:06:30.142496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed169c0-2fa3-4855-82d5-f0b2064ae411 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:30.161922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed169c0-2fa3-4855-82d5-f0b2064ae411 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:30.170661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0da1db-eb9b-47a4-86a0-3d535ea8eea0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:30.171355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0da1db-eb9b-47a4-86a0-3d535ea8eea0 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:30.171678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0da1db-eb9b-47a4-86a0-3d535ea8eea0 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:30.172120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0da1db-eb9b-47a4-86a0-3d535ea8eea0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:30.185798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0da1db-eb9b-47a4-86a0-3d535ea8eea0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:30.187938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0da1db-eb9b-47a4-86a0-3d535ea8eea0 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:30.211171 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1865/3703] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:06:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:06:35.543294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fcbb7-9d26-4e43-8e5e-6044bf87553d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:35.543892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fcbb7-9d26-4e43-8e5e-6044bf87553d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:35.543892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fcbb7-9d26-4e43-8e5e-6044bf87553d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:35.610518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a55fcbb7-9d26-4e43-8e5e-6044bf87553d None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:06:35.612389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a55fcbb7-9d26-4e43-8e5e-6044bf87553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:35.638711 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1839/3704] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:06:35.654486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0da1db-eb9b-47a4-86a0-3d535ea8eea0 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:35.671882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0da1db-eb9b-47a4-86a0-3d535ea8eea0 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:35.695706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1542615-f444-414d-bed3-d50804a08efe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:35.696150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542615-f444-414d-bed3-d50804a08efe service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:35.696538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542615-f444-414d-bed3-d50804a08efe service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:35.696728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542615-f444-414d-bed3-d50804a08efe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:35.730320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1542615-f444-414d-bed3-d50804a08efe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:35.734873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1542615-f444-414d-bed3-d50804a08efe service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:35.751491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1866/3705] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:06:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:06:45.596336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=12 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:45.744845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=56 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:46.794690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fcbb7-9d26-4e43-8e5e-6044bf87553d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:46.799841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a55fcbb7-9d26-4e43-8e5e-6044bf87553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:46.829762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fcbb7-9d26-4e43-8e5e-6044bf87553d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:46.916986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:46.917725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:46.917961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:46.928092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] PATH_INFO: `/v3/users/84c19569dc6d49f4a29c814ef921e144` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:46.928092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c19569dc6d49f4a29c814ef921e144)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:46.928398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:47.075047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c19569dc6d49f4a29c814ef921e144'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:47.092479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c19569dc6d49f4a29c814ef921e144'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:47.123102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c19569dc6d49f4a29c814ef921e144'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:47.145102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c19569dc6d49f4a29c814ef921e144'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:47.146017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c19569dc6d49f4a29c814ef921e144'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:47.162442 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1840/3706] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:06:46 2022] DELETE /identity/v3/users/84c19569dc6d49f4a29c814ef921e144 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:06:47.683429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1542615-f444-414d-bed3-d50804a08efe service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:47.715167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1542615-f444-414d-bed3-d50804a08efe service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:47.893593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:47.894292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:47.894573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:47.901437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] PATH_INFO: `/v3/projects/12a874b752564923ad6a4747c7a44d61` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:47.903892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a874b752564923ad6a4747c7a44d61)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:47.905896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:48.085555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a874b752564923ad6a4747c7a44d61', 'request_id': 'req-eb11a036-6b32-4850-838d-77fee51eae04'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:48.112118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a874b752564923ad6a4747c7a44d61', 'request_id': 'req-eb11a036-6b32-4850-838d-77fee51eae04'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:48.123196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a874b752564923ad6a4747c7a44d61', 'request_id': 'req-eb11a036-6b32-4850-838d-77fee51eae04'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:48.162637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] The token cache is being invalidate because project 12a874b752564923ad6a4747c7a44d61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:06:48.164475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:48.166041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:48.171932 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1867/3707] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:06:47 2022] DELETE /identity/v3/projects/12a874b752564923ad6a4747c7a44d61 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:06:48.177058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:48.178781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:48.229898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e3088-4f89-4ff5-94e2-33e299ff721a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:48.418804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:48.420092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:48.420738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:48.421552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] PATH_INFO: `/v3/users/03fc75503c7b426cba42614c38545fd3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:48.424308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03fc75503c7b426cba42614c38545fd3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:48.427304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:48.710235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03fc75503c7b426cba42614c38545fd3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:48.752803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03fc75503c7b426cba42614c38545fd3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:48.778970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03fc75503c7b426cba42614c38545fd3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:48.783344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:48.792738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03fc75503c7b426cba42614c38545fd3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:48.795155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-657e130e-f31b-4f84-af66-c96201974898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03fc75503c7b426cba42614c38545fd3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:48.810729 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1841/3708] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:06:48 2022] DELETE /identity/v3/users/03fc75503c7b426cba42614c38545fd3 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:06:49.336604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:49.338722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:49.378698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11a036-6b32-4850-838d-77fee51eae04 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:49.579187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:49.579793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:49.580877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:49.582809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] PATH_INFO: `/v3/projects/562fbd30b3b547ad84cef4c396b0b68e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:49.585498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] RBAC: Authorizing `identity:delete_project(project_id=562fbd30b3b547ad84cef4c396b0b68e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:49.597764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:49.718361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:49.741540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '562fbd30b3b547ad84cef4c396b0b68e', 'request_id': 'req-e26ff343-8fc2-4519-a279-f55b605de6be'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:49.769592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '562fbd30b3b547ad84cef4c396b0b68e', 'request_id': 'req-e26ff343-8fc2-4519-a279-f55b605de6be'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:49.790392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '562fbd30b3b547ad84cef4c396b0b68e', 'request_id': 'req-e26ff343-8fc2-4519-a279-f55b605de6be'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:49.830780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] The token cache is being invalidate because project 562fbd30b3b547ad84cef4c396b0b68e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:06:49.831143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:49.834242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e26ff343-8fc2-4519-a279-f55b605de6be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:49.837231 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1868/3709] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:06:49 2022] DELETE /identity/v3/projects/562fbd30b3b547ad84cef4c396b0b68e => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:06:49.842140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dbafa-aedc-4b6f-a190-d8f31c17d740 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:49.842711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dbafa-aedc-4b6f-a190-d8f31c17d740 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:49.842821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dbafa-aedc-4b6f-a190-d8f31c17d740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:49.934256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-816dbafa-aedc-4b6f-a190-d8f31c17d740 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:06:49.935493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-816dbafa-aedc-4b6f-a190-d8f31c17d740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:50.156788 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1842/3710] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:06:50.163616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255fc2d-91da-42be-be9c-2e352a855ae7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:50.163889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255fc2d-91da-42be-be9c-2e352a855ae7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:50.164094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255fc2d-91da-42be-be9c-2e352a855ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:50.225058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5255fc2d-91da-42be-be9c-2e352a855ae7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:06:50.226624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5255fc2d-91da-42be-be9c-2e352a855ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:50.238031 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1869/3711] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:06:50.239219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dbafa-aedc-4b6f-a190-d8f31c17d740 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:50.254789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dbafa-aedc-4b6f-a190-d8f31c17d740 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:50.261438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38500fa4-bf51-45c7-aacd-10a540ab8ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:50.262046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38500fa4-bf51-45c7-aacd-10a540ab8ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:50.262320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38500fa4-bf51-45c7-aacd-10a540ab8ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:50.262586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38500fa4-bf51-45c7-aacd-10a540ab8ea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:50.263236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38500fa4-bf51-45c7-aacd-10a540ab8ea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:06:50.264458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38500fa4-bf51-45c7-aacd-10a540ab8ea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:50.269453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38500fa4-bf51-45c7-aacd-10a540ab8ea9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:50.323491 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1843/3712] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:06:50.328312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255fc2d-91da-42be-be9c-2e352a855ae7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:50.341086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255fc2d-91da-42be-be9c-2e352a855ae7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:50.358222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b08e27-e158-4520-b9a6-8f06e86a9723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:50.359183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b08e27-e158-4520-b9a6-8f06e86a9723 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:50.359747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b08e27-e158-4520-b9a6-8f06e86a9723 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:50.360234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b08e27-e158-4520-b9a6-8f06e86a9723 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:50.363683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b08e27-e158-4520-b9a6-8f06e86a9723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:50.368751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b08e27-e158-4520-b9a6-8f06e86a9723 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:50.475840 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1870/3713] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:06:50 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:06:50.477520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38500fa4-bf51-45c7-aacd-10a540ab8ea9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:50.506931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38500fa4-bf51-45c7-aacd-10a540ab8ea9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:50.702386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7981483-784c-41ec-b0d6-871976fbe7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:50.706711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7981483-784c-41ec-b0d6-871976fbe7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:50.706996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7981483-784c-41ec-b0d6-871976fbe7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:50.710360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7981483-784c-41ec-b0d6-871976fbe7a7 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:50.712164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7981483-784c-41ec-b0d6-871976fbe7a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:50.719576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7981483-784c-41ec-b0d6-871976fbe7a7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:50.803680 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1844/3714] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:06:50 2022] POST /identity/v3/users => generated 292 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:06:50.807647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b08e27-e158-4520-b9a6-8f06e86a9723 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:50.823830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b08e27-e158-4520-b9a6-8f06e86a9723 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:50.833021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d9b70-0368-45fc-9178-b984125cead8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:50.833701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d9b70-0368-45fc-9178-b984125cead8 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:50.833941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d9b70-0368-45fc-9178-b984125cead8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:50.834204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d9b70-0368-45fc-9178-b984125cead8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:50.834853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d9b70-0368-45fc-9178-b984125cead8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:50.836482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d9b70-0368-45fc-9178-b984125cead8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:50.875357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7981483-784c-41ec-b0d6-871976fbe7a7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:50.895771 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1871/3715] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:06:50.898049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7981483-784c-41ec-b0d6-871976fbe7a7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:50.921872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec20a4-f302-4282-aa37-ac34e5a6c43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:50.922889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec20a4-f302-4282-aa37-ac34e5a6c43c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:50.923365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec20a4-f302-4282-aa37-ac34e5a6c43c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:50.923901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec20a4-f302-4282-aa37-ac34e5a6c43c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:50.924751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec20a4-f302-4282-aa37-ac34e5a6c43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:50.926668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec20a4-f302-4282-aa37-ac34e5a6c43c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:50.956444 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1845/3716] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:06:50.957808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d9b70-0368-45fc-9178-b984125cead8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:50.977796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d9b70-0368-45fc-9178-b984125cead8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:50.984889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e8e4-7e3a-4eb2-b325-3d771ef4ad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:50.986459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e8e4-7e3a-4eb2-b325-3d771ef4ad5d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:50.986837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e8e4-7e3a-4eb2-b325-3d771ef4ad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:50.987244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e8e4-7e3a-4eb2-b325-3d771ef4ad5d admin admin] PATH_INFO: `/v3/projects/96091c3dac03410dbb4f59faa1482781/users/dd615ca74887465b8e21b7f4011fb53f/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:51.022495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277e8e4-7e3a-4eb2-b325-3d771ef4ad5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=96091c3dac03410dbb4f59faa1482781, user_id=dd615ca74887465b8e21b7f4011fb53f, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:51.029607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277e8e4-7e3a-4eb2-b325-3d771ef4ad5d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:51.092936 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1872/3717] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:06:50 2022] PUT /identity/v3/projects/96091c3dac03410dbb4f59faa1482781/users/dd615ca74887465b8e21b7f4011fb53f/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:06:51.098658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090118a-8579-42f3-a2de-d167328c94b1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:51.099376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090118a-8579-42f3-a2de-d167328c94b1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:51.099980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090118a-8579-42f3-a2de-d167328c94b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:51.127273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0090118a-8579-42f3-a2de-d167328c94b1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:06:51.129454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0090118a-8579-42f3-a2de-d167328c94b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:51.204637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:51.297547 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1846/3718] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:06:51.309567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e8e4-7e3a-4eb2-b325-3d771ef4ad5d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:51.311910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0277e8e4-7e3a-4eb2-b325-3d771ef4ad5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:51.531015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e8e4-7e3a-4eb2-b325-3d771ef4ad5d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:51.537598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20067498-e128-4748-9bbc-13ba0e50748a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:51.538448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20067498-e128-4748-9bbc-13ba0e50748a service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:51.538874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20067498-e128-4748-9bbc-13ba0e50748a service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:51.539288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20067498-e128-4748-9bbc-13ba0e50748a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:51.550880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20067498-e128-4748-9bbc-13ba0e50748a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:51.552657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20067498-e128-4748-9bbc-13ba0e50748a service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:51.567267 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1873/3719] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:06:53.743954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090118a-8579-42f3-a2de-d167328c94b1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:53.771546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090118a-8579-42f3-a2de-d167328c94b1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:53.789319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2d844-d0ae-4bc0-901f-b64f46c30c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:53.792697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d844-d0ae-4bc0-901f-b64f46c30c5b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:53.793184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d844-d0ae-4bc0-901f-b64f46c30c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:53.793861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d844-d0ae-4bc0-901f-b64f46c30c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:53.796697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-27f2d844-d0ae-4bc0-901f-b64f46c30c5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:53.833554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2d844-d0ae-4bc0-901f-b64f46c30c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:53.834928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2d844-d0ae-4bc0-901f-b64f46c30c5b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:53.858003 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1847/3720] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:06:57.405370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20067498-e128-4748-9bbc-13ba0e50748a service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:57.414852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-20067498-e128-4748-9bbc-13ba0e50748a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:57.431912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20067498-e128-4748-9bbc-13ba0e50748a service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:57.439315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:57.440420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:57.440853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:57.441486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] PATH_INFO: `/v3/users/be3724ed7cfa41cabc1c1a5e8fb1f218` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:57.442907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3724ed7cfa41cabc1c1a5e8fb1f218)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:57.445925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:57.563904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3724ed7cfa41cabc1c1a5e8fb1f218'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:57.577335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:57.598786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3724ed7cfa41cabc1c1a5e8fb1f218'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:57.614298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3724ed7cfa41cabc1c1a5e8fb1f218'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:57.633953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3724ed7cfa41cabc1c1a5e8fb1f218'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:57.637744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0f75e800-31db-43cb-ae3b-886b4f97a99e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3724ed7cfa41cabc1c1a5e8fb1f218'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:57.669358 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1874/3721] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:06:57 2022] DELETE /identity/v3/users/be3724ed7cfa41cabc1c1a5e8fb1f218 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:06:58.277211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2d844-d0ae-4bc0-901f-b64f46c30c5b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:58.291871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2d844-d0ae-4bc0-901f-b64f46c30c5b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:58.433487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:58.434146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:58.434413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:58.434683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] PATH_INFO: `/v3/projects/b2c267ee2a9d4191a6e9de6e7bdee449` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:58.436334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c267ee2a9d4191a6e9de6e7bdee449)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:58.438285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:58.521497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'request_id': 'req-7bbdf1e5-d267-442e-9e99-d68e06219082'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:58.534944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'request_id': 'req-7bbdf1e5-d267-442e-9e99-d68e06219082'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:58.547990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'request_id': 'req-7bbdf1e5-d267-442e-9e99-d68e06219082'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:58.565808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] The token cache is being invalidate because project b2c267ee2a9d4191a6e9de6e7bdee449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:06:58.566312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:58.567694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7bbdf1e5-d267-442e-9e99-d68e06219082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:06:58.570544 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1848/3722] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:06:58 2022] DELETE /identity/v3/projects/b2c267ee2a9d4191a6e9de6e7bdee449 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:06:58.575095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc87bc-60c5-41d2-b512-b68ce1d6c31d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:58.575772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc87bc-60c5-41d2-b512-b68ce1d6c31d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:58.575772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc87bc-60c5-41d2-b512-b68ce1d6c31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:58.607637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-62dc87bc-60c5-41d2-b512-b68ce1d6c31d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:06:58.609823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-62dc87bc-60c5-41d2-b512-b68ce1d6c31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:58.744518 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1875/3723] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:06:58.759879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81669a-c92b-4654-b56a-2b9437983dfc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:58.760884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81669a-c92b-4654-b56a-2b9437983dfc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:58.761236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81669a-c92b-4654-b56a-2b9437983dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:58.811705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6c81669a-c92b-4654-b56a-2b9437983dfc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:06:58.815241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6c81669a-c92b-4654-b56a-2b9437983dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:58.839778 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1849/3724] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:06:58.844112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc87bc-60c5-41d2-b512-b68ce1d6c31d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:58.860591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc87bc-60c5-41d2-b512-b68ce1d6c31d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:58.875530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fbb72-d0fc-49dd-93cf-a55d5cbc043d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:58.876158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fbb72-d0fc-49dd-93cf-a55d5cbc043d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:58.876483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fbb72-d0fc-49dd-93cf-a55d5cbc043d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:58.876808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fbb72-d0fc-49dd-93cf-a55d5cbc043d admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:58.877793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fbb72-d0fc-49dd-93cf-a55d5cbc043d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:06:58.878091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fbb72-d0fc-49dd-93cf-a55d5cbc043d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:58.886222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fbb72-d0fc-49dd-93cf-a55d5cbc043d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:58.913541 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1876/3725] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:06:58.914812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81669a-c92b-4654-b56a-2b9437983dfc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:58.937773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81669a-c92b-4654-b56a-2b9437983dfc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:58.944933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24b650-c83c-4098-93eb-025999a8792b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:58.945780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24b650-c83c-4098-93eb-025999a8792b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:58.946261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24b650-c83c-4098-93eb-025999a8792b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:58.946697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24b650-c83c-4098-93eb-025999a8792b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:58.947511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24b650-c83c-4098-93eb-025999a8792b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:58.952253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24b650-c83c-4098-93eb-025999a8792b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:59.021076 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1850/3726] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:06:58 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:06:59.021346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fbb72-d0fc-49dd-93cf-a55d5cbc043d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:59.036859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fbb72-d0fc-49dd-93cf-a55d5cbc043d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:59.166027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d39a-3307-4578-b1cc-7dd08c8c00cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:59.167024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d39a-3307-4578-b1cc-7dd08c8c00cb admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:59.168237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d39a-3307-4578-b1cc-7dd08c8c00cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:59.168658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d39a-3307-4578-b1cc-7dd08c8c00cb admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:59.169712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8d39a-3307-4578-b1cc-7dd08c8c00cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:59.171937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8d39a-3307-4578-b1cc-7dd08c8c00cb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:59.215788 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1877/3727] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:06:59 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:06:59.217155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24b650-c83c-4098-93eb-025999a8792b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:59.235050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24b650-c83c-4098-93eb-025999a8792b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:59.249819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcce12-2d70-4b2f-b70b-0ff7111d60e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:59.250901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcce12-2d70-4b2f-b70b-0ff7111d60e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:59.251391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcce12-2d70-4b2f-b70b-0ff7111d60e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:59.252070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcce12-2d70-4b2f-b70b-0ff7111d60e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:59.252969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcce12-2d70-4b2f-b70b-0ff7111d60e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:59.254781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcce12-2d70-4b2f-b70b-0ff7111d60e9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:59.277093 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1851/3728] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:06:59.278322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d39a-3307-4578-b1cc-7dd08c8c00cb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:59.294959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d39a-3307-4578-b1cc-7dd08c8c00cb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:59.308120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffc25a-0e78-4cdc-af36-f292040412b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:59.308781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffc25a-0e78-4cdc-af36-f292040412b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:59.309699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffc25a-0e78-4cdc-af36-f292040412b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:59.310329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffc25a-0e78-4cdc-af36-f292040412b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:59.310749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffc25a-0e78-4cdc-af36-f292040412b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:59.312444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffc25a-0e78-4cdc-af36-f292040412b3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:59.335054 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1878/3729] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:06:59.338136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcce12-2d70-4b2f-b70b-0ff7111d60e9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:59.355285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcce12-2d70-4b2f-b70b-0ff7111d60e9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:59.374861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fd5e0-a480-4183-bc55-def39c2e1919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:59.378281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fd5e0-a480-4183-bc55-def39c2e1919 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:59.378581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fd5e0-a480-4183-bc55-def39c2e1919 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:59.378860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fd5e0-a480-4183-bc55-def39c2e1919 admin admin] PATH_INFO: `/v3/projects/5572b130042c482384a1a06e6a91c784/users/461a5c1e334a47eb9d70908227cca0b2/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:59.398743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fd5e0-a480-4183-bc55-def39c2e1919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5572b130042c482384a1a06e6a91c784, user_id=461a5c1e334a47eb9d70908227cca0b2, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:59.405711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fd5e0-a480-4183-bc55-def39c2e1919 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:59.410848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:59.431134 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1852/3730] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:06:59 2022] PUT /identity/v3/projects/5572b130042c482384a1a06e6a91c784/users/461a5c1e334a47eb9d70908227cca0b2/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:06:59.436167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5ed44-d9ba-4050-83e3-695bb8ed2c83 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:59.436448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5ed44-d9ba-4050-83e3-695bb8ed2c83 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:59.436721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5ed44-d9ba-4050-83e3-695bb8ed2c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:59.484572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d9e5ed44-d9ba-4050-83e3-695bb8ed2c83 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:06:59.487945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d9e5ed44-d9ba-4050-83e3-695bb8ed2c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:59.575810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:59.650196 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1879/3731] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:06:59.680669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fd5e0-a480-4183-bc55-def39c2e1919 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:06:59.682749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d00fd5e0-a480-4183-bc55-def39c2e1919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:06:59.833727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fd5e0-a480-4183-bc55-def39c2e1919 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:06:59.843182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a5208-e2b7-48d5-bdc8-30b982be32ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:06:59.843816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a5208-e2b7-48d5-bdc8-30b982be32ec service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:06:59.844091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a5208-e2b7-48d5-bdc8-30b982be32ec service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:06:59.844359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a5208-e2b7-48d5-bdc8-30b982be32ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:06:59.869013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a5208-e2b7-48d5-bdc8-30b982be32ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:06:59.870519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a5208-e2b7-48d5-bdc8-30b982be32ec service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:06:59.884152 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1853/3732] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:07:02.107911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5ed44-d9ba-4050-83e3-695bb8ed2c83 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:02.128176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5ed44-d9ba-4050-83e3-695bb8ed2c83 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:02.131955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf41ba-a8d3-43f2-8d81-e1a9379c726e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:02.132578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf41ba-a8d3-43f2-8d81-e1a9379c726e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:02.132856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf41ba-a8d3-43f2-8d81-e1a9379c726e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:02.133151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf41ba-a8d3-43f2-8d81-e1a9379c726e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:02.135272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ccaf41ba-a8d3-43f2-8d81-e1a9379c726e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:02.247219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf41ba-a8d3-43f2-8d81-e1a9379c726e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:02.248667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf41ba-a8d3-43f2-8d81-e1a9379c726e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:02.263547 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1880/3733] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:07:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:07:05.135452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfbe8a-3c91-4d95-8968-1c233c1205b2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:05.136800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfbe8a-3c91-4d95-8968-1c233c1205b2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:05.137304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfbe8a-3c91-4d95-8968-1c233c1205b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:05.169739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-abcfbe8a-3c91-4d95-8968-1c233c1205b2 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:05.171158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-abcfbe8a-3c91-4d95-8968-1c233c1205b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:05.302500 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1854/3734] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:07:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:05.307485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf41ba-a8d3-43f2-8d81-e1a9379c726e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:05.309370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ccaf41ba-a8d3-43f2-8d81-e1a9379c726e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:05.398957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf41ba-a8d3-43f2-8d81-e1a9379c726e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:05.405299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ad6fe-1df0-4a3b-b36c-6bd52ead5742 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:05.406153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad6fe-1df0-4a3b-b36c-6bd52ead5742 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:05.406676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad6fe-1df0-4a3b-b36c-6bd52ead5742 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:05.407336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad6fe-1df0-4a3b-b36c-6bd52ead5742 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:05.419121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ad6fe-1df0-4a3b-b36c-6bd52ead5742 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:05.421932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ad6fe-1df0-4a3b-b36c-6bd52ead5742 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:05.436017 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1881/3735] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:07:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:05.871204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca7d8b-e2fc-4f53-9285-93abe8c2f160 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:05.871789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca7d8b-e2fc-4f53-9285-93abe8c2f160 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:05.872252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca7d8b-e2fc-4f53-9285-93abe8c2f160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:05.994096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-89ca7d8b-e2fc-4f53-9285-93abe8c2f160 None None] MFA Rules not processed for user `dd615ca74887465b8e21b7f4011fb53f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:05.994096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-89ca7d8b-e2fc-4f53-9285-93abe8c2f160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:06.141215 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1855/3736] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:06.158857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861b473-58ee-47bf-8f91-3d675c9b27b2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:06.158857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861b473-58ee-47bf-8f91-3d675c9b27b2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:06.158857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861b473-58ee-47bf-8f91-3d675c9b27b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:06.199426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3861b473-58ee-47bf-8f91-3d675c9b27b2 None None] MFA Rules not processed for user `dd615ca74887465b8e21b7f4011fb53f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:06.201254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3861b473-58ee-47bf-8f91-3d675c9b27b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:06.224493 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1882/3737] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:06.430207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe9180-740d-47f3-b4aa-aaaa91a0dab9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:06.430491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe9180-740d-47f3-b4aa-aaaa91a0dab9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:06.430765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe9180-740d-47f3-b4aa-aaaa91a0dab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:06.481920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d9fe9180-740d-47f3-b4aa-aaaa91a0dab9 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:06.488039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d9fe9180-740d-47f3-b4aa-aaaa91a0dab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:06.518538 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1856/3738] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:06.519318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861b473-58ee-47bf-8f91-3d675c9b27b2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:06.537558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861b473-58ee-47bf-8f91-3d675c9b27b2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:06.543884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab594b-86ed-4394-9c73-ea6fed491275 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:06.544697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab594b-86ed-4394-9c73-ea6fed491275 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:06.545149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab594b-86ed-4394-9c73-ea6fed491275 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:06.545557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab594b-86ed-4394-9c73-ea6fed491275 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:06.557813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab594b-86ed-4394-9c73-ea6fed491275 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:06.559604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab594b-86ed-4394-9c73-ea6fed491275 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:06.582592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe9180-740d-47f3-b4aa-aaaa91a0dab9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:06.584680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d9fe9180-740d-47f3-b4aa-aaaa91a0dab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:06.597631 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1883/3739] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:07:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:06.619742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe9180-740d-47f3-b4aa-aaaa91a0dab9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:06.635669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37b934-4fac-405f-8d1c-30cd69630959 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:06.636130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37b934-4fac-405f-8d1c-30cd69630959 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:06.636407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37b934-4fac-405f-8d1c-30cd69630959 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:06.636677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37b934-4fac-405f-8d1c-30cd69630959 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:06.656598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37b934-4fac-405f-8d1c-30cd69630959 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:06.671887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37b934-4fac-405f-8d1c-30cd69630959 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:06.710156 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1857/3740] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:07:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:07:11.705624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ffbb6-b954-47ee-aeef-86a6f1acc20d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:11.706110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ffbb6-b954-47ee-aeef-86a6f1acc20d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:11.706219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ffbb6-b954-47ee-aeef-86a6f1acc20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:11.799402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f48ffbb6-b954-47ee-aeef-86a6f1acc20d None None] MFA Rules not processed for user `461a5c1e334a47eb9d70908227cca0b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:11.800712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f48ffbb6-b954-47ee-aeef-86a6f1acc20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:11.909454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:12.050825 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1884/3741] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:12.052963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88d9f2-2a39-49aa-b494-a8863a1727eb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:12.053945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88d9f2-2a39-49aa-b494-a8863a1727eb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:12.054362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88d9f2-2a39-49aa-b494-a8863a1727eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:12.095930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3f88d9f2-2a39-49aa-b494-a8863a1727eb None None] MFA Rules not processed for user `461a5c1e334a47eb9d70908227cca0b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:12.097546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3f88d9f2-2a39-49aa-b494-a8863a1727eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:12.127818 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1858/3742] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:16.916322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ffbb6-b954-47ee-aeef-86a6f1acc20d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:16.963125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ffbb6-b954-47ee-aeef-86a6f1acc20d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:16.988411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596a620-6b9f-420c-a31e-4fd1836da46b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:16.989067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596a620-6b9f-420c-a31e-4fd1836da46b service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:16.989381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596a620-6b9f-420c-a31e-4fd1836da46b service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:16.989652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596a620-6b9f-420c-a31e-4fd1836da46b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:17.032916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a596a620-6b9f-420c-a31e-4fd1836da46b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:17.034602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a596a620-6b9f-420c-a31e-4fd1836da46b service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:17.083898 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1885/3743] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:07:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:07:17.988602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88d9f2-2a39-49aa-b494-a8863a1727eb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:17.991723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3f88d9f2-2a39-49aa-b494-a8863a1727eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:18.010816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:18.078554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596a620-6b9f-420c-a31e-4fd1836da46b service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:18.080826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a596a620-6b9f-420c-a31e-4fd1836da46b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:18.112909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596a620-6b9f-420c-a31e-4fd1836da46b service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:18.127236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:18.128374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:18.128885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:18.130215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] PATH_INFO: `/v3/users/f00da962dc434ae18a1ab29228846688` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:18.135421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00da962dc434ae18a1ab29228846688)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:18.138811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:18.168643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88d9f2-2a39-49aa-b494-a8863a1727eb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:18.196919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2feb488-456a-4ecb-b00f-19c09b0d5ba0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:18.198118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feb488-456a-4ecb-b00f-19c09b0d5ba0 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:18.199046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feb488-456a-4ecb-b00f-19c09b0d5ba0 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:18.199764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feb488-456a-4ecb-b00f-19c09b0d5ba0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:18.202872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f2feb488-456a-4ecb-b00f-19c09b0d5ba0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:18.232468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2feb488-456a-4ecb-b00f-19c09b0d5ba0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:18.234455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2feb488-456a-4ecb-b00f-19c09b0d5ba0 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:18.240155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00da962dc434ae18a1ab29228846688'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:18.247493 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1859/3744] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:07:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:07:18.280629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00da962dc434ae18a1ab29228846688'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:18.293766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00da962dc434ae18a1ab29228846688'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:18.305644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00da962dc434ae18a1ab29228846688'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:18.306264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00da962dc434ae18a1ab29228846688'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:18.314345 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1886/3745] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:07:18 2022] DELETE /identity/v3/users/f00da962dc434ae18a1ab29228846688 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:18.589205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2feb488-456a-4ecb-b00f-19c09b0d5ba0 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:18.607077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2feb488-456a-4ecb-b00f-19c09b0d5ba0 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:18.756320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:18.757504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:18.758114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:18.758738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] PATH_INFO: `/v3/projects/90ca0b4db2b64900aaa1a4e05dec8a6f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:18.771190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ca0b4db2b64900aaa1a4e05dec8a6f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:18.782226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:18.898647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'request_id': 'req-12436adb-1ba5-4276-b648-02b038e0d327'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:18.912927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'request_id': 'req-12436adb-1ba5-4276-b648-02b038e0d327'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:18.924656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'request_id': 'req-12436adb-1ba5-4276-b648-02b038e0d327'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:18.951507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] The token cache is being invalidate because project 90ca0b4db2b64900aaa1a4e05dec8a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:07:18.951742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:18.952792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:18.958994 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1860/3746] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:07:18 2022] DELETE /identity/v3/projects/90ca0b4db2b64900aaa1a4e05dec8a6f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:18.959963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:18.961987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:19.011995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dd300-e7e2-4ee9-a96f-960c46f7158c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:19.153986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:19.155877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:19.158934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:19.159383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] PATH_INFO: `/v3/users/3d212845339449069a8ecd805d1d8eef` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:19.162234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d212845339449069a8ecd805d1d8eef)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:19.167252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:19.231232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:19.286855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d212845339449069a8ecd805d1d8eef'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:19.318305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d212845339449069a8ecd805d1d8eef'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:19.332695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d212845339449069a8ecd805d1d8eef'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:19.353060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d212845339449069a8ecd805d1d8eef'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:19.355827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d212845339449069a8ecd805d1d8eef'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:19.372372 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1887/3747] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:07:18 2022] DELETE /identity/v3/users/3d212845339449069a8ecd805d1d8eef => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:19.970907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:19.972984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:19.986165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12436adb-1ba5-4276-b648-02b038e0d327 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:20.013604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:20.021652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:20.023964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:20.064219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:20.065267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.065768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.066250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] PATH_INFO: `/v3/projects/d3c58c35393d47688321147f21e69d4a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.067744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c58c35393d47688321147f21e69d4a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:20.070972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:20.150115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c58c35393d47688321147f21e69d4a', 'request_id': 'req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.150806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04906b1c-6e9d-449f-a10e-3759749cdb3b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:20.160604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c58c35393d47688321147f21e69d4a', 'request_id': 'req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.169446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c58c35393d47688321147f21e69d4a', 'request_id': 'req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.183121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] The token cache is being invalidate because project d3c58c35393d47688321147f21e69d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:07:20.183383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.184472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f07dcfb-ebd0-4a5f-8223-7fc8fbdfc9ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.186888 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1861/3748] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:07:19 2022] DELETE /identity/v3/projects/d3c58c35393d47688321147f21e69d4a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:20.193285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046a1f6-9287-4127-bb3f-9c013b06d4b3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.193841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046a1f6-9287-4127-bb3f-9c013b06d4b3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.194218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046a1f6-9287-4127-bb3f-9c013b06d4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.219663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4046a1f6-9287-4127-bb3f-9c013b06d4b3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:20.220943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4046a1f6-9287-4127-bb3f-9c013b06d4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:20.230170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:20.230919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.231265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.231604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] PATH_INFO: `/v3/users/86129e282ee8469eba032a84e9a55096` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.233826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86129e282ee8469eba032a84e9a55096)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:20.235934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:20.301190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86129e282ee8469eba032a84e9a55096'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.304924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:20.309838 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1862/3749] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:20.324609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ba50f-efd9-4db9-a5ef-b1b8189a3134 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.325165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ba50f-efd9-4db9-a5ef-b1b8189a3134 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.325557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86129e282ee8469eba032a84e9a55096'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.325866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ba50f-efd9-4db9-a5ef-b1b8189a3134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.352056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86129e282ee8469eba032a84e9a55096'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.381895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86129e282ee8469eba032a84e9a55096'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.383050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86129e282ee8469eba032a84e9a55096'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.386042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ba5ba50f-efd9-4db9-a5ef-b1b8189a3134 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:20.387291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ba5ba50f-efd9-4db9-a5ef-b1b8189a3134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:20.394528 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1888/3750] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:07:20 2022] DELETE /identity/v3/users/86129e282ee8469eba032a84e9a55096 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:20.400423 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1863/3751] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:20.404390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:20.491368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f68afa-a6f5-4f9b-a08a-c65a90c5d7b7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:20.497062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245b532-f612-4b98-8252-b0b16679cec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:20.497680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245b532-f612-4b98-8252-b0b16679cec0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.498549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245b532-f612-4b98-8252-b0b16679cec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.498662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245b532-f612-4b98-8252-b0b16679cec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.499885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245b532-f612-4b98-8252-b0b16679cec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:07:20.500077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245b532-f612-4b98-8252-b0b16679cec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:20.502329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245b532-f612-4b98-8252-b0b16679cec0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:20.521591 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1889/3752] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:07:20.523155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ba50f-efd9-4db9-a5ef-b1b8189a3134 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:20.534186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ba50f-efd9-4db9-a5ef-b1b8189a3134 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:20.540295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768774f-b455-4d0f-bd27-91f1be363d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:20.541358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768774f-b455-4d0f-bd27-91f1be363d0d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.541838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768774f-b455-4d0f-bd27-91f1be363d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.542201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768774f-b455-4d0f-bd27-91f1be363d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.543046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768774f-b455-4d0f-bd27-91f1be363d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:20.545230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768774f-b455-4d0f-bd27-91f1be363d0d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:20.581560 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1864/3753] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:07:20 2022] POST /identity/v3/projects => generated 395 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:07:20.586125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245b532-f612-4b98-8252-b0b16679cec0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:20.597121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245b532-f612-4b98-8252-b0b16679cec0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:20.601610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768774f-b455-4d0f-bd27-91f1be363d0d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:20.603701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c768774f-b455-4d0f-bd27-91f1be363d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:20.657327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:20.689378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca6cab-edc5-4999-9ab1-ec77bbc5ce66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:20.689984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca6cab-edc5-4999-9ab1-ec77bbc5ce66 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.690264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca6cab-edc5-4999-9ab1-ec77bbc5ce66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.690651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca6cab-edc5-4999-9ab1-ec77bbc5ce66 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.691276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca6cab-edc5-4999-9ab1-ec77bbc5ce66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:20.693507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca6cab-edc5-4999-9ab1-ec77bbc5ce66 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:20.709892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768774f-b455-4d0f-bd27-91f1be363d0d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:20.715142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:20.715774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.716043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.716632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] PATH_INFO: `/v3/projects/87739141ec764f6ca347224117dc45c2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.718265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=87739141ec764f6ca347224117dc45c2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:20.721665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:20.723772 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1890/3754] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:07:20 2022] POST /identity/v3/users => generated 298 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:07:20.727639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca6cab-edc5-4999-9ab1-ec77bbc5ce66 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:20.748199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca6cab-edc5-4999-9ab1-ec77bbc5ce66 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:20.758238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f3a78-0a8a-4d92-976e-361340997ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:20.758888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f3a78-0a8a-4d92-976e-361340997ddb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.759161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f3a78-0a8a-4d92-976e-361340997ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.759431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f3a78-0a8a-4d92-976e-361340997ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.762304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f3a78-0a8a-4d92-976e-361340997ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:20.770089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f3a78-0a8a-4d92-976e-361340997ddb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:20.795007 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1891/3755] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:07:20.796068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87739141ec764f6ca347224117dc45c2', 'request_id': 'req-abf7ae51-b43c-415c-80f2-bf25b0978dcb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.796191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f3a78-0a8a-4d92-976e-361340997ddb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:20.807102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87739141ec764f6ca347224117dc45c2', 'request_id': 'req-abf7ae51-b43c-415c-80f2-bf25b0978dcb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.811375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f3a78-0a8a-4d92-976e-361340997ddb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:20.814724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87739141ec764f6ca347224117dc45c2', 'request_id': 'req-abf7ae51-b43c-415c-80f2-bf25b0978dcb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.827668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] The token cache is being invalidate because project 87739141ec764f6ca347224117dc45c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:07:20.827954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.829152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-abf7ae51-b43c-415c-80f2-bf25b0978dcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:20.831004 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1865/3756] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:07:20 2022] DELETE /identity/v3/projects/87739141ec764f6ca347224117dc45c2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:07:20.837294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fae8eb-28fa-4496-97bc-536f32b217a7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.837648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fae8eb-28fa-4496-97bc-536f32b217a7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.837872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fae8eb-28fa-4496-97bc-536f32b217a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.862878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:20.864429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-80fae8eb-28fa-4496-97bc-536f32b217a7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:20.866173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80fae8eb-28fa-4496-97bc-536f32b217a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:20.898040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08642cbd-755d-4041-80d8-93e1029801c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:20.899040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08642cbd-755d-4041-80d8-93e1029801c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.899328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08642cbd-755d-4041-80d8-93e1029801c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.899600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08642cbd-755d-4041-80d8-93e1029801c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.900582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08642cbd-755d-4041-80d8-93e1029801c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:20.902475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08642cbd-755d-4041-80d8-93e1029801c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:20.921229 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1892/3757] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:07:20.925409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08642cbd-755d-4041-80d8-93e1029801c6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:20.928411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-08642cbd-755d-4041-80d8-93e1029801c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:20.940861 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1866/3758] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:07:20.945173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08642cbd-755d-4041-80d8-93e1029801c6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:20.948412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64eae4-c17f-4fba-b37f-58b67c28fb44 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.948687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64eae4-c17f-4fba-b37f-58b67c28fb44 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.948953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64eae4-c17f-4fba-b37f-58b67c28fb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.951295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3608b-0706-45a3-9d36-f068b6374935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:20.952130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3608b-0706-45a3-9d36-f068b6374935 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:20.952569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3608b-0706-45a3-9d36-f068b6374935 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:20.953170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3608b-0706-45a3-9d36-f068b6374935 admin admin] PATH_INFO: `/v3/projects/8fbc8a431f1249c3a7fa3b33e2990c39/users/c5d4c5f85b7e4a09b56a423e42274a79/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:20.975845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0d64eae4-c17f-4fba-b37f-58b67c28fb44 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:20.977066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0d64eae4-c17f-4fba-b37f-58b67c28fb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:20.978706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3608b-0706-45a3-9d36-f068b6374935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fbc8a431f1249c3a7fa3b33e2990c39, user_id=c5d4c5f85b7e4a09b56a423e42274a79, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:20.981383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3608b-0706-45a3-9d36-f068b6374935 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:20.985781 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1867/3759] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:07:20.990414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64eae4-c17f-4fba-b37f-58b67c28fb44 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:20.997026 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1893/3760] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:07:20 2022] PUT /identity/v3/projects/8fbc8a431f1249c3a7fa3b33e2990c39/users/c5d4c5f85b7e4a09b56a423e42274a79/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:21.007961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64eae4-c17f-4fba-b37f-58b67c28fb44 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:21.008325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0e27e-77cf-47e1-9998-1b2a7185aeb5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.008601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0e27e-77cf-47e1-9998-1b2a7185aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.008865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0e27e-77cf-47e1-9998-1b2a7185aeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.039988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9fe0e27e-77cf-47e1-9998-1b2a7185aeb5 None None] MFA Rules not processed for user `c5d4c5f85b7e4a09b56a423e42274a79`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:21.041237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9fe0e27e-77cf-47e1-9998-1b2a7185aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:21.065111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:21.077118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565a85e-3b2b-482f-8cdf-9965e1a7d983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:21.077713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565a85e-3b2b-482f-8cdf-9965e1a7d983 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.077992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565a85e-3b2b-482f-8cdf-9965e1a7d983 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.078274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565a85e-3b2b-482f-8cdf-9965e1a7d983 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.079522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565a85e-3b2b-482f-8cdf-9965e1a7d983 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:07:21.080120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565a85e-3b2b-482f-8cdf-9965e1a7d983 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:21.081964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565a85e-3b2b-482f-8cdf-9965e1a7d983 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:21.094840 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1868/3761] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:07:21.098878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565a85e-3b2b-482f-8cdf-9965e1a7d983 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:21.112396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565a85e-3b2b-482f-8cdf-9965e1a7d983 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:21.119706 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1894/3762] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:21 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:07:21.120277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ebf6e-f61d-4d1b-af5b-d14e52776ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:21.120893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ebf6e-f61d-4d1b-af5b-d14e52776ccc admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.121205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ebf6e-f61d-4d1b-af5b-d14e52776ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.121477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ebf6e-f61d-4d1b-af5b-d14e52776ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.122706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ebf6e-f61d-4d1b-af5b-d14e52776ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:21.125343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ebf6e-f61d-4d1b-af5b-d14e52776ccc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:21.131931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01f8d1-3fce-4601-99fe-6ee1774b4790 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.132259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01f8d1-3fce-4601-99fe-6ee1774b4790 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.132784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01f8d1-3fce-4601-99fe-6ee1774b4790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.158738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ad01f8d1-3fce-4601-99fe-6ee1774b4790 None None] MFA Rules not processed for user `c5d4c5f85b7e4a09b56a423e42274a79`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:21.161649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ad01f8d1-3fce-4601-99fe-6ee1774b4790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:21.170038 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1869/3763] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:07:21 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:07:21.174641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ebf6e-f61d-4d1b-af5b-d14e52776ccc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:21.175537 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1895/3764] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:21 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:07:21.251860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ebf6e-f61d-4d1b-af5b-d14e52776ccc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:21.257101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb356f8a-01bb-47cd-a53f-d2162a1b5f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:21.257698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb356f8a-01bb-47cd-a53f-d2162a1b5f23 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.257973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb356f8a-01bb-47cd-a53f-d2162a1b5f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.258245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb356f8a-01bb-47cd-a53f-d2162a1b5f23 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.259111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb356f8a-01bb-47cd-a53f-d2162a1b5f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:21.261055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb356f8a-01bb-47cd-a53f-d2162a1b5f23 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:21.284600 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1870/3765] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:07:21 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:07:21.288878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01f8d1-3fce-4601-99fe-6ee1774b4790 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:21.302963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01f8d1-3fce-4601-99fe-6ee1774b4790 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:21.311792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd54ad-3a6e-477d-9969-5359370a7828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:21.312414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd54ad-3a6e-477d-9969-5359370a7828 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.312981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd54ad-3a6e-477d-9969-5359370a7828 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.313336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd54ad-3a6e-477d-9969-5359370a7828 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.314119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd54ad-3a6e-477d-9969-5359370a7828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:21.315799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd54ad-3a6e-477d-9969-5359370a7828 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:21.327897 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1896/3766] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:07:21.332644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb356f8a-01bb-47cd-a53f-d2162a1b5f23 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:21.361780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb356f8a-01bb-47cd-a53f-d2162a1b5f23 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:21.367123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50557327-caba-4f78-9139-4bd9f3e5ea58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:21.367823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50557327-caba-4f78-9139-4bd9f3e5ea58 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.368077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50557327-caba-4f78-9139-4bd9f3e5ea58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.368341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50557327-caba-4f78-9139-4bd9f3e5ea58 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.368970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50557327-caba-4f78-9139-4bd9f3e5ea58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:21.370762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50557327-caba-4f78-9139-4bd9f3e5ea58 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:21.381990 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1871/3767] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:07:21.386223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd54ad-3a6e-477d-9969-5359370a7828 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:21.397656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd54ad-3a6e-477d-9969-5359370a7828 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:21.402829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf2c6e-e9b2-4ab3-b52f-2cc923737839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:21.403568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf2c6e-e9b2-4ab3-b52f-2cc923737839 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.403849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf2c6e-e9b2-4ab3-b52f-2cc923737839 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.404118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf2c6e-e9b2-4ab3-b52f-2cc923737839 admin admin] PATH_INFO: `/v3/projects/8c7c33c41d0a4f96bd720b6235d0acad/users/508ea6505b99421d8d65a42f1b6b3876/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.424103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf2c6e-e9b2-4ab3-b52f-2cc923737839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7c33c41d0a4f96bd720b6235d0acad, user_id=508ea6505b99421d8d65a42f1b6b3876, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:21.426871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf2c6e-e9b2-4ab3-b52f-2cc923737839 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:21.445076 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1897/3768] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:07:21 2022] PUT /identity/v3/projects/8c7c33c41d0a4f96bd720b6235d0acad/users/508ea6505b99421d8d65a42f1b6b3876/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:21.453332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81175185-98cc-415b-b08e-727f2b73b484 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.454524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81175185-98cc-415b-b08e-727f2b73b484 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.454798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81175185-98cc-415b-b08e-727f2b73b484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.503473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-81175185-98cc-415b-b08e-727f2b73b484 None None] MFA Rules not processed for user `508ea6505b99421d8d65a42f1b6b3876`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:21.504758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-81175185-98cc-415b-b08e-727f2b73b484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:21.580763 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1872/3769] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:07:21.593230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585433a3-9393-49ff-8acf-deabff87d54d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.593831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585433a3-9393-49ff-8acf-deabff87d54d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.596157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585433a3-9393-49ff-8acf-deabff87d54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.618994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-585433a3-9393-49ff-8acf-deabff87d54d None None] MFA Rules not processed for user `508ea6505b99421d8d65a42f1b6b3876`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:21.621449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-585433a3-9393-49ff-8acf-deabff87d54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:21.630711 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1898/3770] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:21.708275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81175185-98cc-415b-b08e-727f2b73b484 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:21.709965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-81175185-98cc-415b-b08e-727f2b73b484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:21.747291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:21.789313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81175185-98cc-415b-b08e-727f2b73b484 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:21.794733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566a97c-42d8-479d-b9c5-801d2b253d84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:21.795329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566a97c-42d8-479d-b9c5-801d2b253d84 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.795598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566a97c-42d8-479d-b9c5-801d2b253d84 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.795859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566a97c-42d8-479d-b9c5-801d2b253d84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.852757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566a97c-42d8-479d-b9c5-801d2b253d84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:21.854396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566a97c-42d8-479d-b9c5-801d2b253d84 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:21.864486 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1873/3771] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:07:21 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:07:21.899389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1d79-9f18-46c5-9d21-4ce3f438e797 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:21.899672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1d79-9f18-46c5-9d21-4ce3f438e797 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:21.899953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1d79-9f18-46c5-9d21-4ce3f438e797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:21.933417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e8fe1d79-9f18-46c5-9d21-4ce3f438e797 None None] MFA Rules not processed for user `c5d4c5f85b7e4a09b56a423e42274a79`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:21.934673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e8fe1d79-9f18-46c5-9d21-4ce3f438e797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:21.942395 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1899/3772] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:21 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:22.161359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566a97c-42d8-479d-b9c5-801d2b253d84 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:22.171016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566a97c-42d8-479d-b9c5-801d2b253d84 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:22.177417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f002721-a4a0-416b-a6b7-0f0709c79bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:22.178157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f002721-a4a0-416b-a6b7-0f0709c79bd2 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:22.178493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f002721-a4a0-416b-a6b7-0f0709c79bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:22.178788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f002721-a4a0-416b-a6b7-0f0709c79bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:22.187263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f002721-a4a0-416b-a6b7-0f0709c79bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:22.188845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f002721-a4a0-416b-a6b7-0f0709c79bd2 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:22.200020 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1874/3773] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:07:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:07:22.803839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a6c7e-a4cf-4b16-8bb3-b048c3f2400b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:22.804366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a6c7e-a4cf-4b16-8bb3-b048c3f2400b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:22.804366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a6c7e-a4cf-4b16-8bb3-b048c3f2400b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:22.825872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a00a6c7e-a4cf-4b16-8bb3-b048c3f2400b None None] MFA Rules not processed for user `c5d4c5f85b7e4a09b56a423e42274a79`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:22.827097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a00a6c7e-a4cf-4b16-8bb3-b048c3f2400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:22.836850 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1900/3774] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:22 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:23.183435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99dc0c-4600-4852-ab4b-b59283f56222 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:23.184859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99dc0c-4600-4852-ab4b-b59283f56222 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:23.185246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99dc0c-4600-4852-ab4b-b59283f56222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:23.218430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8c99dc0c-4600-4852-ab4b-b59283f56222 None None] MFA Rules not processed for user `508ea6505b99421d8d65a42f1b6b3876`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:23.220409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8c99dc0c-4600-4852-ab4b-b59283f56222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:23.242843 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1875/3775] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:23.353913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4851c69-7ffe-414b-b0bc-d8d2a5afce3b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:23.354189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4851c69-7ffe-414b-b0bc-d8d2a5afce3b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:23.354452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4851c69-7ffe-414b-b0bc-d8d2a5afce3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:23.381925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b4851c69-7ffe-414b-b0bc-d8d2a5afce3b None None] MFA Rules not processed for user `c5d4c5f85b7e4a09b56a423e42274a79`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:23.383407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b4851c69-7ffe-414b-b0bc-d8d2a5afce3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:23.393725 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1901/3776] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:23.874774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffbc26-d8dd-47df-b515-c9ee06b1429b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:23.875053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffbc26-d8dd-47df-b515-c9ee06b1429b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:23.875316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffbc26-d8dd-47df-b515-c9ee06b1429b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:23.900098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-79ffbc26-d8dd-47df-b515-c9ee06b1429b None None] MFA Rules not processed for user `c5d4c5f85b7e4a09b56a423e42274a79`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:23.901665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-79ffbc26-d8dd-47df-b515-c9ee06b1429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:23.914832 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1876/3777] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:24.392583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4851c69-7ffe-414b-b0bc-d8d2a5afce3b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:24.426992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:24.458359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4851c69-7ffe-414b-b0bc-d8d2a5afce3b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:24.462804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:24.463419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:24.463695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:24.463958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] PATH_INFO: `/v3/users/c5d4c5f85b7e4a09b56a423e42274a79` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:24.465083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d4c5f85b7e4a09b56a423e42274a79)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:24.466989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:24.518776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d4c5f85b7e4a09b56a423e42274a79'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:24.531381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d4c5f85b7e4a09b56a423e42274a79'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:24.540190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d4c5f85b7e4a09b56a423e42274a79'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:24.550768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d4c5f85b7e4a09b56a423e42274a79'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:24.551791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d4c5f85b7e4a09b56a423e42274a79'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:24.559627 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1902/3778] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:07:24 2022] DELETE /identity/v3/users/c5d4c5f85b7e4a09b56a423e42274a79 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:24.564086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924c98e-5249-4db0-a531-8544154e02fa None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:24.564600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924c98e-5249-4db0-a531-8544154e02fa None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:24.564857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924c98e-5249-4db0-a531-8544154e02fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:24.589113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3924c98e-5249-4db0-a531-8544154e02fa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:24.590772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3924c98e-5249-4db0-a531-8544154e02fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:24.663588 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1877/3779] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:07:24.674580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:24.676377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:24.763598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49ca98-ff66-4fec-815b-40c564159f51 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:24.770302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf082459-5749-460c-8f91-995486a939f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:24.770916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf082459-5749-460c-8f91-995486a939f3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:24.771186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf082459-5749-460c-8f91-995486a939f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:24.771451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf082459-5749-460c-8f91-995486a939f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:24.782917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf082459-5749-460c-8f91-995486a939f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:24.784378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf082459-5749-460c-8f91-995486a939f3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:24.794808 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1903/3780] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:07:25.370506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924c98e-5249-4db0-a531-8544154e02fa None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:25.373528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3924c98e-5249-4db0-a531-8544154e02fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:25.395789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924c98e-5249-4db0-a531-8544154e02fa None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:25.409991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:25.410648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:25.410939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:25.411191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] PATH_INFO: `/v3/projects/8fbc8a431f1249c3a7fa3b33e2990c39` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:25.412425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fbc8a431f1249c3a7fa3b33e2990c39)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:25.414655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:25.468229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:25.480250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fbc8a431f1249c3a7fa3b33e2990c39', 'request_id': 'req-e81884ba-913c-40a6-800b-063de2e54ffc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:25.492302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fbc8a431f1249c3a7fa3b33e2990c39', 'request_id': 'req-e81884ba-913c-40a6-800b-063de2e54ffc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:25.501066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fbc8a431f1249c3a7fa3b33e2990c39', 'request_id': 'req-e81884ba-913c-40a6-800b-063de2e54ffc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:25.514368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] The token cache is being invalidate because project 8fbc8a431f1249c3a7fa3b33e2990c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:07:25.514710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:25.515323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e81884ba-913c-40a6-800b-063de2e54ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:25.517336 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1878/3781] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:07:25 2022] DELETE /identity/v3/projects/8fbc8a431f1249c3a7fa3b33e2990c39 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:25.522430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9ec91-2429-42fa-8491-89b1afb1bed5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:25.523034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9ec91-2429-42fa-8491-89b1afb1bed5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:25.523034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9ec91-2429-42fa-8491-89b1afb1bed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:25.545880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ffe9ec91-2429-42fa-8491-89b1afb1bed5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:25.546611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ffe9ec91-2429-42fa-8491-89b1afb1bed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:25.605861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:25.620846 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1904/3782] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:25.629000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11019a5-e85c-41f7-aee9-da9be2671874 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:25.629516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11019a5-e85c-41f7-aee9-da9be2671874 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:25.629604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11019a5-e85c-41f7-aee9-da9be2671874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:25.650542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e11019a5-e85c-41f7-aee9-da9be2671874 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:25.651805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e11019a5-e85c-41f7-aee9-da9be2671874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:25.660614 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1879/3783] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:25.665181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9ec91-2429-42fa-8491-89b1afb1bed5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:25.676529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9ec91-2429-42fa-8491-89b1afb1bed5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:25.681471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518f78e-6322-446e-af32-07e06c21f829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:25.682169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518f78e-6322-446e-af32-07e06c21f829 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:25.682510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518f78e-6322-446e-af32-07e06c21f829 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:25.682845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518f78e-6322-446e-af32-07e06c21f829 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:25.683583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518f78e-6322-446e-af32-07e06c21f829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:07:25.683945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518f78e-6322-446e-af32-07e06c21f829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:25.685677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518f78e-6322-446e-af32-07e06c21f829 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:25.697990 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1905/3784] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:07:25.702563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11019a5-e85c-41f7-aee9-da9be2671874 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:25.713451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11019a5-e85c-41f7-aee9-da9be2671874 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:25.718456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53b830-cea1-498f-b821-9477d7ab66e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:25.719092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53b830-cea1-498f-b821-9477d7ab66e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:25.720049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53b830-cea1-498f-b821-9477d7ab66e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:25.720381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53b830-cea1-498f-b821-9477d7ab66e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:25.721057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53b830-cea1-498f-b821-9477d7ab66e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:25.722979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53b830-cea1-498f-b821-9477d7ab66e5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:25.767543 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1880/3785] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:07:25 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:07:25.771375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518f78e-6322-446e-af32-07e06c21f829 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:25.784594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518f78e-6322-446e-af32-07e06c21f829 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:25.862874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb31a1-19c2-44f1-8350-dc15ca67b6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:25.863484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb31a1-19c2-44f1-8350-dc15ca67b6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:25.863761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb31a1-19c2-44f1-8350-dc15ca67b6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:25.864030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb31a1-19c2-44f1-8350-dc15ca67b6e1 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:25.865150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb31a1-19c2-44f1-8350-dc15ca67b6e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:25.867077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb31a1-19c2-44f1-8350-dc15ca67b6e1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:25.892844 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1906/3786] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:07:25 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:07:25.896786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53b830-cea1-498f-b821-9477d7ab66e5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:25.909444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53b830-cea1-498f-b821-9477d7ab66e5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:25.916996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc94a0-ff60-41d4-a0cf-77c6d7241f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:25.917807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc94a0-ff60-41d4-a0cf-77c6d7241f56 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:25.918086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc94a0-ff60-41d4-a0cf-77c6d7241f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:25.918393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc94a0-ff60-41d4-a0cf-77c6d7241f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:25.919208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc94a0-ff60-41d4-a0cf-77c6d7241f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:25.921482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc94a0-ff60-41d4-a0cf-77c6d7241f56 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:25.939795 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1881/3787] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:07:25.946150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb31a1-19c2-44f1-8350-dc15ca67b6e1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:25.958697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb31a1-19c2-44f1-8350-dc15ca67b6e1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:25.964188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c82bc6-0180-4c12-8b7e-615b5a42ca35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:25.964908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c82bc6-0180-4c12-8b7e-615b5a42ca35 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:25.965310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c82bc6-0180-4c12-8b7e-615b5a42ca35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:25.965689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c82bc6-0180-4c12-8b7e-615b5a42ca35 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:25.966433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c82bc6-0180-4c12-8b7e-615b5a42ca35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:25.968833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c82bc6-0180-4c12-8b7e-615b5a42ca35 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:25.981673 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1907/3788] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:07:25.985962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc94a0-ff60-41d4-a0cf-77c6d7241f56 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:25.997340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc94a0-ff60-41d4-a0cf-77c6d7241f56 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:26.003403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6d30f-8a4e-409d-aee3-c8b156dda6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:26.004316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d30f-8a4e-409d-aee3-c8b156dda6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:26.004931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d30f-8a4e-409d-aee3-c8b156dda6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:26.005437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d30f-8a4e-409d-aee3-c8b156dda6cf admin admin] PATH_INFO: `/v3/projects/6edf9c4fb3cc4276999855482ca29697/users/1d78f5fa0ab549cfba0a91e02d3379b4/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:26.027070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6d30f-8a4e-409d-aee3-c8b156dda6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edf9c4fb3cc4276999855482ca29697, user_id=1d78f5fa0ab549cfba0a91e02d3379b4, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:26.030463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6d30f-8a4e-409d-aee3-c8b156dda6cf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:26.049999 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1882/3789] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:07:25 2022] PUT /identity/v3/projects/6edf9c4fb3cc4276999855482ca29697/users/1d78f5fa0ab549cfba0a91e02d3379b4/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:07:26.055548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd326e-cecb-4c03-a7ab-dbc139854b96 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:26.056307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd326e-cecb-4c03-a7ab-dbc139854b96 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:26.056742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd326e-cecb-4c03-a7ab-dbc139854b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:26.084231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7fcd326e-cecb-4c03-a7ab-dbc139854b96 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:26.085489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7fcd326e-cecb-4c03-a7ab-dbc139854b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:26.166409 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1908/3790] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:26.176556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6d30f-8a4e-409d-aee3-c8b156dda6cf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:26.178712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5ed6d30f-8a4e-409d-aee3-c8b156dda6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:26.246105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6d30f-8a4e-409d-aee3-c8b156dda6cf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:26.249888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98483d4f-8b38-4668-81ac-2007b13250d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:26.251025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98483d4f-8b38-4668-81ac-2007b13250d5 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:26.251294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98483d4f-8b38-4668-81ac-2007b13250d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:26.251998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98483d4f-8b38-4668-81ac-2007b13250d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:26.262118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98483d4f-8b38-4668-81ac-2007b13250d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:26.263495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98483d4f-8b38-4668-81ac-2007b13250d5 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:26.274100 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1883/3791] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:07:31.447259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098833e-9e8e-4538-93e8-4f53c57cceb9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:31.447259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098833e-9e8e-4538-93e8-4f53c57cceb9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:31.447702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098833e-9e8e-4538-93e8-4f53c57cceb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:31.480009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3098833e-9e8e-4538-93e8-4f53c57cceb9 None None] MFA Rules not processed for user `1d78f5fa0ab549cfba0a91e02d3379b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:31.481278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3098833e-9e8e-4538-93e8-4f53c57cceb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:31.538079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:31.560160 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1909/3792] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:32.156702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98483d4f-8b38-4668-81ac-2007b13250d5 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:32.158309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-98483d4f-8b38-4668-81ac-2007b13250d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:32.241498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:32.246802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98483d4f-8b38-4668-81ac-2007b13250d5 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:32.255462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71754094-e504-427e-ae88-f3866195e1e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:32.256286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71754094-e504-427e-ae88-f3866195e1e3 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:32.256730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71754094-e504-427e-ae88-f3866195e1e3 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:32.256992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71754094-e504-427e-ae88-f3866195e1e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:32.269740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71754094-e504-427e-ae88-f3866195e1e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:32.271421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71754094-e504-427e-ae88-f3866195e1e3 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:32.281808 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1884/3793] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:07:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:07:35.382591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea5c2-fba5-4e93-aee7-5f12a16886c4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:35.383489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea5c2-fba5-4e93-aee7-5f12a16886c4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:35.383749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea5c2-fba5-4e93-aee7-5f12a16886c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:35.418647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5d1ea5c2-fba5-4e93-aee7-5f12a16886c4 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:35.419988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5d1ea5c2-fba5-4e93-aee7-5f12a16886c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:35.484863 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1910/3794] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:07:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:35.510047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71754094-e504-427e-ae88-f3866195e1e3 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:35.511481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-71754094-e504-427e-ae88-f3866195e1e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:35.611402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71754094-e504-427e-ae88-f3866195e1e3 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:35.617330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e5d44-8e5a-46b9-8305-6cdd1d9603bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:35.618160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e5d44-8e5a-46b9-8305-6cdd1d9603bc service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:35.618538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e5d44-8e5a-46b9-8305-6cdd1d9603bc service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:35.619005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e5d44-8e5a-46b9-8305-6cdd1d9603bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:35.633457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e5d44-8e5a-46b9-8305-6cdd1d9603bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:35.635054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e5d44-8e5a-46b9-8305-6cdd1d9603bc service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:35.644792 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1885/3795] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:07:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:36.230454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7dad4-1590-40ca-9950-6fcdbaf2bd59 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:36.230733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7dad4-1590-40ca-9950-6fcdbaf2bd59 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:36.231206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7dad4-1590-40ca-9950-6fcdbaf2bd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:36.261865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-98d7dad4-1590-40ca-9950-6fcdbaf2bd59 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:36.262993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-98d7dad4-1590-40ca-9950-6fcdbaf2bd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:36.274492 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1911/3796] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:07:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:36.288934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e5d44-8e5a-46b9-8305-6cdd1d9603bc service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:36.301607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e5d44-8e5a-46b9-8305-6cdd1d9603bc service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:36.313932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360eca27-df3c-47d5-bf82-70f4da7a4ac7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:36.314585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eca27-df3c-47d5-bf82-70f4da7a4ac7 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:36.315819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eca27-df3c-47d5-bf82-70f4da7a4ac7 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:36.316095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eca27-df3c-47d5-bf82-70f4da7a4ac7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:36.329211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360eca27-df3c-47d5-bf82-70f4da7a4ac7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:36.332551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360eca27-df3c-47d5-bf82-70f4da7a4ac7 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:36.342434 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1886/3797] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:07:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:38.040962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fbd9c-c742-4b35-ac4c-8e92eea86586 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:38.041999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fbd9c-c742-4b35-ac4c-8e92eea86586 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:38.042408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fbd9c-c742-4b35-ac4c-8e92eea86586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:38.081723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6f4fbd9c-c742-4b35-ac4c-8e92eea86586 None None] MFA Rules not processed for user `1d78f5fa0ab549cfba0a91e02d3379b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:38.083303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f4fbd9c-c742-4b35-ac4c-8e92eea86586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:38.091771 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1912/3798] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:46.338792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=42 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:48.068622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=35 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:53.051567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360eca27-df3c-47d5-bf82-70f4da7a4ac7 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:53.053024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-360eca27-df3c-47d5-bf82-70f4da7a4ac7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:53.073559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360eca27-df3c-47d5-bf82-70f4da7a4ac7 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:53.078716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:53.079368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:53.079644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:53.079909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] PATH_INFO: `/v3/users/90e8754a100240a498e422de07c76bbf` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:53.081181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e8754a100240a498e422de07c76bbf)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:53.083163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:53.150420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e8754a100240a498e422de07c76bbf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:53.159743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e8754a100240a498e422de07c76bbf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:53.178031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e8754a100240a498e422de07c76bbf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:53.207490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e8754a100240a498e422de07c76bbf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:53.208578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9d617901-8a7d-485f-b5fa-2823907529df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e8754a100240a498e422de07c76bbf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:53.215905 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1887/3799] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:07:53 2022] DELETE /identity/v3/users/90e8754a100240a498e422de07c76bbf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:53.449177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fbd9c-c742-4b35-ac4c-8e92eea86586 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:53.470000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fbd9c-c742-4b35-ac4c-8e92eea86586 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:53.532212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:53.533018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:53.533421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:53.533693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] PATH_INFO: `/v3/projects/172114b9a46e4df8ae8076ed582f0b5f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:53.534985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=172114b9a46e4df8ae8076ed582f0b5f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:53.536977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:53.595979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172114b9a46e4df8ae8076ed582f0b5f', 'request_id': 'req-b065cb84-d301-4346-82b7-446f81d9ac7d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:53.603954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172114b9a46e4df8ae8076ed582f0b5f', 'request_id': 'req-b065cb84-d301-4346-82b7-446f81d9ac7d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:53.611267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172114b9a46e4df8ae8076ed582f0b5f', 'request_id': 'req-b065cb84-d301-4346-82b7-446f81d9ac7d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:53.622396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] The token cache is being invalidate because project 172114b9a46e4df8ae8076ed582f0b5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:07:53.622679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:53.623339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b065cb84-d301-4346-82b7-446f81d9ac7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:07:53.625114 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1913/3800] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:07:53 2022] DELETE /identity/v3/projects/172114b9a46e4df8ae8076ed582f0b5f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:53.630260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348fa92-6eab-46da-ab57-def29001e5a7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:53.630608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348fa92-6eab-46da-ab57-def29001e5a7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:53.630938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348fa92-6eab-46da-ab57-def29001e5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:53.653952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d348fa92-6eab-46da-ab57-def29001e5a7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:53.655322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d348fa92-6eab-46da-ab57-def29001e5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:53.719450 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1888/3801] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:53.727221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee78de-f053-40ca-ab8d-59d4c1d25d98 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:53.727607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee78de-f053-40ca-ab8d-59d4c1d25d98 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:53.727946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee78de-f053-40ca-ab8d-59d4c1d25d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:53.749749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e9ee78de-f053-40ca-ab8d-59d4c1d25d98 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:53.751031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e9ee78de-f053-40ca-ab8d-59d4c1d25d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:53.759141 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1914/3802] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:53.763750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348fa92-6eab-46da-ab57-def29001e5a7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:53.774483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348fa92-6eab-46da-ab57-def29001e5a7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:53.779378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e0b50-fd30-4448-b0a7-ce2e6ae8e5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:53.780086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e0b50-fd30-4448-b0a7-ce2e6ae8e5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:53.780425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e0b50-fd30-4448-b0a7-ce2e6ae8e5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:53.780928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e0b50-fd30-4448-b0a7-ce2e6ae8e5c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:53.781887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e0b50-fd30-4448-b0a7-ce2e6ae8e5c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:07:53.782254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e0b50-fd30-4448-b0a7-ce2e6ae8e5c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:53.784295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e0b50-fd30-4448-b0a7-ce2e6ae8e5c4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:53.797097 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1889/3803] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:07:53.801132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee78de-f053-40ca-ab8d-59d4c1d25d98 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:53.810474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee78de-f053-40ca-ab8d-59d4c1d25d98 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:53.815877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380ccbe-94b2-49f0-aaa3-97eef81b9e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:53.816462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380ccbe-94b2-49f0-aaa3-97eef81b9e2d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:53.816733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380ccbe-94b2-49f0-aaa3-97eef81b9e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:53.817161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380ccbe-94b2-49f0-aaa3-97eef81b9e2d admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:53.817773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380ccbe-94b2-49f0-aaa3-97eef81b9e2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:53.819658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380ccbe-94b2-49f0-aaa3-97eef81b9e2d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:53.848479 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1915/3804] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:07:53 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:07:53.852643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e0b50-fd30-4448-b0a7-ce2e6ae8e5c4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:53.864034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e0b50-fd30-4448-b0a7-ce2e6ae8e5c4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:53.937918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a551c-f621-4f56-9fb1-4c1f79ce5608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:53.938516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a551c-f621-4f56-9fb1-4c1f79ce5608 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:53.938785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a551c-f621-4f56-9fb1-4c1f79ce5608 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:53.939048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a551c-f621-4f56-9fb1-4c1f79ce5608 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:53.939673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a551c-f621-4f56-9fb1-4c1f79ce5608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:53.941575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a551c-f621-4f56-9fb1-4c1f79ce5608 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:53.949729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:53.965372 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1890/3805] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:07:53 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:07:53.970184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380ccbe-94b2-49f0-aaa3-97eef81b9e2d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:53.980774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380ccbe-94b2-49f0-aaa3-97eef81b9e2d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:53.987002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65ef4b-5c94-4f5b-8943-7ce370985e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:53.987696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65ef4b-5c94-4f5b-8943-7ce370985e7a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:53.987972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65ef4b-5c94-4f5b-8943-7ce370985e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:53.988249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65ef4b-5c94-4f5b-8943-7ce370985e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:53.989075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65ef4b-5c94-4f5b-8943-7ce370985e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:53.990927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65ef4b-5c94-4f5b-8943-7ce370985e7a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:54.002364 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1916/3806] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:07:54.006327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a551c-f621-4f56-9fb1-4c1f79ce5608 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:54.016981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a551c-f621-4f56-9fb1-4c1f79ce5608 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:54.021587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d34c01-d471-46ab-b76b-1aa1fb5ec28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:54.022172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34c01-d471-46ab-b76b-1aa1fb5ec28d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:54.022441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34c01-d471-46ab-b76b-1aa1fb5ec28d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:54.022705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34c01-d471-46ab-b76b-1aa1fb5ec28d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:54.024040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d34c01-d471-46ab-b76b-1aa1fb5ec28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:54.025652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d34c01-d471-46ab-b76b-1aa1fb5ec28d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:54.037193 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1891/3807] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:07:54.041334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65ef4b-5c94-4f5b-8943-7ce370985e7a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:54.051580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65ef4b-5c94-4f5b-8943-7ce370985e7a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:54.057555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c47eca-a4b4-47b9-a3ab-5bf251bf1d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:54.058197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c47eca-a4b4-47b9-a3ab-5bf251bf1d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:54.058469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c47eca-a4b4-47b9-a3ab-5bf251bf1d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:54.058731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c47eca-a4b4-47b9-a3ab-5bf251bf1d01 admin admin] PATH_INFO: `/v3/projects/9f733281111e4320a7a9fac31a821023/users/a4d021129a274e58ba6f543b9affc2ac/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:54.078592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c47eca-a4b4-47b9-a3ab-5bf251bf1d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f733281111e4320a7a9fac31a821023, user_id=a4d021129a274e58ba6f543b9affc2ac, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:54.081251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c47eca-a4b4-47b9-a3ab-5bf251bf1d01 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:54.095312 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1917/3808] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:07:54 2022] PUT /identity/v3/projects/9f733281111e4320a7a9fac31a821023/users/a4d021129a274e58ba6f543b9affc2ac/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:07:54.100320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c72f2-0287-48b8-9a48-481231c93aee None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:54.100851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c72f2-0287-48b8-9a48-481231c93aee None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:54.100851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c72f2-0287-48b8-9a48-481231c93aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:54.121184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a10c72f2-0287-48b8-9a48-481231c93aee None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:54.122321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a10c72f2-0287-48b8-9a48-481231c93aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:54.185154 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1892/3809] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:07:54.201179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c47eca-a4b4-47b9-a3ab-5bf251bf1d01 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:07:54.201179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4c47eca-a4b4-47b9-a3ab-5bf251bf1d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:54.238913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:54.269259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c47eca-a4b4-47b9-a3ab-5bf251bf1d01 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:07:54.273189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503061ec-8c2a-40c8-bc19-4639bb93e29e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:07:54.273840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503061ec-8c2a-40c8-bc19-4639bb93e29e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:54.274485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503061ec-8c2a-40c8-bc19-4639bb93e29e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:54.274485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503061ec-8c2a-40c8-bc19-4639bb93e29e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:54.285206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503061ec-8c2a-40c8-bc19-4639bb93e29e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:07:54.286702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503061ec-8c2a-40c8-bc19-4639bb93e29e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:07:54.298111 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1918/3810] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:07:59.618486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce248f9-7001-4494-a897-845a6984dadb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:07:59.618978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce248f9-7001-4494-a897-845a6984dadb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:07:59.619086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce248f9-7001-4494-a897-845a6984dadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:07:59.681311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fce248f9-7001-4494-a897-845a6984dadb None None] MFA Rules not processed for user `a4d021129a274e58ba6f543b9affc2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:07:59.682980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fce248f9-7001-4494-a897-845a6984dadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:07:59.877577 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1893/3811] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:07:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:00.363689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d5711-3a6a-4a5c-9daf-cbe0b2b170df None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:00.364352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d5711-3a6a-4a5c-9daf-cbe0b2b170df None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:00.364352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d5711-3a6a-4a5c-9daf-cbe0b2b170df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:00.385953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-826d5711-3a6a-4a5c-9daf-cbe0b2b170df None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:00.387145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-826d5711-3a6a-4a5c-9daf-cbe0b2b170df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:00.457899 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1919/3812] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:00.474680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce248f9-7001-4494-a897-845a6984dadb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:00.476559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fce248f9-7001-4494-a897-845a6984dadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:00.567038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce248f9-7001-4494-a897-845a6984dadb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:00.580379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb71111-beb9-4c5a-9b38-f6449c295488 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:00.581015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb71111-beb9-4c5a-9b38-f6449c295488 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:00.583654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb71111-beb9-4c5a-9b38-f6449c295488 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:00.583963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb71111-beb9-4c5a-9b38-f6449c295488 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:00.609496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb71111-beb9-4c5a-9b38-f6449c295488 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:00.611063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb71111-beb9-4c5a-9b38-f6449c295488 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:00.622811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:00.630435 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1894/3813] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:08:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:07.287607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d5711-3a6a-4a5c-9daf-cbe0b2b170df None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:07.308564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d5711-3a6a-4a5c-9daf-cbe0b2b170df None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:07.336501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:07.343519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:07.343834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:07.349969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] PATH_INFO: `/v3/users/a4d021129a274e58ba6f543b9affc2ac` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:07.353210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d021129a274e58ba6f543b9affc2ac)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:07.355172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:07.478207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d021129a274e58ba6f543b9affc2ac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:07.509831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d021129a274e58ba6f543b9affc2ac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:07.539500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d021129a274e58ba6f543b9affc2ac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:07.552253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d021129a274e58ba6f543b9affc2ac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:07.553152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0eafcfe-f3ce-48f8-968c-2f22b64886ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d021129a274e58ba6f543b9affc2ac'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:07.562270 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1920/3814] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:07 2022] DELETE /identity/v3/users/a4d021129a274e58ba6f543b9affc2ac => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:07.837291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb71111-beb9-4c5a-9b38-f6449c295488 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:07.854120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb71111-beb9-4c5a-9b38-f6449c295488 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:07.937078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:07.937990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:07.938682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:07.939194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] PATH_INFO: `/v3/projects/9f733281111e4320a7a9fac31a821023` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:07.940926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f733281111e4320a7a9fac31a821023)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:07.943068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:08.005340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f733281111e4320a7a9fac31a821023', 'request_id': 'req-c0fc9999-0dbf-4086-ba0c-871e980dc494'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:08.052962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f733281111e4320a7a9fac31a821023', 'request_id': 'req-c0fc9999-0dbf-4086-ba0c-871e980dc494'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:08.064910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f733281111e4320a7a9fac31a821023', 'request_id': 'req-c0fc9999-0dbf-4086-ba0c-871e980dc494'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:08.098864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] The token cache is being invalidate because project 9f733281111e4320a7a9fac31a821023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:08.099206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:08.100029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c0fc9999-0dbf-4086-ba0c-871e980dc494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:08.110184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211479f-5de1-42f8-8acc-659f570967fe None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:08.110978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211479f-5de1-42f8-8acc-659f570967fe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:08.112039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211479f-5de1-42f8-8acc-659f570967fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:08.120252 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1895/3815] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:07 2022] DELETE /identity/v3/projects/9f733281111e4320a7a9fac31a821023 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:08.152452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9211479f-5de1-42f8-8acc-659f570967fe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:08.154121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9211479f-5de1-42f8-8acc-659f570967fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:08.250906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:08.291679 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1921/3816] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:08.299668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561d022-bfc1-4a84-9865-08b09e4f2168 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:08.300020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561d022-bfc1-4a84-9865-08b09e4f2168 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:08.300225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561d022-bfc1-4a84-9865-08b09e4f2168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:08.329964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a561d022-bfc1-4a84-9865-08b09e4f2168 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:08.331501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a561d022-bfc1-4a84-9865-08b09e4f2168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:08.362099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211479f-5de1-42f8-8acc-659f570967fe None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:08.362493 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1896/3817] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:08.377492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211479f-5de1-42f8-8acc-659f570967fe None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:08.386372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330c50d-5c59-46df-af06-b471cb29e75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:08.387214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330c50d-5c59-46df-af06-b471cb29e75b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:08.387561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330c50d-5c59-46df-af06-b471cb29e75b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:08.387924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330c50d-5c59-46df-af06-b471cb29e75b admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:08.388870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330c50d-5c59-46df-af06-b471cb29e75b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:08.389443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330c50d-5c59-46df-af06-b471cb29e75b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:08.392171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330c50d-5c59-46df-af06-b471cb29e75b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:08.407211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561d022-bfc1-4a84-9865-08b09e4f2168 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:08.414105 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1922/3818] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:08.415357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330c50d-5c59-46df-af06-b471cb29e75b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:08.429321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330c50d-5c59-46df-af06-b471cb29e75b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:08.429667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a561d022-bfc1-4a84-9865-08b09e4f2168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:08.436053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deba8dc-7492-48f8-99ba-0e6f322430e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:08.437403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deba8dc-7492-48f8-99ba-0e6f322430e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:08.437843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deba8dc-7492-48f8-99ba-0e6f322430e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:08.439052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deba8dc-7492-48f8-99ba-0e6f322430e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:08.439950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deba8dc-7492-48f8-99ba-0e6f322430e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:08.442888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deba8dc-7492-48f8-99ba-0e6f322430e9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:08.500058 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1923/3819] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:08 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:08.508013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deba8dc-7492-48f8-99ba-0e6f322430e9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:08.602927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561d022-bfc1-4a84-9865-08b09e4f2168 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:08.668734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:08.712699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8384a3f-346b-40e6-8b51-fc7ef0108118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:08.712699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8384a3f-346b-40e6-8b51-fc7ef0108118 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:08.712942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8384a3f-346b-40e6-8b51-fc7ef0108118 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:08.714142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8384a3f-346b-40e6-8b51-fc7ef0108118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:08.720153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f8384a3f-346b-40e6-8b51-fc7ef0108118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:08.725788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deba8dc-7492-48f8-99ba-0e6f322430e9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:08.743039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4f941-164f-41de-a9ba-838a39b58d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:08.744799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4f941-164f-41de-a9ba-838a39b58d10 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:08.746375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4f941-164f-41de-a9ba-838a39b58d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:08.746728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4f941-164f-41de-a9ba-838a39b58d10 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:08.747775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4f941-164f-41de-a9ba-838a39b58d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:08.753632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4f941-164f-41de-a9ba-838a39b58d10 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:08.804132 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1924/3820] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:08 2022] POST /identity/v3/users => generated 287 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:08.806199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4f941-164f-41de-a9ba-838a39b58d10 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:08.824052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4f941-164f-41de-a9ba-838a39b58d10 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:08.831034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9e46a-03b2-4b37-b8bb-a86239b7fe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:08.831650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9e46a-03b2-4b37-b8bb-a86239b7fe4b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:08.832417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9e46a-03b2-4b37-b8bb-a86239b7fe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:08.832697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9e46a-03b2-4b37-b8bb-a86239b7fe4b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:08.833449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9e46a-03b2-4b37-b8bb-a86239b7fe4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:08.836983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9e46a-03b2-4b37-b8bb-a86239b7fe4b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:08.857209 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1925/3821] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:08.858592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9e46a-03b2-4b37-b8bb-a86239b7fe4b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:08.866257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8384a3f-346b-40e6-8b51-fc7ef0108118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:08.869027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8384a3f-346b-40e6-8b51-fc7ef0108118 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:08.886833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9e46a-03b2-4b37-b8bb-a86239b7fe4b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:08.893536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b01ef-f962-40c2-a6c7-3cc66deda7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:08.895288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b01ef-f962-40c2-a6c7-3cc66deda7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:08.895578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b01ef-f962-40c2-a6c7-3cc66deda7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:08.895829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b01ef-f962-40c2-a6c7-3cc66deda7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:08.896462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b01ef-f962-40c2-a6c7-3cc66deda7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:08.898880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b01ef-f962-40c2-a6c7-3cc66deda7f0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:08.914157 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1897/3822] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:08.917289 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1926/3823] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:08.917955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8384a3f-346b-40e6-8b51-fc7ef0108118 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:08.935595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8384a3f-346b-40e6-8b51-fc7ef0108118 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:08.942511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9eb3d-ca34-4760-a3ae-66956e7bb537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:08.943607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9eb3d-ca34-4760-a3ae-66956e7bb537 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:08.944083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9eb3d-ca34-4760-a3ae-66956e7bb537 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:08.945025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9eb3d-ca34-4760-a3ae-66956e7bb537 admin admin] PATH_INFO: `/v3/projects/3cd52aaddd184ef890002aacf0dd160c/users/e0c117b4333843c89f1f44c283d807cd/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:08.965156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9eb3d-ca34-4760-a3ae-66956e7bb537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cd52aaddd184ef890002aacf0dd160c, user_id=e0c117b4333843c89f1f44c283d807cd, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:08.970329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9eb3d-ca34-4760-a3ae-66956e7bb537 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:09.012156 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1898/3824] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:08 2022] PUT /identity/v3/projects/3cd52aaddd184ef890002aacf0dd160c/users/e0c117b4333843c89f1f44c283d807cd/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:09.012345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3591280-20e1-41df-813b-b802b0e9ce79 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:09.012577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3591280-20e1-41df-813b-b802b0e9ce79 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:09.012854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3591280-20e1-41df-813b-b802b0e9ce79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:09.060066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f3591280-20e1-41df-813b-b802b0e9ce79 None None] MFA Rules not processed for user `e0c117b4333843c89f1f44c283d807cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:09.062662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f3591280-20e1-41df-813b-b802b0e9ce79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:09.226711 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1927/3825] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:09.228562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9eb3d-ca34-4760-a3ae-66956e7bb537 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:09.335635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9eb3d-ca34-4760-a3ae-66956e7bb537 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:09.342363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aa2ff-9855-43ac-9cf2-10aef1bd90ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:09.343384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aa2ff-9855-43ac-9cf2-10aef1bd90ef admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:09.343772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aa2ff-9855-43ac-9cf2-10aef1bd90ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:09.344131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aa2ff-9855-43ac-9cf2-10aef1bd90ef admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:09.344972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801aa2ff-9855-43ac-9cf2-10aef1bd90ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:09.347344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801aa2ff-9855-43ac-9cf2-10aef1bd90ef admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:09.435982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3591280-20e1-41df-813b-b802b0e9ce79 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:09.462791 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1899/3826] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:09 2022] POST /identity/v3/projects => generated 375 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:09.481603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3591280-20e1-41df-813b-b802b0e9ce79 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:09.493480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:09.674966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ffc5-b3ef-47ef-bf67-f3279d3380cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:09.676176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ffc5-b3ef-47ef-bf67-f3279d3380cf admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:09.676666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ffc5-b3ef-47ef-bf67-f3279d3380cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:09.677156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ffc5-b3ef-47ef-bf67-f3279d3380cf admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:09.677971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2ffc5-b3ef-47ef-bf67-f3279d3380cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:09.680793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2ffc5-b3ef-47ef-bf67-f3279d3380cf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:09.731386 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1928/3827] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:09 2022] POST /identity/v3/users => generated 294 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:09.737577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aa2ff-9855-43ac-9cf2-10aef1bd90ef admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:09.757817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aa2ff-9855-43ac-9cf2-10aef1bd90ef admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:09.798869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81173b78-022b-49b7-bb8f-f51a0779b36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:09.799921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81173b78-022b-49b7-bb8f-f51a0779b36f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:09.800047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81173b78-022b-49b7-bb8f-f51a0779b36f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:09.800416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81173b78-022b-49b7-bb8f-f51a0779b36f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:09.801218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81173b78-022b-49b7-bb8f-f51a0779b36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:09.803496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81173b78-022b-49b7-bb8f-f51a0779b36f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:09.840825 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1900/3828] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:09.841565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ffc5-b3ef-47ef-bf67-f3279d3380cf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:09.856748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ffc5-b3ef-47ef-bf67-f3279d3380cf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:09.866965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d4b46-84e7-4b51-b5aa-a2d978552221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:09.867698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d4b46-84e7-4b51-b5aa-a2d978552221 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:09.867977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d4b46-84e7-4b51-b5aa-a2d978552221 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:09.868246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d4b46-84e7-4b51-b5aa-a2d978552221 admin admin] PATH_INFO: `/v3/projects/23372f4e6f8e4d3a83a0dacc0d6e5635/users/e7c0815cc1bc4333836dffef48aae959/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:09.891300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d4b46-84e7-4b51-b5aa-a2d978552221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23372f4e6f8e4d3a83a0dacc0d6e5635, user_id=e7c0815cc1bc4333836dffef48aae959, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:09.897013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d4b46-84e7-4b51-b5aa-a2d978552221 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:09.937699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f458924-c9a2-4f34-9d11-0bdc9ac8bf3b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:09.938229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f458924-c9a2-4f34-9d11-0bdc9ac8bf3b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:09.938229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f458924-c9a2-4f34-9d11-0bdc9ac8bf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:09.942073 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1929/3829] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:09 2022] PUT /identity/v3/projects/23372f4e6f8e4d3a83a0dacc0d6e5635/users/e7c0815cc1bc4333836dffef48aae959/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:09.946198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d4b46-84e7-4b51-b5aa-a2d978552221 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:09.991092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1f458924-c9a2-4f34-9d11-0bdc9ac8bf3b None None] MFA Rules not processed for user `2f4e4fc2f97f4c1a832c6348a97fd155`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:09.992744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1f458924-c9a2-4f34-9d11-0bdc9ac8bf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:10.043303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:10.120676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d4b46-84e7-4b51-b5aa-a2d978552221 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:10.127888 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1901/3830] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:10.138951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c489f-7477-46ca-9e98-88434ed04718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:10.140254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c489f-7477-46ca-9e98-88434ed04718 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:10.141070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c489f-7477-46ca-9e98-88434ed04718 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:10.141547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c489f-7477-46ca-9e98-88434ed04718 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:10.142212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c489f-7477-46ca-9e98-88434ed04718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:10.144947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c489f-7477-46ca-9e98-88434ed04718 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:10.163689 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1930/3831] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:10.166294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f458924-c9a2-4f34-9d11-0bdc9ac8bf3b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:10.209526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f458924-c9a2-4f34-9d11-0bdc9ac8bf3b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:10.240002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d0a68-a259-4a5f-bb71-834c387fa3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:10.240690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d0a68-a259-4a5f-bb71-834c387fa3ec admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:10.240971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d0a68-a259-4a5f-bb71-834c387fa3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:10.241339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d0a68-a259-4a5f-bb71-834c387fa3ec admin admin] PATH_INFO: `/v3/projects/23372f4e6f8e4d3a83a0dacc0d6e5635/users/e7c0815cc1bc4333836dffef48aae959/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:10.258390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d0a68-a259-4a5f-bb71-834c387fa3ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=23372f4e6f8e4d3a83a0dacc0d6e5635, user_id=e7c0815cc1bc4333836dffef48aae959, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:10.262426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d0a68-a259-4a5f-bb71-834c387fa3ec admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:10.295309 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1902/3832] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:10 2022] PUT /identity/v3/projects/23372f4e6f8e4d3a83a0dacc0d6e5635/users/e7c0815cc1bc4333836dffef48aae959/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:10.305710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0d5a7-c79e-48f5-a1be-93c09c63de25 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:10.305990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0d5a7-c79e-48f5-a1be-93c09c63de25 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:10.306269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0d5a7-c79e-48f5-a1be-93c09c63de25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:10.373081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-16c0d5a7-c79e-48f5-a1be-93c09c63de25 None None] MFA Rules not processed for user `e7c0815cc1bc4333836dffef48aae959`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:10.376303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-16c0d5a7-c79e-48f5-a1be-93c09c63de25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:10.503410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:10.522650 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1931/3833] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:10.526658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d0a68-a259-4a5f-bb71-834c387fa3ec admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:10.540802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d0a68-a259-4a5f-bb71-834c387fa3ec admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:10.563810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71de89-a4d9-4416-a201-deec131c9678 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:10.564583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71de89-a4d9-4416-a201-deec131c9678 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:10.564860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71de89-a4d9-4416-a201-deec131c9678 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:10.568233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71de89-a4d9-4416-a201-deec131c9678 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:10.569011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71de89-a4d9-4416-a201-deec131c9678 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:10.573069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71de89-a4d9-4416-a201-deec131c9678 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:10.583197 ncdv-master-1 devstack@keystone.service[99099]: WARNING keystone.server.flask.common [None req-5e71de89-a4d9-4416-a201-deec131c9678 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 15 21:08:10.657541 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1903/3834] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:10 2022] POST /identity/v3/projects => generated 351 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:10.661435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0d5a7-c79e-48f5-a1be-93c09c63de25 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:10.674083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0d5a7-c79e-48f5-a1be-93c09c63de25 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:10.828799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbfbf1-3c64-46e3-a74c-4e064d70e496 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:10.829445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbfbf1-3c64-46e3-a74c-4e064d70e496 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:10.829725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbfbf1-3c64-46e3-a74c-4e064d70e496 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:10.833231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbfbf1-3c64-46e3-a74c-4e064d70e496 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:10.833896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbfbf1-3c64-46e3-a74c-4e064d70e496 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:10.835801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbfbf1-3c64-46e3-a74c-4e064d70e496 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:10.918529 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1932/3835] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:08:10 2022] POST /identity/v3/projects => generated 349 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:10.922785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71de89-a4d9-4416-a201-deec131c9678 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:10.940095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71de89-a4d9-4416-a201-deec131c9678 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:11.032896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db39ab-75bb-497c-b039-91fb34d31857 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:11.033532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db39ab-75bb-497c-b039-91fb34d31857 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:11.033807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db39ab-75bb-497c-b039-91fb34d31857 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:11.034073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db39ab-75bb-497c-b039-91fb34d31857 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/users/e0c117b4333843c89f1f44c283d807cd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:11.035933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db39ab-75bb-497c-b039-91fb34d31857 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:get_user(user_id=e0c117b4333843c89f1f44c283d807cd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.038460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db39ab-75bb-497c-b039-91fb34d31857 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.052589 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1904/3836] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:08:10 2022] GET /identity/v3/users/e0c117b4333843c89f1f44c283d807cd => generated 287 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:11.053877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbfbf1-3c64-46e3-a74c-4e064d70e496 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:11.074825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbfbf1-3c64-46e3-a74c-4e064d70e496 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:11.085061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faad03-4aa6-431f-8e90-6162cc15cfd5 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:11.085942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faad03-4aa6-431f-8e90-6162cc15cfd5 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:11.087003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faad03-4aa6-431f-8e90-6162cc15cfd5 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:11.087143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faad03-4aa6-431f-8e90-6162cc15cfd5 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:11.087792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faad03-4aa6-431f-8e90-6162cc15cfd5 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.089725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faad03-4aa6-431f-8e90-6162cc15cfd5 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.171970 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1933/3837] 10.222.0.16 () {60 vars in 1162 bytes} [Sat Oct 15 21:08:11 2022] POST /identity/v3/credentials => generated 430 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:11.176119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db39ab-75bb-497c-b039-91fb34d31857 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:11.208043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db39ab-75bb-497c-b039-91fb34d31857 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:11.228023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcdbbb-5814-47f8-8e55-fc7913575ebf tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:11.228985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcdbbb-5814-47f8-8e55-fc7913575ebf tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:11.233473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcdbbb-5814-47f8-8e55-fc7913575ebf tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:11.233852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcdbbb-5814-47f8-8e55-fc7913575ebf tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/credentials/aa0d2f36704c3b9580197235c8b67de0fba7355845c950fc97999bff1656b3a8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:11.242709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcdbbb-5814-47f8-8e55-fc7913575ebf tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=aa0d2f36704c3b9580197235c8b67de0fba7355845c950fc97999bff1656b3a8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.245820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcdbbb-5814-47f8-8e55-fc7913575ebf tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.256538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcdbbb-5814-47f8-8e55-fc7913575ebf tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=aa0d2f36704c3b9580197235c8b67de0fba7355845c950fc97999bff1656b3a8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.259157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcdbbb-5814-47f8-8e55-fc7913575ebf tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.297591 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1905/3838] 10.222.0.16 () {60 vars in 1358 bytes} [Sat Oct 15 21:08:11 2022] PATCH /identity/v3/credentials/aa0d2f36704c3b9580197235c8b67de0fba7355845c950fc97999bff1656b3a8 => generated 436 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:11.299654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faad03-4aa6-431f-8e90-6162cc15cfd5 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:11.324221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faad03-4aa6-431f-8e90-6162cc15cfd5 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:11.366965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181ea8a-5dd0-4b89-bc10-ca4393f9e7a6 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:11.367732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181ea8a-5dd0-4b89-bc10-ca4393f9e7a6 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:11.368007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181ea8a-5dd0-4b89-bc10-ca4393f9e7a6 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:11.368274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181ea8a-5dd0-4b89-bc10-ca4393f9e7a6 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/credentials/aa0d2f36704c3b9580197235c8b67de0fba7355845c950fc97999bff1656b3a8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:11.388539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181ea8a-5dd0-4b89-bc10-ca4393f9e7a6 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:get_credential(credential_id=aa0d2f36704c3b9580197235c8b67de0fba7355845c950fc97999bff1656b3a8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.396137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181ea8a-5dd0-4b89-bc10-ca4393f9e7a6 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.433333 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1934/3839] 10.222.0.16 () {58 vars in 1335 bytes} [Sat Oct 15 21:08:11 2022] GET /identity/v3/credentials/aa0d2f36704c3b9580197235c8b67de0fba7355845c950fc97999bff1656b3a8 => generated 436 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:11.440057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcdbbb-5814-47f8-8e55-fc7913575ebf tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:11.468610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcdbbb-5814-47f8-8e55-fc7913575ebf tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:11.483651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d4af4-5afa-4caa-b07b-38d80d52138b tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:11.486377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d4af4-5afa-4caa-b07b-38d80d52138b tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:11.486903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d4af4-5afa-4caa-b07b-38d80d52138b tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:11.487360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d4af4-5afa-4caa-b07b-38d80d52138b tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/credentials/aa0d2f36704c3b9580197235c8b67de0fba7355845c950fc97999bff1656b3a8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:11.493466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d4af4-5afa-4caa-b07b-38d80d52138b tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=aa0d2f36704c3b9580197235c8b67de0fba7355845c950fc97999bff1656b3a8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.497825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d4af4-5afa-4caa-b07b-38d80d52138b tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.538756 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1906/3840] 10.222.0.16 () {58 vars in 1338 bytes} [Sat Oct 15 21:08:11 2022] DELETE /identity/v3/credentials/aa0d2f36704c3b9580197235c8b67de0fba7355845c950fc97999bff1656b3a8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:11.547739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181ea8a-5dd0-4b89-bc10-ca4393f9e7a6 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:11.559813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181ea8a-5dd0-4b89-bc10-ca4393f9e7a6 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:11.583313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8e2d2-b3b4-497f-a832-5d519cbea1ad tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:11.589438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8e2d2-b3b4-497f-a832-5d519cbea1ad tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:11.589738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8e2d2-b3b4-497f-a832-5d519cbea1ad tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:11.589999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8e2d2-b3b4-497f-a832-5d519cbea1ad tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:11.590661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8e2d2-b3b4-497f-a832-5d519cbea1ad tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.600016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8e2d2-b3b4-497f-a832-5d519cbea1ad tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.643221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1935/3841] 10.222.0.16 () {60 vars in 1162 bytes} [Sat Oct 15 21:08:11 2022] POST /identity/v3/credentials => generated 430 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:11.644525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d4af4-5afa-4caa-b07b-38d80d52138b tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:11.664223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d4af4-5afa-4caa-b07b-38d80d52138b tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:11.689842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c200b5-dcf2-489d-8034-dd6830cdf1ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:11.690478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c200b5-dcf2-489d-8034-dd6830cdf1ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:11.690757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c200b5-dcf2-489d-8034-dd6830cdf1ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:11.691031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c200b5-dcf2-489d-8034-dd6830cdf1ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:11.691687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c200b5-dcf2-489d-8034-dd6830cdf1ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.696076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c200b5-dcf2-489d-8034-dd6830cdf1ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.750785 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1907/3842] 10.222.0.16 () {60 vars in 1162 bytes} [Sat Oct 15 21:08:11 2022] POST /identity/v3/credentials => generated 430 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:11.756302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8e2d2-b3b4-497f-a832-5d519cbea1ad tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:11.773578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8e2d2-b3b4-497f-a832-5d519cbea1ad tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:11.786103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:11.787865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:11.788145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:11.788413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:11.790535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:list_credentials()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.796671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.815625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.824167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.840806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.845267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.853410 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1936/3843] 10.222.0.16 () {58 vars in 1140 bytes} [Sat Oct 15 21:08:11 2022] GET /identity/v3/credentials => generated 945 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:11.856345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c200b5-dcf2-489d-8034-dd6830cdf1ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:11.874187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c200b5-dcf2-489d-8034-dd6830cdf1ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:11.898786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ef2f7-3702-4762-8dc8-852ad224cc2c tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:11.899497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ef2f7-3702-4762-8dc8-852ad224cc2c tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:11.903821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ef2f7-3702-4762-8dc8-852ad224cc2c tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:11.903999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ef2f7-3702-4762-8dc8-852ad224cc2c tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/credentials/afbcb1f29b2a20e29440e49fa48117d227c218c5b8adf9837022ad804e3ac61e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:11.910369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ef2f7-3702-4762-8dc8-852ad224cc2c tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=afbcb1f29b2a20e29440e49fa48117d227c218c5b8adf9837022ad804e3ac61e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:11.912124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ef2f7-3702-4762-8dc8-852ad224cc2c tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:11.971555 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1908/3844] 10.222.0.16 () {58 vars in 1338 bytes} [Sat Oct 15 21:08:11 2022] DELETE /identity/v3/credentials/afbcb1f29b2a20e29440e49fa48117d227c218c5b8adf9837022ad804e3ac61e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:11.975448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:11.989273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f18d20-3357-4760-bd6b-7ad55390ed6a tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:12.003560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15c94d-ed3c-42ff-9c0b-1ac054b8b7be tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:12.006146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15c94d-ed3c-42ff-9c0b-1ac054b8b7be tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:12.007768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15c94d-ed3c-42ff-9c0b-1ac054b8b7be tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:12.012090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15c94d-ed3c-42ff-9c0b-1ac054b8b7be tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/credentials/2446ef1e85c9bea0070af2b57e629962f66b109fe8552a4810c9f642263fc26a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:12.015533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15c94d-ed3c-42ff-9c0b-1ac054b8b7be tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=2446ef1e85c9bea0070af2b57e629962f66b109fe8552a4810c9f642263fc26a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:12.020899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15c94d-ed3c-42ff-9c0b-1ac054b8b7be tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:12.065767 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1937/3845] 10.222.0.16 () {58 vars in 1338 bytes} [Sat Oct 15 21:08:11 2022] DELETE /identity/v3/credentials/2446ef1e85c9bea0070af2b57e629962f66b109fe8552a4810c9f642263fc26a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:12.070454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ef2f7-3702-4762-8dc8-852ad224cc2c tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:12.086001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ef2f7-3702-4762-8dc8-852ad224cc2c tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:12.101805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:12.102839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:12.103119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:12.103457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/projects/6110ebf7499044fe9d95efa39795e276` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:12.109707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6110ebf7499044fe9d95efa39795e276)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:12.112167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:12.256853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:12.280005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6110ebf7499044fe9d95efa39795e276', 'request_id': 'req-206f36e5-75f4-4d0a-93a5-3f0f81715a50'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:12.307893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6110ebf7499044fe9d95efa39795e276', 'request_id': 'req-206f36e5-75f4-4d0a-93a5-3f0f81715a50'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:12.320020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6110ebf7499044fe9d95efa39795e276', 'request_id': 'req-206f36e5-75f4-4d0a-93a5-3f0f81715a50'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:12.335290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] The token cache is being invalidate because project 6110ebf7499044fe9d95efa39795e276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:12.335701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:12.336822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:12.341314 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1909/3846] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:12 2022] DELETE /identity/v3/projects/6110ebf7499044fe9d95efa39795e276 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:12.343135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15c94d-ed3c-42ff-9c0b-1ac054b8b7be tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:12.345488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2f15c94d-ed3c-42ff-9c0b-1ac054b8b7be tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:12.379085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15c94d-ed3c-42ff-9c0b-1ac054b8b7be tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:12.570495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c0815cc1bc4333836dffef48aae959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:12.571252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:12.572260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:12.574198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] PATH_INFO: `/v3/projects/68b74b27b5f64ce6a9ff7a23a40959ab` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:12.577927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorizing `identity:delete_project(project_id=68b74b27b5f64ce6a9ff7a23a40959ab)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:12.582188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:12.658663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:12.723502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68b74b27b5f64ce6a9ff7a23a40959ab', 'request_id': 'req-94818db1-5134-4df2-a63e-7d77565629ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:12.747677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68b74b27b5f64ce6a9ff7a23a40959ab', 'request_id': 'req-94818db1-5134-4df2-a63e-7d77565629ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:12.772653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68b74b27b5f64ce6a9ff7a23a40959ab', 'request_id': 'req-94818db1-5134-4df2-a63e-7d77565629ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:12.815556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] The token cache is being invalidate because project 68b74b27b5f64ce6a9ff7a23a40959ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:12.815853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:12.819906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:12.829117 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1938/3847] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:12 2022] DELETE /identity/v3/projects/68b74b27b5f64ce6a9ff7a23a40959ab => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:12.831594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:12.833287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:12.982452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f36e5-75f4-4d0a-93a5-3f0f81715a50 tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:13.225160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:13.226410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:13.230239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:13.230239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] PATH_INFO: `/v3/users/e0c117b4333843c89f1f44c283d807cd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:13.237512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c117b4333843c89f1f44c283d807cd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:13.247802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:13.369539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c117b4333843c89f1f44c283d807cd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:13.385446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c117b4333843c89f1f44c283d807cd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:13.415234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c117b4333843c89f1f44c283d807cd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:13.427959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:13.437373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c117b4333843c89f1f44c283d807cd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:13.438754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bd94d6d-27ec-4cbc-959d-104b356f813c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c117b4333843c89f1f44c283d807cd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:13.449768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:13.451272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:13.451606 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1910/3848] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:12 2022] DELETE /identity/v3/users/e0c117b4333843c89f1f44c283d807cd => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:13.461882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16743935-1df9-472e-a990-e6dc6656570b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:13.462175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16743935-1df9-472e-a990-e6dc6656570b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:13.462434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16743935-1df9-472e-a990-e6dc6656570b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:13.482044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94818db1-5134-4df2-a63e-7d77565629ee tempest-CredentialsTestJSON-1313667812 tempest-CredentialsTestJSON-1313667812-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:13.521557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-16743935-1df9-472e-a990-e6dc6656570b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:13.522965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-16743935-1df9-472e-a990-e6dc6656570b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:13.637435 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1911/3849] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:13.641017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:13.641707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:13.641989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:13.642262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] PATH_INFO: `/v3/users/1d78f5fa0ab549cfba0a91e02d3379b4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:13.648863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16743935-1df9-472e-a990-e6dc6656570b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:13.649071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d78f5fa0ab549cfba0a91e02d3379b4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:13.650627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-16743935-1df9-472e-a990-e6dc6656570b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:13.652506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:13.788825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d78f5fa0ab549cfba0a91e02d3379b4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:13.804350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16743935-1df9-472e-a990-e6dc6656570b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:13.813937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115721b-3d74-4f18-8b7e-6b4d9b6e2cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:13.814811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115721b-3d74-4f18-8b7e-6b4d9b6e2cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:13.815255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115721b-3d74-4f18-8b7e-6b4d9b6e2cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:13.815536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115721b-3d74-4f18-8b7e-6b4d9b6e2cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:13.821907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d78f5fa0ab549cfba0a91e02d3379b4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:13.830588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115721b-3d74-4f18-8b7e-6b4d9b6e2cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:13.833931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115721b-3d74-4f18-8b7e-6b4d9b6e2cd6 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:13.841784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d78f5fa0ab549cfba0a91e02d3379b4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:13.870892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1912/3850] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:13.877400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d78f5fa0ab549cfba0a91e02d3379b4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:13.878183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d78f5fa0ab549cfba0a91e02d3379b4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:13.888499 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1939/3851] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:13 2022] DELETE /identity/v3/users/1d78f5fa0ab549cfba0a91e02d3379b4 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:14.341758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115721b-3d74-4f18-8b7e-6b4d9b6e2cd6 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:14.343846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2115721b-3d74-4f18-8b7e-6b4d9b6e2cd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:14.361368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115721b-3d74-4f18-8b7e-6b4d9b6e2cd6 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:14.485800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:14.486863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:14.486974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:14.487569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:14.487843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] PATH_INFO: `/v3/projects/6edf9c4fb3cc4276999855482ca29697` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:14.489211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:14.491937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6edf9c4fb3cc4276999855482ca29697)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:14.493907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:14.516563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77482262-1c62-4fe3-bbd0-1d2bf920aa55 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:14.535207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:14.536013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:14.536366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:14.536714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] PATH_INFO: `/v3/projects/3cd52aaddd184ef890002aacf0dd160c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:14.538268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cd52aaddd184ef890002aacf0dd160c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:14.540520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:14.595364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:14.605609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6edf9c4fb3cc4276999855482ca29697', 'request_id': 'req-39c51b38-49d7-4a7c-8e89-dc0403b8db74'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.619915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:14.621411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6edf9c4fb3cc4276999855482ca29697', 'request_id': 'req-39c51b38-49d7-4a7c-8e89-dc0403b8db74'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.632141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6edf9c4fb3cc4276999855482ca29697', 'request_id': 'req-39c51b38-49d7-4a7c-8e89-dc0403b8db74'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.650718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cd52aaddd184ef890002aacf0dd160c', 'request_id': 'req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.651906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] The token cache is being invalidate because project 6edf9c4fb3cc4276999855482ca29697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:14.652401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.653163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-39c51b38-49d7-4a7c-8e89-dc0403b8db74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.660041 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1913/3852] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:14 2022] DELETE /identity/v3/projects/6edf9c4fb3cc4276999855482ca29697 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:14.662316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7d89c-0893-42f5-8bd0-56f2b736cdf7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:14.662658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7d89c-0893-42f5-8bd0-56f2b736cdf7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:14.662884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7d89c-0893-42f5-8bd0-56f2b736cdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:14.667703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cd52aaddd184ef890002aacf0dd160c', 'request_id': 'req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.687750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cd52aaddd184ef890002aacf0dd160c', 'request_id': 'req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.691437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bfc7d89c-0893-42f5-8bd0-56f2b736cdf7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:14.693106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bfc7d89c-0893-42f5-8bd0-56f2b736cdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:14.702459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] The token cache is being invalidate because project 3cd52aaddd184ef890002aacf0dd160c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:14.702989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.703967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c07c7c3-03e5-4744-ba79-2f4ac6ba1a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.708423 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1940/3853] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:14 2022] DELETE /identity/v3/projects/3cd52aaddd184ef890002aacf0dd160c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:14.708423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b786-a668-4d42-9b60-a2ac0ddc5639 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:14.708423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b786-a668-4d42-9b60-a2ac0ddc5639 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:14.708423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b786-a668-4d42-9b60-a2ac0ddc5639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:14.734394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9d26b786-a668-4d42-9b60-a2ac0ddc5639 None None] MFA Rules not processed for user `dd615ca74887465b8e21b7f4011fb53f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:14.735767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9d26b786-a668-4d42-9b60-a2ac0ddc5639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:14.807857 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1914/3854] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:14.810503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7d89c-0893-42f5-8bd0-56f2b736cdf7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:14.813695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bfc7d89c-0893-42f5-8bd0-56f2b736cdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:14.831873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7d89c-0893-42f5-8bd0-56f2b736cdf7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:14.836968 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1941/3855] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:14.843685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123cba8-b3f9-41a5-b45e-d2dd2092ddd4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:14.843974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123cba8-b3f9-41a5-b45e-d2dd2092ddd4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:14.844245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123cba8-b3f9-41a5-b45e-d2dd2092ddd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:14.844351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:14.845012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:14.845313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:14.845582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] PATH_INFO: `/v3/users/e7c0815cc1bc4333836dffef48aae959` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:14.847664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c0815cc1bc4333836dffef48aae959)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:14.852006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:14.870996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c123cba8-b3f9-41a5-b45e-d2dd2092ddd4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:14.872400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c123cba8-b3f9-41a5-b45e-d2dd2092ddd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:14.888693 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1942/3856] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:14.890598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123cba8-b3f9-41a5-b45e-d2dd2092ddd4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:14.907562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123cba8-b3f9-41a5-b45e-d2dd2092ddd4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:14.914856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcf1bf-450a-40e8-802c-5212f06cc69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:14.915489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcf1bf-450a-40e8-802c-5212f06cc69f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:14.918053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcf1bf-450a-40e8-802c-5212f06cc69f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:14.918339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcf1bf-450a-40e8-802c-5212f06cc69f admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:14.922499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcf1bf-450a-40e8-802c-5212f06cc69f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:14.922831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcf1bf-450a-40e8-802c-5212f06cc69f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:14.926896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcf1bf-450a-40e8-802c-5212f06cc69f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:14.931992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c0815cc1bc4333836dffef48aae959'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.945100 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1943/3857] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:14.946750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcf1bf-450a-40e8-802c-5212f06cc69f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:14.956215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c0815cc1bc4333836dffef48aae959'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:14.965238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcf1bf-450a-40e8-802c-5212f06cc69f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:14.972964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cafbc2-6d85-40b8-91e1-d045f4a58f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:14.977011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cafbc2-6d85-40b8-91e1-d045f4a58f8a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:14.979962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cafbc2-6d85-40b8-91e1-d045f4a58f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:14.980648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cafbc2-6d85-40b8-91e1-d045f4a58f8a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:14.982093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cafbc2-6d85-40b8-91e1-d045f4a58f8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:14.991733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c0815cc1bc4333836dffef48aae959'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:15.004668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cafbc2-6d85-40b8-91e1-d045f4a58f8a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:15.010087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c0815cc1bc4333836dffef48aae959'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:15.012273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c0815cc1bc4333836dffef48aae959'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:15.024395 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1915/3858] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:14 2022] DELETE /identity/v3/users/e7c0815cc1bc4333836dffef48aae959 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:15.089917 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1944/3859] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:14 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:15.091476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:15.093919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:15.219286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:15.299685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d47696-c1c3-4809-b462-db1af5e08869 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:15.446630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893e178-2907-40af-8d6c-8db0bdf8c150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:15.447288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893e178-2907-40af-8d6c-8db0bdf8c150 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:15.447569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893e178-2907-40af-8d6c-8db0bdf8c150 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:15.447736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893e178-2907-40af-8d6c-8db0bdf8c150 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:15.448417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893e178-2907-40af-8d6c-8db0bdf8c150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:15.450563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893e178-2907-40af-8d6c-8db0bdf8c150 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:15.497498 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1916/3860] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:15 2022] POST /identity/v3/users => generated 290 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:15.499471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cafbc2-6d85-40b8-91e1-d045f4a58f8a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:15.514564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cafbc2-6d85-40b8-91e1-d045f4a58f8a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:15.529012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf8059-493d-4762-a9c9-563bd96eb2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:15.529798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf8059-493d-4762-a9c9-563bd96eb2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:15.529798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf8059-493d-4762-a9c9-563bd96eb2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:15.531119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf8059-493d-4762-a9c9-563bd96eb2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:15.531451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf8059-493d-4762-a9c9-563bd96eb2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:15.536235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf8059-493d-4762-a9c9-563bd96eb2d7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:15.562988 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1945/3861] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:15.568734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893e178-2907-40af-8d6c-8db0bdf8c150 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:15.598408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893e178-2907-40af-8d6c-8db0bdf8c150 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:15.618013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d16cbd-fe70-4152-b76e-e97fb4bd30e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:15.618895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d16cbd-fe70-4152-b76e-e97fb4bd30e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:15.619388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d16cbd-fe70-4152-b76e-e97fb4bd30e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:15.619924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d16cbd-fe70-4152-b76e-e97fb4bd30e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:15.621970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d16cbd-fe70-4152-b76e-e97fb4bd30e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:15.625278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d16cbd-fe70-4152-b76e-e97fb4bd30e9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:15.645572 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1917/3862] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:15.646271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf8059-493d-4762-a9c9-563bd96eb2d7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:15.661841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf8059-493d-4762-a9c9-563bd96eb2d7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:15.676288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82432e2-3d85-4cbb-92a9-343ea7ab93e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:15.677029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82432e2-3d85-4cbb-92a9-343ea7ab93e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:15.677358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82432e2-3d85-4cbb-92a9-343ea7ab93e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:15.677548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82432e2-3d85-4cbb-92a9-343ea7ab93e7 admin admin] PATH_INFO: `/v3/projects/93dac014c11c45029f5b8569b7f509d0/users/06e2c573568846d095c2a42a1df4e918/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:15.713980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82432e2-3d85-4cbb-92a9-343ea7ab93e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93dac014c11c45029f5b8569b7f509d0, user_id=06e2c573568846d095c2a42a1df4e918, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:15.716563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82432e2-3d85-4cbb-92a9-343ea7ab93e7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:15.739284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d16cbd-fe70-4152-b76e-e97fb4bd30e9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:15.740407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-36d16cbd-fe70-4152-b76e-e97fb4bd30e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:15.749620 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1946/3863] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:15 2022] PUT /identity/v3/projects/93dac014c11c45029f5b8569b7f509d0/users/06e2c573568846d095c2a42a1df4e918/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:15.755758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74838f68-3452-46d2-bac6-6fe7cbce575c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:15.756513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74838f68-3452-46d2-bac6-6fe7cbce575c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:15.757003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74838f68-3452-46d2-bac6-6fe7cbce575c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:15.766254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d16cbd-fe70-4152-b76e-e97fb4bd30e9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:15.824278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-74838f68-3452-46d2-bac6-6fe7cbce575c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:15.827693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-74838f68-3452-46d2-bac6-6fe7cbce575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:15.883773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:15.884592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:15.884966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:15.885365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] PATH_INFO: `/v3/projects/23372f4e6f8e4d3a83a0dacc0d6e5635` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:15.886617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23372f4e6f8e4d3a83a0dacc0d6e5635)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:15.892788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:15.894198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:15.933938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1947/3864] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:15.934946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74838f68-3452-46d2-bac6-6fe7cbce575c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:15.936769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-74838f68-3452-46d2-bac6-6fe7cbce575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:15.991991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'request_id': 'req-732bcab2-27dc-4f72-829b-63ef3d3e02f6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:16.002964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'request_id': 'req-732bcab2-27dc-4f72-829b-63ef3d3e02f6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:16.013062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23372f4e6f8e4d3a83a0dacc0d6e5635', 'request_id': 'req-732bcab2-27dc-4f72-829b-63ef3d3e02f6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:16.024915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:16.034218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] The token cache is being invalidate because project 23372f4e6f8e4d3a83a0dacc0d6e5635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:16.034403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:16.035117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:16.042048 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1918/3865] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:15 2022] DELETE /identity/v3/projects/23372f4e6f8e4d3a83a0dacc0d6e5635 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:16.042048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:16.042954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:16.075870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74838f68-3452-46d2-bac6-6fe7cbce575c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:16.149864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bcab2-27dc-4f72-829b-63ef3d3e02f6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:16.156609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d1203-5c31-49e9-a67f-371bd5250441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:16.157555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d1203-5c31-49e9-a67f-371bd5250441 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:16.160309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d1203-5c31-49e9-a67f-371bd5250441 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:16.160808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d1203-5c31-49e9-a67f-371bd5250441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:16.163795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ce6d1203-5c31-49e9-a67f-371bd5250441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:16.209502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908e1b3-6afa-47e3-bb2c-41a2adeee13f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:16.210262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908e1b3-6afa-47e3-bb2c-41a2adeee13f service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:16.210644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908e1b3-6afa-47e3-bb2c-41a2adeee13f service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:16.211209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908e1b3-6afa-47e3-bb2c-41a2adeee13f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:16.214393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2908e1b3-6afa-47e3-bb2c-41a2adeee13f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:16.235428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908e1b3-6afa-47e3-bb2c-41a2adeee13f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:16.236988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908e1b3-6afa-47e3-bb2c-41a2adeee13f service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:16.251328 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1948/3866] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:16.252647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eae6a3-31b2-439e-8eae-c24ecb2359ee None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:16.252988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eae6a3-31b2-439e-8eae-c24ecb2359ee None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:16.253289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eae6a3-31b2-439e-8eae-c24ecb2359ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:16.278838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-66eae6a3-31b2-439e-8eae-c24ecb2359ee None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:16.281011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-66eae6a3-31b2-439e-8eae-c24ecb2359ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:16.314068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d1203-5c31-49e9-a67f-371bd5250441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:16.316366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d1203-5c31-49e9-a67f-371bd5250441 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:16.332254 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1919/3867] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:16.389301 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1949/3868] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:16.396882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa68f9-6316-4da2-bb2b-97687fb287ac None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:16.397330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa68f9-6316-4da2-bb2b-97687fb287ac None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:16.397697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa68f9-6316-4da2-bb2b-97687fb287ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:16.446606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-faaa68f9-6316-4da2-bb2b-97687fb287ac None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:16.452202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-faaa68f9-6316-4da2-bb2b-97687fb287ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:16.471378 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1920/3869] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:16.476105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eae6a3-31b2-439e-8eae-c24ecb2359ee None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:16.489288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eae6a3-31b2-439e-8eae-c24ecb2359ee None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:16.495649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43738182-d292-482c-bc7c-8af8b573da4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:16.496846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738182-d292-482c-bc7c-8af8b573da4f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:16.497505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738182-d292-482c-bc7c-8af8b573da4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:16.498048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738182-d292-482c-bc7c-8af8b573da4f admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:16.499197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43738182-d292-482c-bc7c-8af8b573da4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:16.499772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43738182-d292-482c-bc7c-8af8b573da4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:16.501913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43738182-d292-482c-bc7c-8af8b573da4f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:16.519826 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1950/3870] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:16.524273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa68f9-6316-4da2-bb2b-97687fb287ac None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:16.538516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa68f9-6316-4da2-bb2b-97687fb287ac None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:16.544668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d72d63-fd52-46d1-9946-150d12fde9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:16.546479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d72d63-fd52-46d1-9946-150d12fde9ce admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:16.547322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d72d63-fd52-46d1-9946-150d12fde9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:16.547829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d72d63-fd52-46d1-9946-150d12fde9ce admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:16.548856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d72d63-fd52-46d1-9946-150d12fde9ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:16.551335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d72d63-fd52-46d1-9946-150d12fde9ce admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:16.611354 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1921/3871] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:16 2022] POST /identity/v3/projects => generated 365 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:16.615621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43738182-d292-482c-bc7c-8af8b573da4f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:16.646145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43738182-d292-482c-bc7c-8af8b573da4f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:16.777663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:16.907741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fde7b-36bb-4c71-b1dd-b164bca43043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:16.908418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fde7b-36bb-4c71-b1dd-b164bca43043 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:16.908703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fde7b-36bb-4c71-b1dd-b164bca43043 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:16.908975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fde7b-36bb-4c71-b1dd-b164bca43043 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:16.909648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fde7b-36bb-4c71-b1dd-b164bca43043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:16.911591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fde7b-36bb-4c71-b1dd-b164bca43043 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:16.968466 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1951/3872] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:16 2022] POST /identity/v3/users => generated 283 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:16.972191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d72d63-fd52-46d1-9946-150d12fde9ce admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:16.990685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d72d63-fd52-46d1-9946-150d12fde9ce admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:17.000150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7baee-4a42-4e9d-91f2-a6ef4f02a33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:17.000832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7baee-4a42-4e9d-91f2-a6ef4f02a33a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:17.001364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7baee-4a42-4e9d-91f2-a6ef4f02a33a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:17.001867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7baee-4a42-4e9d-91f2-a6ef4f02a33a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:17.002394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7baee-4a42-4e9d-91f2-a6ef4f02a33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:17.004930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7baee-4a42-4e9d-91f2-a6ef4f02a33a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:17.023009 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1922/3873] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:17.023574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fde7b-36bb-4c71-b1dd-b164bca43043 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:17.047681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fde7b-36bb-4c71-b1dd-b164bca43043 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:17.061919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b59ea1-b0b4-4ad9-8cbb-7cdf3cfe10a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:17.063102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b59ea1-b0b4-4ad9-8cbb-7cdf3cfe10a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:17.063627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b59ea1-b0b4-4ad9-8cbb-7cdf3cfe10a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:17.064049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b59ea1-b0b4-4ad9-8cbb-7cdf3cfe10a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:17.064854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b59ea1-b0b4-4ad9-8cbb-7cdf3cfe10a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:17.066670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b59ea1-b0b4-4ad9-8cbb-7cdf3cfe10a1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:17.080812 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1952/3874] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:08:17.084847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7baee-4a42-4e9d-91f2-a6ef4f02a33a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:17.103924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7baee-4a42-4e9d-91f2-a6ef4f02a33a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:17.112240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccfbf8-7833-4d61-bb3d-895d0afcd789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:17.112918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccfbf8-7833-4d61-bb3d-895d0afcd789 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:17.113356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccfbf8-7833-4d61-bb3d-895d0afcd789 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:17.113554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccfbf8-7833-4d61-bb3d-895d0afcd789 admin admin] PATH_INFO: `/v3/projects/c2555efc656846f49ac0c2ab65c05d98/users/32f8a4e461644978be5fed785315974a/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:17.142687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccfbf8-7833-4d61-bb3d-895d0afcd789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2555efc656846f49ac0c2ab65c05d98, user_id=32f8a4e461644978be5fed785315974a, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:17.145912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccfbf8-7833-4d61-bb3d-895d0afcd789 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:17.164799 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1923/3875] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:17 2022] PUT /identity/v3/projects/c2555efc656846f49ac0c2ab65c05d98/users/32f8a4e461644978be5fed785315974a/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:17.175895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4fe44-7ac8-41f9-9490-274843b9d797 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:17.176190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4fe44-7ac8-41f9-9490-274843b9d797 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:17.176450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4fe44-7ac8-41f9-9490-274843b9d797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:17.214734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d0a4fe44-7ac8-41f9-9490-274843b9d797 None None] MFA Rules not processed for user `32f8a4e461644978be5fed785315974a`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:17.216596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d0a4fe44-7ac8-41f9-9490-274843b9d797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:17.386916 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1953/3876] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:17.392943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccfbf8-7833-4d61-bb3d-895d0afcd789 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:17.496503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:17.551822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccfbf8-7833-4d61-bb3d-895d0afcd789 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:17.572528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94782a49-d608-4e3c-aea9-5e3bd11f13f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:17.573182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94782a49-d608-4e3c-aea9-5e3bd11f13f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:17.577631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94782a49-d608-4e3c-aea9-5e3bd11f13f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:17.578997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94782a49-d608-4e3c-aea9-5e3bd11f13f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:17.582240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94782a49-d608-4e3c-aea9-5e3bd11f13f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:17.584288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94782a49-d608-4e3c-aea9-5e3bd11f13f3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:17.683892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1924/3877] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:17 2022] POST /identity/v3/projects => generated 367 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:17.684526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4fe44-7ac8-41f9-9490-274843b9d797 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:17.703086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4fe44-7ac8-41f9-9490-274843b9d797 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:17.854274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa20de2-7cf6-482f-b850-994313a40776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:17.855773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa20de2-7cf6-482f-b850-994313a40776 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:17.856109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa20de2-7cf6-482f-b850-994313a40776 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:17.856517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa20de2-7cf6-482f-b850-994313a40776 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:17.858038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa20de2-7cf6-482f-b850-994313a40776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:17.862619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa20de2-7cf6-482f-b850-994313a40776 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:17.949082 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1954/3878] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:17 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:17.952926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94782a49-d608-4e3c-aea9-5e3bd11f13f3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:17.994065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94782a49-d608-4e3c-aea9-5e3bd11f13f3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:18.001438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160519a6-e249-4875-884c-926fdc4a2bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:18.002065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160519a6-e249-4875-884c-926fdc4a2bce admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:18.002341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160519a6-e249-4875-884c-926fdc4a2bce admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:18.002608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160519a6-e249-4875-884c-926fdc4a2bce admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:18.003247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160519a6-e249-4875-884c-926fdc4a2bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:18.004849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160519a6-e249-4875-884c-926fdc4a2bce admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:18.034588 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1925/3879] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:18.038435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa20de2-7cf6-482f-b850-994313a40776 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:18.055419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa20de2-7cf6-482f-b850-994313a40776 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:18.071688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ac3c3-f4a6-4660-bb6b-dad50218cac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:18.072290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ac3c3-f4a6-4660-bb6b-dad50218cac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:18.072566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ac3c3-f4a6-4660-bb6b-dad50218cac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:18.072829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ac3c3-f4a6-4660-bb6b-dad50218cac7 admin admin] PATH_INFO: `/v3/projects/5d06616134974ebea540c734183e78e1/users/fa211f5d30dd4f50976ebe5093cce89a/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:18.111886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ac3c3-f4a6-4660-bb6b-dad50218cac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d06616134974ebea540c734183e78e1, user_id=fa211f5d30dd4f50976ebe5093cce89a, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:18.114854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ac3c3-f4a6-4660-bb6b-dad50218cac7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:18.146913 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1955/3880] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:18 2022] PUT /identity/v3/projects/5d06616134974ebea540c734183e78e1/users/fa211f5d30dd4f50976ebe5093cce89a/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:18.150833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160519a6-e249-4875-884c-926fdc4a2bce admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:18.169851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160519a6-e249-4875-884c-926fdc4a2bce admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:18.348920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53bb56-dc94-4271-92df-53138596b45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:18.349810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53bb56-dc94-4271-92df-53138596b45b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:18.350184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53bb56-dc94-4271-92df-53138596b45b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:18.350828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53bb56-dc94-4271-92df-53138596b45b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:18.351620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53bb56-dc94-4271-92df-53138596b45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:18.355522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53bb56-dc94-4271-92df-53138596b45b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:18.375829 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1926/3881] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:18.380267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ac3c3-f4a6-4660-bb6b-dad50218cac7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:18.396252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ac3c3-f4a6-4660-bb6b-dad50218cac7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:18.411501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1fe7d-2fcd-4cd0-b49c-ebbb784aef6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:18.417207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1fe7d-2fcd-4cd0-b49c-ebbb784aef6f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:18.421907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1fe7d-2fcd-4cd0-b49c-ebbb784aef6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:18.421907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1fe7d-2fcd-4cd0-b49c-ebbb784aef6f admin admin] PATH_INFO: `/v3/projects/5d06616134974ebea540c734183e78e1/users/fa211f5d30dd4f50976ebe5093cce89a/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:18.443641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1fe7d-2fcd-4cd0-b49c-ebbb784aef6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d06616134974ebea540c734183e78e1, user_id=fa211f5d30dd4f50976ebe5093cce89a, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:18.448700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1fe7d-2fcd-4cd0-b49c-ebbb784aef6f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:18.460997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:18.486267 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1956/3882] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:18 2022] PUT /identity/v3/projects/5d06616134974ebea540c734183e78e1/users/fa211f5d30dd4f50976ebe5093cce89a/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:18.500251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be3bf6-5ae4-468a-8c5d-e58c6dcc1a8a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:18.502676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be3bf6-5ae4-468a-8c5d-e58c6dcc1a8a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:18.502952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be3bf6-5ae4-468a-8c5d-e58c6dcc1a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:18.561818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-90be3bf6-5ae4-468a-8c5d-e58c6dcc1a8a None None] MFA Rules not processed for user `fa211f5d30dd4f50976ebe5093cce89a`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:18.564324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-90be3bf6-5ae4-468a-8c5d-e58c6dcc1a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:18.784349 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1927/3883] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:18.786840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1fe7d-2fcd-4cd0-b49c-ebbb784aef6f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:18.803234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1fe7d-2fcd-4cd0-b49c-ebbb784aef6f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:18.841651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d25ce-a0b0-42cf-b3a7-d10470299b93 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:18.842219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d25ce-a0b0-42cf-b3a7-d10470299b93 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:18.842508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d25ce-a0b0-42cf-b3a7-d10470299b93 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:18.848016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d25ce-a0b0-42cf-b3a7-d10470299b93 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:18.848744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d25ce-a0b0-42cf-b3a7-d10470299b93 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:18.854095 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-d90d25ce-a0b0-42cf-b3a7-d10470299b93 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:18.854095 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:18.854095 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:18.855710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d25ce-a0b0-42cf-b3a7-d10470299b93 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:18.937871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be3bf6-5ae4-468a-8c5d-e58c6dcc1a8a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:18.946725 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1957/3884] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Oct 15 21:08:18 2022] POST /identity/v3/domains => generated 276 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:18.949612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be3bf6-5ae4-468a-8c5d-e58c6dcc1a8a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:19.066631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:19.117494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1740b67-4dc8-4932-a8ed-a3ef4bd83d05 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:19.119794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740b67-4dc8-4932-a8ed-a3ef4bd83d05 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:19.120569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740b67-4dc8-4932-a8ed-a3ef4bd83d05 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:19.121622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740b67-4dc8-4932-a8ed-a3ef4bd83d05 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:19.124039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1740b67-4dc8-4932-a8ed-a3ef4bd83d05 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:19.127900 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-f1740b67-4dc8-4932-a8ed-a3ef4bd83d05 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:19.127900 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:19.127900 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:19.129687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1740b67-4dc8-4932-a8ed-a3ef4bd83d05 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:19.167613 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1928/3885] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Oct 15 21:08:18 2022] POST /identity/v3/domains => generated 274 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:19.171985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d25ce-a0b0-42cf-b3a7-d10470299b93 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:19.184823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d25ce-a0b0-42cf-b3a7-d10470299b93 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:19.430435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcb03d-ebe5-4ff9-9c77-2cd744aab30b tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:19.431715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb03d-ebe5-4ff9-9c77-2cd744aab30b tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:19.433433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb03d-ebe5-4ff9-9c77-2cd744aab30b tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:19.433823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb03d-ebe5-4ff9-9c77-2cd744aab30b tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:19.438703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdcb03d-ebe5-4ff9-9c77-2cd744aab30b tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:19.438703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdcb03d-ebe5-4ff9-9c77-2cd744aab30b tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:19.486210 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1958/3886] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Oct 15 21:08:19 2022] POST /identity/v3/domains => generated 277 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:19.489415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1740b67-4dc8-4932-a8ed-a3ef4bd83d05 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:19.504793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1740b67-4dc8-4932-a8ed-a3ef4bd83d05 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:19.708338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c2856-bdac-4515-89ff-29fc50a9e8e4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:19.708965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c2856-bdac-4515-89ff-29fc50a9e8e4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:19.709413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c2856-bdac-4515-89ff-29fc50a9e8e4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:19.709757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c2856-bdac-4515-89ff-29fc50a9e8e4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:19.710890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c2856-bdac-4515-89ff-29fc50a9e8e4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:19.712802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c2856-bdac-4515-89ff-29fc50a9e8e4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:19.748658 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1929/3887] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Oct 15 21:08:19 2022] POST /identity/v3/domains => generated 277 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:19.752696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcb03d-ebe5-4ff9-9c77-2cd744aab30b tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:19.773707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcb03d-ebe5-4ff9-9c77-2cd744aab30b tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:19.941885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:19.943635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:19.944073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:19.944491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/5d8c6af5aa2845daae7cd997b0d00d83` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:19.968075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=5d8c6af5aa2845daae7cd997b0d00d83)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:19.969438 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:19.969438 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:19.969438 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:19.970674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:20.032836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '5d8c6af5aa2845daae7cd997b0d00d83'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.050673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '5d8c6af5aa2845daae7cd997b0d00d83'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.275099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=3,keystone:SELECT=94,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:20.286449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8c6af5aa2845daae7cd997b0d00d83', 'request_id': 'req-76ef010f-a31d-4b97-b11e-c03270f9f54a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.302061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8c6af5aa2845daae7cd997b0d00d83', 'request_id': 'req-76ef010f-a31d-4b97-b11e-c03270f9f54a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.325517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8c6af5aa2845daae7cd997b0d00d83', 'request_id': 'req-76ef010f-a31d-4b97-b11e-c03270f9f54a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.359579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] The token cache is being invalidate because project 5d8c6af5aa2845daae7cd997b0d00d83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:20.360387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.362133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.394526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '5d8c6af5aa2845daae7cd997b0d00d83', 'request_id': 'req-76ef010f-a31d-4b97-b11e-c03270f9f54a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.395652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '5d8c6af5aa2845daae7cd997b0d00d83', 'request_id': 'req-76ef010f-a31d-4b97-b11e-c03270f9f54a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.396771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '5d8c6af5aa2845daae7cd997b0d00d83', 'request_id': 'req-76ef010f-a31d-4b97-b11e-c03270f9f54a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.417937 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1959/3888] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:19 2022] DELETE /identity/v3/domains/5d8c6af5aa2845daae7cd997b0d00d83 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:20.423033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c2856-bdac-4515-89ff-29fc50a9e8e4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:20.425283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-391c2856-bdac-4515-89ff-29fc50a9e8e4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:20.441428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c2856-bdac-4515-89ff-29fc50a9e8e4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:20.546036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667df73-4bb7-4a54-9a68-ee6b645dbe59 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:20.547464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667df73-4bb7-4a54-9a68-ee6b645dbe59 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:20.557645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667df73-4bb7-4a54-9a68-ee6b645dbe59 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:20.558268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667df73-4bb7-4a54-9a68-ee6b645dbe59 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:20.559191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667df73-4bb7-4a54-9a68-ee6b645dbe59 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:20.562691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667df73-4bb7-4a54-9a68-ee6b645dbe59 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:20.590354 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1930/3889] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:20 2022] POST /identity/v3/domains => generated 248 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:20.594157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:20.616489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef010f-a31d-4b97-b11e-c03270f9f54a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:20.764869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:20.767389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:20.767839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:20.769525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/a8cdcb4a2c414d7bb0ccdc2ef087035c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:20.805796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=a8cdcb4a2c414d7bb0ccdc2ef087035c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:20.808675 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:20.808675 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:20.808675 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:20.816965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:20.880893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:20.880893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.143084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.143740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.144633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf272c-27d4-4267-acbf-f3f243318bd3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:21.145198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf272c-27d4-4267-acbf-f3f243318bd3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:21.146033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf272c-27d4-4267-acbf-f3f243318bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:21.151168 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1960/3890] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:20 2022] PATCH /identity/v3/domains/a8cdcb4a2c414d7bb0ccdc2ef087035c => generated 249 bytes in 557 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:21.155672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:21.157954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:21.193264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-eedf272c-27d4-4267-acbf-f3f243318bd3 None None] MFA Rules not processed for user `461a5c1e334a47eb9d70908227cca0b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:21.194766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eedf272c-27d4-4267-acbf-f3f243318bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:21.337396 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1931/3891] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:21.345539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cde90-7f3c-4123-9098-dbdbaac93616 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:21.364903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:21.365593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:21.365874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:21.366148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/a8cdcb4a2c414d7bb0ccdc2ef087035c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:21.382862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a8cdcb4a2c414d7bb0ccdc2ef087035c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:21.385151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:21.386939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:21.419213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.426186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.511478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c', 'request_id': 'req-d3ab2801-2fff-404d-8ebf-e82209265c40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.533266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c', 'request_id': 'req-d3ab2801-2fff-404d-8ebf-e82209265c40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.558338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c', 'request_id': 'req-d3ab2801-2fff-404d-8ebf-e82209265c40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.589584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] The token cache is being invalidate because project a8cdcb4a2c414d7bb0ccdc2ef087035c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:21.589812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.592699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.615037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c', 'request_id': 'req-d3ab2801-2fff-404d-8ebf-e82209265c40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.616190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c', 'request_id': 'req-d3ab2801-2fff-404d-8ebf-e82209265c40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.619468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a8cdcb4a2c414d7bb0ccdc2ef087035c', 'request_id': 'req-d3ab2801-2fff-404d-8ebf-e82209265c40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:21.643254 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1961/3892] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:21 2022] DELETE /identity/v3/domains/a8cdcb4a2c414d7bb0ccdc2ef087035c => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:21.644954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf272c-27d4-4267-acbf-f3f243318bd3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:21.669460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf272c-27d4-4267-acbf-f3f243318bd3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:21.687672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:21.854891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a9ace-1673-4633-8478-f53f81e6ec80 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:21.855534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a9ace-1673-4633-8478-f53f81e6ec80 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:21.855805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a9ace-1673-4633-8478-f53f81e6ec80 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:21.856073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a9ace-1673-4633-8478-f53f81e6ec80 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:21.861668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a9ace-1673-4633-8478-f53f81e6ec80 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:21.862754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a9ace-1673-4633-8478-f53f81e6ec80 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:21.898400 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1932/3893] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:21 2022] POST /identity/v3/domains => generated 278 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:21.903002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:21.905568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:21.951916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab2801-2fff-404d-8ebf-e82209265c40 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:22.214946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:22.215700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:22.215881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:22.216153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/526d5a411f494fc5a892cccdb68d0def` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:22.247655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=526d5a411f494fc5a892cccdb68d0def)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:22.249392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:22.293031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '526d5a411f494fc5a892cccdb68d0def'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.293606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '526d5a411f494fc5a892cccdb68d0def'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.383522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '526d5a411f494fc5a892cccdb68d0def'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.384113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '526d5a411f494fc5a892cccdb68d0def'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.390340 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1962/3894] 10.222.0.16 () {60 vars in 1250 bytes} [Sat Oct 15 21:08:21 2022] PATCH /identity/v3/domains/526d5a411f494fc5a892cccdb68d0def => generated 279 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:22.391537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a9ace-1673-4633-8478-f53f81e6ec80 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:22.393408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ce9a9ace-1673-4633-8478-f53f81e6ec80 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:22.415230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a9ace-1673-4633-8478-f53f81e6ec80 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:22.569980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed264d19-8398-46c3-9f27-ae982148ce15 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:22.570658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed264d19-8398-46c3-9f27-ae982148ce15 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:22.570920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed264d19-8398-46c3-9f27-ae982148ce15 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:22.571193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed264d19-8398-46c3-9f27-ae982148ce15 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/526d5a411f494fc5a892cccdb68d0def` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:22.598117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed264d19-8398-46c3-9f27-ae982148ce15 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=526d5a411f494fc5a892cccdb68d0def)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:22.600302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed264d19-8398-46c3-9f27-ae982148ce15 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:22.606214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1933/3895] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:08:22 2022] GET /identity/v3/domains/526d5a411f494fc5a892cccdb68d0def => generated 279 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:22.606934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:22.614900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:22.623517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a366936-145f-448e-8344-2451e134923c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:22.635624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:22.637423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:22.637946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:22.638443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/526d5a411f494fc5a892cccdb68d0def` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:22.640840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=526d5a411f494fc5a892cccdb68d0def)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:22.643100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:22.672333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '526d5a411f494fc5a892cccdb68d0def'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.678163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '526d5a411f494fc5a892cccdb68d0def'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.775309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '526d5a411f494fc5a892cccdb68d0def', 'request_id': 'req-66fdd060-3fad-4638-a937-e2682bf6a900'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.795501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '526d5a411f494fc5a892cccdb68d0def', 'request_id': 'req-66fdd060-3fad-4638-a937-e2682bf6a900'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.825101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '526d5a411f494fc5a892cccdb68d0def', 'request_id': 'req-66fdd060-3fad-4638-a937-e2682bf6a900'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.870712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] The token cache is being invalidate because project 526d5a411f494fc5a892cccdb68d0def was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:22.870924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.871610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.887792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '526d5a411f494fc5a892cccdb68d0def', 'request_id': 'req-66fdd060-3fad-4638-a937-e2682bf6a900'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.889161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '526d5a411f494fc5a892cccdb68d0def', 'request_id': 'req-66fdd060-3fad-4638-a937-e2682bf6a900'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.890098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '526d5a411f494fc5a892cccdb68d0def', 'request_id': 'req-66fdd060-3fad-4638-a937-e2682bf6a900'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:22.905971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed264d19-8398-46c3-9f27-ae982148ce15 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:22.908712 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1963/3896] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:22 2022] DELETE /identity/v3/domains/526d5a411f494fc5a892cccdb68d0def => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:22.909890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ed264d19-8398-46c3-9f27-ae982148ce15 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:22.927546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed264d19-8398-46c3-9f27-ae982148ce15 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:23.058767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ac528-4603-4a4f-aeec-16f0806b5443 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:23.063065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ac528-4603-4a4f-aeec-16f0806b5443 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:23.063678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ac528-4603-4a4f-aeec-16f0806b5443 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:23.065755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ac528-4603-4a4f-aeec-16f0806b5443 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:23.066741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ac528-4603-4a4f-aeec-16f0806b5443 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:23.071119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ac528-4603-4a4f-aeec-16f0806b5443 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:23.189060 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1934/3897] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 21:08:22 2022] GET /identity/v3/domains => generated 1345 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:23.197287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:23.217900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdd060-3fad-4638-a937-e2682bf6a900 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:23.229873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a0c5c-fb34-4e72-b069-ce13cf7e35df tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:23.230842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a0c5c-fb34-4e72-b069-ce13cf7e35df tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:23.231296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a0c5c-fb34-4e72-b069-ce13cf7e35df tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:23.231767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a0c5c-fb34-4e72-b069-ce13cf7e35df tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/526d5a411f494fc5a892cccdb68d0def` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:23.258172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a0c5c-fb34-4e72-b069-ce13cf7e35df tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-6f7a0c5c-fb34-4e72-b069-ce13cf7e35df tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Could not find domain: 526d5a411f494fc5a892cccdb68d0def.: keystone.exception.DomainNotFound: Could not find domain: 526d5a411f494fc5a892cccdb68d0def. Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 526d5a411f494fc5a892cccdb68d0def. Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:23.326844 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:23.327973 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 526d5a411f494fc5a892cccdb68d0def. Oct 15 21:08:23.328903 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:23.331238 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1964/3898] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:23 2022] PATCH /identity/v3/domains/526d5a411f494fc5a892cccdb68d0def => generated 112 bytes in 134 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:08:23.335244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ac528-4603-4a4f-aeec-16f0806b5443 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:23.367965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ac528-4603-4a4f-aeec-16f0806b5443 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:23.391388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab1698-dfde-4d1f-b83f-498b95eaad07 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:23.391955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab1698-dfde-4d1f-b83f-498b95eaad07 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:23.392230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab1698-dfde-4d1f-b83f-498b95eaad07 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:23.392495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab1698-dfde-4d1f-b83f-498b95eaad07 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:23.393209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab1698-dfde-4d1f-b83f-498b95eaad07 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:23.395706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab1698-dfde-4d1f-b83f-498b95eaad07 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:23.480231 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1935/3899] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:23 2022] POST /identity/v3/domains => generated 274 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:23.481049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a0c5c-fb34-4e72-b069-ce13cf7e35df tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:23.510675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a0c5c-fb34-4e72-b069-ce13cf7e35df tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:23.677839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ef385-f442-4d76-a764-3ed45381964a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:23.678805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ef385-f442-4d76-a764-3ed45381964a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:23.679180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ef385-f442-4d76-a764-3ed45381964a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:23.679553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ef385-f442-4d76-a764-3ed45381964a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:23.680522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ef385-f442-4d76-a764-3ed45381964a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:23.685074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ef385-f442-4d76-a764-3ed45381964a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:23.741653 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1965/3900] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:23 2022] POST /identity/v3/users => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:23.743291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab1698-dfde-4d1f-b83f-498b95eaad07 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:23.761812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab1698-dfde-4d1f-b83f-498b95eaad07 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:23.768343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db04598-baf8-422c-8a7d-378b0db2bc22 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:23.769364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db04598-baf8-422c-8a7d-378b0db2bc22 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:23.769889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db04598-baf8-422c-8a7d-378b0db2bc22 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:23.770414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db04598-baf8-422c-8a7d-378b0db2bc22 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/groups` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:23.771273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db04598-baf8-422c-8a7d-378b0db2bc22 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:23.772525 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-1db04598-baf8-422c-8a7d-378b0db2bc22 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:23.772525 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:23.772525 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:23.776773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db04598-baf8-422c-8a7d-378b0db2bc22 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:23.803112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ef385-f442-4d76-a764-3ed45381964a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:23.813085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1936/3901] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:08:23 2022] POST /identity/v3/groups => generated 324 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:23.818689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ef385-f442-4d76-a764-3ed45381964a tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:23.827057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:23.828495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:23.829207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:23.829694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/a01c61f6f9714b7a8535d330d85d09d2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:23.831979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=a01c61f6f9714b7a8535d330d85d09d2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:23.834688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:23.853488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:23.877406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:23.878711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:23.923072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:23.924744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:23.933220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db04598-baf8-422c-8a7d-378b0db2bc22 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:23.934213 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1966/3902] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:23 2022] PATCH /identity/v3/domains/a01c61f6f9714b7a8535d330d85d09d2 => generated 275 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:08:23.935030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1db04598-baf8-422c-8a7d-378b0db2bc22 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:23.963646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:23.978485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db04598-baf8-422c-8a7d-378b0db2bc22 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:24.073021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:24.074068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:24.074512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:24.074943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/a01c61f6f9714b7a8535d330d85d09d2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:24.089572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a01c61f6f9714b7a8535d330d85d09d2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:24.090948 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:24.090948 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:24.090948 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:24.092736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:24.142802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.441659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b167202199d4f5e98fefb920371ec9a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.469430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b167202199d4f5e98fefb920371ec9a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.512854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=12 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:24.521207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b167202199d4f5e98fefb920371ec9a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.549958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b167202199d4f5e98fefb920371ec9a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.550667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b167202199d4f5e98fefb920371ec9a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.558069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.657584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2', 'request_id': 'req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.672125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2', 'request_id': 'req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.681801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2', 'request_id': 'req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.725394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] The token cache is being invalidate because project a01c61f6f9714b7a8535d330d85d09d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:24.725837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.729366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.745993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2', 'request_id': 'req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.749465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2', 'request_id': 'req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.749627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a01c61f6f9714b7a8535d330d85d09d2', 'request_id': 'req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:24.767317 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1937/3903] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:23 2022] DELETE /identity/v3/domains/a01c61f6f9714b7a8535d330d85d09d2 => generated 0 bytes in 835 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:24.771235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:24.794903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e66a4-a1a0-4eb6-a1eb-2e9cedcc90a4 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:24.959879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf36cf-fe19-4745-a0fb-0dfd5b271360 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:24.961997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf36cf-fe19-4745-a0fb-0dfd5b271360 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:24.962279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf36cf-fe19-4745-a0fb-0dfd5b271360 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:24.962626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf36cf-fe19-4745-a0fb-0dfd5b271360 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/a01c61f6f9714b7a8535d330d85d09d2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:24.973565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf36cf-fe19-4745-a0fb-0dfd5b271360 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=a01c61f6f9714b7a8535d330d85d09d2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:24.976311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf36cf-fe19-4745-a0fb-0dfd5b271360 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:24.998275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.000215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-e9bf36cf-fe19-4745-a0fb-0dfd5b271360 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Could not find domain: a01c61f6f9714b7a8535d330d85d09d2.: keystone.exception.DomainNotFound: Could not find domain: a01c61f6f9714b7a8535d330d85d09d2. Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a01c61f6f9714b7a8535d330d85d09d2. Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:25.006384 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:08:25.007488 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: a01c61f6f9714b7a8535d330d85d09d2. Oct 15 21:08:25.008637 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:25.027103 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1967/3904] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:08:24 2022] GET /identity/v3/domains/a01c61f6f9714b7a8535d330d85d09d2 => generated 112 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:08:25.034117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf36cf-fe19-4745-a0fb-0dfd5b271360 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.043067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e9bf36cf-fe19-4745-a0fb-0dfd5b271360 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:25.074769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf36cf-fe19-4745-a0fb-0dfd5b271360 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.094341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1743f-eae4-4c0d-b54b-23a8b46a700d tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.095021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1743f-eae4-4c0d-b54b-23a8b46a700d tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.095277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1743f-eae4-4c0d-b54b-23a8b46a700d tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.095564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1743f-eae4-4c0d-b54b-23a8b46a700d tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/users/7b167202199d4f5e98fefb920371ec9a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.126375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1743f-eae4-4c0d-b54b-23a8b46a700d tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:get_user(user_id=7b167202199d4f5e98fefb920371ec9a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:25.128380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1743f-eae4-4c0d-b54b-23a8b46a700d tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-77d1743f-eae4-4c0d-b54b-23a8b46a700d tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Could not find user: 7b167202199d4f5e98fefb920371ec9a.: keystone.exception.UserNotFound: Could not find user: 7b167202199d4f5e98fefb920371ec9a. Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 21:08:25.168364 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:08:25.170141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 7b167202199d4f5e98fefb920371ec9a. Oct 15 21:08:25.171071 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:25.174728 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1968/3905] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:08:25 2022] GET /identity/v3/users/7b167202199d4f5e98fefb920371ec9a => generated 110 bytes in 141 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:08:25.176858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabaf51-647f-4020-8ad9-fd0830e2e6f0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.178644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1743f-eae4-4c0d-b54b-23a8b46a700d tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.194400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1743f-eae4-4c0d-b54b-23a8b46a700d tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.197314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2d86e-25b9-4cde-9fe8-a6bf542ff28f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.198230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2d86e-25b9-4cde-9fe8-a6bf542ff28f service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.198712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2d86e-25b9-4cde-9fe8-a6bf542ff28f service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.199179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2d86e-25b9-4cde-9fe8-a6bf542ff28f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.202537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c0c2d86e-25b9-4cde-9fe8-a6bf542ff28f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:25.216014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73633f44-3136-491b-984b-44c372749bea tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.217170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73633f44-3136-491b-984b-44c372749bea tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.217753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73633f44-3136-491b-984b-44c372749bea tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.218229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73633f44-3136-491b-984b-44c372749bea tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/groups/dce00e0dd21f4283813f460d597f5a16` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.224485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2d86e-25b9-4cde-9fe8-a6bf542ff28f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:25.226698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73633f44-3136-491b-984b-44c372749bea tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:get_group(group_id=dce00e0dd21f4283813f460d597f5a16)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:25.227414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2d86e-25b9-4cde-9fe8-a6bf542ff28f service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:25.229159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73633f44-3136-491b-984b-44c372749bea tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:25.246582 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1938/3906] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-73633f44-3136-491b-984b-44c372749bea tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Could not find group: dce00e0dd21f4283813f460d597f5a16.: keystone.exception.GroupNotFound: Could not find group: dce00e0dd21f4283813f460d597f5a16. Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 58, in get Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_group(group_id) Oct 15 21:08:25.258615 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 70, in _get_group Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.identity_api.get_group(group_id)) Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:08:25.259503 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: dce00e0dd21f4283813f460d597f5a16. Oct 15 21:08:25.260402 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:25.264073 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1969/3907] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:25 2022] GET /identity/v3/groups/dce00e0dd21f4283813f460d597f5a16 => generated 111 bytes in 85 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:08:25.268350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2d86e-25b9-4cde-9fe8-a6bf542ff28f service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.283721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2d86e-25b9-4cde-9fe8-a6bf542ff28f service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.303298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa96bb-c8da-4207-8662-23e39769439c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.303937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa96bb-c8da-4207-8662-23e39769439c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.304203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa96bb-c8da-4207-8662-23e39769439c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.304528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa96bb-c8da-4207-8662-23e39769439c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/groups/dce00e0dd21f4283813f460d597f5a16` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.311170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa96bb-c8da-4207-8662-23e39769439c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-15fa96bb-c8da-4207-8662-23e39769439c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Could not find group: dce00e0dd21f4283813f460d597f5a16.: keystone.exception.GroupNotFound: Could not find group: dce00e0dd21f4283813f460d597f5a16. Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 133, in delete Oct 15 21:08:25.316893 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_group') Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:08:25.318103 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: dce00e0dd21f4283813f460d597f5a16. Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:25.322362 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1939/3908] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:08:25 2022] DELETE /identity/v3/groups/dce00e0dd21f4283813f460d597f5a16 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:08:25.323404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73633f44-3136-491b-984b-44c372749bea tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.333088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73633f44-3136-491b-984b-44c372749bea tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.352235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12616804-eab4-4eef-aeb2-addeeceac0ad tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.352841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12616804-eab4-4eef-aeb2-addeeceac0ad tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.353141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12616804-eab4-4eef-aeb2-addeeceac0ad tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.353412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12616804-eab4-4eef-aeb2-addeeceac0ad tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/users/7b167202199d4f5e98fefb920371ec9a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.373954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12616804-eab4-4eef-aeb2-addeeceac0ad tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:delete_user(user_id=7b167202199d4f5e98fefb920371ec9a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:25.375898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12616804-eab4-4eef-aeb2-addeeceac0ad tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-12616804-eab4-4eef-aeb2-addeeceac0ad tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Could not find user: 7b167202199d4f5e98fefb920371ec9a.: keystone.exception.UserNotFound: Could not find user: 7b167202199d4f5e98fefb920371ec9a. Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 275, in delete Oct 15 21:08:25.397055 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.identity_api.delete_user(user_id) Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1207, in delete_user Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application user_old = self.get_user(user_id) Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:08:25.398149 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 7b167202199d4f5e98fefb920371ec9a. Oct 15 21:08:25.399191 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:25.406476 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1970/3909] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:25 2022] DELETE /identity/v3/users/7b167202199d4f5e98fefb920371ec9a => generated 110 bytes in 84 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:08:25.406788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa96bb-c8da-4207-8662-23e39769439c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.436399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa96bb-c8da-4207-8662-23e39769439c tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.462598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a3e96-a77f-46c9-bc90-0fcfdc8ecb8e tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.463245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a3e96-a77f-46c9-bc90-0fcfdc8ecb8e tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.463515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a3e96-a77f-46c9-bc90-0fcfdc8ecb8e tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.463783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a3e96-a77f-46c9-bc90-0fcfdc8ecb8e tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/a01c61f6f9714b7a8535d330d85d09d2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.485372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a3e96-a77f-46c9-bc90-0fcfdc8ecb8e tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-5d1a3e96-a77f-46c9-bc90-0fcfdc8ecb8e tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Could not find domain: a01c61f6f9714b7a8535d330d85d09d2.: keystone.exception.DomainNotFound: Could not find domain: a01c61f6f9714b7a8535d330d85d09d2. Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a01c61f6f9714b7a8535d330d85d09d2. Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:25.492767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:25.493911 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: a01c61f6f9714b7a8535d330d85d09d2. Oct 15 21:08:25.494861 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:25.497672 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1940/3910] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:25 2022] PATCH /identity/v3/domains/a01c61f6f9714b7a8535d330d85d09d2 => generated 112 bytes in 91 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 15 21:08:25.502740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12616804-eab4-4eef-aeb2-addeeceac0ad tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.519150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12616804-eab4-4eef-aeb2-addeeceac0ad tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.527193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d36ac3-f25e-47b9-b770-d1308412da34 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.527692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d36ac3-f25e-47b9-b770-d1308412da34 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.527979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d36ac3-f25e-47b9-b770-d1308412da34 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.528248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d36ac3-f25e-47b9-b770-d1308412da34 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.528909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d36ac3-f25e-47b9-b770-d1308412da34 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:25.534305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d36ac3-f25e-47b9-b770-d1308412da34 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:25.567171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063a866-34ae-42ad-8223-57363198df02 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.567636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063a866-34ae-42ad-8223-57363198df02 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.568159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063a866-34ae-42ad-8223-57363198df02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.594173 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1971/3911] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 21:08:25 2022] GET /identity/v3/domains => generated 1345 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:25.598326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d36ac3-f25e-47b9-b770-d1308412da34 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.624054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d36ac3-f25e-47b9-b770-d1308412da34 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.637318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c2d97-ff6f-4ccf-abaa-ca75c7daa3b3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.637935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c2d97-ff6f-4ccf-abaa-ca75c7daa3b3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.638212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c2d97-ff6f-4ccf-abaa-ca75c7daa3b3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.638602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c2d97-ff6f-4ccf-abaa-ca75c7daa3b3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.639344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c2d97-ff6f-4ccf-abaa-ca75c7daa3b3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Adding query filter params (enabled=True) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:25.639638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c2d97-ff6f-4ccf-abaa-ca75c7daa3b3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:25.641626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f063a866-34ae-42ad-8223-57363198df02 None None] MFA Rules not processed for user `06e2c573568846d095c2a42a1df4e918`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:25.642792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f063a866-34ae-42ad-8223-57363198df02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:25.644874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c2d97-ff6f-4ccf-abaa-ca75c7daa3b3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:25.678262 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1972/3912] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:25 2022] GET /identity/v3/domains?enabled=True => generated 1092 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:25.679449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c2d97-ff6f-4ccf-abaa-ca75c7daa3b3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.692503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:25.697781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c2d97-ff6f-4ccf-abaa-ca75c7daa3b3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.712397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7073dd2-7b82-4663-9712-f1cadffbf7d0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.713335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7073dd2-7b82-4663-9712-f1cadffbf7d0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.715535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7073dd2-7b82-4663-9712-f1cadffbf7d0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.716032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7073dd2-7b82-4663-9712-f1cadffbf7d0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.717092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7073dd2-7b82-4663-9712-f1cadffbf7d0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Adding query filter params (name=tempest-test_domain-259087262) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:25.718938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7073dd2-7b82-4663-9712-f1cadffbf7d0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:25.727620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7073dd2-7b82-4663-9712-f1cadffbf7d0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:25.774012 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1973/3913] 10.222.0.16 () {58 vars in 1197 bytes} [Sat Oct 15 21:08:25 2022] GET /identity/v3/domains?name=tempest-test_domain-259087262 => generated 407 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:08:25.781741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7073dd2-7b82-4663-9712-f1cadffbf7d0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.794400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7073dd2-7b82-4663-9712-f1cadffbf7d0 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.800790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda0f61-2a1d-4a03-a63d-2871fb3cf792 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.801470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda0f61-2a1d-4a03-a63d-2871fb3cf792 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.801750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda0f61-2a1d-4a03-a63d-2871fb3cf792 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.802025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda0f61-2a1d-4a03-a63d-2871fb3cf792 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/a01c61f6f9714b7a8535d330d85d09d2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.813194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda0f61-2a1d-4a03-a63d-2871fb3cf792 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-7fda0f61-2a1d-4a03-a63d-2871fb3cf792 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Could not find domain: a01c61f6f9714b7a8535d330d85d09d2.: keystone.exception.DomainNotFound: Could not find domain: a01c61f6f9714b7a8535d330d85d09d2. Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a01c61f6f9714b7a8535d330d85d09d2. Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:25.821421 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:25.822429 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: a01c61f6f9714b7a8535d330d85d09d2. Oct 15 21:08:25.823267 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:25.827367 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1974/3914] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:25 2022] PATCH /identity/v3/domains/a01c61f6f9714b7a8535d330d85d09d2 => generated 112 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:08:25.829849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda0f61-2a1d-4a03-a63d-2871fb3cf792 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.832387 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1941/3915] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:25.848128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda0f61-2a1d-4a03-a63d-2871fb3cf792 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.848913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0079-0e2f-4f1a-889b-97dddd044853 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.849303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0079-0e2f-4f1a-889b-97dddd044853 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.849572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0079-0e2f-4f1a-889b-97dddd044853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.854307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf39f4-f824-4884-b795-375a1149bba3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.855043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf39f4-f824-4884-b795-375a1149bba3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.855322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf39f4-f824-4884-b795-375a1149bba3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.855596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf39f4-f824-4884-b795-375a1149bba3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/495c82f0cd244c98b2d9cf1c8f90a6b7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:25.871462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf39f4-f824-4884-b795-375a1149bba3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=495c82f0cd244c98b2d9cf1c8f90a6b7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:25.874196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf39f4-f824-4884-b795-375a1149bba3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:25.915977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:25.925690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9ebd0079-0e2f-4f1a-889b-97dddd044853 None None] MFA Rules not processed for user `06e2c573568846d095c2a42a1df4e918`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:25.927930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9ebd0079-0e2f-4f1a-889b-97dddd044853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:25.946432 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1942/3916] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:25.946432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd0079-0e2f-4f1a-889b-97dddd044853 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.946432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9ebd0079-0e2f-4f1a-889b-97dddd044853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:25.958821 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1975/3917] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:25 2022] PATCH /identity/v3/domains/495c82f0cd244c98b2d9cf1c8f90a6b7 => generated 277 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:25.964507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf39f4-f824-4884-b795-375a1149bba3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:25.977825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf39f4-f824-4884-b795-375a1149bba3 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:25.987558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:25.988125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:25.988403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:25.988672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/495c82f0cd244c98b2d9cf1c8f90a6b7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:26.006297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=495c82f0cd244c98b2d9cf1c8f90a6b7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:26.008373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:26.080955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd0079-0e2f-4f1a-889b-97dddd044853 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:26.083950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '495c82f0cd244c98b2d9cf1c8f90a6b7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.088806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:26.090615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:26.090992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:26.091953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] PATH_INFO: `/v3/users/508ea6505b99421d8d65a42f1b6b3876` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:26.093883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '495c82f0cd244c98b2d9cf1c8f90a6b7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.094232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508ea6505b99421d8d65a42f1b6b3876)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:26.098546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:26.176077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495c82f0cd244c98b2d9cf1c8f90a6b7', 'request_id': 'req-ffcd7c12-0586-4c58-995a-7f2c53db1d32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.183787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508ea6505b99421d8d65a42f1b6b3876'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.193154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508ea6505b99421d8d65a42f1b6b3876'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.212113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495c82f0cd244c98b2d9cf1c8f90a6b7', 'request_id': 'req-ffcd7c12-0586-4c58-995a-7f2c53db1d32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.223560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495c82f0cd244c98b2d9cf1c8f90a6b7', 'request_id': 'req-ffcd7c12-0586-4c58-995a-7f2c53db1d32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.234725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508ea6505b99421d8d65a42f1b6b3876'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.237821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] The token cache is being invalidate because project 495c82f0cd244c98b2d9cf1c8f90a6b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:26.238592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.240355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.246529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508ea6505b99421d8d65a42f1b6b3876'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.247235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-be39b226-6f02-45f4-9596-2c28833233a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508ea6505b99421d8d65a42f1b6b3876'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.256153 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1943/3918] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:25 2022] DELETE /identity/v3/users/508ea6505b99421d8d65a42f1b6b3876 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:26.260742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34cad7-d9a0-4bee-8ffd-2543b3585f5e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:26.261179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34cad7-d9a0-4bee-8ffd-2543b3585f5e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:26.261887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34cad7-d9a0-4bee-8ffd-2543b3585f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:26.275230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '495c82f0cd244c98b2d9cf1c8f90a6b7', 'request_id': 'req-ffcd7c12-0586-4c58-995a-7f2c53db1d32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.276322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '495c82f0cd244c98b2d9cf1c8f90a6b7', 'request_id': 'req-ffcd7c12-0586-4c58-995a-7f2c53db1d32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.277112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '495c82f0cd244c98b2d9cf1c8f90a6b7', 'request_id': 'req-ffcd7c12-0586-4c58-995a-7f2c53db1d32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.287364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8c34cad7-d9a0-4bee-8ffd-2543b3585f5e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:26.288692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8c34cad7-d9a0-4bee-8ffd-2543b3585f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:26.290338 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1976/3919] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:25 2022] DELETE /identity/v3/domains/495c82f0cd244c98b2d9cf1c8f90a6b7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:26.294194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:26.296280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:26.343985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:26.405254 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1944/3920] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:26.414871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34cad7-d9a0-4bee-8ffd-2543b3585f5e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:26.416916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8c34cad7-d9a0-4bee-8ffd-2543b3585f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:26.428716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd7c12-0586-4c58-995a-7f2c53db1d32 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:26.459207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:26.497592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34cad7-d9a0-4bee-8ffd-2543b3585f5e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:26.501800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb13914-9a64-4e6c-bb70-3369f4029c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:26.502488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb13914-9a64-4e6c-bb70-3369f4029c11 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:26.502858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb13914-9a64-4e6c-bb70-3369f4029c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:26.503320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb13914-9a64-4e6c-bb70-3369f4029c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:26.504085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:26.505120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:26.505545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:26.506087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/30c6a8058c4a4faa99aa8e3de7c55c90` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:26.515929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb13914-9a64-4e6c-bb70-3369f4029c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:26.518139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb13914-9a64-4e6c-bb70-3369f4029c11 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:26.520884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=30c6a8058c4a4faa99aa8e3de7c55c90)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:26.522730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:26.532729 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1945/3921] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:26.579382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.579973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.650506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.651172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.655254 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1977/3922] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:26 2022] PATCH /identity/v3/domains/30c6a8058c4a4faa99aa8e3de7c55c90 => generated 275 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:26.659102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb13914-9a64-4e6c-bb70-3369f4029c11 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:26.673280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb13914-9a64-4e6c-bb70-3369f4029c11 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:26.674113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:26.675579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:26.772545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:26.774645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:26.775137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:26.775278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/30c6a8058c4a4faa99aa8e3de7c55c90` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:26.789128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=30c6a8058c4a4faa99aa8e3de7c55c90)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:26.791079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:26.791179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffa481-f4fc-4427-8cbe-761e4e43c92f tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:26.799654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754fcac-4ea1-49e0-88a5-53f07f82f610 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:26.800297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754fcac-4ea1-49e0-88a5-53f07f82f610 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:26.800789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754fcac-4ea1-49e0-88a5-53f07f82f610 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:26.800982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754fcac-4ea1-49e0-88a5-53f07f82f610 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:26.804168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f754fcac-4ea1-49e0-88a5-53f07f82f610 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:26.820025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.850516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.921030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:26.926263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90', 'request_id': 'req-4151e25e-373f-4b06-a641-5309c0ed7c57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.927561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754fcac-4ea1-49e0-88a5-53f07f82f610 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:26.929359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754fcac-4ea1-49e0-88a5-53f07f82f610 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:26.940385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90', 'request_id': 'req-4151e25e-373f-4b06-a641-5309c0ed7c57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.954053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90', 'request_id': 'req-4151e25e-373f-4b06-a641-5309c0ed7c57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.964688 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1978/3923] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:26.977531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] The token cache is being invalidate because project 30c6a8058c4a4faa99aa8e3de7c55c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:26.977809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.978805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.990446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90', 'request_id': 'req-4151e25e-373f-4b06-a641-5309c0ed7c57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.991355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90', 'request_id': 'req-4151e25e-373f-4b06-a641-5309c0ed7c57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:26.992217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '30c6a8058c4a4faa99aa8e3de7c55c90', 'request_id': 'req-4151e25e-373f-4b06-a641-5309c0ed7c57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.010319 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1946/3924] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:26 2022] DELETE /identity/v3/domains/30c6a8058c4a4faa99aa8e3de7c55c90 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:27.014904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754fcac-4ea1-49e0-88a5-53f07f82f610 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:27.016755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f754fcac-4ea1-49e0-88a5-53f07f82f610 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:27.039674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754fcac-4ea1-49e0-88a5-53f07f82f610 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:27.073691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:27.164038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:27.164530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:27.164930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:27.166907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/6d311869cf074de6a91f12de26c3f30f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:27.187056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=6d311869cf074de6a91f12de26c3f30f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:27.195746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:27.238142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '6d311869cf074de6a91f12de26c3f30f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.238894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '6d311869cf074de6a91f12de26c3f30f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.281486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '6d311869cf074de6a91f12de26c3f30f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.282480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:27.282936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '6d311869cf074de6a91f12de26c3f30f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.284189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:27.288121 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1979/3925] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:27 2022] PATCH /identity/v3/domains/6d311869cf074de6a91f12de26c3f30f => generated 277 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:27.289135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:27.291016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:27.358525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151e25e-373f-4b06-a641-5309c0ed7c57 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:27.378047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4aa5c8-8e20-4fe1-bc91-cf4ebd91ac97 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:27.384028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa211f5d30dd4f50976ebe5093cce89a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d06616134974ebea540c734183e78e1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:27.384691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:27.385024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:27.385522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] PATH_INFO: `/v3/domains/6d311869cf074de6a91f12de26c3f30f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:27.399078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=6d311869cf074de6a91f12de26c3f30f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:27.400947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:27.424139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:27.424889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:27.425301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:27.425663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] PATH_INFO: `/v3/projects/8c7c33c41d0a4f96bd720b6235d0acad` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:27.427172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7c33c41d0a4f96bd720b6235d0acad)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:27.429456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:27.432531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '6d311869cf074de6a91f12de26c3f30f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.437504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '6d311869cf074de6a91f12de26c3f30f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.491488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7c33c41d0a4f96bd720b6235d0acad', 'request_id': 'req-6439992c-210b-47b2-8bb1-bf40ea93b34b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.501185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7c33c41d0a4f96bd720b6235d0acad', 'request_id': 'req-6439992c-210b-47b2-8bb1-bf40ea93b34b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.502003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:27.507445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d311869cf074de6a91f12de26c3f30f', 'request_id': 'req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.514143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7c33c41d0a4f96bd720b6235d0acad', 'request_id': 'req-6439992c-210b-47b2-8bb1-bf40ea93b34b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.516955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:27.521081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d311869cf074de6a91f12de26c3f30f', 'request_id': 'req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.535785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d311869cf074de6a91f12de26c3f30f', 'request_id': 'req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.537355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] The token cache is being invalidate because project 8c7c33c41d0a4f96bd720b6235d0acad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:27.537842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.538855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6439992c-210b-47b2-8bb1-bf40ea93b34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.543858 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1947/3926] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:27 2022] DELETE /identity/v3/projects/8c7c33c41d0a4f96bd720b6235d0acad => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:27.546769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56763e0-35e9-44cf-8a57-53fc0c2f6073 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:27.547127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56763e0-35e9-44cf-8a57-53fc0c2f6073 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:27.547464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56763e0-35e9-44cf-8a57-53fc0c2f6073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:27.549945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] The token cache is being invalidate because project 6d311869cf074de6a91f12de26c3f30f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:27.550240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.550923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.562723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '6d311869cf074de6a91f12de26c3f30f', 'request_id': 'req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.563817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '6d311869cf074de6a91f12de26c3f30f', 'request_id': 'req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.564834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '6d311869cf074de6a91f12de26c3f30f', 'request_id': 'req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.582872 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1980/3927] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:27 2022] DELETE /identity/v3/domains/6d311869cf074de6a91f12de26c3f30f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:27.587898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:27.590652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:27.600056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a56763e0-35e9-44cf-8a57-53fc0c2f6073 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:27.602676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a56763e0-35e9-44cf-8a57-53fc0c2f6073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:27.679462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b045b6-ac46-4e27-bffd-3e73cdcfb844 tempest-DomainsTestJSON-2044823831 tempest-DomainsTestJSON-2044823831-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:27.684274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:27.684934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:27.685294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:27.685718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] PATH_INFO: `/v3/users/32f8a4e461644978be5fed785315974a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:27.687114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f8a4e461644978be5fed785315974a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:27.689062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:27.698640 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1948/3928] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:27.699233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6c789-e7e1-4319-b5f0-ebf8e657b4a6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:27.699520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6c789-e7e1-4319-b5f0-ebf8e657b4a6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:27.699797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6c789-e7e1-4319-b5f0-ebf8e657b4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:27.729322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a0b6c789-e7e1-4319-b5f0-ebf8e657b4a6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:27.730674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a0b6c789-e7e1-4319-b5f0-ebf8e657b4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:27.742638 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1949/3929] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:27.747477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6c789-e7e1-4319-b5f0-ebf8e657b4a6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:27.770527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6c789-e7e1-4319-b5f0-ebf8e657b4a6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:27.777631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c368505-d897-48cb-b020-48e171d4cad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:27.779814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c368505-d897-48cb-b020-48e171d4cad7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:27.780545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c368505-d897-48cb-b020-48e171d4cad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:27.780978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f8a4e461644978be5fed785315974a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.781494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c368505-d897-48cb-b020-48e171d4cad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:27.782753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c368505-d897-48cb-b020-48e171d4cad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:27.784213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c368505-d897-48cb-b020-48e171d4cad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:27.787429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c368505-d897-48cb-b020-48e171d4cad7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:27.811070 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1950/3930] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:27.812279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c368505-d897-48cb-b020-48e171d4cad7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:27.821550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f8a4e461644978be5fed785315974a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.833376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f8a4e461644978be5fed785315974a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.836264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c368505-d897-48cb-b020-48e171d4cad7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:27.846529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320b1e5-7e95-43b2-ad4d-f99a9cbe017e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:27.847139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320b1e5-7e95-43b2-ad4d-f99a9cbe017e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:27.847420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320b1e5-7e95-43b2-ad4d-f99a9cbe017e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:27.847744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320b1e5-7e95-43b2-ad4d-f99a9cbe017e admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:27.848305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320b1e5-7e95-43b2-ad4d-f99a9cbe017e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:27.852002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320b1e5-7e95-43b2-ad4d-f99a9cbe017e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:27.852322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f8a4e461644978be5fed785315974a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.853435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-761dca73-bf9d-40c0-9b11-b29f274a059b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f8a4e461644978be5fed785315974a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:27.865231 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1981/3931] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:27 2022] DELETE /identity/v3/users/32f8a4e461644978be5fed785315974a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:27.869449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a14737-0e30-47e7-b237-c75fd0d066df None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:27.870500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a14737-0e30-47e7-b237-c75fd0d066df None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:27.871136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a14737-0e30-47e7-b237-c75fd0d066df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:27.899449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-70a14737-0e30-47e7-b237-c75fd0d066df None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:27.902224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-70a14737-0e30-47e7-b237-c75fd0d066df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:27.906940 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1951/3932] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:27 2022] POST /identity/v3/projects => generated 365 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:27.910054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320b1e5-7e95-43b2-ad4d-f99a9cbe017e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:27.912721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a320b1e5-7e95-43b2-ad4d-f99a9cbe017e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:27.958232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:28.021764 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1982/3933] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:28.034447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a14737-0e30-47e7-b237-c75fd0d066df None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:28.037363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-70a14737-0e30-47e7-b237-c75fd0d066df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:28.064873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320b1e5-7e95-43b2-ad4d-f99a9cbe017e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:28.078808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bfc4c-ac99-4319-8f61-ae25e3498f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:28.079512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bfc4c-ac99-4319-8f61-ae25e3498f4f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:28.080328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bfc4c-ac99-4319-8f61-ae25e3498f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:28.080328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bfc4c-ac99-4319-8f61-ae25e3498f4f admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:28.081361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bfc4c-ac99-4319-8f61-ae25e3498f4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:28.085860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bfc4c-ac99-4319-8f61-ae25e3498f4f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:28.119510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a14737-0e30-47e7-b237-c75fd0d066df None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:28.125882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa6f00-e525-44a9-aa5c-8bfe0caa47a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:28.127979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa6f00-e525-44a9-aa5c-8bfe0caa47a9 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:28.128583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa6f00-e525-44a9-aa5c-8bfe0caa47a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:28.128983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa6f00-e525-44a9-aa5c-8bfe0caa47a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:28.141625 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1952/3934] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:27 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:28.145636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bfc4c-ac99-4319-8f61-ae25e3498f4f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:28.148139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa6f00-e525-44a9-aa5c-8bfe0caa47a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:28.151432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa6f00-e525-44a9-aa5c-8bfe0caa47a9 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:28.167111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bfc4c-ac99-4319-8f61-ae25e3498f4f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:28.173127 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1983/3935] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:08:28.180229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08146c79-dd62-4138-aa07-ea94eb5e6fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:28.180858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08146c79-dd62-4138-aa07-ea94eb5e6fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:28.181164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08146c79-dd62-4138-aa07-ea94eb5e6fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:28.181431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08146c79-dd62-4138-aa07-ea94eb5e6fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:28.182377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08146c79-dd62-4138-aa07-ea94eb5e6fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:28.195343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08146c79-dd62-4138-aa07-ea94eb5e6fb4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:28.222156 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1953/3936] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:08:28.225410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa6f00-e525-44a9-aa5c-8bfe0caa47a9 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:28.249733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa6f00-e525-44a9-aa5c-8bfe0caa47a9 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:28.260648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2952af3-d86b-45f2-bb4f-f1e77f10a446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:28.261277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2952af3-d86b-45f2-bb4f-f1e77f10a446 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:28.261552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2952af3-d86b-45f2-bb4f-f1e77f10a446 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:28.263193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2952af3-d86b-45f2-bb4f-f1e77f10a446 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:28.263193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2952af3-d86b-45f2-bb4f-f1e77f10a446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:28.265255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2952af3-d86b-45f2-bb4f-f1e77f10a446 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:28.289807 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1984/3937] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:28.294106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08146c79-dd62-4138-aa07-ea94eb5e6fb4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:28.305903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08146c79-dd62-4138-aa07-ea94eb5e6fb4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:28.328824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1c6f3-00b4-4d4c-9098-2b8806e4e9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:28.329670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1c6f3-00b4-4d4c-9098-2b8806e4e9e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:28.329923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1c6f3-00b4-4d4c-9098-2b8806e4e9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:28.330194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1c6f3-00b4-4d4c-9098-2b8806e4e9e3 admin admin] PATH_INFO: `/v3/projects/0c80653d957b4afc975c59e0794c8aff/users/30a3470452114c74828cd576ab7369f8/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:28.392814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1c6f3-00b4-4d4c-9098-2b8806e4e9e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c80653d957b4afc975c59e0794c8aff, user_id=30a3470452114c74828cd576ab7369f8, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:28.409678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1c6f3-00b4-4d4c-9098-2b8806e4e9e3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:28.418028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:28.461326 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1954/3938] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:28 2022] PUT /identity/v3/projects/0c80653d957b4afc975c59e0794c8aff/users/30a3470452114c74828cd576ab7369f8/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:28.470212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3805ae9-bfba-4955-ad13-c5051a50aa62 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:28.470708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3805ae9-bfba-4955-ad13-c5051a50aa62 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:28.470977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3805ae9-bfba-4955-ad13-c5051a50aa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:28.562885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e3805ae9-bfba-4955-ad13-c5051a50aa62 None None] MFA Rules not processed for user `30a3470452114c74828cd576ab7369f8`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:28.564500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e3805ae9-bfba-4955-ad13-c5051a50aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:28.701993 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1985/3939] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:28.947034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1c6f3-00b4-4d4c-9098-2b8806e4e9e3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:28.950048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-69d1c6f3-00b4-4d4c-9098-2b8806e4e9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:29.088843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1c6f3-00b4-4d4c-9098-2b8806e4e9e3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:29.094770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:29.095721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:29.095995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:29.096292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] PATH_INFO: `/v3/projects/c2555efc656846f49ac0c2ab65c05d98` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:29.097624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2555efc656846f49ac0c2ab65c05d98)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:29.100306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:29.188520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2555efc656846f49ac0c2ab65c05d98', 'request_id': 'req-56eb6333-6604-411a-bd34-648d615ab6e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:29.228671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2555efc656846f49ac0c2ab65c05d98', 'request_id': 'req-56eb6333-6604-411a-bd34-648d615ab6e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:29.249389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2555efc656846f49ac0c2ab65c05d98', 'request_id': 'req-56eb6333-6604-411a-bd34-648d615ab6e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:29.280489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] The token cache is being invalidate because project c2555efc656846f49ac0c2ab65c05d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:29.281986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:29.283081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:29.287157 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1955/3940] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:28 2022] DELETE /identity/v3/projects/c2555efc656846f49ac0c2ab65c05d98 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:29.292282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3805ae9-bfba-4955-ad13-c5051a50aa62 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:29.310935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3805ae9-bfba-4955-ad13-c5051a50aa62 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:29.372755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:29.383229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:29.384068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:29.384576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:29.384947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] PATH_INFO: `/v3/users/fa211f5d30dd4f50976ebe5093cce89a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:29.386301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa211f5d30dd4f50976ebe5093cce89a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:29.388622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:29.460956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa211f5d30dd4f50976ebe5093cce89a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:29.476049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa211f5d30dd4f50976ebe5093cce89a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:29.493719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa211f5d30dd4f50976ebe5093cce89a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:29.516520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa211f5d30dd4f50976ebe5093cce89a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:29.517837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e199d20c-49a4-4f8d-98bd-107894000e52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa211f5d30dd4f50976ebe5093cce89a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:29.529092 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1986/3941] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:29 2022] DELETE /identity/v3/users/fa211f5d30dd4f50976ebe5093cce89a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:29.902619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:29.926567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:29.974498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb6333-6604-411a-bd34-648d615ab6e5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:30.125732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:30.126581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:30.126957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:30.127328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] PATH_INFO: `/v3/projects/5d06616134974ebea540c734183e78e1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:30.128606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d06616134974ebea540c734183e78e1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:30.131055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:30.203621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:30.243647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d06616134974ebea540c734183e78e1', 'request_id': 'req-22a286cc-e538-428e-a773-772b19b0b7e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:30.256244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d06616134974ebea540c734183e78e1', 'request_id': 'req-22a286cc-e538-428e-a773-772b19b0b7e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:30.263397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d06616134974ebea540c734183e78e1', 'request_id': 'req-22a286cc-e538-428e-a773-772b19b0b7e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:30.280342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] The token cache is being invalidate because project 5d06616134974ebea540c734183e78e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:30.280634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:30.281355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22a286cc-e538-428e-a773-772b19b0b7e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:30.282911 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1956/3942] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:29 2022] DELETE /identity/v3/projects/5d06616134974ebea540c734183e78e1 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:30.287771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f636f-6cbf-4711-8d6b-4ea90d6175f1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:30.288136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f636f-6cbf-4711-8d6b-4ea90d6175f1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:30.288405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f636f-6cbf-4711-8d6b-4ea90d6175f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:30.311065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-193f636f-6cbf-4711-8d6b-4ea90d6175f1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:30.312271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-193f636f-6cbf-4711-8d6b-4ea90d6175f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:30.405476 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1987/3943] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:30.412553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ab06a-9f6a-4934-a3ef-1e1a92ffc84c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:30.412838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ab06a-9f6a-4934-a3ef-1e1a92ffc84c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:30.413262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ab06a-9f6a-4934-a3ef-1e1a92ffc84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:30.451596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6c5ab06a-9f6a-4934-a3ef-1e1a92ffc84c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:30.454309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6c5ab06a-9f6a-4934-a3ef-1e1a92ffc84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:30.467611 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1957/3944] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:30.471539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f636f-6cbf-4711-8d6b-4ea90d6175f1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:30.483696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f636f-6cbf-4711-8d6b-4ea90d6175f1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:30.489161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540157-1d37-40dc-9422-cd968a6f8ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:30.490544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540157-1d37-40dc-9422-cd968a6f8ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:30.490998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540157-1d37-40dc-9422-cd968a6f8ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:30.491937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540157-1d37-40dc-9422-cd968a6f8ec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:30.492426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9540157-1d37-40dc-9422-cd968a6f8ec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:30.492951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9540157-1d37-40dc-9422-cd968a6f8ec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:30.494974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9540157-1d37-40dc-9422-cd968a6f8ec1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:30.508590 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1988/3945] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:30.513011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ab06a-9f6a-4934-a3ef-1e1a92ffc84c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:30.525061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ab06a-9f6a-4934-a3ef-1e1a92ffc84c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:30.530862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bdab5a-cd4b-492c-96c4-5df09bf2d794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:30.531849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdab5a-cd4b-492c-96c4-5df09bf2d794 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:30.532784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdab5a-cd4b-492c-96c4-5df09bf2d794 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:30.532784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdab5a-cd4b-492c-96c4-5df09bf2d794 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:30.533247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bdab5a-cd4b-492c-96c4-5df09bf2d794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:30.535372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bdab5a-cd4b-492c-96c4-5df09bf2d794 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:30.570202 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1958/3946] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:30 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:30.574726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540157-1d37-40dc-9422-cd968a6f8ec1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:30.587160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540157-1d37-40dc-9422-cd968a6f8ec1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:30.672673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8340aad-f9fe-467f-85dd-74c6fdd2d8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:30.673361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8340aad-f9fe-467f-85dd-74c6fdd2d8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:30.673730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8340aad-f9fe-467f-85dd-74c6fdd2d8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:30.674068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8340aad-f9fe-467f-85dd-74c6fdd2d8d4 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:30.674779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8340aad-f9fe-467f-85dd-74c6fdd2d8d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:30.677194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8340aad-f9fe-467f-85dd-74c6fdd2d8d4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:30.697620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:30.735731 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1989/3947] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:30 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:30.737129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bdab5a-cd4b-492c-96c4-5df09bf2d794 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:30.758034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bdab5a-cd4b-492c-96c4-5df09bf2d794 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:30.766396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c8acc-505c-4ff2-bb7a-9f22b3d2c103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:30.766976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c8acc-505c-4ff2-bb7a-9f22b3d2c103 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:30.767264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c8acc-505c-4ff2-bb7a-9f22b3d2c103 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:30.767536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c8acc-505c-4ff2-bb7a-9f22b3d2c103 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:30.768184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c8acc-505c-4ff2-bb7a-9f22b3d2c103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:30.772389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c8acc-505c-4ff2-bb7a-9f22b3d2c103 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:30.793001 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1959/3948] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:30.798475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8340aad-f9fe-467f-85dd-74c6fdd2d8d4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:30.812195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8340aad-f9fe-467f-85dd-74c6fdd2d8d4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:30.818661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8a67e-998f-45a8-98b4-175482afae11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:30.818661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a67e-998f-45a8-98b4-175482afae11 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:30.818928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a67e-998f-45a8-98b4-175482afae11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:30.819137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a67e-998f-45a8-98b4-175482afae11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:30.820115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8a67e-998f-45a8-98b4-175482afae11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:30.821986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8a67e-998f-45a8-98b4-175482afae11 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:30.837193 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1990/3949] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:30.842771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c8acc-505c-4ff2-bb7a-9f22b3d2c103 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:30.854970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c8acc-505c-4ff2-bb7a-9f22b3d2c103 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:30.861366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d81a9-6986-4f15-80d1-154f0a381d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:30.862031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d81a9-6986-4f15-80d1-154f0a381d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:30.862303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d81a9-6986-4f15-80d1-154f0a381d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:30.862857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d81a9-6986-4f15-80d1-154f0a381d13 admin admin] PATH_INFO: `/v3/projects/40f63f7d53f94851811e439ba16dafa8/users/51bdfb93d9124fc2b29d74765f3022be/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:30.882933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d81a9-6986-4f15-80d1-154f0a381d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f63f7d53f94851811e439ba16dafa8, user_id=51bdfb93d9124fc2b29d74765f3022be, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:30.886606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d81a9-6986-4f15-80d1-154f0a381d13 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:30.906718 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1960/3950] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:30 2022] PUT /identity/v3/projects/40f63f7d53f94851811e439ba16dafa8/users/51bdfb93d9124fc2b29d74765f3022be/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:30.917851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a51e3-11a8-4a86-8565-e91da854835f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:30.918655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a51e3-11a8-4a86-8565-e91da854835f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:30.919080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a51e3-11a8-4a86-8565-e91da854835f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:30.964710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-003a51e3-11a8-4a86-8565-e91da854835f None None] MFA Rules not processed for user `51bdfb93d9124fc2b29d74765f3022be`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:30.966356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-003a51e3-11a8-4a86-8565-e91da854835f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:31.029412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d81a9-6986-4f15-80d1-154f0a381d13 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.031316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ab7d81a9-6986-4f15-80d1-154f0a381d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:31.082006 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1991/3951] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:31.083136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a51e3-11a8-4a86-8565-e91da854835f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.169112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a51e3-11a8-4a86-8565-e91da854835f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:31.176372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068b988-2e7d-4539-8db9-9fc6c5125111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:31.177060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068b988-2e7d-4539-8db9-9fc6c5125111 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.177393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068b988-2e7d-4539-8db9-9fc6c5125111 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.177666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068b988-2e7d-4539-8db9-9fc6c5125111 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.178676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068b988-2e7d-4539-8db9-9fc6c5125111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:31.181022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068b988-2e7d-4539-8db9-9fc6c5125111 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:31.212830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d81a9-6986-4f15-80d1-154f0a381d13 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:31.225999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814bd1c-027e-40db-9829-8673b2453e36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:31.227604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814bd1c-027e-40db-9829-8673b2453e36 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.228270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814bd1c-027e-40db-9829-8673b2453e36 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.228784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814bd1c-027e-40db-9829-8673b2453e36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.234200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d814bd1c-027e-40db-9829-8673b2453e36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:31.244850 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1992/3952] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:31 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:31.250911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068b988-2e7d-4539-8db9-9fc6c5125111 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.324215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:31.328532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:31.332404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068b988-2e7d-4539-8db9-9fc6c5125111 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:31.346585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e543e8c-9c27-429d-8b0b-73211e9ac35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:31.348285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e543e8c-9c27-429d-8b0b-73211e9ac35a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.348760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e543e8c-9c27-429d-8b0b-73211e9ac35a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.349196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e543e8c-9c27-429d-8b0b-73211e9ac35a admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.349989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e543e8c-9c27-429d-8b0b-73211e9ac35a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:31.353149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e543e8c-9c27-429d-8b0b-73211e9ac35a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:31.357817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814bd1c-027e-40db-9829-8673b2453e36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:31.359412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814bd1c-027e-40db-9829-8673b2453e36 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:31.377493 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1961/3953] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:08:31.409852 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1993/3954] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:31 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:31.414070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814bd1c-027e-40db-9829-8673b2453e36 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.427251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814bd1c-027e-40db-9829-8673b2453e36 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:31.435548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8385a-fd6a-462f-8a3f-93d66f899dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:31.436426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8385a-fd6a-462f-8a3f-93d66f899dfd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.436941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8385a-fd6a-462f-8a3f-93d66f899dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.437430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8385a-fd6a-462f-8a3f-93d66f899dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.438253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8385a-fd6a-462f-8a3f-93d66f899dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:31.440286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8385a-fd6a-462f-8a3f-93d66f899dfd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:31.455741 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1962/3955] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:31.459948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e543e8c-9c27-429d-8b0b-73211e9ac35a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.472282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e543e8c-9c27-429d-8b0b-73211e9ac35a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:31.478106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eeefb4-7076-4ee6-b0bd-4653d0b75e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:31.478870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeefb4-7076-4ee6-b0bd-4653d0b75e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.479063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeefb4-7076-4ee6-b0bd-4653d0b75e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.479350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeefb4-7076-4ee6-b0bd-4653d0b75e93 admin admin] PATH_INFO: `/v3/projects/896ca2d11a7144d886405c44ea28ca33/users/2e725968179a441099a3e13ae2d6d069/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.502109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eeefb4-7076-4ee6-b0bd-4653d0b75e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896ca2d11a7144d886405c44ea28ca33, user_id=2e725968179a441099a3e13ae2d6d069, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:31.504984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eeefb4-7076-4ee6-b0bd-4653d0b75e93 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:31.528497 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1994/3956] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:31 2022] PUT /identity/v3/projects/896ca2d11a7144d886405c44ea28ca33/users/2e725968179a441099a3e13ae2d6d069/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:31.530327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8385a-fd6a-462f-8a3f-93d66f899dfd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.541233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8385a-fd6a-462f-8a3f-93d66f899dfd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:31.626357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e8013-8ef3-45ca-b9fc-af34088861e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:31.627194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e8013-8ef3-45ca-b9fc-af34088861e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.627411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e8013-8ef3-45ca-b9fc-af34088861e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.627764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e8013-8ef3-45ca-b9fc-af34088861e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.628879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e8013-8ef3-45ca-b9fc-af34088861e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:31.630411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e8013-8ef3-45ca-b9fc-af34088861e2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:31.643931 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1963/3957] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:31.647853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eeefb4-7076-4ee6-b0bd-4653d0b75e93 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.660269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eeefb4-7076-4ee6-b0bd-4653d0b75e93 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:31.666448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92a572-f57e-4ada-b350-5d35ffcac1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:31.667118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92a572-f57e-4ada-b350-5d35ffcac1ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.667436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92a572-f57e-4ada-b350-5d35ffcac1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.667699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92a572-f57e-4ada-b350-5d35ffcac1ee admin admin] PATH_INFO: `/v3/projects/896ca2d11a7144d886405c44ea28ca33/users/2e725968179a441099a3e13ae2d6d069/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.670303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92a572-f57e-4ada-b350-5d35ffcac1ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=896ca2d11a7144d886405c44ea28ca33, user_id=2e725968179a441099a3e13ae2d6d069, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:31.673114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92a572-f57e-4ada-b350-5d35ffcac1ee admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:31.693381 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1995/3958] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:31 2022] PUT /identity/v3/projects/896ca2d11a7144d886405c44ea28ca33/users/2e725968179a441099a3e13ae2d6d069/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:31.696775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d50a3-3bc2-480c-9857-7d56052d60a4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.697095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d50a3-3bc2-480c-9857-7d56052d60a4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.697374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d50a3-3bc2-480c-9857-7d56052d60a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.731014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c55d50a3-3bc2-480c-9857-7d56052d60a4 None None] MFA Rules not processed for user `2e725968179a441099a3e13ae2d6d069`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:31.732448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c55d50a3-3bc2-480c-9857-7d56052d60a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:31.807905 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1964/3959] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:31.812944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92a572-f57e-4ada-b350-5d35ffcac1ee admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.823797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92a572-f57e-4ada-b350-5d35ffcac1ee admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:31.829498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde67b72-85d6-4049-8f8b-67f7e609c0b4 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:31.830110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde67b72-85d6-4049-8f8b-67f7e609c0b4 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.830411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde67b72-85d6-4049-8f8b-67f7e609c0b4 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.830681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde67b72-85d6-4049-8f8b-67f7e609c0b4 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.831662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde67b72-85d6-4049-8f8b-67f7e609c0b4 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:31.833371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde67b72-85d6-4049-8f8b-67f7e609c0b4 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-fde67b72-85d6-4049-8f8b-67f7e609c0b4 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invalid input for field 'name': '' is too short Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: Failed validating 'minLength' in schema['properties']['name']: Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: On instance['name']: Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: Failed validating 'minLength' in schema['properties']['name']: Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: On instance['name']: Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: '' Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise error Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application On instance['name']: Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application '' Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:31.854959 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 15 21:08:31.856281 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 15 21:08:31.858221 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 15 21:08:31.858221 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Oct 15 21:08:31.858221 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:31.858221 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Oct 15 21:08:31.858221 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:08:31.858221 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:31.858221 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application On instance['name']: Oct 15 21:08:31.858221 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application '' Oct 15 21:08:31.858221 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:31.858221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1996/3960] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:31 2022] POST /identity/v3/domains => generated 283 bytes in 45 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 15 21:08:31.863514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d50a3-3bc2-480c-9857-7d56052d60a4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.875489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d50a3-3bc2-480c-9857-7d56052d60a4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:31.881057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccede34b-4555-4968-9472-52d42065a761 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:31.881521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccede34b-4555-4968-9472-52d42065a761 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.881795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccede34b-4555-4968-9472-52d42065a761 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.882063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccede34b-4555-4968-9472-52d42065a761 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.882713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccede34b-4555-4968-9472-52d42065a761 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:31.884303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccede34b-4555-4968-9472-52d42065a761 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-ccede34b-4555-4968-9472-52d42065a761 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: Failed validating 'maxLength' in schema['properties']['name']: Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: On instance['name']: Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: Failed validating 'maxLength' in schema['properties']['name']: Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: On instance['name']: Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise error Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:08:31.892822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application On instance['name']: Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Oct 15 21:08:31.894005 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application On instance['name']: Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:31.899374 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1965/3961] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:31 2022] POST /identity/v3/domains => generated 412 bytes in 32 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 15 21:08:31.900220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde67b72-85d6-4049-8f8b-67f7e609c0b4 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.911515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde67b72-85d6-4049-8f8b-67f7e609c0b4 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:31.916246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2883504-42b9-44e2-86a2-73fa6fd4bb07 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:31.916924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883504-42b9-44e2-86a2-73fa6fd4bb07 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:31.917290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883504-42b9-44e2-86a2-73fa6fd4bb07 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:31.917624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883504-42b9-44e2-86a2-73fa6fd4bb07 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:31.918327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2883504-42b9-44e2-86a2-73fa6fd4bb07 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:31.920196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2883504-42b9-44e2-86a2-73fa6fd4bb07 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:31.939357 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1997/3962] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:31 2022] POST /identity/v3/domains => generated 276 bytes in 37 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:31.940539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccede34b-4555-4968-9472-52d42065a761 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:31.952157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccede34b-4555-4968-9472-52d42065a761 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:32.008226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:32.017957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed03fa4-dedc-4720-a9db-5f273c706fa5 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:32.018595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed03fa4-dedc-4720-a9db-5f273c706fa5 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:32.018873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed03fa4-dedc-4720-a9db-5f273c706fa5 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:32.019139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed03fa4-dedc-4720-a9db-5f273c706fa5 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains/bb9da3601cda4565b3c7898fc217f615` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:32.034086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed03fa4-dedc-4720-a9db-5f273c706fa5 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=bb9da3601cda4565b3c7898fc217f615)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:32.035898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed03fa4-dedc-4720-a9db-5f273c706fa5 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-aed03fa4-dedc-4720-a9db-5f273c706fa5 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Cannot delete a domain that is enabled, please disable it first.: keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 151, in delete Oct 15 21:08:32.051544 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_domain( Oct 15 21:08:32.052589 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:32.052589 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:32.052589 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 802, in delete_domain Oct 15 21:08:32.052589 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._delete_domain(domain, initiator) Oct 15 21:08:32.052589 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 815, in _delete_domain Oct 15 21:08:32.052589 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Oct 15 21:08:32.052589 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Oct 15 21:08:32.052589 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:32.053028 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1966/3963] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:31 2022] DELETE /identity/v3/domains/bb9da3601cda4565b3c7898fc217f615 => generated 120 bytes in 112 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:08:32.056702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2883504-42b9-44e2-86a2-73fa6fd4bb07 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:32.069090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2883504-42b9-44e2-86a2-73fa6fd4bb07 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:32.073821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:32.074520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:32.074855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:32.075187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains/bb9da3601cda4565b3c7898fc217f615` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:32.077026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=bb9da3601cda4565b3c7898fc217f615)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:32.078746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:32.104083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.104717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.142016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.143541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.149001 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1998/3964] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:32 2022] PATCH /identity/v3/domains/bb9da3601cda4565b3c7898fc217f615 => generated 277 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:08:32.150899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed03fa4-dedc-4720-a9db-5f273c706fa5 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:32.153683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aed03fa4-dedc-4720-a9db-5f273c706fa5 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:32.166513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed03fa4-dedc-4720-a9db-5f273c706fa5 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:32.261898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:32.262553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:32.262857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:32.263164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains/bb9da3601cda4565b3c7898fc217f615` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:32.279729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=bb9da3601cda4565b3c7898fc217f615)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:32.281816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:32.314425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.326347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.382133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615', 'request_id': 'req-0d2b264f-efcb-4a3b-b71a-e92949870fad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.390879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615', 'request_id': 'req-0d2b264f-efcb-4a3b-b71a-e92949870fad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.398460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615', 'request_id': 'req-0d2b264f-efcb-4a3b-b71a-e92949870fad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.414454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] The token cache is being invalidate because project bb9da3601cda4565b3c7898fc217f615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:32.414826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.415579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.424774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615', 'request_id': 'req-0d2b264f-efcb-4a3b-b71a-e92949870fad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.426236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615', 'request_id': 'req-0d2b264f-efcb-4a3b-b71a-e92949870fad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.426986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'bb9da3601cda4565b3c7898fc217f615', 'request_id': 'req-0d2b264f-efcb-4a3b-b71a-e92949870fad'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.436464 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1967/3965] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:32 2022] DELETE /identity/v3/domains/bb9da3601cda4565b3c7898fc217f615 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:32.441683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:32.452965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7c134-13ec-4d76-b646-7d6e59a1052c tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:32.511823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2dde1-fd23-47f1-a3f0-8f9544e5a175 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:32.512472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2dde1-fd23-47f1-a3f0-8f9544e5a175 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:32.512731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2dde1-fd23-47f1-a3f0-8f9544e5a175 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:32.513001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2dde1-fd23-47f1-a3f0-8f9544e5a175 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains/c501ea00c4944b57b2dc2b90d6541cd4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:32.520724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2dde1-fd23-47f1-a3f0-8f9544e5a175 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-a0d2dde1-fd23-47f1-a3f0-8f9544e5a175 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Could not find domain: c501ea00c4944b57b2dc2b90d6541cd4.: keystone.exception.DomainNotFound: Could not find domain: c501ea00c4944b57b2dc2b90d6541cd4. Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c501ea00c4944b57b2dc2b90d6541cd4. Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:32.525709 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 150, in delete Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_domain') Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:32.526828 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:08:32.528506 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: c501ea00c4944b57b2dc2b90d6541cd4. Oct 15 21:08:32.529756 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:32.529756 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 1999/3966] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:32 2022] DELETE /identity/v3/domains/c501ea00c4944b57b2dc2b90d6541cd4 => generated 112 bytes in 86 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 15 21:08:32.534575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:32.536141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:32.550171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b264f-efcb-4a3b-b71a-e92949870fad tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:32.556492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d57357-69c7-4456-80de-3a19e3a94645 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:32.557277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d57357-69c7-4456-80de-3a19e3a94645 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:32.557636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d57357-69c7-4456-80de-3a19e3a94645 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:32.557996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d57357-69c7-4456-80de-3a19e3a94645 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:32.559134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d57357-69c7-4456-80de-3a19e3a94645 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:32.562300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d57357-69c7-4456-80de-3a19e3a94645 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:32.582936 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1968/3967] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:32 2022] POST /identity/v3/domains => generated 253 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:32.586918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2dde1-fd23-47f1-a3f0-8f9544e5a175 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:32.601559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2dde1-fd23-47f1-a3f0-8f9544e5a175 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:32.689904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:32.691764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d57357-69c7-4456-80de-3a19e3a94645 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:32.694292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-85d57357-69c7-4456-80de-3a19e3a94645 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:32.702609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:32.718688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b5a42-4d5f-4fc9-a9e4-22235a43455d tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:32.719461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b5a42-4d5f-4fc9-a9e4-22235a43455d tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:32.719847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b5a42-4d5f-4fc9-a9e4-22235a43455d tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:32.720194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b5a42-4d5f-4fc9-a9e4-22235a43455d tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:32.720986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b5a42-4d5f-4fc9-a9e4-22235a43455d tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:32.722820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b5a42-4d5f-4fc9-a9e4-22235a43455d tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:32.732143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.sql.core [None req-927b5a42-4d5f-4fc9-a9e4-22235a43455d tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1429858441' for key 'project.ixu_project_name_domain_id'") Oct 15 21:08:32.732143 ncdv-master-1 devstack@keystone.service[99100]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 21:08:32.732143 ncdv-master-1 devstack@keystone.service[99100]: [parameters: {'id': '816315033f574dbdbe5db79dbba1ffc4', 'name': 'tempest-domain-dup-1429858441', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 21:08:32.732143 ncdv-master-1 devstack@keystone.service[99100]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99100) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-927b5a42-4d5f-4fc9-a9e4-22235a43455d tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1429858441.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1429858441. Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application conn.query(q) Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 21:08:32.743820 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result.read() Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-tempest-domain-dup-1429858441' for key 'project.ixu_project_name_domain_id'") Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 21:08:32.744680 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application next(self.gen) Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application yield resource Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application next(self.gen) Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise self.value Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application session.commit() Oct 15 21:08:32.746583 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.session.flush() Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application compat.raise_( Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 21:08:32.747588 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 21:08:32.749405 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application conn.query(q) Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result.read() Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 21:08:32.751494 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1429858441' for key 'project.ixu_project_name_domain_id'") Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [parameters: {'id': '816315033f574dbdbe5db79dbba1ffc4', 'name': 'tempest-domain-dup-1429858441', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:32.754123 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-domain-dup-1429858441. Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:32.756156 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1429858441. Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:32.760060 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2000/3968] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:32 2022] POST /identity/v3/domains => generated 212 bytes in 162 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 21:08:32.762581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b5a42-4d5f-4fc9-a9e4-22235a43455d tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:32.762581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b5a42-4d5f-4fc9-a9e4-22235a43455d tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:32.768505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:32.770185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:32.770185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:32.770185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains/30b17328bec4472db0aed20e5090acce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:32.788877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=30b17328bec4472db0aed20e5090acce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:32.791708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:32.808462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d57357-69c7-4456-80de-3a19e3a94645 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:32.818693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '30b17328bec4472db0aed20e5090acce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.819704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '30b17328bec4472db0aed20e5090acce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.826693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:32.827347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:32.827730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:32.827914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] PATH_INFO: `/v3/users/30a3470452114c74828cd576ab7369f8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:32.829322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=30a3470452114c74828cd576ab7369f8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:32.832219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:32.857683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '30b17328bec4472db0aed20e5090acce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.858412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '30b17328bec4472db0aed20e5090acce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.865567 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2001/3969] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:32 2022] PATCH /identity/v3/domains/30b17328bec4472db0aed20e5090acce => generated 254 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:08:32.866617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:32.868074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:32.896738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30a3470452114c74828cd576ab7369f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.908610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30a3470452114c74828cd576ab7369f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.925448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30a3470452114c74828cd576ab7369f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.938431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30a3470452114c74828cd576ab7369f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.940670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-90478001-9cf8-4135-a32b-5d2284b8e0ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30a3470452114c74828cd576ab7369f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:32.949397 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1969/3970] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:32 2022] DELETE /identity/v3/users/30a3470452114c74828cd576ab7369f8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:32.954982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6dd1d-826d-44b1-9067-8827c5ab94f9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:32.955458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6dd1d-826d-44b1-9067-8827c5ab94f9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:32.956346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6dd1d-826d-44b1-9067-8827c5ab94f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:32.971825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34846b48-5c01-4526-9d56-291ad5e8b034 tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:32.987749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6aa6dd1d-826d-44b1-9067-8827c5ab94f9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:32.989272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6aa6dd1d-826d-44b1-9067-8827c5ab94f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:33.071183 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1970/3971] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:33.071440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:33.072183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:33.072408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:33.072677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains/30b17328bec4472db0aed20e5090acce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:33.085431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6dd1d-826d-44b1-9067-8827c5ab94f9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:33.086621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=30b17328bec4472db0aed20e5090acce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:33.087424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6aa6dd1d-826d-44b1-9067-8827c5ab94f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:33.088295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:33.115211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '30b17328bec4472db0aed20e5090acce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.120700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '30b17328bec4472db0aed20e5090acce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.126030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:33.134572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:33.172218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6dd1d-826d-44b1-9067-8827c5ab94f9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:33.184554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30b17328bec4472db0aed20e5090acce', 'request_id': 'req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.204219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30b17328bec4472db0aed20e5090acce', 'request_id': 'req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.214878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30b17328bec4472db0aed20e5090acce', 'request_id': 'req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.235983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] The token cache is being invalidate because project 30b17328bec4472db0aed20e5090acce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:33.236503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.237352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.244097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916263ef-2a05-4d10-9fdd-2cf24a04f8c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:33.244786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916263ef-2a05-4d10-9fdd-2cf24a04f8c0 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:33.245269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916263ef-2a05-4d10-9fdd-2cf24a04f8c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:33.245615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916263ef-2a05-4d10-9fdd-2cf24a04f8c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:33.247462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-916263ef-2a05-4d10-9fdd-2cf24a04f8c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:33.248687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '30b17328bec4472db0aed20e5090acce', 'request_id': 'req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.249601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '30b17328bec4472db0aed20e5090acce', 'request_id': 'req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.250386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '30b17328bec4472db0aed20e5090acce', 'request_id': 'req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.265295 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2002/3972] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:32 2022] DELETE /identity/v3/domains/30b17328bec4472db0aed20e5090acce => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:33.267003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:33.267614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:33.338122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916263ef-2a05-4d10-9fdd-2cf24a04f8c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:33.339890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916263ef-2a05-4d10-9fdd-2cf24a04f8c0 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:33.352535 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1971/3973] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:08:33.361153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d348a-e6b2-4438-9512-bc5b2fe5bc4b tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:33.366537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf65364-8eef-4815-a9e5-797b8669e76a tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e725968179a441099a3e13ae2d6d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '896ca2d11a7144d886405c44ea28ca33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:33.367296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf65364-8eef-4815-a9e5-797b8669e76a tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:33.367845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf65364-8eef-4815-a9e5-797b8669e76a tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:33.368162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf65364-8eef-4815-a9e5-797b8669e76a tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] PATH_INFO: `/v3/domains/bb9da3601cda4565b3c7898fc217f615` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:33.377433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf65364-8eef-4815-a9e5-797b8669e76a tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-8cf65364-8eef-4815-a9e5-797b8669e76a tempest-DomainsNegativeTestJSON-94171329 tempest-DomainsNegativeTestJSON-94171329-project-admin] Could not find domain: bb9da3601cda4565b3c7898fc217f615.: keystone.exception.DomainNotFound: Could not find domain: bb9da3601cda4565b3c7898fc217f615. Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: bb9da3601cda4565b3c7898fc217f615. Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:33.383545 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:33.384664 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:08:33.386162 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: bb9da3601cda4565b3c7898fc217f615. Oct 15 21:08:33.387296 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:33.387296 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2003/3974] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:33 2022] PATCH /identity/v3/domains/bb9da3601cda4565b3c7898fc217f615 => generated 112 bytes in 121 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:08:33.392717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916263ef-2a05-4d10-9fdd-2cf24a04f8c0 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:33.394556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-916263ef-2a05-4d10-9fdd-2cf24a04f8c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:33.420756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916263ef-2a05-4d10-9fdd-2cf24a04f8c0 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:33.431319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:33.432561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:33.433874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:33.434305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] PATH_INFO: `/v3/users/51bdfb93d9124fc2b29d74765f3022be` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:33.436563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51bdfb93d9124fc2b29d74765f3022be)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:33.439226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:33.523056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51bdfb93d9124fc2b29d74765f3022be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.533243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51bdfb93d9124fc2b29d74765f3022be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.549214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51bdfb93d9124fc2b29d74765f3022be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.558123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51bdfb93d9124fc2b29d74765f3022be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.559104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51bdfb93d9124fc2b29d74765f3022be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:33.566227 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1972/3975] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:33 2022] DELETE /identity/v3/users/51bdfb93d9124fc2b29d74765f3022be => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:33.570643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3259da-66fd-4dff-95d5-caa23ee10ab3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:33.571130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3259da-66fd-4dff-95d5-caa23ee10ab3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:33.571210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3259da-66fd-4dff-95d5-caa23ee10ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:33.598948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0d3259da-66fd-4dff-95d5-caa23ee10ab3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:33.600576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0d3259da-66fd-4dff-95d5-caa23ee10ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:33.678710 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2004/3976] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:33.729893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:33.731689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:33.740670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:33.809974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c153-bfa3-4874-8a08-9796c3255079 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:33.815028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519de27-383e-4059-b510-4b0c8d628455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:33.815320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519de27-383e-4059-b510-4b0c8d628455 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:33.815718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519de27-383e-4059-b510-4b0c8d628455 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:33.816054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519de27-383e-4059-b510-4b0c8d628455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:33.826886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519de27-383e-4059-b510-4b0c8d628455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:33.828470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519de27-383e-4059-b510-4b0c8d628455 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:33.846492 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1973/3977] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:34.230755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3259da-66fd-4dff-95d5-caa23ee10ab3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:34.234149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0d3259da-66fd-4dff-95d5-caa23ee10ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:34.266658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3259da-66fd-4dff-95d5-caa23ee10ab3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:34.291358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:34.292251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:34.292676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:34.293021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] PATH_INFO: `/v3/projects/0c80653d957b4afc975c59e0794c8aff` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:34.295166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c80653d957b4afc975c59e0794c8aff)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:34.299368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:34.316582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:34.364481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c80653d957b4afc975c59e0794c8aff', 'request_id': 'req-7985df66-c68f-48ce-9cc3-0b7a1039dd32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:34.377417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c80653d957b4afc975c59e0794c8aff', 'request_id': 'req-7985df66-c68f-48ce-9cc3-0b7a1039dd32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:34.384802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c80653d957b4afc975c59e0794c8aff', 'request_id': 'req-7985df66-c68f-48ce-9cc3-0b7a1039dd32'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:34.398480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] The token cache is being invalidate because project 0c80653d957b4afc975c59e0794c8aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:34.399077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:34.399736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:34.401543 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2005/3978] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:34 2022] DELETE /identity/v3/projects/0c80653d957b4afc975c59e0794c8aff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:34.407883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1226f-0242-4613-94dd-fd2d29bc712c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:34.408418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1226f-0242-4613-94dd-fd2d29bc712c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:34.408621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1226f-0242-4613-94dd-fd2d29bc712c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:34.439490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8ad1226f-0242-4613-94dd-fd2d29bc712c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:34.440788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8ad1226f-0242-4613-94dd-fd2d29bc712c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:34.461916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:34.463473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:34.530989 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1974/3979] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:34.539576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437a7fd-9f2f-4237-a1cd-0ecb3758b10a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:34.540033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437a7fd-9f2f-4237-a1cd-0ecb3758b10a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:34.540889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437a7fd-9f2f-4237-a1cd-0ecb3758b10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:34.571718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3437a7fd-9f2f-4237-a1cd-0ecb3758b10a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:34.573965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3437a7fd-9f2f-4237-a1cd-0ecb3758b10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:34.584134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985df66-c68f-48ce-9cc3-0b7a1039dd32 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:34.588168 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1975/3980] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:34.589283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:34.589811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437a7fd-9f2f-4237-a1cd-0ecb3758b10a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:34.589928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:34.590193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:34.590452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] PATH_INFO: `/v3/projects/40f63f7d53f94851811e439ba16dafa8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:34.592669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] RBAC: Authorizing `identity:delete_project(project_id=40f63f7d53f94851811e439ba16dafa8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:34.594761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:34.608247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437a7fd-9f2f-4237-a1cd-0ecb3758b10a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:34.619666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4a0f8-d611-4041-968d-45d1c14887b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:34.620278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4a0f8-d611-4041-968d-45d1c14887b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:34.620548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4a0f8-d611-4041-968d-45d1c14887b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:34.621001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4a0f8-d611-4041-968d-45d1c14887b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:34.621688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4a0f8-d611-4041-968d-45d1c14887b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:34.621984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4a0f8-d611-4041-968d-45d1c14887b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:34.624298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4a0f8-d611-4041-968d-45d1c14887b8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:34.649072 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1976/3981] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:34.650723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4a0f8-d611-4041-968d-45d1c14887b8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:34.662244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4a0f8-d611-4041-968d-45d1c14887b8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:34.665480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40f63f7d53f94851811e439ba16dafa8', 'request_id': 'req-b74a59db-2b6b-4217-9157-e97d6e8de43c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:34.678145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40f63f7d53f94851811e439ba16dafa8', 'request_id': 'req-b74a59db-2b6b-4217-9157-e97d6e8de43c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:34.687994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40f63f7d53f94851811e439ba16dafa8', 'request_id': 'req-b74a59db-2b6b-4217-9157-e97d6e8de43c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:34.709132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] The token cache is being invalidate because project 40f63f7d53f94851811e439ba16dafa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:34.709500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:34.710140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:34.712932 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2006/3982] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:34 2022] DELETE /identity/v3/projects/40f63f7d53f94851811e439ba16dafa8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:34.716963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:34.718723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:34.768780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70555ae0-9ab0-4096-bc7f-d4c4b9c17b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:34.769421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70555ae0-9ab0-4096-bc7f-d4c4b9c17b9d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:34.769691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70555ae0-9ab0-4096-bc7f-d4c4b9c17b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:34.769951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70555ae0-9ab0-4096-bc7f-d4c4b9c17b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:34.770581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70555ae0-9ab0-4096-bc7f-d4c4b9c17b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:34.772503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70555ae0-9ab0-4096-bc7f-d4c4b9c17b9d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:34.815254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:34.824199 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1977/3983] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:34 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:34.829813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70555ae0-9ab0-4096-bc7f-d4c4b9c17b9d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:34.833576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-70555ae0-9ab0-4096-bc7f-d4c4b9c17b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:34.853666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:34.862949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a59db-2b6b-4217-9157-e97d6e8de43c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:34.963802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70555ae0-9ab0-4096-bc7f-d4c4b9c17b9d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:34.976308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980ce3e-678f-4102-9a3b-54ab45bf6409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:34.978191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980ce3e-678f-4102-9a3b-54ab45bf6409 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:34.978719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980ce3e-678f-4102-9a3b-54ab45bf6409 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:34.979365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980ce3e-678f-4102-9a3b-54ab45bf6409 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:34.982672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980ce3e-678f-4102-9a3b-54ab45bf6409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:34.985167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980ce3e-678f-4102-9a3b-54ab45bf6409 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:34.993067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:34.994355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:34.996091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:34.996592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] PATH_INFO: `/v3/users/2e725968179a441099a3e13ae2d6d069` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:35.001236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e725968179a441099a3e13ae2d6d069)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:35.005500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:35.033727 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1978/3984] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:34 2022] POST /identity/v3/users => generated 284 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:35.037949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980ce3e-678f-4102-9a3b-54ab45bf6409 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:35.069444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980ce3e-678f-4102-9a3b-54ab45bf6409 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:35.078760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435fbbb-8583-4d8f-8f4d-ffe2519d0992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:35.080808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435fbbb-8583-4d8f-8f4d-ffe2519d0992 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:35.085212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435fbbb-8583-4d8f-8f4d-ffe2519d0992 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:35.085212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435fbbb-8583-4d8f-8f4d-ffe2519d0992 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:35.085212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435fbbb-8583-4d8f-8f4d-ffe2519d0992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:35.085212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435fbbb-8583-4d8f-8f4d-ffe2519d0992 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:35.099719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e725968179a441099a3e13ae2d6d069'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:35.104819 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1979/3985] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:08:35.106237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435fbbb-8583-4d8f-8f4d-ffe2519d0992 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:35.123314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435fbbb-8583-4d8f-8f4d-ffe2519d0992 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:35.125425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e725968179a441099a3e13ae2d6d069'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:35.133771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bde8d3-6e87-404e-bbf6-8ee6f3c61d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:35.134487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bde8d3-6e87-404e-bbf6-8ee6f3c61d72 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:35.134972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bde8d3-6e87-404e-bbf6-8ee6f3c61d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:35.135452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bde8d3-6e87-404e-bbf6-8ee6f3c61d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:35.136148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bde8d3-6e87-404e-bbf6-8ee6f3c61d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:35.138081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e725968179a441099a3e13ae2d6d069'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:35.138932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bde8d3-6e87-404e-bbf6-8ee6f3c61d72 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:35.151246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e725968179a441099a3e13ae2d6d069'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:35.152949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f45e8de5-604d-428f-9bf4-e10f1dd31b7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e725968179a441099a3e13ae2d6d069'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:35.158914 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1980/3986] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:35.160393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bde8d3-6e87-404e-bbf6-8ee6f3c61d72 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:35.162476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-19bde8d3-6e87-404e-bbf6-8ee6f3c61d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:35.166082 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2007/3987] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:34 2022] DELETE /identity/v3/users/2e725968179a441099a3e13ae2d6d069 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:35.182550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bde8d3-6e87-404e-bbf6-8ee6f3c61d72 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:35.301763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39cc17-354c-47c4-bca0-af5fc42a7671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:35.302844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39cc17-354c-47c4-bca0-af5fc42a7671 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:35.303234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39cc17-354c-47c4-bca0-af5fc42a7671 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:35.303691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39cc17-354c-47c4-bca0-af5fc42a7671 admin admin] PATH_INFO: `/v3/projects/9ff9752441b2458d962bc20c9dc70f63/users/a7cd1ae2e4ea4aa48b33c7e9eda52b00/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:35.332269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39cc17-354c-47c4-bca0-af5fc42a7671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff9752441b2458d962bc20c9dc70f63, user_id=a7cd1ae2e4ea4aa48b33c7e9eda52b00, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:35.336233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39cc17-354c-47c4-bca0-af5fc42a7671 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:35.363085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1981/3988] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:35 2022] PUT /identity/v3/projects/9ff9752441b2458d962bc20c9dc70f63/users/a7cd1ae2e4ea4aa48b33c7e9eda52b00/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:35.376279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed58e27-8faf-4259-a741-f8c21c0e286a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:35.376634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed58e27-8faf-4259-a741-f8c21c0e286a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:35.376977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed58e27-8faf-4259-a741-f8c21c0e286a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:35.409648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4ed58e27-8faf-4259-a741-f8c21c0e286a None None] MFA Rules not processed for user `a7cd1ae2e4ea4aa48b33c7e9eda52b00`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:35.411136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4ed58e27-8faf-4259-a741-f8c21c0e286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:35.472284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:35.490805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39cc17-354c-47c4-bca0-af5fc42a7671 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:35.492750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2d39cc17-354c-47c4-bca0-af5fc42a7671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:35.505426 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2008/3989] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:35.519715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed58e27-8faf-4259-a741-f8c21c0e286a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:35.521362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4ed58e27-8faf-4259-a741-f8c21c0e286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:35.635133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39cc17-354c-47c4-bca0-af5fc42a7671 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:35.642614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f809847-7506-406c-99d5-ec128fbe01d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:35.645049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f809847-7506-406c-99d5-ec128fbe01d6 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:35.645327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f809847-7506-406c-99d5-ec128fbe01d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:35.645588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f809847-7506-406c-99d5-ec128fbe01d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:35.649308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5f809847-7506-406c-99d5-ec128fbe01d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:35.675660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f809847-7506-406c-99d5-ec128fbe01d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:35.677708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f809847-7506-406c-99d5-ec128fbe01d6 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:35.680009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed58e27-8faf-4259-a741-f8c21c0e286a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:35.689227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01375a9e-3846-4b0e-a564-202e6e268d09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:35.689922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01375a9e-3846-4b0e-a564-202e6e268d09 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:35.690196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01375a9e-3846-4b0e-a564-202e6e268d09 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:35.690464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01375a9e-3846-4b0e-a564-202e6e268d09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:35.701622 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1982/3990] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:08:35.709178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01375a9e-3846-4b0e-a564-202e6e268d09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:35.710638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01375a9e-3846-4b0e-a564-202e6e268d09 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:35.744170 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2009/3991] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:35.785937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f809847-7506-406c-99d5-ec128fbe01d6 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:35.787825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5f809847-7506-406c-99d5-ec128fbe01d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:35.799373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:35.912277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f809847-7506-406c-99d5-ec128fbe01d6 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:35.920822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:35.921490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:35.921761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:35.922026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] PATH_INFO: `/v3/projects/896ca2d11a7144d886405c44ea28ca33` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:35.923371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=896ca2d11a7144d886405c44ea28ca33)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:35.926101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:36.011280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896ca2d11a7144d886405c44ea28ca33', 'request_id': 'req-e6c12b5c-9230-439d-bceb-c5dace6aee54'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:36.031713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896ca2d11a7144d886405c44ea28ca33', 'request_id': 'req-e6c12b5c-9230-439d-bceb-c5dace6aee54'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:36.043397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896ca2d11a7144d886405c44ea28ca33', 'request_id': 'req-e6c12b5c-9230-439d-bceb-c5dace6aee54'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:36.060564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] The token cache is being invalidate because project 896ca2d11a7144d886405c44ea28ca33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:36.061017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:36.062132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6c12b5c-9230-439d-bceb-c5dace6aee54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:36.064195 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1983/3992] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:35 2022] DELETE /identity/v3/projects/896ca2d11a7144d886405c44ea28ca33 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:36.069158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d68c0-fd2e-4673-b0e7-7b0fd16ab9b7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:36.069444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d68c0-fd2e-4673-b0e7-7b0fd16ab9b7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:36.069718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d68c0-fd2e-4673-b0e7-7b0fd16ab9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:36.107830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fb6d68c0-fd2e-4673-b0e7-7b0fd16ab9b7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:36.109836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fb6d68c0-fd2e-4673-b0e7-7b0fd16ab9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:36.282437 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2010/3993] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:36.284857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c45c0-bcfd-4423-96f0-5727f2a9f027 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:36.286161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c45c0-bcfd-4423-96f0-5727f2a9f027 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:36.286161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c45c0-bcfd-4423-96f0-5727f2a9f027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:36.307919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b28c45c0-bcfd-4423-96f0-5727f2a9f027 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:36.309297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b28c45c0-bcfd-4423-96f0-5727f2a9f027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:36.321340 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1984/3994] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:36.326370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d68c0-fd2e-4673-b0e7-7b0fd16ab9b7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:36.338442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d68c0-fd2e-4673-b0e7-7b0fd16ab9b7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:36.345795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8241a14-d41b-4938-8d58-7957ef89b10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:36.346527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8241a14-d41b-4938-8d58-7957ef89b10b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:36.346917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8241a14-d41b-4938-8d58-7957ef89b10b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:36.347300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8241a14-d41b-4938-8d58-7957ef89b10b admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:36.348045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8241a14-d41b-4938-8d58-7957ef89b10b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:36.348444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8241a14-d41b-4938-8d58-7957ef89b10b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:36.350474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8241a14-d41b-4938-8d58-7957ef89b10b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:36.369792 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2011/3995] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:08:36.371064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c45c0-bcfd-4423-96f0-5727f2a9f027 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:36.383202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c45c0-bcfd-4423-96f0-5727f2a9f027 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:36.392246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878fad0-38f6-4c84-bbd4-65c6857e81c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:36.392864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878fad0-38f6-4c84-bbd4-65c6857e81c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:36.393181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878fad0-38f6-4c84-bbd4-65c6857e81c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:36.393426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878fad0-38f6-4c84-bbd4-65c6857e81c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:36.394056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878fad0-38f6-4c84-bbd4-65c6857e81c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:36.395989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878fad0-38f6-4c84-bbd4-65c6857e81c3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:36.446764 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1985/3996] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:36 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:36.451456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8241a14-d41b-4938-8d58-7957ef89b10b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:36.465851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8241a14-d41b-4938-8d58-7957ef89b10b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:36.542112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:36.551355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d17390-17a4-49cb-b198-30a8374200a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:36.552068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d17390-17a4-49cb-b198-30a8374200a8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:36.552723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d17390-17a4-49cb-b198-30a8374200a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:36.553091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d17390-17a4-49cb-b198-30a8374200a8 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:36.553801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d17390-17a4-49cb-b198-30a8374200a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:36.555919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d17390-17a4-49cb-b198-30a8374200a8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:36.586604 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2012/3997] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:36 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:36.590345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878fad0-38f6-4c84-bbd4-65c6857e81c3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:36.605850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878fad0-38f6-4c84-bbd4-65c6857e81c3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:36.612188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb2d33-ea83-4f0a-b005-1028d1a55a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:36.612821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2d33-ea83-4f0a-b005-1028d1a55a5f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:36.613287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2d33-ea83-4f0a-b005-1028d1a55a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:36.613588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2d33-ea83-4f0a-b005-1028d1a55a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:36.614238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb2d33-ea83-4f0a-b005-1028d1a55a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:36.616212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb2d33-ea83-4f0a-b005-1028d1a55a5f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:36.632769 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1986/3998] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:08:36.636604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d17390-17a4-49cb-b198-30a8374200a8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:36.655229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d17390-17a4-49cb-b198-30a8374200a8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:36.663752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8014e-00a5-465f-a00f-3cff91296f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:36.664484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8014e-00a5-465f-a00f-3cff91296f91 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:36.664625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8014e-00a5-465f-a00f-3cff91296f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:36.664896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8014e-00a5-465f-a00f-3cff91296f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:36.665566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8014e-00a5-465f-a00f-3cff91296f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:36.667194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8014e-00a5-465f-a00f-3cff91296f91 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:36.685184 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2013/3999] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:36.690463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb2d33-ea83-4f0a-b005-1028d1a55a5f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:36.702804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb2d33-ea83-4f0a-b005-1028d1a55a5f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:36.709469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd892cb1-656f-421d-802c-ed0896ad68a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:36.711457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd892cb1-656f-421d-802c-ed0896ad68a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:36.711741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd892cb1-656f-421d-802c-ed0896ad68a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:36.712005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd892cb1-656f-421d-802c-ed0896ad68a7 admin admin] PATH_INFO: `/v3/projects/1c6262061eb24eadbe654f2e7fc67757/users/f44a283fc85348d99de077f18498bbfd/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:36.735436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd892cb1-656f-421d-802c-ed0896ad68a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c6262061eb24eadbe654f2e7fc67757, user_id=f44a283fc85348d99de077f18498bbfd, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:36.738125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd892cb1-656f-421d-802c-ed0896ad68a7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:36.754405 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1987/4000] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:36 2022] PUT /identity/v3/projects/1c6262061eb24eadbe654f2e7fc67757/users/f44a283fc85348d99de077f18498bbfd/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:36.764831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a973f-3e57-4798-a9fc-0f5cd27400ea None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:36.765129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a973f-3e57-4798-a9fc-0f5cd27400ea None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:36.765399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a973f-3e57-4798-a9fc-0f5cd27400ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:36.800574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bb5a973f-3e57-4798-a9fc-0f5cd27400ea None None] MFA Rules not processed for user `f44a283fc85348d99de077f18498bbfd`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:36.801839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bb5a973f-3e57-4798-a9fc-0f5cd27400ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:36.877741 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2014/4001] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:36.886321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd892cb1-656f-421d-802c-ed0896ad68a7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:36.927883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:36.961372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd892cb1-656f-421d-802c-ed0896ad68a7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:36.969815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1322b9e-26d5-4578-a49c-29e963ace9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:36.970298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1322b9e-26d5-4578-a49c-29e963ace9c5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:36.970642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1322b9e-26d5-4578-a49c-29e963ace9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:36.971034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1322b9e-26d5-4578-a49c-29e963ace9c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:36.971921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1322b9e-26d5-4578-a49c-29e963ace9c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:36.973867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1322b9e-26d5-4578-a49c-29e963ace9c5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.008549 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1988/4002] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:36 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:37.009973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a973f-3e57-4798-a9fc-0f5cd27400ea None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.021363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a973f-3e57-4798-a9fc-0f5cd27400ea None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.082830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bf84a-a6d1-40e7-96ff-2083bc470fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.083430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bf84a-a6d1-40e7-96ff-2083bc470fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.083855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bf84a-a6d1-40e7-96ff-2083bc470fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.084128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bf84a-a6d1-40e7-96ff-2083bc470fd5 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.084759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6bf84a-a6d1-40e7-96ff-2083bc470fd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.086909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6bf84a-a6d1-40e7-96ff-2083bc470fd5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.118031 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2015/4003] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:37 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:37.120019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1322b9e-26d5-4578-a49c-29e963ace9c5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.132370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1322b9e-26d5-4578-a49c-29e963ace9c5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.138528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b3a07-529e-4ab5-8975-10bedc219241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.139165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b3a07-529e-4ab5-8975-10bedc219241 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.139446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b3a07-529e-4ab5-8975-10bedc219241 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.139745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b3a07-529e-4ab5-8975-10bedc219241 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.140447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b3a07-529e-4ab5-8975-10bedc219241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.142089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b3a07-529e-4ab5-8975-10bedc219241 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.154783 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1989/4004] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:37.158996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bf84a-a6d1-40e7-96ff-2083bc470fd5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.171148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bf84a-a6d1-40e7-96ff-2083bc470fd5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.177489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9be652-37e0-4586-9148-b8c9d8467cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.178578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9be652-37e0-4586-9148-b8c9d8467cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.178578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9be652-37e0-4586-9148-b8c9d8467cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.178905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9be652-37e0-4586-9148-b8c9d8467cfb admin admin] PATH_INFO: `/v3/projects/ba5914f1eac440e98bd6bb35685a635a/users/2ab448c386034e17a4619849a2fa6e68/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.206884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9be652-37e0-4586-9148-b8c9d8467cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba5914f1eac440e98bd6bb35685a635a, user_id=2ab448c386034e17a4619849a2fa6e68, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.210202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9be652-37e0-4586-9148-b8c9d8467cfb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.229332 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2016/4005] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:37 2022] PUT /identity/v3/projects/ba5914f1eac440e98bd6bb35685a635a/users/2ab448c386034e17a4619849a2fa6e68/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:37.234013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b3a07-529e-4ab5-8975-10bedc219241 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.248317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b3a07-529e-4ab5-8975-10bedc219241 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.361913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b40508-8e6b-42ea-a092-3369d4e002ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.362448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40508-8e6b-42ea-a092-3369d4e002ce admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.362716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40508-8e6b-42ea-a092-3369d4e002ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.362980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40508-8e6b-42ea-a092-3369d4e002ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.363609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b40508-8e6b-42ea-a092-3369d4e002ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.365995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b40508-8e6b-42ea-a092-3369d4e002ce admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.377896 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1990/4006] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:37.382103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9be652-37e0-4586-9148-b8c9d8467cfb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.393345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9be652-37e0-4586-9148-b8c9d8467cfb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.399249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99636554-a441-409d-9269-0cc3b003225f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.399720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99636554-a441-409d-9269-0cc3b003225f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.399998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99636554-a441-409d-9269-0cc3b003225f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.400269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99636554-a441-409d-9269-0cc3b003225f admin admin] PATH_INFO: `/v3/projects/ba5914f1eac440e98bd6bb35685a635a/users/2ab448c386034e17a4619849a2fa6e68/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.402374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99636554-a441-409d-9269-0cc3b003225f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba5914f1eac440e98bd6bb35685a635a, user_id=2ab448c386034e17a4619849a2fa6e68, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.405618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99636554-a441-409d-9269-0cc3b003225f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.420028 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2017/4007] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:37 2022] PUT /identity/v3/projects/ba5914f1eac440e98bd6bb35685a635a/users/2ab448c386034e17a4619849a2fa6e68/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:37.431850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5694db8-84c2-47f8-882a-635861c7d952 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.432484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5694db8-84c2-47f8-882a-635861c7d952 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.433706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5694db8-84c2-47f8-882a-635861c7d952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.467691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c5694db8-84c2-47f8-882a-635861c7d952 None None] MFA Rules not processed for user `2ab448c386034e17a4619849a2fa6e68`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:37.469379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c5694db8-84c2-47f8-882a-635861c7d952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:37.558959 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1991/4008] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:37.563770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99636554-a441-409d-9269-0cc3b003225f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.578942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99636554-a441-409d-9269-0cc3b003225f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.587948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95019d66-3489-412b-bcb8-18699fa05c2c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.588813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95019d66-3489-412b-bcb8-18699fa05c2c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.589593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95019d66-3489-412b-bcb8-18699fa05c2c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.589874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95019d66-3489-412b-bcb8-18699fa05c2c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.590628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95019d66-3489-412b-bcb8-18699fa05c2c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.592453 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-95019d66-3489-412b-bcb8-18699fa05c2c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:37.592453 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:37.592453 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:37.593389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95019d66-3489-412b-bcb8-18699fa05c2c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.616666 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2018/4009] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:37 2022] POST /identity/v3/services => generated 290 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:37.621424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5694db8-84c2-47f8-882a-635861c7d952 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.633255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5694db8-84c2-47f8-882a-635861c7d952 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.651141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459762e-9b49-4964-8542-d9c7dafc81d0 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.651759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459762e-9b49-4964-8542-d9c7dafc81d0 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.652054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459762e-9b49-4964-8542-d9c7dafc81d0 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.652314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459762e-9b49-4964-8542-d9c7dafc81d0 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.652964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459762e-9b49-4964-8542-d9c7dafc81d0 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.654232 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-1459762e-9b49-4964-8542-d9c7dafc81d0 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:37.654232 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:37.654232 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:37.655118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459762e-9b49-4964-8542-d9c7dafc81d0 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-1459762e-9b49-4964-8542-d9c7dafc81d0 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: On instance['enabled']: Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: On instance['enabled']: Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: 'False' Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise error Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application On instance['enabled']: Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application 'False' Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.667436 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 15 21:08:37.668705 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 15 21:08:37.670008 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 15 21:08:37.670008 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 15 21:08:37.670008 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.670008 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.670008 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.670008 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.670008 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application On instance['enabled']: Oct 15 21:08:37.670008 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application 'False' Oct 15 21:08:37.670008 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.672607 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1992/4010] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:08:37 2022] POST /identity/v3/endpoints => generated 283 bytes in 52 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 15 21:08:37.674003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95019d66-3489-412b-bcb8-18699fa05c2c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.681332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:37.689376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95019d66-3489-412b-bcb8-18699fa05c2c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.696478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6d062-23d5-4922-b061-181afe4cb75c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.697140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6d062-23d5-4922-b061-181afe4cb75c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.697675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6d062-23d5-4922-b061-181afe4cb75c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.698046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6d062-23d5-4922-b061-181afe4cb75c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.698821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6d062-23d5-4922-b061-181afe4cb75c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.699928 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-fea6d062-23d5-4922-b061-181afe4cb75c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:37.699928 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:37.699928 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:37.701886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6d062-23d5-4922-b061-181afe4cb75c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-fea6d062-23d5-4922-b061-181afe4cb75c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: On instance['enabled']: Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: On instance['enabled']: Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: 'True' Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise error Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application On instance['enabled']: Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application 'True' Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:37.711662 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 15 21:08:37.718786 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 15 21:08:37.719985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 15 21:08:37.719985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 15 21:08:37.719985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:37.719985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.719985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.719985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:37.719985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application On instance['enabled']: Oct 15 21:08:37.719985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application 'True' Oct 15 21:08:37.719985 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:37.719985 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2019/4011] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:08:37 2022] POST /identity/v3/endpoints => generated 281 bytes in 40 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Oct 15 21:08:37.720690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459762e-9b49-4964-8542-d9c7dafc81d0 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.734303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459762e-9b49-4964-8542-d9c7dafc81d0 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.739947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccd56d-c0b4-4efe-9d2c-602548336921 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.740263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccd56d-c0b4-4efe-9d2c-602548336921 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.740539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccd56d-c0b4-4efe-9d2c-602548336921 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.740805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccd56d-c0b4-4efe-9d2c-602548336921 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.741518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccd56d-c0b4-4efe-9d2c-602548336921 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.743429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccd56d-c0b4-4efe-9d2c-602548336921 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.756244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:37.788065 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-a1ccd56d-c0b4-4efe-9d2c-602548336921 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-425320595. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:08:37.788065 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:08:37.788065 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:37.822148 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1993/4012] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:08:37 2022] POST /identity/v3/endpoints => generated 358 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:37.826577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6d062-23d5-4922-b061-181afe4cb75c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.837538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6d062-23d5-4922-b061-181afe4cb75c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.857928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d967b-00c6-413e-8dcf-086bf96bae56 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.858660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d967b-00c6-413e-8dcf-086bf96bae56 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.858828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d967b-00c6-413e-8dcf-086bf96bae56 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.859092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d967b-00c6-413e-8dcf-086bf96bae56 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/regions/tempest-region-425320595` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.859664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d967b-00c6-413e-8dcf-086bf96bae56 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-425320595)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.860740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d967b-00c6-413e-8dcf-086bf96bae56 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.865073 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2020/4013] 10.222.0.16 () {58 vars in 1203 bytes} [Sat Oct 15 21:08:37 2022] GET /identity/v3/regions/tempest-region-425320595 => generated 178 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:37.867081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccd56d-c0b4-4efe-9d2c-602548336921 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.878550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccd56d-c0b4-4efe-9d2c-602548336921 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.883781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f6b09-cb61-472e-86cb-244d7e355563 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.884488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f6b09-cb61-472e-86cb-244d7e355563 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.884764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f6b09-cb61-472e-86cb-244d7e355563 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.885067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f6b09-cb61-472e-86cb-244d7e355563 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/endpoints/0d20b77f9cf1423ca13be71a059710ca` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.892209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f6b09-cb61-472e-86cb-244d7e355563 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=0d20b77f9cf1423ca13be71a059710ca)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.893574 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-d36f6b09-cb61-472e-86cb-244d7e355563 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:37.893574 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:37.893574 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:37.894456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f6b09-cb61-472e-86cb-244d7e355563 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-d36f6b09-cb61-472e-86cb-244d7e355563 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: On instance['enabled']: Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: On instance['enabled']: Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: 'False' Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise error Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application On instance['enabled']: Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application 'False' Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.901653 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 15 21:08:37.903188 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 15 21:08:37.907829 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 15 21:08:37.907829 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 15 21:08:37.907829 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.907829 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:37.907829 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:37.907829 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.907829 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application On instance['enabled']: Oct 15 21:08:37.907829 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application 'False' Oct 15 21:08:37.907829 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:37.907829 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1994/4014] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Oct 15 21:08:37 2022] PATCH /identity/v3/endpoints/0d20b77f9cf1423ca13be71a059710ca => generated 283 bytes in 37 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 15 21:08:37.910119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d967b-00c6-413e-8dcf-086bf96bae56 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.921316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d967b-00c6-413e-8dcf-086bf96bae56 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:37.928297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:37.929126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:37.929405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:37.929672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/endpoints/0d20b77f9cf1423ca13be71a059710ca` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:37.930861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=0d20b77f9cf1423ca13be71a059710ca)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:37.931862 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:37.931862 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:37.931862 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:37.932864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:37.950192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '0d20b77f9cf1423ca13be71a059710ca', 'request_id': 'req-c81124e7-5aac-409c-a215-d4e186f6b7f3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:37.963509 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2021/4015] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:37 2022] DELETE /identity/v3/endpoints/0d20b77f9cf1423ca13be71a059710ca => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:37.967548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f6b09-cb61-472e-86cb-244d7e355563 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:37.978937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f6b09-cb61-472e-86cb-244d7e355563 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:38.001350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:38.001566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eb843-3ac8-4001-9eda-bb7890d4b99c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:38.002524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eb843-3ac8-4001-9eda-bb7890d4b99c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.002945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eb843-3ac8-4001-9eda-bb7890d4b99c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.003059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:38.003468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eb843-3ac8-4001-9eda-bb7890d4b99c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/regions/tempest-region-425320595` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.004291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9eb843-3ac8-4001-9eda-bb7890d4b99c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-425320595)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:38.005658 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-7c9eb843-3ac8-4001-9eda-bb7890d4b99c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:38.005658 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:38.005658 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:38.007197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9eb843-3ac8-4001-9eda-bb7890d4b99c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:38.042076 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1995/4016] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 21:08:37 2022] DELETE /identity/v3/regions/tempest-region-425320595 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:38.042861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eb843-3ac8-4001-9eda-bb7890d4b99c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:38.070080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eb843-3ac8-4001-9eda-bb7890d4b99c tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:38.076284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7c488-df31-4ca2-b582-340462f0026e tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:38.076494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7c488-df31-4ca2-b582-340462f0026e tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.076751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7c488-df31-4ca2-b582-340462f0026e tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.077017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7c488-df31-4ca2-b582-340462f0026e tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.077703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7c488-df31-4ca2-b582-340462f0026e tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:38.079702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7c488-df31-4ca2-b582-340462f0026e tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:38.102968 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-c3c7c488-df31-4ca2-b582-340462f0026e tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1082560078. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:08:38.102968 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:08:38.102968 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:38.129568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81124e7-5aac-409c-a215-d4e186f6b7f3 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:38.142865 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1996/4017] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:08:38 2022] POST /identity/v3/endpoints => generated 360 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:38.143926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7c488-df31-4ca2-b582-340462f0026e tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:38.153084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e61d-0986-477b-94b9-07db2f3b9df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:38.154500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e61d-0986-477b-94b9-07db2f3b9df0 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.154782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e61d-0986-477b-94b9-07db2f3b9df0 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.155056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e61d-0986-477b-94b9-07db2f3b9df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.157301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b407e61d-0986-477b-94b9-07db2f3b9df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:38.167174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7c488-df31-4ca2-b582-340462f0026e tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:38.171916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407e61d-0986-477b-94b9-07db2f3b9df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:38.172996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab824a-44d0-4abf-93ec-f128ff627011 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:38.173674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab824a-44d0-4abf-93ec-f128ff627011 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.174118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab824a-44d0-4abf-93ec-f128ff627011 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.174255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407e61d-0986-477b-94b9-07db2f3b9df0 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:38.174412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab824a-44d0-4abf-93ec-f128ff627011 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/regions/tempest-region-1082560078` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.174956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab824a-44d0-4abf-93ec-f128ff627011 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1082560078)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:38.177004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab824a-44d0-4abf-93ec-f128ff627011 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:38.180687 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1997/4018] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 21:08:38 2022] GET /identity/v3/regions/tempest-region-1082560078 => generated 180 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:38.185589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab824a-44d0-4abf-93ec-f128ff627011 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:38.188765 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2022/4019] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:38 2022] GET /identity/v3/auth/tokens => generated 3688 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:38.197278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab824a-44d0-4abf-93ec-f128ff627011 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:38.202406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fc84c-8a6b-4b9c-90e1-a6bd691d05eb tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:38.203139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fc84c-8a6b-4b9c-90e1-a6bd691d05eb tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.203419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fc84c-8a6b-4b9c-90e1-a6bd691d05eb tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.203683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fc84c-8a6b-4b9c-90e1-a6bd691d05eb tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/endpoints/d2ed4ea452e94a9b8348dbbe903a2822` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.209416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fc84c-8a6b-4b9c-90e1-a6bd691d05eb tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=d2ed4ea452e94a9b8348dbbe903a2822)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:38.211040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fc84c-8a6b-4b9c-90e1-a6bd691d05eb tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-f00fc84c-8a6b-4b9c-90e1-a6bd691d05eb tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: On instance['enabled']: Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: On instance['enabled']: Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: 'True' Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise error Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application On instance['enabled']: Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application 'True' Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:38.218777 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 15 21:08:38.220267 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 15 21:08:38.221295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 15 21:08:38.221295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 15 21:08:38.221295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:38.221295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 15 21:08:38.221295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 15 21:08:38.221295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:38.221295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application On instance['enabled']: Oct 15 21:08:38.221295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application 'True' Oct 15 21:08:38.221295 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:08:38.221295 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1998/4020] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Oct 15 21:08:38 2022] PATCH /identity/v3/endpoints/d2ed4ea452e94a9b8348dbbe903a2822 => generated 281 bytes in 36 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Oct 15 21:08:38.224180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e61d-0986-477b-94b9-07db2f3b9df0 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:38.261435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e61d-0986-477b-94b9-07db2f3b9df0 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:38.267103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0a190-dbf1-477e-99fe-dff0a09a5eb4 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:38.267730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0a190-dbf1-477e-99fe-dff0a09a5eb4 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.268011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0a190-dbf1-477e-99fe-dff0a09a5eb4 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.268443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0a190-dbf1-477e-99fe-dff0a09a5eb4 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/endpoints/d2ed4ea452e94a9b8348dbbe903a2822` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.269892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0a190-dbf1-477e-99fe-dff0a09a5eb4 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=d2ed4ea452e94a9b8348dbbe903a2822)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:38.271485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0a190-dbf1-477e-99fe-dff0a09a5eb4 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:38.281392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2ab0a190-dbf1-477e-99fe-dff0a09a5eb4 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'd2ed4ea452e94a9b8348dbbe903a2822', 'request_id': 'req-2ab0a190-dbf1-477e-99fe-dff0a09a5eb4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:38.293098 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2023/4021] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:38 2022] DELETE /identity/v3/endpoints/d2ed4ea452e94a9b8348dbbe903a2822 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:38.297984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fc84c-8a6b-4b9c-90e1-a6bd691d05eb tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:38.310862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fc84c-8a6b-4b9c-90e1-a6bd691d05eb tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:38.330757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa35b7-7978-478f-b223-4f9c70d2cefa tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:38.331618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa35b7-7978-478f-b223-4f9c70d2cefa tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.331958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa35b7-7978-478f-b223-4f9c70d2cefa tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.332322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa35b7-7978-478f-b223-4f9c70d2cefa tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/regions/tempest-region-1082560078` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.333001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa35b7-7978-478f-b223-4f9c70d2cefa tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1082560078)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:38.335031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa35b7-7978-478f-b223-4f9c70d2cefa tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:38.369241 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 1999/4022] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Oct 15 21:08:38 2022] DELETE /identity/v3/regions/tempest-region-1082560078 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:38.369332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0a190-dbf1-477e-99fe-dff0a09a5eb4 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:38.382459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0a190-dbf1-477e-99fe-dff0a09a5eb4 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:38.411662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e295f-03af-4147-890a-3b66bba32f76 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ab448c386034e17a4619849a2fa6e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba5914f1eac440e98bd6bb35685a635a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:38.413007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e295f-03af-4147-890a-3b66bba32f76 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.413634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e295f-03af-4147-890a-3b66bba32f76 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.413982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e295f-03af-4147-890a-3b66bba32f76 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] PATH_INFO: `/v3/services/b826504f2a204bb0b2534e32ba96ed8f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.414595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e295f-03af-4147-890a-3b66bba32f76 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorizing `identity:delete_service(service_id=b826504f2a204bb0b2534e32ba96ed8f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:38.417110 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-970e295f-03af-4147-890a-3b66bba32f76 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:38.417110 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:38.417110 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:38.417110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e295f-03af-4147-890a-3b66bba32f76 tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:38.453191 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2024/4023] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:38 2022] DELETE /identity/v3/services/b826504f2a204bb0b2534e32ba96ed8f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:38.454728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa35b7-7978-478f-b223-4f9c70d2cefa tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:38.523979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:38.544734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa35b7-7978-478f-b223-4f9c70d2cefa tempest-EndpointsNegativeTestJSON-608190400 tempest-EndpointsNegativeTestJSON-608190400-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:38.573108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:38.573770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.574048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.574318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] PATH_INFO: `/v3/users/f44a283fc85348d99de077f18498bbfd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.576167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f44a283fc85348d99de077f18498bbfd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:38.578339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:38.634488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f44a283fc85348d99de077f18498bbfd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:38.644254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f44a283fc85348d99de077f18498bbfd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:38.659309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f44a283fc85348d99de077f18498bbfd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:38.666784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f44a283fc85348d99de077f18498bbfd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:38.667444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f44a283fc85348d99de077f18498bbfd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:38.673960 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2000/4024] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:38 2022] DELETE /identity/v3/users/f44a283fc85348d99de077f18498bbfd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:38.678494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4cfca-33b3-4a97-872d-b761171c25de None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.678856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4cfca-33b3-4a97-872d-b761171c25de None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.679132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4cfca-33b3-4a97-872d-b761171c25de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.698466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a9f4cfca-33b3-4a97-872d-b761171c25de None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:38.699679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a9f4cfca-33b3-4a97-872d-b761171c25de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:38.736274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:38.768545 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2025/4025] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:38.778356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:38.780432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:38.849479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe7b2b-af38-4ee8-9c9a-22b00ea6bf0a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:38.853267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640546b7-4aa0-455f-94ba-65661ba93846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:38.853871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640546b7-4aa0-455f-94ba-65661ba93846 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:38.854150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640546b7-4aa0-455f-94ba-65661ba93846 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:38.854410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640546b7-4aa0-455f-94ba-65661ba93846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:38.865344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640546b7-4aa0-455f-94ba-65661ba93846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:38.866770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640546b7-4aa0-455f-94ba-65661ba93846 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:38.877320 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2001/4026] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:39.170653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4cfca-33b3-4a97-872d-b761171c25de None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:39.172609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a9f4cfca-33b3-4a97-872d-b761171c25de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:39.190733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4cfca-33b3-4a97-872d-b761171c25de None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:39.197519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:39.198162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:39.198440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:39.198711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] PATH_INFO: `/v3/projects/1c6262061eb24eadbe654f2e7fc67757` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:39.200173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c6262061eb24eadbe654f2e7fc67757)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:39.203015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:39.278933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c6262061eb24eadbe654f2e7fc67757', 'request_id': 'req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:39.295625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c6262061eb24eadbe654f2e7fc67757', 'request_id': 'req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:39.305009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c6262061eb24eadbe654f2e7fc67757', 'request_id': 'req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:39.335783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] The token cache is being invalidate because project 1c6262061eb24eadbe654f2e7fc67757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:39.336457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:39.337351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93696f70-d2ed-4abc-80cd-5ed9f91ebf8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:39.339047 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2026/4027] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:39 2022] DELETE /identity/v3/projects/1c6262061eb24eadbe654f2e7fc67757 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:39.343980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640546b7-4aa0-455f-94ba-65661ba93846 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:39.355485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640546b7-4aa0-455f-94ba-65661ba93846 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:39.430888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:39.448487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:39.449506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:39.450098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:39.451323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] PATH_INFO: `/v3/users/2ab448c386034e17a4619849a2fa6e68` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:39.453138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab448c386034e17a4619849a2fa6e68)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:39.457091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:39.535613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab448c386034e17a4619849a2fa6e68'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:39.545489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab448c386034e17a4619849a2fa6e68'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:39.568266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab448c386034e17a4619849a2fa6e68'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:39.588451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab448c386034e17a4619849a2fa6e68'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:39.589308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab448c386034e17a4619849a2fa6e68'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:39.607925 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2002/4028] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:39 2022] DELETE /identity/v3/users/2ab448c386034e17a4619849a2fa6e68 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:39.623370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568fd1-d05a-413b-a27e-9a3ce03b179b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:39.623904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568fd1-d05a-413b-a27e-9a3ce03b179b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:39.624352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568fd1-d05a-413b-a27e-9a3ce03b179b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:39.753626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-57568fd1-d05a-413b-a27e-9a3ce03b179b None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:39.759935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-57568fd1-d05a-413b-a27e-9a3ce03b179b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:39.873955 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2027/4029] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:08:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:39.908890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:39.911567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:40.114968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568fd1-d05a-413b-a27e-9a3ce03b179b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:40.116479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-57568fd1-d05a-413b-a27e-9a3ce03b179b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:40.149284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a90edf-99d0-444a-bfaf-cddc6b1b42b2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:40.159952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbb4db-0da6-41ca-a721-95a5a836253d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:40.160628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb4db-0da6-41ca-a721-95a5a836253d service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:40.160934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb4db-0da6-41ca-a721-95a5a836253d service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:40.161792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb4db-0da6-41ca-a721-95a5a836253d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:40.173665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbb4db-0da6-41ca-a721-95a5a836253d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:40.175628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbb4db-0da6-41ca-a721-95a5a836253d service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:40.186696 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2003/4030] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:08:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:40.208719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:40.318003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568fd1-d05a-413b-a27e-9a3ce03b179b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:40.336239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:40.337208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:40.337486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:40.337916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] PATH_INFO: `/v3/projects/ba5914f1eac440e98bd6bb35685a635a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:40.345709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba5914f1eac440e98bd6bb35685a635a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:40.359946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:40.480726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba5914f1eac440e98bd6bb35685a635a', 'request_id': 'req-4d3686d9-6871-47fb-92f8-655812ef89f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:40.496277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba5914f1eac440e98bd6bb35685a635a', 'request_id': 'req-4d3686d9-6871-47fb-92f8-655812ef89f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:40.505029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba5914f1eac440e98bd6bb35685a635a', 'request_id': 'req-4d3686d9-6871-47fb-92f8-655812ef89f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:40.534760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] The token cache is being invalidate because project ba5914f1eac440e98bd6bb35685a635a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:40.538238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:40.539334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d3686d9-6871-47fb-92f8-655812ef89f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:40.543586 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2028/4031] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:40 2022] DELETE /identity/v3/projects/ba5914f1eac440e98bd6bb35685a635a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:40.549421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99137b-0f91-4a0b-af84-0fa28444dcd1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:40.549707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99137b-0f91-4a0b-af84-0fa28444dcd1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:40.549974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99137b-0f91-4a0b-af84-0fa28444dcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:40.588456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bd99137b-0f91-4a0b-af84-0fa28444dcd1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:40.593768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bd99137b-0f91-4a0b-af84-0fa28444dcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:40.663513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:40.698425 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2004/4032] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:40.707105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8cabf-71a9-4015-80bc-60c714232daa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:40.707638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8cabf-71a9-4015-80bc-60c714232daa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:40.708113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8cabf-71a9-4015-80bc-60c714232daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:40.741089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-acb8cabf-71a9-4015-80bc-60c714232daa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:40.743825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-acb8cabf-71a9-4015-80bc-60c714232daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:40.774311 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2029/4033] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:40.780970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99137b-0f91-4a0b-af84-0fa28444dcd1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:40.797104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99137b-0f91-4a0b-af84-0fa28444dcd1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:40.815001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fb48f-053d-4aa8-894d-2da9b848a089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:40.815571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fb48f-053d-4aa8-894d-2da9b848a089 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:40.815910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fb48f-053d-4aa8-894d-2da9b848a089 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:40.816190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fb48f-053d-4aa8-894d-2da9b848a089 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:40.816917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fb48f-053d-4aa8-894d-2da9b848a089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:40.817321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fb48f-053d-4aa8-894d-2da9b848a089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:40.818999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fb48f-053d-4aa8-894d-2da9b848a089 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:40.851043 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2005/4034] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:40.852619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8cabf-71a9-4015-80bc-60c714232daa None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:40.865442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8cabf-71a9-4015-80bc-60c714232daa None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:40.877692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833f321-8458-400a-8317-cc68c1cd580a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:40.879176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833f321-8458-400a-8317-cc68c1cd580a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:40.879455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833f321-8458-400a-8317-cc68c1cd580a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:40.879807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833f321-8458-400a-8317-cc68c1cd580a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:40.881321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833f321-8458-400a-8317-cc68c1cd580a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:40.884795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833f321-8458-400a-8317-cc68c1cd580a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:40.937613 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2030/4035] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:40 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:40.939247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fb48f-053d-4aa8-894d-2da9b848a089 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:40.954947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fb48f-053d-4aa8-894d-2da9b848a089 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:41.034846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e87cd-f3a3-46d7-b8c3-50c94bf24d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:41.035483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e87cd-f3a3-46d7-b8c3-50c94bf24d87 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:41.035758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e87cd-f3a3-46d7-b8c3-50c94bf24d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:41.036028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e87cd-f3a3-46d7-b8c3-50c94bf24d87 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:41.036649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e87cd-f3a3-46d7-b8c3-50c94bf24d87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:41.039567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e87cd-f3a3-46d7-b8c3-50c94bf24d87 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:41.065620 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2006/4036] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:40 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:41.069721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833f321-8458-400a-8317-cc68c1cd580a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:41.082896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833f321-8458-400a-8317-cc68c1cd580a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:41.087899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614c4b4-9a79-4090-9e29-434ab4547f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:41.088496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614c4b4-9a79-4090-9e29-434ab4547f2f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:41.088771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614c4b4-9a79-4090-9e29-434ab4547f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:41.089057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614c4b4-9a79-4090-9e29-434ab4547f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:41.089705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614c4b4-9a79-4090-9e29-434ab4547f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:41.092322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614c4b4-9a79-4090-9e29-434ab4547f2f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:41.107820 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2031/4037] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:41.109821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e87cd-f3a3-46d7-b8c3-50c94bf24d87 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:41.123080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e87cd-f3a3-46d7-b8c3-50c94bf24d87 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:41.129463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d5543-f947-43d5-9aa4-1ee299840bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:41.130072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d5543-f947-43d5-9aa4-1ee299840bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:41.130349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d5543-f947-43d5-9aa4-1ee299840bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:41.130615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d5543-f947-43d5-9aa4-1ee299840bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:41.131260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d5543-f947-43d5-9aa4-1ee299840bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:41.132842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d5543-f947-43d5-9aa4-1ee299840bc6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:41.148393 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2007/4038] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:41.150367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614c4b4-9a79-4090-9e29-434ab4547f2f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:41.171587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614c4b4-9a79-4090-9e29-434ab4547f2f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:41.182453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c193e8c-52c2-4605-a402-3a338cf34339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:41.183364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c193e8c-52c2-4605-a402-3a338cf34339 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:41.183718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c193e8c-52c2-4605-a402-3a338cf34339 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:41.184078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c193e8c-52c2-4605-a402-3a338cf34339 admin admin] PATH_INFO: `/v3/projects/4d78f7c68b5d47f6b7d97f48d4ecc82e/users/e015d0722993404490ffa222cec39446/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:41.237259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c193e8c-52c2-4605-a402-3a338cf34339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d78f7c68b5d47f6b7d97f48d4ecc82e, user_id=e015d0722993404490ffa222cec39446, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:41.242130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c193e8c-52c2-4605-a402-3a338cf34339 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:41.262102 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2032/4039] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:41 2022] PUT /identity/v3/projects/4d78f7c68b5d47f6b7d97f48d4ecc82e/users/e015d0722993404490ffa222cec39446/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:41.276758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6f541-053d-40ae-be67-721f881d90cc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:41.277300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6f541-053d-40ae-be67-721f881d90cc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:41.277767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6f541-053d-40ae-be67-721f881d90cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:41.320478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4af6f541-053d-40ae-be67-721f881d90cc None None] MFA Rules not processed for user `e015d0722993404490ffa222cec39446`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:41.322174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4af6f541-053d-40ae-be67-721f881d90cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:41.469387 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2008/4040] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:41.470558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c193e8c-52c2-4605-a402-3a338cf34339 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:41.583656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:41.594221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c193e8c-52c2-4605-a402-3a338cf34339 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:41.605948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8245d4-fe63-402b-a566-2d2b11a00ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:41.606601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8245d4-fe63-402b-a566-2d2b11a00ece admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:41.606881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8245d4-fe63-402b-a566-2d2b11a00ece admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:41.607150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8245d4-fe63-402b-a566-2d2b11a00ece admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:41.607791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8245d4-fe63-402b-a566-2d2b11a00ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:41.610590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8245d4-fe63-402b-a566-2d2b11a00ece admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:41.658086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2033/4041] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:41 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:41.662108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6f541-053d-40ae-be67-721f881d90cc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:41.680354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6f541-053d-40ae-be67-721f881d90cc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:41.753345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:41.800361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e71eb3-055d-4cde-b2db-53064f468db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:41.800982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e71eb3-055d-4cde-b2db-53064f468db0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:41.801818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e71eb3-055d-4cde-b2db-53064f468db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:41.801818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e71eb3-055d-4cde-b2db-53064f468db0 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:41.802191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e71eb3-055d-4cde-b2db-53064f468db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:41.804636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e71eb3-055d-4cde-b2db-53064f468db0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:41.853085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2009/4042] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:41 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:41.854985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8245d4-fe63-402b-a566-2d2b11a00ece admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:41.873147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8245d4-fe63-402b-a566-2d2b11a00ece admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:41.878692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f21a5-4036-4e89-bcf5-3d666f743466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:41.879536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f21a5-4036-4e89-bcf5-3d666f743466 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:41.879817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f21a5-4036-4e89-bcf5-3d666f743466 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:41.880080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f21a5-4036-4e89-bcf5-3d666f743466 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:41.880748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f21a5-4036-4e89-bcf5-3d666f743466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:41.882401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f21a5-4036-4e89-bcf5-3d666f743466 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:41.897995 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2034/4043] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:41.900116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e71eb3-055d-4cde-b2db-53064f468db0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:41.911697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e71eb3-055d-4cde-b2db-53064f468db0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:41.917229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae20328-fe28-4e2d-ba4d-05f4a927602d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:41.918003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae20328-fe28-4e2d-ba4d-05f4a927602d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:41.918344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae20328-fe28-4e2d-ba4d-05f4a927602d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:41.918687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae20328-fe28-4e2d-ba4d-05f4a927602d admin admin] PATH_INFO: `/v3/projects/7a1e08ae0e92498b8f0ce0f45b15a1b3/users/073feb0f3ef3445c83b5d75a081451fb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:41.937185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae20328-fe28-4e2d-ba4d-05f4a927602d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1e08ae0e92498b8f0ce0f45b15a1b3, user_id=073feb0f3ef3445c83b5d75a081451fb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:41.940416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae20328-fe28-4e2d-ba4d-05f4a927602d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:41.961114 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2010/4044] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:41 2022] PUT /identity/v3/projects/7a1e08ae0e92498b8f0ce0f45b15a1b3/users/073feb0f3ef3445c83b5d75a081451fb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:41.965104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f21a5-4036-4e89-bcf5-3d666f743466 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:41.978599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f21a5-4036-4e89-bcf5-3d666f743466 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:42.063501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fad65-6d53-4fe1-8e7e-99a47e05fe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:42.064095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fad65-6d53-4fe1-8e7e-99a47e05fe24 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:42.064415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fad65-6d53-4fe1-8e7e-99a47e05fe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:42.064689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fad65-6d53-4fe1-8e7e-99a47e05fe24 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:42.065527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fad65-6d53-4fe1-8e7e-99a47e05fe24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:42.067181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fad65-6d53-4fe1-8e7e-99a47e05fe24 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:42.082199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae20328-fe28-4e2d-ba4d-05f4a927602d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:42.083089 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2035/4045] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:42.094193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae20328-fe28-4e2d-ba4d-05f4a927602d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:42.098971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c843863-50e1-4b0d-aa13-0f1b87d4a10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:42.100225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c843863-50e1-4b0d-aa13-0f1b87d4a10e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:42.100225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c843863-50e1-4b0d-aa13-0f1b87d4a10e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:42.100225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c843863-50e1-4b0d-aa13-0f1b87d4a10e admin admin] PATH_INFO: `/v3/projects/7a1e08ae0e92498b8f0ce0f45b15a1b3/users/073feb0f3ef3445c83b5d75a081451fb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:42.102548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c843863-50e1-4b0d-aa13-0f1b87d4a10e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1e08ae0e92498b8f0ce0f45b15a1b3, user_id=073feb0f3ef3445c83b5d75a081451fb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:42.104923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c843863-50e1-4b0d-aa13-0f1b87d4a10e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:42.118660 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2011/4046] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:42 2022] PUT /identity/v3/projects/7a1e08ae0e92498b8f0ce0f45b15a1b3/users/073feb0f3ef3445c83b5d75a081451fb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:42.137907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315113f-5d4a-468e-9d42-d592e2765daa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:42.137907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315113f-5d4a-468e-9d42-d592e2765daa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:42.138408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315113f-5d4a-468e-9d42-d592e2765daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:42.180259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a315113f-5d4a-468e-9d42-d592e2765daa None None] MFA Rules not processed for user `073feb0f3ef3445c83b5d75a081451fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:42.181983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a315113f-5d4a-468e-9d42-d592e2765daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:42.287343 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2036/4047] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:42.288523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c843863-50e1-4b0d-aa13-0f1b87d4a10e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:42.302645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c843863-50e1-4b0d-aa13-0f1b87d4a10e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:42.308718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532aa0-99f0-44d7-9427-f5b825c1d400 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073feb0f3ef3445c83b5d75a081451fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a1e08ae0e92498b8f0ce0f45b15a1b3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:42.309672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532aa0-99f0-44d7-9427-f5b825c1d400 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:42.310555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532aa0-99f0-44d7-9427-f5b825c1d400 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:42.310928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532aa0-99f0-44d7-9427-f5b825c1d400 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:42.312333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c532aa0-99f0-44d7-9427-f5b825c1d400 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:42.314792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c532aa0-99f0-44d7-9427-f5b825c1d400 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:42.419891 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2012/4048] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:42 2022] POST /identity/v3/projects => generated 360 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:42.421349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315113f-5d4a-468e-9d42-d592e2765daa None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:42.536592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:42.551038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315113f-5d4a-468e-9d42-d592e2765daa None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:42.646508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ea0ea-a981-4fc3-a414-d2d8765f7b9e tempest-ProjectsNegativeTestJSON-1245610467 tempest-ProjectsNegativeTestJSON-1245610467-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e015d0722993404490ffa222cec39446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d78f7c68b5d47f6b7d97f48d4ecc82e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:42.647138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ea0ea-a981-4fc3-a414-d2d8765f7b9e tempest-ProjectsNegativeTestJSON-1245610467 tempest-ProjectsNegativeTestJSON-1245610467-project] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:42.647458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ea0ea-a981-4fc3-a414-d2d8765f7b9e tempest-ProjectsNegativeTestJSON-1245610467 tempest-ProjectsNegativeTestJSON-1245610467-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:42.647693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ea0ea-a981-4fc3-a414-d2d8765f7b9e tempest-ProjectsNegativeTestJSON-1245610467 tempest-ProjectsNegativeTestJSON-1245610467-project] PATH_INFO: `/v3/projects/82071c45f2f64067a5dc22f4b5564419` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:42.649520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ea0ea-a981-4fc3-a414-d2d8765f7b9e tempest-ProjectsNegativeTestJSON-1245610467 tempest-ProjectsNegativeTestJSON-1245610467-project] RBAC: Authorizing `identity:delete_project(project_id=82071c45f2f64067a5dc22f4b5564419)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-f44ea0ea-a981-4fc3-a414-d2d8765f7b9e tempest-ProjectsNegativeTestJSON-1245610467 tempest-ProjectsNegativeTestJSON-1245610467-project] You are not authorized to perform the requested action: identity:delete_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 207, in delete Oct 15 21:08:42.657630 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Oct 15 21:08:42.658653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Oct 15 21:08:42.658653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application enforcer_obj._enforce( Oct 15 21:08:42.658653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Oct 15 21:08:42.658653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Oct 15 21:08:42.658653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 21:08:42.658653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Oct 15 21:08:42.658653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Oct 15 21:08:42.658653 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:42.658653 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2037/4049] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:42 2022] DELETE /identity/v3/projects/82071c45f2f64067a5dc22f4b5564419 => generated 136 bytes in 238 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:08:42.663321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532aa0-99f0-44d7-9427-f5b825c1d400 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:42.744144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532aa0-99f0-44d7-9427-f5b825c1d400 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:42.750206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073feb0f3ef3445c83b5d75a081451fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a1e08ae0e92498b8f0ce0f45b15a1b3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:42.750844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:42.751126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:42.751399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] PATH_INFO: `/v3/projects/82071c45f2f64067a5dc22f4b5564419` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:42.752598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] RBAC: Authorizing `identity:delete_project(project_id=82071c45f2f64067a5dc22f4b5564419)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:42.755324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:42.822316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82071c45f2f64067a5dc22f4b5564419', 'request_id': 'req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:42.845078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82071c45f2f64067a5dc22f4b5564419', 'request_id': 'req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:42.856081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82071c45f2f64067a5dc22f4b5564419', 'request_id': 'req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:42.866876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:42.875899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] The token cache is being invalidate because project 82071c45f2f64067a5dc22f4b5564419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:42.877882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:42.878772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94d5a236-82bb-4c0e-a706-f7f8e61aa6d1 tempest-ProjectsNegativeTestJSON-1863101894 tempest-ProjectsNegativeTestJSON-1863101894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:42.884658 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2013/4050] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:42 2022] DELETE /identity/v3/projects/82071c45f2f64067a5dc22f4b5564419 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:42.886197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ea0ea-a981-4fc3-a414-d2d8765f7b9e tempest-ProjectsNegativeTestJSON-1245610467 tempest-ProjectsNegativeTestJSON-1245610467-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:42.893246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f44ea0ea-a981-4fc3-a414-d2d8765f7b9e tempest-ProjectsNegativeTestJSON-1245610467 tempest-ProjectsNegativeTestJSON-1245610467-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:42.999647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ea0ea-a981-4fc3-a414-d2d8765f7b9e tempest-ProjectsNegativeTestJSON-1245610467 tempest-ProjectsNegativeTestJSON-1245610467-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:43.073625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:43.074430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:43.074847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:43.075265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] PATH_INFO: `/v3/users/e015d0722993404490ffa222cec39446` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:43.076611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e015d0722993404490ffa222cec39446)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:43.078766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:43.165637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e015d0722993404490ffa222cec39446'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:43.180084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:43.187205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e015d0722993404490ffa222cec39446'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:43.211234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e015d0722993404490ffa222cec39446'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:43.225464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e015d0722993404490ffa222cec39446'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:43.226493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e015d0722993404490ffa222cec39446'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:43.238819 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2038/4051] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:42 2022] DELETE /identity/v3/users/e015d0722993404490ffa222cec39446 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:43.241157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819243-8c11-4254-ad81-701127f1a9b3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:43.242528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819243-8c11-4254-ad81-701127f1a9b3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:43.243006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819243-8c11-4254-ad81-701127f1a9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:43.272961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-48819243-8c11-4254-ad81-701127f1a9b3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:43.274943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-48819243-8c11-4254-ad81-701127f1a9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:43.379369 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2014/4052] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:43.380371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:43.382597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:43.459013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400eab7b-4ada-4d2c-8629-bbb146d61c2f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:43.463006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acb42f-24b1-4771-acb2-305852aa677e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:43.463676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acb42f-24b1-4771-acb2-305852aa677e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:43.464060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acb42f-24b1-4771-acb2-305852aa677e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:43.465665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acb42f-24b1-4771-acb2-305852aa677e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:43.476199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acb42f-24b1-4771-acb2-305852aa677e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:43.477799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acb42f-24b1-4771-acb2-305852aa677e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:43.489344 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2039/4053] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:08:43.887200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48819243-8c11-4254-ad81-701127f1a9b3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:43.889780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-48819243-8c11-4254-ad81-701127f1a9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:43.928037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48819243-8c11-4254-ad81-701127f1a9b3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:43.935961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:43.936678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:43.937018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:43.937426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] PATH_INFO: `/v3/projects/4d78f7c68b5d47f6b7d97f48d4ecc82e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:43.942479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d78f7c68b5d47f6b7d97f48d4ecc82e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:43.945656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:44.027962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d78f7c68b5d47f6b7d97f48d4ecc82e', 'request_id': 'req-ff4e953c-796d-44c1-a3fc-89b61915fe9a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:44.051956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d78f7c68b5d47f6b7d97f48d4ecc82e', 'request_id': 'req-ff4e953c-796d-44c1-a3fc-89b61915fe9a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:44.060594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d78f7c68b5d47f6b7d97f48d4ecc82e', 'request_id': 'req-ff4e953c-796d-44c1-a3fc-89b61915fe9a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:44.089122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] The token cache is being invalidate because project 4d78f7c68b5d47f6b7d97f48d4ecc82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:44.089122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:44.089122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:44.089525 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2015/4054] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:43 2022] DELETE /identity/v3/projects/4d78f7c68b5d47f6b7d97f48d4ecc82e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:44.091748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acb42f-24b1-4771-acb2-305852aa677e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:44.103077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acb42f-24b1-4771-acb2-305852aa677e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:44.252135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:44.252925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:44.253483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:44.253987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] PATH_INFO: `/v3/users/073feb0f3ef3445c83b5d75a081451fb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:44.255276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=073feb0f3ef3445c83b5d75a081451fb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:44.261162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:44.341111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073feb0f3ef3445c83b5d75a081451fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:44.360574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073feb0f3ef3445c83b5d75a081451fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:44.373716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073feb0f3ef3445c83b5d75a081451fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:44.383743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073feb0f3ef3445c83b5d75a081451fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:44.384661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4460e755-5986-4291-b4aa-7234a3f62348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073feb0f3ef3445c83b5d75a081451fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:44.388749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:44.396531 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2040/4055] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:44 2022] DELETE /identity/v3/users/073feb0f3ef3445c83b5d75a081451fb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:44.831936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:44.833866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:44.859719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e953c-796d-44c1-a3fc-89b61915fe9a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:45.002437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:45.003090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:45.003366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:45.003629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] PATH_INFO: `/v3/projects/7a1e08ae0e92498b8f0ce0f45b15a1b3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:45.005717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1e08ae0e92498b8f0ce0f45b15a1b3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:45.011882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:45.048491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:45.096223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1e08ae0e92498b8f0ce0f45b15a1b3', 'request_id': 'req-3c8dd0f7-5521-430d-8e15-4e34158b065c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:45.113246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1e08ae0e92498b8f0ce0f45b15a1b3', 'request_id': 'req-3c8dd0f7-5521-430d-8e15-4e34158b065c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:45.124707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1e08ae0e92498b8f0ce0f45b15a1b3', 'request_id': 'req-3c8dd0f7-5521-430d-8e15-4e34158b065c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:45.151885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] The token cache is being invalidate because project 7a1e08ae0e92498b8f0ce0f45b15a1b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:45.152401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:45.155814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3c8dd0f7-5521-430d-8e15-4e34158b065c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:45.158494 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2016/4056] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:44 2022] DELETE /identity/v3/projects/7a1e08ae0e92498b8f0ce0f45b15a1b3 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:45.164076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d3d75-388c-4a4c-ac29-96d503917a86 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:45.164612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d3d75-388c-4a4c-ac29-96d503917a86 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:45.164696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d3d75-388c-4a4c-ac29-96d503917a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:45.213832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ec7d3d75-388c-4a4c-ac29-96d503917a86 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:45.215263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ec7d3d75-388c-4a4c-ac29-96d503917a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:45.324098 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2041/4057] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:45.335444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efdf46-deb5-456a-b4c9-a290599a2439 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:45.335834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efdf46-deb5-456a-b4c9-a290599a2439 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:45.335957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efdf46-deb5-456a-b4c9-a290599a2439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:45.368906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-16efdf46-deb5-456a-b4c9-a290599a2439 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:45.370572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-16efdf46-deb5-456a-b4c9-a290599a2439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:45.385110 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2017/4058] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:45.388972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d3d75-388c-4a4c-ac29-96d503917a86 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:45.400428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d3d75-388c-4a4c-ac29-96d503917a86 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:45.412646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353a602-2b9f-46dc-a67e-355eb31e6111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:45.413298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353a602-2b9f-46dc-a67e-355eb31e6111 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:45.413567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353a602-2b9f-46dc-a67e-355eb31e6111 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:45.413830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353a602-2b9f-46dc-a67e-355eb31e6111 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:45.414477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353a602-2b9f-46dc-a67e-355eb31e6111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:45.414782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353a602-2b9f-46dc-a67e-355eb31e6111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:45.417961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353a602-2b9f-46dc-a67e-355eb31e6111 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:45.437920 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2042/4059] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:45.441987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efdf46-deb5-456a-b4c9-a290599a2439 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:45.455706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efdf46-deb5-456a-b4c9-a290599a2439 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:45.464460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7780-8ce8-43ac-881c-afc4eedc4e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:45.465600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7780-8ce8-43ac-881c-afc4eedc4e70 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:45.465897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7780-8ce8-43ac-881c-afc4eedc4e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:45.466167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7780-8ce8-43ac-881c-afc4eedc4e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:45.466799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f7780-8ce8-43ac-881c-afc4eedc4e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:45.471715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f7780-8ce8-43ac-881c-afc4eedc4e70 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:45.552966 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2018/4060] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:45 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:45.557229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353a602-2b9f-46dc-a67e-355eb31e6111 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:45.598451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353a602-2b9f-46dc-a67e-355eb31e6111 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:45.717641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d6a63-2040-4d3f-8aa2-61472e507e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:45.718524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d6a63-2040-4d3f-8aa2-61472e507e4d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:45.718804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d6a63-2040-4d3f-8aa2-61472e507e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:45.719442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d6a63-2040-4d3f-8aa2-61472e507e4d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:45.720445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d6a63-2040-4d3f-8aa2-61472e507e4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:45.722781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d6a63-2040-4d3f-8aa2-61472e507e4d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:45.754311 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2043/4061] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:45 2022] POST /identity/v3/users => generated 284 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:45.755777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7780-8ce8-43ac-881c-afc4eedc4e70 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:45.767835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7780-8ce8-43ac-881c-afc4eedc4e70 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:45.774860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9917c7-fb8e-4334-ad7a-f8863b3dbb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:45.775573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9917c7-fb8e-4334-ad7a-f8863b3dbb29 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:45.776079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9917c7-fb8e-4334-ad7a-f8863b3dbb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:45.776536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9917c7-fb8e-4334-ad7a-f8863b3dbb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:45.777325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9917c7-fb8e-4334-ad7a-f8863b3dbb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:45.789661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9917c7-fb8e-4334-ad7a-f8863b3dbb29 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:45.804595 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2019/4062] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:45.808255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d6a63-2040-4d3f-8aa2-61472e507e4d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:45.825811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d6a63-2040-4d3f-8aa2-61472e507e4d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:45.833639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b32c38-50af-4105-81cb-b0d7d4b43da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:45.834288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32c38-50af-4105-81cb-b0d7d4b43da5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:45.834745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32c38-50af-4105-81cb-b0d7d4b43da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:45.835070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32c38-50af-4105-81cb-b0d7d4b43da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:45.836095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b32c38-50af-4105-81cb-b0d7d4b43da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:45.838195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b32c38-50af-4105-81cb-b0d7d4b43da5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:45.857774 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2044/4063] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:45.861834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9917c7-fb8e-4334-ad7a-f8863b3dbb29 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:45.874804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9917c7-fb8e-4334-ad7a-f8863b3dbb29 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:45.879854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6375ee-9ce2-4aea-8369-5e892b742cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:45.880592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6375ee-9ce2-4aea-8369-5e892b742cba admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:45.881659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6375ee-9ce2-4aea-8369-5e892b742cba admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:45.882010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6375ee-9ce2-4aea-8369-5e892b742cba admin admin] PATH_INFO: `/v3/projects/c6fb088529854b909d10965a99ead2e5/users/99e6e49193d1409fa1b552eb736ef8f9/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:45.907054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6375ee-9ce2-4aea-8369-5e892b742cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6fb088529854b909d10965a99ead2e5, user_id=99e6e49193d1409fa1b552eb736ef8f9, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:45.910237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6375ee-9ce2-4aea-8369-5e892b742cba admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:45.930337 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2020/4064] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:45 2022] PUT /identity/v3/projects/c6fb088529854b909d10965a99ead2e5/users/99e6e49193d1409fa1b552eb736ef8f9/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:46.037935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fcb75-5fb6-46c2-bef0-ef32a1610201 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:46.037935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fcb75-5fb6-46c2-bef0-ef32a1610201 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:46.038135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fcb75-5fb6-46c2-bef0-ef32a1610201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:46.089356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ed6fcb75-5fb6-46c2-bef0-ef32a1610201 None None] MFA Rules not processed for user `99e6e49193d1409fa1b552eb736ef8f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:46.090813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ed6fcb75-5fb6-46c2-bef0-ef32a1610201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:46.165715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:46.182696 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2045/4065] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:46.186764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6375ee-9ce2-4aea-8369-5e892b742cba admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:46.332899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6375ee-9ce2-4aea-8369-5e892b742cba admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:46.343652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f582e-b7db-475b-a9ad-3e3c4613b90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:46.344392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f582e-b7db-475b-a9ad-3e3c4613b90e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:46.344795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f582e-b7db-475b-a9ad-3e3c4613b90e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:46.344991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f582e-b7db-475b-a9ad-3e3c4613b90e admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:46.345807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f582e-b7db-475b-a9ad-3e3c4613b90e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:46.348065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f582e-b7db-475b-a9ad-3e3c4613b90e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:46.420482 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2021/4066] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:46 2022] POST /identity/v3/projects => generated 367 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:46.421680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fcb75-5fb6-46c2-bef0-ef32a1610201 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:46.438853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fcb75-5fb6-46c2-bef0-ef32a1610201 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:46.568517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f4a0d-94cd-46da-a5e1-1af790f93d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:46.569195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f4a0d-94cd-46da-a5e1-1af790f93d86 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:46.569601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f4a0d-94cd-46da-a5e1-1af790f93d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:46.569876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f4a0d-94cd-46da-a5e1-1af790f93d86 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:46.570561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f4a0d-94cd-46da-a5e1-1af790f93d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:46.573354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f4a0d-94cd-46da-a5e1-1af790f93d86 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:46.617174 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2046/4067] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:46 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:46.618561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f582e-b7db-475b-a9ad-3e3c4613b90e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:46.632928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f582e-b7db-475b-a9ad-3e3c4613b90e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:46.641974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ff45b-4d8f-49a9-9abe-a5a8cb29b6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:46.643088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ff45b-4d8f-49a9-9abe-a5a8cb29b6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:46.643585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ff45b-4d8f-49a9-9abe-a5a8cb29b6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:46.644127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ff45b-4d8f-49a9-9abe-a5a8cb29b6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:46.644997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ff45b-4d8f-49a9-9abe-a5a8cb29b6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:46.649631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ff45b-4d8f-49a9-9abe-a5a8cb29b6e8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:46.669603 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2022/4068] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:08:46.673434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f4a0d-94cd-46da-a5e1-1af790f93d86 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:46.687843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f4a0d-94cd-46da-a5e1-1af790f93d86 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:46.698662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d3162-7395-4607-b81a-43065da3a7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:46.699792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d3162-7395-4607-b81a-43065da3a7de admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:46.701200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d3162-7395-4607-b81a-43065da3a7de admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:46.701600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d3162-7395-4607-b81a-43065da3a7de admin admin] PATH_INFO: `/v3/projects/54dcb34136f34cd59125b833205a50b5/users/74e48edf46d54884ad3bd49fec65e1c6/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:46.740663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d3162-7395-4607-b81a-43065da3a7de admin admin] RBAC: Authorizing `identity:create_grant(project_id=54dcb34136f34cd59125b833205a50b5, user_id=74e48edf46d54884ad3bd49fec65e1c6, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:46.749529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d3162-7395-4607-b81a-43065da3a7de admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:46.780212 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2047/4069] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:46 2022] PUT /identity/v3/projects/54dcb34136f34cd59125b833205a50b5/users/74e48edf46d54884ad3bd49fec65e1c6/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:46.784412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ff45b-4d8f-49a9-9abe-a5a8cb29b6e8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:46.797362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ff45b-4d8f-49a9-9abe-a5a8cb29b6e8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:46.822876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:46.881260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896f479-1242-4a82-8f7c-04abe7ae8888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:46.887328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896f479-1242-4a82-8f7c-04abe7ae8888 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:46.888459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896f479-1242-4a82-8f7c-04abe7ae8888 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:46.889198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896f479-1242-4a82-8f7c-04abe7ae8888 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:46.890113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896f479-1242-4a82-8f7c-04abe7ae8888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:46.893074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896f479-1242-4a82-8f7c-04abe7ae8888 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:46.906160 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2023/4070] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:46.910171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d3162-7395-4607-b81a-43065da3a7de admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:46.925907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d3162-7395-4607-b81a-43065da3a7de admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:46.933613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3ce78-cd54-4925-8bc6-ad1158956b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:46.934622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3ce78-cd54-4925-8bc6-ad1158956b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:46.935365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3ce78-cd54-4925-8bc6-ad1158956b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:46.935945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3ce78-cd54-4925-8bc6-ad1158956b10 admin admin] PATH_INFO: `/v3/projects/54dcb34136f34cd59125b833205a50b5/users/74e48edf46d54884ad3bd49fec65e1c6/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:46.939484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3ce78-cd54-4925-8bc6-ad1158956b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54dcb34136f34cd59125b833205a50b5, user_id=74e48edf46d54884ad3bd49fec65e1c6, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:46.942765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3ce78-cd54-4925-8bc6-ad1158956b10 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:46.962615 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2048/4071] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:46 2022] PUT /identity/v3/projects/54dcb34136f34cd59125b833205a50b5/users/74e48edf46d54884ad3bd49fec65e1c6/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:46.975486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f6af7-f8dc-4b78-8d41-ad294c2ce324 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:46.976066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f6af7-f8dc-4b78-8d41-ad294c2ce324 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:46.976354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f6af7-f8dc-4b78-8d41-ad294c2ce324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:47.018850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a85f6af7-f8dc-4b78-8d41-ad294c2ce324 None None] MFA Rules not processed for user `74e48edf46d54884ad3bd49fec65e1c6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:47.020261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a85f6af7-f8dc-4b78-8d41-ad294c2ce324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:47.117365 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2024/4072] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:47.121766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3ce78-cd54-4925-8bc6-ad1158956b10 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:47.226925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3ce78-cd54-4925-8bc6-ad1158956b10 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:47.233609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c5609-d36b-4895-9da9-ba97d7a7a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:47.235103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c5609-d36b-4895-9da9-ba97d7a7a8aa admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:47.235666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c5609-d36b-4895-9da9-ba97d7a7a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:47.236375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c5609-d36b-4895-9da9-ba97d7a7a8aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:47.238226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c5609-d36b-4895-9da9-ba97d7a7a8aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:47.241348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c5609-d36b-4895-9da9-ba97d7a7a8aa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:47.302913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f6af7-f8dc-4b78-8d41-ad294c2ce324 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:47.303792 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2049/4073] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:47 2022] POST /identity/v3/projects => generated 367 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:47.319858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f6af7-f8dc-4b78-8d41-ad294c2ce324 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:47.412271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47202bb0-f736-42ad-8577-09f4c9b81352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:47.413188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47202bb0-f736-42ad-8577-09f4c9b81352 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:47.413470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47202bb0-f736-42ad-8577-09f4c9b81352 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:47.413828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47202bb0-f736-42ad-8577-09f4c9b81352 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:47.414534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47202bb0-f736-42ad-8577-09f4c9b81352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:47.416624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47202bb0-f736-42ad-8577-09f4c9b81352 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:47.448762 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2025/4074] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:47 2022] POST /identity/v3/users => generated 284 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:47.450735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c5609-d36b-4895-9da9-ba97d7a7a8aa admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:47.464653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c5609-d36b-4895-9da9-ba97d7a7a8aa admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:47.489090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79879d6e-0492-4d99-96af-4b69e2d3d688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:47.490062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79879d6e-0492-4d99-96af-4b69e2d3d688 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:47.490448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79879d6e-0492-4d99-96af-4b69e2d3d688 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:47.490787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79879d6e-0492-4d99-96af-4b69e2d3d688 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:47.492663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79879d6e-0492-4d99-96af-4b69e2d3d688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:47.495552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79879d6e-0492-4d99-96af-4b69e2d3d688 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:47.517105 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2050/4075] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:47.519190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47202bb0-f736-42ad-8577-09f4c9b81352 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:47.533774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47202bb0-f736-42ad-8577-09f4c9b81352 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:47.542990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca3bf4-7903-4c4c-95d1-acaa09433878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:47.544327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3bf4-7903-4c4c-95d1-acaa09433878 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:47.545460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3bf4-7903-4c4c-95d1-acaa09433878 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:47.545814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3bf4-7903-4c4c-95d1-acaa09433878 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:47.550507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca3bf4-7903-4c4c-95d1-acaa09433878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:47.552275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca3bf4-7903-4c4c-95d1-acaa09433878 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:47.570256 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2026/4076] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:47.575163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79879d6e-0492-4d99-96af-4b69e2d3d688 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:47.589245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79879d6e-0492-4d99-96af-4b69e2d3d688 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:47.595036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8c65b-9848-4fc8-be40-feca0f3b8047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:47.595760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c65b-9848-4fc8-be40-feca0f3b8047 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:47.596142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c65b-9848-4fc8-be40-feca0f3b8047 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:47.596403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c65b-9848-4fc8-be40-feca0f3b8047 admin admin] PATH_INFO: `/v3/projects/642622a2b3404e1095d1c89ba0da1f36/users/f4004fb706684648a08e7a131d5b16a0/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:47.619136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8c65b-9848-4fc8-be40-feca0f3b8047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642622a2b3404e1095d1c89ba0da1f36, user_id=f4004fb706684648a08e7a131d5b16a0, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:47.621928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8c65b-9848-4fc8-be40-feca0f3b8047 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:47.626696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:47.649170 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2051/4077] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:47 2022] PUT /identity/v3/projects/642622a2b3404e1095d1c89ba0da1f36/users/f4004fb706684648a08e7a131d5b16a0/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:47.653300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b543c-0863-4c67-8a81-c901333f5107 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:47.653625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b543c-0863-4c67-8a81-c901333f5107 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:47.653923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b543c-0863-4c67-8a81-c901333f5107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:47.692534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8e8b543c-0863-4c67-8a81-c901333f5107 None None] MFA Rules not processed for user `f4004fb706684648a08e7a131d5b16a0`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:47.693797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8e8b543c-0863-4c67-8a81-c901333f5107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:47.706360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:47.792897 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2027/4078] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:47.794208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8c65b-9848-4fc8-be40-feca0f3b8047 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:47.924829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8c65b-9848-4fc8-be40-feca0f3b8047 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:47.932432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e4bc0-beaa-41c2-a071-c6da6038324a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:47.933219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e4bc0-beaa-41c2-a071-c6da6038324a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:47.933566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e4bc0-beaa-41c2-a071-c6da6038324a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:47.933936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e4bc0-beaa-41c2-a071-c6da6038324a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:47.935369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e4bc0-beaa-41c2-a071-c6da6038324a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:47.938233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e4bc0-beaa-41c2-a071-c6da6038324a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:47.965080 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2052/4079] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:47 2022] POST /identity/v3/domains => generated 277 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:47.967770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b543c-0863-4c67-8a81-c901333f5107 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:47.982265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b543c-0863-4c67-8a81-c901333f5107 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:48.095764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21d3ce-d967-45f2-a780-53bc55dedf99 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:48.096535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21d3ce-d967-45f2-a780-53bc55dedf99 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:48.096999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21d3ce-d967-45f2-a780-53bc55dedf99 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:48.097393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21d3ce-d967-45f2-a780-53bc55dedf99 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:48.098144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21d3ce-d967-45f2-a780-53bc55dedf99 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:48.099541 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-bf21d3ce-d967-45f2-a780-53bc55dedf99 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:48.099541 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:48.099541 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:48.100751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21d3ce-d967-45f2-a780-53bc55dedf99 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:48.124888 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2028/4080] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:08:47 2022] POST /identity/v3/roles => generated 239 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:48.126116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e4bc0-beaa-41c2-a071-c6da6038324a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:48.139039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e4bc0-beaa-41c2-a071-c6da6038324a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:48.144199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19d326-cf22-4a1f-9c83-4f69434af53e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:48.144964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19d326-cf22-4a1f-9c83-4f69434af53e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:48.145371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19d326-cf22-4a1f-9c83-4f69434af53e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:48.145738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19d326-cf22-4a1f-9c83-4f69434af53e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/ff50684481b948a2a34522a1182f4652/users/f4004fb706684648a08e7a131d5b16a0/roles/df26696f83d645268f114d64d7f8ea66` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:48.170686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19d326-cf22-4a1f-9c83-4f69434af53e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=ff50684481b948a2a34522a1182f4652, user_id=f4004fb706684648a08e7a131d5b16a0, role_id=df26696f83d645268f114d64d7f8ea66)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:48.173656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19d326-cf22-4a1f-9c83-4f69434af53e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:48.202321 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2053/4081] 10.222.0.16 () {60 vars in 1480 bytes} [Sat Oct 15 21:08:48 2022] PUT /identity/v3/domains/ff50684481b948a2a34522a1182f4652/users/f4004fb706684648a08e7a131d5b16a0/roles/df26696f83d645268f114d64d7f8ea66 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:48.205939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21d3ce-d967-45f2-a780-53bc55dedf99 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:48.219757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21d3ce-d967-45f2-a780-53bc55dedf99 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:48.310467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d971c-3f4a-4c69-a329-895c02061bb3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:48.311054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d971c-3f4a-4c69-a329-895c02061bb3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:48.311336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d971c-3f4a-4c69-a329-895c02061bb3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:48.311602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d971c-3f4a-4c69-a329-895c02061bb3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:48.312468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d971c-3f4a-4c69-a329-895c02061bb3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:48.314201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d971c-3f4a-4c69-a329-895c02061bb3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:48.381328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19d326-cf22-4a1f-9c83-4f69434af53e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:48.408997 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2029/4082] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:48 2022] POST /identity/v3/domains => generated 276 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:48.409140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19d326-cf22-4a1f-9c83-4f69434af53e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:48.521533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891248fd-fab0-4472-be68-ba57a2aa7ad0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:48.522578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891248fd-fab0-4472-be68-ba57a2aa7ad0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:48.523071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891248fd-fab0-4472-be68-ba57a2aa7ad0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:48.523565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891248fd-fab0-4472-be68-ba57a2aa7ad0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:48.524452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891248fd-fab0-4472-be68-ba57a2aa7ad0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:48.526144 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-891248fd-fab0-4472-be68-ba57a2aa7ad0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:48.526144 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:48.526144 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:48.528895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891248fd-fab0-4472-be68-ba57a2aa7ad0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:48.556419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d971c-3f4a-4c69-a329-895c02061bb3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:48.558689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4a2d971c-3f4a-4c69-a329-895c02061bb3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:48.573945 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2054/4083] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:08:48 2022] POST /identity/v3/roles => generated 240 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:48.581450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891248fd-fab0-4472-be68-ba57a2aa7ad0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:48.593431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891248fd-fab0-4472-be68-ba57a2aa7ad0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:48.598470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046eff4-3d58-41cc-8fb6-92ce1251dc54 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:48.599174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046eff4-3d58-41cc-8fb6-92ce1251dc54 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:48.599512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046eff4-3d58-41cc-8fb6-92ce1251dc54 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:48.599847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046eff4-3d58-41cc-8fb6-92ce1251dc54 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:48.600537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046eff4-3d58-41cc-8fb6-92ce1251dc54 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:48.601799 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-8046eff4-3d58-41cc-8fb6-92ce1251dc54 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:48.601799 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:48.601799 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:48.603173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046eff4-3d58-41cc-8fb6-92ce1251dc54 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:48.624383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:48.646188 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2055/4084] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:08:48 2022] POST /identity/v3/groups => generated 327 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:48.650651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046eff4-3d58-41cc-8fb6-92ce1251dc54 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:48.661774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d971c-3f4a-4c69-a329-895c02061bb3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:48.663944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046eff4-3d58-41cc-8fb6-92ce1251dc54 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:48.669913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:48.670668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:48.670802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270497a-4fff-46db-b437-a76b4c6e5b89 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:48.671077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:48.671537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270497a-4fff-46db-b437-a76b4c6e5b89 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:48.671818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270497a-4fff-46db-b437-a76b4c6e5b89 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:48.672073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] PATH_INFO: `/v3/users/a7cd1ae2e4ea4aa48b33c7e9eda52b00` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:48.672196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270497a-4fff-46db-b437-a76b4c6e5b89 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/groups/a836b2ccdd6343c19d82dbb135117f0e/users/f4004fb706684648a08e7a131d5b16a0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:48.673669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7cd1ae2e4ea4aa48b33c7e9eda52b00)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:48.676445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:48.680012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270497a-4fff-46db-b437-a76b4c6e5b89 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=a836b2ccdd6343c19d82dbb135117f0e, user_id=f4004fb706684648a08e7a131d5b16a0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:48.681394 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-0270497a-4fff-46db-b437-a76b4c6e5b89 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:48.681394 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:48.681394 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:48.682909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270497a-4fff-46db-b437-a76b4c6e5b89 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:48.732332 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2056/4085] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Oct 15 21:08:48 2022] PUT /identity/v3/groups/a836b2ccdd6343c19d82dbb135117f0e/users/f4004fb706684648a08e7a131d5b16a0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:48.739405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270497a-4fff-46db-b437-a76b4c6e5b89 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:48.769439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:48.776512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7cd1ae2e4ea4aa48b33c7e9eda52b00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:48.813119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7cd1ae2e4ea4aa48b33c7e9eda52b00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:48.848693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7cd1ae2e4ea4aa48b33c7e9eda52b00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:48.859679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7cd1ae2e4ea4aa48b33c7e9eda52b00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:48.860628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59e0ebed-2b25-4939-9d04-d3caeac205c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7cd1ae2e4ea4aa48b33c7e9eda52b00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:48.874186 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2030/4086] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:48 2022] DELETE /identity/v3/users/a7cd1ae2e4ea4aa48b33c7e9eda52b00 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:48.881450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe7b32-e2dc-488a-94d8-5a8f203c7f7c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:48.881771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe7b32-e2dc-488a-94d8-5a8f203c7f7c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:48.882049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe7b32-e2dc-488a-94d8-5a8f203c7f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:48.910805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270497a-4fff-46db-b437-a76b4c6e5b89 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:48.932219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-24fe7b32-e2dc-488a-94d8-5a8f203c7f7c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:48.933718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-24fe7b32-e2dc-488a-94d8-5a8f203c7f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:49.011913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070359b-5fee-4326-bfbc-7b94ea239596 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:49.014325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070359b-5fee-4326-bfbc-7b94ea239596 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:49.014618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070359b-5fee-4326-bfbc-7b94ea239596 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:49.014897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070359b-5fee-4326-bfbc-7b94ea239596 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/d5ed0d337b2546e6a03f4c220ad1124d/groups/a836b2ccdd6343c19d82dbb135117f0e/roles/5b3e87e24fb544899b6c89a7de7b7175` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:49.019028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070359b-5fee-4326-bfbc-7b94ea239596 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=d5ed0d337b2546e6a03f4c220ad1124d, group_id=a836b2ccdd6343c19d82dbb135117f0e, role_id=5b3e87e24fb544899b6c89a7de7b7175)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:49.033089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070359b-5fee-4326-bfbc-7b94ea239596 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:49.053436 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2031/4087] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:49.061707 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2057/4088] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:48 2022] PUT /identity/v3/domains/d5ed0d337b2546e6a03f4c220ad1124d/groups/a836b2ccdd6343c19d82dbb135117f0e/roles/5b3e87e24fb544899b6c89a7de7b7175 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:49.063018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe7b32-e2dc-488a-94d8-5a8f203c7f7c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:49.064733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-24fe7b32-e2dc-488a-94d8-5a8f203c7f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:49.079371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070359b-5fee-4326-bfbc-7b94ea239596 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:49.082029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7070359b-5fee-4326-bfbc-7b94ea239596 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:49.091612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe7b32-e2dc-488a-94d8-5a8f203c7f7c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:49.225239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070359b-5fee-4326-bfbc-7b94ea239596 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:49.228796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac139a1-556b-4b88-a562-ccb334d7c62a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:49.230013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac139a1-556b-4b88-a562-ccb334d7c62a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:49.230523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac139a1-556b-4b88-a562-ccb334d7c62a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:49.230981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac139a1-556b-4b88-a562-ccb334d7c62a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:49.231902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac139a1-556b-4b88-a562-ccb334d7c62a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:49.232921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2011f1-71c9-4d09-9bc0-903c186417ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:49.233670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2011f1-71c9-4d09-9bc0-903c186417ef service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:49.234040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2011f1-71c9-4d09-9bc0-903c186417ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:49.234268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac139a1-556b-4b88-a562-ccb334d7c62a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:49.234498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2011f1-71c9-4d09-9bc0-903c186417ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:49.238904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:49.267267 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2032/4089] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:49 2022] POST /identity/v3/domains => generated 275 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:49.271072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac139a1-556b-4b88-a562-ccb334d7c62a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:49.357885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac139a1-556b-4b88-a562-ccb334d7c62a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:49.364312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4c438-c4a3-469d-88a9-38eec520cf0c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:49.365220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4c438-c4a3-469d-88a9-38eec520cf0c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:49.365846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4c438-c4a3-469d-88a9-38eec520cf0c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:49.366605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4c438-c4a3-469d-88a9-38eec520cf0c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:49.367379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4c438-c4a3-469d-88a9-38eec520cf0c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:49.369926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4c438-c4a3-469d-88a9-38eec520cf0c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:49.400474 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2033/4090] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:08:49 2022] POST /identity/v3/roles => generated 239 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:49.402396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2011f1-71c9-4d09-9bc0-903c186417ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:49.404481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2011f1-71c9-4d09-9bc0-903c186417ef service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:49.405489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4c438-c4a3-469d-88a9-38eec520cf0c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:49.421942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4c438-c4a3-469d-88a9-38eec520cf0c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:49.428753 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2058/4091] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:08:49.430850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91b159-5d1c-4a39-a4bb-d9d4286d0a26 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:49.431662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91b159-5d1c-4a39-a4bb-d9d4286d0a26 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:49.431945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91b159-5d1c-4a39-a4bb-d9d4286d0a26 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:49.432224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91b159-5d1c-4a39-a4bb-d9d4286d0a26 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/d2518f06a2f54b25b22683720a4220ff/users/f4004fb706684648a08e7a131d5b16a0/roles/fc489bfa855646d6aaa2d88274be50c8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:49.449899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91b159-5d1c-4a39-a4bb-d9d4286d0a26 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=d2518f06a2f54b25b22683720a4220ff, user_id=f4004fb706684648a08e7a131d5b16a0, role_id=fc489bfa855646d6aaa2d88274be50c8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:49.452646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91b159-5d1c-4a39-a4bb-d9d4286d0a26 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:49.477760 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2034/4092] 10.222.0.16 () {60 vars in 1480 bytes} [Sat Oct 15 21:08:49 2022] PUT /identity/v3/domains/d2518f06a2f54b25b22683720a4220ff/users/f4004fb706684648a08e7a131d5b16a0/roles/fc489bfa855646d6aaa2d88274be50c8 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:49.482211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2011f1-71c9-4d09-9bc0-903c186417ef service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:49.489891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:49.495371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2011f1-71c9-4d09-9bc0-903c186417ef service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:49.582328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7934f75-8efa-49e6-80e1-328a6b03b58f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:49.582943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7934f75-8efa-49e6-80e1-328a6b03b58f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:49.583237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7934f75-8efa-49e6-80e1-328a6b03b58f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:49.583486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7934f75-8efa-49e6-80e1-328a6b03b58f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:49.584557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7934f75-8efa-49e6-80e1-328a6b03b58f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:49.586421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7934f75-8efa-49e6-80e1-328a6b03b58f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:49.640658 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2059/4093] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:08:49 2022] POST /identity/v3/domains => generated 276 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:49.642177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91b159-5d1c-4a39-a4bb-d9d4286d0a26 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:49.658481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91b159-5d1c-4a39-a4bb-d9d4286d0a26 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:49.770624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34795198-3ed6-4ae0-af9d-e29ca95982c7 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:49.771249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795198-3ed6-4ae0-af9d-e29ca95982c7 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:49.771913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795198-3ed6-4ae0-af9d-e29ca95982c7 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:49.771913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795198-3ed6-4ae0-af9d-e29ca95982c7 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:49.772765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34795198-3ed6-4ae0-af9d-e29ca95982c7 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:49.775165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34795198-3ed6-4ae0-af9d-e29ca95982c7 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:49.807628 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2035/4094] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:08:49 2022] POST /identity/v3/roles => generated 239 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:49.808003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7934f75-8efa-49e6-80e1-328a6b03b58f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:49.827335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34795198-3ed6-4ae0-af9d-e29ca95982c7 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:49.829116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-34795198-3ed6-4ae0-af9d-e29ca95982c7 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:49.844165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7934f75-8efa-49e6-80e1-328a6b03b58f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:49.863760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59a8c4-f21b-4aff-b8c9-2f8621b029cd tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:49.864387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a8c4-f21b-4aff-b8c9-2f8621b029cd tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:49.864917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a8c4-f21b-4aff-b8c9-2f8621b029cd tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:49.865223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a8c4-f21b-4aff-b8c9-2f8621b029cd tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:49.865889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59a8c4-f21b-4aff-b8c9-2f8621b029cd tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:49.871383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59a8c4-f21b-4aff-b8c9-2f8621b029cd tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:49.928281 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2060/4095] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:08:49 2022] POST /identity/v3/groups => generated 325 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:49.934720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59a8c4-f21b-4aff-b8c9-2f8621b029cd tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:49.952882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59a8c4-f21b-4aff-b8c9-2f8621b029cd tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:49.965591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665980c1-4961-4969-bfc2-e9961d9e73f0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:49.968609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665980c1-4961-4969-bfc2-e9961d9e73f0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:49.970028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665980c1-4961-4969-bfc2-e9961d9e73f0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:49.970169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665980c1-4961-4969-bfc2-e9961d9e73f0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/groups/1c728d9e6a52431aabab11bdc502f7fb/users/f4004fb706684648a08e7a131d5b16a0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:49.979055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:49.981935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665980c1-4961-4969-bfc2-e9961d9e73f0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=1c728d9e6a52431aabab11bdc502f7fb, user_id=f4004fb706684648a08e7a131d5b16a0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:49.984711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665980c1-4961-4969-bfc2-e9961d9e73f0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:50.023962 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2061/4096] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Oct 15 21:08:49 2022] PUT /identity/v3/groups/1c728d9e6a52431aabab11bdc502f7fb/users/f4004fb706684648a08e7a131d5b16a0 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:50.027865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665980c1-4961-4969-bfc2-e9961d9e73f0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:50.032484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34795198-3ed6-4ae0-af9d-e29ca95982c7 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:50.127520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665980c1-4961-4969-bfc2-e9961d9e73f0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:50.139097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf08167-199c-4bd9-ae1d-6897650972e0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:50.140286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf08167-199c-4bd9-ae1d-6897650972e0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:50.140832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf08167-199c-4bd9-ae1d-6897650972e0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:50.141489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf08167-199c-4bd9-ae1d-6897650972e0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/c83c4b95067b4383823c9745fc40e79f/groups/1c728d9e6a52431aabab11bdc502f7fb/roles/ada4c36d93fa44c9aa79296199e43317` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:50.143973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf08167-199c-4bd9-ae1d-6897650972e0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=c83c4b95067b4383823c9745fc40e79f, group_id=1c728d9e6a52431aabab11bdc502f7fb, role_id=ada4c36d93fa44c9aa79296199e43317)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:50.149523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf08167-199c-4bd9-ae1d-6897650972e0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:50.158272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af24c7-0280-48c0-9f7b-36464917a58c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:50.158975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af24c7-0280-48c0-9f7b-36464917a58c service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:50.159369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af24c7-0280-48c0-9f7b-36464917a58c service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:50.159731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af24c7-0280-48c0-9f7b-36464917a58c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:50.162306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c3af24c7-0280-48c0-9f7b-36464917a58c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:50.172520 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2062/4097] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:50 2022] PUT /identity/v3/domains/c83c4b95067b4383823c9745fc40e79f/groups/1c728d9e6a52431aabab11bdc502f7fb/roles/ada4c36d93fa44c9aa79296199e43317 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:50.173680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf08167-199c-4bd9-ae1d-6897650972e0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:50.176095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fcf08167-199c-4bd9-ae1d-6897650972e0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:50.179728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af24c7-0280-48c0-9f7b-36464917a58c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:50.181570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af24c7-0280-48c0-9f7b-36464917a58c service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:50.209423 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2036/4098] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:08:50.210312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af24c7-0280-48c0-9f7b-36464917a58c service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:50.212519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c3af24c7-0280-48c0-9f7b-36464917a58c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:50.263356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:50.289619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf08167-199c-4bd9-ae1d-6897650972e0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:50.305818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:50.306637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:50.307016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:50.307875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] PATH_INFO: `/v3/projects/9ff9752441b2458d962bc20c9dc70f63` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:50.310585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff9752441b2458d962bc20c9dc70f63)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:50.313578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:50.425642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff9752441b2458d962bc20c9dc70f63', 'request_id': 'req-a0b9c505-99be-44d3-877f-7957b9d0007a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:50.433315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af24c7-0280-48c0-9f7b-36464917a58c service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:50.442719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff9752441b2458d962bc20c9dc70f63', 'request_id': 'req-a0b9c505-99be-44d3-877f-7957b9d0007a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:50.456685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff9752441b2458d962bc20c9dc70f63', 'request_id': 'req-a0b9c505-99be-44d3-877f-7957b9d0007a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:50.468279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] The token cache is being invalidate because project 9ff9752441b2458d962bc20c9dc70f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:50.468718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:50.469510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0b9c505-99be-44d3-877f-7957b9d0007a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:50.477082 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2063/4099] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:50 2022] DELETE /identity/v3/projects/9ff9752441b2458d962bc20c9dc70f63 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:50.478623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d224da-d241-49fd-8fca-535a811aac48 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:50.479011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d224da-d241-49fd-8fca-535a811aac48 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:50.479393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d224da-d241-49fd-8fca-535a811aac48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:50.503515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-44d224da-d241-49fd-8fca-535a811aac48 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:50.504893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-44d224da-d241-49fd-8fca-535a811aac48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:50.533048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:50.600426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa1122-b7b1-4ee2-90da-2d997feaa98f tempest-TokensV3TestJSON-696550206 tempest-TokensV3TestJSON-696550206-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4004fb706684648a08e7a131d5b16a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '642622a2b3404e1095d1c89ba0da1f36', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:50.601139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa1122-b7b1-4ee2-90da-2d997feaa98f tempest-TokensV3TestJSON-696550206 tempest-TokensV3TestJSON-696550206-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:50.608823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa1122-b7b1-4ee2-90da-2d997feaa98f tempest-TokensV3TestJSON-696550206 tempest-TokensV3TestJSON-696550206-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:50.608998 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2064/4100] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:50.609127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa1122-b7b1-4ee2-90da-2d997feaa98f tempest-TokensV3TestJSON-696550206 tempest-TokensV3TestJSON-696550206-project] PATH_INFO: `/v3/auth/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:50.609127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa1122-b7b1-4ee2-90da-2d997feaa98f tempest-TokensV3TestJSON-696550206 tempest-TokensV3TestJSON-696550206-project] RBAC: Authorizing `identity:get_auth_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:50.609127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa1122-b7b1-4ee2-90da-2d997feaa98f tempest-TokensV3TestJSON-696550206 tempest-TokensV3TestJSON-696550206-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:50.610219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efd84d-6bac-4e56-a51b-bbe8864b5d4a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:50.610498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efd84d-6bac-4e56-a51b-bbe8864b5d4a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:50.610767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efd84d-6bac-4e56-a51b-bbe8864b5d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:50.657450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a6efd84d-6bac-4e56-a51b-bbe8864b5d4a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:50.658670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a6efd84d-6bac-4e56-a51b-bbe8864b5d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:50.684181 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2065/4101] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:50.685853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efd84d-6bac-4e56-a51b-bbe8864b5d4a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:50.698944 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2037/4102] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 21:08:50 2022] GET /identity/v3/auth/domains => generated 1172 bytes in 489 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:50.705385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa1122-b7b1-4ee2-90da-2d997feaa98f tempest-TokensV3TestJSON-696550206 tempest-TokensV3TestJSON-696550206-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:50.705640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efd84d-6bac-4e56-a51b-bbe8864b5d4a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:50.707562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-09fa1122-b7b1-4ee2-90da-2d997feaa98f tempest-TokensV3TestJSON-696550206 tempest-TokensV3TestJSON-696550206-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:50.718606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405caea6-1457-47b6-8b9d-0f6e2f8bdae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:50.719213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405caea6-1457-47b6-8b9d-0f6e2f8bdae3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:50.719484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405caea6-1457-47b6-8b9d-0f6e2f8bdae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:50.719811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405caea6-1457-47b6-8b9d-0f6e2f8bdae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:50.720471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405caea6-1457-47b6-8b9d-0f6e2f8bdae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:50.720753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405caea6-1457-47b6-8b9d-0f6e2f8bdae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:50.724561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405caea6-1457-47b6-8b9d-0f6e2f8bdae3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:50.745489 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2066/4103] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:08:50.750742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405caea6-1457-47b6-8b9d-0f6e2f8bdae3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:50.767743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405caea6-1457-47b6-8b9d-0f6e2f8bdae3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:50.781162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd16f7-260a-4382-a213-c6dcfa1a40be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:50.782942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd16f7-260a-4382-a213-c6dcfa1a40be admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:50.785529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd16f7-260a-4382-a213-c6dcfa1a40be admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:50.785815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd16f7-260a-4382-a213-c6dcfa1a40be admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:50.786871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd16f7-260a-4382-a213-c6dcfa1a40be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:50.790421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd16f7-260a-4382-a213-c6dcfa1a40be admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:50.862974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa1122-b7b1-4ee2-90da-2d997feaa98f tempest-TokensV3TestJSON-696550206 tempest-TokensV3TestJSON-696550206-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:50.867203 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2067/4104] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:50 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:50.872152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd16f7-260a-4382-a213-c6dcfa1a40be admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:50.876325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:50.877008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:50.877311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:50.877586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/c83c4b95067b4383823c9745fc40e79f/groups/1c728d9e6a52431aabab11bdc502f7fb/roles/ada4c36d93fa44c9aa79296199e43317` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:50.884096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=c83c4b95067b4383823c9745fc40e79f, group_id=1c728d9e6a52431aabab11bdc502f7fb, role_id=ada4c36d93fa44c9aa79296199e43317)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:50.888087 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:50.888087 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:50.888087 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:50.891477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:50.924877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invalidating the token cache because role ada4c36d93fa44c9aa79296199e43317 was removed from group 1c728d9e6a52431aabab11bdc502f7fb on domain c83c4b95067b4383823c9745fc40e79f. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:50.925208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:50.926305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:50.963453 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2038/4105] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Oct 15 21:08:50 2022] DELETE /identity/v3/domains/c83c4b95067b4383823c9745fc40e79f/groups/1c728d9e6a52431aabab11bdc502f7fb/roles/ada4c36d93fa44c9aa79296199e43317 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:50.964479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:50.967727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:50.969287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:51.019256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd16f7-260a-4382-a213-c6dcfa1a40be admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:51.088433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bac5f-cee4-4896-b729-760347055326 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:51.099658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068d434-3256-4d7b-bd0c-cce7a60b40da tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:51.102403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068d434-3256-4d7b-bd0c-cce7a60b40da tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:51.102403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068d434-3256-4d7b-bd0c-cce7a60b40da tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:51.102403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068d434-3256-4d7b-bd0c-cce7a60b40da tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/groups/1c728d9e6a52431aabab11bdc502f7fb/users/f4004fb706684648a08e7a131d5b16a0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:51.103102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068d434-3256-4d7b-bd0c-cce7a60b40da tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=1c728d9e6a52431aabab11bdc502f7fb, user_id=f4004fb706684648a08e7a131d5b16a0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:51.104131 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-c068d434-3256-4d7b-bd0c-cce7a60b40da tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:51.104131 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:51.104131 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:51.105710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068d434-3256-4d7b-bd0c-cce7a60b40da tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:51.111002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:51.126341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c068d434-3256-4d7b-bd0c-cce7a60b40da tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f4004fb706684648a08e7a131d5b16a0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:51.137514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d875e-acd8-4d0a-b373-05db75cd95b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:51.138205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d875e-acd8-4d0a-b373-05db75cd95b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:51.138646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d875e-acd8-4d0a-b373-05db75cd95b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:51.138987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d875e-acd8-4d0a-b373-05db75cd95b2 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:51.140402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d875e-acd8-4d0a-b373-05db75cd95b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:51.148641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d875e-acd8-4d0a-b373-05db75cd95b2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:51.153451 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2039/4106] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Oct 15 21:08:50 2022] DELETE /identity/v3/groups/1c728d9e6a52431aabab11bdc502f7fb/users/f4004fb706684648a08e7a131d5b16a0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:51.159826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068d434-3256-4d7b-bd0c-cce7a60b40da tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:51.214875 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2068/4107] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:50 2022] POST /identity/v3/users => generated 289 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:51.221505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d875e-acd8-4d0a-b373-05db75cd95b2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:51.225372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f5d875e-acd8-4d0a-b373-05db75cd95b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:51.239881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d875e-acd8-4d0a-b373-05db75cd95b2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:51.294149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068d434-3256-4d7b-bd0c-cce7a60b40da tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:51.300423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd697bf-9552-4ec3-a50c-5597e0686d45 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:51.301174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd697bf-9552-4ec3-a50c-5597e0686d45 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:51.301520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd697bf-9552-4ec3-a50c-5597e0686d45 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:51.301862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd697bf-9552-4ec3-a50c-5597e0686d45 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/groups/1c728d9e6a52431aabab11bdc502f7fb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:51.303159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd697bf-9552-4ec3-a50c-5597e0686d45 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_group(group_id=1c728d9e6a52431aabab11bdc502f7fb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:51.304241 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-1dd697bf-9552-4ec3-a50c-5597e0686d45 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:51.304241 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:51.304241 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:51.306296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd697bf-9552-4ec3-a50c-5597e0686d45 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:51.340095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5980539-caff-4448-a20c-ab305091b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:51.343116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5980539-caff-4448-a20c-ab305091b3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:51.343482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5980539-caff-4448-a20c-ab305091b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:51.346603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5980539-caff-4448-a20c-ab305091b3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:51.347614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5980539-caff-4448-a20c-ab305091b3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:51.349853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5980539-caff-4448-a20c-ab305091b3f8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:51.364686 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2069/4108] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:51 2022] GET /identity/v3/roles => generated 2341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:51.368341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5980539-caff-4448-a20c-ab305091b3f8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:51.384049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5980539-caff-4448-a20c-ab305091b3f8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:51.391939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f0cb9-8ed9-47b8-a32e-336ad94d8f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:51.392677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f0cb9-8ed9-47b8-a32e-336ad94d8f8f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:51.393022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f0cb9-8ed9-47b8-a32e-336ad94d8f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:51.393374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f0cb9-8ed9-47b8-a32e-336ad94d8f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:51.394127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f0cb9-8ed9-47b8-a32e-336ad94d8f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:51.394594 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2040/4109] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:08:51 2022] DELETE /identity/v3/groups/1c728d9e6a52431aabab11bdc502f7fb => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:51.396311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f0cb9-8ed9-47b8-a32e-336ad94d8f8f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:51.398797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd697bf-9552-4ec3-a50c-5597e0686d45 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:51.423809 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2070/4110] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:51 2022] GET /identity/v3/roles => generated 2341 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:08:51.427338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f0cb9-8ed9-47b8-a32e-336ad94d8f8f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:51.438374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f0cb9-8ed9-47b8-a32e-336ad94d8f8f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:51.517800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd697bf-9552-4ec3-a50c-5597e0686d45 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:51.519909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:51.527812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:51.528658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:51.529285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:51.529575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles/ada4c36d93fa44c9aa79296199e43317` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:51.531534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_role(role_id=ada4c36d93fa44c9aa79296199e43317)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:51.532555 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:51.532555 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:51.532555 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:51.533625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:51.535577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53db41-f075-44bf-a8bf-98560b8234d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:51.536805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53db41-f075-44bf-a8bf-98560b8234d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:51.537192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53db41-f075-44bf-a8bf-98560b8234d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:51.537541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53db41-f075-44bf-a8bf-98560b8234d9 admin admin] PATH_INFO: `/v3/projects/bd273f1984eb48ed960715c7bab4d8d8/users/9325167a28da4f298d6b581802cc7865/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:51.557824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53db41-f075-44bf-a8bf-98560b8234d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd273f1984eb48ed960715c7bab4d8d8, user_id=9325167a28da4f298d6b581802cc7865, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:51.561560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53db41-f075-44bf-a8bf-98560b8234d9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:51.589837 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2071/4111] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:51 2022] PUT /identity/v3/projects/bd273f1984eb48ed960715c7bab4d8d8/users/9325167a28da4f298d6b581802cc7865/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:51.611633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ed51f-cdf5-40ac-ad47-58eaad91622b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:51.612900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ed51f-cdf5-40ac-ad47-58eaad91622b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:51.613889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ed51f-cdf5-40ac-ad47-58eaad91622b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:51.627849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invalidating the token cache because role ada4c36d93fa44c9aa79296199e43317 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:51.628300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:51.629103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:51.636055 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2041/4112] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:51 2022] DELETE /identity/v3/roles/ada4c36d93fa44c9aa79296199e43317 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:51.637107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:51.642270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:51.677479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:51.694131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-219ed51f-cdf5-40ac-ad47-58eaad91622b None None] MFA Rules not processed for user `9325167a28da4f298d6b581802cc7865`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:51.695250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-219ed51f-cdf5-40ac-ad47-58eaad91622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:51.793087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0d17d-2c88-431c-8229-8e8673e4dc3b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:51.800598 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2072/4113] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:51.801525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ed51f-cdf5-40ac-ad47-58eaad91622b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:51.803611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-219ed51f-cdf5-40ac-ad47-58eaad91622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:51.804575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:51.805345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:51.805676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:51.806027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/c83c4b95067b4383823c9745fc40e79f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:51.812863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=c83c4b95067b4383823c9745fc40e79f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:51.814402 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:51.814402 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:08:51.814402 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:08:51.815730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:51.856441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:51.859020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:51.886181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ed51f-cdf5-40ac-ad47-58eaad91622b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:51.917145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:51.920307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:51.926823 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2042/4114] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:51 2022] PATCH /identity/v3/domains/c83c4b95067b4383823c9745fc40e79f => generated 277 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:51.931005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:51.937602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:51.990926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39efd1-69c2-4bf0-9976-562fe65a5bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:51.991588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39efd1-69c2-4bf0-9976-562fe65a5bee admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:51.992111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39efd1-69c2-4bf0-9976-562fe65a5bee admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:51.992388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39efd1-69c2-4bf0-9976-562fe65a5bee admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:51.993095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39efd1-69c2-4bf0-9976-562fe65a5bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:51.996049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39efd1-69c2-4bf0-9976-562fe65a5bee admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:52.044869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:52.057070 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2073/4115] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:51 2022] POST /identity/v3/projects => generated 377 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:52.062982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39efd1-69c2-4bf0-9976-562fe65a5bee admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:52.064723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fcd29-531c-4e27-b584-47d55527e14d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:52.069677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ad39efd1-69c2-4bf0-9976-562fe65a5bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:52.195105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:52.195764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:52.196038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:52.196303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/c83c4b95067b4383823c9745fc40e79f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:52.227758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c83c4b95067b4383823c9745fc40e79f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:52.229722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:52.272941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:52.282601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39efd1-69c2-4bf0-9976-562fe65a5bee admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:52.298108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6bb22-3ba3-47c7-811b-0d33df34c2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:52.299187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.305155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6bb22-3ba3-47c7-811b-0d33df34c2fa admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:52.305851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6bb22-3ba3-47c7-811b-0d33df34c2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:52.306308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6bb22-3ba3-47c7-811b-0d33df34c2fa admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:52.307249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6bb22-3ba3-47c7-811b-0d33df34c2fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:52.310450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6bb22-3ba3-47c7-811b-0d33df34c2fa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:52.336036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.361698 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2074/4116] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:52 2022] POST /identity/v3/users => generated 295 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:52.363067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6bb22-3ba3-47c7-811b-0d33df34c2fa admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:52.386860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6bb22-3ba3-47c7-811b-0d33df34c2fa admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:52.403543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ad453-9763-4374-a176-b6cb03746b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:52.404382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ad453-9763-4374-a176-b6cb03746b07 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:52.404818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ad453-9763-4374-a176-b6cb03746b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:52.405638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ad453-9763-4374-a176-b6cb03746b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:52.406464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ad453-9763-4374-a176-b6cb03746b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:52.411352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ad453-9763-4374-a176-b6cb03746b07 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:52.432505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f', 'request_id': 'req-16fc130f-fac0-4f71-a11f-df6ead85ee5a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.436291 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2075/4117] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:52 2022] GET /identity/v3/roles => generated 2111 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:52.440812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ad453-9763-4374-a176-b6cb03746b07 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:52.460701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f', 'request_id': 'req-16fc130f-fac0-4f71-a11f-df6ead85ee5a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.472767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ad453-9763-4374-a176-b6cb03746b07 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:52.479441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f', 'request_id': 'req-16fc130f-fac0-4f71-a11f-df6ead85ee5a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.518515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] The token cache is being invalidate because project c83c4b95067b4383823c9745fc40e79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:52.518810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.521340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.546206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f', 'request_id': 'req-16fc130f-fac0-4f71-a11f-df6ead85ee5a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.549243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f', 'request_id': 'req-16fc130f-fac0-4f71-a11f-df6ead85ee5a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.551409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c83c4b95067b4383823c9745fc40e79f', 'request_id': 'req-16fc130f-fac0-4f71-a11f-df6ead85ee5a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.574824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e708734-2465-4e4f-acb9-4fcbdb82e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:52.575996 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2043/4118] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:51 2022] DELETE /identity/v3/domains/c83c4b95067b4383823c9745fc40e79f => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:52.579352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e708734-2465-4e4f-acb9-4fcbdb82e071 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:52.579953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:52.581094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e708734-2465-4e4f-acb9-4fcbdb82e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:52.582956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e708734-2465-4e4f-acb9-4fcbdb82e071 admin admin] PATH_INFO: `/v3/projects/f694762209cd4f8d89c405b3926c8d7d/users/41c8b53752a049eebfe107aa0f969203/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:52.584510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:52.616716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e708734-2465-4e4f-acb9-4fcbdb82e071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f694762209cd4f8d89c405b3926c8d7d, user_id=41c8b53752a049eebfe107aa0f969203, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:52.620699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e708734-2465-4e4f-acb9-4fcbdb82e071 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:52.657367 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2076/4119] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:52 2022] PUT /identity/v3/projects/f694762209cd4f8d89c405b3926c8d7d/users/41c8b53752a049eebfe107aa0f969203/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:52.661281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e708734-2465-4e4f-acb9-4fcbdb82e071 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:52.663823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2e708734-2465-4e4f-acb9-4fcbdb82e071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:52.758652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc130f-fac0-4f71-a11f-df6ead85ee5a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:52.844100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e708734-2465-4e4f-acb9-4fcbdb82e071 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:52.865129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562d920-095c-47b1-a071-ea447f9d891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:52.865752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562d920-095c-47b1-a071-ea447f9d891b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:52.866076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562d920-095c-47b1-a071-ea447f9d891b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:52.866286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562d920-095c-47b1-a071-ea447f9d891b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:52.867252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562d920-095c-47b1-a071-ea447f9d891b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:52.869140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562d920-095c-47b1-a071-ea447f9d891b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:52.875368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:52.884546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:52.885264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:52.885541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:52.885806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/d2518f06a2f54b25b22683720a4220ff/users/f4004fb706684648a08e7a131d5b16a0/roles/fc489bfa855646d6aaa2d88274be50c8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:52.899195 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2077/4120] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:52 2022] GET /identity/v3/roles => generated 2111 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:08:52.899967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=d2518f06a2f54b25b22683720a4220ff, user_id=f4004fb706684648a08e7a131d5b16a0, role_id=fc489bfa855646d6aaa2d88274be50c8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:52.900595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562d920-095c-47b1-a071-ea447f9d891b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:52.909970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:52.928081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562d920-095c-47b1-a071-ea447f9d891b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:52.942272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invalidating the token cache because role fc489bfa855646d6aaa2d88274be50c8 was removed from user f4004fb706684648a08e7a131d5b16a0 on domain d2518f06a2f54b25b22683720a4220ff. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:52.942573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.944118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d9c2e-68a4-4912-b625-7bee5e268d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:52.944783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d9c2e-68a4-4912-b625-7bee5e268d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:52.945120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d9c2e-68a4-4912-b625-7bee5e268d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:52.945408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d9c2e-68a4-4912-b625-7bee5e268d6c admin admin] PATH_INFO: `/v3/projects/f694762209cd4f8d89c405b3926c8d7d/users/41c8b53752a049eebfe107aa0f969203/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:52.946493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:52.950357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d9c2e-68a4-4912-b625-7bee5e268d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f694762209cd4f8d89c405b3926c8d7d, user_id=41c8b53752a049eebfe107aa0f969203, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:52.957330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d9c2e-68a4-4912-b625-7bee5e268d6c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:52.968829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:52.978748 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2044/4121] 10.222.0.16 () {58 vars in 1464 bytes} [Sat Oct 15 21:08:52 2022] DELETE /identity/v3/domains/d2518f06a2f54b25b22683720a4220ff/users/f4004fb706684648a08e7a131d5b16a0/roles/fc489bfa855646d6aaa2d88274be50c8 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:52.984602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:52.987197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:52.994434 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2078/4122] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:52 2022] PUT /identity/v3/projects/f694762209cd4f8d89c405b3926c8d7d/users/41c8b53752a049eebfe107aa0f969203/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:53.005678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0ebce-0cfe-4b13-baec-0a080ffb42f2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:53.005959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0ebce-0cfe-4b13-baec-0a080ffb42f2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:53.006294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0ebce-0cfe-4b13-baec-0a080ffb42f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:53.079178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d3b0ebce-0cfe-4b13-baec-0a080ffb42f2 None None] MFA Rules not processed for user `41c8b53752a049eebfe107aa0f969203`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:53.080696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d3b0ebce-0cfe-4b13-baec-0a080ffb42f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:53.217876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f4a53-de4d-40f0-ac15-b0d20cf4eec5 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:53.256849 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2079/4123] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:08:53.258956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0ebce-0cfe-4b13-baec-0a080ffb42f2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:53.276540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0ebce-0cfe-4b13-baec-0a080ffb42f2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:53.284357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e52a7-177f-425f-81d2-69cabb0ae496 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:53.284983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e52a7-177f-425f-81d2-69cabb0ae496 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:53.285362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e52a7-177f-425f-81d2-69cabb0ae496 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:53.285633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e52a7-177f-425f-81d2-69cabb0ae496 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:53.286638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e52a7-177f-425f-81d2-69cabb0ae496 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:53.287726 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-bb4e52a7-177f-425f-81d2-69cabb0ae496 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:53.287726 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:53.287726 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:53.288764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e52a7-177f-425f-81d2-69cabb0ae496 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:53.327367 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2080/4124] 10.222.0.16 () {60 vars in 1185 bytes} [Sat Oct 15 21:08:53 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:53.345489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e52a7-177f-425f-81d2-69cabb0ae496 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:53.373959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e52a7-177f-425f-81d2-69cabb0ae496 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:53.389932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fbf9e-62b8-4f08-aebe-0fe1e15c4020 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:53.390589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fbf9e-62b8-4f08-aebe-0fe1e15c4020 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:53.390924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fbf9e-62b8-4f08-aebe-0fe1e15c4020 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:53.391154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fbf9e-62b8-4f08-aebe-0fe1e15c4020 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/89de3e3c9a4a4e529c960917ce91ab37` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:53.391748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fbf9e-62b8-4f08-aebe-0fe1e15c4020 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=89de3e3c9a4a4e529c960917ce91ab37)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:53.392777 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-991fbf9e-62b8-4f08-aebe-0fe1e15c4020 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:53.392777 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:53.392777 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:53.396196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:53.396859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:53.397291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:53.397561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles/fc489bfa855646d6aaa2d88274be50c8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:53.398428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fbf9e-62b8-4f08-aebe-0fe1e15c4020 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:53.398989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_role(role_id=fc489bfa855646d6aaa2d88274be50c8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:53.402524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:53.413398 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2081/4125] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:08:53 2022] GET /identity/v3/OS-OAUTH1/consumers/89de3e3c9a4a4e529c960917ce91ab37 => generated 220 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:08:53.419163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fbf9e-62b8-4f08-aebe-0fe1e15c4020 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:53.443704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fbf9e-62b8-4f08-aebe-0fe1e15c4020 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:53.457770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:53.458423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:53.458707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:53.458981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/89de3e3c9a4a4e529c960917ce91ab37` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:53.459573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=89de3e3c9a4a4e529c960917ce91ab37)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:53.460597 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:53.460597 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:53.460597 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:53.466253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:53.466544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invalidating token cache because consumer 89de3e3c9a4a4e529c960917ce91ab37 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:53.466833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:53.471439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:53.487934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invalidating the token cache because role fc489bfa855646d6aaa2d88274be50c8 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:53.488298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:53.488928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:53.491770 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2045/4126] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:52 2022] DELETE /identity/v3/roles/fc489bfa855646d6aaa2d88274be50c8 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:53.495897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:53.498226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:53.529959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '89de3e3c9a4a4e529c960917ce91ab37', 'request_id': 'req-e82ceb8d-a414-488d-a810-2312db23821f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:53.557574 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2082/4127] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:53 2022] DELETE /identity/v3/OS-OAUTH1/consumers/89de3e3c9a4a4e529c960917ce91ab37 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:53.558728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:53.567353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:53.599605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ceb8d-a414-488d-a810-2312db23821f tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:53.687151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5559d8e-8714-4adf-a0c1-00da0833d83c tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:53.687772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5559d8e-8714-4adf-a0c1-00da0833d83c tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:53.688053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5559d8e-8714-4adf-a0c1-00da0833d83c tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:53.688321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5559d8e-8714-4adf-a0c1-00da0833d83c tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:53.689690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5559d8e-8714-4adf-a0c1-00da0833d83c tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:53.691511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5559d8e-8714-4adf-a0c1-00da0833d83c tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:53.699243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dac02c-0222-48b3-98be-363ddb0265a2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:53.713320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:53.713980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:53.714805 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2083/4128] 10.222.0.16 () {60 vars in 1185 bytes} [Sat Oct 15 21:08:53 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:08:53.716413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:53.716704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/d2518f06a2f54b25b22683720a4220ff` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:53.719499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d2518f06a2f54b25b22683720a4220ff)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:53.722079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5559d8e-8714-4adf-a0c1-00da0833d83c tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:53.725761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:53.730244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f5559d8e-8714-4adf-a0c1-00da0833d83c tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:53.757297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5559d8e-8714-4adf-a0c1-00da0833d83c tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:53.763354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:53.778430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8aaad-5e20-4087-a2a4-5cead498e8d1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:53.779136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8aaad-5e20-4087-a2a4-5cead498e8d1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:53.779418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8aaad-5e20-4087-a2a4-5cead498e8d1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:53.779693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8aaad-5e20-4087-a2a4-5cead498e8d1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/29e5dbb769a14c0cb5ed934d2c2a0bad` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:53.780293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8aaad-5e20-4087-a2a4-5cead498e8d1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=29e5dbb769a14c0cb5ed934d2c2a0bad)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:53.781938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8aaad-5e20-4087-a2a4-5cead498e8d1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:53.787113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:53.788827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:53.804932 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2084/4129] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:08:53 2022] GET /identity/v3/OS-OAUTH1/consumers/29e5dbb769a14c0cb5ed934d2c2a0bad => generated 221 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:53.805933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8aaad-5e20-4087-a2a4-5cead498e8d1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:53.811656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3db8aaad-5e20-4087-a2a4-5cead498e8d1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:53.843952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8aaad-5e20-4087-a2a4-5cead498e8d1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:53.848123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:53.848885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:53.853763 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2046/4130] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:53 2022] PATCH /identity/v3/domains/d2518f06a2f54b25b22683720a4220ff => generated 276 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:08:53.857937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:53.860731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:53.865057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:53.984897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:53.985644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:53.985879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:53.986255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/29e5dbb769a14c0cb5ed934d2c2a0bad` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:53.986911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=29e5dbb769a14c0cb5ed934d2c2a0bad)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:53.988423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:53.988691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invalidating token cache because consumer 29e5dbb769a14c0cb5ed934d2c2a0bad has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:53.988970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:53.991632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.016876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '29e5dbb769a14c0cb5ed934d2c2a0bad', 'request_id': 'req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.043614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54564d97-2a26-4e2c-a1e3-d675f57db028 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:54.047773 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2085/4131] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:53 2022] DELETE /identity/v3/OS-OAUTH1/consumers/29e5dbb769a14c0cb5ed934d2c2a0bad => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:54.048732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:54.054560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:54.055446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:54.056107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:54.056382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:54.056652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/d2518f06a2f54b25b22683720a4220ff` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:54.083872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d2518f06a2f54b25b22683720a4220ff)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:54.085549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:54.087195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9e79e-a01c-4fd4-9fc6-9ebadb4326aa tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:54.099644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b02ea8-7e0e-43bd-90ce-cd77a3c2b611 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:54.100396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b02ea8-7e0e-43bd-90ce-cd77a3c2b611 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:54.100683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b02ea8-7e0e-43bd-90ce-cd77a3c2b611 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:54.100962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b02ea8-7e0e-43bd-90ce-cd77a3c2b611 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/29e5dbb769a14c0cb5ed934d2c2a0bad` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:54.102147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b02ea8-7e0e-43bd-90ce-cd77a3c2b611 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=29e5dbb769a14c0cb5ed934d2c2a0bad)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:54.103994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b02ea8-7e0e-43bd-90ce-cd77a3c2b611 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-83b02ea8-7e0e-43bd-90ce-cd77a3c2b611 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 100, in get Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_consumer(consumer_id) Oct 15 21:08:54.151270 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 95, in _get_consumer Oct 15 21:08:54.152355 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.oauth_api.get_consumer(consumer_id)) Oct 15 21:08:54.152355 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 101, in get_consumer Oct 15 21:08:54.152355 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.get_consumer_with_secret(consumer_id)) Oct 15 21:08:54.152355 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 96, in get_consumer_with_secret Oct 15 21:08:54.152355 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Oct 15 21:08:54.152355 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Oct 15 21:08:54.152355 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Oct 15 21:08:54.152355 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Oct 15 21:08:54.152355 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:54.154188 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2086/4132] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:08:54 2022] GET /identity/v3/OS-OAUTH1/consumers/29e5dbb769a14c0cb5ed934d2c2a0bad => generated 74 bytes in 106 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 21:08:54.159010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b02ea8-7e0e-43bd-90ce-cd77a3c2b611 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:54.161097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.182784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.204914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b02ea8-7e0e-43bd-90ce-cd77a3c2b611 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:54.232041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:54.234087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:54.236235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:54.237943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/29e5dbb769a14c0cb5ed934d2c2a0bad` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:54.239961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=29e5dbb769a14c0cb5ed934d2c2a0bad)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:54.241725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:54.245478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invalidating token cache because consumer 29e5dbb769a14c0cb5ed934d2c2a0bad has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:54.247584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.251986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 123, in delete Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.oauth_api.delete_consumer( Oct 15 21:08:54.302223 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/core.py", line 139, in delete_consumer Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self.driver.delete_consumer(consumer_id) Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 133, in delete_consumer Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._delete_consumer(session, consumer_id) Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 110, in _delete_consumer Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Oct 15 21:08:54.303335 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:08:54.305367 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2087/4133] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:54 2022] DELETE /identity/v3/OS-OAUTH1/consumers/29e5dbb769a14c0cb5ed934d2c2a0bad => generated 74 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 21:08:54.311141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:54.318574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:54.320430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff', 'request_id': 'req-4faa9087-0cab-4142-b90c-dca494c57a6a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.342588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff', 'request_id': 'req-4faa9087-0cab-4142-b90c-dca494c57a6a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.346127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8abd0f-f1f2-4ddf-a530-5e4a71d9e3a1 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:54.358130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff', 'request_id': 'req-4faa9087-0cab-4142-b90c-dca494c57a6a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.391580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] The token cache is being invalidate because project d2518f06a2f54b25b22683720a4220ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:54.392016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.393058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.411952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff', 'request_id': 'req-4faa9087-0cab-4142-b90c-dca494c57a6a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.415658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff', 'request_id': 'req-4faa9087-0cab-4142-b90c-dca494c57a6a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.416313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd2518f06a2f54b25b22683720a4220ff', 'request_id': 'req-4faa9087-0cab-4142-b90c-dca494c57a6a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.433906 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2047/4134] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:53 2022] DELETE /identity/v3/domains/d2518f06a2f54b25b22683720a4220ff => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:54.439498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:54.442192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:54.472510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:54.475200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff7bbc-c012-4bd6-b8c0-0f05d5ba9f8b tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:54.477114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7bbc-c012-4bd6-b8c0-0f05d5ba9f8b tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:54.477588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7bbc-c012-4bd6-b8c0-0f05d5ba9f8b tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:54.478019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7bbc-c012-4bd6-b8c0-0f05d5ba9f8b tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:54.478840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff7bbc-c012-4bd6-b8c0-0f05d5ba9f8b tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:54.480673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff7bbc-c012-4bd6-b8c0-0f05d5ba9f8b tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:54.545220 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2088/4135] 10.222.0.16 () {60 vars in 1185 bytes} [Sat Oct 15 21:08:54 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:54.546114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff7bbc-c012-4bd6-b8c0-0f05d5ba9f8b tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:54.550056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-63ff7bbc-c012-4bd6-b8c0-0f05d5ba9f8b tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:54.574465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff7bbc-c012-4bd6-b8c0-0f05d5ba9f8b tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:54.584559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34e866-098b-46d2-9afd-de85487b31c2 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:54.587253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34e866-098b-46d2-9afd-de85487b31c2 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:54.587559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34e866-098b-46d2-9afd-de85487b31c2 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:54.588179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34e866-098b-46d2-9afd-de85487b31c2 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:54.589221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34e866-098b-46d2-9afd-de85487b31c2 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:54.592632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34e866-098b-46d2-9afd-de85487b31c2 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:54.601660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa9087-0cab-4142-b90c-dca494c57a6a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:54.634511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:54.635434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:54.638095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:54.639083 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2089/4136] 10.222.0.16 () {60 vars in 1185 bytes} [Sat Oct 15 21:08:54 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:54.639631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/d5ed0d337b2546e6a03f4c220ad1124d/groups/a836b2ccdd6343c19d82dbb135117f0e/roles/5b3e87e24fb544899b6c89a7de7b7175` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:54.643352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34e866-098b-46d2-9afd-de85487b31c2 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:54.645324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=d5ed0d337b2546e6a03f4c220ad1124d, group_id=a836b2ccdd6343c19d82dbb135117f0e, role_id=5b3e87e24fb544899b6c89a7de7b7175)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:54.654942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:54.670220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34e866-098b-46d2-9afd-de85487b31c2 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:54.671041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invalidating the token cache because role 5b3e87e24fb544899b6c89a7de7b7175 was removed from group a836b2ccdd6343c19d82dbb135117f0e on domain d5ed0d337b2546e6a03f4c220ad1124d. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:54.672228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.673825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:54.681541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e6a8-02b1-4b47-9756-c24810784cea tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:54.684042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e6a8-02b1-4b47-9756-c24810784cea tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:54.685386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e6a8-02b1-4b47-9756-c24810784cea tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:54.686051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e6a8-02b1-4b47-9756-c24810784cea tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:54.687331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83e6a8-02b1-4b47-9756-c24810784cea tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:list_consumers()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:54.688878 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-1e83e6a8-02b1-4b47-9756-c24810784cea tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_consumers": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:54.688878 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:54.688878 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:54.690495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83e6a8-02b1-4b47-9756-c24810784cea tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:54.713452 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2048/4137] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Oct 15 21:08:54 2022] DELETE /identity/v3/domains/d5ed0d337b2546e6a03f4c220ad1124d/groups/a836b2ccdd6343c19d82dbb135117f0e/roles/5b3e87e24fb544899b6c89a7de7b7175 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:54.717340 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2090/4138] 10.222.0.16 () {58 vars in 1164 bytes} [Sat Oct 15 21:08:54 2022] GET /identity/v3/OS-OAUTH1/consumers => generated 536 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:54.721507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e6a8-02b1-4b47-9756-c24810784cea tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:54.722767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:54.757846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:54.758948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e83e6a8-02b1-4b47-9756-c24810784cea tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:54.803809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e6a8-02b1-4b47-9756-c24810784cea tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:54.924500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:55.008260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:55.008903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:55.009297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:55.009580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/d040dfcf67f241acac6d27fab1fc4e4b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:55.010293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=d040dfcf67f241acac6d27fab1fc4e4b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:55.012469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:55.012744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invalidating token cache because consumer d040dfcf67f241acac6d27fab1fc4e4b has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:55.013024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.013768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.034643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'd040dfcf67f241acac6d27fab1fc4e4b', 'request_id': 'req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.043157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71f341-47e1-4a00-ae62-d39353ac9783 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:55.053673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca8ffb-9bc7-47b4-9fbc-8a80c2a7ca61 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:55.054054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca8ffb-9bc7-47b4-9fbc-8a80c2a7ca61 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:55.054483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca8ffb-9bc7-47b4-9fbc-8a80c2a7ca61 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:55.054762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca8ffb-9bc7-47b4-9fbc-8a80c2a7ca61 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/groups/a836b2ccdd6343c19d82dbb135117f0e/users/f4004fb706684648a08e7a131d5b16a0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:55.057417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca8ffb-9bc7-47b4-9fbc-8a80c2a7ca61 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=a836b2ccdd6343c19d82dbb135117f0e, user_id=f4004fb706684648a08e7a131d5b16a0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:55.065732 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2091/4139] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:54 2022] DELETE /identity/v3/OS-OAUTH1/consumers/d040dfcf67f241acac6d27fab1fc4e4b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:55.067638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:55.068132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca8ffb-9bc7-47b4-9fbc-8a80c2a7ca61 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:55.074163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:55.086204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-70ca8ffb-9bc7-47b4-9fbc-8a80c2a7ca61 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f4004fb706684648a08e7a131d5b16a0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.123966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98999a-9858-4a4a-9d6f-ea18b7e38d35 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:55.127045 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2049/4140] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Oct 15 21:08:54 2022] DELETE /identity/v3/groups/a836b2ccdd6343c19d82dbb135117f0e/users/f4004fb706684648a08e7a131d5b16a0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:55.131178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca8ffb-9bc7-47b4-9fbc-8a80c2a7ca61 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:55.133905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-70ca8ffb-9bc7-47b4-9fbc-8a80c2a7ca61 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:55.273533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:55.274182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:55.274459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:55.274725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f3a228e1a59f4ef888224d26ac4dbd48` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:55.275303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=f3a228e1a59f4ef888224d26ac4dbd48)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:55.279394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:55.279724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invalidating token cache because consumer f3a228e1a59f4ef888224d26ac4dbd48 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:55.280008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.280713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.295875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca8ffb-9bc7-47b4-9fbc-8a80c2a7ca61 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:55.301202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da876055-cb71-4bfc-8b05-ddcb0266c047 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:55.301982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da876055-cb71-4bfc-8b05-ddcb0266c047 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:55.302228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da876055-cb71-4bfc-8b05-ddcb0266c047 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:55.302658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da876055-cb71-4bfc-8b05-ddcb0266c047 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/groups/a836b2ccdd6343c19d82dbb135117f0e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:55.303948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da876055-cb71-4bfc-8b05-ddcb0266c047 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_group(group_id=a836b2ccdd6343c19d82dbb135117f0e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:55.306117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da876055-cb71-4bfc-8b05-ddcb0266c047 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:55.307990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'f3a228e1a59f4ef888224d26ac4dbd48', 'request_id': 'req-696c72b0-1e63-4baf-bdc0-59ccec9c5688'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.332475 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2092/4141] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:55 2022] DELETE /identity/v3/OS-OAUTH1/consumers/f3a228e1a59f4ef888224d26ac4dbd48 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:55.337452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:55.339098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:55.357998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c72b0-1e63-4baf-bdc0-59ccec9c5688 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:55.358541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:55.366184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c9a0-5715-4064-821a-79b55286c02a tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:55.367539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c9a0-5715-4064-821a-79b55286c02a tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:55.367936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c9a0-5715-4064-821a-79b55286c02a tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:55.368313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c9a0-5715-4064-821a-79b55286c02a tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:55.369289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c9a0-5715-4064-821a-79b55286c02a tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:55.371254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c9a0-5715-4064-821a-79b55286c02a tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:55.380563 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2050/4142] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:08:55 2022] DELETE /identity/v3/groups/a836b2ccdd6343c19d82dbb135117f0e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:55.385614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da876055-cb71-4bfc-8b05-ddcb0266c047 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:55.387562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-da876055-cb71-4bfc-8b05-ddcb0266c047 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:55.420187 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2093/4143] 10.222.0.16 () {60 vars in 1185 bytes} [Sat Oct 15 21:08:55 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:55.421712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c9a0-5715-4064-821a-79b55286c02a tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:55.433397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c9a0-5715-4064-821a-79b55286c02a tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:55.515071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da876055-cb71-4bfc-8b05-ddcb0266c047 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:55.520393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:55.521532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:55.522005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:55.522694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles/5b3e87e24fb544899b6c89a7de7b7175` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:55.524389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_role(role_id=5b3e87e24fb544899b6c89a7de7b7175)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:55.531625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:55.584379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12bc52-8ea1-436f-8e8c-07fb47a451f0 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:55.585179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12bc52-8ea1-436f-8e8c-07fb47a451f0 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:55.585565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12bc52-8ea1-436f-8e8c-07fb47a451f0 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:55.585933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12bc52-8ea1-436f-8e8c-07fb47a451f0 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/860dd8871ebd4c0bbae50e0634f74562` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:55.587060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12bc52-8ea1-436f-8e8c-07fb47a451f0 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:update_consumer(consumer_id=860dd8871ebd4c0bbae50e0634f74562)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:55.599259 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-0a12bc52-8ea1-436f-8e8c-07fb47a451f0 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:55.599259 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:55.599259 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:55.599915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invalidating the token cache because role 5b3e87e24fb544899b6c89a7de7b7175 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:55.600262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.601828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.602231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12bc52-8ea1-436f-8e8c-07fb47a451f0 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:55.604286 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2051/4144] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:55 2022] DELETE /identity/v3/roles/5b3e87e24fb544899b6c89a7de7b7175 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:55.608527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:55.610604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:55.660313 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2094/4145] 10.222.0.16 () {60 vars in 1285 bytes} [Sat Oct 15 21:08:55 2022] PATCH /identity/v3/OS-OAUTH1/consumers/860dd8871ebd4c0bbae50e0634f74562 => generated 221 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:55.661359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12bc52-8ea1-436f-8e8c-07fb47a451f0 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:55.664722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0a12bc52-8ea1-436f-8e8c-07fb47a451f0 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:55.685228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12bc52-8ea1-436f-8e8c-07fb47a451f0 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:55.720282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:55.744800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8e0a7-1732-453f-a71b-4a20af80f83d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:55.751664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:55.755926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:55.756223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:55.756492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/d5ed0d337b2546e6a03f4c220ad1124d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:55.757882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d5ed0d337b2546e6a03f4c220ad1124d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:55.765183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:55.832389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecabc6-9f5e-4f51-b1d9-b15d3a96ecd4 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:55.833488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecabc6-9f5e-4f51-b1d9-b15d3a96ecd4 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:55.833958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecabc6-9f5e-4f51-b1d9-b15d3a96ecd4 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:55.835663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecabc6-9f5e-4f51-b1d9-b15d3a96ecd4 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/860dd8871ebd4c0bbae50e0634f74562` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:55.836376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecabc6-9f5e-4f51-b1d9-b15d3a96ecd4 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=860dd8871ebd4c0bbae50e0634f74562)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:55.839460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecabc6-9f5e-4f51-b1d9-b15d3a96ecd4 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:55.846034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.846802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.858496 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2095/4146] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:08:55 2022] GET /identity/v3/OS-OAUTH1/consumers/860dd8871ebd4c0bbae50e0634f74562 => generated 221 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:55.867127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecabc6-9f5e-4f51-b1d9-b15d3a96ecd4 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:55.877739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a7ecabc6-9f5e-4f51-b1d9-b15d3a96ecd4 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:55.885011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.886037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:55.893626 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2052/4147] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:55 2022] PATCH /identity/v3/domains/d5ed0d337b2546e6a03f4c220ad1124d => generated 277 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:08:55.897335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:55.900308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:55.923181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecabc6-9f5e-4f51-b1d9-b15d3a96ecd4 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:55.960737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:56.062223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41c8b53752a049eebfe107aa0f969203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f694762209cd4f8d89c405b3926c8d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:56.063002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:56.063386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:56.063761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/860dd8871ebd4c0bbae50e0634f74562` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:56.064449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=860dd8871ebd4c0bbae50e0634f74562)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:56.069133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:56.069516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invalidating token cache because consumer 860dd8871ebd4c0bbae50e0634f74562 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:56.070805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.075610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.091609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129008e0-817a-4843-93d3-e0fd65cdf00d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:56.107840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '860dd8871ebd4c0bbae50e0634f74562', 'request_id': 'req-5fb8f627-f827-4003-83a4-00e487eaf044'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.113163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:56.114674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:56.115892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:56.116545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/d5ed0d337b2546e6a03f4c220ad1124d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:56.136159 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2096/4148] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:55 2022] DELETE /identity/v3/OS-OAUTH1/consumers/860dd8871ebd4c0bbae50e0634f74562 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:56.142553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:56.144317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:56.169411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d5ed0d337b2546e6a03f4c220ad1124d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:56.170942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:56.249833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.275796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8f627-f827-4003-83a4-00e487eaf044 tempest-OAUTHConsumersV3Test-1368762151 tempest-OAUTHConsumersV3Test-1368762151-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:56.279617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.285907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:56.288239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:56.288513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:56.288788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] PATH_INFO: `/v3/users/9325167a28da4f298d6b581802cc7865` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:56.291233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9325167a28da4f298d6b581802cc7865)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:56.296167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:56.401957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d', 'request_id': 'req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.411879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:56.424649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d', 'request_id': 'req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.431737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9325167a28da4f298d6b581802cc7865'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.437658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d', 'request_id': 'req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.469574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9325167a28da4f298d6b581802cc7865'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.479293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] The token cache is being invalidate because project d5ed0d337b2546e6a03f4c220ad1124d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:56.479922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.485551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9325167a28da4f298d6b581802cc7865'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.486164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.503416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9325167a28da4f298d6b581802cc7865'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.504455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1f1e5b2f-6f4e-4d2c-a972-8db0e89d2e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9325167a28da4f298d6b581802cc7865'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.505115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d', 'request_id': 'req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.505830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d', 'request_id': 'req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.506609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd5ed0d337b2546e6a03f4c220ad1124d', 'request_id': 'req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.517083 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2097/4149] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:56 2022] DELETE /identity/v3/users/9325167a28da4f298d6b581802cc7865 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:56.521876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f74980-2f89-4b34-9ddf-9554008c71fe None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:56.522375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f74980-2f89-4b34-9ddf-9554008c71fe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:56.525473 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2053/4150] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:55 2022] DELETE /identity/v3/domains/d5ed0d337b2546e6a03f4c220ad1124d => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:56.526093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f74980-2f89-4b34-9ddf-9554008c71fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:56.526521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:56.528030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:56.578042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f1f74980-2f89-4b34-9ddf-9554008c71fe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:56.579310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f1f74980-2f89-4b34-9ddf-9554008c71fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:56.645083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:56.659389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b0c95-5b3e-47cd-86d3-4054ecc735b0 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:56.757534 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2098/4151] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:56.783700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f74980-2f89-4b34-9ddf-9554008c71fe None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:56.785957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f1f74980-2f89-4b34-9ddf-9554008c71fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:56.854810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:56.855490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:56.855817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:56.856103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/ff50684481b948a2a34522a1182f4652/users/f4004fb706684648a08e7a131d5b16a0/roles/df26696f83d645268f114d64d7f8ea66` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:56.859001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=ff50684481b948a2a34522a1182f4652, user_id=f4004fb706684648a08e7a131d5b16a0, role_id=df26696f83d645268f114d64d7f8ea66)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:56.862007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:56.870183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invalidating the token cache because role df26696f83d645268f114d64d7f8ea66 was removed from user f4004fb706684648a08e7a131d5b16a0 on domain ff50684481b948a2a34522a1182f4652. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:56.870569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.871178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:56.896549 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2054/4152] 10.222.0.16 () {58 vars in 1464 bytes} [Sat Oct 15 21:08:56 2022] DELETE /identity/v3/domains/ff50684481b948a2a34522a1182f4652/users/f4004fb706684648a08e7a131d5b16a0/roles/df26696f83d645268f114d64d7f8ea66 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:08:56.897660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:56.899301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:56.953738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f74980-2f89-4b34-9ddf-9554008c71fe None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:57.040099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f000bc-44a8-43e0-b3b5-edc8c5900506 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:57.047295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfc80b-d6aa-4481-82ac-41771b722bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:57.047520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:57.048019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:57.048142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfc80b-d6aa-4481-82ac-41771b722bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:57.048332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:57.048435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfc80b-d6aa-4481-82ac-41771b722bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:57.048586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfc80b-d6aa-4481-82ac-41771b722bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:57.048691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles/df26696f83d645268f114d64d7f8ea66` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:57.050455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_role(role_id=df26696f83d645268f114d64d7f8ea66)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:57.050859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-68bfc80b-d6aa-4481-82ac-41771b722bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:57.052295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:57.097942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:57.125308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invalidating the token cache because role df26696f83d645268f114d64d7f8ea66 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:57.126477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.127258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.135417 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2055/4153] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:56 2022] DELETE /identity/v3/roles/df26696f83d645268f114d64d7f8ea66 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:57.136340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:57.138439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:57.214222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfc80b-d6aa-4481-82ac-41771b722bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:57.217795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfc80b-d6aa-4481-82ac-41771b722bc1 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:57.237518 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2099/4154] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:08:57.277406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:57.286561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527d193-527b-451d-babd-672e733ab18b tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:57.294085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:57.295315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:57.295826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:57.296339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/ff50684481b948a2a34522a1182f4652` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:57.297905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=ff50684481b948a2a34522a1182f4652)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:57.299894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:57.333591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'ff50684481b948a2a34522a1182f4652'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.341769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'ff50684481b948a2a34522a1182f4652'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.344160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfc80b-d6aa-4481-82ac-41771b722bc1 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:57.346740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-68bfc80b-d6aa-4481-82ac-41771b722bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:57.376953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfc80b-d6aa-4481-82ac-41771b722bc1 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:57.429189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'ff50684481b948a2a34522a1182f4652'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.431330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'ff50684481b948a2a34522a1182f4652'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.436651 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2056/4155] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:08:57 2022] PATCH /identity/v3/domains/ff50684481b948a2a34522a1182f4652 => generated 278 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:57.438003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:57.439573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:57.484364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:57.485315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:57.485731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:57.486197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] PATH_INFO: `/v3/users/06e2c573568846d095c2a42a1df4e918` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:57.487727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e2c573568846d095c2a42a1df4e918)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:57.491288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:57.541275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8aad1a-acad-4b5d-800a-699b6b7e1964 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:57.549879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:57.550717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:57.551387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:57.551867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/ff50684481b948a2a34522a1182f4652` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:57.567328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=ff50684481b948a2a34522a1182f4652)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:57.569253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:57.588030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e2c573568846d095c2a42a1df4e918'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.602327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'ff50684481b948a2a34522a1182f4652'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.609626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e2c573568846d095c2a42a1df4e918'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.618397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'ff50684481b948a2a34522a1182f4652'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.620934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e2c573568846d095c2a42a1df4e918'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.635289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e2c573568846d095c2a42a1df4e918'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.636095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e2c573568846d095c2a42a1df4e918'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.649632 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2100/4156] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:57 2022] DELETE /identity/v3/users/06e2c573568846d095c2a42a1df4e918 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:57.679559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff50684481b948a2a34522a1182f4652', 'request_id': 'req-7f550c59-d709-4a32-b9f0-a1b33fc803c3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.691746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff50684481b948a2a34522a1182f4652', 'request_id': 'req-7f550c59-d709-4a32-b9f0-a1b33fc803c3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.701264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff50684481b948a2a34522a1182f4652', 'request_id': 'req-7f550c59-d709-4a32-b9f0-a1b33fc803c3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.716098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] The token cache is being invalidate because project ff50684481b948a2a34522a1182f4652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:57.716475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.717062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.725672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'ff50684481b948a2a34522a1182f4652', 'request_id': 'req-7f550c59-d709-4a32-b9f0-a1b33fc803c3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.726430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'ff50684481b948a2a34522a1182f4652', 'request_id': 'req-7f550c59-d709-4a32-b9f0-a1b33fc803c3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.727108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'ff50684481b948a2a34522a1182f4652', 'request_id': 'req-7f550c59-d709-4a32-b9f0-a1b33fc803c3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:57.732625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:57.743858 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2057/4157] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:08:57 2022] DELETE /identity/v3/domains/ff50684481b948a2a34522a1182f4652 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:57.751728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:57.754618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:57.835273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:57.862155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd1072-310d-4028-bc86-3ddc323b5c8f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:57.917706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:57.919640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:57.935489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76c451-4c74-4fbc-a4d4-17670fd8f579 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:57.937027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76c451-4c74-4fbc-a4d4-17670fd8f579 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:57.937509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76c451-4c74-4fbc-a4d4-17670fd8f579 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:57.937967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76c451-4c74-4fbc-a4d4-17670fd8f579 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:57.939212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76c451-4c74-4fbc-a4d4-17670fd8f579 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:57.941007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76c451-4c74-4fbc-a4d4-17670fd8f579 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:57.954788 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2101/4158] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:08:57.959399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76c451-4c74-4fbc-a4d4-17670fd8f579 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:57.961534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5b76c451-4c74-4fbc-a4d4-17670fd8f579 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:57.977211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76c451-4c74-4fbc-a4d4-17670fd8f579 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:57.982778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6520897-727f-4013-ba81-8423c722cae2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:57.984060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6520897-727f-4013-ba81-8423c722cae2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:57.984319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6520897-727f-4013-ba81-8423c722cae2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:57.984740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6520897-727f-4013-ba81-8423c722cae2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/projects/c6fb088529854b909d10965a99ead2e5/users/74e48edf46d54884ad3bd49fec65e1c6/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:57.986719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6520897-727f-4013-ba81-8423c722cae2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c6fb088529854b909d10965a99ead2e5, user_id=74e48edf46d54884ad3bd49fec65e1c6, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:57.990068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6520897-727f-4013-ba81-8423c722cae2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:58.008712 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2102/4159] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:08:57 2022] PUT /identity/v3/projects/c6fb088529854b909d10965a99ead2e5/users/74e48edf46d54884ad3bd49fec65e1c6/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:58.013096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6520897-727f-4013-ba81-8423c722cae2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:58.022096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f550c59-d709-4a32-b9f0-a1b33fc803c3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:58.095015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6520897-727f-4013-ba81-8423c722cae2 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:58.100729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51faa7-8dd2-4ce4-8272-f59f3127a65f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:58.101400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51faa7-8dd2-4ce4-8272-f59f3127a65f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:58.101785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51faa7-8dd2-4ce4-8272-f59f3127a65f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:58.101948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51faa7-8dd2-4ce4-8272-f59f3127a65f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/auth/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:58.102543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f51faa7-8dd2-4ce4-8272-f59f3127a65f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:get_auth_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:58.103636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f51faa7-8dd2-4ce4-8272-f59f3127a65f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:58.110475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:58.111384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:58.112089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:58.112536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] PATH_INFO: `/v3/projects/93dac014c11c45029f5b8569b7f509d0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:58.113940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=93dac014c11c45029f5b8569b7f509d0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:58.116315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:58.180765 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2103/4160] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 21:08:58 2022] GET /identity/v3/auth/projects => generated 824 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:08:58.184884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51faa7-8dd2-4ce4-8272-f59f3127a65f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:58.197806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51faa7-8dd2-4ce4-8272-f59f3127a65f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:58.217311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93dac014c11c45029f5b8569b7f509d0', 'request_id': 'req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.231702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93dac014c11c45029f5b8569b7f509d0', 'request_id': 'req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.241591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93dac014c11c45029f5b8569b7f509d0', 'request_id': 'req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.256835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] The token cache is being invalidate because project 93dac014c11c45029f5b8569b7f509d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:58.257407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.258398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-82ff0191-96de-4e5c-9e22-f1bb43e1e6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.264584 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2058/4161] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:57 2022] DELETE /identity/v3/projects/93dac014c11c45029f5b8569b7f509d0 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:58.266644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c23a9e-9f1e-4373-b0c7-14bdfc9328ea None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:58.268599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c23a9e-9f1e-4373-b0c7-14bdfc9328ea None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:58.269085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c23a9e-9f1e-4373-b0c7-14bdfc9328ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:58.282214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:58.307356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-68c23a9e-9f1e-4373-b0c7-14bdfc9328ea None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:58.309860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-68c23a9e-9f1e-4373-b0c7-14bdfc9328ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:58.339403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:58.339403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:58.339403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:58.339927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/projects/c6fb088529854b909d10965a99ead2e5/users/74e48edf46d54884ad3bd49fec65e1c6/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:58.345299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=c6fb088529854b909d10965a99ead2e5, user_id=74e48edf46d54884ad3bd49fec65e1c6, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:58.345299 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:08:58.345299 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:08:58.345299 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:08:58.352700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:58.363220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invalidating the token cache because role fab66a1a6ba64c9583182c78978b218e was removed from user 74e48edf46d54884ad3bd49fec65e1c6 on project c6fb088529854b909d10965a99ead2e5. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:58.363656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.364806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.390601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:58.402229 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2104/4162] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Oct 15 21:08:58 2022] DELETE /identity/v3/projects/c6fb088529854b909d10965a99ead2e5/users/74e48edf46d54884ad3bd49fec65e1c6/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:58.404387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:58.407141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:58.495639 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2059/4163] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:58.500602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c23a9e-9f1e-4373-b0c7-14bdfc9328ea None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:58.505509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-68c23a9e-9f1e-4373-b0c7-14bdfc9328ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:58.539956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c23a9e-9f1e-4373-b0c7-14bdfc9328ea None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:58.611898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14fe72-0e15-45cc-b46d-e3d95ec26137 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:58.628488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:58.629218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:58.629531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:58.629802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] PATH_INFO: `/v3/projects/bd273f1984eb48ed960715c7bab4d8d8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:58.631196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd273f1984eb48ed960715c7bab4d8d8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:58.633011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:58.635536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b54f67-9520-4965-b4ca-20433708ca5c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:58.636375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b54f67-9520-4965-b4ca-20433708ca5c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:58.636757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b54f67-9520-4965-b4ca-20433708ca5c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:58.638337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b54f67-9520-4965-b4ca-20433708ca5c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:58.639101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b54f67-9520-4965-b4ca-20433708ca5c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:58.641339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b54f67-9520-4965-b4ca-20433708ca5c tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:58.674786 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2060/4164] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:58 2022] POST /identity/v3/users => generated 323 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:58.677302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ecb1f-c100-41fe-85cb-da0d9b85bf57 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:58.677771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ecb1f-c100-41fe-85cb-da0d9b85bf57 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:58.678283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ecb1f-c100-41fe-85cb-da0d9b85bf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:58.697986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd273f1984eb48ed960715c7bab4d8d8', 'request_id': 'req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.708438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c34ecb1f-c100-41fe-85cb-da0d9b85bf57 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:58.709998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c34ecb1f-c100-41fe-85cb-da0d9b85bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:58.721396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd273f1984eb48ed960715c7bab4d8d8', 'request_id': 'req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.741125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd273f1984eb48ed960715c7bab4d8d8', 'request_id': 'req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.761621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] The token cache is being invalidate because project bd273f1984eb48ed960715c7bab4d8d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:08:58.763676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.765153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8610677f-0d63-4ff5-a8f5-ac8efbe08ed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:58.767749 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2105/4165] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:58 2022] DELETE /identity/v3/projects/bd273f1984eb48ed960715c7bab4d8d8 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:58.769681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5873a-42d7-497a-a4b3-e149ae193cc5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:58.770432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5873a-42d7-497a-a4b3-e149ae193cc5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:58.770557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5873a-42d7-497a-a4b3-e149ae193cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:58.802482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-aad5873a-42d7-497a-a4b3-e149ae193cc5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:08:58.804293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aad5873a-42d7-497a-a4b3-e149ae193cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:58.812601 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2061/4166] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:08:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:58.814413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ecb1f-c100-41fe-85cb-da0d9b85bf57 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:58.816518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c34ecb1f-c100-41fe-85cb-da0d9b85bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:58.900120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:58.921420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ecb1f-c100-41fe-85cb-da0d9b85bf57 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:58.929362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146a069-02a5-471a-abbe-b0022364a922 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:58.930339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a069-02a5-471a-abbe-b0022364a922 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:58.930849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a069-02a5-471a-abbe-b0022364a922 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:58.931346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a069-02a5-471a-abbe-b0022364a922 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:58.932273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146a069-02a5-471a-abbe-b0022364a922 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:58.935837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146a069-02a5-471a-abbe-b0022364a922 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:58.939437 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2106/4167] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:08:58.940567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5873a-42d7-497a-a4b3-e149ae193cc5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:58.942693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aad5873a-42d7-497a-a4b3-e149ae193cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:58.970621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5873a-42d7-497a-a4b3-e149ae193cc5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:58.972838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:58.982846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:58.984146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:58.984790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:58.985407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] PATH_INFO: `/v3/users/41c8b53752a049eebfe107aa0f969203` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:58.986637 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2062/4168] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:58 2022] POST /identity/v3/projects => generated 363 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:58.987736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146a069-02a5-471a-abbe-b0022364a922 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:58.991982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7146a069-02a5-471a-abbe-b0022364a922 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:58.992980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c8b53752a049eebfe107aa0f969203)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:58.995253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:59.111051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c8b53752a049eebfe107aa0f969203'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:59.142165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c8b53752a049eebfe107aa0f969203'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:59.153888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c8b53752a049eebfe107aa0f969203'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:59.166789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c8b53752a049eebfe107aa0f969203'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:59.168840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c8b53752a049eebfe107aa0f969203'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:08:59.182933 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2107/4169] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:08:58 2022] DELETE /identity/v3/users/41c8b53752a049eebfe107aa0f969203 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:08:59.185082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:59.187990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:59.256290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146a069-02a5-471a-abbe-b0022364a922 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:59.367216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7969c4-8307-403f-8882-07475f54676a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:59.372820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2f0b2-03b0-4559-81cb-9a1a571aa408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:59.373493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2f0b2-03b0-4559-81cb-9a1a571aa408 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:59.373807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2f0b2-03b0-4559-81cb-9a1a571aa408 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:59.374024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2f0b2-03b0-4559-81cb-9a1a571aa408 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:59.374704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2f0b2-03b0-4559-81cb-9a1a571aa408 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:08:59.374967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da3bec-a928-4ffc-9946-8bc898b6d166 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:59.375091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2f0b2-03b0-4559-81cb-9a1a571aa408 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:59.377306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2f0b2-03b0-4559-81cb-9a1a571aa408 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:59.378347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da3bec-a928-4ffc-9946-8bc898b6d166 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:59.378641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da3bec-a928-4ffc-9946-8bc898b6d166 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:59.379765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da3bec-a928-4ffc-9946-8bc898b6d166 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:59.385514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a7da3bec-a928-4ffc-9946-8bc898b6d166 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:59.391600 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2108/4170] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:08:59.392442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2f0b2-03b0-4559-81cb-9a1a571aa408 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:59.394346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2bb2f0b2-03b0-4559-81cb-9a1a571aa408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:59.489098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da3bec-a928-4ffc-9946-8bc898b6d166 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:59.490555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2f0b2-03b0-4559-81cb-9a1a571aa408 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:59.491620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da3bec-a928-4ffc-9946-8bc898b6d166 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:59.496019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dfefd-7e5e-4a9e-ac72-b51088e56b19 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:59.497084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dfefd-7e5e-4a9e-ac72-b51088e56b19 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:59.497466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dfefd-7e5e-4a9e-ac72-b51088e56b19 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:59.498105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dfefd-7e5e-4a9e-ac72-b51088e56b19 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:59.498920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4dfefd-7e5e-4a9e-ac72-b51088e56b19 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:59.501308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4dfefd-7e5e-4a9e-ac72-b51088e56b19 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:59.507526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:59.510683 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2063/4171] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:08:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:59.514564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da3bec-a928-4ffc-9946-8bc898b6d166 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:59.578761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da3bec-a928-4ffc-9946-8bc898b6d166 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:59.587895 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2109/4172] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:59 2022] POST /identity/v3/projects => generated 364 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:59.592047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dfefd-7e5e-4a9e-ac72-b51088e56b19 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:59.602738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c1c60-d9c5-45fa-bc75-a386f444553d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:59.602892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c1c60-d9c5-45fa-bc75-a386f444553d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:59.603087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c1c60-d9c5-45fa-bc75-a386f444553d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:59.603417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c1c60-d9c5-45fa-bc75-a386f444553d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:59.604030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c1c60-d9c5-45fa-bc75-a386f444553d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:59.612857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c1c60-d9c5-45fa-bc75-a386f444553d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:59.679481 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2064/4173] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:08:59 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:59.684665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c1c60-d9c5-45fa-bc75-a386f444553d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:59.754239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:59.855274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dfefd-7e5e-4a9e-ac72-b51088e56b19 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:59.890758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c1c60-d9c5-45fa-bc75-a386f444553d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:59.896050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7743fc-3b97-4825-80ed-b27d4c79b127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:59.896771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7743fc-3b97-4825-80ed-b27d4c79b127 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:59.897263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7743fc-3b97-4825-80ed-b27d4c79b127 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:59.897570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7743fc-3b97-4825-80ed-b27d4c79b127 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:59.898213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7743fc-3b97-4825-80ed-b27d4c79b127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:59.900225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7743fc-3b97-4825-80ed-b27d4c79b127 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:59.938419 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2065/4174] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:08:59 2022] POST /identity/v3/users => generated 296 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:08:59.939612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7743fc-3b97-4825-80ed-b27d4c79b127 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:08:59.942717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4f7743fc-3b97-4825-80ed-b27d4c79b127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:08:59.967810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7743fc-3b97-4825-80ed-b27d4c79b127 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:08:59.973790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98a89d-542c-46f5-b7d8-fe152f7dd6e3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:59.975192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98a89d-542c-46f5-b7d8-fe152f7dd6e3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:59.975617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98a89d-542c-46f5-b7d8-fe152f7dd6e3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:59.976048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98a89d-542c-46f5-b7d8-fe152f7dd6e3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:59.976974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98a89d-542c-46f5-b7d8-fe152f7dd6e3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:59.978881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98a89d-542c-46f5-b7d8-fe152f7dd6e3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:08:59.985304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:08:59.988428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:08:59.988858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:08:59.989163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] PATH_INFO: `/v3/projects/f694762209cd4f8d89c405b3926c8d7d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:08:59.990681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f694762209cd4f8d89c405b3926c8d7d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:08:59.993620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:00.005632 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2110/4175] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:08:59 2022] POST /identity/v3/roles => generated 240 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:00.007211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98a89d-542c-46f5-b7d8-fe152f7dd6e3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:00.026919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98a89d-542c-46f5-b7d8-fe152f7dd6e3 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:00.038196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e4b50-3162-454f-b35b-d23a48461bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:00.039668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e4b50-3162-454f-b35b-d23a48461bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.040265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e4b50-3162-454f-b35b-d23a48461bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.040862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e4b50-3162-454f-b35b-d23a48461bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.041941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e4b50-3162-454f-b35b-d23a48461bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:00.044242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e4b50-3162-454f-b35b-d23a48461bf4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:00.058267 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2111/4176] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:00 2022] GET /identity/v3/roles => generated 1651 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:00.060418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e4b50-3162-454f-b35b-d23a48461bf4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:00.077763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e4b50-3162-454f-b35b-d23a48461bf4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:00.079983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f694762209cd4f8d89c405b3926c8d7d', 'request_id': 'req-22be0094-996a-4206-b5d0-31374077eb15'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:00.091203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f694762209cd4f8d89c405b3926c8d7d', 'request_id': 'req-22be0094-996a-4206-b5d0-31374077eb15'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:00.098936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f694762209cd4f8d89c405b3926c8d7d', 'request_id': 'req-22be0094-996a-4206-b5d0-31374077eb15'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:00.111971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] The token cache is being invalidate because project f694762209cd4f8d89c405b3926c8d7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:00.112267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:00.112948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:00.118333 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2066/4177] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:08:59 2022] DELETE /identity/v3/projects/f694762209cd4f8d89c405b3926c8d7d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:00.118333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:00.118333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:00.166024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455e487-1b92-405a-9d70-2091a0f9b52e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:00.167886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e487-1b92-405a-9d70-2091a0f9b52e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.169403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e487-1b92-405a-9d70-2091a0f9b52e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.169862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e487-1b92-405a-9d70-2091a0f9b52e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/projects/ccedecec882141958f02d83eba688636/users/ec5f6e34e59840cc88ba252e3f32e4f9/roles/7d290f5bff694b44b37abd5ff35d8f58` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.201113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455e487-1b92-405a-9d70-2091a0f9b52e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ccedecec882141958f02d83eba688636, user_id=ec5f6e34e59840cc88ba252e3f32e4f9, role_id=7d290f5bff694b44b37abd5ff35d8f58)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:00.204077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455e487-1b92-405a-9d70-2091a0f9b52e tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:00.219758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:00.224382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be0094-996a-4206-b5d0-31374077eb15 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:00.240272 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2112/4178] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:00 2022] PUT /identity/v3/projects/ccedecec882141958f02d83eba688636/users/ec5f6e34e59840cc88ba252e3f32e4f9/roles/7d290f5bff694b44b37abd5ff35d8f58 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:00.244410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d1ea8-d580-46c9-baa2-9f134615384f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.244751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d1ea8-d580-46c9-baa2-9f134615384f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.245023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d1ea8-d580-46c9-baa2-9f134615384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.282849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dd8d1ea8-d580-46c9-baa2-9f134615384f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:00.284898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dd8d1ea8-d580-46c9-baa2-9f134615384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:00.298689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:00.341337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2f4b5-d4f9-4dca-a7f9-24e13ec0042a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:00.342271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2f4b5-d4f9-4dca-a7f9-24e13ec0042a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.342726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2f4b5-d4f9-4dca-a7f9-24e13ec0042a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.343223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2f4b5-d4f9-4dca-a7f9-24e13ec0042a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.344039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2f4b5-d4f9-4dca-a7f9-24e13ec0042a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:00.345850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2f4b5-d4f9-4dca-a7f9-24e13ec0042a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:00.362483 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2067/4179] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:00 2022] GET /identity/v3/roles => generated 1651 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:00.362483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2f4b5-d4f9-4dca-a7f9-24e13ec0042a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:00.362483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-05c2f4b5-d4f9-4dca-a7f9-24e13ec0042a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:00.385691 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2113/4180] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:00.391195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d1ea8-d580-46c9-baa2-9f134615384f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:00.398666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d1ea8-d580-46c9-baa2-9f134615384f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:00.405804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057a82c-c2f7-46f0-8c4f-3a3eb888e3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:00.406566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057a82c-c2f7-46f0-8c4f-3a3eb888e3c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.406937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057a82c-c2f7-46f0-8c4f-3a3eb888e3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.407293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057a82c-c2f7-46f0-8c4f-3a3eb888e3c4 admin admin] PATH_INFO: `/v3/projects/e2b1936403344a4994393f2b098327a0/users/c48346e56c47481cab7f248b67369a27/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.424985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057a82c-c2f7-46f0-8c4f-3a3eb888e3c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b1936403344a4994393f2b098327a0, user_id=c48346e56c47481cab7f248b67369a27, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:00.427656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057a82c-c2f7-46f0-8c4f-3a3eb888e3c4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:00.447172 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2114/4181] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:00 2022] PUT /identity/v3/projects/e2b1936403344a4994393f2b098327a0/users/c48346e56c47481cab7f248b67369a27/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:00.448501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e489fe-ae12-482e-9064-568b65c6723a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.449017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e489fe-ae12-482e-9064-568b65c6723a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.449321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e489fe-ae12-482e-9064-568b65c6723a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.463693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2f4b5-d4f9-4dca-a7f9-24e13ec0042a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:00.472902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-02e489fe-ae12-482e-9064-568b65c6723a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:00.474419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-02e489fe-ae12-482e-9064-568b65c6723a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:00.528800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261a2e3-85a3-43dc-8993-d6287a513106 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:00.529510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261a2e3-85a3-43dc-8993-d6287a513106 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.529787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261a2e3-85a3-43dc-8993-d6287a513106 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.530062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261a2e3-85a3-43dc-8993-d6287a513106 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/projects/3c3449da326f41c79e8752d29be8ee29/users/ec5f6e34e59840cc88ba252e3f32e4f9/roles/7d290f5bff694b44b37abd5ff35d8f58` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.532302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261a2e3-85a3-43dc-8993-d6287a513106 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3c3449da326f41c79e8752d29be8ee29, user_id=ec5f6e34e59840cc88ba252e3f32e4f9, role_id=7d290f5bff694b44b37abd5ff35d8f58)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:00.535700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261a2e3-85a3-43dc-8993-d6287a513106 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:00.544456 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2115/4182] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:00.546134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f33ba-f5c0-4d29-a325-f2c53c8b19e6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.546703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f33ba-f5c0-4d29-a325-f2c53c8b19e6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.547104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f33ba-f5c0-4d29-a325-f2c53c8b19e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.552324 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2068/4183] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:00 2022] PUT /identity/v3/projects/3c3449da326f41c79e8752d29be8ee29/users/ec5f6e34e59840cc88ba252e3f32e4f9/roles/7d290f5bff694b44b37abd5ff35d8f58 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:00.553028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261a2e3-85a3-43dc-8993-d6287a513106 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:00.569592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-656f33ba-f5c0-4d29-a325-f2c53c8b19e6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:00.571001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-656f33ba-f5c0-4d29-a325-f2c53c8b19e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:00.631555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261a2e3-85a3-43dc-8993-d6287a513106 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:00.640796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b557f-bf86-4a22-bae4-fa90a9994a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:00.641657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b557f-bf86-4a22-bae4-fa90a9994a14 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.641961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b557f-bf86-4a22-bae4-fa90a9994a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.642359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b557f-bf86-4a22-bae4-fa90a9994a14 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.643008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b557f-bf86-4a22-bae4-fa90a9994a14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:00.643362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b557f-bf86-4a22-bae4-fa90a9994a14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:00.645305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b557f-bf86-4a22-bae4-fa90a9994a14 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:00.649006 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2116/4184] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:00.650119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2706c3-9a35-4918-827a-3cebd549c8b2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.652125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2706c3-9a35-4918-827a-3cebd549c8b2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.652413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2706c3-9a35-4918-827a-3cebd549c8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.660094 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2069/4185] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:00.661022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b557f-bf86-4a22-bae4-fa90a9994a14 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:00.663094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9c3b557f-bf86-4a22-bae4-fa90a9994a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:00.679153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:00.686218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6f2706c3-9a35-4918-827a-3cebd549c8b2 None None] MFA Rules not processed for user `ec5f6e34e59840cc88ba252e3f32e4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:00.687350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f2706c3-9a35-4918-827a-3cebd549c8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:00.692812 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2117/4186] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:00 2022] POST /identity/v3/auth/tokens => generated 335 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:09:00.696540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2706c3-9a35-4918-827a-3cebd549c8b2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:00.704176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:00.711708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2706c3-9a35-4918-827a-3cebd549c8b2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:00.719160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd646fe-45cf-4faa-ad60-03a6f3ed5a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:00.719841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd646fe-45cf-4faa-ad60-03a6f3ed5a18 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.720085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd646fe-45cf-4faa-ad60-03a6f3ed5a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.720354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd646fe-45cf-4faa-ad60-03a6f3ed5a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.720989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd646fe-45cf-4faa-ad60-03a6f3ed5a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:00.722995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd646fe-45cf-4faa-ad60-03a6f3ed5a18 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:00.736089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b557f-bf86-4a22-bae4-fa90a9994a14 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:00.740354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf40d74-3359-499c-8403-65a2c9feb089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:00.741028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf40d74-3359-499c-8403-65a2c9feb089 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.741400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf40d74-3359-499c-8403-65a2c9feb089 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.741738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf40d74-3359-499c-8403-65a2c9feb089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.757079 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2118/4187] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:00 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:00.757079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39679495-fa30-401f-99eb-93f2991ff05a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.757079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39679495-fa30-401f-99eb-93f2991ff05a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.757079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39679495-fa30-401f-99eb-93f2991ff05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.763499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf40d74-3359-499c-8403-65a2c9feb089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:00.765214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf40d74-3359-499c-8403-65a2c9feb089 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:00.776773 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2070/4188] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:00.778534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222b69-d56c-4ae7-8465-12aaa68be0b6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:00.778880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222b69-d56c-4ae7-8465-12aaa68be0b6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:00.779153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222b69-d56c-4ae7-8465-12aaa68be0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:00.780106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-39679495-fa30-401f-99eb-93f2991ff05a None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:00.782049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-39679495-fa30-401f-99eb-93f2991ff05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:00.791104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8f222b69-d56c-4ae7-8465-12aaa68be0b6 None None] MFA Rules not processed for user `ec5f6e34e59840cc88ba252e3f32e4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:00.792288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f222b69-d56c-4ae7-8465-12aaa68be0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:00.854555 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2119/4189] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:09:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:00.855232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39679495-fa30-401f-99eb-93f2991ff05a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:00.883151 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2071/4190] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:00 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Oct 15 21:09:00.883151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f222b69-d56c-4ae7-8465-12aaa68be0b6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:00.884923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f222b69-d56c-4ae7-8465-12aaa68be0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:00.993388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39679495-fa30-401f-99eb-93f2991ff05a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:00.999159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63facf-e9f8-4718-a9bc-5254002a383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:00.999923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63facf-e9f8-4718-a9bc-5254002a383f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.000201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63facf-e9f8-4718-a9bc-5254002a383f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.000471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63facf-e9f8-4718-a9bc-5254002a383f admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.001261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63facf-e9f8-4718-a9bc-5254002a383f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:01.004124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63facf-e9f8-4718-a9bc-5254002a383f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:01.028422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f222b69-d56c-4ae7-8465-12aaa68be0b6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:01.034576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb710c-0054-431e-abe9-68d5c18fa001 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:01.035144 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2120/4191] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:00 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:01.036412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb710c-0054-431e-abe9-68d5c18fa001 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.036700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb710c-0054-431e-abe9-68d5c18fa001 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.036968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb710c-0054-431e-abe9-68d5c18fa001 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.040063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63facf-e9f8-4718-a9bc-5254002a383f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:01.056321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb710c-0054-431e-abe9-68d5c18fa001 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:01.059056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb710c-0054-431e-abe9-68d5c18fa001 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:01.070658 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2072/4192] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:09:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:01.071697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb710c-0054-431e-abe9-68d5c18fa001 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:01.085477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb710c-0054-431e-abe9-68d5c18fa001 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:01.093424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf8e7c-c4cc-41b0-8e27-c248a7ad4b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:01.093989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8e7c-c4cc-41b0-8e27-c248a7ad4b16 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.094293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8e7c-c4cc-41b0-8e27-c248a7ad4b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.094588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8e7c-c4cc-41b0-8e27-c248a7ad4b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.095375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf8e7c-c4cc-41b0-8e27-c248a7ad4b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:01.097433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf8e7c-c4cc-41b0-8e27-c248a7ad4b16 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:01.110701 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2073/4193] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:01 2022] GET /identity/v3/roles => generated 1651 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:01.115227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf8e7c-c4cc-41b0-8e27-c248a7ad4b16 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:01.129208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf8e7c-c4cc-41b0-8e27-c248a7ad4b16 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:01.136319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90bc08-a0d0-4690-93df-b119ba43696b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:01.137736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90bc08-a0d0-4690-93df-b119ba43696b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.138367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90bc08-a0d0-4690-93df-b119ba43696b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.139044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90bc08-a0d0-4690-93df-b119ba43696b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.139610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90bc08-a0d0-4690-93df-b119ba43696b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:01.140622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63facf-e9f8-4718-a9bc-5254002a383f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:01.141561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90bc08-a0d0-4690-93df-b119ba43696b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:01.148273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e5539-c0e0-4a95-ad21-c29f65639572 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:01.149460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e5539-c0e0-4a95-ad21-c29f65639572 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.149750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e5539-c0e0-4a95-ad21-c29f65639572 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.150356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e5539-c0e0-4a95-ad21-c29f65639572 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.155894 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2074/4194] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:01 2022] GET /identity/v3/roles => generated 1651 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:09:01.162692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90bc08-a0d0-4690-93df-b119ba43696b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:01.177523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90bc08-a0d0-4690-93df-b119ba43696b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:01.178224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e5539-c0e0-4a95-ad21-c29f65639572 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:revoke_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:01.182998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675b898-6c4b-48e2-bb6b-72fd1e467584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:01.183288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e5539-c0e0-4a95-ad21-c29f65639572 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:01.184731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675b898-6c4b-48e2-bb6b-72fd1e467584 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.185085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675b898-6c4b-48e2-bb6b-72fd1e467584 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.185991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675b898-6c4b-48e2-bb6b-72fd1e467584 admin admin] PATH_INFO: `/v3/projects/077b5e8062594b64b54eb5304095fd93/users/10e29d80b807404cbf407436d84156d9/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.204654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675b898-6c4b-48e2-bb6b-72fd1e467584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=077b5e8062594b64b54eb5304095fd93, user_id=10e29d80b807404cbf407436d84156d9, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:01.207607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675b898-6c4b-48e2-bb6b-72fd1e467584 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:01.222718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:01.232746 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2075/4195] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:01 2022] PUT /identity/v3/projects/077b5e8062594b64b54eb5304095fd93/users/10e29d80b807404cbf407436d84156d9/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:01.245025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3480d35-7dfb-497e-938f-232ec4931c97 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.245478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3480d35-7dfb-497e-938f-232ec4931c97 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.247695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3480d35-7dfb-497e-938f-232ec4931c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.257631 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2121/4196] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Oct 15 21:09:01 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:01.262385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af34cd-33f7-4857-9954-80fdbfa9e7ad None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.263050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af34cd-33f7-4857-9954-80fdbfa9e7ad None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.263684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af34cd-33f7-4857-9954-80fdbfa9e7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.274848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:01.289258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-58af34cd-33f7-4857-9954-80fdbfa9e7ad None None] MFA Rules not processed for user `ec5f6e34e59840cc88ba252e3f32e4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:01.291598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-58af34cd-33f7-4857-9954-80fdbfa9e7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:01.305727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a3480d35-7dfb-497e-938f-232ec4931c97 None None] MFA Rules not processed for user `10e29d80b807404cbf407436d84156d9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:01.307060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a3480d35-7dfb-497e-938f-232ec4931c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:01.399125 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2122/4197] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Oct 15 21:09:01.403550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af34cd-33f7-4857-9954-80fdbfa9e7ad None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:01.409193 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2076/4198] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:09:01.415597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3480d35-7dfb-497e-938f-232ec4931c97 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:01.450829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3480d35-7dfb-497e-938f-232ec4931c97 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:01.467441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9342dda-be85-4b28-9e7b-84b857e01a8f tempest-DefaultDomainTestJSON-1140468521 tempest-DefaultDomainTestJSON-1140468521-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10e29d80b807404cbf407436d84156d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '077b5e8062594b64b54eb5304095fd93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:01.468541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9342dda-be85-4b28-9e7b-84b857e01a8f tempest-DefaultDomainTestJSON-1140468521 tempest-DefaultDomainTestJSON-1140468521-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.469006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9342dda-be85-4b28-9e7b-84b857e01a8f tempest-DefaultDomainTestJSON-1140468521 tempest-DefaultDomainTestJSON-1140468521-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.469645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9342dda-be85-4b28-9e7b-84b857e01a8f tempest-DefaultDomainTestJSON-1140468521 tempest-DefaultDomainTestJSON-1140468521-project] PATH_INFO: `/v3/domains/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.473472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9342dda-be85-4b28-9e7b-84b857e01a8f tempest-DefaultDomainTestJSON-1140468521 tempest-DefaultDomainTestJSON-1140468521-project] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:01.476880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9342dda-be85-4b28-9e7b-84b857e01a8f tempest-DefaultDomainTestJSON-1140468521 tempest-DefaultDomainTestJSON-1140468521-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:01.483610 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2077/4199] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 21:09:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:01.509227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9342dda-be85-4b28-9e7b-84b857e01a8f tempest-DefaultDomainTestJSON-1140468521 tempest-DefaultDomainTestJSON-1140468521-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:01.570856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af34cd-33f7-4857-9954-80fdbfa9e7ad None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:01.576612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:01.577306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.579317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.579452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/roles/7d290f5bff694b44b37abd5ff35d8f58` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.581466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_role(role_id=7d290f5bff694b44b37abd5ff35d8f58)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:01.583492 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:01.583492 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:09:01.583492 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:09:01.584704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:01.653389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invalidating the token cache because role 7d290f5bff694b44b37abd5ff35d8f58 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:01.654401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:01.655520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:01.658855 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2123/4200] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:01 2022] DELETE /identity/v3/roles/7d290f5bff694b44b37abd5ff35d8f58 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:01.664762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:01.667968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:01.708692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9342dda-be85-4b28-9e7b-84b857e01a8f tempest-DefaultDomainTestJSON-1140468521 tempest-DefaultDomainTestJSON-1140468521-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:01.792402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bdb48-3f32-4fa9-839d-0469ecf60941 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:01.798355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:01.799458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.799739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.800223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/projects/3c3449da326f41c79e8752d29be8ee29` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.801696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3c3449da326f41c79e8752d29be8ee29)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:01.803533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:01.821890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:01.822593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.823063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:01.823691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.823691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] PATH_INFO: `/v3/users/10e29d80b807404cbf407436d84156d9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:01.827527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e29d80b807404cbf407436d84156d9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:01.832321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:01.904670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c3449da326f41c79e8752d29be8ee29', 'request_id': 'req-19803a4b-e5d3-4ec5-abc2-596bff81798a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:01.929591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c3449da326f41c79e8752d29be8ee29', 'request_id': 'req-19803a4b-e5d3-4ec5-abc2-596bff81798a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:01.934966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:01.949896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c3449da326f41c79e8752d29be8ee29', 'request_id': 'req-19803a4b-e5d3-4ec5-abc2-596bff81798a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:01.969813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e29d80b807404cbf407436d84156d9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:01.976794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] The token cache is being invalidate because project 3c3449da326f41c79e8752d29be8ee29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:01.977244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:01.979579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-19803a4b-e5d3-4ec5-abc2-596bff81798a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:01.983106 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2124/4201] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:01 2022] DELETE /identity/v3/projects/3c3449da326f41c79e8752d29be8ee29 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:01.984797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874ecf0-5d96-4d59-b620-f897c1ce6654 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:01.985177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874ecf0-5d96-4d59-b620-f897c1ce6654 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:01.985516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874ecf0-5d96-4d59-b620-f897c1ce6654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:02.003915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e29d80b807404cbf407436d84156d9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.007712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e29d80b807404cbf407436d84156d9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.028942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7874ecf0-5d96-4d59-b620-f897c1ce6654 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:02.029509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e29d80b807404cbf407436d84156d9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.031716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7874ecf0-5d96-4d59-b620-f897c1ce6654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:02.031923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e29d80b807404cbf407436d84156d9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.046927 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2078/4202] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:01 2022] DELETE /identity/v3/users/10e29d80b807404cbf407436d84156d9 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:02.053130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:02.055171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:02.154972 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2125/4203] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:02.156817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52de8a1-e7ce-4927-acf7-37a0c9dea092 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:02.157853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52de8a1-e7ce-4927-acf7-37a0c9dea092 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:02.157853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52de8a1-e7ce-4927-acf7-37a0c9dea092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:02.204140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e52de8a1-e7ce-4927-acf7-37a0c9dea092 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:02.209710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e52de8a1-e7ce-4927-acf7-37a0c9dea092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:02.236220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266da982-7d1d-42eb-adcd-8af9957d3c90 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:02.245565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba08d8-d6be-457c-a5cf-d92faf077c90 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:02.247052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba08d8-d6be-457c-a5cf-d92faf077c90 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:02.248214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba08d8-d6be-457c-a5cf-d92faf077c90 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:02.248964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba08d8-d6be-457c-a5cf-d92faf077c90 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/projects/3c3449da326f41c79e8752d29be8ee29` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:02.259983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba08d8-d6be-457c-a5cf-d92faf077c90 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3c3449da326f41c79e8752d29be8ee29)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:02.263829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba08d8-d6be-457c-a5cf-d92faf077c90 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-09ba08d8-d6be-457c-a5cf-d92faf077c90 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Could not find project: 3c3449da326f41c79e8752d29be8ee29.: keystone.exception.ProjectNotFound: Could not find project: 3c3449da326f41c79e8752d29be8ee29. Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Oct 15 21:09:02.285092 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Oct 15 21:09:02.286102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:02.286102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:02.286102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Oct 15 21:09:02.286102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Oct 15 21:09:02.286102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:09:02.286102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:09:02.286102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:09:02.286102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:09:02.286102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 3c3449da326f41c79e8752d29be8ee29. Oct 15 21:09:02.286102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:02.287117 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2079/4204] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:02 2022] DELETE /identity/v3/projects/3c3449da326f41c79e8752d29be8ee29 => generated 113 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:02.288056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba08d8-d6be-457c-a5cf-d92faf077c90 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:02.292292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-09ba08d8-d6be-457c-a5cf-d92faf077c90 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:02.359924 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2126/4205] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:02.361173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52de8a1-e7ce-4927-acf7-37a0c9dea092 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:02.388560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52de8a1-e7ce-4927-acf7-37a0c9dea092 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:02.395204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:02.395401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:02.395621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:02.395853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/projects/ccedecec882141958f02d83eba688636` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:02.397232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ccedecec882141958f02d83eba688636)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:02.399371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:02.438441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba08d8-d6be-457c-a5cf-d92faf077c90 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:02.451929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dbeda-6823-42d7-a645-7eccdae3ec02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:02.452556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dbeda-6823-42d7-a645-7eccdae3ec02 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:02.452833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dbeda-6823-42d7-a645-7eccdae3ec02 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:02.454601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dbeda-6823-42d7-a645-7eccdae3ec02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:02.471182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:02.482276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccedecec882141958f02d83eba688636', 'request_id': 'req-5489f583-771d-495a-aeb3-cc3eb6004e4a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.494745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccedecec882141958f02d83eba688636', 'request_id': 'req-5489f583-771d-495a-aeb3-cc3eb6004e4a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.510253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccedecec882141958f02d83eba688636', 'request_id': 'req-5489f583-771d-495a-aeb3-cc3eb6004e4a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.526901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] The token cache is being invalidate because project ccedecec882141958f02d83eba688636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:02.527325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.528569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.530693 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2127/4206] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:02 2022] DELETE /identity/v3/projects/ccedecec882141958f02d83eba688636 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:02.531653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:02.533621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:02.548864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:02.572977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56dbeda-6823-42d7-a645-7eccdae3ec02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:02.575797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56dbeda-6823-42d7-a645-7eccdae3ec02 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:02.609571 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2080/4207] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:09:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:02.611189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dbeda-6823-42d7-a645-7eccdae3ec02 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:02.629561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dbeda-6823-42d7-a645-7eccdae3ec02 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:02.700793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489f583-771d-495a-aeb3-cc3eb6004e4a tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:02.711744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8a49c-355d-492e-acdb-ca66a8a4a727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:02.712319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8a49c-355d-492e-acdb-ca66a8a4a727 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:02.712595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8a49c-355d-492e-acdb-ca66a8a4a727 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:02.712856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8a49c-355d-492e-acdb-ca66a8a4a727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:02.718089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83e8a49c-355d-492e-acdb-ca66a8a4a727 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:02.803232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:02.804016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:02.804264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:02.804540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/users/ec5f6e34e59840cc88ba252e3f32e4f9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:02.809479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorizing `identity:delete_user(user_id=ec5f6e34e59840cc88ba252e3f32e4f9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:02.818371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:02.935327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5f6e34e59840cc88ba252e3f32e4f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.958265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5f6e34e59840cc88ba252e3f32e4f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.990537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5f6e34e59840cc88ba252e3f32e4f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:02.994562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8a49c-355d-492e-acdb-ca66a8a4a727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:02.998214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8a49c-355d-492e-acdb-ca66a8a4a727 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:03.011606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5f6e34e59840cc88ba252e3f32e4f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:03.012587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5f6e34e59840cc88ba252e3f32e4f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:03.021525 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2081/4208] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:02 2022] DELETE /identity/v3/users/ec5f6e34e59840cc88ba252e3f32e4f9 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:03.025316 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2128/4209] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:03.026017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:03.028323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:03.147905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71b3c6-c3df-4203-8088-cce1edfcb574 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:03.156012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50229669-00c6-4a87-b12b-11c0fb9cad2f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:03.156996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50229669-00c6-4a87-b12b-11c0fb9cad2f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:03.157993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50229669-00c6-4a87-b12b-11c0fb9cad2f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:03.158460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50229669-00c6-4a87-b12b-11c0fb9cad2f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/c83c4b95067b4383823c9745fc40e79f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:03.182360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50229669-00c6-4a87-b12b-11c0fb9cad2f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-50229669-00c6-4a87-b12b-11c0fb9cad2f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Could not find domain: c83c4b95067b4383823c9745fc40e79f.: keystone.exception.DomainNotFound: Could not find domain: c83c4b95067b4383823c9745fc40e79f. Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c83c4b95067b4383823c9745fc40e79f. Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:03.190689 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:03.191976 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: c83c4b95067b4383823c9745fc40e79f. Oct 15 21:09:03.193639 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:03.196756 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2082/4210] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:03 2022] PATCH /identity/v3/domains/c83c4b95067b4383823c9745fc40e79f => generated 112 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:03.201439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8a49c-355d-492e-acdb-ca66a8a4a727 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:03.203578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83e8a49c-355d-492e-acdb-ca66a8a4a727 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:03.223457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8a49c-355d-492e-acdb-ca66a8a4a727 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:03.231980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18188a7-0c7f-463c-b3ff-71a6531de280 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:03.232958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18188a7-0c7f-463c-b3ff-71a6531de280 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:03.233265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18188a7-0c7f-463c-b3ff-71a6531de280 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:03.233542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18188a7-0c7f-463c-b3ff-71a6531de280 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/d2518f06a2f54b25b22683720a4220ff` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:03.242673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18188a7-0c7f-463c-b3ff-71a6531de280 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-a18188a7-0c7f-463c-b3ff-71a6531de280 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Could not find domain: d2518f06a2f54b25b22683720a4220ff.: keystone.exception.DomainNotFound: Could not find domain: d2518f06a2f54b25b22683720a4220ff. Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d2518f06a2f54b25b22683720a4220ff. Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:03.250622 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:03.251696 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d2518f06a2f54b25b22683720a4220ff. Oct 15 21:09:03.252704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:03.256277 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2129/4211] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:03 2022] PATCH /identity/v3/domains/d2518f06a2f54b25b22683720a4220ff => generated 112 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:03.260163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50229669-00c6-4a87-b12b-11c0fb9cad2f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:03.268603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:03.279314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50229669-00c6-4a87-b12b-11c0fb9cad2f tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:03.288541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e026a8-76fe-46f8-aebc-6628405d30bf tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:03.289352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e026a8-76fe-46f8-aebc-6628405d30bf tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:03.290760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e026a8-76fe-46f8-aebc-6628405d30bf tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:03.290893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e026a8-76fe-46f8-aebc-6628405d30bf tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/d5ed0d337b2546e6a03f4c220ad1124d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:03.306221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e026a8-76fe-46f8-aebc-6628405d30bf tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-57e026a8-76fe-46f8-aebc-6628405d30bf tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Could not find domain: d5ed0d337b2546e6a03f4c220ad1124d.: keystone.exception.DomainNotFound: Could not find domain: d5ed0d337b2546e6a03f4c220ad1124d. Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d5ed0d337b2546e6a03f4c220ad1124d. Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:03.315017 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:03.316077 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d5ed0d337b2546e6a03f4c220ad1124d. Oct 15 21:09:03.316969 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:03.317950 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2083/4212] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:03 2022] PATCH /identity/v3/domains/d5ed0d337b2546e6a03f4c220ad1124d => generated 112 bytes in 57 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:03.320632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18188a7-0c7f-463c-b3ff-71a6531de280 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:03.335384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18188a7-0c7f-463c-b3ff-71a6531de280 tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:03.340939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f94e3-c745-4b3d-8dcd-bb9e063ef16d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74e48edf46d54884ad3bd49fec65e1c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54dcb34136f34cd59125b833205a50b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:03.341924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f94e3-c745-4b3d-8dcd-bb9e063ef16d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:03.342413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f94e3-c745-4b3d-8dcd-bb9e063ef16d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:03.342996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f94e3-c745-4b3d-8dcd-bb9e063ef16d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] PATH_INFO: `/v3/domains/ff50684481b948a2a34522a1182f4652` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:03.351692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f94e3-c745-4b3d-8dcd-bb9e063ef16d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-f27f94e3-c745-4b3d-8dcd-bb9e063ef16d tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Could not find domain: ff50684481b948a2a34522a1182f4652.: keystone.exception.DomainNotFound: Could not find domain: ff50684481b948a2a34522a1182f4652. Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: ff50684481b948a2a34522a1182f4652. Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:03.359108 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:03.360192 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: ff50684481b948a2a34522a1182f4652. Oct 15 21:09:03.361110 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:03.364821 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2130/4213] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:03 2022] PATCH /identity/v3/domains/ff50684481b948a2a34522a1182f4652 => generated 112 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:03.368588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e026a8-76fe-46f8-aebc-6628405d30bf tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:03.370465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-57e026a8-76fe-46f8-aebc-6628405d30bf tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:03.486333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e026a8-76fe-46f8-aebc-6628405d30bf tempest-TokensV3TestJSON-383503785 tempest-TokensV3TestJSON-383503785-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:03.492466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:03.493162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:03.493439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:03.493945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] PATH_INFO: `/v3/users/99e6e49193d1409fa1b552eb736ef8f9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:03.495140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e6e49193d1409fa1b552eb736ef8f9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:03.497917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:03.602317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e6e49193d1409fa1b552eb736ef8f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:03.610749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e6e49193d1409fa1b552eb736ef8f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:03.628251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e6e49193d1409fa1b552eb736ef8f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:03.643011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e6e49193d1409fa1b552eb736ef8f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:03.644081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e6e49193d1409fa1b552eb736ef8f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:03.660726 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2084/4214] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:03 2022] DELETE /identity/v3/users/99e6e49193d1409fa1b552eb736ef8f9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:03.662388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c95317-20f8-4507-b8df-36a790693ebf None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:03.662671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c95317-20f8-4507-b8df-36a790693ebf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:03.662937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c95317-20f8-4507-b8df-36a790693ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:03.710239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e9c95317-20f8-4507-b8df-36a790693ebf None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:03.713194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e9c95317-20f8-4507-b8df-36a790693ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:03.858151 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2131/4215] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:03.892623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:03.894112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:03.994227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324c69b-d17d-43b9-8f5b-614b471aefad admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:04.000785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7d50f-2a15-429f-a313-80da1e698201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:04.001710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7d50f-2a15-429f-a313-80da1e698201 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:04.001710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7d50f-2a15-429f-a313-80da1e698201 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:04.001973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7d50f-2a15-429f-a313-80da1e698201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:04.012951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7d50f-2a15-429f-a313-80da1e698201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:04.014547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7d50f-2a15-429f-a313-80da1e698201 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:04.025563 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2085/4216] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:09:04.361631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c95317-20f8-4507-b8df-36a790693ebf None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:04.363079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e9c95317-20f8-4507-b8df-36a790693ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:04.370857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:04.380165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c95317-20f8-4507-b8df-36a790693ebf None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:04.388823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:04.389603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:04.390682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:04.391041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] PATH_INFO: `/v3/projects/077b5e8062594b64b54eb5304095fd93` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:04.393326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=077b5e8062594b64b54eb5304095fd93)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:04.396031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:04.463712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '077b5e8062594b64b54eb5304095fd93', 'request_id': 'req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:04.478317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '077b5e8062594b64b54eb5304095fd93', 'request_id': 'req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:04.493744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '077b5e8062594b64b54eb5304095fd93', 'request_id': 'req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:04.512774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] The token cache is being invalidate because project 077b5e8062594b64b54eb5304095fd93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:04.513148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:04.513743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6324e9fd-32b6-4ef0-aac6-dec5b7a4deaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:04.518829 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2132/4217] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:04 2022] DELETE /identity/v3/projects/077b5e8062594b64b54eb5304095fd93 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:04.520639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97e8c9-46ec-4a70-96f4-e7d7b0825856 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:04.521575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97e8c9-46ec-4a70-96f4-e7d7b0825856 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:04.521996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97e8c9-46ec-4a70-96f4-e7d7b0825856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:04.545965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4c97e8c9-46ec-4a70-96f4-e7d7b0825856 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:04.547393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4c97e8c9-46ec-4a70-96f4-e7d7b0825856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:04.565676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:04.650684 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2086/4218] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:04.653861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c607ce-6be9-4e3e-97a0-ae5365926e8c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:04.654144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c607ce-6be9-4e3e-97a0-ae5365926e8c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:04.654413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c607ce-6be9-4e3e-97a0-ae5365926e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:04.687214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-79c607ce-6be9-4e3e-97a0-ae5365926e8c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:04.690652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-79c607ce-6be9-4e3e-97a0-ae5365926e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:04.698626 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2133/4219] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:09:04.703253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97e8c9-46ec-4a70-96f4-e7d7b0825856 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:04.715824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97e8c9-46ec-4a70-96f4-e7d7b0825856 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:04.720624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffafd17-897e-4afc-8b86-a3812b0e870b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:04.721003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffafd17-897e-4afc-8b86-a3812b0e870b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:04.721450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffafd17-897e-4afc-8b86-a3812b0e870b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:04.721910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffafd17-897e-4afc-8b86-a3812b0e870b admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:04.722769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffafd17-897e-4afc-8b86-a3812b0e870b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:04.723070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffafd17-897e-4afc-8b86-a3812b0e870b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:04.724744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffafd17-897e-4afc-8b86-a3812b0e870b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:04.742408 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2087/4220] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:09:04.746427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c607ce-6be9-4e3e-97a0-ae5365926e8c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:04.775741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c607ce-6be9-4e3e-97a0-ae5365926e8c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:04.791932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679bf78-9ec5-43b2-ba11-755065e83333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:04.791932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679bf78-9ec5-43b2-ba11-755065e83333 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:04.791932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679bf78-9ec5-43b2-ba11-755065e83333 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:04.791932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679bf78-9ec5-43b2-ba11-755065e83333 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:04.792305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c679bf78-9ec5-43b2-ba11-755065e83333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:04.795088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c679bf78-9ec5-43b2-ba11-755065e83333 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:04.887617 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2134/4221] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:04 2022] POST /identity/v3/projects => generated 361 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:04.891391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffafd17-897e-4afc-8b86-a3812b0e870b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:04.923771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffafd17-897e-4afc-8b86-a3812b0e870b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:05.081765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d70050-e542-4bc6-a571-f9235609a6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:05.083447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d70050-e542-4bc6-a571-f9235609a6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:05.083735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d70050-e542-4bc6-a571-f9235609a6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:05.084052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d70050-e542-4bc6-a571-f9235609a6f6 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:05.085378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d70050-e542-4bc6-a571-f9235609a6f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:05.094022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d70050-e542-4bc6-a571-f9235609a6f6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:05.117917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679bf78-9ec5-43b2-ba11-755065e83333 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:05.120289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c679bf78-9ec5-43b2-ba11-755065e83333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:05.158842 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2088/4222] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:04 2022] POST /identity/v3/users => generated 281 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:05.163426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d70050-e542-4bc6-a571-f9235609a6f6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:05.166975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679bf78-9ec5-43b2-ba11-755065e83333 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:05.193780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:05.194491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:05.194820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:05.195099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] PATH_INFO: `/v3/projects/c6fb088529854b909d10965a99ead2e5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:05.202340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6fb088529854b909d10965a99ead2e5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:05.202496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:05.209547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d70050-e542-4bc6-a571-f9235609a6f6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:05.244028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b92687-2113-4b84-b9d5-17b85fa19ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:05.244906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b92687-2113-4b84-b9d5-17b85fa19ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:05.245236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b92687-2113-4b84-b9d5-17b85fa19ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:05.245505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b92687-2113-4b84-b9d5-17b85fa19ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:05.246686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b92687-2113-4b84-b9d5-17b85fa19ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:05.248523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b92687-2113-4b84-b9d5-17b85fa19ba5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:05.269114 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2089/4223] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:05.270033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b92687-2113-4b84-b9d5-17b85fa19ba5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:05.307259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b92687-2113-4b84-b9d5-17b85fa19ba5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:05.322703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682091f3-3a9f-4431-a98e-374cb61bc076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:05.325073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682091f3-3a9f-4431-a98e-374cb61bc076 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:05.325621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682091f3-3a9f-4431-a98e-374cb61bc076 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:05.325621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682091f3-3a9f-4431-a98e-374cb61bc076 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:05.326239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682091f3-3a9f-4431-a98e-374cb61bc076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:05.331774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682091f3-3a9f-4431-a98e-374cb61bc076 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:05.358939 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2090/4224] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:05.363131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682091f3-3a9f-4431-a98e-374cb61bc076 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:05.380866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682091f3-3a9f-4431-a98e-374cb61bc076 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:05.388098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6fb088529854b909d10965a99ead2e5', 'request_id': 'req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:05.405390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6fb088529854b909d10965a99ead2e5', 'request_id': 'req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:05.418446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6fb088529854b909d10965a99ead2e5', 'request_id': 'req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:05.459198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] The token cache is being invalidate because project c6fb088529854b909d10965a99ead2e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:05.459531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:05.460238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:05.462237 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2135/4225] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:05 2022] DELETE /identity/v3/projects/c6fb088529854b909d10965a99ead2e5 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:05.466102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:05.467722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:05.553524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081aafe-b39b-4a0c-a782-98c6fd0eba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:05.554618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081aafe-b39b-4a0c-a782-98c6fd0eba71 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:05.555034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081aafe-b39b-4a0c-a782-98c6fd0eba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:05.555434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081aafe-b39b-4a0c-a782-98c6fd0eba71 admin admin] PATH_INFO: `/v3/projects/511a967fbd29437097e5dee333295f42/users/7208bb0d6d5c4922bbbbd0733b2d73f9/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:05.614724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081aafe-b39b-4a0c-a782-98c6fd0eba71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=511a967fbd29437097e5dee333295f42, user_id=7208bb0d6d5c4922bbbbd0733b2d73f9, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:05.621289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081aafe-b39b-4a0c-a782-98c6fd0eba71 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:05.624446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:05.634483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:05.689399 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2091/4226] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:05 2022] PUT /identity/v3/projects/511a967fbd29437097e5dee333295f42/users/7208bb0d6d5c4922bbbbd0733b2d73f9/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:05.691229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a88dc-856b-45f9-a41e-a1d4e5dac882 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:05.701417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ea520-faa9-4c4b-a037-cdb2ffd30944 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:05.701724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ea520-faa9-4c4b-a037-cdb2ffd30944 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:05.702074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ea520-faa9-4c4b-a037-cdb2ffd30944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:05.752760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-824ea520-faa9-4c4b-a037-cdb2ffd30944 None None] MFA Rules not processed for user `7208bb0d6d5c4922bbbbd0733b2d73f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:05.760541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-824ea520-faa9-4c4b-a037-cdb2ffd30944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:05.876064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:05.876715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:05.876988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:05.877291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] PATH_INFO: `/v3/users/74e48edf46d54884ad3bd49fec65e1c6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:05.880517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e48edf46d54884ad3bd49fec65e1c6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:05.884286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:05.941368 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2092/4227] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:05.946374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977360ee-b63a-41e4-b939-965914cab262 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:05.946879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977360ee-b63a-41e4-b939-965914cab262 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:05.947214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977360ee-b63a-41e4-b939-965914cab262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:05.986703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-977360ee-b63a-41e4-b939-965914cab262 None None] MFA Rules not processed for user `7208bb0d6d5c4922bbbbd0733b2d73f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:05.988758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-977360ee-b63a-41e4-b939-965914cab262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:05.998364 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2093/4228] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:05 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:09:06.002714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e48edf46d54884ad3bd49fec65e1c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:06.006316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4482-1cb6-42b6-9d3b-729397b64466 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:06.007637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4482-1cb6-42b6-9d3b-729397b64466 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:06.008170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4482-1cb6-42b6-9d3b-729397b64466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:06.029904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e48edf46d54884ad3bd49fec65e1c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:06.047401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0a2d4482-1cb6-42b6-9d3b-729397b64466 None None] MFA Rules not processed for user `7208bb0d6d5c4922bbbbd0733b2d73f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:06.048674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0a2d4482-1cb6-42b6-9d3b-729397b64466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:06.050954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e48edf46d54884ad3bd49fec65e1c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:06.059497 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2094/4229] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:06 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:09:06.060410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d4482-1cb6-42b6-9d3b-729397b64466 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:06.062978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e48edf46d54884ad3bd49fec65e1c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:06.066441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e48edf46d54884ad3bd49fec65e1c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:06.076428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d4482-1cb6-42b6-9d3b-729397b64466 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:06.078486 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2136/4230] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:05 2022] DELETE /identity/v3/users/74e48edf46d54884ad3bd49fec65e1c6 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:06.156641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235b3bb-feaa-4ed2-8290-22516a473519 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7208bb0d6d5c4922bbbbd0733b2d73f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a967fbd29437097e5dee333295f42', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:06.157521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235b3bb-feaa-4ed2-8290-22516a473519 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:06.157868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235b3bb-feaa-4ed2-8290-22516a473519 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:06.158283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235b3bb-feaa-4ed2-8290-22516a473519 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:06.161457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5235b3bb-feaa-4ed2-8290-22516a473519 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:06.173071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235b3bb-feaa-4ed2-8290-22516a473519 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: Authorizing `identity:check_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:06.175109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235b3bb-feaa-4ed2-8290-22516a473519 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:06.185718 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2095/4231] 10.222.0.16 () {56 vars in 1264 bytes} [Sat Oct 15 21:09:06 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 358 bytes (0 switches on core 0) Oct 15 21:09:06.189551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:06.191748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:06.207930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607cf5f-455d-422b-b7d4-957d76103d7c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:06.216875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c6595-524d-4de3-913e-eb3b2ed81d7f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7208bb0d6d5c4922bbbbd0733b2d73f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a967fbd29437097e5dee333295f42', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:06.217624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c6595-524d-4de3-913e-eb3b2ed81d7f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:06.217958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c6595-524d-4de3-913e-eb3b2ed81d7f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:06.218256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c6595-524d-4de3-913e-eb3b2ed81d7f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:06.227395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c6595-524d-4de3-913e-eb3b2ed81d7f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:06.231823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c6595-524d-4de3-913e-eb3b2ed81d7f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:06.243030 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2137/4232] 10.222.0.16 () {56 vars in 1263 bytes} [Sat Oct 15 21:09:06 2022] GET /identity/v3/auth/tokens => generated 346 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:06.247159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235b3bb-feaa-4ed2-8290-22516a473519 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:06.256553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235b3bb-feaa-4ed2-8290-22516a473519 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:06.262102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3fceb-2c88-4b1a-97c9-86e083453490 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7208bb0d6d5c4922bbbbd0733b2d73f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a967fbd29437097e5dee333295f42', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:06.262740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3fceb-2c88-4b1a-97c9-86e083453490 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:06.263010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3fceb-2c88-4b1a-97c9-86e083453490 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:06.263270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3fceb-2c88-4b1a-97c9-86e083453490 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:06.271722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3fceb-2c88-4b1a-97c9-86e083453490 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:06.273388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3fceb-2c88-4b1a-97c9-86e083453490 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:06.289695 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2096/4233] 10.222.0.16 () {56 vars in 1266 bytes} [Sat Oct 15 21:09:06 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:06.294302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c6595-524d-4de3-913e-eb3b2ed81d7f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:06.309139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c6595-524d-4de3-913e-eb3b2ed81d7f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:06.315290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bfca3-2b79-469c-b622-efeb9bd4440f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7208bb0d6d5c4922bbbbd0733b2d73f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a967fbd29437097e5dee333295f42', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:06.316083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bfca3-2b79-469c-b622-efeb9bd4440f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:06.316616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bfca3-2b79-469c-b622-efeb9bd4440f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:06.317107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bfca3-2b79-469c-b622-efeb9bd4440f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:06.321006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-df4bfca3-2b79-469c-b622-efeb9bd4440f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-df4bfca3-2b79-469c-b622-efeb9bd4440f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 283, in get Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:check_token') Oct 15 21:09:06.385229 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.check_revocation(token) Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Oct 15 21:09:06.386466 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Oct 15 21:09:06.388896 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:06.388896 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:06.388896 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Oct 15 21:09:06.388896 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Oct 15 21:09:06.388896 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Oct 15 21:09:06.388896 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:06.400587 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2138/4234] 10.222.0.16 () {56 vars in 1264 bytes} [Sat Oct 15 21:09:06 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 103 msecs (HTTP/1.1 404) 5 headers in 183 bytes (0 switches on core 0) Oct 15 21:09:06.402127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f768d-eef0-4b61-bc58-5605496e88ce None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:06.403091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f768d-eef0-4b61-bc58-5605496e88ce None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:06.403433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f768d-eef0-4b61-bc58-5605496e88ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:06.457992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4b0f768d-eef0-4b61-bc58-5605496e88ce None None] MFA Rules not processed for user `7208bb0d6d5c4922bbbbd0733b2d73f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:06.460048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4b0f768d-eef0-4b61-bc58-5605496e88ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:06.464127 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2097/4235] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:06 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 21:09:06.468718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bfca3-2b79-469c-b622-efeb9bd4440f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:06.484897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bfca3-2b79-469c-b622-efeb9bd4440f tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:06.493028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2bdd7-8e51-4040-bddb-465a2e23b026 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7208bb0d6d5c4922bbbbd0733b2d73f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a967fbd29437097e5dee333295f42', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:06.494099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2bdd7-8e51-4040-bddb-465a2e23b026 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:06.496241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2bdd7-8e51-4040-bddb-465a2e23b026 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:06.497009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2bdd7-8e51-4040-bddb-465a2e23b026 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:06.510627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2bdd7-8e51-4040-bddb-465a2e23b026 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:06.534343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2bdd7-8e51-4040-bddb-465a2e23b026 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:06.548351 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2139/4236] 10.222.0.16 () {56 vars in 1263 bytes} [Sat Oct 15 21:09:06 2022] GET /identity/v3/auth/tokens => generated 346 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:06.551591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f768d-eef0-4b61-bc58-5605496e88ce None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:06.563914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f768d-eef0-4b61-bc58-5605496e88ce None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:06.568778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e849e8c-4c7e-4c20-8048-fc08913d8747 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7208bb0d6d5c4922bbbbd0733b2d73f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a967fbd29437097e5dee333295f42', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:06.569617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e849e8c-4c7e-4c20-8048-fc08913d8747 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:06.570207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e849e8c-4c7e-4c20-8048-fc08913d8747 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:06.570618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e849e8c-4c7e-4c20-8048-fc08913d8747 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:06.581535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e849e8c-4c7e-4c20-8048-fc08913d8747 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:06.583982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e849e8c-4c7e-4c20-8048-fc08913d8747 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:06.609494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2bdd7-8e51-4040-bddb-465a2e23b026 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:06.610070 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2098/4237] 10.222.0.16 () {56 vars in 1266 bytes} [Sat Oct 15 21:09:06 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:06.621873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2bdd7-8e51-4040-bddb-465a2e23b026 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:06.632179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc57a7f-bb5a-41e6-b544-9b3780ba3b7c tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7208bb0d6d5c4922bbbbd0733b2d73f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a967fbd29437097e5dee333295f42', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:06.633290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57a7f-bb5a-41e6-b544-9b3780ba3b7c tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:06.633290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57a7f-bb5a-41e6-b544-9b3780ba3b7c tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:06.633846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57a7f-bb5a-41e6-b544-9b3780ba3b7c tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:06.635927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-adc57a7f-bb5a-41e6-b544-9b3780ba3b7c tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-adc57a7f-bb5a-41e6-b544-9b3780ba3b7c tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 15 21:09:06.650934 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.check_revocation(token) Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Oct 15 21:09:06.652127 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Oct 15 21:09:06.654069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:06.654069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:06.654069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Oct 15 21:09:06.654069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Oct 15 21:09:06.654069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Oct 15 21:09:06.654069 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:06.659958 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2140/4238] 10.222.0.16 () {56 vars in 1263 bytes} [Sat Oct 15 21:09:06 2022] GET /identity/v3/auth/tokens => generated 80 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 21:09:06.661984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e849e8c-4c7e-4c20-8048-fc08913d8747 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:06.663955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5e849e8c-4c7e-4c20-8048-fc08913d8747 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:06.712910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:06.807632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e849e8c-4c7e-4c20-8048-fc08913d8747 tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:06.813556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:06.814521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:06.814803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:06.815224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] PATH_INFO: `/v3/users/7208bb0d6d5c4922bbbbd0733b2d73f9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:06.816596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7208bb0d6d5c4922bbbbd0733b2d73f9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:06.820805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:06.900420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc57a7f-bb5a-41e6-b544-9b3780ba3b7c tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:06.904913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-adc57a7f-bb5a-41e6-b544-9b3780ba3b7c tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:06.917237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7208bb0d6d5c4922bbbbd0733b2d73f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:06.959558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7208bb0d6d5c4922bbbbd0733b2d73f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:06.994188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:07.005823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7208bb0d6d5c4922bbbbd0733b2d73f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.023230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7208bb0d6d5c4922bbbbd0733b2d73f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.024232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-196aefde-3b32-4078-b066-f79a01bc6cbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7208bb0d6d5c4922bbbbd0733b2d73f9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.047905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc57a7f-bb5a-41e6-b544-9b3780ba3b7c tempest-TokensV3Test-2133153237 tempest-TokensV3Test-2133153237-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:07.048557 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2099/4239] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:06 2022] DELETE /identity/v3/users/7208bb0d6d5c4922bbbbd0733b2d73f9 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:07.051910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d732c9-3b05-43d9-918a-ff8731d8d325 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:07.052195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d732c9-3b05-43d9-918a-ff8731d8d325 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:07.052470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d732c9-3b05-43d9-918a-ff8731d8d325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:07.084690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-29d732c9-3b05-43d9-918a-ff8731d8d325 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:07.086495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-29d732c9-3b05-43d9-918a-ff8731d8d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:07.161161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796bdc3-be56-4a8d-bb7a-3b5eede0bf09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:07.162308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796bdc3-be56-4a8d-bb7a-3b5eede0bf09 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:07.162607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796bdc3-be56-4a8d-bb7a-3b5eede0bf09 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:07.162879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796bdc3-be56-4a8d-bb7a-3b5eede0bf09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:07.167325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7796bdc3-be56-4a8d-bb7a-3b5eede0bf09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:07.198060 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2100/4240] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:07.198060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d732c9-3b05-43d9-918a-ff8731d8d325 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:07.198920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-29d732c9-3b05-43d9-918a-ff8731d8d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:07.239991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d732c9-3b05-43d9-918a-ff8731d8d325 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:07.248407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:07.249569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:07.249969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:07.250348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] PATH_INFO: `/v3/projects/54dcb34136f34cd59125b833205a50b5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:07.251839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] RBAC: Authorizing `identity:delete_project(project_id=54dcb34136f34cd59125b833205a50b5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:07.255704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:07.345439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796bdc3-be56-4a8d-bb7a-3b5eede0bf09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:07.350744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796bdc3-be56-4a8d-bb7a-3b5eede0bf09 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:07.360905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:07.392995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54dcb34136f34cd59125b833205a50b5', 'request_id': 'req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.397253 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2141/4241] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:07.398329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796bdc3-be56-4a8d-bb7a-3b5eede0bf09 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:07.402123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7796bdc3-be56-4a8d-bb7a-3b5eede0bf09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:07.414964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54dcb34136f34cd59125b833205a50b5', 'request_id': 'req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.423954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54dcb34136f34cd59125b833205a50b5', 'request_id': 'req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.434338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796bdc3-be56-4a8d-bb7a-3b5eede0bf09 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:07.452973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] The token cache is being invalidate because project 54dcb34136f34cd59125b833205a50b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:07.455973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.457481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.464092 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2101/4242] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:07 2022] DELETE /identity/v3/projects/54dcb34136f34cd59125b833205a50b5 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:07.466577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:07.470468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:07.569555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be400b-8b4a-4193-99a2-2032ba5704e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:07.573857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be400b-8b4a-4193-99a2-2032ba5704e2 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:07.574366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be400b-8b4a-4193-99a2-2032ba5704e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:07.574706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be400b-8b4a-4193-99a2-2032ba5704e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:07.582065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d7be400b-8b4a-4193-99a2-2032ba5704e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:07.654718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd98ce6-4e75-49e1-802e-9c4f6eaafdae admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:07.661904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:07.663165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:07.663599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:07.664054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] PATH_INFO: `/v3/users/f4004fb706684648a08e7a131d5b16a0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:07.665941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4004fb706684648a08e7a131d5b16a0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:07.668493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:07.698597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:07.731273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be400b-8b4a-4193-99a2-2032ba5704e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:07.733221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be400b-8b4a-4193-99a2-2032ba5704e2 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:07.747224 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2142/4243] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:07.780780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4004fb706684648a08e7a131d5b16a0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.796181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4004fb706684648a08e7a131d5b16a0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.816411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4004fb706684648a08e7a131d5b16a0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.829629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4004fb706684648a08e7a131d5b16a0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.832231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4004fb706684648a08e7a131d5b16a0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:07.849109 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2102/4244] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:07 2022] DELETE /identity/v3/users/f4004fb706684648a08e7a131d5b16a0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:08.477902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cd04a-03f8-4e89-9f5a-a955eb3b0461 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:08.478493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cd04a-03f8-4e89-9f5a-a955eb3b0461 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:08.478722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cd04a-03f8-4e89-9f5a-a955eb3b0461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:08.584690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-958cd04a-03f8-4e89-9f5a-a955eb3b0461 None None] MFA Rules not processed for user `c48346e56c47481cab7f248b67369a27`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:08.588884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-958cd04a-03f8-4e89-9f5a-a955eb3b0461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:08.634996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:08.638492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:08.729143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:08.752977 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2143/4245] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:08.754167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cd04a-03f8-4e89-9f5a-a955eb3b0461 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:08.755840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-958cd04a-03f8-4e89-9f5a-a955eb3b0461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:08.771415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c8ff5-e383-46eb-962f-a9de6264b001 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:08.779340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:08.779987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:08.780365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:08.780646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] PATH_INFO: `/v3/projects/642622a2b3404e1095d1c89ba0da1f36` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:08.782301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=642622a2b3404e1095d1c89ba0da1f36)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:08.782517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cd04a-03f8-4e89-9f5a-a955eb3b0461 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:08.787476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:08.791348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:08.792075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:08.792354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:08.792620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] PATH_INFO: `/v3/projects/511a967fbd29437097e5dee333295f42` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:08.797868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=511a967fbd29437097e5dee333295f42)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:08.800242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:08.872417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642622a2b3404e1095d1c89ba0da1f36', 'request_id': 'req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:08.883984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642622a2b3404e1095d1c89ba0da1f36', 'request_id': 'req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:08.899796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642622a2b3404e1095d1c89ba0da1f36', 'request_id': 'req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:08.913607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '511a967fbd29437097e5dee333295f42', 'request_id': 'req-85673e1e-d749-44d9-86ca-0edd23948417'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:08.916780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] The token cache is being invalidate because project 642622a2b3404e1095d1c89ba0da1f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:08.917235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:08.918273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d4e3ca2-8d03-49ee-bbf1-1ebd2a7f8aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:08.921628 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2103/4246] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:08 2022] DELETE /identity/v3/projects/642622a2b3404e1095d1c89ba0da1f36 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:08.923353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab54147-66e2-4a2e-9cdd-ac20b3de931c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:08.923735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab54147-66e2-4a2e-9cdd-ac20b3de931c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:08.924105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab54147-66e2-4a2e-9cdd-ac20b3de931c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:08.927565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '511a967fbd29437097e5dee333295f42', 'request_id': 'req-85673e1e-d749-44d9-86ca-0edd23948417'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:08.938767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '511a967fbd29437097e5dee333295f42', 'request_id': 'req-85673e1e-d749-44d9-86ca-0edd23948417'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:08.957058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] The token cache is being invalidate because project 511a967fbd29437097e5dee333295f42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:08.957652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:08.959102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-85673e1e-d749-44d9-86ca-0edd23948417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:08.961491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2144/4247] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:08 2022] DELETE /identity/v3/projects/511a967fbd29437097e5dee333295f42 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:08.963162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8df71-0841-465c-bbc1-e9a842d2c85f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:08.963606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8df71-0841-465c-bbc1-e9a842d2c85f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:08.964033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8df71-0841-465c-bbc1-e9a842d2c85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:08.967341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4ab54147-66e2-4a2e-9cdd-ac20b3de931c None None] MFA Rules not processed for user `c48346e56c47481cab7f248b67369a27`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:08.968577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4ab54147-66e2-4a2e-9cdd-ac20b3de931c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:09.021161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f0d8df71-0841-465c-bbc1-e9a842d2c85f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:09.022969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f0d8df71-0841-465c-bbc1-e9a842d2c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:09.068819 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2104/4248] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:09.070328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3faddd-8216-403d-b5e8-a57591fa4546 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.070613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3faddd-8216-403d-b5e8-a57591fa4546 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.071291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3faddd-8216-403d-b5e8-a57591fa4546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.101111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7f3faddd-8216-403d-b5e8-a57591fa4546 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:09.101327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7f3faddd-8216-403d-b5e8-a57591fa4546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:09.139409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:09.161850 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2145/4249] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:09.170108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b597b7-8c95-4432-8d73-58b5b0398291 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.170108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b597b7-8c95-4432-8d73-58b5b0398291 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.170675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b597b7-8c95-4432-8d73-58b5b0398291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.225962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-54b597b7-8c95-4432-8d73-58b5b0398291 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:09.227628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-54b597b7-8c95-4432-8d73-58b5b0398291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:09.236662 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2105/4250] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:09.238848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e05fed-be88-4ee0-90df-bbcf5d23bd8c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.239640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e05fed-be88-4ee0-90df-bbcf5d23bd8c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.239916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e05fed-be88-4ee0-90df-bbcf5d23bd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.244766 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2146/4251] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:09.250644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b597b7-8c95-4432-8d73-58b5b0398291 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.264521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b597b7-8c95-4432-8d73-58b5b0398291 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:09.271567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d7135-3012-4d56-ba95-843e4c3f5c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:09.272308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d7135-3012-4d56-ba95-843e4c3f5c2c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.272824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d7135-3012-4d56-ba95-843e4c3f5c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.273203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d7135-3012-4d56-ba95-843e4c3f5c2c admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.273960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d7135-3012-4d56-ba95-843e4c3f5c2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:09.274419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d7135-3012-4d56-ba95-843e4c3f5c2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:09.276567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d7135-3012-4d56-ba95-843e4c3f5c2c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:09.294139 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2147/4252] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:09:09.298631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d7135-3012-4d56-ba95-843e4c3f5c2c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.319699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d7135-3012-4d56-ba95-843e4c3f5c2c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:09.328927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4dd44-4540-4daf-85d0-9dedbf4df7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:09.329592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4dd44-4540-4daf-85d0-9dedbf4df7b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.329871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4dd44-4540-4daf-85d0-9dedbf4df7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.330138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4dd44-4540-4daf-85d0-9dedbf4df7b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.330934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4dd44-4540-4daf-85d0-9dedbf4df7b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:09.334410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4dd44-4540-4daf-85d0-9dedbf4df7b8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:09.371800 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2148/4253] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:09 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:09.376098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4dd44-4540-4daf-85d0-9dedbf4df7b8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.432560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-97e05fed-be88-4ee0-90df-bbcf5d23bd8c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:09.434769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-97e05fed-be88-4ee0-90df-bbcf5d23bd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:09.453819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:09.468977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4dd44-4540-4daf-85d0-9dedbf4df7b8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:09.474958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9a07c-4511-473c-ac4d-86f51fe3d909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:09.475790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9a07c-4511-473c-ac4d-86f51fe3d909 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.476047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9a07c-4511-473c-ac4d-86f51fe3d909 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.476349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9a07c-4511-473c-ac4d-86f51fe3d909 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.477607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9a07c-4511-473c-ac4d-86f51fe3d909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:09.481013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9a07c-4511-473c-ac4d-86f51fe3d909 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:09.508313 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2149/4254] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:09 2022] POST /identity/v3/users => generated 284 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:09.508313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9a07c-4511-473c-ac4d-86f51fe3d909 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.510165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-50a9a07c-4511-473c-ac4d-86f51fe3d909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:09.517958 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2106/4255] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:09.518476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e05fed-be88-4ee0-90df-bbcf5d23bd8c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.530058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e05fed-be88-4ee0-90df-bbcf5d23bd8c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:09.535367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294f02a-1c25-4d66-9039-06ea242db7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:09.535990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294f02a-1c25-4d66-9039-06ea242db7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.536345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294f02a-1c25-4d66-9039-06ea242db7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.536611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294f02a-1c25-4d66-9039-06ea242db7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.537341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294f02a-1c25-4d66-9039-06ea242db7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:09.539059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294f02a-1c25-4d66-9039-06ea242db7d0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:09.550360 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2107/4256] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:09.553872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294f02a-1c25-4d66-9039-06ea242db7d0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.565900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294f02a-1c25-4d66-9039-06ea242db7d0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:09.571307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee81f47-d7f0-48d1-8471-3a1e81950de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:09.572259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee81f47-d7f0-48d1-8471-3a1e81950de3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.572526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee81f47-d7f0-48d1-8471-3a1e81950de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.572784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee81f47-d7f0-48d1-8471-3a1e81950de3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.573454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee81f47-d7f0-48d1-8471-3a1e81950de3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:09.573743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee81f47-d7f0-48d1-8471-3a1e81950de3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:09.576126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee81f47-d7f0-48d1-8471-3a1e81950de3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:09.593573 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2108/4257] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:09.595617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee81f47-d7f0-48d1-8471-3a1e81950de3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.615347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee81f47-d7f0-48d1-8471-3a1e81950de3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:09.620632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11d132-9a0c-40cb-88b0-ac098d6f6867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:09.621817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11d132-9a0c-40cb-88b0-ac098d6f6867 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.622104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11d132-9a0c-40cb-88b0-ac098d6f6867 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.622362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11d132-9a0c-40cb-88b0-ac098d6f6867 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.623035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11d132-9a0c-40cb-88b0-ac098d6f6867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:09.626372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11d132-9a0c-40cb-88b0-ac098d6f6867 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:09.630656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9a07c-4511-473c-ac4d-86f51fe3d909 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:09.636814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1dc02-e769-4388-ab86-ba9a4d7fc37e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:09.637726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1dc02-e769-4388-ab86-ba9a4d7fc37e service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.638011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1dc02-e769-4388-ab86-ba9a4d7fc37e service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.638284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1dc02-e769-4388-ab86-ba9a4d7fc37e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.638509 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2109/4258] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:09.641956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11d132-9a0c-40cb-88b0-ac098d6f6867 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.643806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fee1dc02-e769-4388-ab86-ba9a4d7fc37e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:09.653446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11d132-9a0c-40cb-88b0-ac098d6f6867 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:09.660422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73799e2e-bac6-4bea-9459-ae5ade47d0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:09.661028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73799e2e-bac6-4bea-9459-ae5ade47d0cb admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.661332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73799e2e-bac6-4bea-9459-ae5ade47d0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.661599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73799e2e-bac6-4bea-9459-ae5ade47d0cb admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.662224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73799e2e-bac6-4bea-9459-ae5ade47d0cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:09.664109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73799e2e-bac6-4bea-9459-ae5ade47d0cb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:09.701092 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2110/4259] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:09 2022] POST /identity/v3/projects => generated 399 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:09.702256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73799e2e-bac6-4bea-9459-ae5ade47d0cb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.704306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:09.738871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1dc02-e769-4388-ab86-ba9a4d7fc37e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:09.740941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1dc02-e769-4388-ab86-ba9a4d7fc37e service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:09.751792 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2150/4260] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:09.752725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1dc02-e769-4388-ab86-ba9a4d7fc37e service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.765469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1dc02-e769-4388-ab86-ba9a4d7fc37e service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:09.808331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73799e2e-bac6-4bea-9459-ae5ade47d0cb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:09.824201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05ac5e-4ff2-4a1d-bbcb-10a73aed948e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:09.825134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ac5e-4ff2-4a1d-bbcb-10a73aed948e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.825476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ac5e-4ff2-4a1d-bbcb-10a73aed948e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.825827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ac5e-4ff2-4a1d-bbcb-10a73aed948e admin admin] PATH_INFO: `/v3/projects/2902b82407cb43e289a6f03178142dc7/users/2707e97bf01f4941ba74c578b57e080c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.871700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05ac5e-4ff2-4a1d-bbcb-10a73aed948e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2902b82407cb43e289a6f03178142dc7, user_id=2707e97bf01f4941ba74c578b57e080c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:09.875565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05ac5e-4ff2-4a1d-bbcb-10a73aed948e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:09.905715 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2111/4261] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:09 2022] PUT /identity/v3/projects/2902b82407cb43e289a6f03178142dc7/users/2707e97bf01f4941ba74c578b57e080c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:09.916437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b2260-5e77-4bf6-a175-2b27e6352c84 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.916786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b2260-5e77-4bf6-a175-2b27e6352c84 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.917346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b2260-5e77-4bf6-a175-2b27e6352c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.927564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d75ec4-014d-450c-863b-e019cebcb8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:09.928500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d75ec4-014d-450c-863b-e019cebcb8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:09.928772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d75ec4-014d-450c-863b-e019cebcb8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:09.929076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d75ec4-014d-450c-863b-e019cebcb8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:09.929741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d75ec4-014d-450c-863b-e019cebcb8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:09.936076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d75ec4-014d-450c-863b-e019cebcb8f0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:09.953475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d28b2260-5e77-4bf6-a175-2b27e6352c84 None None] MFA Rules not processed for user `2707e97bf01f4941ba74c578b57e080c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:09.955139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d28b2260-5e77-4bf6-a175-2b27e6352c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:09.979581 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2151/4262] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:09 2022] POST /identity/v3/users => generated 306 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:09.981480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d75ec4-014d-450c-863b-e019cebcb8f0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:09.992699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d75ec4-014d-450c-863b-e019cebcb8f0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.006234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:10.068088 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2112/4263] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:10.069518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b2260-5e77-4bf6-a175-2b27e6352c84 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.095828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ada92-077a-4752-9e80-f87309a37be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.096607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ada92-077a-4752-9e80-f87309a37be4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.096889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ada92-077a-4752-9e80-f87309a37be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.097193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ada92-077a-4752-9e80-f87309a37be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.097842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ada92-077a-4752-9e80-f87309a37be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.099758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ada92-077a-4752-9e80-f87309a37be4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.118370 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2152/4264] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:10.122612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ada92-077a-4752-9e80-f87309a37be4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.145063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ada92-077a-4752-9e80-f87309a37be4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.153454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02d300-7ddc-4a61-ac3f-122bfac8b72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.155848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02d300-7ddc-4a61-ac3f-122bfac8b72f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.163995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02d300-7ddc-4a61-ac3f-122bfac8b72f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.164578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02d300-7ddc-4a61-ac3f-122bfac8b72f admin admin] PATH_INFO: `/v3/projects/34a9201c85c24d61a8ed55a7cec69d67/users/ea567d4cfe1d4073b55a532ca672acdc/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.174040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b2260-5e77-4bf6-a175-2b27e6352c84 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.185446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61f7de-3a7b-420f-8670-71a88b4f32d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.185772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61f7de-3a7b-420f-8670-71a88b4f32d3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.186424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61f7de-3a7b-420f-8670-71a88b4f32d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.186702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61f7de-3a7b-420f-8670-71a88b4f32d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.187342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61f7de-3a7b-420f-8670-71a88b4f32d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.196233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61f7de-3a7b-420f-8670-71a88b4f32d3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.196431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02d300-7ddc-4a61-ac3f-122bfac8b72f admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a9201c85c24d61a8ed55a7cec69d67, user_id=ea567d4cfe1d4073b55a532ca672acdc, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.196431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02d300-7ddc-4a61-ac3f-122bfac8b72f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.231556 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2153/4265] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:10 2022] PUT /identity/v3/projects/34a9201c85c24d61a8ed55a7cec69d67/users/ea567d4cfe1d4073b55a532ca672acdc/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:10.233005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02d300-7ddc-4a61-ac3f-122bfac8b72f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.249630 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2113/4266] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:10 2022] POST /identity/v3/projects => generated 369 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:10.253308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61f7de-3a7b-420f-8670-71a88b4f32d3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.314291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:10.346219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02d300-7ddc-4a61-ac3f-122bfac8b72f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.381713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61f7de-3a7b-420f-8670-71a88b4f32d3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.396332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0a3d4-619a-411d-84a9-f4040ae8c287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.396967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0a3d4-619a-411d-84a9-f4040ae8c287 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.397492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0a3d4-619a-411d-84a9-f4040ae8c287 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.397779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0a3d4-619a-411d-84a9-f4040ae8c287 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.398804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0a3d4-619a-411d-84a9-f4040ae8c287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.401678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0a3d4-619a-411d-84a9-f4040ae8c287 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.436987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5e556-9c82-4e77-bb56-ac736c809c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.437859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5e556-9c82-4e77-bb56-ac736c809c9b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.438287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5e556-9c82-4e77-bb56-ac736c809c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.438711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5e556-9c82-4e77-bb56-ac736c809c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.439669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5e556-9c82-4e77-bb56-ac736c809c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.442090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5e556-9c82-4e77-bb56-ac736c809c9b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.457727 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2154/4267] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:09:10.458825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5e556-9c82-4e77-bb56-ac736c809c9b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.459887 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2114/4268] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:10 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:10.463779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0a3d4-619a-411d-84a9-f4040ae8c287 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.479221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0a3d4-619a-411d-84a9-f4040ae8c287 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.480744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5e556-9c82-4e77-bb56-ac736c809c9b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.486304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e901d37-33d8-4d2a-9a54-331d49b5dc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.487199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901d37-33d8-4d2a-9a54-331d49b5dc29 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.487710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901d37-33d8-4d2a-9a54-331d49b5dc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.488135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901d37-33d8-4d2a-9a54-331d49b5dc29 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.489059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bcf18-3f1b-4038-aefa-fdfddfe258a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.490853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bcf18-3f1b-4038-aefa-fdfddfe258a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.491013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e901d37-33d8-4d2a-9a54-331d49b5dc29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.491332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bcf18-3f1b-4038-aefa-fdfddfe258a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.491703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bcf18-3f1b-4038-aefa-fdfddfe258a1 admin admin] PATH_INFO: `/v3/projects/34a9201c85c24d61a8ed55a7cec69d67/users/ea567d4cfe1d4073b55a532ca672acdc/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.494397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e901d37-33d8-4d2a-9a54-331d49b5dc29 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.494577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bcf18-3f1b-4038-aefa-fdfddfe258a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a9201c85c24d61a8ed55a7cec69d67, user_id=ea567d4cfe1d4073b55a532ca672acdc, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.500298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bcf18-3f1b-4038-aefa-fdfddfe258a1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.510438 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2115/4269] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:09:10.514809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e901d37-33d8-4d2a-9a54-331d49b5dc29 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.531065 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2155/4270] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:10 2022] PUT /identity/v3/projects/34a9201c85c24d61a8ed55a7cec69d67/users/ea567d4cfe1d4073b55a532ca672acdc/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:10.535893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a83ece-88f2-4de2-a272-191f87fb1d4a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.536298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a83ece-88f2-4de2-a272-191f87fb1d4a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.536680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a83ece-88f2-4de2-a272-191f87fb1d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.543553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e901d37-33d8-4d2a-9a54-331d49b5dc29 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.594269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-28a83ece-88f2-4de2-a272-191f87fb1d4a None None] MFA Rules not processed for user `ea567d4cfe1d4073b55a532ca672acdc`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:10.596897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-28a83ece-88f2-4de2-a272-191f87fb1d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:10.618413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:10.649017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d3b95-edb3-4ad3-a711-6f593a8d839e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.649705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d3b95-edb3-4ad3-a711-6f593a8d839e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.649925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d3b95-edb3-4ad3-a711-6f593a8d839e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.650144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d3b95-edb3-4ad3-a711-6f593a8d839e admin admin] PATH_INFO: `/v3/projects/cc5915c40f3e440db9abb899c7e16961/users/56f42b0f02f44b509e1ee5b60ff14475/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.677655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d3b95-edb3-4ad3-a711-6f593a8d839e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5915c40f3e440db9abb899c7e16961, user_id=56f42b0f02f44b509e1ee5b60ff14475, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.680447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d3b95-edb3-4ad3-a711-6f593a8d839e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.705384 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2116/4271] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:10 2022] PUT /identity/v3/projects/cc5915c40f3e440db9abb899c7e16961/users/56f42b0f02f44b509e1ee5b60ff14475/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:10.709736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d3b95-edb3-4ad3-a711-6f593a8d839e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.744676 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2156/4272] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:10 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:10.746026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a83ece-88f2-4de2-a272-191f87fb1d4a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.758754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a83ece-88f2-4de2-a272-191f87fb1d4a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.842010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d3b95-edb3-4ad3-a711-6f593a8d839e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.849653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011dfc7-f96a-421a-b065-df3f3366541e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.851085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:10.852229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011dfc7-f96a-421a-b065-df3f3366541e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.852675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011dfc7-f96a-421a-b065-df3f3366541e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.853232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011dfc7-f96a-421a-b065-df3f3366541e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.854181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011dfc7-f96a-421a-b065-df3f3366541e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.857287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011dfc7-f96a-421a-b065-df3f3366541e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.872662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c821eb-956c-415f-a3ea-8fda22028e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.873471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c821eb-956c-415f-a3ea-8fda22028e63 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.873827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c821eb-956c-415f-a3ea-8fda22028e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.874163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c821eb-956c-415f-a3ea-8fda22028e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.874968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c821eb-956c-415f-a3ea-8fda22028e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.878288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c821eb-956c-415f-a3ea-8fda22028e63 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.888535 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2117/4273] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:09:10.891880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011dfc7-f96a-421a-b065-df3f3366541e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.918776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011dfc7-f96a-421a-b065-df3f3366541e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:10.927958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a341d6-e373-4d92-b268-435fb352b32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:10.928711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a341d6-e373-4d92-b268-435fb352b32e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.929070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a341d6-e373-4d92-b268-435fb352b32e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.929369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a341d6-e373-4d92-b268-435fb352b32e admin admin] PATH_INFO: `/v3/projects/cc5915c40f3e440db9abb899c7e16961/users/56f42b0f02f44b509e1ee5b60ff14475/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:10.934034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a341d6-e373-4d92-b268-435fb352b32e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5915c40f3e440db9abb899c7e16961, user_id=56f42b0f02f44b509e1ee5b60ff14475, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:10.934865 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2157/4274] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:10 2022] POST /identity/v3/projects => generated 399 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:10.940085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c821eb-956c-415f-a3ea-8fda22028e63 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:10.942932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a341d6-e373-4d92-b268-435fb352b32e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:10.985096 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2118/4275] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:10 2022] PUT /identity/v3/projects/cc5915c40f3e440db9abb899c7e16961/users/56f42b0f02f44b509e1ee5b60ff14475/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:10.994980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355834b-7273-4294-a851-49bd45083d43 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:10.995333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355834b-7273-4294-a851-49bd45083d43 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:10.995674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355834b-7273-4294-a851-49bd45083d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:11.040158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c821eb-956c-415f-a3ea-8fda22028e63 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:11.093948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d355834b-7273-4294-a851-49bd45083d43 None None] MFA Rules not processed for user `56f42b0f02f44b509e1ee5b60ff14475`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:11.095319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d355834b-7273-4294-a851-49bd45083d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:11.154452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24391dc-dec3-4e2d-b05c-d6ed522ee5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:11.155072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24391dc-dec3-4e2d-b05c-d6ed522ee5dd admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:11.155348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24391dc-dec3-4e2d-b05c-d6ed522ee5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:11.155615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24391dc-dec3-4e2d-b05c-d6ed522ee5dd admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:11.156241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24391dc-dec3-4e2d-b05c-d6ed522ee5dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:11.158499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24391dc-dec3-4e2d-b05c-d6ed522ee5dd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:11.198670 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2158/4276] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:10 2022] POST /identity/v3/users => generated 300 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:11.203088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24391dc-dec3-4e2d-b05c-d6ed522ee5dd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:11.238032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24391dc-dec3-4e2d-b05c-d6ed522ee5dd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:11.258173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1439b58-e3b5-42f8-a276-67f9c4627675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:11.259074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1439b58-e3b5-42f8-a276-67f9c4627675 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:11.259520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1439b58-e3b5-42f8-a276-67f9c4627675 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:11.260236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1439b58-e3b5-42f8-a276-67f9c4627675 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:11.261166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1439b58-e3b5-42f8-a276-67f9c4627675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:11.263503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1439b58-e3b5-42f8-a276-67f9c4627675 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:11.276524 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2119/4277] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:11.279262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355834b-7273-4294-a851-49bd45083d43 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:11.294487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355834b-7273-4294-a851-49bd45083d43 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:11.311855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ec10b-5c0d-4e36-ba7a-8b4ed740dcb8 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:11.312536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ec10b-5c0d-4e36-ba7a-8b4ed740dcb8 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:11.312744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ec10b-5c0d-4e36-ba7a-8b4ed740dcb8 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:11.313012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ec10b-5c0d-4e36-ba7a-8b4ed740dcb8 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:11.313724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ec10b-5c0d-4e36-ba7a-8b4ed740dcb8 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:11.315707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ec10b-5c0d-4e36-ba7a-8b4ed740dcb8 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:11.324531 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2159/4278] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:11.327858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1439b58-e3b5-42f8-a276-67f9c4627675 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:11.356937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1439b58-e3b5-42f8-a276-67f9c4627675 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:11.372504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c189fbf-abc8-42bf-a93b-286bd7dd008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:11.374437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c189fbf-abc8-42bf-a93b-286bd7dd008b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:11.374785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c189fbf-abc8-42bf-a93b-286bd7dd008b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:11.374977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c189fbf-abc8-42bf-a93b-286bd7dd008b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:11.376290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c189fbf-abc8-42bf-a93b-286bd7dd008b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:11.380008 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2120/4279] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:09:11 2022] POST /identity/v3/projects => generated 328 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:11.386068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ec10b-5c0d-4e36-ba7a-8b4ed740dcb8 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:11.390695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c189fbf-abc8-42bf-a93b-286bd7dd008b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:11.415330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:11.419748 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2160/4280] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:11.420917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c189fbf-abc8-42bf-a93b-286bd7dd008b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:11.454448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c189fbf-abc8-42bf-a93b-286bd7dd008b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:11.562965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ec10b-5c0d-4e36-ba7a-8b4ed740dcb8 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:11.580752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0741063-b7b5-4b89-afee-d9fb6e6ae920 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:11.582712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0741063-b7b5-4b89-afee-d9fb6e6ae920 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:11.585181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0741063-b7b5-4b89-afee-d9fb6e6ae920 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:11.585965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0741063-b7b5-4b89-afee-d9fb6e6ae920 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:11.586780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0741063-b7b5-4b89-afee-d9fb6e6ae920 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:11.593481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0741063-b7b5-4b89-afee-d9fb6e6ae920 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:11.600818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:11.628274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1639125-b5dc-47db-b3fe-2456242529ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:11.630290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1639125-b5dc-47db-b3fe-2456242529ab admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:11.630492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1639125-b5dc-47db-b3fe-2456242529ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:11.631953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1639125-b5dc-47db-b3fe-2456242529ab admin admin] PATH_INFO: `/v3/projects/900945ce55e14648a881f27796caaa44/users/de947d89fd994f6eb9c492b202c41520/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:11.655278 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2121/4281] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:11 2022] POST /identity/v3/users => generated 413 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:11.659263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0741063-b7b5-4b89-afee-d9fb6e6ae920 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:11.698098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1639125-b5dc-47db-b3fe-2456242529ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=900945ce55e14648a881f27796caaa44, user_id=de947d89fd994f6eb9c492b202c41520, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:11.701361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1639125-b5dc-47db-b3fe-2456242529ab admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:11.708635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0741063-b7b5-4b89-afee-d9fb6e6ae920 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:11.736077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc605b0-ecb7-459c-9b50-d24dc3f50d9e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:11.736780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc605b0-ecb7-459c-9b50-d24dc3f50d9e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:11.737590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc605b0-ecb7-459c-9b50-d24dc3f50d9e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:11.737590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc605b0-ecb7-459c-9b50-d24dc3f50d9e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:11.738195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc605b0-ecb7-459c-9b50-d24dc3f50d9e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:11.740032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc605b0-ecb7-459c-9b50-d24dc3f50d9e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:11.759523 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2161/4282] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:11 2022] PUT /identity/v3/projects/900945ce55e14648a881f27796caaa44/users/de947d89fd994f6eb9c492b202c41520/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:11.770796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf04d3d-bc37-4378-a4b9-09efd4e46aa5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:11.771067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf04d3d-bc37-4378-a4b9-09efd4e46aa5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:11.772134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf04d3d-bc37-4378-a4b9-09efd4e46aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:11.797465 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2122/4283] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:11 2022] POST /identity/v3/roles => generated 244 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:11.800556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc605b0-ecb7-459c-9b50-d24dc3f50d9e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:11.823421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc605b0-ecb7-459c-9b50-d24dc3f50d9e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:11.857634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dcf04d3d-bc37-4378-a4b9-09efd4e46aa5 None None] MFA Rules not processed for user `de947d89fd994f6eb9c492b202c41520`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:11.858909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dcf04d3d-bc37-4378-a4b9-09efd4e46aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:12.009301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f38fae-9122-4103-9b94-203ef97ae617 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:12.012280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f38fae-9122-4103-9b94-203ef97ae617 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:12.012971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f38fae-9122-4103-9b94-203ef97ae617 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:12.013118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f38fae-9122-4103-9b94-203ef97ae617 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:12.013956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f38fae-9122-4103-9b94-203ef97ae617 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:12.015868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f38fae-9122-4103-9b94-203ef97ae617 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:12.019277 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2162/4284] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:09:12.032412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf04d3d-bc37-4378-a4b9-09efd4e46aa5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:12.034067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dcf04d3d-bc37-4378-a4b9-09efd4e46aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:12.048270 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2123/4285] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:11 2022] POST /identity/v3/roles => generated 246 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:12.051749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f38fae-9122-4103-9b94-203ef97ae617 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:12.074812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f38fae-9122-4103-9b94-203ef97ae617 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:12.087458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b2c46-15d7-4cf9-ad7d-54ff69e0e443 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:12.090270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b2c46-15d7-4cf9-ad7d-54ff69e0e443 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:12.092370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b2c46-15d7-4cf9-ad7d-54ff69e0e443 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:12.093528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b2c46-15d7-4cf9-ad7d-54ff69e0e443 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/759749c69fdf4013bd05ecdde086418b/users/5690d84099354671bc41cfad427111cf/roles/00fe0f90fd804b5883724a24aa8909af` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:12.135739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b2c46-15d7-4cf9-ad7d-54ff69e0e443 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=759749c69fdf4013bd05ecdde086418b, user_id=5690d84099354671bc41cfad427111cf, role_id=00fe0f90fd804b5883724a24aa8909af)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:12.138657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b2c46-15d7-4cf9-ad7d-54ff69e0e443 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:12.159465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf04d3d-bc37-4378-a4b9-09efd4e46aa5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:12.165871 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2124/4286] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:12 2022] PUT /identity/v3/projects/759749c69fdf4013bd05ecdde086418b/users/5690d84099354671bc41cfad427111cf/roles/00fe0f90fd804b5883724a24aa8909af => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:12.170114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b2c46-15d7-4cf9-ad7d-54ff69e0e443 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:12.274667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140bb58-5a24-48f7-8110-a7c6d75b9663 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:12.275484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140bb58-5a24-48f7-8110-a7c6d75b9663 service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:12.276079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140bb58-5a24-48f7-8110-a7c6d75b9663 service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:12.276462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140bb58-5a24-48f7-8110-a7c6d75b9663 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:12.316125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:12.333719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:12.339721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b2c46-15d7-4cf9-ad7d-54ff69e0e443 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:12.359426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce78271-568b-41ce-917c-e53c44f6415a tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:12.361557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce78271-568b-41ce-917c-e53c44f6415a tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:12.361845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce78271-568b-41ce-917c-e53c44f6415a tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:12.362117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce78271-568b-41ce-917c-e53c44f6415a tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/759749c69fdf4013bd05ecdde086418b/users/5690d84099354671bc41cfad427111cf/roles/6a2a3216774345ea9a2852b303ae6e1f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:12.364800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce78271-568b-41ce-917c-e53c44f6415a tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=759749c69fdf4013bd05ecdde086418b, user_id=5690d84099354671bc41cfad427111cf, role_id=6a2a3216774345ea9a2852b303ae6e1f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:12.367550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce78271-568b-41ce-917c-e53c44f6415a tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:12.370340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140bb58-5a24-48f7-8110-a7c6d75b9663 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:12.374053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140bb58-5a24-48f7-8110-a7c6d75b9663 service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:12.398941 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2125/4287] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:12 2022] PUT /identity/v3/projects/759749c69fdf4013bd05ecdde086418b/users/5690d84099354671bc41cfad427111cf/roles/6a2a3216774345ea9a2852b303ae6e1f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:12.401196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce78271-568b-41ce-917c-e53c44f6415a tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:12.401414 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2163/4288] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:12 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:09:12.647180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce78271-568b-41ce-917c-e53c44f6415a tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:12.684425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2d51a-0514-4dda-a2da-c16ed375061c tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:12.685191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d51a-0514-4dda-a2da-c16ed375061c tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:12.685511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d51a-0514-4dda-a2da-c16ed375061c tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:12.685735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d51a-0514-4dda-a2da-c16ed375061c tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:12.686389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2d51a-0514-4dda-a2da-c16ed375061c tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:12.699488 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-1ef2d51a-0514-4dda-a2da-c16ed375061c tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:12.699488 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:09:12.699488 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:09:12.703958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2d51a-0514-4dda-a2da-c16ed375061c tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:12.717078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2c3ae-665b-4a0a-8074-e3cd03d459a1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:12.718577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2c3ae-665b-4a0a-8074-e3cd03d459a1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:12.719044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2c3ae-665b-4a0a-8074-e3cd03d459a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:12.778568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-16b2c3ae-665b-4a0a-8074-e3cd03d459a1 None None] MFA Rules not processed for user `c48346e56c47481cab7f248b67369a27`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:12.789286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-16b2c3ae-665b-4a0a-8074-e3cd03d459a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:12.927119 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2126/4289] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:12 2022] GET /identity/v3/users => generated 8361 bytes in 526 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:12.935934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4f1ff-7cfa-448b-9069-5f656b03d68d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:12.936203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4f1ff-7cfa-448b-9069-5f656b03d68d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:12.936465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4f1ff-7cfa-448b-9069-5f656b03d68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:13.060286 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2164/4290] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:09:13.071806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-14d4f1ff-7cfa-448b-9069-5f656b03d68d None None] MFA Rules not processed for user `5690d84099354671bc41cfad427111cf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:13.073328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-14d4f1ff-7cfa-448b-9069-5f656b03d68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:13.269287 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2127/4291] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:13.286506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dedb94-3a2f-4f52-b161-e5eb4ff1e33c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:13.287344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dedb94-3a2f-4f52-b161-e5eb4ff1e33c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:13.288436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dedb94-3a2f-4f52-b161-e5eb4ff1e33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:13.348569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c9dedb94-3a2f-4f52-b161-e5eb4ff1e33c None None] MFA Rules not processed for user `5690d84099354671bc41cfad427111cf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:13.350769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c9dedb94-3a2f-4f52-b161-e5eb4ff1e33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:13.382053 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2165/4292] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:13 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:13.382600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4f1ff-7cfa-448b-9069-5f656b03d68d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:13.400592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4f1ff-7cfa-448b-9069-5f656b03d68d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:13.430332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77888c62-5b66-42cb-871c-042d2b952a1d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5690d84099354671bc41cfad427111cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '759749c69fdf4013bd05ecdde086418b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1607046426', 'tempest-NotDelegatedRole-662290822'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:13.433721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77888c62-5b66-42cb-871c-042d2b952a1d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:13.434267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77888c62-5b66-42cb-871c-042d2b952a1d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:13.434806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77888c62-5b66-42cb-871c-042d2b952a1d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:13.435727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77888c62-5b66-42cb-871c-042d2b952a1d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] RBAC: Authorizing `identity:create_trust()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:13.438831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77888c62-5b66-42cb-871c-042d2b952a1d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:13.606104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dedb94-3a2f-4f52-b161-e5eb4ff1e33c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:13.645545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:13.656178 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2128/4293] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Oct 15 21:09:13 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:13.658750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77888c62-5b66-42cb-871c-042d2b952a1d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:13.681375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77888c62-5b66-42cb-871c-042d2b952a1d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:13.691072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b419d-e364-4962-8df8-ade5acef773d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:13.691991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b419d-e364-4962-8df8-ade5acef773d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:13.692414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b419d-e364-4962-8df8-ade5acef773d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:13.692784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b419d-e364-4962-8df8-ade5acef773d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:13.693863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b419d-e364-4962-8df8-ade5acef773d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:list_trusts()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:13.697182 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-102b419d-e364-4962-8df8-ade5acef773d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_trusts": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:13.697182 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:09:13.697182 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:09:13.698680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b419d-e364-4962-8df8-ade5acef773d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:13.700214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dedb94-3a2f-4f52-b161-e5eb4ff1e33c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:13.707470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:13.708182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:13.708464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:13.708736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] PATH_INFO: `/v3/users/ea567d4cfe1d4073b55a532ca672acdc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:13.709751 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2129/4294] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 21:09:13 2022] GET /identity/v3/OS-TRUST/trusts/ => generated 562 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:13.713095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea567d4cfe1d4073b55a532ca672acdc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:13.713716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b419d-e364-4962-8df8-ade5acef773d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:13.715283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:13.732039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b419d-e364-4962-8df8-ade5acef773d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:13.747585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5690d84099354671bc41cfad427111cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '759749c69fdf4013bd05ecdde086418b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1607046426', 'tempest-NotDelegatedRole-662290822'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:13.763517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:13.764514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:13.781018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] PATH_INFO: `/v3/OS-TRUST/trusts/1377a34ab973404f9daac40798052a1d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:13.832179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] RBAC: Authorizing `identity:delete_trust(trust_id=1377a34ab973404f9daac40798052a1d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:13.839560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:13.877092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea567d4cfe1d4073b55a532ca672acdc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:13.907969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:13.909838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea567d4cfe1d4073b55a532ca672acdc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:13.925066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea567d4cfe1d4073b55a532ca672acdc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:13.946218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea567d4cfe1d4073b55a532ca672acdc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:13.946456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '1377a34ab973404f9daac40798052a1d', 'request_id': 'req-925a4918-936d-4a7a-ac27-493c827024c8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:13.947118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '1377a34ab973404f9daac40798052a1d', 'request_id': 'req-925a4918-936d-4a7a-ac27-493c827024c8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:13.948599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7746b054-773d-43e8-87fc-b83060743db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea567d4cfe1d4073b55a532ca672acdc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:13.962934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '1377a34ab973404f9daac40798052a1d', 'request_id': 'req-925a4918-936d-4a7a-ac27-493c827024c8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:13.974929 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2130/4295] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Oct 15 21:09:13 2022] DELETE /identity/v3/OS-TRUST/trusts/1377a34ab973404f9daac40798052a1d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:13.976170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:13.981195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:13.985899 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2166/4296] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:13 2022] DELETE /identity/v3/users/ea567d4cfe1d4073b55a532ca672acdc => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:13.987630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048becf-3f9b-4264-82c7-b9f25a606fce None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:13.988029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048becf-3f9b-4264-82c7-b9f25a606fce None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:13.988281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048becf-3f9b-4264-82c7-b9f25a606fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:14.025507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a4918-936d-4a7a-ac27-493c827024c8 tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:14.043587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0048becf-3f9b-4264-82c7-b9f25a606fce None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:14.044949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0048becf-3f9b-4264-82c7-b9f25a606fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:14.212339 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2167/4297] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:14.212505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5981eed-a01d-4074-913f-bd492b34af9d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5690d84099354671bc41cfad427111cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '759749c69fdf4013bd05ecdde086418b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1607046426', 'tempest-NotDelegatedRole-662290822'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:14.215298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5981eed-a01d-4074-913f-bd492b34af9d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:14.218038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5981eed-a01d-4074-913f-bd492b34af9d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:14.218962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5981eed-a01d-4074-913f-bd492b34af9d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] PATH_INFO: `/v3/OS-TRUST/trusts/1377a34ab973404f9daac40798052a1d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:14.233727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048becf-3f9b-4264-82c7-b9f25a606fce None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:14.237735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0048becf-3f9b-4264-82c7-b9f25a606fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-d5981eed-a01d-4074-913f-bd492b34af9d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Could not find trust: 1377a34ab973404f9daac40798052a1d.: keystone.exception.TrustNotFound: Could not find trust: 1377a34ab973404f9daac40798052a1d. Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Oct 15 21:09:14.273845 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 1377a34ab973404f9daac40798052a1d. Oct 15 21:09:14.274876 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:14.281721 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2131/4298] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:13 2022] GET /identity/v3/OS-TRUST/trusts/1377a34ab973404f9daac40798052a1d => generated 111 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:14.282933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5981eed-a01d-4074-913f-bd492b34af9d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:14.285171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d5981eed-a01d-4074-913f-bd492b34af9d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:14.395775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048becf-3f9b-4264-82c7-b9f25a606fce None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:14.404446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0563c-7629-4156-9fd0-08c903812025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:14.405635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0563c-7629-4156-9fd0-08c903812025 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:14.406010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0563c-7629-4156-9fd0-08c903812025 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:14.406305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0563c-7629-4156-9fd0-08c903812025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:14.427383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0563c-7629-4156-9fd0-08c903812025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:14.428901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0563c-7629-4156-9fd0-08c903812025 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:14.441221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5981eed-a01d-4074-913f-bd492b34af9d tempest-TrustsV3TestJSON-186333077 tempest-user-844234086] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:14.453531 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2168/4299] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:09:14.453852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:14.454751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:14.455140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:14.455545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/6a2a3216774345ea9a2852b303ae6e1f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:14.458360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=6a2a3216774345ea9a2852b303ae6e1f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:14.461558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:14.563766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role 6a2a3216774345ea9a2852b303ae6e1f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:14.564058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:14.565414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:14.568791 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2132/4300] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:14 2022] DELETE /identity/v3/roles/6a2a3216774345ea9a2852b303ae6e1f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:14.572722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0563c-7629-4156-9fd0-08c903812025 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:14.591813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0563c-7629-4156-9fd0-08c903812025 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:14.714476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:14.715125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:14.715497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:14.715777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/00fe0f90fd804b5883724a24aa8909af` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:14.735803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=00fe0f90fd804b5883724a24aa8909af)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:14.738671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:14.820703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:14.876950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role 00fe0f90fd804b5883724a24aa8909af has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:14.877563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:14.879102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:14.887176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:14.889227 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2169/4301] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:14 2022] DELETE /identity/v3/roles/00fe0f90fd804b5883724a24aa8909af => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:14.890539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:14.907177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b85a6d-752e-4e81-b279-7dbd717e3988 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:14.943026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:15.031822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:15.032486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:15.032766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:15.033062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users/5690d84099354671bc41cfad427111cf` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:15.034270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_user(user_id=5690d84099354671bc41cfad427111cf)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:15.036209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:15.127382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5690d84099354671bc41cfad427111cf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.144893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5690d84099354671bc41cfad427111cf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.174548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5690d84099354671bc41cfad427111cf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.193887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5690d84099354671bc41cfad427111cf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.194684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5690d84099354671bc41cfad427111cf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.207718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:15.208256 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2133/4302] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:14 2022] DELETE /identity/v3/users/5690d84099354671bc41cfad427111cf => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:15.209386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:15.224764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe3de7-c329-4fc9-aad3-440d7264921f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:15.338543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:15.339196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:15.341511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:15.341889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/759749c69fdf4013bd05ecdde086418b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:15.344178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_project(project_id=759749c69fdf4013bd05ecdde086418b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:15.348713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:15.361074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:15.362531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:15.470730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '759749c69fdf4013bd05ecdde086418b', 'request_id': 'req-de7dc5d3-482d-4da5-9a1e-633fefe181f7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.482343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '759749c69fdf4013bd05ecdde086418b', 'request_id': 'req-de7dc5d3-482d-4da5-9a1e-633fefe181f7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.500156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '759749c69fdf4013bd05ecdde086418b', 'request_id': 'req-de7dc5d3-482d-4da5-9a1e-633fefe181f7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.519076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] The token cache is being invalidate because project 759749c69fdf4013bd05ecdde086418b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:15.519617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.521022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.528231 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2170/4303] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:15 2022] DELETE /identity/v3/projects/759749c69fdf4013bd05ecdde086418b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:15.531979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:15.535071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:15.538341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e2180-89b9-4c07-9f29-8b09c7a1e4f6 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:15.658335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:15.696096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:15.696908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:15.697319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:15.697676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] PATH_INFO: `/v3/users/fb8dc371c65f4bbb92d559c7ed78964c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:15.700789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb8dc371c65f4bbb92d559c7ed78964c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:15.702762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dc5d3-482d-4da5-9a1e-633fefe181f7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:15.703001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:15.710035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09a777-f9e9-4d3d-8bf0-061db874590e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:15.710991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09a777-f9e9-4d3d-8bf0-061db874590e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:15.712377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09a777-f9e9-4d3d-8bf0-061db874590e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:15.712791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09a777-f9e9-4d3d-8bf0-061db874590e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:15.713634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09a777-f9e9-4d3d-8bf0-061db874590e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:15.716828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09a777-f9e9-4d3d-8bf0-061db874590e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:15.773696 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2171/4304] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:09:15 2022] POST /identity/v3/projects => generated 329 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:15.780540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09a777-f9e9-4d3d-8bf0-061db874590e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:15.791015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:15.800932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb8dc371c65f4bbb92d559c7ed78964c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.837386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb8dc371c65f4bbb92d559c7ed78964c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.863870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb8dc371c65f4bbb92d559c7ed78964c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.883608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb8dc371c65f4bbb92d559c7ed78964c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.884334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb8dc371c65f4bbb92d559c7ed78964c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:15.908872 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2134/4305] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:15 2022] DELETE /identity/v3/users/fb8dc371c65f4bbb92d559c7ed78964c => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:15.928363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09a777-f9e9-4d3d-8bf0-061db874590e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:16.036019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39814d62-e01e-43e2-9eae-47bdba2cdcca tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:16.037188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814d62-e01e-43e2-9eae-47bdba2cdcca tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:16.037706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814d62-e01e-43e2-9eae-47bdba2cdcca tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:16.038227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814d62-e01e-43e2-9eae-47bdba2cdcca tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:16.044880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39814d62-e01e-43e2-9eae-47bdba2cdcca tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:16.047302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:16.048016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39814d62-e01e-43e2-9eae-47bdba2cdcca tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:16.048968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:16.083987 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2172/4306] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:15 2022] POST /identity/v3/users => generated 416 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:16.084992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39814d62-e01e-43e2-9eae-47bdba2cdcca tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:16.087183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-39814d62-e01e-43e2-9eae-47bdba2cdcca tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:16.107284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39814d62-e01e-43e2-9eae-47bdba2cdcca tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:16.112972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ee9b-4630-496a-9eb8-552eb77de11d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:16.114491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ee9b-4630-496a-9eb8-552eb77de11d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:16.115105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ee9b-4630-496a-9eb8-552eb77de11d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:16.115579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ee9b-4630-496a-9eb8-552eb77de11d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:16.116724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ee9b-4630-496a-9eb8-552eb77de11d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:16.120421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ee9b-4630-496a-9eb8-552eb77de11d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:16.160619 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2173/4307] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:16 2022] POST /identity/v3/roles => generated 244 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:16.161588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ee9b-4630-496a-9eb8-552eb77de11d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:16.187080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ee9b-4630-496a-9eb8-552eb77de11d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:16.203583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6b9e2-00ce-4c23-a878-fee569d6fc0f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:16.203871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6b9e2-00ce-4c23-a878-fee569d6fc0f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:16.204240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6b9e2-00ce-4c23-a878-fee569d6fc0f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:16.205515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6b9e2-00ce-4c23-a878-fee569d6fc0f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:16.206205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6b9e2-00ce-4c23-a878-fee569d6fc0f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:16.211903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6b9e2-00ce-4c23-a878-fee569d6fc0f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:16.251766 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2174/4308] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:16 2022] POST /identity/v3/roles => generated 246 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:16.256242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6b9e2-00ce-4c23-a878-fee569d6fc0f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:16.285781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80b13d-d3e6-4caf-9290-6eb04a38927d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:16.291244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6b9e2-00ce-4c23-a878-fee569d6fc0f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:16.298906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:16.299667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:16.299949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:16.302660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] PATH_INFO: `/v3/projects/34a9201c85c24d61a8ed55a7cec69d67` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:16.306745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a9201c85c24d61a8ed55a7cec69d67)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:16.309370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efa284-ad99-490f-882b-69327ebf36f2 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:16.310043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efa284-ad99-490f-882b-69327ebf36f2 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:16.310323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efa284-ad99-490f-882b-69327ebf36f2 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:16.313480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:16.315115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efa284-ad99-490f-882b-69327ebf36f2 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/db3604c2ccf24d8fb076cecc5bf34013/users/e741a510294e45afadf6605699edbee1/roles/2dcbcd4cc0a94e579242fcef27a6f0b5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:16.352207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efa284-ad99-490f-882b-69327ebf36f2 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=db3604c2ccf24d8fb076cecc5bf34013, user_id=e741a510294e45afadf6605699edbee1, role_id=2dcbcd4cc0a94e579242fcef27a6f0b5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:16.356776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efa284-ad99-490f-882b-69327ebf36f2 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:16.390942 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2175/4309] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:16 2022] PUT /identity/v3/projects/db3604c2ccf24d8fb076cecc5bf34013/users/e741a510294e45afadf6605699edbee1/roles/2dcbcd4cc0a94e579242fcef27a6f0b5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:16.395255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efa284-ad99-490f-882b-69327ebf36f2 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:16.462543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a9201c85c24d61a8ed55a7cec69d67', 'request_id': 'req-f8fe8345-94da-444a-8628-85918a7637b5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:16.481788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a9201c85c24d61a8ed55a7cec69d67', 'request_id': 'req-f8fe8345-94da-444a-8628-85918a7637b5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:16.491657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a9201c85c24d61a8ed55a7cec69d67', 'request_id': 'req-f8fe8345-94da-444a-8628-85918a7637b5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:16.516951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:16.522570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:16.549325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] The token cache is being invalidate because project 34a9201c85c24d61a8ed55a7cec69d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:16.549620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:16.549993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efa284-ad99-490f-882b-69327ebf36f2 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:16.552614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:16.557135 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2135/4310] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:16 2022] DELETE /identity/v3/projects/34a9201c85c24d61a8ed55a7cec69d67 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:16.557135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:16.560864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:16.674994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78401e89-b6cc-4acf-b600-0426bb597300 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:16.677130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78401e89-b6cc-4acf-b600-0426bb597300 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:16.677485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78401e89-b6cc-4acf-b600-0426bb597300 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:16.679326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78401e89-b6cc-4acf-b600-0426bb597300 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/db3604c2ccf24d8fb076cecc5bf34013/users/e741a510294e45afadf6605699edbee1/roles/70689ceba97f4a3b8463e9cadbcb0ccf` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:16.685308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78401e89-b6cc-4acf-b600-0426bb597300 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=db3604c2ccf24d8fb076cecc5bf34013, user_id=e741a510294e45afadf6605699edbee1, role_id=70689ceba97f4a3b8463e9cadbcb0ccf)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:16.689967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78401e89-b6cc-4acf-b600-0426bb597300 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:16.726757 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2176/4311] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:16 2022] PUT /identity/v3/projects/db3604c2ccf24d8fb076cecc5bf34013/users/e741a510294e45afadf6605699edbee1/roles/70689ceba97f4a3b8463e9cadbcb0ccf => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:16.726757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78401e89-b6cc-4acf-b600-0426bb597300 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:16.728983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-78401e89-b6cc-4acf-b600-0426bb597300 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:16.771637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe8345-94da-444a-8628-85918a7637b5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:16.928046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78401e89-b6cc-4acf-b600-0426bb597300 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:16.938404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:16.939327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:16.939704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:16.940105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] PATH_INFO: `/v3/users/de947d89fd994f6eb9c492b202c41520` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:16.941757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de947d89fd994f6eb9c492b202c41520)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:16.945261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:16.974330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:16.974935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:16.981433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:16.981797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] PATH_INFO: `/v3/projects/c04b2976577d42799a800d21545d1969` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:16.988791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c04b2976577d42799a800d21545d1969)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:16.991891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:17.072038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de947d89fd994f6eb9c492b202c41520'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.087011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c04b2976577d42799a800d21545d1969', 'request_id': 'req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.100928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de947d89fd994f6eb9c492b202c41520'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.107084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c04b2976577d42799a800d21545d1969', 'request_id': 'req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.115176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de947d89fd994f6eb9c492b202c41520'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.121091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c04b2976577d42799a800d21545d1969', 'request_id': 'req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.135677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de947d89fd994f6eb9c492b202c41520'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.138209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f05976e2-0605-4d0e-b653-7ea504bba23f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de947d89fd994f6eb9c492b202c41520'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.147575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] The token cache is being invalidate because project c04b2976577d42799a800d21545d1969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:17.147924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.148618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.156634 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2136/4312] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:16 2022] DELETE /identity/v3/projects/c04b2976577d42799a800d21545d1969 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:17.156634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:17.156634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:17.177372 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2177/4313] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:16 2022] DELETE /identity/v3/users/de947d89fd994f6eb9c492b202c41520 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:17.179148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618cb65-b8aa-4cc6-a1c8-daa95e46664b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:17.181166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618cb65-b8aa-4cc6-a1c8-daa95e46664b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:17.182030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618cb65-b8aa-4cc6-a1c8-daa95e46664b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:17.235118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9618cb65-b8aa-4cc6-a1c8-daa95e46664b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:17.238420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9618cb65-b8aa-4cc6-a1c8-daa95e46664b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:17.248138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:17.255621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:17.352503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceb1e6-7f56-4ef8-86de-6f1d01d851b2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:17.383031 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2178/4314] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:17.389196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261719a-e358-4ae7-aeda-d552cdee7f36 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:17.389577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261719a-e358-4ae7-aeda-d552cdee7f36 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:17.389857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261719a-e358-4ae7-aeda-d552cdee7f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:17.446189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1261719a-e358-4ae7-aeda-d552cdee7f36 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:17.446645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1261719a-e358-4ae7-aeda-d552cdee7f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:17.471511 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2179/4315] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:17.472292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0f586-0f7b-4975-9491-5c70400bddb4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:17.473526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0f586-0f7b-4975-9491-5c70400bddb4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:17.474745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0f586-0f7b-4975-9491-5c70400bddb4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:17.475196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0f586-0f7b-4975-9491-5c70400bddb4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:17.475748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261719a-e358-4ae7-aeda-d552cdee7f36 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:17.476353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0f586-0f7b-4975-9491-5c70400bddb4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:17.478805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0f586-0f7b-4975-9491-5c70400bddb4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:17.492256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261719a-e358-4ae7-aeda-d552cdee7f36 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:17.498291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa58b25-5cd9-46e9-af2a-12b7e7bdc136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:17.499374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa58b25-5cd9-46e9-af2a-12b7e7bdc136 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:17.499848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa58b25-5cd9-46e9-af2a-12b7e7bdc136 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:17.500295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa58b25-5cd9-46e9-af2a-12b7e7bdc136 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:17.501829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa58b25-5cd9-46e9-af2a-12b7e7bdc136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:17.502362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa58b25-5cd9-46e9-af2a-12b7e7bdc136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:17.504324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa58b25-5cd9-46e9-af2a-12b7e7bdc136 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:17.526882 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2137/4316] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:17 2022] GET /identity/v3/users => generated 7491 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:17.531953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d2000-540b-43d0-9767-35f55406e004 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:17.533260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d2000-540b-43d0-9767-35f55406e004 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:17.533539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d2000-540b-43d0-9767-35f55406e004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:17.557109 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2180/4317] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:09:17.560644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa58b25-5cd9-46e9-af2a-12b7e7bdc136 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:17.585125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa58b25-5cd9-46e9-af2a-12b7e7bdc136 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:17.598397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6cbc5-9f78-4c00-9096-d065ae09e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:17.600157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6cbc5-9f78-4c00-9096-d065ae09e7cf admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:17.600500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6cbc5-9f78-4c00-9096-d065ae09e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:17.602397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6cbc5-9f78-4c00-9096-d065ae09e7cf admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:17.603264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6cbc5-9f78-4c00-9096-d065ae09e7cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:17.608527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6cbc5-9f78-4c00-9096-d065ae09e7cf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:17.654565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d98d2000-540b-43d0-9767-35f55406e004 None None] MFA Rules not processed for user `e741a510294e45afadf6605699edbee1`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:17.655790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d98d2000-540b-43d0-9767-35f55406e004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:17.690437 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2181/4318] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:17 2022] POST /identity/v3/projects => generated 389 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:17.691329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6cbc5-9f78-4c00-9096-d065ae09e7cf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:17.809875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6cbc5-9f78-4c00-9096-d065ae09e7cf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:17.822050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5cfb9-6c93-4040-9f7d-5b7fb02a6951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:17.822556 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2138/4319] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:17.822680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5cfb9-6c93-4040-9f7d-5b7fb02a6951 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:17.824354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5cfb9-6c93-4040-9f7d-5b7fb02a6951 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:17.824512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d2000-540b-43d0-9767-35f55406e004 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:17.826556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d98d2000-540b-43d0-9767-35f55406e004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:17.836455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5cfb9-6c93-4040-9f7d-5b7fb02a6951 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:17.841739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5cfb9-6c93-4040-9f7d-5b7fb02a6951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:17.844537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5cfb9-6c93-4040-9f7d-5b7fb02a6951 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:17.853716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d2000-540b-43d0-9767-35f55406e004 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:17.884862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:17.888447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:17.888927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:17.889323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] PATH_INFO: `/v3/projects/900945ce55e14648a881f27796caaa44` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:17.892599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] RBAC: Authorizing `identity:delete_project(project_id=900945ce55e14648a881f27796caaa44)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:17.892974 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2182/4320] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:17 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:17.895572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:17.896193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b978137-e908-4860-bf38-d20e9e702ac0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:17.896544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b978137-e908-4860-bf38-d20e9e702ac0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:17.896927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b978137-e908-4860-bf38-d20e9e702ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:17.937254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3b978137-e908-4860-bf38-d20e9e702ac0 None None] MFA Rules not processed for user `e741a510294e45afadf6605699edbee1`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:17.938770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3b978137-e908-4860-bf38-d20e9e702ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:17.988044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '900945ce55e14648a881f27796caaa44', 'request_id': 'req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:17.993567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:18.004531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '900945ce55e14648a881f27796caaa44', 'request_id': 'req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:18.020360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '900945ce55e14648a881f27796caaa44', 'request_id': 'req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:18.046229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] The token cache is being invalidate because project 900945ce55e14648a881f27796caaa44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:18.046865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:18.047869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:18.050410 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2139/4321] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:17 2022] DELETE /identity/v3/projects/900945ce55e14648a881f27796caaa44 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:18.051430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:18.053265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:18.097243 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2183/4322] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:18.105976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932ef8d-f8bd-4a06-b2c9-ada419a2c558 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.106403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932ef8d-f8bd-4a06-b2c9-ada419a2c558 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.106936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932ef8d-f8bd-4a06-b2c9-ada419a2c558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.132936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:18.166030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b932ef8d-f8bd-4a06-b2c9-ada419a2c558 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:18.169845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b932ef8d-f8bd-4a06-b2c9-ada419a2c558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:18.178914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fa2d5-4e40-4762-8eea-baeed8e5b14c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:18.184297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62be5c3-c62c-4daf-9f56-30877c9bc364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:18.184944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62be5c3-c62c-4daf-9f56-30877c9bc364 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.185258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62be5c3-c62c-4daf-9f56-30877c9bc364 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.185529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62be5c3-c62c-4daf-9f56-30877c9bc364 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.186166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62be5c3-c62c-4daf-9f56-30877c9bc364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:18.187767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62be5c3-c62c-4daf-9f56-30877c9bc364 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:18.195983 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2184/4323] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:18.203575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932ef8d-f8bd-4a06-b2c9-ada419a2c558 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:18.220558 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2140/4324] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:18 2022] GET /identity/v3/roles => generated 1892 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:18.226727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeef0dd-87a1-47d8-a18f-2547dcbd90e6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.228896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeef0dd-87a1-47d8-a18f-2547dcbd90e6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.228984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeef0dd-87a1-47d8-a18f-2547dcbd90e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.272729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2aeef0dd-87a1-47d8-a18f-2547dcbd90e6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:18.278482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2aeef0dd-87a1-47d8-a18f-2547dcbd90e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:18.294673 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2141/4325] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:18.299425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeef0dd-87a1-47d8-a18f-2547dcbd90e6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:18.317526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeef0dd-87a1-47d8-a18f-2547dcbd90e6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:18.322686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932ef8d-f8bd-4a06-b2c9-ada419a2c558 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:18.325181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c40f37-09d5-4289-bf62-ef815bd14376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:18.325790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c40f37-09d5-4289-bf62-ef815bd14376 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.326063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c40f37-09d5-4289-bf62-ef815bd14376 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.326437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c40f37-09d5-4289-bf62-ef815bd14376 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.327073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c40f37-09d5-4289-bf62-ef815bd14376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:18.327474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f3881-b068-4c0d-8f7b-227cb4b5010d tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e741a510294e45afadf6605699edbee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db3604c2ccf24d8fb076cecc5bf34013', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-715316997', 'tempest-DelegatedRole-1868483118'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:18.328076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f3881-b068-4c0d-8f7b-227cb4b5010d tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.328347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f3881-b068-4c0d-8f7b-227cb4b5010d tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.328611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f3881-b068-4c0d-8f7b-227cb4b5010d tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.328737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c40f37-09d5-4289-bf62-ef815bd14376 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:18.329297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f3881-b068-4c0d-8f7b-227cb4b5010d tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] RBAC: Authorizing `identity:create_trust()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:18.330440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f3881-b068-4c0d-8f7b-227cb4b5010d tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:18.343072 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2142/4326] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:18 2022] GET /identity/v3/roles => generated 1892 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:18.343932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c40f37-09d5-4289-bf62-ef815bd14376 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:18.358390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c40f37-09d5-4289-bf62-ef815bd14376 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:18.369554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2e3fa-b758-42c3-af77-c6f4cb326c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:18.370264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2e3fa-b758-42c3-af77-c6f4cb326c9d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.371421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2e3fa-b758-42c3-af77-c6f4cb326c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.371794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2e3fa-b758-42c3-af77-c6f4cb326c9d admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.372616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2e3fa-b758-42c3-af77-c6f4cb326c9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:18.373342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2e3fa-b758-42c3-af77-c6f4cb326c9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:18.374950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2e3fa-b758-42c3-af77-c6f4cb326c9d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:18.393832 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2143/4327] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:18.394729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2e3fa-b758-42c3-af77-c6f4cb326c9d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:18.412548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2e3fa-b758-42c3-af77-c6f4cb326c9d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:18.418731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205278c-f8c3-4895-9194-376842d565a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:18.419401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205278c-f8c3-4895-9194-376842d565a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.419777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205278c-f8c3-4895-9194-376842d565a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.420174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205278c-f8c3-4895-9194-376842d565a4 admin admin] PATH_INFO: `/v3/projects/a7416e28c90a4a4a8b1fdcf7753a7bc2/users/4d029b25bf304d5c9c9115d649f106d8/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.445063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205278c-f8c3-4895-9194-376842d565a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7416e28c90a4a4a8b1fdcf7753a7bc2, user_id=4d029b25bf304d5c9c9115d649f106d8, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:18.448270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205278c-f8c3-4895-9194-376842d565a4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:18.457942 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2185/4328] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Oct 15 21:09:18 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:18.459172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f3881-b068-4c0d-8f7b-227cb4b5010d tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:18.471803 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2144/4329] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:18 2022] PUT /identity/v3/projects/a7416e28c90a4a4a8b1fdcf7753a7bc2/users/4d029b25bf304d5c9c9115d649f106d8/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:18.472682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205278c-f8c3-4895-9194-376842d565a4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:18.475529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f3881-b068-4c0d-8f7b-227cb4b5010d tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:18.602848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af3ea4-dda6-4d0a-b0b3-a484d2619483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:18.604287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af3ea4-dda6-4d0a-b0b3-a484d2619483 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.604885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af3ea4-dda6-4d0a-b0b3-a484d2619483 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.605459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af3ea4-dda6-4d0a-b0b3-a484d2619483 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.606516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af3ea4-dda6-4d0a-b0b3-a484d2619483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:18.608680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af3ea4-dda6-4d0a-b0b3-a484d2619483 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:18.626193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205278c-f8c3-4895-9194-376842d565a4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:18.630592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cf894-a376-4308-aa24-349c97903a03 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e741a510294e45afadf6605699edbee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db3604c2ccf24d8fb076cecc5bf34013', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-715316997', 'tempest-DelegatedRole-1868483118'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:18.631204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cf894-a376-4308-aa24-349c97903a03 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.631474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cf894-a376-4308-aa24-349c97903a03 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.631739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cf894-a376-4308-aa24-349c97903a03 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.632720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cf894-a376-4308-aa24-349c97903a03 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] RBAC: Authorizing `identity:list_trusts_for_trustor()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:18.634148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cf894-a376-4308-aa24-349c97903a03 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:18.645835 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2145/4330] 10.222.0.16 () {58 vars in 1252 bytes} [Sat Oct 15 21:09:18 2022] GET /identity/v3/OS-TRUST/trusts/?trustor_user_id=e741a510294e45afadf6605699edbee1 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:18.647207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19a220-f239-4237-9314-e9e6c915ea45 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.647487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19a220-f239-4237-9314-e9e6c915ea45 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.647769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19a220-f239-4237-9314-e9e6c915ea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.678047 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2186/4331] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:18 2022] POST /identity/v3/projects => generated 387 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:18.678401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af3ea4-dda6-4d0a-b0b3-a484d2619483 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:18.693870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3c19a220-f239-4237-9314-e9e6c915ea45 None None] MFA Rules not processed for user `4d029b25bf304d5c9c9115d649f106d8`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:18.695172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3c19a220-f239-4237-9314-e9e6c915ea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:18.710438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:18.778959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:18.835111 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2146/4332] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:18.836251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19a220-f239-4237-9314-e9e6c915ea45 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:18.851871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af3ea4-dda6-4d0a-b0b3-a484d2619483 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:18.858460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e741a510294e45afadf6605699edbee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db3604c2ccf24d8fb076cecc5bf34013', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-715316997', 'tempest-DelegatedRole-1868483118'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:18.859686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.860220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.862170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] PATH_INFO: `/v3/OS-TRUST/trusts/1d9b91a9772e42c5aed8125e85cbb886` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.884347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] RBAC: Authorizing `identity:delete_trust(trust_id=1d9b91a9772e42c5aed8125e85cbb886)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:18.887835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:18.947530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '1d9b91a9772e42c5aed8125e85cbb886', 'request_id': 'req-bf255622-45f2-4d40-8ec6-15422a7c6033'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:18.948413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '1d9b91a9772e42c5aed8125e85cbb886', 'request_id': 'req-bf255622-45f2-4d40-8ec6-15422a7c6033'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:18.949553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '1d9b91a9772e42c5aed8125e85cbb886', 'request_id': 'req-bf255622-45f2-4d40-8ec6-15422a7c6033'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:18.977533 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2187/4333] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Oct 15 21:09:18 2022] DELETE /identity/v3/OS-TRUST/trusts/1d9b91a9772e42c5aed8125e85cbb886 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:18.980756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:18.985598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:18.990971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19a220-f239-4237-9314-e9e6c915ea45 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:18.996757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83453ee-f957-4d25-b5e3-84cb907da2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:18.997660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83453ee-f957-4d25-b5e3-84cb907da2ed admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:18.998182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83453ee-f957-4d25-b5e3-84cb907da2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:18.998586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83453ee-f957-4d25-b5e3-84cb907da2ed admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:18.999398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83453ee-f957-4d25-b5e3-84cb907da2ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.001665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83453ee-f957-4d25-b5e3-84cb907da2ed admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.015153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf255622-45f2-4d40-8ec6-15422a7c6033 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.021648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53428312-5bdb-4206-bc51-04a7a8745a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.022322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53428312-5bdb-4206-bc51-04a7a8745a1a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.022664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53428312-5bdb-4206-bc51-04a7a8745a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.023019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53428312-5bdb-4206-bc51-04a7a8745a1a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.023730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53428312-5bdb-4206-bc51-04a7a8745a1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.025854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53428312-5bdb-4206-bc51-04a7a8745a1a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.034679 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2147/4334] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:18 2022] POST /identity/v3/users => generated 300 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:19.035817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83453ee-f957-4d25-b5e3-84cb907da2ed admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.037882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c83453ee-f957-4d25-b5e3-84cb907da2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:19.060476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83453ee-f957-4d25-b5e3-84cb907da2ed admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.067931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21ab45-ee45-44bc-846d-0e65d51d14d7 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e741a510294e45afadf6605699edbee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db3604c2ccf24d8fb076cecc5bf34013', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-715316997', 'tempest-DelegatedRole-1868483118'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.068771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21ab45-ee45-44bc-846d-0e65d51d14d7 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.068944 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2188/4335] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:18 2022] POST /identity/v3/projects => generated 391 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:19.069261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21ab45-ee45-44bc-846d-0e65d51d14d7 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.069542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21ab45-ee45-44bc-846d-0e65d51d14d7 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] PATH_INFO: `/v3/OS-TRUST/trusts/1d9b91a9772e42c5aed8125e85cbb886` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.070385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53428312-5bdb-4206-bc51-04a7a8745a1a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.074868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-53428312-5bdb-4206-bc51-04a7a8745a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-6f21ab45-ee45-44bc-846d-0e65d51d14d7 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Could not find trust: 1d9b91a9772e42c5aed8125e85cbb886.: keystone.exception.TrustNotFound: Could not find trust: 1d9b91a9772e42c5aed8125e85cbb886. Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Oct 15 21:09:19.080739 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 1d9b91a9772e42c5aed8125e85cbb886. Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:19.083132 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2148/4336] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:19 2022] GET /identity/v3/OS-TRUST/trusts/1d9b91a9772e42c5aed8125e85cbb886 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:19.084809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21ab45-ee45-44bc-846d-0e65d51d14d7 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.109112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21ab45-ee45-44bc-846d-0e65d51d14d7 tempest-TrustsV3TestJSON-1838125174 tempest-user-1091973642] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.164087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53428312-5bdb-4206-bc51-04a7a8745a1a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.171561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14977ec-d291-4fa8-a12f-012cd4fe495e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.172821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14977ec-d291-4fa8-a12f-012cd4fe495e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.172821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14977ec-d291-4fa8-a12f-012cd4fe495e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.173014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14977ec-d291-4fa8-a12f-012cd4fe495e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.173878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14977ec-d291-4fa8-a12f-012cd4fe495e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.175546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14977ec-d291-4fa8-a12f-012cd4fe495e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.191189 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2189/4337] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:19 2022] GET /identity/v3/roles => generated 1892 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:19.192744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14977ec-d291-4fa8-a12f-012cd4fe495e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.195846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9b89d-7462-4cb9-af38-491a33d15bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.196392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b14977ec-d291-4fa8-a12f-012cd4fe495e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:19.196524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b89d-7462-4cb9-af38-491a33d15bdc admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.196755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b89d-7462-4cb9-af38-491a33d15bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.197030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b89d-7462-4cb9-af38-491a33d15bdc admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.198203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9b89d-7462-4cb9-af38-491a33d15bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.202794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9b89d-7462-4cb9-af38-491a33d15bdc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.245091 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2149/4338] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:19 2022] POST /identity/v3/users => generated 302 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:19.246113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9b89d-7462-4cb9-af38-491a33d15bdc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.258717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9b89d-7462-4cb9-af38-491a33d15bdc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.265225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aa287-ab9d-457a-b011-58e8e3679b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.265889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aa287-ab9d-457a-b011-58e8e3679b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.266162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aa287-ab9d-457a-b011-58e8e3679b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.266432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aa287-ab9d-457a-b011-58e8e3679b12 admin admin] PATH_INFO: `/v3/projects/643cc125b98144ca8760f973f1c2ca37/users/23120408275c437ab72d6247415bf2c1/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.281930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3aa287-ab9d-457a-b011-58e8e3679b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643cc125b98144ca8760f973f1c2ca37, user_id=23120408275c437ab72d6247415bf2c1, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.284544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3aa287-ab9d-457a-b011-58e8e3679b12 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.294643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14977ec-d291-4fa8-a12f-012cd4fe495e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.301821 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2150/4339] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:19 2022] PUT /identity/v3/projects/643cc125b98144ca8760f973f1c2ca37/users/23120408275c437ab72d6247415bf2c1/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:19.302428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aa287-ab9d-457a-b011-58e8e3679b12 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.307475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:19.309088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:19.367113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.367747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.368414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.368414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/70689ceba97f4a3b8463e9cadbcb0ccf` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.369856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=70689ceba97f4a3b8463e9cadbcb0ccf)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.372041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.385117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aa287-ab9d-457a-b011-58e8e3679b12 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.390858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbe058-6b19-422b-a221-4472a2713ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.391473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbe058-6b19-422b-a221-4472a2713ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.391743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbe058-6b19-422b-a221-4472a2713ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.392028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbe058-6b19-422b-a221-4472a2713ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.392651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbe058-6b19-422b-a221-4472a2713ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.394281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbe058-6b19-422b-a221-4472a2713ad4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.406905 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2151/4340] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:19 2022] GET /identity/v3/roles => generated 1892 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:19.407888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbe058-6b19-422b-a221-4472a2713ad4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.419167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role 70689ceba97f4a3b8463e9cadbcb0ccf has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:19.419550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:19.420002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbe058-6b19-422b-a221-4472a2713ad4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.420351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:19.422520 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2190/4341] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:19 2022] DELETE /identity/v3/roles/70689ceba97f4a3b8463e9cadbcb0ccf => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:19.423407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.424957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:19.428456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fb7ce-3851-4741-9582-73de724a9ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.429439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fb7ce-3851-4741-9582-73de724a9ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.429439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fb7ce-3851-4741-9582-73de724a9ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.429870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fb7ce-3851-4741-9582-73de724a9ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.430672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fb7ce-3851-4741-9582-73de724a9ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.432607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fb7ce-3851-4741-9582-73de724a9ba4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.447556 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2152/4342] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:19 2022] GET /identity/v3/roles => generated 1655 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:19.447556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fb7ce-3851-4741-9582-73de724a9ba4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.449425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-159fb7ce-3851-4741-9582-73de724a9ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:19.466997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fb7ce-3851-4741-9582-73de724a9ba4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.527726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11df9d7-dfa7-4507-bccb-534de2484a70 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.534836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351a7dd-d1b4-4f06-a4b1-bb1bc910f168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.536679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a7dd-d1b4-4f06-a4b1-bb1bc910f168 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.539031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a7dd-d1b4-4f06-a4b1-bb1bc910f168 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.539401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a7dd-d1b4-4f06-a4b1-bb1bc910f168 admin admin] PATH_INFO: `/v3/projects/154c05e6fafa4f0fb4b06e4a1437cd2b/users/7f68c907c4484d3dbcb5b1d372ed3502/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.557154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.557788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.558577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.558871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/2dcbcd4cc0a94e579242fcef27a6f0b5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.559767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351a7dd-d1b4-4f06-a4b1-bb1bc910f168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=154c05e6fafa4f0fb4b06e4a1437cd2b, user_id=7f68c907c4484d3dbcb5b1d372ed3502, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.560251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2dcbcd4cc0a94e579242fcef27a6f0b5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.562244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.564279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351a7dd-d1b4-4f06-a4b1-bb1bc910f168 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.584703 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2191/4343] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:19 2022] PUT /identity/v3/projects/154c05e6fafa4f0fb4b06e4a1437cd2b/users/7f68c907c4484d3dbcb5b1d372ed3502/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:19.585730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351a7dd-d1b4-4f06-a4b1-bb1bc910f168 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.587611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d351a7dd-d1b4-4f06-a4b1-bb1bc910f168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:19.620237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role 2dcbcd4cc0a94e579242fcef27a6f0b5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:19.620568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:19.622822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:19.625506 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2153/4344] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:19 2022] DELETE /identity/v3/roles/2dcbcd4cc0a94e579242fcef27a6f0b5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:19.626253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.627542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:19.679785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351a7dd-d1b4-4f06-a4b1-bb1bc910f168 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.691735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:19.697060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:19.722820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b99eb-7cdb-40cc-9ff6-62eb14ffaae4 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.729200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743148ec-ca15-470c-9068-f21027325db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.729971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743148ec-ca15-470c-9068-f21027325db6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.730488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743148ec-ca15-470c-9068-f21027325db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.730859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743148ec-ca15-470c-9068-f21027325db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.731601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743148ec-ca15-470c-9068-f21027325db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.733353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743148ec-ca15-470c-9068-f21027325db6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.746886 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2154/4345] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:19.747944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743148ec-ca15-470c-9068-f21027325db6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.751585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-743148ec-ca15-470c-9068-f21027325db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:19.758462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58a2ec-d05b-4517-ad70-03d650564e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.759331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58a2ec-d05b-4517-ad70-03d650564e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.759608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58a2ec-d05b-4517-ad70-03d650564e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.759871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58a2ec-d05b-4517-ad70-03d650564e30 admin admin] PATH_INFO: `/v3/projects/643cc125b98144ca8760f973f1c2ca37/users/23120408275c437ab72d6247415bf2c1/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.762565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58a2ec-d05b-4517-ad70-03d650564e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643cc125b98144ca8760f973f1c2ca37, user_id=23120408275c437ab72d6247415bf2c1, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.765226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58a2ec-d05b-4517-ad70-03d650564e30 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.781642 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2192/4346] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:19 2022] PUT /identity/v3/projects/643cc125b98144ca8760f973f1c2ca37/users/23120408275c437ab72d6247415bf2c1/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:19.781642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58a2ec-d05b-4517-ad70-03d650564e30 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:19.852985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743148ec-ca15-470c-9068-f21027325db6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.863023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58a2ec-d05b-4517-ad70-03d650564e30 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:19.868861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bbe2b-2512-4f70-8e87-3fbffd5c446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.869750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bbe2b-2512-4f70-8e87-3fbffd5c446c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.870173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bbe2b-2512-4f70-8e87-3fbffd5c446c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.871401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bbe2b-2512-4f70-8e87-3fbffd5c446c admin admin] PATH_INFO: `/v3/projects/154c05e6fafa4f0fb4b06e4a1437cd2b/users/7f68c907c4484d3dbcb5b1d372ed3502/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.874537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bbe2b-2512-4f70-8e87-3fbffd5c446c admin admin] RBAC: Authorizing `identity:create_grant(project_id=154c05e6fafa4f0fb4b06e4a1437cd2b, user_id=7f68c907c4484d3dbcb5b1d372ed3502, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.879730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bbe2b-2512-4f70-8e87-3fbffd5c446c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:19.905026 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2193/4347] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:19 2022] PUT /identity/v3/projects/154c05e6fafa4f0fb4b06e4a1437cd2b/users/7f68c907c4484d3dbcb5b1d372ed3502/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:19.905026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d306-f918-4772-ae0b-7f757659f185 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.905026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d306-f918-4772-ae0b-7f757659f185 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.905026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d306-f918-4772-ae0b-7f757659f185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.934016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8c01d306-f918-4772-ae0b-7f757659f185 None None] MFA Rules not processed for user `23120408275c437ab72d6247415bf2c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:19.935388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8c01d306-f918-4772-ae0b-7f757659f185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:19.953964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:19.954605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:19.954887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:19.955592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users/e741a510294e45afadf6605699edbee1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:19.957668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_user(user_id=e741a510294e45afadf6605699edbee1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:19.959921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.038192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e741a510294e45afadf6605699edbee1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:20.047530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e741a510294e45afadf6605699edbee1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:20.048076 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2194/4348] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:20.049912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5bf0-99fa-46a5-a31d-b9cd9b1b8817 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.050147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5bf0-99fa-46a5-a31d-b9cd9b1b8817 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.050426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5bf0-99fa-46a5-a31d-b9cd9b1b8817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.063735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e741a510294e45afadf6605699edbee1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:20.075164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e741a510294e45afadf6605699edbee1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:20.075980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e741a510294e45afadf6605699edbee1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:20.084165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0c5a5bf0-99fa-46a5-a31d-b9cd9b1b8817 None None] MFA Rules not processed for user `7f68c907c4484d3dbcb5b1d372ed3502`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:20.085552 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2155/4349] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:19 2022] DELETE /identity/v3/users/e741a510294e45afadf6605699edbee1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:20.086369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.087828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:20.089012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0c5a5bf0-99fa-46a5-a31d-b9cd9b1b8817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:20.104304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:20.112282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:20.171582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a84cd-86a1-4c07-aa28-ea8898d4be18 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.176874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a74bb-0989-44d7-b4b5-7b9ba4c358f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.177654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a74bb-0989-44d7-b4b5-7b9ba4c358f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.178067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a74bb-0989-44d7-b4b5-7b9ba4c358f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.178346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a74bb-0989-44d7-b4b5-7b9ba4c358f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.178986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a74bb-0989-44d7-b4b5-7b9ba4c358f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.184131 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2195/4350] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:20 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:20.184131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a5bf0-99fa-46a5-a31d-b9cd9b1b8817 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.184425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a74bb-0989-44d7-b4b5-7b9ba4c358f5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.185011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0c5a5bf0-99fa-46a5-a31d-b9cd9b1b8817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:20.224342 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2156/4351] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:20 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:20.224342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a74bb-0989-44d7-b4b5-7b9ba4c358f5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.277952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a5bf0-99fa-46a5-a31d-b9cd9b1b8817 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.300538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a74bb-0989-44d7-b4b5-7b9ba4c358f5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.308626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bc5ba-713a-4378-b84b-081b3124f0a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.309818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bc5ba-713a-4378-b84b-081b3124f0a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.310170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bc5ba-713a-4378-b84b-081b3124f0a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.310554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bc5ba-713a-4378-b84b-081b3124f0a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.311723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bc5ba-713a-4378-b84b-081b3124f0a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.313877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bc5ba-713a-4378-b84b-081b3124f0a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.338568 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2157/4352] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:09:20 2022] POST /identity/v3/domains => generated 276 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:20.338568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bc5ba-713a-4378-b84b-081b3124f0a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.348564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.349236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.349516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.349787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/db3604c2ccf24d8fb076cecc5bf34013` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.350938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_project(project_id=db3604c2ccf24d8fb076cecc5bf34013)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.353479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.422436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bc5ba-713a-4378-b84b-081b3124f0a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.424802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3604c2ccf24d8fb076cecc5bf34013', 'request_id': 'req-9b747ce8-3b45-4392-ba35-7629fdbe31a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:20.435342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3604c2ccf24d8fb076cecc5bf34013', 'request_id': 'req-9b747ce8-3b45-4392-ba35-7629fdbe31a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:20.438119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:20.452474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3604c2ccf24d8fb076cecc5bf34013', 'request_id': 'req-9b747ce8-3b45-4392-ba35-7629fdbe31a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:20.477168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] The token cache is being invalidate because project db3604c2ccf24d8fb076cecc5bf34013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:20.477632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:20.478463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:20.481471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:20.483719 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2196/4353] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:20 2022] DELETE /identity/v3/projects/db3604c2ccf24d8fb076cecc5bf34013 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:20.484609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.492839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:20.510011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f082896-50df-4240-a8bd-ae8f55af4412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.510629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f082896-50df-4240-a8bd-ae8f55af4412 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.511075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f082896-50df-4240-a8bd-ae8f55af4412 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.511764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f082896-50df-4240-a8bd-ae8f55af4412 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.512552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f082896-50df-4240-a8bd-ae8f55af4412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.514802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f082896-50df-4240-a8bd-ae8f55af4412 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.539118 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2158/4354] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:20 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:20.542984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f082896-50df-4240-a8bd-ae8f55af4412 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.546054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3f082896-50df-4240-a8bd-ae8f55af4412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:20.582844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b747ce8-3b45-4392-ba35-7629fdbe31a0 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.591029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faed767-ce0e-4bb0-aca2-057fd2e8b151 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.591994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faed767-ce0e-4bb0-aca2-057fd2e8b151 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.592312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faed767-ce0e-4bb0-aca2-057fd2e8b151 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.593029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faed767-ce0e-4bb0-aca2-057fd2e8b151 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/46506ef720b644548b535384c28f1604/config` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.594076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faed767-ce0e-4bb0-aca2-057fd2e8b151 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=46506ef720b644548b535384c28f1604)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.595755 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-2faed767-ce0e-4bb0-aca2-057fd2e8b151 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:20.595755 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:09:20.595755 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:09:20.597671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faed767-ce0e-4bb0-aca2-057fd2e8b151 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.635352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f082896-50df-4240-a8bd-ae8f55af4412 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.642513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a058d-290e-4d3b-8b79-fecae8bccd43 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.643120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a058d-290e-4d3b-8b79-fecae8bccd43 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.643393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a058d-290e-4d3b-8b79-fecae8bccd43 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.643677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a058d-290e-4d3b-8b79-fecae8bccd43 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.644403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a058d-290e-4d3b-8b79-fecae8bccd43 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.646955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a058d-290e-4d3b-8b79-fecae8bccd43 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.659599 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2197/4355] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:09:20 2022] PUT /identity/v3/domains/46506ef720b644548b535384c28f1604/config => generated 138 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:20.661275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faed767-ce0e-4bb0-aca2-057fd2e8b151 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.663637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2faed767-ce0e-4bb0-aca2-057fd2e8b151 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:20.682161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faed767-ce0e-4bb0-aca2-057fd2e8b151 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.690417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adec49-e838-4011-b9aa-1ad76fd601e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.691032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adec49-e838-4011-b9aa-1ad76fd601e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.691369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adec49-e838-4011-b9aa-1ad76fd601e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.691647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adec49-e838-4011-b9aa-1ad76fd601e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.692303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adec49-e838-4011-b9aa-1ad76fd601e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.694393 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2159/4356] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:09:20 2022] POST /identity/v3/projects => generated 329 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:20.695424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a058d-290e-4d3b-8b79-fecae8bccd43 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.698177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adec49-e838-4011-b9aa-1ad76fd601e2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.712907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2198/4357] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:20.714612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adec49-e838-4011-b9aa-1ad76fd601e2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.729569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adec49-e838-4011-b9aa-1ad76fd601e2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.801505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a058d-290e-4d3b-8b79-fecae8bccd43 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.806723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c8632-acfb-4115-8865-63499b1d7de8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.807407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c8632-acfb-4115-8865-63499b1d7de8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.807766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c8632-acfb-4115-8865-63499b1d7de8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.808506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c8632-acfb-4115-8865-63499b1d7de8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/46506ef720b644548b535384c28f1604/config/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.810031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c8632-acfb-4115-8865-63499b1d7de8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=46506ef720b644548b535384c28f1604, group=identity)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.811275 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-bd2c8632-acfb-4115-8865-63499b1d7de8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:20.811275 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:09:20.811275 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:09:20.812376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c8632-acfb-4115-8865-63499b1d7de8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.821615 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2160/4358] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:09:20 2022] GET /identity/v3/domains/46506ef720b644548b535384c28f1604/config/identity => generated 45 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:20.823034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c8632-acfb-4115-8865-63499b1d7de8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.829961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb8e29-9089-4539-a630-de3e80ccf397 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.830809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb8e29-9089-4539-a630-de3e80ccf397 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.831193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb8e29-9089-4539-a630-de3e80ccf397 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.831570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb8e29-9089-4539-a630-de3e80ccf397 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.837467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb8e29-9089-4539-a630-de3e80ccf397 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.849877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb8e29-9089-4539-a630-de3e80ccf397 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.882608 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2199/4359] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:20 2022] POST /identity/v3/users => generated 416 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:20.882608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb8e29-9089-4539-a630-de3e80ccf397 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.895269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb8e29-9089-4539-a630-de3e80ccf397 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.904592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156a63c-aad6-4453-9b58-24bbc996d308 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.905295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156a63c-aad6-4453-9b58-24bbc996d308 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.905610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156a63c-aad6-4453-9b58-24bbc996d308 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.905883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156a63c-aad6-4453-9b58-24bbc996d308 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/46506ef720b644548b535384c28f1604/config/identity/driver` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.907751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156a63c-aad6-4453-9b58-24bbc996d308 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=46506ef720b644548b535384c28f1604, group=identity, option=driver)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.920615 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-f156a63c-aad6-4453-9b58-24bbc996d308 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:20.920615 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:09:20.920615 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:09:20.921938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156a63c-aad6-4453-9b58-24bbc996d308 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.926769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:20.929559 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2200/4360] 10.222.0.16 () {58 vars in 1296 bytes} [Sat Oct 15 21:09:20 2022] GET /identity/v3/domains/46506ef720b644548b535384c28f1604/config/identity/driver => generated 31 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:20.930402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156a63c-aad6-4453-9b58-24bbc996d308 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.939692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c8632-acfb-4115-8865-63499b1d7de8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.948858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156a63c-aad6-4453-9b58-24bbc996d308 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:20.949723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce3d5b-0f13-4265-943d-1692d4e9c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.952598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce3d5b-0f13-4265-943d-1692d4e9c104 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.952881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce3d5b-0f13-4265-943d-1692d4e9c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.953257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce3d5b-0f13-4265-943d-1692d4e9c104 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.955059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce3d5b-0f13-4265-943d-1692d4e9c104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.955219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf045ce-5e80-483b-a133-62a2bd0d4605 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:20.955988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf045ce-5e80-483b-a133-62a2bd0d4605 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:20.956270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf045ce-5e80-483b-a133-62a2bd0d4605 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:20.956540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf045ce-5e80-483b-a133-62a2bd0d4605 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:20.957568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf045ce-5e80-483b-a133-62a2bd0d4605 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:20.957974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce3d5b-0f13-4265-943d-1692d4e9c104 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.958804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf045ce-5e80-483b-a133-62a2bd0d4605 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:20.984031 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2161/4361] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:20.985435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce3d5b-0f13-4265-943d-1692d4e9c104 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:20.999802 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2201/4362] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:20 2022] POST /identity/v3/roles => generated 243 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:20.999802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf045ce-5e80-483b-a133-62a2bd0d4605 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.022872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce3d5b-0f13-4265-943d-1692d4e9c104 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.038474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36c83b-a0e2-46c9-9589-d5f4882de751 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:21.039096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf045ce-5e80-483b-a133-62a2bd0d4605 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.039555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36c83b-a0e2-46c9-9589-d5f4882de751 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.040052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36c83b-a0e2-46c9-9589-d5f4882de751 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.040458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36c83b-a0e2-46c9-9589-d5f4882de751 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/46506ef720b644548b535384c28f1604/config/ldap` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.041989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36c83b-a0e2-46c9-9589-d5f4882de751 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=46506ef720b644548b535384c28f1604, group=ldap)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:21.044548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36c83b-a0e2-46c9-9589-d5f4882de751 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:21.055075 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2162/4363] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:09:20 2022] GET /identity/v3/domains/46506ef720b644548b535384c28f1604/config/ldap => generated 106 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:21.056141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36c83b-a0e2-46c9-9589-d5f4882de751 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.065837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bde14-5de2-4d86-abf1-294740c8c138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:21.066535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bde14-5de2-4d86-abf1-294740c8c138 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.066817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bde14-5de2-4d86-abf1-294740c8c138 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.067098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bde14-5de2-4d86-abf1-294740c8c138 admin admin] PATH_INFO: `/v3/projects/543f9dfa44a044e0950b04b7848aa1e0/users/3ebb8e252e6647af801b85689ca39e73/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.087376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:21.097272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bde14-5de2-4d86-abf1-294740c8c138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=543f9dfa44a044e0950b04b7848aa1e0, user_id=3ebb8e252e6647af801b85689ca39e73, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:21.100209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bde14-5de2-4d86-abf1-294740c8c138 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:21.105246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36c83b-a0e2-46c9-9589-d5f4882de751 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.119828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22200f81-1a86-4f91-b5c3-c5e315e01305 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:21.120087 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2202/4364] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:20 2022] PUT /identity/v3/projects/543f9dfa44a044e0950b04b7848aa1e0/users/3ebb8e252e6647af801b85689ca39e73/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:21.121888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22200f81-1a86-4f91-b5c3-c5e315e01305 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.122345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22200f81-1a86-4f91-b5c3-c5e315e01305 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.122789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22200f81-1a86-4f91-b5c3-c5e315e01305 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.124051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22200f81-1a86-4f91-b5c3-c5e315e01305 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:21.127338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22200f81-1a86-4f91-b5c3-c5e315e01305 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:21.133247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493347b-b8f5-4e60-a908-b8fb3dcd37c6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.134259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493347b-b8f5-4e60-a908-b8fb3dcd37c6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.135023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493347b-b8f5-4e60-a908-b8fb3dcd37c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.156205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2163/4365] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:21 2022] POST /identity/v3/roles => generated 247 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:21.156205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22200f81-1a86-4f91-b5c3-c5e315e01305 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.176121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22200f81-1a86-4f91-b5c3-c5e315e01305 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.224116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d493347b-b8f5-4e60-a908-b8fb3dcd37c6 None None] MFA Rules not processed for user `1aadd87b97c247cb95321664ba0c32a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:21.227992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d493347b-b8f5-4e60-a908-b8fb3dcd37c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:21.272997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dfb69-5eb2-44de-93ab-02c2bda8f0f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:21.273768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dfb69-5eb2-44de-93ab-02c2bda8f0f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.274270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dfb69-5eb2-44de-93ab-02c2bda8f0f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.275193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dfb69-5eb2-44de-93ab-02c2bda8f0f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/46506ef720b644548b535384c28f1604/config/ldap/url` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.277262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dfb69-5eb2-44de-93ab-02c2bda8f0f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=46506ef720b644548b535384c28f1604, group=ldap, option=url)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:21.279381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dfb69-5eb2-44de-93ab-02c2bda8f0f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:21.290005 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2164/4366] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:09:21 2022] GET /identity/v3/domains/46506ef720b644548b535384c28f1604/config/ldap/url => generated 46 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:21.290005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56356e-7b5f-4857-872b-232bbeb890fc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.290005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56356e-7b5f-4857-872b-232bbeb890fc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.290005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56356e-7b5f-4857-872b-232bbeb890fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.318507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4e56356e-7b5f-4857-872b-232bbeb890fc None None] MFA Rules not processed for user `3ebb8e252e6647af801b85689ca39e73`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:21.319844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4e56356e-7b5f-4857-872b-232bbeb890fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:21.358422 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2203/4367] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:21.358422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493347b-b8f5-4e60-a908-b8fb3dcd37c6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.412732 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2165/4368] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:21.412732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56356e-7b5f-4857-872b-232bbeb890fc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.426817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56356e-7b5f-4857-872b-232bbeb890fc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.448194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23ed77-be22-4b38-a469-e3989a85c9a8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:21.449255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23ed77-be22-4b38-a469-e3989a85c9a8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.449569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23ed77-be22-4b38-a469-e3989a85c9a8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.449917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23ed77-be22-4b38-a469-e3989a85c9a8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/46506ef720b644548b535384c28f1604/config/ldap/user_tree_dn` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.451156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23ed77-be22-4b38-a469-e3989a85c9a8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=46506ef720b644548b535384c28f1604, group=ldap, option=user_tree_dn)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:21.453459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23ed77-be22-4b38-a469-e3989a85c9a8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:21.458815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493347b-b8f5-4e60-a908-b8fb3dcd37c6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.464326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10fca6-003e-47b8-8768-c8d29e61a0d1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:21.465243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10fca6-003e-47b8-8768-c8d29e61a0d1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.465511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10fca6-003e-47b8-8768-c8d29e61a0d1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.465622 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2166/4369] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:09:21 2022] GET /identity/v3/domains/46506ef720b644548b535384c28f1604/config/ldap/user_tree_dn => generated 63 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:21.465788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10fca6-003e-47b8-8768-c8d29e61a0d1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/2e4f7e4e7e6c49258dfd8a3b6de5f564/users/8dfef6500629450798fc836065ac0d86/roles/c240f6abeaa043c99ed42180ce01ecc1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.466680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23ed77-be22-4b38-a469-e3989a85c9a8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.468433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6a23ed77-be22-4b38-a469-e3989a85c9a8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:21.485729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10fca6-003e-47b8-8768-c8d29e61a0d1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=2e4f7e4e7e6c49258dfd8a3b6de5f564, user_id=8dfef6500629450798fc836065ac0d86, role_id=c240f6abeaa043c99ed42180ce01ecc1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:21.488581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10fca6-003e-47b8-8768-c8d29e61a0d1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:21.521098 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2204/4370] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:21 2022] PUT /identity/v3/projects/2e4f7e4e7e6c49258dfd8a3b6de5f564/users/8dfef6500629450798fc836065ac0d86/roles/c240f6abeaa043c99ed42180ce01ecc1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:21.522418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10fca6-003e-47b8-8768-c8d29e61a0d1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.558032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:21.563569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23ed77-be22-4b38-a469-e3989a85c9a8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.633626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10fca6-003e-47b8-8768-c8d29e61a0d1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.643246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ad7ab-8853-400c-95d2-a068416fd8d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:21.644002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ad7ab-8853-400c-95d2-a068416fd8d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.644372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ad7ab-8853-400c-95d2-a068416fd8d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.644713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ad7ab-8853-400c-95d2-a068416fd8d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/46506ef720b644548b535384c28f1604/config` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.645435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ad7ab-8853-400c-95d2-a068416fd8d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=46506ef720b644548b535384c28f1604)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:21.646537 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-dc4ad7ab-8853-400c-95d2-a068416fd8d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:21.646537 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:09:21.646537 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:09:21.647635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ad7ab-8853-400c-95d2-a068416fd8d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:21.647769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa77adc-5275-4451-b720-f48dead640f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:21.648516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa77adc-5275-4451-b720-f48dead640f3 service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.648861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa77adc-5275-4451-b720-f48dead640f3 service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.649222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa77adc-5275-4451-b720-f48dead640f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.651179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eaa77adc-5275-4451-b720-f48dead640f3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:21.664907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2205/4371] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:21 2022] DELETE /identity/v3/domains/46506ef720b644548b535384c28f1604/config => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:21.666393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ad7ab-8853-400c-95d2-a068416fd8d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.674968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:21.741590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa77adc-5275-4451-b720-f48dead640f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:21.743054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa77adc-5275-4451-b720-f48dead640f3 service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:21.752858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ad7ab-8853-400c-95d2-a068416fd8d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.757320 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2167/4372] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:21.759208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016ef05-f1c9-4111-a189-59c156e5877f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:21.759967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016ef05-f1c9-4111-a189-59c156e5877f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.760340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016ef05-f1c9-4111-a189-59c156e5877f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.760710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016ef05-f1c9-4111-a189-59c156e5877f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/2e4f7e4e7e6c49258dfd8a3b6de5f564/users/8dfef6500629450798fc836065ac0d86/roles/0f233ffbce7740dc9d5d9bfa258dee80` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.762332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa77adc-5275-4451-b720-f48dead640f3 service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.764103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016ef05-f1c9-4111-a189-59c156e5877f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=2e4f7e4e7e6c49258dfd8a3b6de5f564, user_id=8dfef6500629450798fc836065ac0d86, role_id=0f233ffbce7740dc9d5d9bfa258dee80)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:21.766969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016ef05-f1c9-4111-a189-59c156e5877f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:21.776777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa77adc-5275-4451-b720-f48dead640f3 service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.786565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:21.787582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:21.788076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:21.788553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/46506ef720b644548b535384c28f1604` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:21.790339 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2206/4373] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:21 2022] PUT /identity/v3/projects/2e4f7e4e7e6c49258dfd8a3b6de5f564/users/8dfef6500629450798fc836065ac0d86/roles/0f233ffbce7740dc9d5d9bfa258dee80 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:21.791066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=46506ef720b644548b535384c28f1604)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:21.793234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:21.794695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016ef05-f1c9-4111-a189-59c156e5877f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.833468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '46506ef720b644548b535384c28f1604'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:21.834346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '46506ef720b644548b535384c28f1604'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:21.870857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '46506ef720b644548b535384c28f1604'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:21.871924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '46506ef720b644548b535384c28f1604'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:21.879280 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2168/4374] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:21 2022] PATCH /identity/v3/domains/46506ef720b644548b535384c28f1604 => generated 277 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:21.883384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:21.890609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:21.896245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016ef05-f1c9-4111-a189-59c156e5877f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:21.994658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95e09e-3b94-41a7-89dd-9e879e542248 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:22.000146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:22.000303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731d4da-88d8-4e60-8ab1-ae31c5c9b927 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:22.001709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731d4da-88d8-4e60-8ab1-ae31c5c9b927 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.002935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731d4da-88d8-4e60-8ab1-ae31c5c9b927 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.003090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731d4da-88d8-4e60-8ab1-ae31c5c9b927 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.004009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731d4da-88d8-4e60-8ab1-ae31c5c9b927 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:22.005658 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-b731d4da-88d8-4e60-8ab1-ae31c5c9b927 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:22.005658 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:09:22.005658 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:09:22.008007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731d4da-88d8-4e60-8ab1-ae31c5c9b927 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:22.008797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.009264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.009691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/46506ef720b644548b535384c28f1604` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.064224 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2207/4375] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:21 2022] GET /identity/v3/users => generated 8645 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:22.066356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=46506ef720b644548b535384c28f1604)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:22.070586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:22.070922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b8afb-3a3c-4ebf-a7f0-9e7200fbc2e8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.071765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b8afb-3a3c-4ebf-a7f0-9e7200fbc2e8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.072250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b8afb-3a3c-4ebf-a7f0-9e7200fbc2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.112597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:22.140472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '46506ef720b644548b535384c28f1604'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.154075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '46506ef720b644548b535384c28f1604'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.194673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f52b8afb-3a3c-4ebf-a7f0-9e7200fbc2e8 None None] MFA Rules not processed for user `8dfef6500629450798fc836065ac0d86`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:22.194673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f52b8afb-3a3c-4ebf-a7f0-9e7200fbc2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:22.241556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:22.243990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46506ef720b644548b535384c28f1604', 'request_id': 'req-247cc32e-5e99-42df-9fc2-deadb51ca84e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.259569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46506ef720b644548b535384c28f1604', 'request_id': 'req-247cc32e-5e99-42df-9fc2-deadb51ca84e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.275163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46506ef720b644548b535384c28f1604', 'request_id': 'req-247cc32e-5e99-42df-9fc2-deadb51ca84e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.292970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] The token cache is being invalidate because project 46506ef720b644548b535384c28f1604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:22.294488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.295386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.313189 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2208/4376] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:22.313189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b8afb-3a3c-4ebf-a7f0-9e7200fbc2e8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:22.313189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f52b8afb-3a3c-4ebf-a7f0-9e7200fbc2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:22.316589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '46506ef720b644548b535384c28f1604', 'request_id': 'req-247cc32e-5e99-42df-9fc2-deadb51ca84e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.318082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '46506ef720b644548b535384c28f1604', 'request_id': 'req-247cc32e-5e99-42df-9fc2-deadb51ca84e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.319020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-247cc32e-5e99-42df-9fc2-deadb51ca84e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '46506ef720b644548b535384c28f1604', 'request_id': 'req-247cc32e-5e99-42df-9fc2-deadb51ca84e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.332215 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2169/4377] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:21 2022] DELETE /identity/v3/domains/46506ef720b644548b535384c28f1604 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:22.333969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ae3ad-2087-49e0-9a83-a15d5dd6cd95 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.334736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ae3ad-2087-49e0-9a83-a15d5dd6cd95 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.336188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ae3ad-2087-49e0-9a83-a15d5dd6cd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.360868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f77ae3ad-2087-49e0-9a83-a15d5dd6cd95 None None] MFA Rules not processed for user `8dfef6500629450798fc836065ac0d86`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:22.363057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f77ae3ad-2087-49e0-9a83-a15d5dd6cd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:22.424557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b8afb-3a3c-4ebf-a7f0-9e7200fbc2e8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:22.451251 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2170/4378] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:22.452478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ae3ad-2087-49e0-9a83-a15d5dd6cd95 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:22.454094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f77ae3ad-2087-49e0-9a83-a15d5dd6cd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:22.515124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:22.516924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.516924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.516924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] PATH_INFO: `/v3/users/23120408275c437ab72d6247415bf2c1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.518019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23120408275c437ab72d6247415bf2c1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:22.520198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:22.541819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:22.551644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ae3ad-2087-49e0-9a83-a15d5dd6cd95 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:22.556997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe82753-2d21-4d7c-9031-fc9b711e58b7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:22.558550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe82753-2d21-4d7c-9031-fc9b711e58b7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.559252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe82753-2d21-4d7c-9031-fc9b711e58b7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.559729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe82753-2d21-4d7c-9031-fc9b711e58b7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.560536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe82753-2d21-4d7c-9031-fc9b711e58b7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:22.562411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe82753-2d21-4d7c-9031-fc9b711e58b7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:22.587890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23120408275c437ab72d6247415bf2c1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.593904 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2171/4379] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:09:22 2022] POST /identity/v3/domains => generated 275 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:22.594952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe82753-2d21-4d7c-9031-fc9b711e58b7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:22.604849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23120408275c437ab72d6247415bf2c1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.617741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23120408275c437ab72d6247415bf2c1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.629158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:22.632614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23120408275c437ab72d6247415bf2c1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.633901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23120408275c437ab72d6247415bf2c1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:22.645311 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2209/4380] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:22 2022] DELETE /identity/v3/users/23120408275c437ab72d6247415bf2c1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:22.646296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:22.648629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:22.675831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe82753-2d21-4d7c-9031-fc9b711e58b7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:22.742080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c50155-0e18-4580-b039-206a099aea34 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:22.747773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c78bf7-85d4-4648-a978-0ba853ad0840 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:22.748687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c78bf7-85d4-4648-a978-0ba853ad0840 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.748992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c78bf7-85d4-4648-a978-0ba853ad0840 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.749385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c78bf7-85d4-4648-a978-0ba853ad0840 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.750071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c78bf7-85d4-4648-a978-0ba853ad0840 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=af581daa4ed24de59e1d61c6a168f52e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:22.750383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df957f2-365a-4c38-8151-5204155b0446 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfef6500629450798fc836065ac0d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-866084460', 'tempest-NotDelegatedRole-1782315395'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:22.751007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df957f2-365a-4c38-8151-5204155b0446 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.751380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df957f2-365a-4c38-8151-5204155b0446 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.751690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df957f2-365a-4c38-8151-5204155b0446 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.751956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c78bf7-85d4-4648-a978-0ba853ad0840 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:22.752434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df957f2-365a-4c38-8151-5204155b0446 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorizing `identity:create_trust()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:22.753515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df957f2-365a-4c38-8151-5204155b0446 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:22.803108 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2210/4381] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:09:22 2022] PUT /identity/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config => generated 138 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:22.803108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcd2cd-31d8-4ed7-bace-60b798817ea5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.803108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcd2cd-31d8-4ed7-bace-60b798817ea5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.803108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcd2cd-31d8-4ed7-bace-60b798817ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.831511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-53fcd2cd-31d8-4ed7-bace-60b798817ea5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:22.832768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-53fcd2cd-31d8-4ed7-bace-60b798817ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:22.851473 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2172/4382] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Oct 15 21:09:22 2022] POST /identity/v3/OS-TRUST/trusts => generated 858 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:22.852585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df957f2-365a-4c38-8151-5204155b0446 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:22.871717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df957f2-365a-4c38-8151-5204155b0446 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:22.877724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47409cf-d773-42c5-a644-350f89c27b4f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:22.878600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47409cf-d773-42c5-a644-350f89c27b4f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.878948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47409cf-d773-42c5-a644-350f89c27b4f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.879416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47409cf-d773-42c5-a644-350f89c27b4f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.880449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47409cf-d773-42c5-a644-350f89c27b4f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=af581daa4ed24de59e1d61c6a168f52e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:22.881584 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-e47409cf-d773-42c5-a644-350f89c27b4f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:22.881584 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:09:22.881584 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:09:22.882625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47409cf-d773-42c5-a644-350f89c27b4f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:22.916476 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2173/4383] 10.222.0.16 () {60 vars in 1271 bytes} [Sat Oct 15 21:09:22 2022] PATCH /identity/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config => generated 142 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:22.921172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47409cf-d773-42c5-a644-350f89c27b4f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:22.921962 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2211/4384] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:22.922964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcd2cd-31d8-4ed7-bace-60b798817ea5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:22.923089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e47409cf-d773-42c5-a644-350f89c27b4f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:22.935148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcd2cd-31d8-4ed7-bace-60b798817ea5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:22.941458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a423e-0a7e-46bc-a0ce-ee9e59eda63f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:22.942298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a423e-0a7e-46bc-a0ce-ee9e59eda63f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.942649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a423e-0a7e-46bc-a0ce-ee9e59eda63f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.942972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a423e-0a7e-46bc-a0ce-ee9e59eda63f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.943397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47409cf-d773-42c5-a644-350f89c27b4f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:22.944264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a423e-0a7e-46bc-a0ce-ee9e59eda63f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=af581daa4ed24de59e1d61c6a168f52e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:22.946088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a423e-0a7e-46bc-a0ce-ee9e59eda63f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:22.949825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f68f77-0554-4ca4-8e25-026cc2978fbe tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfef6500629450798fc836065ac0d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-866084460', 'tempest-NotDelegatedRole-1782315395'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:22.951031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f68f77-0554-4ca4-8e25-026cc2978fbe tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:22.951588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f68f77-0554-4ca4-8e25-026cc2978fbe tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:22.952066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f68f77-0554-4ca4-8e25-026cc2978fbe tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] PATH_INFO: `/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:22.952993 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2212/4385] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:09:22 2022] GET /identity/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config => generated 142 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:22.953973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a423e-0a7e-46bc-a0ce-ee9e59eda63f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:22.956112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bb6a423e-0a7e-46bc-a0ce-ee9e59eda63f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:22.963538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:22.967183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f68f77-0554-4ca4-8e25-026cc2978fbe tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorizing `identity:get_trust(trust_id=ce43eb6813074039b529474ccb3ed955)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:22.969953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f68f77-0554-4ca4-8e25-026cc2978fbe tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:22.990218 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2174/4386] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:22 2022] GET /identity/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955 => generated 858 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:22.991829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f68f77-0554-4ca4-8e25-026cc2978fbe tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.009658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f68f77-0554-4ca4-8e25-026cc2978fbe tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.016112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babfaf1-e87e-4341-967c-290ec2608fb9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.016679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babfaf1-e87e-4341-967c-290ec2608fb9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.017071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babfaf1-e87e-4341-967c-290ec2608fb9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.017340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babfaf1-e87e-4341-967c-290ec2608fb9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.018231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babfaf1-e87e-4341-967c-290ec2608fb9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=af581daa4ed24de59e1d61c6a168f52e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.019460 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-6babfaf1-e87e-4341-967c-290ec2608fb9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:23.019460 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:09:23.019460 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:09:23.021150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babfaf1-e87e-4341-967c-290ec2608fb9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.032432 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2175/4387] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:22 2022] DELETE /identity/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:23.033523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babfaf1-e87e-4341-967c-290ec2608fb9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.047092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babfaf1-e87e-4341-967c-290ec2608fb9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.054134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bae2f-9814-4b73-83a1-04af62f672a6 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfef6500629450798fc836065ac0d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-866084460', 'tempest-NotDelegatedRole-1782315395'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.054744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bae2f-9814-4b73-83a1-04af62f672a6 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.055035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bae2f-9814-4b73-83a1-04af62f672a6 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.055296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bae2f-9814-4b73-83a1-04af62f672a6 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] PATH_INFO: `/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.062695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bae2f-9814-4b73-83a1-04af62f672a6 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=ce43eb6813074039b529474ccb3ed955)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.064499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bae2f-9814-4b73-83a1-04af62f672a6 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.066064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a423e-0a7e-46bc-a0ce-ee9e59eda63f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.075217 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2176/4388] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:09:23 2022] GET /identity/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955/roles => generated 370 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:23.075217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bae2f-9814-4b73-83a1-04af62f672a6 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.079018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9a623-4e93-4251-84ca-ace1ec9d54b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.079018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a623-4e93-4251-84ca-ace1ec9d54b6 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.079018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a623-4e93-4251-84ca-ace1ec9d54b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.079018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a623-4e93-4251-84ca-ace1ec9d54b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.095569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bae2f-9814-4b73-83a1-04af62f672a6 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.100920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9a623-4e93-4251-84ca-ace1ec9d54b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.102693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9a623-4e93-4251-84ca-ace1ec9d54b6 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.103244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739ff42-8ff9-4e77-9662-65e88dc53446 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.103970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739ff42-8ff9-4e77-9662-65e88dc53446 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.104293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739ff42-8ff9-4e77-9662-65e88dc53446 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.104572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739ff42-8ff9-4e77-9662-65e88dc53446 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.105796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739ff42-8ff9-4e77-9662-65e88dc53446 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=af581daa4ed24de59e1d61c6a168f52e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.108211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:23.121618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739ff42-8ff9-4e77-9662-65e88dc53446 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.131079 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2213/4389] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:23.133619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9a623-4e93-4251-84ca-ace1ec9d54b6 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-8739ff42-8ff9-4e77-9662-65e88dc53446 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Could not find any options in domain configuration for domain af581daa4ed24de59e1d61c6a168f52e.: keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain af581daa4ed24de59e1d61c6a168f52e. Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:23.142789 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Oct 15 21:09:23.143706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Oct 15 21:09:23.143706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Oct 15 21:09:23.143706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Oct 15 21:09:23.143706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:23.143706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:23.143706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Oct 15 21:09:23.143706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Oct 15 21:09:23.143706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain af581daa4ed24de59e1d61c6a168f52e. Oct 15 21:09:23.143706 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:23.148662 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2177/4390] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:09:23 2022] GET /identity/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config => generated 151 bytes in 71 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:23.149899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739ff42-8ff9-4e77-9662-65e88dc53446 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.161081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9a623-4e93-4251-84ca-ace1ec9d54b6 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.172617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb72f0-898d-446c-8ee8-930b16230815 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfef6500629450798fc836065ac0d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-866084460', 'tempest-NotDelegatedRole-1782315395'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.175157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb72f0-898d-446c-8ee8-930b16230815 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.175924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb72f0-898d-446c-8ee8-930b16230815 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.177773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb72f0-898d-446c-8ee8-930b16230815 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] PATH_INFO: `/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955/roles/c240f6abeaa043c99ed42180ce01ecc1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.196066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb72f0-898d-446c-8ee8-930b16230815 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorizing `identity:get_role_for_trust(trust_id=ce43eb6813074039b529474ccb3ed955, role_id=c240f6abeaa043c99ed42180ce01ecc1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.196066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb72f0-898d-446c-8ee8-930b16230815 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.201446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739ff42-8ff9-4e77-9662-65e88dc53446 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.213308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6fff8-3105-4a5d-bfc4-4ebf974839f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.213989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6fff8-3105-4a5d-bfc4-4ebf974839f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.216446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6fff8-3105-4a5d-bfc4-4ebf974839f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.216924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6fff8-3105-4a5d-bfc4-4ebf974839f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.217557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6fff8-3105-4a5d-bfc4-4ebf974839f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=af581daa4ed24de59e1d61c6a168f52e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.222800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6fff8-3105-4a5d-bfc4-4ebf974839f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.226301 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2214/4391] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Oct 15 21:09:23 2022] GET /identity/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955/roles/c240f6abeaa043c99ed42180ce01ecc1 => generated 243 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:23.229852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb72f0-898d-446c-8ee8-930b16230815 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.251802 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2178/4392] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:23 2022] DELETE /identity/v3/domains/af581daa4ed24de59e1d61c6a168f52e/config => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:23.256901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6fff8-3105-4a5d-bfc4-4ebf974839f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.258864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb72f0-898d-446c-8ee8-930b16230815 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.265558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd0dea-022d-4421-932a-6c012baca26a tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfef6500629450798fc836065ac0d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-866084460', 'tempest-NotDelegatedRole-1782315395'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.267120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd0dea-022d-4421-932a-6c012baca26a tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.267544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd0dea-022d-4421-932a-6c012baca26a tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.267978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd0dea-022d-4421-932a-6c012baca26a tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] PATH_INFO: `/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955/roles/c240f6abeaa043c99ed42180ce01ecc1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.279367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd0dea-022d-4421-932a-6c012baca26a tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorizing `identity:get_role_for_trust(trust_id=ce43eb6813074039b529474ccb3ed955, role_id=c240f6abeaa043c99ed42180ce01ecc1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.281570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd0dea-022d-4421-932a-6c012baca26a tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.289096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6fff8-3105-4a5d-bfc4-4ebf974839f9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.300241 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2215/4393] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Oct 15 21:09:23 2022] HEAD /identity/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955/roles/c240f6abeaa043c99ed42180ce01ecc1 => generated 0 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Oct 15 21:09:23.305113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd0dea-022d-4421-932a-6c012baca26a tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.312944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.313996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.314809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.315316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/af581daa4ed24de59e1d61c6a168f52e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.321209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=af581daa4ed24de59e1d61c6a168f52e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.322789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.328458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd0dea-022d-4421-932a-6c012baca26a tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.333466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7078f-6348-4e1b-89a3-3b2d0c8e5362 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfef6500629450798fc836065ac0d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-866084460', 'tempest-NotDelegatedRole-1782315395'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.334110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7078f-6348-4e1b-89a3-3b2d0c8e5362 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.334389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7078f-6348-4e1b-89a3-3b2d0c8e5362 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.334658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7078f-6348-4e1b-89a3-3b2d0c8e5362 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] PATH_INFO: `/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955/roles/0f233ffbce7740dc9d5d9bfa258dee80` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.350011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7078f-6348-4e1b-89a3-3b2d0c8e5362 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorizing `identity:get_role_for_trust(trust_id=ce43eb6813074039b529474ccb3ed955, role_id=0f233ffbce7740dc9d5d9bfa258dee80)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.351495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7078f-6348-4e1b-89a3-3b2d0c8e5362 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.362700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.363695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-11d7078f-6348-4e1b-89a3-3b2d0c8e5362 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Could not find role: 0f233ffbce7740dc9d5d9bfa258dee80.: keystone.exception.RoleNotFound: Could not find role: 0f233ffbce7740dc9d5d9bfa258dee80. Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 21:09:23.379509 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 0f233ffbce7740dc9d5d9bfa258dee80. Oct 15 21:09:23.380702 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:23.382253 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2216/4394] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Oct 15 21:09:23 2022] GET /identity/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955/roles/0f233ffbce7740dc9d5d9bfa258dee80 => generated 110 bytes in 78 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:23.388342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7078f-6348-4e1b-89a3-3b2d0c8e5362 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.395943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-11d7078f-6348-4e1b-89a3-3b2d0c8e5362 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:23.424886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.426120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7078f-6348-4e1b-89a3-3b2d0c8e5362 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.427505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.434796 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2179/4395] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:23 2022] PATCH /identity/v3/domains/af581daa4ed24de59e1d61c6a168f52e => generated 276 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:23.438350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.443153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:23.563646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91730f0d-19a4-411f-8894-942d9fa7679f tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfef6500629450798fc836065ac0d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-866084460', 'tempest-NotDelegatedRole-1782315395'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.565586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91730f0d-19a4-411f-8894-942d9fa7679f tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.565970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91730f0d-19a4-411f-8894-942d9fa7679f tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.566315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91730f0d-19a4-411f-8894-942d9fa7679f tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] PATH_INFO: `/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955/roles/0f233ffbce7740dc9d5d9bfa258dee80` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.576310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91730f0d-19a4-411f-8894-942d9fa7679f tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorizing `identity:get_role_for_trust(trust_id=ce43eb6813074039b529474ccb3ed955, role_id=0f233ffbce7740dc9d5d9bfa258dee80)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.578459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91730f0d-19a4-411f-8894-942d9fa7679f tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-91730f0d-19a4-411f-8894-942d9fa7679f tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Could not find role: 0f233ffbce7740dc9d5d9bfa258dee80.: keystone.exception.RoleNotFound: Could not find role: 0f233ffbce7740dc9d5d9bfa258dee80. Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 21:09:23.591581 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 0f233ffbce7740dc9d5d9bfa258dee80. Oct 15 21:09:23.592902 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:23.611828 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2217/4396] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Oct 15 21:09:23 2022] HEAD /identity/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955/roles/0f233ffbce7740dc9d5d9bfa258dee80 => generated 0 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Oct 15 21:09:23.614008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91730f0d-19a4-411f-8894-942d9fa7679f tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.615696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f40817-ccf3-49a5-a351-8a14a56a4eb6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.618441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-91730f0d-19a4-411f-8894-942d9fa7679f tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:23.634075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.634731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.635010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.635280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/af581daa4ed24de59e1d61c6a168f52e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.648187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91730f0d-19a4-411f-8894-942d9fa7679f tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.655558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=af581daa4ed24de59e1d61c6a168f52e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.659234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.663691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfef6500629450798fc836065ac0d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-866084460', 'tempest-NotDelegatedRole-1782315395'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:23.664497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:23.666372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:23.666665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] PATH_INFO: `/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:23.688073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorizing `identity:delete_trust(trust_id=ce43eb6813074039b529474ccb3ed955)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:23.690295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:23.726057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.736527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'ce43eb6813074039b529474ccb3ed955', 'request_id': 'req-6e48fa42-467a-43ec-adc5-d14b3f4cf231'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.738255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'ce43eb6813074039b529474ccb3ed955', 'request_id': 'req-6e48fa42-467a-43ec-adc5-d14b3f4cf231'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.739923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'ce43eb6813074039b529474ccb3ed955', 'request_id': 'req-6e48fa42-467a-43ec-adc5-d14b3f4cf231'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.759451 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2218/4397] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Oct 15 21:09:23 2022] DELETE /identity/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:23.760494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.762884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:23.770346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.809796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:UPDATE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:23.842204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e', 'request_id': 'req-5eca7152-438e-4b50-ba6e-86773f674edc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.856272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e', 'request_id': 'req-5eca7152-438e-4b50-ba6e-86773f674edc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.866745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e', 'request_id': 'req-5eca7152-438e-4b50-ba6e-86773f674edc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.879682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] The token cache is being invalidate because project af581daa4ed24de59e1d61c6a168f52e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:23.880053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.880830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.889021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48fa42-467a-43ec-adc5-d14b3f4cf231 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:23.894310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:23.905719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e', 'request_id': 'req-5eca7152-438e-4b50-ba6e-86773f674edc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.908126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e', 'request_id': 'req-5eca7152-438e-4b50-ba6e-86773f674edc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.909796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'af581daa4ed24de59e1d61c6a168f52e', 'request_id': 'req-5eca7152-438e-4b50-ba6e-86773f674edc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:23.926223 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2180/4398] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:23 2022] DELETE /identity/v3/domains/af581daa4ed24de59e1d61c6a168f52e => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:23.927163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:23.930372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:24.005731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:24.006439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:24.006717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:24.006983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] PATH_INFO: `/v3/projects/643cc125b98144ca8760f973f1c2ca37` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:24.008574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=643cc125b98144ca8760f973f1c2ca37)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:24.011779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:24.074740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca7152-438e-4b50-ba6e-86773f674edc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:24.087903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643cc125b98144ca8760f973f1c2ca37', 'request_id': 'req-4f741157-a4d1-4985-b106-962b87696ed3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.105654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643cc125b98144ca8760f973f1c2ca37', 'request_id': 'req-4f741157-a4d1-4985-b106-962b87696ed3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.115129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643cc125b98144ca8760f973f1c2ca37', 'request_id': 'req-4f741157-a4d1-4985-b106-962b87696ed3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.132577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] The token cache is being invalidate because project 643cc125b98144ca8760f973f1c2ca37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:24.132916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.133640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.135726 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2219/4399] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:23 2022] DELETE /identity/v3/projects/643cc125b98144ca8760f973f1c2ca37 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:24.137191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:24.139641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:24.155459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8370ac5-0a3a-4817-83a6-714f6b632df0 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfef6500629450798fc836065ac0d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-866084460', 'tempest-NotDelegatedRole-1782315395'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:24.156358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8370ac5-0a3a-4817-83a6-714f6b632df0 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:24.156641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8370ac5-0a3a-4817-83a6-714f6b632df0 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:24.156914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8370ac5-0a3a-4817-83a6-714f6b632df0 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] PATH_INFO: `/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-c8370ac5-0a3a-4817-83a6-714f6b632df0 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Could not find trust: ce43eb6813074039b529474ccb3ed955.: keystone.exception.TrustNotFound: Could not find trust: ce43eb6813074039b529474ccb3ed955. Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Oct 15 21:09:24.165999 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: ce43eb6813074039b529474ccb3ed955. Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:24.167383 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2181/4400] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:23 2022] GET /identity/v3/OS-TRUST/trusts/ce43eb6813074039b529474ccb3ed955 => generated 111 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:24.168276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8370ac5-0a3a-4817-83a6-714f6b632df0 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:24.176427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c8370ac5-0a3a-4817-83a6-714f6b632df0 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:24.256061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f741157-a4d1-4985-b106-962b87696ed3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:24.263162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb636958-c3a1-41a4-8e39-60d80a66c517 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:24.263875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb636958-c3a1-41a4-8e39-60d80a66c517 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:24.264162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb636958-c3a1-41a4-8e39-60d80a66c517 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:24.264431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb636958-c3a1-41a4-8e39-60d80a66c517 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:24.265107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb636958-c3a1-41a4-8e39-60d80a66c517 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:24.266730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb636958-c3a1-41a4-8e39-60d80a66c517 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:24.293247 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2220/4401] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:09:24 2022] POST /identity/v3/domains => generated 276 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:24.296156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:24.302242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb636958-c3a1-41a4-8e39-60d80a66c517 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:24.304527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eb636958-c3a1-41a4-8e39-60d80a66c517 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:24.314688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8370ac5-0a3a-4817-83a6-714f6b632df0 tempest-TrustsV3TestJSON-1821141800 tempest-user-1876212759] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:24.350071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:24.395142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:24.395875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:24.396226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:24.396564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] PATH_INFO: `/v3/users/3ebb8e252e6647af801b85689ca39e73` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:24.399257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ebb8e252e6647af801b85689ca39e73)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:24.401649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:24.435403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb636958-c3a1-41a4-8e39-60d80a66c517 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:24.457490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:24.458180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:24.458465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:24.459132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/0f233ffbce7740dc9d5d9bfa258dee80` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:24.460356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0f233ffbce7740dc9d5d9bfa258dee80)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:24.462021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:24.488175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ebb8e252e6647af801b85689ca39e73'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.500125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ebb8e252e6647af801b85689ca39e73'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.514455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ebb8e252e6647af801b85689ca39e73'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.528261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ebb8e252e6647af801b85689ca39e73'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.528927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ebb8e252e6647af801b85689ca39e73'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.552159 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2182/4402] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:24 2022] DELETE /identity/v3/users/3ebb8e252e6647af801b85689ca39e73 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:24.552159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:24.552159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:24.586041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role 0f233ffbce7740dc9d5d9bfa258dee80 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:24.586342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.586998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.589806 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2221/4403] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:24 2022] DELETE /identity/v3/roles/0f233ffbce7740dc9d5d9bfa258dee80 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:24.593803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:24.596246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:24.638467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc8cab-9e26-409f-9e4b-baddbaf5a700 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:24.679178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:24.729700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf78a3-a1f1-4aa3-9bcc-80dd87c96906 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:24.740786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:24.741873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:24.742699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:24.744080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/c240f6abeaa043c99ed42180ce01ecc1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:24.746410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=c240f6abeaa043c99ed42180ce01ecc1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:24.748051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9581c8-f083-48f9-b1f7-8879eb2542cc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:24.748812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9581c8-f083-48f9-b1f7-8879eb2542cc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:24.749908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9581c8-f083-48f9-b1f7-8879eb2542cc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:24.751134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9581c8-f083-48f9-b1f7-8879eb2542cc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276/config` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:24.752063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:24.752910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9581c8-f083-48f9-b1f7-8879eb2542cc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=665c2a47a50d4d93b394b925040b1276)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:24.753819 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-9a9581c8-f083-48f9-b1f7-8879eb2542cc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:24.753819 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:09:24.753819 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:09:24.756427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9581c8-f083-48f9-b1f7-8879eb2542cc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:24.833139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role c240f6abeaa043c99ed42180ce01ecc1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:24.833438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.834129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:24.837564 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2222/4404] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:24 2022] DELETE /identity/v3/roles/c240f6abeaa043c99ed42180ce01ecc1 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:24.844339 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2183/4405] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:09:24 2022] PUT /identity/v3/domains/665c2a47a50d4d93b394b925040b1276/config => generated 138 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:24.844874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:24.846396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9581c8-f083-48f9-b1f7-8879eb2542cc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:24.848319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:24.849287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9a9581c8-f083-48f9-b1f7-8879eb2542cc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:24.869826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9581c8-f083-48f9-b1f7-8879eb2542cc tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:24.955247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84057af3-d38b-40eb-9ef8-866c8287a7f2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:24.956302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84057af3-d38b-40eb-9ef8-866c8287a7f2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:24.956637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84057af3-d38b-40eb-9ef8-866c8287a7f2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:24.957400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84057af3-d38b-40eb-9ef8-866c8287a7f2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:24.958194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84057af3-d38b-40eb-9ef8-866c8287a7f2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=665c2a47a50d4d93b394b925040b1276, group=identity)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:24.960481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84057af3-d38b-40eb-9ef8-866c8287a7f2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.016802 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2184/4406] 10.222.0.16 () {60 vars in 1297 bytes} [Sat Oct 15 21:09:24 2022] PATCH /identity/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity => generated 176 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:25.021250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84057af3-d38b-40eb-9ef8-866c8287a7f2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.024347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352c315-b127-4fc2-97a7-a956c0bbe6df tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.038317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.039117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.039553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.039869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users/8dfef6500629450798fc836065ac0d86` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.041724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_user(user_id=8dfef6500629450798fc836065ac0d86)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.046231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.053229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84057af3-d38b-40eb-9ef8-866c8287a7f2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.054460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:25.072627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6f93-5ac9-45c9-8efd-05b1c080a18b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.073916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6f93-5ac9-45c9-8efd-05b1c080a18b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.074263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6f93-5ac9-45c9-8efd-05b1c080a18b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.074539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6f93-5ac9-45c9-8efd-05b1c080a18b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity/driver` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.075205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde6f93-5ac9-45c9-8efd-05b1c080a18b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=665c2a47a50d4d93b394b925040b1276, group=identity, option=driver)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.081164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde6f93-5ac9-45c9-8efd-05b1c080a18b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.123272 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2185/4407] 10.222.0.16 () {60 vars in 1318 bytes} [Sat Oct 15 21:09:25 2022] PATCH /identity/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity/driver => generated 176 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:25.124310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6f93-5ac9-45c9-8efd-05b1c080a18b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.131749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dfef6500629450798fc836065ac0d86'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.142756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6f93-5ac9-45c9-8efd-05b1c080a18b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.148405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c3562-c9ce-49b6-a9a5-4272cf2c5f3d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.149227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c3562-c9ce-49b6-a9a5-4272cf2c5f3d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.149770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c3562-c9ce-49b6-a9a5-4272cf2c5f3d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.149889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c3562-c9ce-49b6-a9a5-4272cf2c5f3d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity/driver` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.150501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c3562-c9ce-49b6-a9a5-4272cf2c5f3d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=665c2a47a50d4d93b394b925040b1276, group=identity, option=driver)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.152482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c3562-c9ce-49b6-a9a5-4272cf2c5f3d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.159269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dfef6500629450798fc836065ac0d86'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.169921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dfef6500629450798fc836065ac0d86'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.180102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dfef6500629450798fc836065ac0d86'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.180984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dfef6500629450798fc836065ac0d86'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.191540 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2223/4408] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:24 2022] DELETE /identity/v3/users/8dfef6500629450798fc836065ac0d86 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:25.196972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.199635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:25.201123 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2186/4409] 10.222.0.16 () {58 vars in 1299 bytes} [Sat Oct 15 21:09:25 2022] DELETE /identity/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity/driver => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:25.204843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c3562-c9ce-49b6-a9a5-4272cf2c5f3d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.206891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0f4c3562-c9ce-49b6-a9a5-4272cf2c5f3d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:25.225629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c3562-c9ce-49b6-a9a5-4272cf2c5f3d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.275948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:25.316288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627097a-8840-42ed-abee-5489a9ecaa32 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.318926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7123077-dfa0-476e-9560-a4cb9136716f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.319843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123077-dfa0-476e-9560-a4cb9136716f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.320324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123077-dfa0-476e-9560-a4cb9136716f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.320795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123077-dfa0-476e-9560-a4cb9136716f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity/driver` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.322117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.322553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7123077-dfa0-476e-9560-a4cb9136716f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=665c2a47a50d4d93b394b925040b1276, group=identity, option=driver)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.322754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.323036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.323386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/2e4f7e4e7e6c49258dfd8a3b6de5f564` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.324615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2e4f7e4e7e6c49258dfd8a3b6de5f564)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.325391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7123077-dfa0-476e-9560-a4cb9136716f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.327149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-e7123077-dfa0-476e-9560-a4cb9136716f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Could not find option driver in group identity in domain configuration for domain 665c2a47a50d4d93b394b925040b1276.: keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain 665c2a47a50d4d93b394b925040b1276. Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:25.346236 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Oct 15 21:09:25.347164 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Oct 15 21:09:25.347164 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Oct 15 21:09:25.347164 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Oct 15 21:09:25.347164 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:25.347164 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:25.347164 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Oct 15 21:09:25.347164 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Oct 15 21:09:25.347164 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain 665c2a47a50d4d93b394b925040b1276. Oct 15 21:09:25.347164 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:25.351100 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2187/4410] 10.222.0.16 () {58 vars in 1296 bytes} [Sat Oct 15 21:09:25 2022] GET /identity/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity/driver => generated 171 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 15 21:09:25.355542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7123077-dfa0-476e-9560-a4cb9136716f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.370428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7123077-dfa0-476e-9560-a4cb9136716f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.377539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c46990-2fc6-471b-a21f-8a29cd610c88 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.379033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c46990-2fc6-471b-a21f-8a29cd610c88 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.379280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c46990-2fc6-471b-a21f-8a29cd610c88 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.379846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c46990-2fc6-471b-a21f-8a29cd610c88 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.380929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c46990-2fc6-471b-a21f-8a29cd610c88 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=665c2a47a50d4d93b394b925040b1276, group=identity)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.383628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c46990-2fc6-471b-a21f-8a29cd610c88 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.408639 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2188/4411] 10.222.0.16 () {58 vars in 1278 bytes} [Sat Oct 15 21:09:25 2022] DELETE /identity/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:25.410086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c46990-2fc6-471b-a21f-8a29cd610c88 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.426372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c46990-2fc6-471b-a21f-8a29cd610c88 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.438051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'request_id': 'req-ffae1a19-f69a-401b-abcd-8b877cf4a91d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.455441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'request_id': 'req-ffae1a19-f69a-401b-abcd-8b877cf4a91d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.465954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e4f7e4e7e6c49258dfd8a3b6de5f564', 'request_id': 'req-ffae1a19-f69a-401b-abcd-8b877cf4a91d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.481972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] The token cache is being invalidate because project 2e4f7e4e7e6c49258dfd8a3b6de5f564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:25.482472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.483162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.485093 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2224/4412] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:25 2022] DELETE /identity/v3/projects/2e4f7e4e7e6c49258dfd8a3b6de5f564 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:25.492297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.494141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:25.505335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63b36c-ea50-4de9-a833-4613739e0610 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.506174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63b36c-ea50-4de9-a833-4613739e0610 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.506456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63b36c-ea50-4de9-a833-4613739e0610 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.506724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63b36c-ea50-4de9-a833-4613739e0610 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.506836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:25.508214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63b36c-ea50-4de9-a833-4613739e0610 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=665c2a47a50d4d93b394b925040b1276, group=identity)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.509997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63b36c-ea50-4de9-a833-4613739e0610 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-ae63b36c-ea50-4de9-a833-4613739e0610 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Could not find group identity in domain configuration for domain 665c2a47a50d4d93b394b925040b1276.: keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain 665c2a47a50d4d93b394b925040b1276. Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:25.520113 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Oct 15 21:09:25.521076 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Oct 15 21:09:25.521076 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Oct 15 21:09:25.521076 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Oct 15 21:09:25.521076 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:25.521076 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:25.521076 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Oct 15 21:09:25.521076 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Oct 15 21:09:25.521076 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain 665c2a47a50d4d93b394b925040b1276. Oct 15 21:09:25.521076 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:25.525474 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2189/4413] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:09:25 2022] GET /identity/v3/domains/665c2a47a50d4d93b394b925040b1276/config/identity => generated 154 bytes in 116 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 15 21:09:25.527160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63b36c-ea50-4de9-a833-4613739e0610 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.529174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ae63b36c-ea50-4de9-a833-4613739e0610 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:25.546485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63b36c-ea50-4de9-a833-4613739e0610 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.551653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d38e88-de88-4dc5-84d6-baeb9dd67d2e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.552411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d38e88-de88-4dc5-84d6-baeb9dd67d2e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.552689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d38e88-de88-4dc5-84d6-baeb9dd67d2e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.552959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d38e88-de88-4dc5-84d6-baeb9dd67d2e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276/config` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.553686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d38e88-de88-4dc5-84d6-baeb9dd67d2e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=665c2a47a50d4d93b394b925040b1276)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.555754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d38e88-de88-4dc5-84d6-baeb9dd67d2e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.567642 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2190/4414] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:25 2022] DELETE /identity/v3/domains/665c2a47a50d4d93b394b925040b1276/config => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:25.571716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d38e88-de88-4dc5-84d6-baeb9dd67d2e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.581389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae1a19-f69a-401b-abcd-8b877cf4a91d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.587748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d38e88-de88-4dc5-84d6-baeb9dd67d2e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.595331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f311bf-9d05-4b19-b65d-7087e73d55cc tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.595966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f311bf-9d05-4b19-b65d-7087e73d55cc tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.596379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f311bf-9d05-4b19-b65d-7087e73d55cc tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.596656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f311bf-9d05-4b19-b65d-7087e73d55cc tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.598078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f311bf-9d05-4b19-b65d-7087e73d55cc tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.600023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f311bf-9d05-4b19-b65d-7087e73d55cc tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.606119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.606904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.607187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.607459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.609254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=665c2a47a50d4d93b394b925040b1276)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.610865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.640268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '665c2a47a50d4d93b394b925040b1276'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.641098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '665c2a47a50d4d93b394b925040b1276'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.648687 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2225/4415] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:09:25 2022] POST /identity/v3/projects => generated 329 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:25.652743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f311bf-9d05-4b19-b65d-7087e73d55cc tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.654663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-96f311bf-9d05-4b19-b65d-7087e73d55cc tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:25.672062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '665c2a47a50d4d93b394b925040b1276'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.673006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '665c2a47a50d4d93b394b925040b1276'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.675827 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2191/4416] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:25 2022] PATCH /identity/v3/domains/665c2a47a50d4d93b394b925040b1276 => generated 277 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:09:25.679818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.681301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:25.737016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f311bf-9d05-4b19-b65d-7087e73d55cc tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.741244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:25.742469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090b746-8a47-443d-9bad-b64bcb579374 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.743235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090b746-8a47-443d-9bad-b64bcb579374 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.743523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090b746-8a47-443d-9bad-b64bcb579374 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.743799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090b746-8a47-443d-9bad-b64bcb579374 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.744613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090b746-8a47-443d-9bad-b64bcb579374 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.747307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090b746-8a47-443d-9bad-b64bcb579374 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.766853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d06b5-49bb-4872-b8c5-85746e8c2c8f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.776967 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2226/4417] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:25 2022] POST /identity/v3/users => generated 416 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:25.781172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090b746-8a47-443d-9bad-b64bcb579374 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.784111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0090b746-8a47-443d-9bad-b64bcb579374 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:25.808237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090b746-8a47-443d-9bad-b64bcb579374 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:25.818288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf51f55-2c07-4a1e-b993-f7d1abfbec83 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.820186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf51f55-2c07-4a1e-b993-f7d1abfbec83 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.820587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf51f55-2c07-4a1e-b993-f7d1abfbec83 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.820878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf51f55-2c07-4a1e-b993-f7d1abfbec83 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.831700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf51f55-2c07-4a1e-b993-f7d1abfbec83 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.836538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf51f55-2c07-4a1e-b993-f7d1abfbec83 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.845779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:25.846665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:25.847023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:25.847486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:25.863003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=665c2a47a50d4d93b394b925040b1276)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:25.864094 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2227/4418] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:25 2022] POST /identity/v3/roles => generated 243 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:25.865940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:25.867671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf51f55-2c07-4a1e-b993-f7d1abfbec83 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:25.872257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fbf51f55-2c07-4a1e-b993-f7d1abfbec83 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:25.933735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '665c2a47a50d4d93b394b925040b1276'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:25.955633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '665c2a47a50d4d93b394b925040b1276'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.004478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf51f55-2c07-4a1e-b993-f7d1abfbec83 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.021881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:26.024042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '665c2a47a50d4d93b394b925040b1276', 'request_id': 'req-45394f11-e81d-415b-a0bc-523a38f2b0b6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.038695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '665c2a47a50d4d93b394b925040b1276', 'request_id': 'req-45394f11-e81d-415b-a0bc-523a38f2b0b6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.051670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '665c2a47a50d4d93b394b925040b1276', 'request_id': 'req-45394f11-e81d-415b-a0bc-523a38f2b0b6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.071482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] The token cache is being invalidate because project 665c2a47a50d4d93b394b925040b1276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:26.071764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.073324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.083816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '665c2a47a50d4d93b394b925040b1276', 'request_id': 'req-45394f11-e81d-415b-a0bc-523a38f2b0b6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.084554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '665c2a47a50d4d93b394b925040b1276', 'request_id': 'req-45394f11-e81d-415b-a0bc-523a38f2b0b6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.085250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '665c2a47a50d4d93b394b925040b1276', 'request_id': 'req-45394f11-e81d-415b-a0bc-523a38f2b0b6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.093090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.095871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.096009 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2192/4419] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:25 2022] DELETE /identity/v3/domains/665c2a47a50d4d93b394b925040b1276 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:26.096463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.096907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.098126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] PATH_INFO: `/v3/projects/543f9dfa44a044e0950b04b7848aa1e0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.099901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=543f9dfa44a044e0950b04b7848aa1e0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.100574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:26.102830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.180143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:26.194919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543f9dfa44a044e0950b04b7848aa1e0', 'request_id': 'req-4285d6e3-aa7d-4397-81c4-834fe875c7a5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.217224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543f9dfa44a044e0950b04b7848aa1e0', 'request_id': 'req-4285d6e3-aa7d-4397-81c4-834fe875c7a5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.217497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394f11-e81d-415b-a0bc-523a38f2b0b6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.227477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543f9dfa44a044e0950b04b7848aa1e0', 'request_id': 'req-4285d6e3-aa7d-4397-81c4-834fe875c7a5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.260424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] The token cache is being invalidate because project 543f9dfa44a044e0950b04b7848aa1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:26.261520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.265093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:26.271494 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2228/4420] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:25 2022] DELETE /identity/v3/projects/543f9dfa44a044e0950b04b7848aa1e0 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:26.271494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.271494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:26.324485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0979ccf-b513-41ab-87f5-3e113201e1e1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.325167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0979ccf-b513-41ab-87f5-3e113201e1e1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.325441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0979ccf-b513-41ab-87f5-3e113201e1e1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.325806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0979ccf-b513-41ab-87f5-3e113201e1e1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.326356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0979ccf-b513-41ab-87f5-3e113201e1e1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.328104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0979ccf-b513-41ab-87f5-3e113201e1e1 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.349084 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2193/4421] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:26 2022] POST /identity/v3/roles => generated 246 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:26.349260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285d6e3-aa7d-4397-81c4-834fe875c7a5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.349755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d8419-50d3-4032-bc53-96ee6763932f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.349892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d8419-50d3-4032-bc53-96ee6763932f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.350158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d8419-50d3-4032-bc53-96ee6763932f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.356476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675520-5342-4a9d-851f-f85163d7519e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.357376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675520-5342-4a9d-851f-f85163d7519e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.357753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675520-5342-4a9d-851f-f85163d7519e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.358146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675520-5342-4a9d-851f-f85163d7519e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.358775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675520-5342-4a9d-851f-f85163d7519e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.360645 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-aa675520-5342-4a9d-851f-f85163d7519e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:26.360645 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:09:26.360645 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:09:26.361911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675520-5342-4a9d-851f-f85163d7519e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.370662 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2229/4422] 10.222.0.16 () {58 vars in 1173 bytes} [Sat Oct 15 21:09:26 2022] GET /identity/v3/domains/config/default => generated 1527 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:26.371960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675520-5342-4a9d-851f-f85163d7519e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.373774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aa675520-5342-4a9d-851f-f85163d7519e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:26.375400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a20d8419-50d3-4032-bc53-96ee6763932f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:26.376587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a20d8419-50d3-4032-bc53-96ee6763932f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:26.388587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675520-5342-4a9d-851f-f85163d7519e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.393777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55321-2c80-43ba-bb30-745b8c92cb47 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.394505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55321-2c80-43ba-bb30-745b8c92cb47 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.394839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55321-2c80-43ba-bb30-745b8c92cb47 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.395120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55321-2c80-43ba-bb30-745b8c92cb47 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/eab69bd688fa4d969422aa1318965e4e/users/1c58c240167e4c5baa3d2f0d3ae8b475/roles/7c7c0104497246218082819eada78d64` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.411805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f55321-2c80-43ba-bb30-745b8c92cb47 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=eab69bd688fa4d969422aa1318965e4e, user_id=1c58c240167e4c5baa3d2f0d3ae8b475, role_id=7c7c0104497246218082819eada78d64)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.414800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f55321-2c80-43ba-bb30-745b8c92cb47 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.432601 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2230/4423] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:26 2022] PUT /identity/v3/projects/eab69bd688fa4d969422aa1318965e4e/users/1c58c240167e4c5baa3d2f0d3ae8b475/roles/7c7c0104497246218082819eada78d64 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:26.433852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55321-2c80-43ba-bb30-745b8c92cb47 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.463472 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2194/4424] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:26.465691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d8419-50d3-4032-bc53-96ee6763932f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.477831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d8419-50d3-4032-bc53-96ee6763932f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.487596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:26.550920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55321-2c80-43ba-bb30-745b8c92cb47 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.563656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa52f-d13d-41d8-8512-34b72b1ba9c6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.564616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa52f-d13d-41d8-8512-34b72b1ba9c6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.564745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa52f-d13d-41d8-8512-34b72b1ba9c6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.564992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa52f-d13d-41d8-8512-34b72b1ba9c6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/identity/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.565618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa52f-d13d-41d8-8512-34b72b1ba9c6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.567232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa52f-d13d-41d8-8512-34b72b1ba9c6 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.568365 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2231/4425] 10.222.0.16 () {58 vars in 1200 bytes} [Sat Oct 15 21:09:26 2022] GET /identity/v3/domains/config/identity/default => generated 64 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:26.572644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b943042-c80f-4430-ab22-a4415b1a2f96 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.572921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b943042-c80f-4430-ab22-a4415b1a2f96 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.573869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b943042-c80f-4430-ab22-a4415b1a2f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.590163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07694-906a-4524-bd94-64866ac360f5 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.591104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07694-906a-4524-bd94-64866ac360f5 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.591485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07694-906a-4524-bd94-64866ac360f5 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.591861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07694-906a-4524-bd94-64866ac360f5 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/eab69bd688fa4d969422aa1318965e4e/users/1c58c240167e4c5baa3d2f0d3ae8b475/roles/a1878681caa446c390c4eb058fa1dfed` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.594874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c07694-906a-4524-bd94-64866ac360f5 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=eab69bd688fa4d969422aa1318965e4e, user_id=1c58c240167e4c5baa3d2f0d3ae8b475, role_id=a1878681caa446c390c4eb058fa1dfed)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.599389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c07694-906a-4524-bd94-64866ac360f5 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.608912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5b943042-c80f-4430-ab22-a4415b1a2f96 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:26.610181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5b943042-c80f-4430-ab22-a4415b1a2f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:26.623423 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2195/4426] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:26 2022] PUT /identity/v3/projects/eab69bd688fa4d969422aa1318965e4e/users/1c58c240167e4c5baa3d2f0d3ae8b475/roles/a1878681caa446c390c4eb058fa1dfed => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:26.624405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07694-906a-4524-bd94-64866ac360f5 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.647462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:26.700412 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2232/4427] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:26.701709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b943042-c80f-4430-ab22-a4415b1a2f96 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.722170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07694-906a-4524-bd94-64866ac360f5 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.727533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae7468-53a5-4b83-b967-7e616061b0fe tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.728199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7468-53a5-4b83-b967-7e616061b0fe tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.728477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7468-53a5-4b83-b967-7e616061b0fe tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.728753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7468-53a5-4b83-b967-7e616061b0fe tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/identity/driver/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.729379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae7468-53a5-4b83-b967-7e616061b0fe tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=driver)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.731029 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-6aae7468-53a5-4b83-b967-7e616061b0fe tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:26.731029 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:09:26.731029 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:09:26.731921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae7468-53a5-4b83-b967-7e616061b0fe tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.736096 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2196/4428] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:09:26 2022] GET /identity/v3/domains/config/identity/driver/default => generated 30 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:26.736096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae7468-53a5-4b83-b967-7e616061b0fe tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.747994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae7468-53a5-4b83-b967-7e616061b0fe tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.753526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf7939-9df3-41ff-86c4-3e19d137521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.754149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf7939-9df3-41ff-86c4-3e19d137521f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.754480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf7939-9df3-41ff-86c4-3e19d137521f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.754770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf7939-9df3-41ff-86c4-3e19d137521f admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.755408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf7939-9df3-41ff-86c4-3e19d137521f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:26.755710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf7939-9df3-41ff-86c4-3e19d137521f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.757492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf7939-9df3-41ff-86c4-3e19d137521f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.775525 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2197/4429] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:26.776534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf7939-9df3-41ff-86c4-3e19d137521f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.782028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b943042-c80f-4430-ab22-a4415b1a2f96 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.788496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa74407-cbbb-41a4-b1b3-08535ce282b3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.789471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa74407-cbbb-41a4-b1b3-08535ce282b3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.789911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa74407-cbbb-41a4-b1b3-08535ce282b3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.790328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa74407-cbbb-41a4-b1b3-08535ce282b3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.790460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf7939-9df3-41ff-86c4-3e19d137521f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.791509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa74407-cbbb-41a4-b1b3-08535ce282b3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.794415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa74407-cbbb-41a4-b1b3-08535ce282b3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.799572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a088a-ad9a-4197-9f98-7556ede5ed8a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.800230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a088a-ad9a-4197-9f98-7556ede5ed8a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.800509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a088a-ad9a-4197-9f98-7556ede5ed8a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.800784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a088a-ad9a-4197-9f98-7556ede5ed8a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/identity/list_limit/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.801395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a088a-ad9a-4197-9f98-7556ede5ed8a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=list_limit)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.803144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a088a-ad9a-4197-9f98-7556ede5ed8a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.804433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2198/4430] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:26 2022] GET /identity/v3/domains/config/identity/list_limit/default => generated 33 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:26.804991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a088a-ad9a-4197-9f98-7556ede5ed8a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.816982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a088a-ad9a-4197-9f98-7556ede5ed8a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.822549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcf482-500b-4e2f-8909-7c349df34f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.823974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcf482-500b-4e2f-8909-7c349df34f9d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.824515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcf482-500b-4e2f-8909-7c349df34f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.824916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcf482-500b-4e2f-8909-7c349df34f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.825755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcf482-500b-4e2f-8909-7c349df34f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.828457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcf482-500b-4e2f-8909-7c349df34f9d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.829014 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2233/4431] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:26 2022] GET /identity/v3/users => generated 8070 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:26.843911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa74407-cbbb-41a4-b1b3-08535ce282b3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.860572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa74407-cbbb-41a4-b1b3-08535ce282b3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.867732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64ec85-dd06-435d-8cb6-d60a47724211 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.868622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64ec85-dd06-435d-8cb6-d60a47724211 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.869170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64ec85-dd06-435d-8cb6-d60a47724211 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.869570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64ec85-dd06-435d-8cb6-d60a47724211 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.870761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64ec85-dd06-435d-8cb6-d60a47724211 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.873276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64ec85-dd06-435d-8cb6-d60a47724211 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.878161 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2234/4432] 10.222.0.16 () {58 vars in 1188 bytes} [Sat Oct 15 21:09:26 2022] GET /identity/v3/domains/config/ldap/default => generated 1476 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:26.878161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8bc01-4c9a-4d98-982e-3b7f02b28d11 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.878161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8bc01-4c9a-4d98-982e-3b7f02b28d11 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.878161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8bc01-4c9a-4d98-982e-3b7f02b28d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.886366 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2199/4433] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:26 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:26.887717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcf482-500b-4e2f-8909-7c349df34f9d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:26.961176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-52c8bc01-4c9a-4d98-982e-3b7f02b28d11 None None] MFA Rules not processed for user `1c58c240167e4c5baa3d2f0d3ae8b475`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:26.962625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-52c8bc01-4c9a-4d98-982e-3b7f02b28d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:26.962987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcf482-500b-4e2f-8909-7c349df34f9d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:26.968565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa81096-d440-48f4-b6ea-280926344a16 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:26.969291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa81096-d440-48f4-b6ea-280926344a16 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:26.969514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa81096-d440-48f4-b6ea-280926344a16 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:26.969787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa81096-d440-48f4-b6ea-280926344a16 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/url/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:26.970372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa81096-d440-48f4-b6ea-280926344a16 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=url)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:26.972350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa81096-d440-48f4-b6ea-280926344a16 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:26.977013 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2200/4434] 10.222.0.16 () {58 vars in 1200 bytes} [Sat Oct 15 21:09:26 2022] GET /identity/v3/domains/config/ldap/url/default => generated 40 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:26.977948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa81096-d440-48f4-b6ea-280926344a16 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.061551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:27.092551 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2235/4435] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:27.093744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8bc01-4c9a-4d98-982e-3b7f02b28d11 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.102098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa81096-d440-48f4-b6ea-280926344a16 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.108618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c5afc-5c94-4f08-958b-b2d16a28e612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.109867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c5afc-5c94-4f08-958b-b2d16a28e612 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.109867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c5afc-5c94-4f08-958b-b2d16a28e612 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.109867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c5afc-5c94-4f08-958b-b2d16a28e612 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.111009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c5afc-5c94-4f08-958b-b2d16a28e612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.113111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c5afc-5c94-4f08-958b-b2d16a28e612 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.126513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8bc01-4c9a-4d98-982e-3b7f02b28d11 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.139982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b01145-25be-48a3-9a0c-ce4cce3d5491 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.149856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b01145-25be-48a3-9a0c-ce4cce3d5491 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.150165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b01145-25be-48a3-9a0c-ce4cce3d5491 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.150432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b01145-25be-48a3-9a0c-ce4cce3d5491 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.151050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b01145-25be-48a3-9a0c-ce4cce3d5491 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.158128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b01145-25be-48a3-9a0c-ce4cce3d5491 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.159616 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2236/4436] 10.222.0.16 () {58 vars in 1203 bytes} [Sat Oct 15 21:09:27 2022] GET /identity/v3/domains/config/ldap/user/default => generated 27 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:27.160799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838cc2e-5ef2-4df6-9cb4-d58cc32986e4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.161105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838cc2e-5ef2-4df6-9cb4-d58cc32986e4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.161375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838cc2e-5ef2-4df6-9cb4-d58cc32986e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.188259 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2201/4437] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:26 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:27.192412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c5afc-5c94-4f08-958b-b2d16a28e612 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.194653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0838cc2e-5ef2-4df6-9cb4-d58cc32986e4 None None] MFA Rules not processed for user `1c58c240167e4c5baa3d2f0d3ae8b475`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:27.196227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0838cc2e-5ef2-4df6-9cb4-d58cc32986e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:27.210825 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2237/4438] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:27 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:27.210825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838cc2e-5ef2-4df6-9cb4-d58cc32986e4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.215807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c5afc-5c94-4f08-958b-b2d16a28e612 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.224147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2f9f5-1399-420d-844b-6d3a1e08a487 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.226173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838cc2e-5ef2-4df6-9cb4-d58cc32986e4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.226916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2f9f5-1399-420d-844b-6d3a1e08a487 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.227269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2f9f5-1399-420d-844b-6d3a1e08a487 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.227704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2f9f5-1399-420d-844b-6d3a1e08a487 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/suffix/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.228223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2f9f5-1399-420d-844b-6d3a1e08a487 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=suffix)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.230498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2f9f5-1399-420d-844b-6d3a1e08a487 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.233214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9aa86-fd7f-4505-ba2a-53486f5e5108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.234295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9aa86-fd7f-4505-ba2a-53486f5e5108 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.234295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9aa86-fd7f-4505-ba2a-53486f5e5108 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.234708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9aa86-fd7f-4505-ba2a-53486f5e5108 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.234942 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2202/4439] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Oct 15 21:09:27 2022] GET /identity/v3/domains/config/ldap/suffix/default => generated 44 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:27.235362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9aa86-fd7f-4505-ba2a-53486f5e5108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.236506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2f9f5-1399-420d-844b-6d3a1e08a487 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.236965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9aa86-fd7f-4505-ba2a-53486f5e5108 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.244222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:27.250644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2f9f5-1399-420d-844b-6d3a1e08a487 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.252816 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2238/4440] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:27 2022] GET /identity/v3/roles => generated 1891 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:27.253969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9aa86-fd7f-4505-ba2a-53486f5e5108 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.261205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b75ab7-efc9-45fc-8d42-00f103b3bb15 tempest-TrustsV3TestJSON-1721812038 tempest-user-1871567219] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c58c240167e4c5baa3d2f0d3ae8b475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab69bd688fa4d969422aa1318965e4e', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-241164761', 'tempest-NotDelegatedRole-986277724'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.262176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b75ab7-efc9-45fc-8d42-00f103b3bb15 tempest-TrustsV3TestJSON-1721812038 tempest-user-1871567219] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.262865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b75ab7-efc9-45fc-8d42-00f103b3bb15 tempest-TrustsV3TestJSON-1721812038 tempest-user-1871567219] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.263751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b75ab7-efc9-45fc-8d42-00f103b3bb15 tempest-TrustsV3TestJSON-1721812038 tempest-user-1871567219] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.264739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b75ab7-efc9-45fc-8d42-00f103b3bb15 tempest-TrustsV3TestJSON-1721812038 tempest-user-1871567219] RBAC: Authorizing `identity:create_trust()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.267813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b75ab7-efc9-45fc-8d42-00f103b3bb15 tempest-TrustsV3TestJSON-1721812038 tempest-user-1871567219] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.270339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9aa86-fd7f-4505-ba2a-53486f5e5108 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.276580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec364a74-5138-4bd9-b7d6-e626a836aa1f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.277717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec364a74-5138-4bd9-b7d6-e626a836aa1f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.278477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec364a74-5138-4bd9-b7d6-e626a836aa1f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.278760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec364a74-5138-4bd9-b7d6-e626a836aa1f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/query_scope/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.279726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec364a74-5138-4bd9-b7d6-e626a836aa1f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=query_scope)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.281350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec364a74-5138-4bd9-b7d6-e626a836aa1f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.282883 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2239/4441] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:27 2022] GET /identity/v3/domains/config/ldap/query_scope/default => generated 35 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:27.283891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec364a74-5138-4bd9-b7d6-e626a836aa1f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.301747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec364a74-5138-4bd9-b7d6-e626a836aa1f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.308589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbe5a-b909-44bf-8186-6de80c820692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.309953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbe5a-b909-44bf-8186-6de80c820692 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.310229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbe5a-b909-44bf-8186-6de80c820692 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.310529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbe5a-b909-44bf-8186-6de80c820692 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.311439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bbe5a-b909-44bf-8186-6de80c820692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.313070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bbe5a-b909-44bf-8186-6de80c820692 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.328819 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2240/4442] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:27 2022] GET /identity/v3/roles => generated 1891 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:27.328819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbe5a-b909-44bf-8186-6de80c820692 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.344800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbe5a-b909-44bf-8186-6de80c820692 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.363707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae21246-75d4-4c6f-a781-f9ec334097d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.367534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae21246-75d4-4c6f-a781-f9ec334097d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.368553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae21246-75d4-4c6f-a781-f9ec334097d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.369009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae21246-75d4-4c6f-a781-f9ec334097d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/page_size/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.373147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae21246-75d4-4c6f-a781-f9ec334097d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=page_size)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.378388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae21246-75d4-4c6f-a781-f9ec334097d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.388537 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2241/4443] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 21:09:27 2022] GET /identity/v3/domains/config/ldap/page_size/default => generated 29 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:27.388537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae21246-75d4-4c6f-a781-f9ec334097d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-59b75ab7-efc9-45fc-8d42-00f103b3bb15 tempest-TrustsV3TestJSON-1721812038 tempest-user-1871567219] Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 44, in parse_isotime Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return iso8601.parse_date(timestr) Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/iso8601/iso8601.py", line 127, in parse_date Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise ParseError(f"Unable to parse date string {datestring!r}") Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application iso8601.iso8601.ParseError: Unable to parse date string 'bad.123Z' Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 346, in parse_expiration_date Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application expiration_time = timeutils.parse_isotime(expiration_date) Oct 15 21:09:27.397565 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 46, in parse_isotime Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise ValueError(str(e)) Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ValueError: Unable to parse date string 'bad.123Z' Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 296, in post Oct 15 21:09:27.398571 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application trust['expires_at'] = self._parse_expiration_date( Oct 15 21:09:27.399910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 127, in _parse_expiration_date Oct 15 21:09:27.399910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return utils.parse_expiration_date(expiration_date) Oct 15 21:09:27.399910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 348, in parse_expiration_date Oct 15 21:09:27.399910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ValidationTimeStampError() Oct 15 21:09:27.399910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 15 21:09:27.399910 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:27.399910 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2203/4444] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Oct 15 21:09:27 2022] POST /identity/v3/OS-TRUST/trusts => generated 227 bytes in 164 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 15 21:09:27.400688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b75ab7-efc9-45fc-8d42-00f103b3bb15 tempest-TrustsV3TestJSON-1721812038 tempest-user-1871567219] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.420147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b75ab7-efc9-45fc-8d42-00f103b3bb15 tempest-TrustsV3TestJSON-1721812038 tempest-user-1871567219] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.426004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d411f6-0595-4dfe-87f3-5a7412155b2f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.426760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d411f6-0595-4dfe-87f3-5a7412155b2f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.427107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d411f6-0595-4dfe-87f3-5a7412155b2f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.427352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae21246-75d4-4c6f-a781-f9ec334097d9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.427481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d411f6-0595-4dfe-87f3-5a7412155b2f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/alias_dereferencing/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.428240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d411f6-0595-4dfe-87f3-5a7412155b2f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=alias_dereferencing)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.429953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d411f6-0595-4dfe-87f3-5a7412155b2f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.436122 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2204/4445] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:09:27 2022] GET /identity/v3/domains/config/ldap/alias_dereferencing/default => generated 47 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:27.436961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f45743-fafe-4055-9bc3-8b515349a1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.437681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f45743-fafe-4055-9bc3-8b515349a1a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.438248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f45743-fafe-4055-9bc3-8b515349a1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.438532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f45743-fafe-4055-9bc3-8b515349a1a8 admin admin] PATH_INFO: `/v3/projects/490af491d7a046b58e3c850f3bdd3b59/users/4b71f0e126194a99bc76fc9cf44f954b/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.439586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d411f6-0595-4dfe-87f3-5a7412155b2f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.474450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f45743-fafe-4055-9bc3-8b515349a1a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=490af491d7a046b58e3c850f3bdd3b59, user_id=4b71f0e126194a99bc76fc9cf44f954b, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.477414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f45743-fafe-4055-9bc3-8b515349a1a8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.499950 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2242/4446] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:27 2022] PUT /identity/v3/projects/490af491d7a046b58e3c850f3bdd3b59/users/4b71f0e126194a99bc76fc9cf44f954b/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:27.500790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f45743-fafe-4055-9bc3-8b515349a1a8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.582496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d411f6-0595-4dfe-87f3-5a7412155b2f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.668002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f45743-fafe-4055-9bc3-8b515349a1a8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.679477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f887a-6df1-44e0-b0b2-2f0c8fdda954 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.680172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f887a-6df1-44e0-b0b2-2f0c8fdda954 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.680625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f887a-6df1-44e0-b0b2-2f0c8fdda954 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.680809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f887a-6df1-44e0-b0b2-2f0c8fdda954 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/debug_level/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.681399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f887a-6df1-44e0-b0b2-2f0c8fdda954 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=debug_level)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.687634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f887a-6df1-44e0-b0b2-2f0c8fdda954 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.688997 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2243/4447] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:27 2022] GET /identity/v3/domains/config/ldap/debug_level/default => generated 34 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:27.693125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0bdc8-45c9-42ac-a967-382334cf3c86 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.693796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0bdc8-45c9-42ac-a967-382334cf3c86 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.694112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0bdc8-45c9-42ac-a967-382334cf3c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.702073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600095c-871b-47c2-b013-b29ced5d88bd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:27.704532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600095c-871b-47c2-b013-b29ced5d88bd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.704817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600095c-871b-47c2-b013-b29ced5d88bd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.705112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600095c-871b-47c2-b013-b29ced5d88bd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/a1878681caa446c390c4eb058fa1dfed` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.711669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600095c-871b-47c2-b013-b29ced5d88bd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=a1878681caa446c390c4eb058fa1dfed)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:27.717295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600095c-871b-47c2-b013-b29ced5d88bd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:27.769849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f3b0bdc8-45c9-42ac-a967-382334cf3c86 None None] MFA Rules not processed for user `4b71f0e126194a99bc76fc9cf44f954b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:27.772384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f3b0bdc8-45c9-42ac-a967-382334cf3c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:27.798389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a600095c-871b-47c2-b013-b29ced5d88bd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role a1878681caa446c390c4eb058fa1dfed has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:27.799059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a600095c-871b-47c2-b013-b29ced5d88bd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:27.800078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a600095c-871b-47c2-b013-b29ced5d88bd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:27.803802 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2205/4448] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:27 2022] DELETE /identity/v3/roles/a1878681caa446c390c4eb058fa1dfed => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:27.805805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8813424-0632-4e08-b958-fd30ebf8255d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:27.806250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8813424-0632-4e08-b958-fd30ebf8255d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:27.806766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8813424-0632-4e08-b958-fd30ebf8255d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:27.838455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d8813424-0632-4e08-b958-fd30ebf8255d None None] MFA Rules not processed for user `dd615ca74887465b8e21b7f4011fb53f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:27.840389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8813424-0632-4e08-b958-fd30ebf8255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:27.850788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:27.885280 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2244/4449] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:27.886761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0bdc8-45c9-42ac-a967-382334cf3c86 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.891347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f3b0bdc8-45c9-42ac-a967-382334cf3c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:27.912979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0bdc8-45c9-42ac-a967-382334cf3c86 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:27.926093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:27.992584 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2206/4450] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:27.993747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8813424-0632-4e08-b958-fd30ebf8255d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:27.996383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8813424-0632-4e08-b958-fd30ebf8255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:28.053869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5415a-e9dd-4cc6-ba2c-196d86bfbdc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.054644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5415a-e9dd-4cc6-ba2c-196d86bfbdc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.054910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5415a-e9dd-4cc6-ba2c-196d86bfbdc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.055186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5415a-e9dd-4cc6-ba2c-196d86bfbdc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/chase_referrals/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.055778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5415a-e9dd-4cc6-ba2c-196d86bfbdc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=chase_referrals)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.059355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5415a-e9dd-4cc6-ba2c-196d86bfbdc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.060536 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2245/4451] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:27 2022] GET /identity/v3/domains/config/ldap/chase_referrals/default => generated 38 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:28.061344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5415a-e9dd-4cc6-ba2c-196d86bfbdc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.066174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-75e5415a-e9dd-4cc6-ba2c-196d86bfbdc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:28.104109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8813424-0632-4e08-b958-fd30ebf8255d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.110027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.110324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.110602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.110919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/7c7c0104497246218082819eada78d64` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.112323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=7c7c0104497246218082819eada78d64)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.113954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.167896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5415a-e9dd-4cc6-ba2c-196d86bfbdc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.173451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ecb45-760a-4f7c-95be-ff868f74db80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.174311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ecb45-760a-4f7c-95be-ff868f74db80 service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.174719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ecb45-760a-4f7c-95be-ff868f74db80 service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.175205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ecb45-760a-4f7c-95be-ff868f74db80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.179671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role 7c7c0104497246218082819eada78d64 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:28.180345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.181288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.185327 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2207/4452] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:27 2022] DELETE /identity/v3/roles/7c7c0104497246218082819eada78d64 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:28.186350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.188599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:28.276727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ecb45-760a-4f7c-95be-ff868f74db80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.278370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ecb45-760a-4f7c-95be-ff868f74db80 service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.282123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb165ca0-413f-4ed2-b3a0-3b0105b8a14b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.294567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6fc6e-af6e-474c-8580-5e1bc073c5a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.295224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6fc6e-af6e-474c-8580-5e1bc073c5a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.296918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6fc6e-af6e-474c-8580-5e1bc073c5a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.297241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6fc6e-af6e-474c-8580-5e1bc073c5a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_tree_dn/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.297830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6fc6e-af6e-474c-8580-5e1bc073c5a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_tree_dn)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.299402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6fc6e-af6e-474c-8580-5e1bc073c5a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.302251 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2208/4453] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:09:28 2022] GET /identity/v3/domains/config/ldap/user_tree_dn/default => generated 35 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:28.303588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6fc6e-af6e-474c-8580-5e1bc073c5a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.305822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-44b6fc6e-af6e-474c-8580-5e1bc073c5a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:28.320071 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2246/4454] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:28.321047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ecb45-760a-4f7c-95be-ff868f74db80 service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.332998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:28.345671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ecb45-760a-4f7c-95be-ff868f74db80 service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.365517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37d837-cbb3-47e6-b59e-42e72f52608e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.366184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37d837-cbb3-47e6-b59e-42e72f52608e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.366462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37d837-cbb3-47e6-b59e-42e72f52608e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.366733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37d837-cbb3-47e6-b59e-42e72f52608e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_filter/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.367321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37d837-cbb3-47e6-b59e-42e72f52608e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_filter)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.368917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37d837-cbb3-47e6-b59e-42e72f52608e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.375747 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2247/4455] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:28 2022] GET /identity/v3/domains/config/ldap/user_filter/default => generated 34 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:28.379817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37d837-cbb3-47e6-b59e-42e72f52608e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.407999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37d837-cbb3-47e6-b59e-42e72f52608e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.416119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591c6e6-c1d1-4a30-9c60-7c99b75d99f1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.417771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591c6e6-c1d1-4a30-9c60-7c99b75d99f1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.418273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591c6e6-c1d1-4a30-9c60-7c99b75d99f1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.418812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591c6e6-c1d1-4a30-9c60-7c99b75d99f1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_objectclass/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.419941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591c6e6-c1d1-4a30-9c60-7c99b75d99f1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_objectclass)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.422159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591c6e6-c1d1-4a30-9c60-7c99b75d99f1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.422368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6fc6e-af6e-474c-8580-5e1bc073c5a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.426739 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2248/4456] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:28 2022] GET /identity/v3/domains/config/ldap/user_objectclass/default => generated 50 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:28.427730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591c6e6-c1d1-4a30-9c60-7c99b75d99f1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.435873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.439332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.439556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.443920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users/1c58c240167e4c5baa3d2f0d3ae8b475` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.447372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1c58c240167e4c5baa3d2f0d3ae8b475)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.451664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591c6e6-c1d1-4a30-9c60-7c99b75d99f1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.453646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.474225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40f110-5974-40c7-ab4f-813691f8a322 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.474893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f110-5974-40c7-ab4f-813691f8a322 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.475179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f110-5974-40c7-ab4f-813691f8a322 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.475456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f110-5974-40c7-ab4f-813691f8a322 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_id_attribute/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.476216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40f110-5974-40c7-ab4f-813691f8a322 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_id_attribute)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.478296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40f110-5974-40c7-ab4f-813691f8a322 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.480206 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2249/4457] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:09:28 2022] GET /identity/v3/domains/config/ldap/user_id_attribute/default => generated 40 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:28.485085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40f110-5974-40c7-ab4f-813691f8a322 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.501924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40f110-5974-40c7-ab4f-813691f8a322 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.509304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ac72f-cd05-40a8-920a-ba01feee398a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.511745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ac72f-cd05-40a8-920a-ba01feee398a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.523142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ac72f-cd05-40a8-920a-ba01feee398a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.524824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ac72f-cd05-40a8-920a-ba01feee398a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_name_attribute/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.526527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ac72f-cd05-40a8-920a-ba01feee398a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_name_attribute)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.531434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ac72f-cd05-40a8-920a-ba01feee398a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.548516 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2250/4458] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:09:28 2022] GET /identity/v3/domains/config/ldap/user_name_attribute/default => generated 42 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 15 21:09:28.552998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c58c240167e4c5baa3d2f0d3ae8b475'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.556514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ac72f-cd05-40a8-920a-ba01feee398a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.567926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c58c240167e4c5baa3d2f0d3ae8b475'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.572001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ac72f-cd05-40a8-920a-ba01feee398a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.582613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d3064-50d6-4ad6-9fdf-b51d6e37f237 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.585788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d3064-50d6-4ad6-9fdf-b51d6e37f237 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.586835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d3064-50d6-4ad6-9fdf-b51d6e37f237 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.587110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d3064-50d6-4ad6-9fdf-b51d6e37f237 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_mail_attribute/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.588527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d3064-50d6-4ad6-9fdf-b51d6e37f237 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_mail_attribute)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.593169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d3064-50d6-4ad6-9fdf-b51d6e37f237 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.598481 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2251/4459] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:09:28 2022] GET /identity/v3/domains/config/ldap/user_mail_attribute/default => generated 44 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:28.599941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb5c8d-4ed7-4a28-a2a6-855e821c0611 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.600220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb5c8d-4ed7-4a28-a2a6-855e821c0611 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.600596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb5c8d-4ed7-4a28-a2a6-855e821c0611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.602329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c58c240167e4c5baa3d2f0d3ae8b475'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.624099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-26bb5c8d-4ed7-4a28-a2a6-855e821c0611 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:28.625371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-26bb5c8d-4ed7-4a28-a2a6-855e821c0611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:28.625575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c58c240167e4c5baa3d2f0d3ae8b475'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.626285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c58c240167e4c5baa3d2f0d3ae8b475'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.635132 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2209/4460] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:28 2022] DELETE /identity/v3/users/1c58c240167e4c5baa3d2f0d3ae8b475 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:28.636530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.638196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:28.700113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:28.725958 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2252/4461] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:28.726834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb5c8d-4ed7-4a28-a2a6-855e821c0611 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.728164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-26bb5c8d-4ed7-4a28-a2a6-855e821c0611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:28.744479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb5c8d-4ed7-4a28-a2a6-855e821c0611 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.767372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a21a4c-42da-4cbc-923e-bbae8d81de14 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.772910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3790fb-999f-4396-ab3e-d6b1198f963e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.773782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3790fb-999f-4396-ab3e-d6b1198f963e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.774053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3790fb-999f-4396-ab3e-d6b1198f963e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.774430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3790fb-999f-4396-ab3e-d6b1198f963e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_description_attribute/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.775109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3790fb-999f-4396-ab3e-d6b1198f963e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_description_attribute)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.777509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3790fb-999f-4396-ab3e-d6b1198f963e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.782030 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2210/4462] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:09:28 2022] GET /identity/v3/domains/config/ldap/user_description_attribute/default => generated 58 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:28.783075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3790fb-999f-4396-ab3e-d6b1198f963e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.784949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de3790fb-999f-4396-ab3e-d6b1198f963e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:28.822397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.824860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.825173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.825585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/eab69bd688fa4d969422aa1318965e4e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.828482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_project(project_id=eab69bd688fa4d969422aa1318965e4e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:28.831341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:28.850048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:28.894517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3790fb-999f-4396-ab3e-d6b1198f963e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:28.898734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab69bd688fa4d969422aa1318965e4e', 'request_id': 'req-f356a95a-345d-456b-8d53-18fc0c4998c3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.908675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab69bd688fa4d969422aa1318965e4e', 'request_id': 'req-f356a95a-345d-456b-8d53-18fc0c4998c3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.916977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab69bd688fa4d969422aa1318965e4e', 'request_id': 'req-f356a95a-345d-456b-8d53-18fc0c4998c3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.931681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] The token cache is being invalidate because project eab69bd688fa4d969422aa1318965e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:28.932246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.933099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:28.934658 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2253/4463] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:28 2022] DELETE /identity/v3/projects/eab69bd688fa4d969422aa1318965e4e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:28.940115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:28.941561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:28.972692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20973403-99c6-4b4c-b059-0ff962d128c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:28.973471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20973403-99c6-4b4c-b059-0ff962d128c6 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:28.973851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20973403-99c6-4b4c-b059-0ff962d128c6 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:28.974226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20973403-99c6-4b4c-b059-0ff962d128c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:28.976345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-20973403-99c6-4b4c-b059-0ff962d128c6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:29.024307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356a95a-345d-456b-8d53-18fc0c4998c3 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.029446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af8ff8-5ae0-44fd-b082-62e6f8424af0 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.030099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af8ff8-5ae0-44fd-b082-62e6f8424af0 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.030378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af8ff8-5ae0-44fd-b082-62e6f8424af0 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.030672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af8ff8-5ae0-44fd-b082-62e6f8424af0 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_pass_attribute/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.031229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af8ff8-5ae0-44fd-b082-62e6f8424af0 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_pass_attribute)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.032848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af8ff8-5ae0-44fd-b082-62e6f8424af0 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.037079 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2254/4464] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:09:28 2022] GET /identity/v3/domains/config/ldap/user_pass_attribute/default => generated 52 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.037079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af8ff8-5ae0-44fd-b082-62e6f8424af0 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.038933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c1af8ff8-5ae0-44fd-b082-62e6f8424af0 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:29.045673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20973403-99c6-4b4c-b059-0ff962d128c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.047381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20973403-99c6-4b4c-b059-0ff962d128c6 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.056480 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2211/4465] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:09:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:29.057326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20973403-99c6-4b4c-b059-0ff962d128c6 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.059364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-20973403-99c6-4b4c-b059-0ff962d128c6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:29.074312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:29.133510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20973403-99c6-4b4c-b059-0ff962d128c6 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.137517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24f55a-3744-4876-aca8-7f92c09946e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.137865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af8ff8-5ae0-44fd-b082-62e6f8424af0 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.138248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24f55a-3744-4876-aca8-7f92c09946e4 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.138524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24f55a-3744-4876-aca8-7f92c09946e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.138890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24f55a-3744-4876-aca8-7f92c09946e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.141587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-af24f55a-3744-4876-aca8-7f92c09946e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:29.149776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497afc-1f35-46da-88f0-269f85621050 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.150812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497afc-1f35-46da-88f0-269f85621050 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.152008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497afc-1f35-46da-88f0-269f85621050 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.152716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497afc-1f35-46da-88f0-269f85621050 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.153676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4497afc-1f35-46da-88f0-269f85621050 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.156096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4497afc-1f35-46da-88f0-269f85621050 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.156515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24f55a-3744-4876-aca8-7f92c09946e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.158339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24f55a-3744-4876-aca8-7f92c09946e4 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.170090 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2212/4466] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:29.170998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24f55a-3744-4876-aca8-7f92c09946e4 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.186476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24f55a-3744-4876-aca8-7f92c09946e4 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.192115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acc834-767c-4f83-a264-343baab58f79 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.192802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acc834-767c-4f83-a264-343baab58f79 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.193119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acc834-767c-4f83-a264-343baab58f79 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.193512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acc834-767c-4f83-a264-343baab58f79 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_attribute/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.194103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acc834-767c-4f83-a264-343baab58f79 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_attribute)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.195705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acc834-767c-4f83-a264-343baab58f79 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.197446 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2213/4467] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/user_enabled_attribute/default => generated 50 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.200912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acc834-767c-4f83-a264-343baab58f79 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.208771 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2255/4468] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:09:29 2022] POST /identity/v3/projects => generated 329 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:29.214323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acc834-767c-4f83-a264-343baab58f79 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.231341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497afc-1f35-46da-88f0-269f85621050 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.244436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:29.312426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f8d12-0ddb-4ed0-b7f1-bdb574eac7ed tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.313359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8d12-0ddb-4ed0-b7f1-bdb574eac7ed tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.314787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8d12-0ddb-4ed0-b7f1-bdb574eac7ed tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.315072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8d12-0ddb-4ed0-b7f1-bdb574eac7ed tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_invert/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.315660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f8d12-0ddb-4ed0-b7f1-bdb574eac7ed tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_invert)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.318015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f8d12-0ddb-4ed0-b7f1-bdb574eac7ed tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.320093 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2214/4469] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/user_enabled_invert/default => generated 43 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.324875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f8d12-0ddb-4ed0-b7f1-bdb574eac7ed tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.346660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f8d12-0ddb-4ed0-b7f1-bdb574eac7ed tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.352804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b39959-13f7-4e28-8418-f18b99581334 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.353631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b39959-13f7-4e28-8418-f18b99581334 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.353774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b39959-13f7-4e28-8418-f18b99581334 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.354316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b39959-13f7-4e28-8418-f18b99581334 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_mask/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.355949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b39959-13f7-4e28-8418-f18b99581334 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_mask)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.358492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b39959-13f7-4e28-8418-f18b99581334 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.362918 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2215/4470] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/user_enabled_mask/default => generated 37 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.363194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497afc-1f35-46da-88f0-269f85621050 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.366230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b39959-13f7-4e28-8418-f18b99581334 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.371562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a448ac-a576-46f7-8ae7-eaacd01a53d7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.372267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a448ac-a576-46f7-8ae7-eaacd01a53d7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.372550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a448ac-a576-46f7-8ae7-eaacd01a53d7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.372847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a448ac-a576-46f7-8ae7-eaacd01a53d7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.373546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a448ac-a576-46f7-8ae7-eaacd01a53d7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.376296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a448ac-a576-46f7-8ae7-eaacd01a53d7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.383313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b39959-13f7-4e28-8418-f18b99581334 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.388889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08eefa-b861-4cbe-9c71-8d59ebadaac9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.389633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08eefa-b861-4cbe-9c71-8d59ebadaac9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.389833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08eefa-b861-4cbe-9c71-8d59ebadaac9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.390125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08eefa-b861-4cbe-9c71-8d59ebadaac9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_default/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.390702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08eefa-b861-4cbe-9c71-8d59ebadaac9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_default)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.392254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08eefa-b861-4cbe-9c71-8d59ebadaac9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.393652 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2216/4471] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/user_enabled_default/default => generated 45 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.398898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08eefa-b861-4cbe-9c71-8d59ebadaac9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.402832 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2256/4472] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:29 2022] POST /identity/v3/users => generated 416 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:29.406514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a448ac-a576-46f7-8ae7-eaacd01a53d7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.418470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08eefa-b861-4cbe-9c71-8d59ebadaac9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.424778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52ba59-9a1b-4b79-aa3d-ffca724b73ee tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.425466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52ba59-9a1b-4b79-aa3d-ffca724b73ee tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.425742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52ba59-9a1b-4b79-aa3d-ffca724b73ee tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.426010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52ba59-9a1b-4b79-aa3d-ffca724b73ee tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_attribute_ignore/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.426591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52ba59-9a1b-4b79-aa3d-ffca724b73ee tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_attribute_ignore)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.427727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a448ac-a576-46f7-8ae7-eaacd01a53d7 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.428181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52ba59-9a1b-4b79-aa3d-ffca724b73ee tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.433812 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2217/4473] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/user_attribute_ignore/default => generated 62 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 15 21:09:29.435432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52ba59-9a1b-4b79-aa3d-ffca724b73ee tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.437300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda7c93-a139-4d7d-a9a6-173c5873353d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.438020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda7c93-a139-4d7d-a9a6-173c5873353d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.438397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda7c93-a139-4d7d-a9a6-173c5873353d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.438814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda7c93-a139-4d7d-a9a6-173c5873353d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.439557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda7c93-a139-4d7d-a9a6-173c5873353d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.441364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda7c93-a139-4d7d-a9a6-173c5873353d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.452239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52ba59-9a1b-4b79-aa3d-ffca724b73ee tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.462160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2a24e-e9c6-4640-8dbf-a7c4e38c4194 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.463308 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2257/4474] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:29 2022] POST /identity/v3/roles => generated 244 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:29.465816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2a24e-e9c6-4640-8dbf-a7c4e38c4194 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.466100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2a24e-e9c6-4640-8dbf-a7c4e38c4194 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.466373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2a24e-e9c6-4640-8dbf-a7c4e38c4194 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_default_project_id_attribute/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.466608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda7c93-a139-4d7d-a9a6-173c5873353d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.467421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2a24e-e9c6-4640-8dbf-a7c4e38c4194 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_default_project_id_attribute)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.471851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2a24e-e9c6-4640-8dbf-a7c4e38c4194 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.476580 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2218/4475] 10.222.0.16 () {58 vars in 1290 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/user_default_project_id_attribute/default => generated 56 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.477487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2a24e-e9c6-4640-8dbf-a7c4e38c4194 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.480220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda7c93-a139-4d7d-a9a6-173c5873353d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.490378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2a24e-e9c6-4640-8dbf-a7c4e38c4194 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.492058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff2c2d-5e15-4335-9dc2-92206d59c83d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.494185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff2c2d-5e15-4335-9dc2-92206d59c83d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.494809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff2c2d-5e15-4335-9dc2-92206d59c83d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.495862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff2c2d-5e15-4335-9dc2-92206d59c83d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.500619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff2c2d-5e15-4335-9dc2-92206d59c83d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.500619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff2c2d-5e15-4335-9dc2-92206d59c83d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.503290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15661d0-3afa-4ace-ab0d-3141f15bda57 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.504159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15661d0-3afa-4ace-ab0d-3141f15bda57 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.504687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15661d0-3afa-4ace-ab0d-3141f15bda57 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.505241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15661d0-3afa-4ace-ab0d-3141f15bda57 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.506167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15661d0-3afa-4ace-ab0d-3141f15bda57 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.507972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15661d0-3afa-4ace-ab0d-3141f15bda57 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.513968 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2219/4476] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation/default => generated 46 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.515282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15661d0-3afa-4ace-ab0d-3141f15bda57 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.529761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15661d0-3afa-4ace-ab0d-3141f15bda57 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.537278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356459f-d60a-4d55-8e5e-083925579449 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.538863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356459f-d60a-4d55-8e5e-083925579449 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.539360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356459f-d60a-4d55-8e5e-083925579449 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.539833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356459f-d60a-4d55-8e5e-083925579449 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_dn/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.540640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356459f-d60a-4d55-8e5e-083925579449 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_dn)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.542875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356459f-d60a-4d55-8e5e-083925579449 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.546179 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2220/4477] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_dn/default => generated 48 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.550376 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2258/4478] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:29 2022] POST /identity/v3/roles => generated 246 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:29.551815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356459f-d60a-4d55-8e5e-083925579449 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.552037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff2c2d-5e15-4335-9dc2-92206d59c83d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.572711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356459f-d60a-4d55-8e5e-083925579449 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.578894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa1d93-0d21-4f5a-bbc5-f233a3fe0a5b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.579653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa1d93-0d21-4f5a-bbc5-f233a3fe0a5b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.580137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa1d93-0d21-4f5a-bbc5-f233a3fe0a5b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.580529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa1d93-0d21-4f5a-bbc5-f233a3fe0a5b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/1469f81b45294aca88bb8bd7b24cc210/users/1e76bc3c301242afa9801aad2e66fe53/roles/f75a7c27dd1e4de1a7d6d66525584df6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.600085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa1d93-0d21-4f5a-bbc5-f233a3fe0a5b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=1469f81b45294aca88bb8bd7b24cc210, user_id=1e76bc3c301242afa9801aad2e66fe53, role_id=f75a7c27dd1e4de1a7d6d66525584df6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.603582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa1d93-0d21-4f5a-bbc5-f233a3fe0a5b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.604537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff2c2d-5e15-4335-9dc2-92206d59c83d tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.625170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867eb069-fbd5-4949-b158-8ce3e4fd688b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.626000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867eb069-fbd5-4949-b158-8ce3e4fd688b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.626335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867eb069-fbd5-4949-b158-8ce3e4fd688b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.626543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867eb069-fbd5-4949-b158-8ce3e4fd688b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.627217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867eb069-fbd5-4949-b158-8ce3e4fd688b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_use_group_config)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.629019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867eb069-fbd5-4949-b158-8ce3e4fd688b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.633085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2259/4479] 10.222.0.16 () {58 vars in 1308 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default => generated 63 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.633265 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2221/4480] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:29 2022] PUT /identity/v3/projects/1469f81b45294aca88bb8bd7b24cc210/users/1e76bc3c301242afa9801aad2e66fe53/roles/f75a7c27dd1e4de1a7d6d66525584df6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:29.636230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867eb069-fbd5-4949-b158-8ce3e4fd688b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.639014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa1d93-0d21-4f5a-bbc5-f233a3fe0a5b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.661660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867eb069-fbd5-4949-b158-8ce3e4fd688b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.757170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa1d93-0d21-4f5a-bbc5-f233a3fe0a5b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.762265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbc468-cb78-4469-b5fd-aee12b697fc3 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.763349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbc468-cb78-4469-b5fd-aee12b697fc3 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.763719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbc468-cb78-4469-b5fd-aee12b697fc3 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.764216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbc468-cb78-4469-b5fd-aee12b697fc3 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_additional_attribute_mapping/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.764627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04d872-44e8-459a-89a7-25903cf3fb6e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.765114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbc468-cb78-4469-b5fd-aee12b697fc3 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_additional_attribute_mapping)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.766457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04d872-44e8-459a-89a7-25903cf3fb6e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.766587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04d872-44e8-459a-89a7-25903cf3fb6e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.766974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04d872-44e8-459a-89a7-25903cf3fb6e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/1469f81b45294aca88bb8bd7b24cc210/users/1e76bc3c301242afa9801aad2e66fe53/roles/cd14a76b2f0b4075ba495566bb33164f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.767763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbc468-cb78-4469-b5fd-aee12b697fc3 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.769913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04d872-44e8-459a-89a7-25903cf3fb6e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=1469f81b45294aca88bb8bd7b24cc210, user_id=1e76bc3c301242afa9801aad2e66fe53, role_id=cd14a76b2f0b4075ba495566bb33164f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.770435 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2260/4481] 10.222.0.16 () {58 vars in 1290 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/user_additional_attribute_mapping/default => generated 54 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.774207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbc468-cb78-4469-b5fd-aee12b697fc3 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.777577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04d872-44e8-459a-89a7-25903cf3fb6e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.789603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbc468-cb78-4469-b5fd-aee12b697fc3 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.800524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:29.803904 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2222/4482] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:29 2022] PUT /identity/v3/projects/1469f81b45294aca88bb8bd7b24cc210/users/1e76bc3c301242afa9801aad2e66fe53/roles/cd14a76b2f0b4075ba495566bb33164f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:29.805799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6635bd-1d0d-4e2d-860d-8249bbdfa5a0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.806228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6635bd-1d0d-4e2d-860d-8249bbdfa5a0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.806661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6635bd-1d0d-4e2d-860d-8249bbdfa5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.840411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0b6635bd-1d0d-4e2d-860d-8249bbdfa5a0 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:29.841453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0b6635bd-1d0d-4e2d-860d-8249bbdfa5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:29.886610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faddd9-6ead-4ba8-9e49-9720e1ada922 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.887305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faddd9-6ead-4ba8-9e49-9720e1ada922 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.887587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faddd9-6ead-4ba8-9e49-9720e1ada922 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.887860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faddd9-6ead-4ba8-9e49-9720e1ada922 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_tree_dn/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.888450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faddd9-6ead-4ba8-9e49-9720e1ada922 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_tree_dn)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.890264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faddd9-6ead-4ba8-9e49-9720e1ada922 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.893125 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2261/4483] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/group_tree_dn/default => generated 36 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.893125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faddd9-6ead-4ba8-9e49-9720e1ada922 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.924876 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2223/4484] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:09:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:29.926185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6635bd-1d0d-4e2d-860d-8249bbdfa5a0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.941091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6635bd-1d0d-4e2d-860d-8249bbdfa5a0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.947541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3934728-7067-436c-9037-f0a78b17a05d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.948747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3934728-7067-436c-9037-f0a78b17a05d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.949029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3934728-7067-436c-9037-f0a78b17a05d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.949407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3934728-7067-436c-9037-f0a78b17a05d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_filter/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.950011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3934728-7067-436c-9037-f0a78b17a05d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_filter)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.952121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3934728-7067-436c-9037-f0a78b17a05d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:29.953729 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2224/4485] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/domains/config/ldap/group_filter/default => generated 35 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:29.954844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3934728-7067-436c-9037-f0a78b17a05d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:29.958802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d3934728-7067-436c-9037-f0a78b17a05d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:29.971733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:29.981087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faddd9-6ead-4ba8-9e49-9720e1ada922 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:29.989780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031f494-0fc2-4912-b20b-46b910c5d59e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:29.990696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031f494-0fc2-4912-b20b-46b910c5d59e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:29.991378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031f494-0fc2-4912-b20b-46b910c5d59e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:29.991802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031f494-0fc2-4912-b20b-46b910c5d59e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:29.992732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031f494-0fc2-4912-b20b-46b910c5d59e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:29.994792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031f494-0fc2-4912-b20b-46b910c5d59e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.028022 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2262/4486] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/users => generated 8355 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:30.029126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031f494-0fc2-4912-b20b-46b910c5d59e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.042344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031f494-0fc2-4912-b20b-46b910c5d59e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.043825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3934728-7067-436c-9037-f0a78b17a05d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.048216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a88cf0-8fa9-42c4-871d-1be010fb1db8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.049081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a88cf0-8fa9-42c4-871d-1be010fb1db8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.049218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f3ef8-639a-4d66-b10a-3de5bdfa28f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.049647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a88cf0-8fa9-42c4-871d-1be010fb1db8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.049768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f3ef8-639a-4d66-b10a-3de5bdfa28f6 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.049969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f3ef8-639a-4d66-b10a-3de5bdfa28f6 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.050238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f3ef8-639a-4d66-b10a-3de5bdfa28f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.050496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a88cf0-8fa9-42c4-871d-1be010fb1db8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_objectclass/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.051477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a88cf0-8fa9-42c4-871d-1be010fb1db8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_objectclass)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.053378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a88cf0-8fa9-42c4-871d-1be010fb1db8 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.055449 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2263/4487] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/group_objectclass/default => generated 50 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.056684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d421a6-0a1e-48bc-abf2-c0e1a76e5d3a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.056961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d421a6-0a1e-48bc-abf2-c0e1a76e5d3a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.057923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d421a6-0a1e-48bc-abf2-c0e1a76e5d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.059696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f3ef8-639a-4d66-b10a-3de5bdfa28f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.061346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f3ef8-639a-4d66-b10a-3de5bdfa28f6 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.071733 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2225/4488] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:09:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:30.072420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f3ef8-639a-4d66-b10a-3de5bdfa28f6 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.086048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f3ef8-639a-4d66-b10a-3de5bdfa28f6 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.092425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ddfe4-1c5d-4c33-997c-c701f9b4525a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.093252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ddfe4-1c5d-4c33-997c-c701f9b4525a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.093533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ddfe4-1c5d-4c33-997c-c701f9b4525a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.093872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ddfe4-1c5d-4c33-997c-c701f9b4525a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_id_attribute/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.094523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ddfe4-1c5d-4c33-997c-c701f9b4525a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_id_attribute)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.099238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ddfe4-1c5d-4c33-997c-c701f9b4525a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.100747 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2226/4489] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/group_id_attribute/default => generated 41 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.105602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ddfe4-1c5d-4c33-997c-c701f9b4525a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.107382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b7d421a6-0a1e-48bc-abf2-c0e1a76e5d3a None None] MFA Rules not processed for user `1e76bc3c301242afa9801aad2e66fe53`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:30.109606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b7d421a6-0a1e-48bc-abf2-c0e1a76e5d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:30.121438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ddfe4-1c5d-4c33-997c-c701f9b4525a tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.133094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3ffbe-3835-465b-96c8-f063a98766a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.133897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ffbe-3835-465b-96c8-f063a98766a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.134028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ffbe-3835-465b-96c8-f063a98766a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.134317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ffbe-3835-465b-96c8-f063a98766a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_name_attribute/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.137486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3ffbe-3835-465b-96c8-f063a98766a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_name_attribute)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.138957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3ffbe-3835-465b-96c8-f063a98766a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.140175 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2227/4490] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/group_name_attribute/default => generated 43 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.150301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3ffbe-3835-465b-96c8-f063a98766a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.171685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3ffbe-3835-465b-96c8-f063a98766a2 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.191785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6e677-dcd4-4758-9fe7-c0bf2f89b9b4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.192738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6e677-dcd4-4758-9fe7-c0bf2f89b9b4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.193020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6e677-dcd4-4758-9fe7-c0bf2f89b9b4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.193322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6e677-dcd4-4758-9fe7-c0bf2f89b9b4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_members_are_ids/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.193911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6e677-dcd4-4758-9fe7-c0bf2f89b9b4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_members_are_ids)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.196546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6e677-dcd4-4758-9fe7-c0bf2f89b9b4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.197983 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2228/4491] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/group_members_are_ids/default => generated 45 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.204767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6e677-dcd4-4758-9fe7-c0bf2f89b9b4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.226766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6e677-dcd4-4758-9fe7-c0bf2f89b9b4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.234671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eb505-b0d8-4b68-ba3f-95ec5d9fe396 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.235383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eb505-b0d8-4b68-ba3f-95ec5d9fe396 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.235739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eb505-b0d8-4b68-ba3f-95ec5d9fe396 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.236766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eb505-b0d8-4b68-ba3f-95ec5d9fe396 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_member_attribute/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.236908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23eb505-b0d8-4b68-ba3f-95ec5d9fe396 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_member_attribute)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.239485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23eb505-b0d8-4b68-ba3f-95ec5d9fe396 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.241279 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2229/4492] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/group_member_attribute/default => generated 49 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.245401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eb505-b0d8-4b68-ba3f-95ec5d9fe396 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.246852 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2264/4493] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:30.257072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af581d15-5a4d-4bfc-aaf1-fbccc172f46c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.257362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af581d15-5a4d-4bfc-aaf1-fbccc172f46c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.257607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af581d15-5a4d-4bfc-aaf1-fbccc172f46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.259777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eb505-b0d8-4b68-ba3f-95ec5d9fe396 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.270125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ce85a-f58f-4cff-afc5-4ea3e4ddc7cd tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.270923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ce85a-f58f-4cff-afc5-4ea3e4ddc7cd tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.271205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ce85a-f58f-4cff-afc5-4ea3e4ddc7cd tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.271551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ce85a-f58f-4cff-afc5-4ea3e4ddc7cd tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_desc_attribute/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.272351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ce85a-f58f-4cff-afc5-4ea3e4ddc7cd tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_desc_attribute)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.274232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ce85a-f58f-4cff-afc5-4ea3e4ddc7cd tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.276862 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2230/4494] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/group_desc_attribute/default => generated 52 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.281490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ce85a-f58f-4cff-afc5-4ea3e4ddc7cd tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.288385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-af581d15-5a4d-4bfc-aaf1-fbccc172f46c None None] MFA Rules not processed for user `1e76bc3c301242afa9801aad2e66fe53`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:30.289874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af581d15-5a4d-4bfc-aaf1-fbccc172f46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:30.302451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ce85a-f58f-4cff-afc5-4ea3e4ddc7cd tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.310899 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2265/4495] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:30.311818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af581d15-5a4d-4bfc-aaf1-fbccc172f46c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.331342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0ebc0-00c8-420a-b29c-861566fe87a5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.332010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ebc0-00c8-420a-b29c-861566fe87a5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.332291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ebc0-00c8-420a-b29c-861566fe87a5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.332567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ebc0-00c8-420a-b29c-861566fe87a5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_attribute_ignore/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.333240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0ebc0-00c8-420a-b29c-861566fe87a5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_attribute_ignore)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.335945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0ebc0-00c8-420a-b29c-861566fe87a5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.338298 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2231/4496] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/group_attribute_ignore/default => generated 43 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.343639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0ebc0-00c8-420a-b29c-861566fe87a5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.347360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af581d15-5a4d-4bfc-aaf1-fbccc172f46c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.364604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a458469-e3e0-41e7-bca4-2e8034393ffc tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e76bc3c301242afa9801aad2e66fe53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1469f81b45294aca88bb8bd7b24cc210', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1831003017', 'tempest-NotDelegatedRole-530683875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.365307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458469-e3e0-41e7-bca4-2e8034393ffc tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.365615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458469-e3e0-41e7-bca4-2e8034393ffc tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.365788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458469-e3e0-41e7-bca4-2e8034393ffc tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.368622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a458469-e3e0-41e7-bca4-2e8034393ffc tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorizing `identity:create_trust()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.376473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a458469-e3e0-41e7-bca4-2e8034393ffc tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.390560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0ebc0-00c8-420a-b29c-861566fe87a5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.404593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecec88-bcaf-48bd-8095-89dd6412cdd7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.405366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecec88-bcaf-48bd-8095-89dd6412cdd7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.405653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecec88-bcaf-48bd-8095-89dd6412cdd7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.406743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecec88-bcaf-48bd-8095-89dd6412cdd7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_additional_attribute_mapping/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.407421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ecec88-bcaf-48bd-8095-89dd6412cdd7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_additional_attribute_mapping)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.409289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ecec88-bcaf-48bd-8095-89dd6412cdd7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.413091 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2232/4497] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/group_additional_attribute_mapping/default => generated 55 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 15 21:09:30.429847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:30.431167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecec88-bcaf-48bd-8095-89dd6412cdd7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.451171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecec88-bcaf-48bd-8095-89dd6412cdd7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.459615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912f1f4-6513-4b63-8bb6-6ce2bd097bb7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.460479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912f1f4-6513-4b63-8bb6-6ce2bd097bb7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.463425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912f1f4-6513-4b63-8bb6-6ce2bd097bb7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.464666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912f1f4-6513-4b63-8bb6-6ce2bd097bb7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertfile/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.465538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912f1f4-6513-4b63-8bb6-6ce2bd097bb7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertfile)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.467657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912f1f4-6513-4b63-8bb6-6ce2bd097bb7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.470386 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2233/4498] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/tls_cacertfile/default => generated 37 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.474445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912f1f4-6513-4b63-8bb6-6ce2bd097bb7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.525755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912f1f4-6513-4b63-8bb6-6ce2bd097bb7 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.542552 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2266/4499] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Oct 15 21:09:30 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:30.543662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a458469-e3e0-41e7-bca4-2e8034393ffc tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.551357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a5187-fa2b-45d0-b015-58b7f0ab15e4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.552020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a5187-fa2b-45d0-b015-58b7f0ab15e4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.552300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a5187-fa2b-45d0-b015-58b7f0ab15e4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.554297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a5187-fa2b-45d0-b015-58b7f0ab15e4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertdir/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.558834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a5187-fa2b-45d0-b015-58b7f0ab15e4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertdir)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.562607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a5187-fa2b-45d0-b015-58b7f0ab15e4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.567914 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2234/4500] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/tls_cacertdir/default => generated 36 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.568338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a5187-fa2b-45d0-b015-58b7f0ab15e4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.569776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a458469-e3e0-41e7-bca4-2e8034393ffc tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.580213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22653a2d-905b-468c-a749-30f15296b8a3 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e76bc3c301242afa9801aad2e66fe53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1469f81b45294aca88bb8bd7b24cc210', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1831003017', 'tempest-NotDelegatedRole-530683875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.580865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22653a2d-905b-468c-a749-30f15296b8a3 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.581166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22653a2d-905b-468c-a749-30f15296b8a3 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.581435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22653a2d-905b-468c-a749-30f15296b8a3 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] PATH_INFO: `/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.582640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a5187-fa2b-45d0-b015-58b7f0ab15e4 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.590170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897323eb-83b2-49b7-85e3-a1f49784d938 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.591115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897323eb-83b2-49b7-85e3-a1f49784d938 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.591261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22653a2d-905b-468c-a749-30f15296b8a3 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorizing `identity:get_trust(trust_id=45c934cf820948998b69a3a720beed8c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.591441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897323eb-83b2-49b7-85e3-a1f49784d938 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.591807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897323eb-83b2-49b7-85e3-a1f49784d938 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_tls/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.592358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897323eb-83b2-49b7-85e3-a1f49784d938 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_tls)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.592795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22653a2d-905b-468c-a749-30f15296b8a3 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.594743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897323eb-83b2-49b7-85e3-a1f49784d938 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.599199 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2235/4501] 10.222.0.16 () {58 vars in 1212 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/use_tls/default => generated 31 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.601253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897323eb-83b2-49b7-85e3-a1f49784d938 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.611836 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2267/4502] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c => generated 834 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:30.615837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22653a2d-905b-468c-a749-30f15296b8a3 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.624113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897323eb-83b2-49b7-85e3-a1f49784d938 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.631651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22653a2d-905b-468c-a749-30f15296b8a3 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.640443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0afdc4-c2f3-40a7-982b-527652d93d47 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e76bc3c301242afa9801aad2e66fe53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1469f81b45294aca88bb8bd7b24cc210', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1831003017', 'tempest-NotDelegatedRole-530683875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.641258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0afdc4-c2f3-40a7-982b-527652d93d47 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.641616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0afdc4-c2f3-40a7-982b-527652d93d47 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.641879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0afdc4-c2f3-40a7-982b-527652d93d47 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] PATH_INFO: `/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.642073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988dffb-38da-4bf9-a613-3131fe303dc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.642755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988dffb-38da-4bf9-a613-3131fe303dc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.643638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988dffb-38da-4bf9-a613-3131fe303dc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.643638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988dffb-38da-4bf9-a613-3131fe303dc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_req_cert/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.643947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988dffb-38da-4bf9-a613-3131fe303dc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_req_cert)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.647860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988dffb-38da-4bf9-a613-3131fe303dc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.653081 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2236/4503] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/tls_req_cert/default => generated 39 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.657599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988dffb-38da-4bf9-a613-3131fe303dc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.658413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0afdc4-c2f3-40a7-982b-527652d93d47 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=45c934cf820948998b69a3a720beed8c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.666503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0afdc4-c2f3-40a7-982b-527652d93d47 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.672256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988dffb-38da-4bf9-a613-3131fe303dc9 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.679825 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2268/4504] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c/roles => generated 371 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:09:30.687999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0afdc4-c2f3-40a7-982b-527652d93d47 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.695499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d1337-d85c-43b7-a3e5-d4b3e72db26e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.697111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d1337-d85c-43b7-a3e5-d4b3e72db26e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.699422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d1337-d85c-43b7-a3e5-d4b3e72db26e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.699732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d1337-d85c-43b7-a3e5-d4b3e72db26e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_pool/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.700727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d1337-d85c-43b7-a3e5-d4b3e72db26e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_pool)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.702613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0afdc4-c2f3-40a7-982b-527652d93d47 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.705852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d1337-d85c-43b7-a3e5-d4b3e72db26e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.708543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3483d2-1906-45e9-ad59-affdd38b3d75 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e76bc3c301242afa9801aad2e66fe53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1469f81b45294aca88bb8bd7b24cc210', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1831003017', 'tempest-NotDelegatedRole-530683875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.710144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3483d2-1906-45e9-ad59-affdd38b3d75 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.710588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3483d2-1906-45e9-ad59-affdd38b3d75 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.710708 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2237/4505] 10.222.0.16 () {58 vars in 1215 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/use_pool/default => generated 31 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 15 21:09:30.711239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3483d2-1906-45e9-ad59-affdd38b3d75 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] PATH_INFO: `/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c/roles/f75a7c27dd1e4de1a7d6d66525584df6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.719516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d1337-d85c-43b7-a3e5-d4b3e72db26e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.737745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d1337-d85c-43b7-a3e5-d4b3e72db26e tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.739009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3483d2-1906-45e9-ad59-affdd38b3d75 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorizing `identity:get_role_for_trust(trust_id=45c934cf820948998b69a3a720beed8c, role_id=f75a7c27dd1e4de1a7d6d66525584df6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.740726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3483d2-1906-45e9-ad59-affdd38b3d75 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.746107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51d786-c16d-40c9-8d1b-ec0d23c6f83b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.747735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51d786-c16d-40c9-8d1b-ec0d23c6f83b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.748254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51d786-c16d-40c9-8d1b-ec0d23c6f83b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.748747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51d786-c16d-40c9-8d1b-ec0d23c6f83b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_size/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.750779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51d786-c16d-40c9-8d1b-ec0d23c6f83b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_size)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.752951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51d786-c16d-40c9-8d1b-ec0d23c6f83b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.754894 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2238/4506] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/pool_size/default => generated 30 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 15 21:09:30.765379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51d786-c16d-40c9-8d1b-ec0d23c6f83b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.777155 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2269/4507] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c/roles/f75a7c27dd1e4de1a7d6d66525584df6 => generated 244 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:30.779817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3483d2-1906-45e9-ad59-affdd38b3d75 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.801357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3483d2-1906-45e9-ad59-affdd38b3d75 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.808190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51d786-c16d-40c9-8d1b-ec0d23c6f83b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.812283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4599bb7-3600-4a11-9dec-5ba9c2e0514d tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e76bc3c301242afa9801aad2e66fe53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1469f81b45294aca88bb8bd7b24cc210', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1831003017', 'tempest-NotDelegatedRole-530683875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.812947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4599bb7-3600-4a11-9dec-5ba9c2e0514d tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.813253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4599bb7-3600-4a11-9dec-5ba9c2e0514d tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.813533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4599bb7-3600-4a11-9dec-5ba9c2e0514d tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] PATH_INFO: `/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c/roles/f75a7c27dd1e4de1a7d6d66525584df6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.820757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e80f62-878f-4588-82a4-65145daab3d1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.821599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e80f62-878f-4588-82a4-65145daab3d1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.822211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e80f62-878f-4588-82a4-65145daab3d1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.822629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e80f62-878f-4588-82a4-65145daab3d1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_max/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.823524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e80f62-878f-4588-82a4-65145daab3d1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_max)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.825546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e80f62-878f-4588-82a4-65145daab3d1 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.829878 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2239/4508] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/pool_retry_max/default => generated 34 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.829878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983081d0-e394-4239-8cc3-0789c6348d01 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.829878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983081d0-e394-4239-8cc3-0789c6348d01 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.829878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983081d0-e394-4239-8cc3-0789c6348d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.836127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4599bb7-3600-4a11-9dec-5ba9c2e0514d tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorizing `identity:get_role_for_trust(trust_id=45c934cf820948998b69a3a720beed8c, role_id=f75a7c27dd1e4de1a7d6d66525584df6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.839169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4599bb7-3600-4a11-9dec-5ba9c2e0514d tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.853941 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2270/4509] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Oct 15 21:09:30 2022] HEAD /identity/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c/roles/f75a7c27dd1e4de1a7d6d66525584df6 => generated 0 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Oct 15 21:09:30.855953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-983081d0-e394-4239-8cc3-0789c6348d01 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:30.856348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4599bb7-3600-4a11-9dec-5ba9c2e0514d tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.857476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-983081d0-e394-4239-8cc3-0789c6348d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:30.875654 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2240/4510] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:09:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:30.876860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983081d0-e394-4239-8cc3-0789c6348d01 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.893124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983081d0-e394-4239-8cc3-0789c6348d01 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.899884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc769-a590-47a6-8595-323ab96eb1b5 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e76bc3c301242afa9801aad2e66fe53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1469f81b45294aca88bb8bd7b24cc210', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1831003017', 'tempest-NotDelegatedRole-530683875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.900701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc769-a590-47a6-8595-323ab96eb1b5 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.901131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc769-a590-47a6-8595-323ab96eb1b5 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.902053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc769-a590-47a6-8595-323ab96eb1b5 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] PATH_INFO: `/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c/roles/cd14a76b2f0b4075ba495566bb33164f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.911588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdc769-a590-47a6-8595-323ab96eb1b5 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorizing `identity:get_role_for_trust(trust_id=45c934cf820948998b69a3a720beed8c, role_id=cd14a76b2f0b4075ba495566bb33164f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.913685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdc769-a590-47a6-8595-323ab96eb1b5 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.921342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4599bb7-3600-4a11-9dec-5ba9c2e0514d tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.937274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2ba21-58de-42e0-8f73-528dc781d413 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.937942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2ba21-58de-42e0-8f73-528dc781d413 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.938226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2ba21-58de-42e0-8f73-528dc781d413 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.938542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2ba21-58de-42e0-8f73-528dc781d413 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_delay/default` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.939425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2ba21-58de-42e0-8f73-528dc781d413 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_delay)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:30.941747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2ba21-58de-42e0-8f73-528dc781d413 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:30.942874 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2271/4511] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/pool_retry_delay/default => generated 38 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:30.944396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2ba21-58de-42e0-8f73-528dc781d413 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-e6cdc769-a590-47a6-8595-323ab96eb1b5 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Could not find role: cd14a76b2f0b4075ba495566bb33164f.: keystone.exception.RoleNotFound: Could not find role: cd14a76b2f0b4075ba495566bb33164f. Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 21:09:30.961694 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: cd14a76b2f0b4075ba495566bb33164f. Oct 15 21:09:30.962807 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:30.966288 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2241/4512] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c/roles/cd14a76b2f0b4075ba495566bb33164f => generated 110 bytes in 90 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:30.967872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2ba21-58de-42e0-8f73-528dc781d413 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:30.968507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc769-a590-47a6-8595-323ab96eb1b5 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:30.980829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88125588-7286-40f9-bb3f-55b56919ed5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:30.982079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88125588-7286-40f9-bb3f-55b56919ed5f service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:30.982369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88125588-7286-40f9-bb3f-55b56919ed5f service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:30.982644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88125588-7286-40f9-bb3f-55b56919ed5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:30.997788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc769-a590-47a6-8595-323ab96eb1b5 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.003268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88125588-7286-40f9-bb3f-55b56919ed5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.005146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88125588-7286-40f9-bb3f-55b56919ed5f service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.010382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8253b1-e418-4d76-bf32-32b09d04ab6c tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.012783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8253b1-e418-4d76-bf32-32b09d04ab6c tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.013103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8253b1-e418-4d76-bf32-32b09d04ab6c tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.014981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8253b1-e418-4d76-bf32-32b09d04ab6c tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_timeout/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.015633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8253b1-e418-4d76-bf32-32b09d04ab6c tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_timeout)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.017248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8253b1-e418-4d76-bf32-32b09d04ab6c tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.021828 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2272/4513] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:31.024113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88125588-7286-40f9-bb3f-55b56919ed5f service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.027930 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2242/4514] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 21:09:30 2022] GET /identity/v3/domains/config/ldap/pool_connection_timeout/default => generated 44 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:31.028951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8253b1-e418-4d76-bf32-32b09d04ab6c tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.044156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8253b1-e418-4d76-bf32-32b09d04ab6c tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.050417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88125588-7286-40f9-bb3f-55b56919ed5f service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.055236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96241-5457-48cb-a427-a1e23e3f2a39 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e76bc3c301242afa9801aad2e66fe53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1469f81b45294aca88bb8bd7b24cc210', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1831003017', 'tempest-NotDelegatedRole-530683875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.056113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2be1-edf7-4454-ab5a-bd3fcf64cf6f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.056253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96241-5457-48cb-a427-a1e23e3f2a39 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.056456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96241-5457-48cb-a427-a1e23e3f2a39 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.056702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96241-5457-48cb-a427-a1e23e3f2a39 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] PATH_INFO: `/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c/roles/cd14a76b2f0b4075ba495566bb33164f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.065248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd96241-5457-48cb-a427-a1e23e3f2a39 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorizing `identity:get_role_for_trust(trust_id=45c934cf820948998b69a3a720beed8c, role_id=cd14a76b2f0b4075ba495566bb33164f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.066498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd96241-5457-48cb-a427-a1e23e3f2a39 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.067376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2be1-edf7-4454-ab5a-bd3fcf64cf6f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.067671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2be1-edf7-4454-ab5a-bd3fcf64cf6f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.068162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2be1-edf7-4454-ab5a-bd3fcf64cf6f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_lifetime/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.068566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c2be1-edf7-4454-ab5a-bd3fcf64cf6f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_lifetime)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.075400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c2be1-edf7-4454-ab5a-bd3fcf64cf6f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.079788 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2243/4515] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:09:31 2022] GET /identity/v3/domains/config/ldap/pool_connection_lifetime/default => generated 46 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-3bd96241-5457-48cb-a427-a1e23e3f2a39 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Could not find role: cd14a76b2f0b4075ba495566bb33164f.: keystone.exception.RoleNotFound: Could not find role: cd14a76b2f0b4075ba495566bb33164f. Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 21:09:31.080213 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: cd14a76b2f0b4075ba495566bb33164f. Oct 15 21:09:31.084952 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:31.085069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2be1-edf7-4454-ab5a-bd3fcf64cf6f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.085166 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2273/4516] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Oct 15 21:09:31 2022] HEAD /identity/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c/roles/cd14a76b2f0b4075ba495566bb33164f => generated 0 bytes in 59 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Oct 15 21:09:31.085166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96241-5457-48cb-a427-a1e23e3f2a39 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.092323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2be1-edf7-4454-ab5a-bd3fcf64cf6f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.098759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96241-5457-48cb-a427-a1e23e3f2a39 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.098911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cca6-24f8-4080-b6c6-e17f0be2e7eb tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.099537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cca6-24f8-4080-b6c6-e17f0be2e7eb tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.100122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cca6-24f8-4080-b6c6-e17f0be2e7eb tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.100381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cca6-24f8-4080-b6c6-e17f0be2e7eb tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_auth_pool/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.101089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57cca6-24f8-4080-b6c6-e17f0be2e7eb tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_auth_pool)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.103686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57cca6-24f8-4080-b6c6-e17f0be2e7eb tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.105128 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2244/4517] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:31 2022] GET /identity/v3/domains/config/ldap/use_auth_pool/default => generated 36 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:31.106239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e76bc3c301242afa9801aad2e66fe53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1469f81b45294aca88bb8bd7b24cc210', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1831003017', 'tempest-NotDelegatedRole-530683875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.107016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.107300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.107713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] PATH_INFO: `/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.109192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cca6-24f8-4080-b6c6-e17f0be2e7eb tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.114981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorizing `identity:delete_trust(trust_id=45c934cf820948998b69a3a720beed8c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.116555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.126558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cca6-24f8-4080-b6c6-e17f0be2e7eb tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.134556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d0c61-1c81-4a55-8784-da2ba24880f5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.136062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d0c61-1c81-4a55-8784-da2ba24880f5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.136986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d0c61-1c81-4a55-8784-da2ba24880f5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.137452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d0c61-1c81-4a55-8784-da2ba24880f5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_size/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.138057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d0c61-1c81-4a55-8784-da2ba24880f5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_size)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.139238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '45c934cf820948998b69a3a720beed8c', 'request_id': 'req-d6c12a55-f5c3-4524-9cfd-7e0384539a84'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:31.139848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d0c61-1c81-4a55-8784-da2ba24880f5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.142083 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2245/4518] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:31 2022] GET /identity/v3/domains/config/ldap/auth_pool_size/default => generated 36 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 15 21:09:31.142936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '45c934cf820948998b69a3a720beed8c', 'request_id': 'req-d6c12a55-f5c3-4524-9cfd-7e0384539a84'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:31.144502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '45c934cf820948998b69a3a720beed8c', 'request_id': 'req-d6c12a55-f5c3-4524-9cfd-7e0384539a84'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:31.146500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d0c61-1c81-4a55-8784-da2ba24880f5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.148859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-596d0c61-1c81-4a55-8784-da2ba24880f5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:31.171772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d0c61-1c81-4a55-8784-da2ba24880f5 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.171967 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2274/4519] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Oct 15 21:09:31 2022] DELETE /identity/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:31.173157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.174849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:31.187711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fde39-4680-4ffa-bdbf-cb7b7a97ca1b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.188542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fde39-4680-4ffa-bdbf-cb7b7a97ca1b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.188863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fde39-4680-4ffa-bdbf-cb7b7a97ca1b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.193066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fde39-4680-4ffa-bdbf-cb7b7a97ca1b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_connection_lifetime/default` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.193066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fde39-4680-4ffa-bdbf-cb7b7a97ca1b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_connection_lifetime)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.193309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c12a55-f5c3-4524-9cfd-7e0384539a84 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.193394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fde39-4680-4ffa-bdbf-cb7b7a97ca1b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.195137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2246/4520] 10.222.0.16 () {58 vars in 1278 bytes} [Sat Oct 15 21:09:31 2022] GET /identity/v3/domains/config/ldap/auth_pool_connection_lifetime/default => generated 50 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:09:31.202707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fde39-4680-4ffa-bdbf-cb7b7a97ca1b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.203776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2288-2f4a-4b7a-af14-5eaa118adb25 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e76bc3c301242afa9801aad2e66fe53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1469f81b45294aca88bb8bd7b24cc210', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1831003017', 'tempest-NotDelegatedRole-530683875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.209358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2288-2f4a-4b7a-af14-5eaa118adb25 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.209675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2288-2f4a-4b7a-af14-5eaa118adb25 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.209960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2288-2f4a-4b7a-af14-5eaa118adb25 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] PATH_INFO: `/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.219467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fde39-4680-4ffa-bdbf-cb7b7a97ca1b tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.225716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99b4bb-0997-401c-a206-67cdde49658d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.226756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b4bb-0997-401c-a206-67cdde49658d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.227036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b4bb-0997-401c-a206-67cdde49658d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.227307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b4bb-0997-401c-a206-67cdde49658d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/665c2a47a50d4d93b394b925040b1276` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-00de2288-2f4a-4b7a-af14-5eaa118adb25 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Could not find trust: 45c934cf820948998b69a3a720beed8c.: keystone.exception.TrustNotFound: Could not find trust: 45c934cf820948998b69a3a720beed8c. Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Oct 15 21:09:31.232704 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 45c934cf820948998b69a3a720beed8c. Oct 15 21:09:31.234636 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:31.238682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99b4bb-0997-401c-a206-67cdde49658d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:09:31.242094 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2275/4521] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:31 2022] GET /identity/v3/OS-TRUST/trusts/45c934cf820948998b69a3a720beed8c => generated 111 bytes in 70 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-8f99b4bb-0997-401c-a206-67cdde49658d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Could not find domain: 665c2a47a50d4d93b394b925040b1276.: keystone.exception.DomainNotFound: Could not find domain: 665c2a47a50d4d93b394b925040b1276. Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 665c2a47a50d4d93b394b925040b1276. Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:31.246226 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:31.247796 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:09:31.249097 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 665c2a47a50d4d93b394b925040b1276. Oct 15 21:09:31.250304 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:31.250304 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2247/4522] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:31 2022] PATCH /identity/v3/domains/665c2a47a50d4d93b394b925040b1276 => generated 112 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:31.251864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99b4bb-0997-401c-a206-67cdde49658d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.252146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2288-2f4a-4b7a-af14-5eaa118adb25 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.261553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-00de2288-2f4a-4b7a-af14-5eaa118adb25 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:31.275403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99b4bb-0997-401c-a206-67cdde49658d tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.293665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a7fd7-cf04-47a3-924a-2001681b2b3f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.295279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a7fd7-cf04-47a3-924a-2001681b2b3f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.295279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a7fd7-cf04-47a3-924a-2001681b2b3f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.295279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a7fd7-cf04-47a3-924a-2001681b2b3f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/af581daa4ed24de59e1d61c6a168f52e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.304028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2288-2f4a-4b7a-af14-5eaa118adb25 tempest-TrustsV3TestJSON-1798113769 tempest-user-1315161014] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.311324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.312148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.312571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.312914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/cd14a76b2f0b4075ba495566bb33164f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.314535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a7fd7-cf04-47a3-924a-2001681b2b3f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:09:31.314694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=cd14a76b2f0b4075ba495566bb33164f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.317134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-327a7fd7-cf04-47a3-924a-2001681b2b3f tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Could not find domain: af581daa4ed24de59e1d61c6a168f52e.: keystone.exception.DomainNotFound: Could not find domain: af581daa4ed24de59e1d61c6a168f52e. Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: af581daa4ed24de59e1d61c6a168f52e. Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:31.330396 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:31.331421 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:09:31.332795 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: af581daa4ed24de59e1d61c6a168f52e. Oct 15 21:09:31.334030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:31.337125 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2248/4523] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:31 2022] PATCH /identity/v3/domains/af581daa4ed24de59e1d61c6a168f52e => generated 112 bytes in 86 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:31.337764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff56a2c-ef26-451e-bd09-e597ea0e968e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.337980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff56a2c-ef26-451e-bd09-e597ea0e968e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.338360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff56a2c-ef26-451e-bd09-e597ea0e968e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.341779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:31.381297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role cd14a76b2f0b4075ba495566bb33164f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:31.381987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:31.382495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:31.384177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0ff56a2c-ef26-451e-bd09-e597ea0e968e None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:31.384682 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2276/4524] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:31 2022] DELETE /identity/v3/roles/cd14a76b2f0b4075ba495566bb33164f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:31.386362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.386765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0ff56a2c-ef26-451e-bd09-e597ea0e968e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:31.388670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:31.437644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:31.483795 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2249/4525] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:09:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:31.484916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff56a2c-ef26-451e-bd09-e597ea0e968e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.486426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0ff56a2c-ef26-451e-bd09-e597ea0e968e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:31.579269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff56a2c-ef26-451e-bd09-e597ea0e968e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.585642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.586414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.586776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.587209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/f75a7c27dd1e4de1a7d6d66525584df6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.588848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f75a7c27dd1e4de1a7d6d66525584df6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.590905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.647572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role f75a7c27dd1e4de1a7d6d66525584df6 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:31.647968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:31.648988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:31.651746 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2250/4526] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:31 2022] DELETE /identity/v3/roles/f75a7c27dd1e4de1a7d6d66525584df6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:31.653085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.655776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:31.688986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4de16-ff8c-4e1c-bf01-a37016f3415f tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.753941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8190565-cdb0-4620-bbe0-8c1203b971fb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.767229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572482ea-ddc9-41a5-8dee-266589aab927 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f68c907c4484d3dbcb5b1d372ed3502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.768430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572482ea-ddc9-41a5-8dee-266589aab927 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.768807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572482ea-ddc9-41a5-8dee-266589aab927 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.770276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572482ea-ddc9-41a5-8dee-266589aab927 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] PATH_INFO: `/v3/domains/46506ef720b644548b535384c28f1604` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.772257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea5b46-a7b8-4f3f-a03d-a9f1b3e7bd80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.773120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5b46-a7b8-4f3f-a03d-a9f1b3e7bd80 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.773479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5b46-a7b8-4f3f-a03d-a9f1b3e7bd80 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.773803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5b46-a7b8-4f3f-a03d-a9f1b3e7bd80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.777629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f5ea5b46-a7b8-4f3f-a03d-a9f1b3e7bd80 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:31.784272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572482ea-ddc9-41a5-8dee-266589aab927 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-572482ea-ddc9-41a5-8dee-266589aab927 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Could not find domain: 46506ef720b644548b535384c28f1604.: keystone.exception.DomainNotFound: Could not find domain: 46506ef720b644548b535384c28f1604. Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 46506ef720b644548b535384c28f1604. Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:31.791415 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:31.792565 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:09:31.793803 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 46506ef720b644548b535384c28f1604. Oct 15 21:09:31.795115 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:31.795777 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2277/4527] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:31 2022] PATCH /identity/v3/domains/46506ef720b644548b535384c28f1604 => generated 112 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:31.797261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572482ea-ddc9-41a5-8dee-266589aab927 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.799118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-572482ea-ddc9-41a5-8dee-266589aab927 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:31.878513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:31.899812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572482ea-ddc9-41a5-8dee-266589aab927 tempest-DomainConfigurationTestJSON-1798624617 tempest-DomainConfigurationTestJSON-1798624617-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:31.906737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:31.907913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:31.908377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:31.908654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users/1e76bc3c301242afa9801aad2e66fe53` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:31.909103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea5b46-a7b8-4f3f-a03d-a9f1b3e7bd80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.910144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1e76bc3c301242afa9801aad2e66fe53)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:31.910864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea5b46-a7b8-4f3f-a03d-a9f1b3e7bd80 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.912110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:31.925620 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2251/4528] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:09:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:31.926684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea5b46-a7b8-4f3f-a03d-a9f1b3e7bd80 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:31.928685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f5ea5b46-a7b8-4f3f-a03d-a9f1b3e7bd80 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:31.970576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:32.040174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea5b46-a7b8-4f3f-a03d-a9f1b3e7bd80 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:32.048264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:32.048595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e76bc3c301242afa9801aad2e66fe53'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.049095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:32.050191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:32.050540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] PATH_INFO: `/v3/users/4d029b25bf304d5c9c9115d649f106d8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:32.054148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d029b25bf304d5c9c9115d649f106d8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:32.065197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:32.083352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e76bc3c301242afa9801aad2e66fe53'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.101937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e76bc3c301242afa9801aad2e66fe53'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.113709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e76bc3c301242afa9801aad2e66fe53'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.116873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e76bc3c301242afa9801aad2e66fe53'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.153207 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2278/4529] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:31 2022] DELETE /identity/v3/users/1e76bc3c301242afa9801aad2e66fe53 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:32.155250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:32.157181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:32.188678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d029b25bf304d5c9c9115d649f106d8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.233418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d029b25bf304d5c9c9115d649f106d8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.261598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d029b25bf304d5c9c9115d649f106d8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.279268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d029b25bf304d5c9c9115d649f106d8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.282769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b90ba1e9-49dc-4739-810f-b4299c55cd8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d029b25bf304d5c9c9115d649f106d8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.297958 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2252/4530] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:31 2022] DELETE /identity/v3/users/4d029b25bf304d5c9c9115d649f106d8 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:32.305307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b448b4-faaa-455d-97bd-152ac5b41d65 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:32.305970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b448b4-faaa-455d-97bd-152ac5b41d65 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:32.306448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b448b4-faaa-455d-97bd-152ac5b41d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:32.348930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e9f78-4370-4011-aaac-2293ab0baeec tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:32.353800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e1b448b4-faaa-455d-97bd-152ac5b41d65 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:32.358879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e1b448b4-faaa-455d-97bd-152ac5b41d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:32.499875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:32.501976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:32.502629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:32.503174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/1469f81b45294aca88bb8bd7b24cc210` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:32.504606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1469f81b45294aca88bb8bd7b24cc210)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:32.507972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:32.565016 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2253/4531] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:32.569493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b448b4-faaa-455d-97bd-152ac5b41d65 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:32.573729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e1b448b4-faaa-455d-97bd-152ac5b41d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:32.591137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:32.640683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:32.642673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1469f81b45294aca88bb8bd7b24cc210', 'request_id': 'req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.657148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1469f81b45294aca88bb8bd7b24cc210', 'request_id': 'req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.670911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1469f81b45294aca88bb8bd7b24cc210', 'request_id': 'req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.696256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] The token cache is being invalidate because project 1469f81b45294aca88bb8bd7b24cc210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:32.696499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.698535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6dbd5de6-a2a6-4174-952d-2df0ad6fb1bb tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:32.701316 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2279/4532] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:32 2022] DELETE /identity/v3/projects/1469f81b45294aca88bb8bd7b24cc210 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:32.704665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec6860-067f-479f-98ab-2b4e08f0cf63 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:32.704864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec6860-067f-479f-98ab-2b4e08f0cf63 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:32.705248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec6860-067f-479f-98ab-2b4e08f0cf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:32.727712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-50ec6860-067f-479f-98ab-2b4e08f0cf63 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:32.729664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-50ec6860-067f-479f-98ab-2b4e08f0cf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:32.748556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b448b4-faaa-455d-97bd-152ac5b41d65 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:32.812384 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2280/4533] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:32.814281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec6860-067f-479f-98ab-2b4e08f0cf63 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:32.818540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-50ec6860-067f-479f-98ab-2b4e08f0cf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:32.841985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d736e-9fe9-499a-b8fd-80aabd1e8aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:32.842672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d736e-9fe9-499a-b8fd-80aabd1e8aac service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:32.843086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d736e-9fe9-499a-b8fd-80aabd1e8aac service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:32.843449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d736e-9fe9-499a-b8fd-80aabd1e8aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:32.845348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-277d736e-9fe9-499a-b8fd-80aabd1e8aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:32.919927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec6860-067f-479f-98ab-2b4e08f0cf63 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:32.926177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff593d2b-0815-45d2-a5aa-7efba9d37400 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:32.926920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff593d2b-0815-45d2-a5aa-7efba9d37400 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:32.927303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff593d2b-0815-45d2-a5aa-7efba9d37400 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:32.928156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff593d2b-0815-45d2-a5aa-7efba9d37400 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:32.937073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d736e-9fe9-499a-b8fd-80aabd1e8aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:32.938820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d736e-9fe9-499a-b8fd-80aabd1e8aac service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:32.939527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff593d2b-0815-45d2-a5aa-7efba9d37400 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:32.942025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff593d2b-0815-45d2-a5aa-7efba9d37400 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:32.973368 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2254/4534] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:32.978254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d736e-9fe9-499a-b8fd-80aabd1e8aac service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:32.982933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-277d736e-9fe9-499a-b8fd-80aabd1e8aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:32.986854 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2281/4535] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:09:32 2022] POST /identity/v3/projects => generated 328 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:32.992430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff593d2b-0815-45d2-a5aa-7efba9d37400 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:33.147685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff593d2b-0815-45d2-a5aa-7efba9d37400 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:33.155603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c0014-a632-48ea-8222-c466379c5b16 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:33.156611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c0014-a632-48ea-8222-c466379c5b16 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:33.157095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c0014-a632-48ea-8222-c466379c5b16 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:33.157672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c0014-a632-48ea-8222-c466379c5b16 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:33.158708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c0014-a632-48ea-8222-c466379c5b16 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:33.161634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c0014-a632-48ea-8222-c466379c5b16 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:33.177023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d736e-9fe9-499a-b8fd-80aabd1e8aac service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:33.182586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:33.207641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:33.212707 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2282/4536] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:32 2022] POST /identity/v3/users => generated 413 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:33.214262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c0014-a632-48ea-8222-c466379c5b16 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:33.241086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c0014-a632-48ea-8222-c466379c5b16 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:33.250651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ce8b6-5442-4b41-959d-ebb1b99da088 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:33.251670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ce8b6-5442-4b41-959d-ebb1b99da088 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:33.253241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ce8b6-5442-4b41-959d-ebb1b99da088 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:33.254144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ce8b6-5442-4b41-959d-ebb1b99da088 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:33.255275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ce8b6-5442-4b41-959d-ebb1b99da088 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:33.258014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ce8b6-5442-4b41-959d-ebb1b99da088 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:33.286443 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2283/4537] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:33 2022] POST /identity/v3/roles => generated 243 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:33.288703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ce8b6-5442-4b41-959d-ebb1b99da088 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:33.292869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8562e4-a742-4b36-875a-bf888e3f4934 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:33.293830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8562e4-a742-4b36-875a-bf888e3f4934 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:33.294318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8562e4-a742-4b36-875a-bf888e3f4934 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:33.294794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8562e4-a742-4b36-875a-bf888e3f4934 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:33.333545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ce8b6-5442-4b41-959d-ebb1b99da088 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:33.339754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d84c-8169-4d8a-bfee-846c138fc08b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:33.340519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d84c-8169-4d8a-bfee-846c138fc08b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:33.341395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d84c-8169-4d8a-bfee-846c138fc08b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:33.341762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d84c-8169-4d8a-bfee-846c138fc08b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:33.342511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d84c-8169-4d8a-bfee-846c138fc08b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:33.344202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d84c-8169-4d8a-bfee-846c138fc08b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:33.374569 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2284/4538] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:09:33 2022] POST /identity/v3/roles => generated 247 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:33.375680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d84c-8169-4d8a-bfee-846c138fc08b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:33.390905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d84c-8169-4d8a-bfee-846c138fc08b tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:33.397910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998270c-634a-491c-8312-8aa16df92586 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:33.398706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998270c-634a-491c-8312-8aa16df92586 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:33.399080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998270c-634a-491c-8312-8aa16df92586 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:33.399491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998270c-634a-491c-8312-8aa16df92586 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/ebe2d8ec04554ad8a3f96713b94d7b01/users/563fe679af214919b3ffb6e33ef58748/roles/0c16a43b62ba4c839b49ade920698e42` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:33.427176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998270c-634a-491c-8312-8aa16df92586 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ebe2d8ec04554ad8a3f96713b94d7b01, user_id=563fe679af214919b3ffb6e33ef58748, role_id=0c16a43b62ba4c839b49ade920698e42)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:33.430244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8562e4-a742-4b36-875a-bf888e3f4934 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:33.430951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998270c-634a-491c-8312-8aa16df92586 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:33.431830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8562e4-a742-4b36-875a-bf888e3f4934 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:33.446700 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2255/4539] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:09:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:33.461269 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2285/4540] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:33 2022] PUT /identity/v3/projects/ebe2d8ec04554ad8a3f96713b94d7b01/users/563fe679af214919b3ffb6e33ef58748/roles/0c16a43b62ba4c839b49ade920698e42 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:33.468559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8562e4-a742-4b36-875a-bf888e3f4934 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:33.486756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8562e4-a742-4b36-875a-bf888e3f4934 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:33.554978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998270c-634a-491c-8312-8aa16df92586 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:33.557139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9998270c-634a-491c-8312-8aa16df92586 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:33.571169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff723ced-1a27-4261-a61c-0652f12d5908 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:33.571991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff723ced-1a27-4261-a61c-0652f12d5908 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:33.572369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff723ced-1a27-4261-a61c-0652f12d5908 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:33.572759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff723ced-1a27-4261-a61c-0652f12d5908 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/ebe2d8ec04554ad8a3f96713b94d7b01/users/563fe679af214919b3ffb6e33ef58748/roles/40daaa9c3188453e911441b565cf6af1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:33.575936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff723ced-1a27-4261-a61c-0652f12d5908 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ebe2d8ec04554ad8a3f96713b94d7b01, user_id=563fe679af214919b3ffb6e33ef58748, role_id=40daaa9c3188453e911441b565cf6af1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:33.578596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff723ced-1a27-4261-a61c-0652f12d5908 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:33.604055 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2256/4541] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:33 2022] PUT /identity/v3/projects/ebe2d8ec04554ad8a3f96713b94d7b01/users/563fe679af214919b3ffb6e33ef58748/roles/40daaa9c3188453e911441b565cf6af1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:33.609006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff723ced-1a27-4261-a61c-0652f12d5908 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:33.677972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998270c-634a-491c-8312-8aa16df92586 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:33.699808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:33.722889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff723ced-1a27-4261-a61c-0652f12d5908 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:33.728915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e37a39-d820-4440-959d-1981207bbf4e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:33.729548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e37a39-d820-4440-959d-1981207bbf4e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:33.729820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e37a39-d820-4440-959d-1981207bbf4e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:33.730087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e37a39-d820-4440-959d-1981207bbf4e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:33.730742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e37a39-d820-4440-959d-1981207bbf4e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:33.732558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e37a39-d820-4440-959d-1981207bbf4e tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:33.775816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:33.776619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:33.776931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:33.777204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] PATH_INFO: `/v3/projects/a7416e28c90a4a4a8b1fdcf7753a7bc2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:33.778532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7416e28c90a4a4a8b1fdcf7753a7bc2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:33.780489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:33.782625 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2257/4542] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:33 2022] GET /identity/v3/users => generated 8066 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:33.798573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bc17a-b519-4fe3-a04a-ca83e53e9802 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:33.800506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bc17a-b519-4fe3-a04a-ca83e53e9802 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:33.800779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bc17a-b519-4fe3-a04a-ca83e53e9802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:33.885202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7416e28c90a4a4a8b1fdcf7753a7bc2', 'request_id': 'req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:33.894053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2d8bc17a-b519-4fe3-a04a-ca83e53e9802 None None] MFA Rules not processed for user `563fe679af214919b3ffb6e33ef58748`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:33.895164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2d8bc17a-b519-4fe3-a04a-ca83e53e9802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:33.895848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:33.904130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7416e28c90a4a4a8b1fdcf7753a7bc2', 'request_id': 'req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:33.913311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7416e28c90a4a4a8b1fdcf7753a7bc2', 'request_id': 'req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:33.925993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] The token cache is being invalidate because project a7416e28c90a4a4a8b1fdcf7753a7bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:33.926353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:33.926957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6646c63e-f3fb-42e8-8e7e-9863c6f1d3bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:33.928837 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2286/4543] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:33 2022] DELETE /identity/v3/projects/a7416e28c90a4a4a8b1fdcf7753a7bc2 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:33.933439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f0c74-feeb-4850-b77c-bb9c1a83c37e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:33.933750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f0c74-feeb-4850-b77c-bb9c1a83c37e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:33.934124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f0c74-feeb-4850-b77c-bb9c1a83c37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:33.955716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ca6f0c74-feeb-4850-b77c-bb9c1a83c37e None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:33.956901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ca6f0c74-feeb-4850-b77c-bb9c1a83c37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:34.005101 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2258/4544] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:34.011691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bc17a-b519-4fe3-a04a-ca83e53e9802 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:34.013948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2d8bc17a-b519-4fe3-a04a-ca83e53e9802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:34.033268 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2287/4545] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:34.034565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e9034-c528-4a41-9122-539c2c68911f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.034844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e9034-c528-4a41-9122-539c2c68911f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.035275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e9034-c528-4a41-9122-539c2c68911f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.058291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-044e9034-c528-4a41-9122-539c2c68911f None None] MFA Rules not processed for user `563fe679af214919b3ffb6e33ef58748`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:34.059827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-044e9034-c528-4a41-9122-539c2c68911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:34.069077 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2288/4546] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:34.069488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e9034-c528-4a41-9122-539c2c68911f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:34.070921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-044e9034-c528-4a41-9122-539c2c68911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:34.154588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bc17a-b519-4fe3-a04a-ca83e53e9802 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:34.160231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:34.161476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.161762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.162032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] PATH_INFO: `/v3/users/7f68c907c4484d3dbcb5b1d372ed3502` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.175776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f68c907c4484d3dbcb5b1d372ed3502)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:34.179249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e9034-c528-4a41-9122-539c2c68911f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:34.180131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:34.186541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b4b48-9862-49a9-876b-092fe3d41973 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:34.187370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b4b48-9862-49a9-876b-092fe3d41973 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.187797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b4b48-9862-49a9-876b-092fe3d41973 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.188305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b4b48-9862-49a9-876b-092fe3d41973 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.220716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b4b48-9862-49a9-876b-092fe3d41973 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:34.223328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b4b48-9862-49a9-876b-092fe3d41973 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:34.236796 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2289/4547] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:09:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:34.237697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b4b48-9862-49a9-876b-092fe3d41973 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:34.258728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b4b48-9862-49a9-876b-092fe3d41973 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:34.273405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c9fcd-0a3a-4ced-87bc-db17c632ce89 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '563fe679af214919b3ffb6e33ef58748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-578903612', 'tempest-NotDelegatedRole-2128293774'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:34.274116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c9fcd-0a3a-4ced-87bc-db17c632ce89 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.275494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c9fcd-0a3a-4ced-87bc-db17c632ce89 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.275800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c9fcd-0a3a-4ced-87bc-db17c632ce89 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.277709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c9fcd-0a3a-4ced-87bc-db17c632ce89 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorizing `identity:create_trust()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:34.280027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c9fcd-0a3a-4ced-87bc-db17c632ce89 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:34.302748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f68c907c4484d3dbcb5b1d372ed3502'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:34.315939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f68c907c4484d3dbcb5b1d372ed3502'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:34.332394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:34.336222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f68c907c4484d3dbcb5b1d372ed3502'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:34.349204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f68c907c4484d3dbcb5b1d372ed3502'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:34.350140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f68c907c4484d3dbcb5b1d372ed3502'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:34.359277 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2259/4548] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:34 2022] DELETE /identity/v3/users/7f68c907c4484d3dbcb5b1d372ed3502 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:34.406452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:34.415167 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2290/4549] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Oct 15 21:09:34 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:34.419357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:34.421698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:34.452481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c9fcd-0a3a-4ced-87bc-db17c632ce89 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:34.454942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0d4c9fcd-0a3a-4ced-87bc-db17c632ce89 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:34.480901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c9fcd-0a3a-4ced-87bc-db17c632ce89 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:34.531980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c705af-aa4d-4c03-8b78-deb9c8f1ddc5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:34.537374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c46390-c655-4a06-8e0c-9c215441dda4 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '563fe679af214919b3ffb6e33ef58748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-578903612', 'tempest-NotDelegatedRole-2128293774'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:34.538695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c46390-c655-4a06-8e0c-9c215441dda4 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.539074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c46390-c655-4a06-8e0c-9c215441dda4 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.539751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c46390-c655-4a06-8e0c-9c215441dda4 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] PATH_INFO: `/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.550223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c46390-c655-4a06-8e0c-9c215441dda4 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorizing `identity:get_trust(trust_id=24de11e963a846eaa543135fb018c385)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:34.552075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c46390-c655-4a06-8e0c-9c215441dda4 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:34.602749 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2260/4550] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:34 2022] GET /identity/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385 => generated 834 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:34.604158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c46390-c655-4a06-8e0c-9c215441dda4 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:34.611055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:34.611713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.611988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.612256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] PATH_INFO: `/v3/users/4b71f0e126194a99bc76fc9cf44f954b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.618845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b71f0e126194a99bc76fc9cf44f954b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:34.623640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:34.674664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c46390-c655-4a06-8e0c-9c215441dda4 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:34.682869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e82117-2e44-4c7b-aaeb-328412d3ae50 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '563fe679af214919b3ffb6e33ef58748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-578903612', 'tempest-NotDelegatedRole-2128293774'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:34.684107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82117-2e44-4c7b-aaeb-328412d3ae50 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.685475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82117-2e44-4c7b-aaeb-328412d3ae50 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.685475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82117-2e44-4c7b-aaeb-328412d3ae50 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] PATH_INFO: `/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.693424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e82117-2e44-4c7b-aaeb-328412d3ae50 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=24de11e963a846eaa543135fb018c385)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:34.695392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e82117-2e44-4c7b-aaeb-328412d3ae50 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:34.707312 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2261/4551] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:09:34 2022] GET /identity/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385/roles => generated 370 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:34.711193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e82117-2e44-4c7b-aaeb-328412d3ae50 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:34.727966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e82117-2e44-4c7b-aaeb-328412d3ae50 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:34.732959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c879ac-de47-4adc-983d-28c415380315 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '563fe679af214919b3ffb6e33ef58748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-578903612', 'tempest-NotDelegatedRole-2128293774'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:34.733776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c879ac-de47-4adc-983d-28c415380315 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.734054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c879ac-de47-4adc-983d-28c415380315 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.734324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c879ac-de47-4adc-983d-28c415380315 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] PATH_INFO: `/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385/roles/0c16a43b62ba4c839b49ade920698e42` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.750552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c879ac-de47-4adc-983d-28c415380315 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorizing `identity:get_role_for_trust(trust_id=24de11e963a846eaa543135fb018c385, role_id=0c16a43b62ba4c839b49ade920698e42)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:34.750552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c879ac-de47-4adc-983d-28c415380315 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:34.756700 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2262/4552] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Oct 15 21:09:34 2022] GET /identity/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385/roles/0c16a43b62ba4c839b49ade920698e42 => generated 243 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:34.758636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b71f0e126194a99bc76fc9cf44f954b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:34.758779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c879ac-de47-4adc-983d-28c415380315 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:34.784416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b71f0e126194a99bc76fc9cf44f954b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:34.786729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c879ac-de47-4adc-983d-28c415380315 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:34.801824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b71f0e126194a99bc76fc9cf44f954b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:34.802047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4585275-bddd-4072-9f03-a8aead798871 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '563fe679af214919b3ffb6e33ef58748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-578903612', 'tempest-NotDelegatedRole-2128293774'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:34.802515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4585275-bddd-4072-9f03-a8aead798871 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.803426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4585275-bddd-4072-9f03-a8aead798871 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.803426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4585275-bddd-4072-9f03-a8aead798871 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] PATH_INFO: `/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385/roles/0c16a43b62ba4c839b49ade920698e42` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.814653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4585275-bddd-4072-9f03-a8aead798871 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorizing `identity:get_role_for_trust(trust_id=24de11e963a846eaa543135fb018c385, role_id=0c16a43b62ba4c839b49ade920698e42)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:34.816107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4585275-bddd-4072-9f03-a8aead798871 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:34.820846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b71f0e126194a99bc76fc9cf44f954b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:34.821935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-68cee977-d9d2-466f-9c22-1fe81e5b166f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b71f0e126194a99bc76fc9cf44f954b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:34.833551 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2263/4553] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Oct 15 21:09:34 2022] HEAD /identity/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385/roles/0c16a43b62ba4c839b49ade920698e42 => generated 0 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:34.840600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4585275-bddd-4072-9f03-a8aead798871 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:34.843631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b4585275-bddd-4072-9f03-a8aead798871 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:34.856506 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2291/4554] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:34 2022] DELETE /identity/v3/users/4b71f0e126194a99bc76fc9cf44f954b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:34.862061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a8e06-6932-4300-a1c5-c2f69127bd27 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.862342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a8e06-6932-4300-a1c5-c2f69127bd27 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.862704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a8e06-6932-4300-a1c5-c2f69127bd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.866753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4585275-bddd-4072-9f03-a8aead798871 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:34.894588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-930a8e06-6932-4300-a1c5-c2f69127bd27 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:34.898723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-930a8e06-6932-4300-a1c5-c2f69127bd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:34.976992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409efd8-bac6-4f6c-b658-b23aaecb62e9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '563fe679af214919b3ffb6e33ef58748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-578903612', 'tempest-NotDelegatedRole-2128293774'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:34.978875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409efd8-bac6-4f6c-b658-b23aaecb62e9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:34.979843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409efd8-bac6-4f6c-b658-b23aaecb62e9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:34.980337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409efd8-bac6-4f6c-b658-b23aaecb62e9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] PATH_INFO: `/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385/roles/40daaa9c3188453e911441b565cf6af1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:34.990179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409efd8-bac6-4f6c-b658-b23aaecb62e9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorizing `identity:get_role_for_trust(trust_id=24de11e963a846eaa543135fb018c385, role_id=40daaa9c3188453e911441b565cf6af1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:34.992887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409efd8-bac6-4f6c-b658-b23aaecb62e9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-2409efd8-bac6-4f6c-b658-b23aaecb62e9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Could not find role: 40daaa9c3188453e911441b565cf6af1.: keystone.exception.RoleNotFound: Could not find role: 40daaa9c3188453e911441b565cf6af1. Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 21:09:35.011275 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 40daaa9c3188453e911441b565cf6af1. Oct 15 21:09:35.012535 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:35.016571 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2264/4555] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Oct 15 21:09:34 2022] GET /identity/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385/roles/40daaa9c3188453e911441b565cf6af1 => generated 110 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:35.017744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409efd8-bac6-4f6c-b658-b23aaecb62e9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.033076 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2292/4556] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:09:35.040014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409efd8-bac6-4f6c-b658-b23aaecb62e9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:35.044672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a8e06-6932-4300-a1c5-c2f69127bd27 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.045544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-930a8e06-6932-4300-a1c5-c2f69127bd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:35.047448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0748c-5b97-4515-87d5-5addd1c38bff tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '563fe679af214919b3ffb6e33ef58748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-578903612', 'tempest-NotDelegatedRole-2128293774'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:35.048521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0748c-5b97-4515-87d5-5addd1c38bff tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.048963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0748c-5b97-4515-87d5-5addd1c38bff tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.049379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0748c-5b97-4515-87d5-5addd1c38bff tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] PATH_INFO: `/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385/roles/40daaa9c3188453e911441b565cf6af1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.060755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0748c-5b97-4515-87d5-5addd1c38bff tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorizing `identity:get_role_for_trust(trust_id=24de11e963a846eaa543135fb018c385, role_id=40daaa9c3188453e911441b565cf6af1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:35.062514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0748c-5b97-4515-87d5-5addd1c38bff tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:35.087451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-f8b0748c-5b97-4515-87d5-5addd1c38bff tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Could not find role: 40daaa9c3188453e911441b565cf6af1.: keystone.exception.RoleNotFound: Could not find role: 40daaa9c3188453e911441b565cf6af1. Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 21:09:35.093822 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 40daaa9c3188453e911441b565cf6af1. Oct 15 21:09:35.095223 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:09:35.105771 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2265/4557] 10.222.0.16 () {58 vars in 1369 bytes} [Sat Oct 15 21:09:35 2022] HEAD /identity/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385/roles/40daaa9c3188453e911441b565cf6af1 => generated 0 bytes in 81 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:35.105771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0748c-5b97-4515-87d5-5addd1c38bff tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.105771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f8b0748c-5b97-4515-87d5-5addd1c38bff tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:35.134057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:35.148468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0748c-5b97-4515-87d5-5addd1c38bff tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:35.157282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:35.157922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.158423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.158695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] PATH_INFO: `/v3/projects/154c05e6fafa4f0fb4b06e4a1437cd2b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.168440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=154c05e6fafa4f0fb4b06e4a1437cd2b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:35.168609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a8e06-6932-4300-a1c5-c2f69127bd27 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:35.168702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:35.174103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6935e7-a9a1-4001-8e7c-42db15b8fd1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:35.175448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6935e7-a9a1-4001-8e7c-42db15b8fd1e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.176165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6935e7-a9a1-4001-8e7c-42db15b8fd1e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.176874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6935e7-a9a1-4001-8e7c-42db15b8fd1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.191938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6935e7-a9a1-4001-8e7c-42db15b8fd1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:35.194223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6935e7-a9a1-4001-8e7c-42db15b8fd1e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:35.215207 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2293/4558] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:35.218297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6935e7-a9a1-4001-8e7c-42db15b8fd1e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.236486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6935e7-a9a1-4001-8e7c-42db15b8fd1e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:35.255217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'request_id': 'req-b6334403-2719-43d5-bf0e-e07902b8fa45'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.270506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'request_id': 'req-b6334403-2719-43d5-bf0e-e07902b8fa45'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.293483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '154c05e6fafa4f0fb4b06e4a1437cd2b', 'request_id': 'req-b6334403-2719-43d5-bf0e-e07902b8fa45'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.315303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] The token cache is being invalidate because project 154c05e6fafa4f0fb4b06e4a1437cd2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:35.315638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.316574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6334403-2719-43d5-bf0e-e07902b8fa45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.319243 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2266/4559] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:35 2022] DELETE /identity/v3/projects/154c05e6fafa4f0fb4b06e4a1437cd2b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:35.325952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098238a5-1a81-4d6f-8d4c-72471763bafe None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.326236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098238a5-1a81-4d6f-8d4c-72471763bafe None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.326659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098238a5-1a81-4d6f-8d4c-72471763bafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.339286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '563fe679af214919b3ffb6e33ef58748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-578903612', 'tempest-NotDelegatedRole-2128293774'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:35.340385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.341162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.343165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] PATH_INFO: `/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.357937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorizing `identity:delete_trust(trust_id=24de11e963a846eaa543135fb018c385)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:35.360171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-098238a5-1a81-4d6f-8d4c-72471763bafe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:35.361185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:35.361466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-098238a5-1a81-4d6f-8d4c-72471763bafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:35.400829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '24de11e963a846eaa543135fb018c385', 'request_id': 'req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.404273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '24de11e963a846eaa543135fb018c385', 'request_id': 'req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.405922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '24de11e963a846eaa543135fb018c385', 'request_id': 'req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.439592 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2294/4560] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Oct 15 21:09:35 2022] DELETE /identity/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:35.447020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.450209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:35.500448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dadd3-6cb8-4456-8b1b-03723088c1b8 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:35.516644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131c440-ee00-4e88-92a4-3a807cb35ea9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '563fe679af214919b3ffb6e33ef58748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-578903612', 'tempest-NotDelegatedRole-2128293774'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:35.518110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131c440-ee00-4e88-92a4-3a807cb35ea9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.518888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131c440-ee00-4e88-92a4-3a807cb35ea9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.518888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131c440-ee00-4e88-92a4-3a807cb35ea9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] PATH_INFO: `/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.527015 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2267/4561] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:35.528539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdaa76-93ec-484b-92e1-83b95dce13d0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.529116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdaa76-93ec-484b-92e1-83b95dce13d0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.529116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdaa76-93ec-484b-92e1-83b95dce13d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-a131c440-ee00-4e88-92a4-3a807cb35ea9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Could not find trust: 24de11e963a846eaa543135fb018c385.: keystone.exception.TrustNotFound: Could not find trust: 24de11e963a846eaa543135fb018c385. Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Oct 15 21:09:35.536831 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 24de11e963a846eaa543135fb018c385. Oct 15 21:09:35.538005 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:09:35.541450 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2295/4562] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:09:35 2022] GET /identity/v3/OS-TRUST/trusts/24de11e963a846eaa543135fb018c385 => generated 111 bytes in 96 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:09:35.543208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131c440-ee00-4e88-92a4-3a807cb35ea9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.544936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a131c440-ee00-4e88-92a4-3a807cb35ea9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:35.574320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3ebdaa76-93ec-484b-92e1-83b95dce13d0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:35.575559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3ebdaa76-93ec-484b-92e1-83b95dce13d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:35.598142 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2268/4563] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:35.603411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdaa76-93ec-484b-92e1-83b95dce13d0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.618953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdaa76-93ec-484b-92e1-83b95dce13d0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:35.626424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05406e0-0246-47ff-8fb4-f8467b959f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:35.627420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05406e0-0246-47ff-8fb4-f8467b959f25 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.627904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05406e0-0246-47ff-8fb4-f8467b959f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.628458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05406e0-0246-47ff-8fb4-f8467b959f25 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.629791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05406e0-0246-47ff-8fb4-f8467b959f25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:35.630353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05406e0-0246-47ff-8fb4-f8467b959f25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:35.632741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05406e0-0246-47ff-8fb4-f8467b959f25 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:35.652508 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2269/4564] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:35.656818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05406e0-0246-47ff-8fb4-f8467b959f25 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.676163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131c440-ee00-4e88-92a4-3a807cb35ea9 tempest-TrustsV3TestJSON-502850102 tempest-user-141712961] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:35.677460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05406e0-0246-47ff-8fb4-f8467b959f25 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:35.683912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:35.685064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.685346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.686670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/40daaa9c3188453e911441b565cf6af1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.688108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd5d81-c56b-49e4-b950-d0df32bff813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:35.688720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5d81-c56b-49e4-b950-d0df32bff813 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.689792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=40daaa9c3188453e911441b565cf6af1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:35.690831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5d81-c56b-49e4-b950-d0df32bff813 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.691151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5d81-c56b-49e4-b950-d0df32bff813 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.691760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd5d81-c56b-49e4-b950-d0df32bff813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:35.693757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd5d81-c56b-49e4-b950-d0df32bff813 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:35.694164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:35.723263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:35.728398 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2270/4565] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:35 2022] POST /identity/v3/projects => generated 387 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:35.730297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd5d81-c56b-49e4-b950-d0df32bff813 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.733162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-44bd5d81-c56b-49e4-b950-d0df32bff813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:35.744155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role 40daaa9c3188453e911441b565cf6af1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:35.745645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.746338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.748478 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2296/4566] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:35 2022] DELETE /identity/v3/roles/40daaa9c3188453e911441b565cf6af1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:35.750285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.752502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:35.800843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:35.843277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd5d81-c56b-49e4-b950-d0df32bff813 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:35.849515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:35.850161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.850439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.850712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] PATH_INFO: `/v3/projects/490af491d7a046b58e3c850f3bdd3b59` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.852506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de27709-a70b-4c2f-9a7f-b70fb6af3ecd tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:35.852657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=490af491d7a046b58e3c850f3bdd3b59)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:35.855800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:35.858883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4aa07-d92f-4da2-847a-36ece4d5d069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:35.859644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4aa07-d92f-4da2-847a-36ece4d5d069 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:35.860032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4aa07-d92f-4da2-847a-36ece4d5d069 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:35.860395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4aa07-d92f-4da2-847a-36ece4d5d069 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:35.861177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4aa07-d92f-4da2-847a-36ece4d5d069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:35.863240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4aa07-d92f-4da2-847a-36ece4d5d069 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:35.901614 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2297/4567] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:35 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:35.901614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4aa07-d92f-4da2-847a-36ece4d5d069 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.902775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-95a4aa07-d92f-4da2-847a-36ece4d5d069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:35.939866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490af491d7a046b58e3c850f3bdd3b59', 'request_id': 'req-cd8c2f2d-742f-4d59-901f-e607999570b4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.954809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490af491d7a046b58e3c850f3bdd3b59', 'request_id': 'req-cd8c2f2d-742f-4d59-901f-e607999570b4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.967070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490af491d7a046b58e3c850f3bdd3b59', 'request_id': 'req-cd8c2f2d-742f-4d59-901f-e607999570b4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.989377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] The token cache is being invalidate because project 490af491d7a046b58e3c850f3bdd3b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:35.989967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.990924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:35.993809 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2271/4568] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:35 2022] DELETE /identity/v3/projects/490af491d7a046b58e3c850f3bdd3b59 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:35.995247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:35.999868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.060047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4aa07-d92f-4da2-847a-36ece4d5d069 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:36.167552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c2f2d-742f-4d59-901f-e607999570b4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:36.169581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:36.170357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.170713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.170995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/roles/0c16a43b62ba4c839b49ade920698e42` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.172436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0c16a43b62ba4c839b49ade920698e42)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:36.173151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a748d-dd04-482c-9e9b-9554e9c4e2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:36.173797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a748d-dd04-482c-9e9b-9554e9c4e2cf admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.174131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a748d-dd04-482c-9e9b-9554e9c4e2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.174237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:36.174407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a748d-dd04-482c-9e9b-9554e9c4e2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.175107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a748d-dd04-482c-9e9b-9554e9c4e2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:36.177075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a748d-dd04-482c-9e9b-9554e9c4e2cf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:36.202948 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2272/4569] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:35 2022] GET /identity/v3/roles => generated 1654 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:36.204363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b754c83-2ca4-4809-be6d-6d07265d8a2e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.205578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b754c83-2ca4-4809-be6d-6d07265d8a2e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.205858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b754c83-2ca4-4809-be6d-6d07265d8a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.211644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:36.231658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invalidating the token cache because role 0c16a43b62ba4c839b49ade920698e42 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:36.231911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.233423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.234124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3b754c83-2ca4-4809-be6d-6d07265d8a2e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:36.235594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3b754c83-2ca4-4809-be6d-6d07265d8a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.236097 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2298/4570] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:35 2022] DELETE /identity/v3/roles/0c16a43b62ba4c839b49ade920698e42 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:36.237229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:36.238599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.315550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a5012-d208-48da-9b5b-ab244025c867 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:36.323400 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2273/4571] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:36.323400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b754c83-2ca4-4809-be6d-6d07265d8a2e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:36.324823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3b754c83-2ca4-4809-be6d-6d07265d8a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.327772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344be3dc-abda-4e6b-b42c-9bb936497ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:36.328547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344be3dc-abda-4e6b-b42c-9bb936497ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.328969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344be3dc-abda-4e6b-b42c-9bb936497ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.329429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344be3dc-abda-4e6b-b42c-9bb936497ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.330440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344be3dc-abda-4e6b-b42c-9bb936497ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:36.333340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344be3dc-abda-4e6b-b42c-9bb936497ad0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:36.346682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:36.349566 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2299/4572] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:36.350943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8fa1e-42b6-481b-b020-2671298d1d5f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.351224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8fa1e-42b6-481b-b020-2671298d1d5f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.351968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8fa1e-42b6-481b-b020-2671298d1d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.377592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-76e8fa1e-42b6-481b-b020-2671298d1d5f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:36.378996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-76e8fa1e-42b6-481b-b020-2671298d1d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.389826 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2300/4573] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:36.390598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8fa1e-42b6-481b-b020-2671298d1d5f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:36.402904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8fa1e-42b6-481b-b020-2671298d1d5f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:36.408552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5655b33-c5d6-4cbd-a2cf-16b3c8364118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:36.409251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5655b33-c5d6-4cbd-a2cf-16b3c8364118 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.409522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5655b33-c5d6-4cbd-a2cf-16b3c8364118 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.409786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5655b33-c5d6-4cbd-a2cf-16b3c8364118 admin admin] PATH_INFO: `/v3/projects/1a9ae846e8d045e2afaf224fb9d65c82/users/0186ef9f655f48f58751903fad6f7b41/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.430718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5655b33-c5d6-4cbd-a2cf-16b3c8364118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9ae846e8d045e2afaf224fb9d65c82, user_id=0186ef9f655f48f58751903fad6f7b41, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:36.434626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5655b33-c5d6-4cbd-a2cf-16b3c8364118 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:36.435038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b754c83-2ca4-4809-be6d-6d07265d8a2e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:36.441470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:36.442268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.442656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.442930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/users/563fe679af214919b3ffb6e33ef58748` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.447629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_user(user_id=563fe679af214919b3ffb6e33ef58748)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:36.449693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:36.461686 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2301/4574] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:36 2022] PUT /identity/v3/projects/1a9ae846e8d045e2afaf224fb9d65c82/users/0186ef9f655f48f58751903fad6f7b41/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:36.466290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5655b33-c5d6-4cbd-a2cf-16b3c8364118 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:36.522960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563fe679af214919b3ffb6e33ef58748'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.532601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563fe679af214919b3ffb6e33ef58748'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.546986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5655b33-c5d6-4cbd-a2cf-16b3c8364118 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:36.548121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563fe679af214919b3ffb6e33ef58748'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.551809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd64af-d3a0-4170-99e5-edbbe0356da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:36.552437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd64af-d3a0-4170-99e5-edbbe0356da4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.552692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd64af-d3a0-4170-99e5-edbbe0356da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.553138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd64af-d3a0-4170-99e5-edbbe0356da4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.553847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd64af-d3a0-4170-99e5-edbbe0356da4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:36.554235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd64af-d3a0-4170-99e5-edbbe0356da4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:36.555883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd64af-d3a0-4170-99e5-edbbe0356da4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:36.556547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563fe679af214919b3ffb6e33ef58748'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.557449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f0689394-cb4a-4069-a836-bc5dc497ebfa tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563fe679af214919b3ffb6e33ef58748'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.564560 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2274/4575] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:36 2022] DELETE /identity/v3/users/563fe679af214919b3ffb6e33ef58748 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:36.565960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b35b64-a4c6-4cab-811f-474b66e84d41 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.566238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b35b64-a4c6-4cab-811f-474b66e84d41 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.566506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b35b64-a4c6-4cab-811f-474b66e84d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.576351 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2302/4576] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:36.576351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd64af-d3a0-4170-99e5-edbbe0356da4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:36.578786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3ddd64af-d3a0-4170-99e5-edbbe0356da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.601889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d5b35b64-a4c6-4cab-811f-474b66e84d41 None None] MFA Rules not processed for user `0186ef9f655f48f58751903fad6f7b41`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:36.604207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d5b35b64-a4c6-4cab-811f-474b66e84d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.640008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:36.662727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd64af-d3a0-4170-99e5-edbbe0356da4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:36.685060 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2275/4577] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:36.686493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b35b64-a4c6-4cab-811f-474b66e84d41 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:36.689108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d5b35b64-a4c6-4cab-811f-474b66e84d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.723853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:36.736430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f42b0f02f44b509e1ee5b60ff14475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5915c40f3e440db9abb899c7e16961', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:36.737086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.737367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.737633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] PATH_INFO: `/v3/projects/ebe2d8ec04554ad8a3f96713b94d7b01` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.738751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ebe2d8ec04554ad8a3f96713b94d7b01)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:36.740943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:36.771811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b35b64-a4c6-4cab-811f-474b66e84d41 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:36.782940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c6ab9-ceea-44db-a6c3-eab1bdb77820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:36.784189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c6ab9-ceea-44db-a6c3-eab1bdb77820 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.784571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c6ab9-ceea-44db-a6c3-eab1bdb77820 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.785121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c6ab9-ceea-44db-a6c3-eab1bdb77820 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.786092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c6ab9-ceea-44db-a6c3-eab1bdb77820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:36.789443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c6ab9-ceea-44db-a6c3-eab1bdb77820 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:36.802007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'request_id': 'req-7215d2fb-052a-47f6-8c8d-c426d7e7e519'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.810085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'request_id': 'req-7215d2fb-052a-47f6-8c8d-c426d7e7e519'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.821652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebe2d8ec04554ad8a3f96713b94d7b01', 'request_id': 'req-7215d2fb-052a-47f6-8c8d-c426d7e7e519'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.823155 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2276/4578] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:36 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:36.824048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c6ab9-ceea-44db-a6c3-eab1bdb77820 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:36.825733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a71c6ab9-ceea-44db-a6c3-eab1bdb77820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.852949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] The token cache is being invalidate because project ebe2d8ec04554ad8a3f96713b94d7b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:36.853111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.853780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:36.855746 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2303/4579] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:36 2022] DELETE /identity/v3/projects/ebe2d8ec04554ad8a3f96713b94d7b01 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:36.856325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:36.857822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.901854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c6ab9-ceea-44db-a6c3-eab1bdb77820 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:36.906612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f5536-b70b-4706-9604-f2e586ca8eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:36.907220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f5536-b70b-4706-9604-f2e586ca8eee admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.907497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f5536-b70b-4706-9604-f2e586ca8eee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.907764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f5536-b70b-4706-9604-f2e586ca8eee admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.908395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f5536-b70b-4706-9604-f2e586ca8eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:36.910335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f5536-b70b-4706-9604-f2e586ca8eee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:36.939188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215d2fb-052a-47f6-8c8d-c426d7e7e519 tempest-TrustsV3TestJSON-1809463703 tempest-TrustsV3TestJSON-1809463703-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:36.943978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5b7c-561e-4c1e-8109-77f26b8f2ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:36.944581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5b7c-561e-4c1e-8109-77f26b8f2ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:36.944848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5b7c-561e-4c1e-8109-77f26b8f2ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:36.945140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5b7c-561e-4c1e-8109-77f26b8f2ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:36.948688 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2277/4580] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:36 2022] POST /identity/v3/projects => generated 389 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:36.948837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed5b7c-561e-4c1e-8109-77f26b8f2ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:36.948955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f5536-b70b-4706-9604-f2e586ca8eee admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:36.949157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed5b7c-561e-4c1e-8109-77f26b8f2ab2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:36.950178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e33f5536-b70b-4706-9604-f2e586ca8eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.974686 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2304/4581] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:36 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:36.976058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5b7c-561e-4c1e-8109-77f26b8f2ab2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:36.978336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ceed5b7c-561e-4c1e-8109-77f26b8f2ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:36.991771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5b7c-561e-4c1e-8109-77f26b8f2ab2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.005094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:37.035978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f5536-b70b-4706-9604-f2e586ca8eee admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.042474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.043337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.044087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.044367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] PATH_INFO: `/v3/users/2707e97bf01f4941ba74c578b57e080c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.045566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2707e97bf01f4941ba74c578b57e080c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.047676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.067781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e35476-a9c5-4a67-b233-0dce9b79d9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.068393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e35476-a9c5-4a67-b233-0dce9b79d9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.068669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e35476-a9c5-4a67-b233-0dce9b79d9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.068938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e35476-a9c5-4a67-b233-0dce9b79d9b1 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.069599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e35476-a9c5-4a67-b233-0dce9b79d9b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.071522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e35476-a9c5-4a67-b233-0dce9b79d9b1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.091979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:37.096921 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2305/4582] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:36 2022] POST /identity/v3/users => generated 301 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:37.097494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e35476-a9c5-4a67-b233-0dce9b79d9b1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.107358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2707e97bf01f4941ba74c578b57e080c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:37.109798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e35476-a9c5-4a67-b233-0dce9b79d9b1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.115340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b5217-289b-467d-97d5-3442447c1966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.116027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b5217-289b-467d-97d5-3442447c1966 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.116377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b5217-289b-467d-97d5-3442447c1966 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.116489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2707e97bf01f4941ba74c578b57e080c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:37.116649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b5217-289b-467d-97d5-3442447c1966 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.117310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b5217-289b-467d-97d5-3442447c1966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.119538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b5217-289b-467d-97d5-3442447c1966 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.131534 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2306/4583] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:37.132425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b5217-289b-467d-97d5-3442447c1966 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.138785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2707e97bf01f4941ba74c578b57e080c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:37.145550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b5217-289b-467d-97d5-3442447c1966 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.149447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2707e97bf01f4941ba74c578b57e080c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:37.150097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2707e97bf01f4941ba74c578b57e080c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:37.151663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0080bc-cafb-43f5-91fd-d8a0c5b98701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.152438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0080bc-cafb-43f5-91fd-d8a0c5b98701 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.152849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0080bc-cafb-43f5-91fd-d8a0c5b98701 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.153320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0080bc-cafb-43f5-91fd-d8a0c5b98701 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.153953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0080bc-cafb-43f5-91fd-d8a0c5b98701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.155660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0080bc-cafb-43f5-91fd-d8a0c5b98701 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.157174 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2278/4584] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:36 2022] DELETE /identity/v3/users/2707e97bf01f4941ba74c578b57e080c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:37.158225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.161480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:37.175325 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2307/4585] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:37.175325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5230d5-cdbb-41c8-8014-0045be117dac None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.175325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5230d5-cdbb-41c8-8014-0045be117dac None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.175325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5230d5-cdbb-41c8-8014-0045be117dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.195414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ac5230d5-cdbb-41c8-8014-0045be117dac None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:37.196913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac5230d5-cdbb-41c8-8014-0045be117dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:37.261084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb39d3-4b7d-4502-abcd-33c49e41b095 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.274077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85075193-038d-4fac-87e0-ed2c6154b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.274833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85075193-038d-4fac-87e0-ed2c6154b7ff admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.275122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85075193-038d-4fac-87e0-ed2c6154b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.275485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85075193-038d-4fac-87e0-ed2c6154b7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.276480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85075193-038d-4fac-87e0-ed2c6154b7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.278546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85075193-038d-4fac-87e0-ed2c6154b7ff admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.278854 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2308/4586] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:37.286342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5230d5-cdbb-41c8-8014-0045be117dac None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.288299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac5230d5-cdbb-41c8-8014-0045be117dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:37.298504 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2279/4587] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:37.299210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85075193-038d-4fac-87e0-ed2c6154b7ff admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.300888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-85075193-038d-4fac-87e0-ed2c6154b7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:37.318011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5230d5-cdbb-41c8-8014-0045be117dac None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.336417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6cfe2-9ea2-4efa-b98b-b3daa93444da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.337239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6cfe2-9ea2-4efa-b98b-b3daa93444da admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.337589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6cfe2-9ea2-4efa-b98b-b3daa93444da admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.337864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6cfe2-9ea2-4efa-b98b-b3daa93444da admin admin] PATH_INFO: `/v3/projects/1d0fda6ebc7e4034a40984ed322e65e7/users/9580d4713fe840cbb533c0ab2b3a075e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.358874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6cfe2-9ea2-4efa-b98b-b3daa93444da admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0fda6ebc7e4034a40984ed322e65e7, user_id=9580d4713fe840cbb533c0ab2b3a075e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.361949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6cfe2-9ea2-4efa-b98b-b3daa93444da admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.379464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85075193-038d-4fac-87e0-ed2c6154b7ff admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.384547 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2309/4588] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:37 2022] PUT /identity/v3/projects/1d0fda6ebc7e4034a40984ed322e65e7/users/9580d4713fe840cbb533c0ab2b3a075e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:37.388738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6cfe2-9ea2-4efa-b98b-b3daa93444da admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.439897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a86b0-3909-488c-b52c-d72d5366ebd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.440829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a86b0-3909-488c-b52c-d72d5366ebd9 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.441321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a86b0-3909-488c-b52c-d72d5366ebd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.442365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a86b0-3909-488c-b52c-d72d5366ebd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.461925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:37.466488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6cfe2-9ea2-4efa-b98b-b3daa93444da admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.472841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd42da3-9069-4057-b3e3-4e9134d6aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.473589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd42da3-9069-4057-b3e3-4e9134d6aec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.473940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd42da3-9069-4057-b3e3-4e9134d6aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.474233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd42da3-9069-4057-b3e3-4e9134d6aec7 admin admin] PATH_INFO: `/v3/projects/d5ce316f5cf94936b501265f6a8ac463/users/f275e67f5dd14885bfed369d4a4fba65/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.478383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:37.496173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd42da3-9069-4057-b3e3-4e9134d6aec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ce316f5cf94936b501265f6a8ac463, user_id=f275e67f5dd14885bfed369d4a4fba65, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.499058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd42da3-9069-4057-b3e3-4e9134d6aec7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.511908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a86b0-3909-488c-b52c-d72d5366ebd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.513016 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2310/4589] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:37 2022] PUT /identity/v3/projects/d5ce316f5cf94936b501265f6a8ac463/users/f275e67f5dd14885bfed369d4a4fba65/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:37.513415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a86b0-3909-488c-b52c-d72d5366ebd9 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.513988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd42da3-9069-4057-b3e3-4e9134d6aec7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.523603 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2280/4590] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:37.525190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a4fd2-8e21-4bc2-b3be-ce4b4e8752a1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.525472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a4fd2-8e21-4bc2-b3be-ce4b4e8752a1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.525997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a4fd2-8e21-4bc2-b3be-ce4b4e8752a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.555915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cb8a4fd2-8e21-4bc2-b3be-ce4b4e8752a1 None None] MFA Rules not processed for user `f275e67f5dd14885bfed369d4a4fba65`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:37.558010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cb8a4fd2-8e21-4bc2-b3be-ce4b4e8752a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:37.593465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd42da3-9069-4057-b3e3-4e9134d6aec7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.598684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0f2d0-c400-4e18-905c-eefe3389fa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.599364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0f2d0-c400-4e18-905c-eefe3389fa10 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.599680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0f2d0-c400-4e18-905c-eefe3389fa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.599944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0f2d0-c400-4e18-905c-eefe3389fa10 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.600601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0f2d0-c400-4e18-905c-eefe3389fa10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.602373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0f2d0-c400-4e18-905c-eefe3389fa10 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.613051 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2311/4591] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:37.617419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0f2d0-c400-4e18-905c-eefe3389fa10 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.633082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0f2d0-c400-4e18-905c-eefe3389fa10 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.639315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8f256-6167-4985-95da-a624ad524c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.639969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f256-6167-4985-95da-a624ad524c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.640249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f256-6167-4985-95da-a624ad524c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.640521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f256-6167-4985-95da-a624ad524c59 admin admin] PATH_INFO: `/v3/projects/1d0fda6ebc7e4034a40984ed322e65e7/users/9580d4713fe840cbb533c0ab2b3a075e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.642962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8f256-6167-4985-95da-a624ad524c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0fda6ebc7e4034a40984ed322e65e7, user_id=9580d4713fe840cbb533c0ab2b3a075e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.646440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8f256-6167-4985-95da-a624ad524c59 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.647153 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2281/4592] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:37.652181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a4fd2-8e21-4bc2-b3be-ce4b4e8752a1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.660146 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2312/4593] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:37 2022] PUT /identity/v3/projects/1d0fda6ebc7e4034a40984ed322e65e7/users/9580d4713fe840cbb533c0ab2b3a075e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:37.664852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a4fd2-8e21-4bc2-b3be-ce4b4e8752a1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.670862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d59a3-0b1a-4885-9ff6-12fd24bfa944 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.671585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d59a3-0b1a-4885-9ff6-12fd24bfa944 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.671585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d59a3-0b1a-4885-9ff6-12fd24bfa944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.706919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1e1d59a3-0b1a-4885-9ff6-12fd24bfa944 None None] MFA Rules not processed for user `9580d4713fe840cbb533c0ab2b3a075e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:37.708457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e1d59a3-0b1a-4885-9ff6-12fd24bfa944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:37.755037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0b8cf-0a61-49fa-b9fa-f80e6f3f5ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.755675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0b8cf-0a61-49fa-b9fa-f80e6f3f5ecf admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.755979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0b8cf-0a61-49fa-b9fa-f80e6f3f5ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.756207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0b8cf-0a61-49fa-b9fa-f80e6f3f5ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.757455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0b8cf-0a61-49fa-b9fa-f80e6f3f5ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.759391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0b8cf-0a61-49fa-b9fa-f80e6f3f5ecf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:37.811244 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2282/4594] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:37 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:37.812384 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2313/4595] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:09:37.815576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0b8cf-0a61-49fa-b9fa-f80e6f3f5ecf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.816628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d59a3-0b1a-4885-9ff6-12fd24bfa944 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:37.826812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d59a3-0b1a-4885-9ff6-12fd24bfa944 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:37.882131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:37.947148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:37.974697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5be68-0892-46a6-8a3a-f4f56fede79c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:37.976781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5be68-0892-46a6-8a3a-f4f56fede79c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:37.978783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5be68-0892-46a6-8a3a-f4f56fede79c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:37.979909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5be68-0892-46a6-8a3a-f4f56fede79c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects/1a9ae846e8d045e2afaf224fb9d65c82` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:37.983079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5be68-0892-46a6-8a3a-f4f56fede79c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:get_project(project_id=1a9ae846e8d045e2afaf224fb9d65c82)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:37.988978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5be68-0892-46a6-8a3a-f4f56fede79c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.001480 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2314/4596] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:37 2022] GET /identity/v3/projects/1a9ae846e8d045e2afaf224fb9d65c82 => generated 387 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:38.003016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5be68-0892-46a6-8a3a-f4f56fede79c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.005021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4d5be68-0892-46a6-8a3a-f4f56fede79c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:38.007271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0b8cf-0a61-49fa-b9fa-f80e6f3f5ecf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.017581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6b2a1-cd48-46db-84f7-771f7d805ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.018402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6b2a1-cd48-46db-84f7-771f7d805ebd admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.018855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6b2a1-cd48-46db-84f7-771f7d805ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.019219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6b2a1-cd48-46db-84f7-771f7d805ebd admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.019961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6b2a1-cd48-46db-84f7-771f7d805ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.022231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6b2a1-cd48-46db-84f7-771f7d805ebd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.052837 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2283/4597] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:37 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:38.052837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6b2a1-cd48-46db-84f7-771f7d805ebd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.065826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5be68-0892-46a6-8a3a-f4f56fede79c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.066061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6b2a1-cd48-46db-84f7-771f7d805ebd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.074687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2da56-02d6-4ebb-9f8e-d663d2534b3b tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.075493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2da56-02d6-4ebb-9f8e-d663d2534b3b tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.075914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2da56-02d6-4ebb-9f8e-d663d2534b3b tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.076718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2da56-02d6-4ebb-9f8e-d663d2534b3b tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.078345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2da56-02d6-4ebb-9f8e-d663d2534b3b tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.080527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2da56-02d6-4ebb-9f8e-d663d2534b3b tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.087387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.088015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.088308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.089238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] PATH_INFO: `/v3/projects/2902b82407cb43e289a6f03178142dc7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.092040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2902b82407cb43e289a6f03178142dc7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.094175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.128636 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2284/4598] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:09:38 2022] POST /identity/v3/projects => generated 338 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:38.129861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2da56-02d6-4ebb-9f8e-d663d2534b3b tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.165607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2902b82407cb43e289a6f03178142dc7', 'request_id': 'req-438d62ab-2a00-46c2-980b-b8d9249df999'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:38.178307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2902b82407cb43e289a6f03178142dc7', 'request_id': 'req-438d62ab-2a00-46c2-980b-b8d9249df999'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:38.194881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2902b82407cb43e289a6f03178142dc7', 'request_id': 'req-438d62ab-2a00-46c2-980b-b8d9249df999'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:38.213709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2da56-02d6-4ebb-9f8e-d663d2534b3b tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.216694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] The token cache is being invalidate because project 2902b82407cb43e289a6f03178142dc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:38.216979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:38.217830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:38.220837 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2315/4599] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:38 2022] DELETE /identity/v3/projects/2902b82407cb43e289a6f03178142dc7 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:38.222157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.223751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:38.301385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7225cf-424a-42e1-8361-c108fe8e133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.301982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7225cf-424a-42e1-8361-c108fe8e133e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.302266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7225cf-424a-42e1-8361-c108fe8e133e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.302665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7225cf-424a-42e1-8361-c108fe8e133e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.303842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7225cf-424a-42e1-8361-c108fe8e133e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.307043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7225cf-424a-42e1-8361-c108fe8e133e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.322993 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2285/4600] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:38.322993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7225cf-424a-42e1-8361-c108fe8e133e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.325413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ff7225cf-424a-42e1-8361-c108fe8e133e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:38.331440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d62ab-2a00-46c2-980b-b8d9249df999 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.337782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac30af-68f3-4d9d-873e-511f5fca2c18 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.338982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac30af-68f3-4d9d-873e-511f5fca2c18 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.339417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac30af-68f3-4d9d-873e-511f5fca2c18 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.339703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac30af-68f3-4d9d-873e-511f5fca2c18 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.340440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac30af-68f3-4d9d-873e-511f5fca2c18 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.342326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac30af-68f3-4d9d-873e-511f5fca2c18 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.343179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7225cf-424a-42e1-8361-c108fe8e133e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.350027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.350656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.350933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.351200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] PATH_INFO: `/v3/users/56f42b0f02f44b509e1ee5b60ff14475` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.352388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f42b0f02f44b509e1ee5b60ff14475)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.356834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.383896 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2316/4601] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Oct 15 21:09:38 2022] GET /identity/v3/projects => generated 6573 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:38.384935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac30af-68f3-4d9d-873e-511f5fca2c18 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.386780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-11ac30af-68f3-4d9d-873e-511f5fca2c18 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:38.404313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac30af-68f3-4d9d-873e-511f5fca2c18 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.411160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dd952-504a-4c70-8458-a5ad4ffd346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.412016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd952-504a-4c70-8458-a5ad4ffd346f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.412016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd952-504a-4c70-8458-a5ad4ffd346f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.412223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd952-504a-4c70-8458-a5ad4ffd346f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.412760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dd952-504a-4c70-8458-a5ad4ffd346f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.414384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dd952-504a-4c70-8458-a5ad4ffd346f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.421468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:38.427373 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2317/4602] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:38.431451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dd952-504a-4c70-8458-a5ad4ffd346f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.443495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f42b0f02f44b509e1ee5b60ff14475'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:38.443683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dd952-504a-4c70-8458-a5ad4ffd346f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.449611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade81d1-8083-4618-92d5-57280ae4e72d tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.450244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade81d1-8083-4618-92d5-57280ae4e72d tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.450520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade81d1-8083-4618-92d5-57280ae4e72d tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.450792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade81d1-8083-4618-92d5-57280ae4e72d tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects/1a9ae846e8d045e2afaf224fb9d65c82` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.452034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade81d1-8083-4618-92d5-57280ae4e72d tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:get_project(project_id=1a9ae846e8d045e2afaf224fb9d65c82)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.454144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade81d1-8083-4618-92d5-57280ae4e72d tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.455877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f42b0f02f44b509e1ee5b60ff14475'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:38.458911 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2318/4603] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:38 2022] GET /identity/v3/projects/1a9ae846e8d045e2afaf224fb9d65c82 => generated 387 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:38.459714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade81d1-8083-4618-92d5-57280ae4e72d tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.470982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade81d1-8083-4618-92d5-57280ae4e72d tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.473161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f42b0f02f44b509e1ee5b60ff14475'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:38.477066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c8f7b-9257-4829-b15a-fd17b923348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.479459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c8f7b-9257-4829-b15a-fd17b923348f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.479459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c8f7b-9257-4829-b15a-fd17b923348f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.479459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c8f7b-9257-4829-b15a-fd17b923348f admin admin] PATH_INFO: `/v3/projects/60b889e1c32345f2aeba98ef12933717/users/d4614972e267487db45505f2ae0a91e0/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.485345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f42b0f02f44b509e1ee5b60ff14475'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:38.486541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f42b0f02f44b509e1ee5b60ff14475'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:38.496239 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2286/4604] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:38 2022] DELETE /identity/v3/users/56f42b0f02f44b509e1ee5b60ff14475 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:38.496558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.497881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:38.507306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c8f7b-9257-4829-b15a-fd17b923348f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b889e1c32345f2aeba98ef12933717, user_id=d4614972e267487db45505f2ae0a91e0, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.511042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c8f7b-9257-4829-b15a-fd17b923348f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.546346 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2319/4605] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:38 2022] PUT /identity/v3/projects/60b889e1c32345f2aeba98ef12933717/users/d4614972e267487db45505f2ae0a91e0/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:38.547943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e31e3f-ec9e-4aa1-896b-fec9215df1d9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.548223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e31e3f-ec9e-4aa1-896b-fec9215df1d9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.548548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e31e3f-ec9e-4aa1-896b-fec9215df1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.588804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d2e31e3f-ec9e-4aa1-896b-fec9215df1d9 None None] MFA Rules not processed for user `d4614972e267487db45505f2ae0a91e0`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:38.590469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d2e31e3f-ec9e-4aa1-896b-fec9215df1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:38.618096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:38.628520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4edb23-b1d1-4543-8ad4-ef3530eff939 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.686040 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2320/4606] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:38.687028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e31e3f-ec9e-4aa1-896b-fec9215df1d9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.690294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d2e31e3f-ec9e-4aa1-896b-fec9215df1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:38.727546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd6f9-70f4-448d-a65a-b98d0323fad8 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.728406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd6f9-70f4-448d-a65a-b98d0323fad8 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.729655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd6f9-70f4-448d-a65a-b98d0323fad8 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.730231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd6f9-70f4-448d-a65a-b98d0323fad8 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects/50d0c294d5984e38a11242f111671ee5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.731633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccd6f9-70f4-448d-a65a-b98d0323fad8 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:get_project(project_id=50d0c294d5984e38a11242f111671ee5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.737919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccd6f9-70f4-448d-a65a-b98d0323fad8 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.741245 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2287/4607] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:38 2022] GET /identity/v3/projects/50d0c294d5984e38a11242f111671ee5 => generated 338 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:38.749375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd6f9-70f4-448d-a65a-b98d0323fad8 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.762765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd6f9-70f4-448d-a65a-b98d0323fad8 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.769613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19586c5-65f8-44c0-88e3-d540dd83f3a5 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.770519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19586c5-65f8-44c0-88e3-d540dd83f3a5 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.771050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19586c5-65f8-44c0-88e3-d540dd83f3a5 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.771424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19586c5-65f8-44c0-88e3-d540dd83f3a5 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.772168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19586c5-65f8-44c0-88e3-d540dd83f3a5 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.773376 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-e19586c5-65f8-44c0-88e3-d540dd83f3a5 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:38.773376 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:09:38.773376 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:09:38.774773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19586c5-65f8-44c0-88e3-d540dd83f3a5 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.788064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e31e3f-ec9e-4aa1-896b-fec9215df1d9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.793403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f849d6-46ee-449e-aac2-cf1a3b1cb984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.794431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f849d6-46ee-449e-aac2-cf1a3b1cb984 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.794557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f849d6-46ee-449e-aac2-cf1a3b1cb984 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.794772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f849d6-46ee-449e-aac2-cf1a3b1cb984 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.795507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f849d6-46ee-449e-aac2-cf1a3b1cb984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.798182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f849d6-46ee-449e-aac2-cf1a3b1cb984 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.801495 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2288/4608] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Oct 15 21:09:38 2022] GET /identity/v3/projects => generated 6573 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:38.806099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19586c5-65f8-44c0-88e3-d540dd83f3a5 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.816879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19586c5-65f8-44c0-88e3-d540dd83f3a5 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.823965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630007b-fb36-47ac-9a07-e8e78cfbb380 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.824936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630007b-fb36-47ac-9a07-e8e78cfbb380 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.825146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630007b-fb36-47ac-9a07-e8e78cfbb380 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.825422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630007b-fb36-47ac-9a07-e8e78cfbb380 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.826352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630007b-fb36-47ac-9a07-e8e78cfbb380 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:38.826352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630007b-fb36-47ac-9a07-e8e78cfbb380 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.829239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630007b-fb36-47ac-9a07-e8e78cfbb380 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.835046 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2321/4609] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:38 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:38.839187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f849d6-46ee-449e-aac2-cf1a3b1cb984 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.859419 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2289/4610] 10.222.0.16 () {58 vars in 1166 bytes} [Sat Oct 15 21:09:38 2022] GET /identity/v3/projects?domain_id=default => generated 6613 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:09:38.863302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630007b-fb36-47ac-9a07-e8e78cfbb380 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:38.885534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630007b-fb36-47ac-9a07-e8e78cfbb380 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.940254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f849d6-46ee-449e-aac2-cf1a3b1cb984 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:38.945887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f1bf4-9ca9-496c-bbce-e7ccd53fb245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:38.946810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f1bf4-9ca9-496c-bbce-e7ccd53fb245 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:38.947213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f1bf4-9ca9-496c-bbce-e7ccd53fb245 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:38.947581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f1bf4-9ca9-496c-bbce-e7ccd53fb245 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:38.948446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f1bf4-9ca9-496c-bbce-e7ccd53fb245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:38.950529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f1bf4-9ca9-496c-bbce-e7ccd53fb245 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:38.965378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:38.985946 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2322/4611] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:38 2022] POST /identity/v3/users => generated 298 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:38.991421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f1bf4-9ca9-496c-bbce-e7ccd53fb245 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.005789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f1bf4-9ca9-496c-bbce-e7ccd53fb245 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.006286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb10f1-64a9-4d2b-940c-926dbbf507fd tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.007199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb10f1-64a9-4d2b-940c-926dbbf507fd tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.007551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb10f1-64a9-4d2b-940c-926dbbf507fd tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.007920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb10f1-64a9-4d2b-940c-926dbbf507fd tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.008771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb10f1-64a9-4d2b-940c-926dbbf507fd tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.011939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb10f1-64a9-4d2b-940c-926dbbf507fd tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.017824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5f814-6b52-4834-b428-956a24bf8d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.018657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5f814-6b52-4834-b428-956a24bf8d19 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.018931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5f814-6b52-4834-b428-956a24bf8d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.019192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5f814-6b52-4834-b428-956a24bf8d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.020150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5f814-6b52-4834-b428-956a24bf8d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.022979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5f814-6b52-4834-b428-956a24bf8d19 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.037674 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2323/4612] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:09:39.044163 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2290/4613] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Oct 15 21:09:38 2022] GET /identity/v3/projects => generated 6944 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:39.044748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5f814-6b52-4834-b428-956a24bf8d19 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.045577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb10f1-64a9-4d2b-940c-926dbbf507fd tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.056660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5f814-6b52-4834-b428-956a24bf8d19 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.057121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb10f1-64a9-4d2b-940c-926dbbf507fd tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.062955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d4a4e-90ac-4892-9925-97ab12a57760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.063664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d4a4e-90ac-4892-9925-97ab12a57760 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.063941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d4a4e-90ac-4892-9925-97ab12a57760 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.064210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d4a4e-90ac-4892-9925-97ab12a57760 admin admin] PATH_INFO: `/v3/projects/d1dc8e5c6e72468f95736e23ef28f86b/users/9e6a3bbcd47d40d08a8e0650de3380cb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.077177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4f67a-56cd-4f23-b098-d89f2c28b6d4 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.081830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d4a4e-90ac-4892-9925-97ab12a57760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1dc8e5c6e72468f95736e23ef28f86b, user_id=9e6a3bbcd47d40d08a8e0650de3380cb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.082033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4f67a-56cd-4f23-b098-d89f2c28b6d4 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.082458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4f67a-56cd-4f23-b098-d89f2c28b6d4 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.082879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4f67a-56cd-4f23-b098-d89f2c28b6d4 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.084122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4f67a-56cd-4f23-b098-d89f2c28b6d4 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:39.084433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4f67a-56cd-4f23-b098-d89f2c28b6d4 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.084706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d4a4e-90ac-4892-9925-97ab12a57760 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.086321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4f67a-56cd-4f23-b098-d89f2c28b6d4 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.103664 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2324/4614] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:39 2022] PUT /identity/v3/projects/d1dc8e5c6e72468f95736e23ef28f86b/users/9e6a3bbcd47d40d08a8e0650de3380cb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:39.108018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d4a4e-90ac-4892-9925-97ab12a57760 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.114454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:39.114691 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2291/4615] 10.222.0.16 () {58 vars in 1166 bytes} [Sat Oct 15 21:09:39 2022] GET /identity/v3/projects?domain_id=default => generated 6613 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:39.121888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4f67a-56cd-4f23-b098-d89f2c28b6d4 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.139231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4f67a-56cd-4f23-b098-d89f2c28b6d4 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.195035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d4a4e-90ac-4892-9925-97ab12a57760 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.203039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd9e5-3150-46f5-a770-42da8f0a65a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.203594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd9e5-3150-46f5-a770-42da8f0a65a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.203865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd9e5-3150-46f5-a770-42da8f0a65a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.204129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd9e5-3150-46f5-a770-42da8f0a65a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.204762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdd9e5-3150-46f5-a770-42da8f0a65a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.207828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdd9e5-3150-46f5-a770-42da8f0a65a7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.218959 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2325/4616] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:39.225191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd9e5-3150-46f5-a770-42da8f0a65a7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.240464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d1486-c181-4667-bfb5-7c285d1b951c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.241107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d1486-c181-4667-bfb5-7c285d1b951c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.241391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d1486-c181-4667-bfb5-7c285d1b951c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.241666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d1486-c181-4667-bfb5-7c285d1b951c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.242357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d1486-c181-4667-bfb5-7c285d1b951c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.244185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d1486-c181-4667-bfb5-7c285d1b951c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.244514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd9e5-3150-46f5-a770-42da8f0a65a7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.251485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da4dd4-8dc5-4a21-a1ce-1694c27f3123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.252270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da4dd4-8dc5-4a21-a1ce-1694c27f3123 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.252539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da4dd4-8dc5-4a21-a1ce-1694c27f3123 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.252809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da4dd4-8dc5-4a21-a1ce-1694c27f3123 admin admin] PATH_INFO: `/v3/projects/d1dc8e5c6e72468f95736e23ef28f86b/users/9e6a3bbcd47d40d08a8e0650de3380cb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.255460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da4dd4-8dc5-4a21-a1ce-1694c27f3123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1dc8e5c6e72468f95736e23ef28f86b, user_id=9e6a3bbcd47d40d08a8e0650de3380cb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.258386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da4dd4-8dc5-4a21-a1ce-1694c27f3123 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.266384 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2292/4617] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Oct 15 21:09:39 2022] GET /identity/v3/projects => generated 6944 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:39.271231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d1486-c181-4667-bfb5-7c285d1b951c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.285505 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2326/4618] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:39 2022] PUT /identity/v3/projects/d1dc8e5c6e72468f95736e23ef28f86b/users/9e6a3bbcd47d40d08a8e0650de3380cb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:39.290224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d1486-c181-4667-bfb5-7c285d1b951c tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.290688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e97c9f-47d8-43a5-8ac4-2d84ecb43678 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.290976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e97c9f-47d8-43a5-8ac4-2d84ecb43678 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.294077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e97c9f-47d8-43a5-8ac4-2d84ecb43678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.332597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b9e97c9f-47d8-43a5-8ac4-2d84ecb43678 None None] MFA Rules not processed for user `9e6a3bbcd47d40d08a8e0650de3380cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:39.333780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9e97c9f-47d8-43a5-8ac4-2d84ecb43678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:39.362249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aea68-a3da-436f-bdde-e364872262e7 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.362898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aea68-a3da-436f-bdde-e364872262e7 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.363181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aea68-a3da-436f-bdde-e364872262e7 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.363455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aea68-a3da-436f-bdde-e364872262e7 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.365027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08aea68-a3da-436f-bdde-e364872262e7 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Adding query filter params (name=tempest-ListProjectsStaticTestJSON-735681008) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:39.365383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08aea68-a3da-436f-bdde-e364872262e7 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.367246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08aea68-a3da-436f-bdde-e364872262e7 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.390951 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2293/4619] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:39 2022] GET /identity/v3/projects?name=tempest-ListProjectsStaticTestJSON-735681008 => generated 534 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:09:39.396805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aea68-a3da-436f-bdde-e364872262e7 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.408632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aea68-a3da-436f-bdde-e364872262e7 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.413733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9580d4713fe840cbb533c0ab2b3a075e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d0fda6ebc7e4034a40984ed322e65e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.414359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.414646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.414914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] PATH_INFO: `/v3/projects/50d0c294d5984e38a11242f111671ee5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.416116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorizing `identity:delete_project(project_id=50d0c294d5984e38a11242f111671ee5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.418588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.427955 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2327/4620] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:39.440569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e97c9f-47d8-43a5-8ac4-2d84ecb43678 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.441948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9e97c9f-47d8-43a5-8ac4-2d84ecb43678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:39.468853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:39.480068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50d0c294d5984e38a11242f111671ee5', 'request_id': 'req-fe113bea-697b-4f96-bbc1-263aafc1d670'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:39.488964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50d0c294d5984e38a11242f111671ee5', 'request_id': 'req-fe113bea-697b-4f96-bbc1-263aafc1d670'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:39.496330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50d0c294d5984e38a11242f111671ee5', 'request_id': 'req-fe113bea-697b-4f96-bbc1-263aafc1d670'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:39.508422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] The token cache is being invalidate because project 50d0c294d5984e38a11242f111671ee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:39.508713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:39.509382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:39.512752 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2294/4621] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:39 2022] DELETE /identity/v3/projects/50d0c294d5984e38a11242f111671ee5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:39.515581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.517080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:39.518737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e97c9f-47d8-43a5-8ac4-2d84ecb43678 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.550142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:39.588792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb8c9a-054e-4f03-9aad-2164e9a84ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.589596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb8c9a-054e-4f03-9aad-2164e9a84ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.589871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb8c9a-054e-4f03-9aad-2164e9a84ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.590134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb8c9a-054e-4f03-9aad-2164e9a84ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.591952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-25fb8c9a-054e-4f03-9aad-2164e9a84ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:39.609543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe113bea-697b-4f96-bbc1-263aafc1d670 tempest-ListProjectsStaticTestJSON-1278894748 tempest-ListProjectsStaticTestJSON-1278894748-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.614937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.615576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.615852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.616129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] PATH_INFO: `/v3/users/0186ef9f655f48f58751903fad6f7b41` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.617366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0186ef9f655f48f58751903fad6f7b41)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.619262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.690386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb8c9a-054e-4f03-9aad-2164e9a84ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.691970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb8c9a-054e-4f03-9aad-2164e9a84ea3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.704469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0186ef9f655f48f58751903fad6f7b41'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:39.713137 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2328/4622] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:09:39.715313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0186ef9f655f48f58751903fad6f7b41'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:39.730427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0186ef9f655f48f58751903fad6f7b41'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:39.743441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0186ef9f655f48f58751903fad6f7b41'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:39.744217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0186ef9f655f48f58751903fad6f7b41'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:39.752503 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2295/4623] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:39 2022] DELETE /identity/v3/users/0186ef9f655f48f58751903fad6f7b41 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:39.757251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f8271-2b44-422e-8d3c-5f168f03a74a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.757542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f8271-2b44-422e-8d3c-5f168f03a74a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.757813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f8271-2b44-422e-8d3c-5f168f03a74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.768724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.771064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:39.778712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-875f8271-2b44-422e-8d3c-5f168f03a74a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:39.780113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-875f8271-2b44-422e-8d3c-5f168f03a74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:39.870680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:39.900467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade0ed8-2a20-4b47-b845-27def8652df4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:39.906669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:39.907320 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2329/4624] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:39.908043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:39.908532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:39.909491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] PATH_INFO: `/v3/projects/cc5915c40f3e440db9abb899c7e16961` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:39.910682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc5915c40f3e440db9abb899c7e16961)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:39.912814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:39.925383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f8271-2b44-422e-8d3c-5f168f03a74a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:39.927967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-875f8271-2b44-422e-8d3c-5f168f03a74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:40.014468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc5915c40f3e440db9abb899c7e16961', 'request_id': 'req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:40.019529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f8271-2b44-422e-8d3c-5f168f03a74a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:40.024278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc5915c40f3e440db9abb899c7e16961', 'request_id': 'req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:40.032853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc5915c40f3e440db9abb899c7e16961', 'request_id': 'req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:40.056916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] The token cache is being invalidate because project cc5915c40f3e440db9abb899c7e16961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:40.058134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:40.059121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3d6cbe8e-a3d2-4657-8fb4-6c11da70fde4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:40.061600 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2296/4625] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:39 2022] DELETE /identity/v3/projects/cc5915c40f3e440db9abb899c7e16961 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:40.067402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c35996-77cd-48d3-942b-1d865bb97f72 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.067866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c35996-77cd-48d3-942b-1d865bb97f72 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.068431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c35996-77cd-48d3-942b-1d865bb97f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.095414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:40.108427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-47c35996-77cd-48d3-942b-1d865bb97f72 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:40.110076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-47c35996-77cd-48d3-942b-1d865bb97f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:40.118967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e668071-b503-470d-9705-67842919d1b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:40.119188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e668071-b503-470d-9705-67842919d1b7 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.120414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e668071-b503-470d-9705-67842919d1b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.120414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e668071-b503-470d-9705-67842919d1b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.123433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0e668071-b503-470d-9705-67842919d1b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:40.193086 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2297/4626] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:09:40.200074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6ca95-9ae3-4734-923e-9697a574e3c9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.200183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6ca95-9ae3-4734-923e-9697a574e3c9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.204776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6ca95-9ae3-4734-923e-9697a574e3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.230392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-51a6ca95-9ae3-4734-923e-9697a574e3c9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:40.232170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-51a6ca95-9ae3-4734-923e-9697a574e3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:40.240738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e668071-b503-470d-9705-67842919d1b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:40.242135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e668071-b503-470d-9705-67842919d1b7 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:40.249413 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2298/4627] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:40.252379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6ca95-9ae3-4734-923e-9697a574e3c9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:40.254711 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2330/4628] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:40.264483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6ca95-9ae3-4734-923e-9697a574e3c9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:40.270472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cae0c-a704-4eec-ad81-9855caa98b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:40.271187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cae0c-a704-4eec-ad81-9855caa98b42 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.271560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cae0c-a704-4eec-ad81-9855caa98b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.271901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cae0c-a704-4eec-ad81-9855caa98b42 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.272658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cae0c-a704-4eec-ad81-9855caa98b42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:40.273063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cae0c-a704-4eec-ad81-9855caa98b42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:40.274821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cae0c-a704-4eec-ad81-9855caa98b42 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:40.289858 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2299/4629] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:09:40.294255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e668071-b503-470d-9705-67842919d1b7 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:40.306079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e668071-b503-470d-9705-67842919d1b7 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:40.313432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28fbfa-a245-4313-9aec-dde64d6da178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:40.314282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28fbfa-a245-4313-9aec-dde64d6da178 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.314575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28fbfa-a245-4313-9aec-dde64d6da178 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.314883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28fbfa-a245-4313-9aec-dde64d6da178 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.315599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28fbfa-a245-4313-9aec-dde64d6da178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:40.318236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28fbfa-a245-4313-9aec-dde64d6da178 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:40.361133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cae0c-a704-4eec-ad81-9855caa98b42 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:40.362204 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2331/4630] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:40 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:40.384265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cae0c-a704-4eec-ad81-9855caa98b42 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:40.477403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:40.504749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb520c47-1acb-4331-84fe-07005efdf0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:40.506290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb520c47-1acb-4331-84fe-07005efdf0cf admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.506527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb520c47-1acb-4331-84fe-07005efdf0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.508366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb520c47-1acb-4331-84fe-07005efdf0cf admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.511966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb520c47-1acb-4331-84fe-07005efdf0cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:40.513977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb520c47-1acb-4331-84fe-07005efdf0cf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:40.549576 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2300/4631] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:40 2022] POST /identity/v3/users => generated 284 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:40.553708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28fbfa-a245-4313-9aec-dde64d6da178 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:40.570028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28fbfa-a245-4313-9aec-dde64d6da178 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:40.579469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1393e5-215e-492f-8683-11e936970629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:40.580092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1393e5-215e-492f-8683-11e936970629 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.580489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1393e5-215e-492f-8683-11e936970629 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.580774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1393e5-215e-492f-8683-11e936970629 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.582120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1393e5-215e-492f-8683-11e936970629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:40.583809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1393e5-215e-492f-8683-11e936970629 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:40.601352 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2332/4632] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:09:40.605476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb520c47-1acb-4331-84fe-07005efdf0cf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:40.621148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb520c47-1acb-4331-84fe-07005efdf0cf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:40.629331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8129e-e0be-4f44-8992-b91089736bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:40.629774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8129e-e0be-4f44-8992-b91089736bbd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.630388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8129e-e0be-4f44-8992-b91089736bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.630388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8129e-e0be-4f44-8992-b91089736bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.631109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8129e-e0be-4f44-8992-b91089736bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:40.632733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8129e-e0be-4f44-8992-b91089736bbd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:40.644745 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2301/4633] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:40.649065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1393e5-215e-492f-8683-11e936970629 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:40.660924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1393e5-215e-492f-8683-11e936970629 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:40.667700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e680675-ab81-4eb0-983d-262395cae4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:40.668324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e680675-ab81-4eb0-983d-262395cae4b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.668598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e680675-ab81-4eb0-983d-262395cae4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.668864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e680675-ab81-4eb0-983d-262395cae4b6 admin admin] PATH_INFO: `/v3/projects/90898632f7014657aa70f1706a339901/users/fc4d3b581a0547a883c2f0c44ddb5bc6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.689270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e680675-ab81-4eb0-983d-262395cae4b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90898632f7014657aa70f1706a339901, user_id=fc4d3b581a0547a883c2f0c44ddb5bc6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:40.692099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e680675-ab81-4eb0-983d-262395cae4b6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:40.709422 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2333/4634] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:40 2022] PUT /identity/v3/projects/90898632f7014657aa70f1706a339901/users/fc4d3b581a0547a883c2f0c44ddb5bc6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:40.720010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd02c17-4f56-4964-b63a-4dbab5652743 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.720328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd02c17-4f56-4964-b63a-4dbab5652743 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.720607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd02c17-4f56-4964-b63a-4dbab5652743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.771593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e680675-ab81-4eb0-983d-262395cae4b6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:40.774800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8e680675-ab81-4eb0-983d-262395cae4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:40.775952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0cd02c17-4f56-4964-b63a-4dbab5652743 None None] MFA Rules not processed for user `fc4d3b581a0547a883c2f0c44ddb5bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:40.777589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0cd02c17-4f56-4964-b63a-4dbab5652743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:40.877956 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2302/4635] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:40.890299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e680675-ab81-4eb0-983d-262395cae4b6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:40.891936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd02c17-4f56-4964-b63a-4dbab5652743 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:40.897349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:40.898725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.899758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.900283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] PATH_INFO: `/v3/projects/1a9ae846e8d045e2afaf224fb9d65c82` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.903942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a9ae846e8d045e2afaf224fb9d65c82)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:40.908200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:40.915878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd02c17-4f56-4964-b63a-4dbab5652743 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:40.922733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b43d4-682e-45f0-93ad-049e5944e328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:40.923529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b43d4-682e-45f0-93ad-049e5944e328 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:40.923815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b43d4-682e-45f0-93ad-049e5944e328 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:40.924104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b43d4-682e-45f0-93ad-049e5944e328 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:40.926835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b43d4-682e-45f0-93ad-049e5944e328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:40.930323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b43d4-682e-45f0-93ad-049e5944e328 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:40.966174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:40.982907 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2303/4636] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:40 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:40.986786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b43d4-682e-45f0-93ad-049e5944e328 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:40.988914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a9ae846e8d045e2afaf224fb9d65c82', 'request_id': 'req-b0ac3f1e-5aad-469a-a342-e036a7116eba'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:40.998523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a9ae846e8d045e2afaf224fb9d65c82', 'request_id': 'req-b0ac3f1e-5aad-469a-a342-e036a7116eba'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:41.006130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a9ae846e8d045e2afaf224fb9d65c82', 'request_id': 'req-b0ac3f1e-5aad-469a-a342-e036a7116eba'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:41.021800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] The token cache is being invalidate because project 1a9ae846e8d045e2afaf224fb9d65c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:41.022084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:41.022788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:41.024565 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2334/4637] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:40 2022] DELETE /identity/v3/projects/1a9ae846e8d045e2afaf224fb9d65c82 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:41.028924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:41.031313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:41.083796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b43d4-682e-45f0-93ad-049e5944e328 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:41.091634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359796b-bdaf-4a66-8a46-dd4ae7bef35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:41.092537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359796b-bdaf-4a66-8a46-dd4ae7bef35a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:41.092886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359796b-bdaf-4a66-8a46-dd4ae7bef35a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:41.093248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359796b-bdaf-4a66-8a46-dd4ae7bef35a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:41.094066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359796b-bdaf-4a66-8a46-dd4ae7bef35a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:41.096394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359796b-bdaf-4a66-8a46-dd4ae7bef35a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:41.111975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac3f1e-5aad-469a-a342-e036a7116eba admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:41.118710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:41.120176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:41.120620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:41.121305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] PATH_INFO: `/v3/users/9580d4713fe840cbb533c0ab2b3a075e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:41.122451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9580d4713fe840cbb533c0ab2b3a075e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:41.125224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:41.128738 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2304/4638] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:40 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:41.132041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359796b-bdaf-4a66-8a46-dd4ae7bef35a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:41.137172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4359796b-bdaf-4a66-8a46-dd4ae7bef35a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:41.152574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359796b-bdaf-4a66-8a46-dd4ae7bef35a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:41.157642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7d5e8-eeb2-4616-9406-121daab50ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:41.158401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7d5e8-eeb2-4616-9406-121daab50ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:41.158708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7d5e8-eeb2-4616-9406-121daab50ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:41.158976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7d5e8-eeb2-4616-9406-121daab50ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:41.160165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7d5e8-eeb2-4616-9406-121daab50ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:41.162053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7d5e8-eeb2-4616-9406-121daab50ef0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:41.177185 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2305/4639] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:41.182476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7d5e8-eeb2-4616-9406-121daab50ef0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:41.195405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7d5e8-eeb2-4616-9406-121daab50ef0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:41.217565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc86d54-ec53-40a9-997f-4a5ddd52bc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:41.218279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86d54-ec53-40a9-997f-4a5ddd52bc24 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:41.218613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86d54-ec53-40a9-997f-4a5ddd52bc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:41.219097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86d54-ec53-40a9-997f-4a5ddd52bc24 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:41.220909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc86d54-ec53-40a9-997f-4a5ddd52bc24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:41.223810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc86d54-ec53-40a9-997f-4a5ddd52bc24 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:41.234359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9580d4713fe840cbb533c0ab2b3a075e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:41.252773 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2306/4640] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:41.253880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9580d4713fe840cbb533c0ab2b3a075e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:41.256347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc86d54-ec53-40a9-997f-4a5ddd52bc24 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:41.264784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9580d4713fe840cbb533c0ab2b3a075e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:41.273830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc86d54-ec53-40a9-997f-4a5ddd52bc24 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:41.279270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9580d4713fe840cbb533c0ab2b3a075e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:41.280347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d36cd732-7639-4cbc-b193-63783c8c148f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9580d4713fe840cbb533c0ab2b3a075e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:41.280922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e1386-c794-4d00-9853-00b30af675ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:41.281742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e1386-c794-4d00-9853-00b30af675ac admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:41.282226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e1386-c794-4d00-9853-00b30af675ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:41.282525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e1386-c794-4d00-9853-00b30af675ac admin admin] PATH_INFO: `/v3/projects/a057913d70df49038d15a3f1c91761b0/users/706bb7d795ac48318000ff4a3ead304e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:41.294071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:41.299354 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2335/4641] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:41 2022] DELETE /identity/v3/users/9580d4713fe840cbb533c0ab2b3a075e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:41.304181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e1386-c794-4d00-9853-00b30af675ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a057913d70df49038d15a3f1c91761b0, user_id=706bb7d795ac48318000ff4a3ead304e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:41.308064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e1386-c794-4d00-9853-00b30af675ac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:41.333099 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2307/4642] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:41 2022] PUT /identity/v3/projects/a057913d70df49038d15a3f1c91761b0/users/706bb7d795ac48318000ff4a3ead304e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:41.336908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6add26-6fde-459b-a195-2db1dc35731a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:41.337333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6add26-6fde-459b-a195-2db1dc35731a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:41.337709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6add26-6fde-459b-a195-2db1dc35731a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:41.376344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5f6add26-6fde-459b-a195-2db1dc35731a None None] MFA Rules not processed for user `706bb7d795ac48318000ff4a3ead304e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:41.377634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5f6add26-6fde-459b-a195-2db1dc35731a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:41.419713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:41.469333 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2336/4643] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:41.484156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e1386-c794-4d00-9853-00b30af675ac admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:41.486107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-208e1386-c794-4d00-9853-00b30af675ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:41.597358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e1386-c794-4d00-9853-00b30af675ac admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:41.605489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4f223-8de5-48f6-8a20-77a70b15752c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:41.606116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4f223-8de5-48f6-8a20-77a70b15752c service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:41.606391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4f223-8de5-48f6-8a20-77a70b15752c service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:41.606660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4f223-8de5-48f6-8a20-77a70b15752c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:41.611061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-deb4f223-8de5-48f6-8a20-77a70b15752c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:41.704874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4f223-8de5-48f6-8a20-77a70b15752c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:41.707177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4f223-8de5-48f6-8a20-77a70b15752c service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:41.719683 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2308/4644] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:42.011964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6add26-6fde-459b-a195-2db1dc35731a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:42.013804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5f6add26-6fde-459b-a195-2db1dc35731a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:42.038858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6add26-6fde-459b-a195-2db1dc35731a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:42.045423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ed823-767b-4916-a1f9-ef30b4945caf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:42.046095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed823-767b-4916-a1f9-ef30b4945caf service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:42.046416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed823-767b-4916-a1f9-ef30b4945caf service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:42.046856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed823-767b-4916-a1f9-ef30b4945caf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:42.058225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ed823-767b-4916-a1f9-ef30b4945caf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:42.060695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ed823-767b-4916-a1f9-ef30b4945caf service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:42.086701 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2337/4645] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:42.115061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4f223-8de5-48f6-8a20-77a70b15752c service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:42.116858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-deb4f223-8de5-48f6-8a20-77a70b15752c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:42.308222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4f223-8de5-48f6-8a20-77a70b15752c service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:42.330675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:42.339600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:42.340546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:42.340828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] PATH_INFO: `/v3/projects/1d0fda6ebc7e4034a40984ed322e65e7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:42.345788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d0fda6ebc7e4034a40984ed322e65e7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:42.352115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:42.461762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:42.471257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d0fda6ebc7e4034a40984ed322e65e7', 'request_id': 'req-d46b41f5-1bdb-493d-853f-f1f9223c435b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:42.502467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d0fda6ebc7e4034a40984ed322e65e7', 'request_id': 'req-d46b41f5-1bdb-493d-853f-f1f9223c435b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:42.527075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d0fda6ebc7e4034a40984ed322e65e7', 'request_id': 'req-d46b41f5-1bdb-493d-853f-f1f9223c435b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:42.547794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] The token cache is being invalidate because project 1d0fda6ebc7e4034a40984ed322e65e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:42.550213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:42.551034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d46b41f5-1bdb-493d-853f-f1f9223c435b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:42.555410 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2309/4646] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:42 2022] DELETE /identity/v3/projects/1d0fda6ebc7e4034a40984ed322e65e7 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:42.562419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed881f-17c8-4841-ad88-ebdecd3e154d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:42.562815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed881f-17c8-4841-ad88-ebdecd3e154d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:42.563189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed881f-17c8-4841-ad88-ebdecd3e154d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:42.593309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e4ed881f-17c8-4841-ad88-ebdecd3e154d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:42.594934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e4ed881f-17c8-4841-ad88-ebdecd3e154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:42.720472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb8580-3437-4627-bec1-fe52136644f8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:42.720807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb8580-3437-4627-bec1-fe52136644f8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:42.721012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb8580-3437-4627-bec1-fe52136644f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:42.732045 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2338/4647] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:42.750589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2feb8580-3437-4627-bec1-fe52136644f8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:42.751905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2feb8580-3437-4627-bec1-fe52136644f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:42.764359 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2310/4648] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:42.769614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed881f-17c8-4841-ad88-ebdecd3e154d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:42.796116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed881f-17c8-4841-ad88-ebdecd3e154d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:42.812308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84351a56-bbc7-4063-a483-2e5c4f859523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:42.813352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84351a56-bbc7-4063-a483-2e5c4f859523 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:42.813669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84351a56-bbc7-4063-a483-2e5c4f859523 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:42.813944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84351a56-bbc7-4063-a483-2e5c4f859523 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:42.814775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84351a56-bbc7-4063-a483-2e5c4f859523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:42.815080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84351a56-bbc7-4063-a483-2e5c4f859523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:42.819212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84351a56-bbc7-4063-a483-2e5c4f859523 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:42.846028 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2339/4649] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:42.850818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb8580-3437-4627-bec1-fe52136644f8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:42.866168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb8580-3437-4627-bec1-fe52136644f8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:42.884670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb66310-0ef6-40d8-b89a-7b532ab0db4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:42.888154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66310-0ef6-40d8-b89a-7b532ab0db4d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:42.888663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66310-0ef6-40d8-b89a-7b532ab0db4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:42.891088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66310-0ef6-40d8-b89a-7b532ab0db4d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:42.891987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb66310-0ef6-40d8-b89a-7b532ab0db4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:42.897657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb66310-0ef6-40d8-b89a-7b532ab0db4d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:42.956334 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2311/4650] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:42 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:42.963193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84351a56-bbc7-4063-a483-2e5c4f859523 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:42.982375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84351a56-bbc7-4063-a483-2e5c4f859523 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:43.082593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0c0fc-b3a6-4541-9ba6-4ae8da258092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:43.083252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0c0fc-b3a6-4541-9ba6-4ae8da258092 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:43.083469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0c0fc-b3a6-4541-9ba6-4ae8da258092 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:43.083739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0c0fc-b3a6-4541-9ba6-4ae8da258092 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:43.084620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0c0fc-b3a6-4541-9ba6-4ae8da258092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:43.087133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0c0fc-b3a6-4541-9ba6-4ae8da258092 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:43.148851 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2340/4651] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:42 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:43.150041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb66310-0ef6-40d8-b89a-7b532ab0db4d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:43.164793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb66310-0ef6-40d8-b89a-7b532ab0db4d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:43.178558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2930f3-7a5b-4267-9d3e-d6a3ff8af2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:43.179182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2930f3-7a5b-4267-9d3e-d6a3ff8af2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:43.179459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2930f3-7a5b-4267-9d3e-d6a3ff8af2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:43.179723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2930f3-7a5b-4267-9d3e-d6a3ff8af2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:43.180519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2930f3-7a5b-4267-9d3e-d6a3ff8af2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:43.184802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2930f3-7a5b-4267-9d3e-d6a3ff8af2b1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:43.231863 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2312/4652] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:43.237160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0c0fc-b3a6-4541-9ba6-4ae8da258092 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:43.255695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0c0fc-b3a6-4541-9ba6-4ae8da258092 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:43.271378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aa0a9-cd7f-40e2-8944-612011a221ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:43.273077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aa0a9-cd7f-40e2-8944-612011a221ba admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:43.273375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aa0a9-cd7f-40e2-8944-612011a221ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:43.273648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aa0a9-cd7f-40e2-8944-612011a221ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:43.275777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988aa0a9-cd7f-40e2-8944-612011a221ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:43.279969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988aa0a9-cd7f-40e2-8944-612011a221ba admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:43.313088 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2341/4653] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:43.319438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2930f3-7a5b-4267-9d3e-d6a3ff8af2b1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:43.340624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2930f3-7a5b-4267-9d3e-d6a3ff8af2b1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:43.354511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8622d04-0c70-408f-a572-a6ddc08d7732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:43.355358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8622d04-0c70-408f-a572-a6ddc08d7732 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:43.355721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8622d04-0c70-408f-a572-a6ddc08d7732 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:43.357995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8622d04-0c70-408f-a572-a6ddc08d7732 admin admin] PATH_INFO: `/v3/projects/735785a804474cd699326cf9f241e879/users/a6723b1d8fe44a8b94638b2d17237913/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:43.382522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8622d04-0c70-408f-a572-a6ddc08d7732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=735785a804474cd699326cf9f241e879, user_id=a6723b1d8fe44a8b94638b2d17237913, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:43.391090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8622d04-0c70-408f-a572-a6ddc08d7732 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:43.455994 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2313/4654] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:43 2022] PUT /identity/v3/projects/735785a804474cd699326cf9f241e879/users/a6723b1d8fe44a8b94638b2d17237913/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:43.457654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1f790-b4e1-4aec-aede-ae57691a7e9b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:43.457937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1f790-b4e1-4aec-aede-ae57691a7e9b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:43.458762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1f790-b4e1-4aec-aede-ae57691a7e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:43.496539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-66d1f790-b4e1-4aec-aede-ae57691a7e9b None None] MFA Rules not processed for user `a6723b1d8fe44a8b94638b2d17237913`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:43.497987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-66d1f790-b4e1-4aec-aede-ae57691a7e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:43.515074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:43.712805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8622d04-0c70-408f-a572-a6ddc08d7732 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:43.735848 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2342/4655] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:43.875294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8622d04-0c70-408f-a572-a6ddc08d7732 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:43.880696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fe37c-88ce-4360-a710-a45fede74328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:43.881545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fe37c-88ce-4360-a710-a45fede74328 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:43.881829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fe37c-88ce-4360-a710-a45fede74328 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:43.882098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fe37c-88ce-4360-a710-a45fede74328 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:43.882742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fe37c-88ce-4360-a710-a45fede74328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:43.884684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fe37c-88ce-4360-a710-a45fede74328 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:43.925503 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2314/4656] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:43 2022] POST /identity/v3/projects => generated 365 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:43.929551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1f790-b4e1-4aec-aede-ae57691a7e9b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:43.948139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1f790-b4e1-4aec-aede-ae57691a7e9b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:44.067001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d486c0-76b0-4f9f-9a81-5f1c5634a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:44.068585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d486c0-76b0-4f9f-9a81-5f1c5634a656 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:44.068866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d486c0-76b0-4f9f-9a81-5f1c5634a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:44.069156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d486c0-76b0-4f9f-9a81-5f1c5634a656 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:44.069816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d486c0-76b0-4f9f-9a81-5f1c5634a656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:44.077903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d486c0-76b0-4f9f-9a81-5f1c5634a656 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:44.147243 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2343/4657] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:43 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:44.152145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fe37c-88ce-4360-a710-a45fede74328 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:44.167534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fe37c-88ce-4360-a710-a45fede74328 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:44.188781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6200c-9f39-4438-8fdf-53a1cafb02d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:44.190236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6200c-9f39-4438-8fdf-53a1cafb02d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:44.193054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6200c-9f39-4438-8fdf-53a1cafb02d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:44.193397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6200c-9f39-4438-8fdf-53a1cafb02d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:44.194092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6200c-9f39-4438-8fdf-53a1cafb02d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:44.198524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6200c-9f39-4438-8fdf-53a1cafb02d5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:44.235683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d486c0-76b0-4f9f-9a81-5f1c5634a656 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:44.242574 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2315/4658] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:44.252031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d486c0-76b0-4f9f-9a81-5f1c5634a656 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:44.267526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeddd86-7e5f-4a75-a3ca-d33729bb843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:44.268819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeddd86-7e5f-4a75-a3ca-d33729bb843e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:44.269229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeddd86-7e5f-4a75-a3ca-d33729bb843e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:44.269498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeddd86-7e5f-4a75-a3ca-d33729bb843e admin admin] PATH_INFO: `/v3/projects/1507e1c38667436f8c3caf9aa1a9cf35/users/442ba1c2564b4c1fa87ff9e660d30969/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:44.305304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeddd86-7e5f-4a75-a3ca-d33729bb843e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1507e1c38667436f8c3caf9aa1a9cf35, user_id=442ba1c2564b4c1fa87ff9e660d30969, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:44.309472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeddd86-7e5f-4a75-a3ca-d33729bb843e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:44.336470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6200c-9f39-4438-8fdf-53a1cafb02d5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:44.337371 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2344/4659] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:44 2022] PUT /identity/v3/projects/1507e1c38667436f8c3caf9aa1a9cf35/users/442ba1c2564b4c1fa87ff9e660d30969/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:44.359137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6200c-9f39-4438-8fdf-53a1cafb02d5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:44.430814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:44.456630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c1039-4ae3-41d4-aba1-dc9aae2ac397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:44.457665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1039-4ae3-41d4-aba1-dc9aae2ac397 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:44.458936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1039-4ae3-41d4-aba1-dc9aae2ac397 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:44.459258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1039-4ae3-41d4-aba1-dc9aae2ac397 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:44.459975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c1039-4ae3-41d4-aba1-dc9aae2ac397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:44.461758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c1039-4ae3-41d4-aba1-dc9aae2ac397 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:44.474995 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2316/4660] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:44.480352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeddd86-7e5f-4a75-a3ca-d33729bb843e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:44.494920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeddd86-7e5f-4a75-a3ca-d33729bb843e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:44.501196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de3999-68e8-423c-b3c9-83bcac949eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:44.501757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de3999-68e8-423c-b3c9-83bcac949eec admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:44.502031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de3999-68e8-423c-b3c9-83bcac949eec admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:44.502441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de3999-68e8-423c-b3c9-83bcac949eec admin admin] PATH_INFO: `/v3/projects/1507e1c38667436f8c3caf9aa1a9cf35/users/442ba1c2564b4c1fa87ff9e660d30969/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:44.504672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de3999-68e8-423c-b3c9-83bcac949eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1507e1c38667436f8c3caf9aa1a9cf35, user_id=442ba1c2564b4c1fa87ff9e660d30969, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:44.507389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de3999-68e8-423c-b3c9-83bcac949eec admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:44.521505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c1039-4ae3-41d4-aba1-dc9aae2ac397 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:44.534325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa8c1039-4ae3-41d4-aba1-dc9aae2ac397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:44.541829 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2345/4661] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:44 2022] PUT /identity/v3/projects/1507e1c38667436f8c3caf9aa1a9cf35/users/442ba1c2564b4c1fa87ff9e660d30969/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:44.554295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc59256-aaba-479d-a551-22b49823162b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:44.554576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc59256-aaba-479d-a551-22b49823162b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:44.554845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc59256-aaba-479d-a551-22b49823162b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:44.560344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c1039-4ae3-41d4-aba1-dc9aae2ac397 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:44.658741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-adc59256-aaba-479d-a551-22b49823162b None None] MFA Rules not processed for user `442ba1c2564b4c1fa87ff9e660d30969`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:44.659326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-adc59256-aaba-479d-a551-22b49823162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:44.687432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:44.688656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:44.689113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:44.689560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] PATH_INFO: `/v3/users/fc4d3b581a0547a883c2f0c44ddb5bc6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:44.691495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc4d3b581a0547a883c2f0c44ddb5bc6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:44.696378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:44.795808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc4d3b581a0547a883c2f0c44ddb5bc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:44.804930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:44.819346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc4d3b581a0547a883c2f0c44ddb5bc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:44.846861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc4d3b581a0547a883c2f0c44ddb5bc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:44.863168 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2346/4662] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:44.866937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc4d3b581a0547a883c2f0c44ddb5bc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:44.867372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc59256-aaba-479d-a551-22b49823162b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:44.867669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7ca8144-2029-45ea-b1b1-97237b545f41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc4d3b581a0547a883c2f0c44ddb5bc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:44.879809 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2317/4663] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:44 2022] DELETE /identity/v3/users/fc4d3b581a0547a883c2f0c44ddb5bc6 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:44.884005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce16463-1035-402c-bdb2-0c12a3cc9589 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:44.884377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce16463-1035-402c-bdb2-0c12a3cc9589 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:44.884789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce16463-1035-402c-bdb2-0c12a3cc9589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:44.893848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc59256-aaba-479d-a551-22b49823162b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:44.911554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5ce16463-1035-402c-bdb2-0c12a3cc9589 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:44.913021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5ce16463-1035-402c-bdb2-0c12a3cc9589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:44.995986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:45.015902 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2318/4664] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:45.038914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce16463-1035-402c-bdb2-0c12a3cc9589 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:45.040597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5ce16463-1035-402c-bdb2-0c12a3cc9589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:45.073570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a9bc7-808e-44ce-8290-100e84006929 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:45.074247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a9bc7-808e-44ce-8290-100e84006929 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:45.074787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a9bc7-808e-44ce-8290-100e84006929 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:45.075456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a9bc7-808e-44ce-8290-100e84006929 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:45.076721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a9bc7-808e-44ce-8290-100e84006929 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:45.079236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a9bc7-808e-44ce-8290-100e84006929 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:45.118660 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2347/4665] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:09:44 2022] POST /identity/v3/domains => generated 276 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:45.120835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a9bc7-808e-44ce-8290-100e84006929 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:45.124050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-121a9bc7-808e-44ce-8290-100e84006929 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:45.159197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce16463-1035-402c-bdb2-0c12a3cc9589 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:45.281959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a9bc7-808e-44ce-8290-100e84006929 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:45.293854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdc598-0cdd-43e2-b684-c3246d32b841 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:45.294493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdc598-0cdd-43e2-b684-c3246d32b841 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:45.295364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdc598-0cdd-43e2-b684-c3246d32b841 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:45.295502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdc598-0cdd-43e2-b684-c3246d32b841 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:45.296154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdc598-0cdd-43e2-b684-c3246d32b841 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:45.298960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdc598-0cdd-43e2-b684-c3246d32b841 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:45.321528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b9cfd-d36c-4562-a504-508baa429a7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:45.322117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b9cfd-d36c-4562-a504-508baa429a7b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:45.323498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b9cfd-d36c-4562-a504-508baa429a7b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:45.323864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b9cfd-d36c-4562-a504-508baa429a7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:45.358229 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2348/4666] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:45 2022] POST /identity/v3/users => generated 347 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:45.362483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdc598-0cdd-43e2-b684-c3246d32b841 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:45.396997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdc598-0cdd-43e2-b684-c3246d32b841 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:45.415122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a729cd-11a4-4f0d-b55d-0aedb93d031e tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:45.415744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a729cd-11a4-4f0d-b55d-0aedb93d031e tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:45.416018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a729cd-11a4-4f0d-b55d-0aedb93d031e tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:45.416934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a729cd-11a4-4f0d-b55d-0aedb93d031e tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:45.423381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a729cd-11a4-4f0d-b55d-0aedb93d031e tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:45.425357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a729cd-11a4-4f0d-b55d-0aedb93d031e tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:45.433638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b9cfd-d36c-4562-a504-508baa429a7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:45.436848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b9cfd-d36c-4562-a504-508baa429a7b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:45.459467 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2319/4667] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:09:45.480649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b9cfd-d36c-4562-a504-508baa429a7b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:45.490976 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2349/4668] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:45 2022] POST /identity/v3/users => generated 323 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:45.494622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b9cfd-d36c-4562-a504-508baa429a7b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:45.500695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd989a-6caa-47e8-95a6-a9c19bbe3381 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:45.501544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd989a-6caa-47e8-95a6-a9c19bbe3381 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:45.501944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd989a-6caa-47e8-95a6-a9c19bbe3381 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:45.502379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd989a-6caa-47e8-95a6-a9c19bbe3381 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/users/52890c58883e43a68e7605bb281cf065` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:45.533623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd989a-6caa-47e8-95a6-a9c19bbe3381 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:get_user(user_id=52890c58883e43a68e7605bb281cf065)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:45.538819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd989a-6caa-47e8-95a6-a9c19bbe3381 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:45.550830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a729cd-11a4-4f0d-b55d-0aedb93d031e tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:45.561554 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2320/4669] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:09:45 2022] GET /identity/v3/users/52890c58883e43a68e7605bb281cf065 => generated 347 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:45.565693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a729cd-11a4-4f0d-b55d-0aedb93d031e tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:45.588949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f2be3-fded-4c73-b5df-af26df5c502c tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:45.590669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f2be3-fded-4c73-b5df-af26df5c502c tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:45.591598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f2be3-fded-4c73-b5df-af26df5c502c tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:45.591878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f2be3-fded-4c73-b5df-af26df5c502c tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:45.597942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f2be3-fded-4c73-b5df-af26df5c502c tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Adding query filter params (domain_id=9ca814855a7a46e0ba968d92c5598dde) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:45.598319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f2be3-fded-4c73-b5df-af26df5c502c tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:45.620612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f2be3-fded-4c73-b5df-af26df5c502c tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:45.670285 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2350/4670] 10.222.0.16 () {58 vars in 1207 bytes} [Sat Oct 15 21:09:45 2022] GET /identity/v3/users?domain_id=9ca814855a7a46e0ba968d92c5598dde => generated 484 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:45.671324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd989a-6caa-47e8-95a6-a9c19bbe3381 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:45.700217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd989a-6caa-47e8-95a6-a9c19bbe3381 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:45.715227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fb604-23d4-45aa-99e1-ca9f10b116c7 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:45.715911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fb604-23d4-45aa-99e1-ca9f10b116c7 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:45.716185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fb604-23d4-45aa-99e1-ca9f10b116c7 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:45.716453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fb604-23d4-45aa-99e1-ca9f10b116c7 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:45.717130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8fb604-23d4-45aa-99e1-ca9f10b116c7 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:45.726628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8fb604-23d4-45aa-99e1-ca9f10b116c7 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:45.811480 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2321/4671] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:45 2022] GET /identity/v3/users => generated 8863 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:45.818016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f2be3-fded-4c73-b5df-af26df5c502c tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:45.831581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f2be3-fded-4c73-b5df-af26df5c502c tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:45.837314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4550241-c93d-4c51-ab6a-80a5d53ecb77 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:45.838164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4550241-c93d-4c51-ab6a-80a5d53ecb77 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:45.838624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4550241-c93d-4c51-ab6a-80a5d53ecb77 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:45.839115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4550241-c93d-4c51-ab6a-80a5d53ecb77 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:45.840294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4550241-c93d-4c51-ab6a-80a5d53ecb77 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Adding query filter params (name=tempest-test_user-1234655194) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:45.840750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4550241-c93d-4c51-ab6a-80a5d53ecb77 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:45.844051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4550241-c93d-4c51-ab6a-80a5d53ecb77 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:45.894449 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2351/4672] 10.222.0.16 () {58 vars in 1189 bytes} [Sat Oct 15 21:09:45 2022] GET /identity/v3/users?name=tempest-test_user-1234655194 => generated 475 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:45.895450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fb604-23d4-45aa-99e1-ca9f10b116c7 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:45.924796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fb604-23d4-45aa-99e1-ca9f10b116c7 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:45.949581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03620f6-5361-4fa7-bac1-beeb32e819fa tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:45.950385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03620f6-5361-4fa7-bac1-beeb32e819fa tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:45.950784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03620f6-5361-4fa7-bac1-beeb32e819fa tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:45.952291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03620f6-5361-4fa7-bac1-beeb32e819fa tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:45.953384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03620f6-5361-4fa7-bac1-beeb32e819fa tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:45.955359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03620f6-5361-4fa7-bac1-beeb32e819fa tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:45.959707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03620f6-5361-4fa7-bac1-beeb32e819fa tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:46.021478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4550241-c93d-4c51-ab6a-80a5d53ecb77 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:46.028693 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2322/4673] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 21:09:45 2022] GET /identity/v3/users?enabled=False => generated 431 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:46.037395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4550241-c93d-4c51-ab6a-80a5d53ecb77 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:46.045344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:46.045998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:46.047023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:46.047023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/users/6aa2cb46fb1d4dcea38630d6e4073b91` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:46.065091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:delete_user(user_id=6aa2cb46fb1d4dcea38630d6e4073b91)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:46.067099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:46.069684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:46.163979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa2cb46fb1d4dcea38630d6e4073b91'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.193874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa2cb46fb1d4dcea38630d6e4073b91'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.216133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa2cb46fb1d4dcea38630d6e4073b91'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.229055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa2cb46fb1d4dcea38630d6e4073b91'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.230838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa2cb46fb1d4dcea38630d6e4073b91'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.240710 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2352/4674] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:46 2022] DELETE /identity/v3/users/6aa2cb46fb1d4dcea38630d6e4073b91 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:46.244555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03620f6-5361-4fa7-bac1-beeb32e819fa tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:46.246573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f03620f6-5361-4fa7-bac1-beeb32e819fa tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:46.264657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03620f6-5361-4fa7-bac1-beeb32e819fa tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:46.305931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:46.365459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:46.366284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:46.369714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:46.370335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/users/52890c58883e43a68e7605bb281cf065` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:46.373797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:delete_user(user_id=52890c58883e43a68e7605bb281cf065)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:46.375910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:46.391331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:46.394334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:46.495118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52890c58883e43a68e7605bb281cf065'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.508736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52890c58883e43a68e7605bb281cf065'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.532320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52890c58883e43a68e7605bb281cf065'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.546570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703eb47-1c97-4886-abf9-ec2ae941e046 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:46.552024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:46.552671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:46.553110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:46.553400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] PATH_INFO: `/v3/projects/90898632f7014657aa70f1706a339901` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:46.554623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90898632f7014657aa70f1706a339901)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:46.556554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:46.559617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52890c58883e43a68e7605bb281cf065'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.565239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52890c58883e43a68e7605bb281cf065'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.588956 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2323/4675] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:46 2022] DELETE /identity/v3/users/52890c58883e43a68e7605bb281cf065 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:46.593536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:46.596761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:46.647211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90898632f7014657aa70f1706a339901', 'request_id': 'req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.658756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90898632f7014657aa70f1706a339901', 'request_id': 'req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.668869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90898632f7014657aa70f1706a339901', 'request_id': 'req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.694099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] The token cache is being invalidate because project 90898632f7014657aa70f1706a339901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:46.694395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.695229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.699096 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2353/4676] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:46 2022] DELETE /identity/v3/projects/90898632f7014657aa70f1706a339901 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:46.702863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:46.705139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:46.731224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:46.755910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7ad73-8a33-4f1e-850b-675d458affe2 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:46.871602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:46.894633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:46.898003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:46.898284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:46.898558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/domains/9ca814855a7a46e0ba968d92c5598dde` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:46.899866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=9ca814855a7a46e0ba968d92c5598dde)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:46.907674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:46.909265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9485c5-5f46-4b0d-a4ff-643fa945f9e2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:46.927882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:46.928286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:46.928569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:46.928843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] PATH_INFO: `/v3/users/706bb7d795ac48318000ff4a3ead304e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:46.935341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] RBAC: Authorizing `identity:delete_user(user_id=706bb7d795ac48318000ff4a3ead304e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:46.935341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:46.973272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:46.976099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.036434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.037835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.051844 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2324/4677] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:46 2022] PATCH /identity/v3/domains/9ca814855a7a46e0ba968d92c5598dde => generated 277 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:47.055083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:47.059776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:47.076784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '706bb7d795ac48318000ff4a3ead304e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.100518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '706bb7d795ac48318000ff4a3ead304e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.133230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '706bb7d795ac48318000ff4a3ead304e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.157994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '706bb7d795ac48318000ff4a3ead304e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.160896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '706bb7d795ac48318000ff4a3ead304e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.178705 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2354/4678] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:46 2022] DELETE /identity/v3/users/706bb7d795ac48318000ff4a3ead304e => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:47.227391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0fdda-23b1-41f2-96a2-7a6b5d75ddf4 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:47.362595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '442ba1c2564b4c1fa87ff9e660d30969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1507e1c38667436f8c3caf9aa1a9cf35', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:47.363441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:47.363821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:47.364492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] PATH_INFO: `/v3/domains/9ca814855a7a46e0ba968d92c5598dde` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:47.379307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=9ca814855a7a46e0ba968d92c5598dde)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:47.381367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:47.416141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.437922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.490298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:47.505537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde', 'request_id': 'req-a1f5b189-2a99-464f-8c10-6d4878b49394'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.518496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde', 'request_id': 'req-a1f5b189-2a99-464f-8c10-6d4878b49394'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.529910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde', 'request_id': 'req-a1f5b189-2a99-464f-8c10-6d4878b49394'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.556688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] The token cache is being invalidate because project 9ca814855a7a46e0ba968d92c5598dde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:47.557231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.558148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.571327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde', 'request_id': 'req-a1f5b189-2a99-464f-8c10-6d4878b49394'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.572153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde', 'request_id': 'req-a1f5b189-2a99-464f-8c10-6d4878b49394'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.573909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1f5b189-2a99-464f-8c10-6d4878b49394 tempest-UsersV3TestJSON-569358903 tempest-UsersV3TestJSON-569358903-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '9ca814855a7a46e0ba968d92c5598dde', 'request_id': 'req-a1f5b189-2a99-464f-8c10-6d4878b49394'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.584506 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2325/4679] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:47 2022] DELETE /identity/v3/domains/9ca814855a7a46e0ba968d92c5598dde => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:47.589743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:47.591467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:47.691299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab4292-5b07-43bf-a8a3-da72a20b233e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:47.741848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:47.800720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:47.802096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:47.802390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:47.802664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] PATH_INFO: `/v3/users/a6723b1d8fe44a8b94638b2d17237913` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:47.803822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6723b1d8fe44a8b94638b2d17237913)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:47.805890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:47.875779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6723b1d8fe44a8b94638b2d17237913'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.892478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6723b1d8fe44a8b94638b2d17237913'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.908574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6723b1d8fe44a8b94638b2d17237913'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.919493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6723b1d8fe44a8b94638b2d17237913'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.920335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6723b1d8fe44a8b94638b2d17237913'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:47.928151 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2355/4680] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:47 2022] DELETE /identity/v3/users/a6723b1d8fe44a8b94638b2d17237913 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:47.933008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b1b59-39e0-4fad-bd21-79273c6c5082 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:47.933402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b1b59-39e0-4fad-bd21-79273c6c5082 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:47.933854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b1b59-39e0-4fad-bd21-79273c6c5082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:47.963851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1e8b1b59-39e0-4fad-bd21-79273c6c5082 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:47.965102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1e8b1b59-39e0-4fad-bd21-79273c6c5082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:48.046125 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2326/4681] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:48.094964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:48.096565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:48.178918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd190-f184-4e2c-9bfd-0fce627d50bb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:48.186709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be814a8-b015-4ead-bf3d-1db131a88abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:48.187575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be814a8-b015-4ead-bf3d-1db131a88abd service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:48.187929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be814a8-b015-4ead-bf3d-1db131a88abd service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:48.188272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be814a8-b015-4ead-bf3d-1db131a88abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:48.220258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be814a8-b015-4ead-bf3d-1db131a88abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:48.222474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be814a8-b015-4ead-bf3d-1db131a88abd service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:48.240598 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2356/4682] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:48.335164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b1b59-39e0-4fad-bd21-79273c6c5082 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:48.337850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1e8b1b59-39e0-4fad-bd21-79273c6c5082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:48.352538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b1b59-39e0-4fad-bd21-79273c6c5082 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:48.359892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:48.360534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:48.361389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:48.361389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] PATH_INFO: `/v3/projects/a057913d70df49038d15a3f1c91761b0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:48.362477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a057913d70df49038d15a3f1c91761b0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:48.364434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:48.466720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a057913d70df49038d15a3f1c91761b0', 'request_id': 'req-36d699e7-4b73-43bc-b30b-5d2b879c9de6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:48.477472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a057913d70df49038d15a3f1c91761b0', 'request_id': 'req-36d699e7-4b73-43bc-b30b-5d2b879c9de6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:48.486810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a057913d70df49038d15a3f1c91761b0', 'request_id': 'req-36d699e7-4b73-43bc-b30b-5d2b879c9de6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:48.503548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] The token cache is being invalidate because project a057913d70df49038d15a3f1c91761b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:48.503906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:48.504642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-36d699e7-4b73-43bc-b30b-5d2b879c9de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:48.507621 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2327/4683] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:48 2022] DELETE /identity/v3/projects/a057913d70df49038d15a3f1c91761b0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:48.513720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11a65-76c3-4e6b-a836-9d9debcc5de1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:48.514011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11a65-76c3-4e6b-a836-9d9debcc5de1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:48.514270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11a65-76c3-4e6b-a836-9d9debcc5de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:48.540430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-24a11a65-76c3-4e6b-a836-9d9debcc5de1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:48.542199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-24a11a65-76c3-4e6b-a836-9d9debcc5de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:48.602015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:48.621687 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2357/4684] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:48.622662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a23f9-a17c-4d98-b946-3b147402db6c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:48.622945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a23f9-a17c-4d98-b946-3b147402db6c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:48.623211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a23f9-a17c-4d98-b946-3b147402db6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:48.646277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6f9a23f9-a17c-4d98-b946-3b147402db6c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:48.647504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6f9a23f9-a17c-4d98-b946-3b147402db6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:48.658034 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2328/4685] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:09:48.661752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a11a65-76c3-4e6b-a836-9d9debcc5de1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:48.672234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a11a65-76c3-4e6b-a836-9d9debcc5de1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:48.678524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e226aa-6ad3-4b1e-b96f-93f6dae149b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:48.679158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e226aa-6ad3-4b1e-b96f-93f6dae149b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:48.679546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e226aa-6ad3-4b1e-b96f-93f6dae149b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:48.679671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e226aa-6ad3-4b1e-b96f-93f6dae149b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:48.680451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e226aa-6ad3-4b1e-b96f-93f6dae149b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:48.680641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e226aa-6ad3-4b1e-b96f-93f6dae149b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:48.685636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e226aa-6ad3-4b1e-b96f-93f6dae149b7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:48.699652 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2358/4686] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:48.704032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a23f9-a17c-4d98-b946-3b147402db6c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:48.715564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a23f9-a17c-4d98-b946-3b147402db6c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:48.721079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673db41-7dd9-4936-8f87-8b52065d13a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:48.721842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673db41-7dd9-4936-8f87-8b52065d13a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:48.722219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673db41-7dd9-4936-8f87-8b52065d13a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:48.722684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673db41-7dd9-4936-8f87-8b52065d13a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:48.723390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673db41-7dd9-4936-8f87-8b52065d13a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:48.727372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673db41-7dd9-4936-8f87-8b52065d13a7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:48.755966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:48.766313 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2329/4687] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:48 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:48.770362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e226aa-6ad3-4b1e-b96f-93f6dae149b7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:48.795456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e226aa-6ad3-4b1e-b96f-93f6dae149b7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:48.882234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff93674-a777-4387-ad76-3013065024fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:48.883139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff93674-a777-4387-ad76-3013065024fe admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:48.883416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff93674-a777-4387-ad76-3013065024fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:48.883680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff93674-a777-4387-ad76-3013065024fe admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:48.884395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff93674-a777-4387-ad76-3013065024fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:48.886998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff93674-a777-4387-ad76-3013065024fe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:48.921095 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2359/4688] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:48 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:48.923548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673db41-7dd9-4936-8f87-8b52065d13a7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:48.940087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673db41-7dd9-4936-8f87-8b52065d13a7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:48.951373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3ab3b-47cb-453d-a77c-c4b2f8a6b5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:48.952113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3ab3b-47cb-453d-a77c-c4b2f8a6b5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:48.952519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3ab3b-47cb-453d-a77c-c4b2f8a6b5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:48.952891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3ab3b-47cb-453d-a77c-c4b2f8a6b5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:48.953799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3ab3b-47cb-453d-a77c-c4b2f8a6b5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:48.956167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3ab3b-47cb-453d-a77c-c4b2f8a6b5f1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:48.974511 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2330/4689] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:48.976408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff93674-a777-4387-ad76-3013065024fe admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:48.993711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff93674-a777-4387-ad76-3013065024fe admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:49.007229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ee1e3-dc11-468f-860b-187d1e9a919c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:49.007845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ee1e3-dc11-468f-860b-187d1e9a919c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:49.008116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ee1e3-dc11-468f-860b-187d1e9a919c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:49.008382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ee1e3-dc11-468f-860b-187d1e9a919c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:49.009069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ee1e3-dc11-468f-860b-187d1e9a919c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:49.011766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ee1e3-dc11-468f-860b-187d1e9a919c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:49.033260 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2360/4690] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:49.053101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3ab3b-47cb-453d-a77c-c4b2f8a6b5f1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:49.069935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3ab3b-47cb-453d-a77c-c4b2f8a6b5f1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:49.082475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e6640-bcbf-4acf-9a8f-45d75732c41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:49.082475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e6640-bcbf-4acf-9a8f-45d75732c41c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:49.082475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e6640-bcbf-4acf-9a8f-45d75732c41c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:49.082475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e6640-bcbf-4acf-9a8f-45d75732c41c admin admin] PATH_INFO: `/v3/projects/549f9ced9a7741e2a5888093586fe8c6/users/42499cd684ac4cf1b77bbc44e7937eb3/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:49.120213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e6640-bcbf-4acf-9a8f-45d75732c41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=549f9ced9a7741e2a5888093586fe8c6, user_id=42499cd684ac4cf1b77bbc44e7937eb3, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:49.123964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e6640-bcbf-4acf-9a8f-45d75732c41c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:49.135616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ee1e3-dc11-468f-860b-187d1e9a919c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:49.146568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-444ee1e3-dc11-468f-860b-187d1e9a919c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:49.177955 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2331/4691] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:49 2022] PUT /identity/v3/projects/549f9ced9a7741e2a5888093586fe8c6/users/42499cd684ac4cf1b77bbc44e7937eb3/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:49.189449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ee1e3-dc11-468f-860b-187d1e9a919c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:49.194752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e93d81-fbc1-4452-92d5-086921193d1d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:49.195056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e93d81-fbc1-4452-92d5-086921193d1d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:49.195355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e93d81-fbc1-4452-92d5-086921193d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:49.292609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-88e93d81-fbc1-4452-92d5-086921193d1d None None] MFA Rules not processed for user `42499cd684ac4cf1b77bbc44e7937eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:49.295524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-88e93d81-fbc1-4452-92d5-086921193d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:49.381964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:49.382526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:49.382803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:49.386063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] PATH_INFO: `/v3/projects/735785a804474cd699326cf9f241e879` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:49.388325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=735785a804474cd699326cf9f241e879)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:49.404151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:49.446923 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2332/4692] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:49.461071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e93d81-fbc1-4452-92d5-086921193d1d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:49.463478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-88e93d81-fbc1-4452-92d5-086921193d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:49.550581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '735785a804474cd699326cf9f241e879', 'request_id': 'req-ac215565-3c74-4de9-b888-f9a31ddc0f63'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:49.560022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '735785a804474cd699326cf9f241e879', 'request_id': 'req-ac215565-3c74-4de9-b888-f9a31ddc0f63'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:49.570054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '735785a804474cd699326cf9f241e879', 'request_id': 'req-ac215565-3c74-4de9-b888-f9a31ddc0f63'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:49.590491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:49.609121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e93d81-fbc1-4452-92d5-086921193d1d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:49.609375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] The token cache is being invalidate because project 735785a804474cd699326cf9f241e879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:49.612410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:49.613417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:49.619972 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2361/4693] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:49 2022] DELETE /identity/v3/projects/735785a804474cd699326cf9f241e879 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:49.622117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:49.624368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:49.756593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce862b3-9247-4b1a-a65e-ae7669239beb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:49.757770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce862b3-9247-4b1a-a65e-ae7669239beb service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:49.763117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce862b3-9247-4b1a-a65e-ae7669239beb service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:49.764103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce862b3-9247-4b1a-a65e-ae7669239beb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:49.768796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fce862b3-9247-4b1a-a65e-ae7669239beb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:49.770525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac215565-3c74-4de9-b888-f9a31ddc0f63 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:49.779427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:49.782784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:49.783849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:49.784678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] PATH_INFO: `/v3/users/442ba1c2564b4c1fa87ff9e660d30969` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:49.788267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=442ba1c2564b4c1fa87ff9e660d30969)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:49.790593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:49.829767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:49.908138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce862b3-9247-4b1a-a65e-ae7669239beb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:49.910032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce862b3-9247-4b1a-a65e-ae7669239beb service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:49.914661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '442ba1c2564b4c1fa87ff9e660d30969'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:49.923469 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2333/4694] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:49.976326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '442ba1c2564b4c1fa87ff9e660d30969'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:50.022463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '442ba1c2564b4c1fa87ff9e660d30969'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:50.041979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '442ba1c2564b4c1fa87ff9e660d30969'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:50.043117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2966f3a7-085f-449d-9bbc-cfb9ca0231c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '442ba1c2564b4c1fa87ff9e660d30969'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:50.076493 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2362/4695] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:49 2022] DELETE /identity/v3/users/442ba1c2564b4c1fa87ff9e660d30969 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:50.705334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce862b3-9247-4b1a-a65e-ae7669239beb service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:50.734089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce862b3-9247-4b1a-a65e-ae7669239beb service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:50.853066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:50.854199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:50.854572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:50.855405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] PATH_INFO: `/v3/projects/1507e1c38667436f8c3caf9aa1a9cf35` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:50.856668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1507e1c38667436f8c3caf9aa1a9cf35)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:50.858880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:50.976236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1507e1c38667436f8c3caf9aa1a9cf35', 'request_id': 'req-992aeffe-309c-4393-a8ed-f74a12d6be0f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:50.995758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1507e1c38667436f8c3caf9aa1a9cf35', 'request_id': 'req-992aeffe-309c-4393-a8ed-f74a12d6be0f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:51.008010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1507e1c38667436f8c3caf9aa1a9cf35', 'request_id': 'req-992aeffe-309c-4393-a8ed-f74a12d6be0f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:51.050977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] The token cache is being invalidate because project 1507e1c38667436f8c3caf9aa1a9cf35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:51.051873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:51.052736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-992aeffe-309c-4393-a8ed-f74a12d6be0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:51.058840 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2334/4696] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:50 2022] DELETE /identity/v3/projects/1507e1c38667436f8c3caf9aa1a9cf35 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:51.062649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d0e32-f91b-44b4-8739-b9557abf2feb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:51.063364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d0e32-f91b-44b4-8739-b9557abf2feb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:51.063552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d0e32-f91b-44b4-8739-b9557abf2feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:51.119600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8c7d0e32-f91b-44b4-8739-b9557abf2feb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:51.122871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8c7d0e32-f91b-44b4-8739-b9557abf2feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:51.332671 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2363/4697] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:51.339133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f41d-9410-4dbe-ba3c-5afb6858c543 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:51.339419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f41d-9410-4dbe-ba3c-5afb6858c543 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:51.339695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f41d-9410-4dbe-ba3c-5afb6858c543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:51.405339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b502f41d-9410-4dbe-ba3c-5afb6858c543 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:51.405339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b502f41d-9410-4dbe-ba3c-5afb6858c543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:51.412624 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2335/4698] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:51.417199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d0e32-f91b-44b4-8739-b9557abf2feb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:51.430257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d0e32-f91b-44b4-8739-b9557abf2feb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:51.438116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82370e12-a821-4e95-a005-9edd60da85cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:51.438749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82370e12-a821-4e95-a005-9edd60da85cb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:51.439027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82370e12-a821-4e95-a005-9edd60da85cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:51.439500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82370e12-a821-4e95-a005-9edd60da85cb admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:51.440529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82370e12-a821-4e95-a005-9edd60da85cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:51.440644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82370e12-a821-4e95-a005-9edd60da85cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:51.443875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82370e12-a821-4e95-a005-9edd60da85cb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:51.465565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502f41d-9410-4dbe-ba3c-5afb6858c543 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:51.470319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b502f41d-9410-4dbe-ba3c-5afb6858c543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:51.480562 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2364/4699] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:51.486049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82370e12-a821-4e95-a005-9edd60da85cb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:51.490785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502f41d-9410-4dbe-ba3c-5afb6858c543 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:51.492950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:51.501250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82370e12-a821-4e95-a005-9edd60da85cb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:51.501987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:51.504546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:51.505090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:51.505710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] PATH_INFO: `/v3/users/42499cd684ac4cf1b77bbc44e7937eb3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:51.509133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42499cd684ac4cf1b77bbc44e7937eb3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:51.510903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441ffd7-70c7-456f-960b-4f944d14378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:51.511609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441ffd7-70c7-456f-960b-4f944d14378e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:51.512146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441ffd7-70c7-456f-960b-4f944d14378e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:51.512949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441ffd7-70c7-456f-960b-4f944d14378e admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:51.514035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441ffd7-70c7-456f-960b-4f944d14378e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:51.514473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:51.516544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441ffd7-70c7-456f-960b-4f944d14378e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:51.569940 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2365/4700] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:51 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:51.574165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441ffd7-70c7-456f-960b-4f944d14378e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:51.592601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:51.601691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42499cd684ac4cf1b77bbc44e7937eb3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:51.612773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42499cd684ac4cf1b77bbc44e7937eb3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:51.647509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42499cd684ac4cf1b77bbc44e7937eb3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:51.668024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42499cd684ac4cf1b77bbc44e7937eb3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:51.668771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85b229d6-7484-4a76-9a60-5b4c34a0a049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42499cd684ac4cf1b77bbc44e7937eb3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:51.680287 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2336/4701] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:51 2022] DELETE /identity/v3/users/42499cd684ac4cf1b77bbc44e7937eb3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:51.680491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441ffd7-70c7-456f-960b-4f944d14378e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:51.682682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f5f1d-4982-4d79-828a-825d382eaea3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:51.683060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f5f1d-4982-4d79-828a-825d382eaea3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:51.683764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f5f1d-4982-4d79-828a-825d382eaea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:51.720517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fa8f5f1d-4982-4d79-828a-825d382eaea3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:51.723787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa8f5f1d-4982-4d79-828a-825d382eaea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:51.801720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d44c8-7cc2-4989-946e-bc51cb4095f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:51.802486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d44c8-7cc2-4989-946e-bc51cb4095f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:51.802773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d44c8-7cc2-4989-946e-bc51cb4095f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:51.803045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d44c8-7cc2-4989-946e-bc51cb4095f7 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:51.803864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d44c8-7cc2-4989-946e-bc51cb4095f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:51.806193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d44c8-7cc2-4989-946e-bc51cb4095f7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:51.858802 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2366/4702] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:51 2022] POST /identity/v3/users => generated 284 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:51.860120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d44c8-7cc2-4989-946e-bc51cb4095f7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:51.863228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-570d44c8-7cc2-4989-946e-bc51cb4095f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:51.873890 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2337/4703] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:51.885703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f5f1d-4982-4d79-828a-825d382eaea3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:51.887323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa8f5f1d-4982-4d79-828a-825d382eaea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:51.912732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d44c8-7cc2-4989-946e-bc51cb4095f7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:51.933925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e659c8-33be-404c-b352-f5e0310d6fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:51.934942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e659c8-33be-404c-b352-f5e0310d6fca admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:51.935357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e659c8-33be-404c-b352-f5e0310d6fca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:51.935798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e659c8-33be-404c-b352-f5e0310d6fca admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:51.936657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e659c8-33be-404c-b352-f5e0310d6fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:51.939194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e659c8-33be-404c-b352-f5e0310d6fca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:51.953779 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2367/4704] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:51.958335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e659c8-33be-404c-b352-f5e0310d6fca admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:51.973342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e659c8-33be-404c-b352-f5e0310d6fca admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:51.980331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4a3ab-ba33-41a9-824e-22470c880d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:51.981184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4a3ab-ba33-41a9-824e-22470c880d1b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:51.981660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4a3ab-ba33-41a9-824e-22470c880d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:51.982199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4a3ab-ba33-41a9-824e-22470c880d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:51.983262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4a3ab-ba33-41a9-824e-22470c880d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:51.987379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4a3ab-ba33-41a9-824e-22470c880d1b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:52.017072 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2368/4705] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:52.024323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4a3ab-ba33-41a9-824e-22470c880d1b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:52.053261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4a3ab-ba33-41a9-824e-22470c880d1b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:52.084804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1473247-ba70-4494-be96-34e9713d4374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:52.086865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1473247-ba70-4494-be96-34e9713d4374 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:52.087352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1473247-ba70-4494-be96-34e9713d4374 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:52.088876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1473247-ba70-4494-be96-34e9713d4374 admin admin] PATH_INFO: `/v3/projects/0010fa073e44461a919b682d04f077da/users/54ab885de090462ebc55dde75589ba63/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:52.123173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f5f1d-4982-4d79-828a-825d382eaea3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:52.130149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb88172-01cd-4800-b073-12a512d2cbd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:52.131156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb88172-01cd-4800-b073-12a512d2cbd9 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:52.131706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb88172-01cd-4800-b073-12a512d2cbd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:52.132079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb88172-01cd-4800-b073-12a512d2cbd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:52.133530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1473247-ba70-4494-be96-34e9713d4374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0010fa073e44461a919b682d04f077da, user_id=54ab885de090462ebc55dde75589ba63, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:52.138056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1473247-ba70-4494-be96-34e9713d4374 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:52.147189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb88172-01cd-4800-b073-12a512d2cbd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:52.148898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb88172-01cd-4800-b073-12a512d2cbd9 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:52.175126 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2369/4706] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:52 2022] PUT /identity/v3/projects/0010fa073e44461a919b682d04f077da/users/54ab885de090462ebc55dde75589ba63/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:52.186376 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2338/4707] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:52.187160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88ab85-1310-485a-a9c9-f57c73274774 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:52.187688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88ab85-1310-485a-a9c9-f57c73274774 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:52.188167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88ab85-1310-485a-a9c9-f57c73274774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:52.246873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cf88ab85-1310-485a-a9c9-f57c73274774 None None] MFA Rules not processed for user `54ab885de090462ebc55dde75589ba63`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:52.248638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cf88ab85-1310-485a-a9c9-f57c73274774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:52.283486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:52.438659 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2370/4708] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:52.443652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb88172-01cd-4800-b073-12a512d2cbd9 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:52.460355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb88172-01cd-4800-b073-12a512d2cbd9 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:52.608071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea0939-4039-4e58-8229-12832c6fa924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:52.609503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea0939-4039-4e58-8229-12832c6fa924 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:52.610053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea0939-4039-4e58-8229-12832c6fa924 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:52.610506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea0939-4039-4e58-8229-12832c6fa924 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:52.611748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea0939-4039-4e58-8229-12832c6fa924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:52.614476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea0939-4039-4e58-8229-12832c6fa924 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:52.623906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:52.667348 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2339/4709] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:52 2022] POST /identity/v3/projects => generated 367 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:52.672150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88ab85-1310-485a-a9c9-f57c73274774 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:52.684229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88ab85-1310-485a-a9c9-f57c73274774 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:52.764363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd11da-65a6-4ad8-82ef-fcf300c541a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:52.765515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd11da-65a6-4ad8-82ef-fcf300c541a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:52.765972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd11da-65a6-4ad8-82ef-fcf300c541a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:52.766314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd11da-65a6-4ad8-82ef-fcf300c541a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:52.767051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd11da-65a6-4ad8-82ef-fcf300c541a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:52.769559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd11da-65a6-4ad8-82ef-fcf300c541a0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:52.821586 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2371/4710] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:52 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:52.826771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea0939-4039-4e58-8229-12832c6fa924 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:52.840137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd11da-65a6-4ad8-82ef-fcf300c541a0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:52.841196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea0939-4039-4e58-8229-12832c6fa924 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:52.843808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5ccd11da-65a6-4ad8-82ef-fcf300c541a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:52.861393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1983e4d-4605-4ff2-9913-40399fddfcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:52.862259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1983e4d-4605-4ff2-9913-40399fddfcdb admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:52.862694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1983e4d-4605-4ff2-9913-40399fddfcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:52.863570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1983e4d-4605-4ff2-9913-40399fddfcdb admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:52.864853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1983e4d-4605-4ff2-9913-40399fddfcdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:52.866649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1983e4d-4605-4ff2-9913-40399fddfcdb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:52.886402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd11da-65a6-4ad8-82ef-fcf300c541a0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:52.894636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:52.895976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:52.896432 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2340/4711] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:52.898010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:52.898667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] PATH_INFO: `/v3/projects/549f9ced9a7741e2a5888093586fe8c6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:52.901475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1983e4d-4605-4ff2-9913-40399fddfcdb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:52.904815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=549f9ced9a7741e2a5888093586fe8c6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:52.909889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:52.931037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1983e4d-4605-4ff2-9913-40399fddfcdb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:52.936062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3eba8e-8ffe-4359-9b82-7b3095896855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:52.936716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3eba8e-8ffe-4359-9b82-7b3095896855 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:52.936987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3eba8e-8ffe-4359-9b82-7b3095896855 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:52.937301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3eba8e-8ffe-4359-9b82-7b3095896855 admin admin] PATH_INFO: `/v3/projects/7826315f559b47bf9cd280a40a0f576d/users/af30b5fefb4e458ea4fbd7dfad725bbe/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:52.955797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3eba8e-8ffe-4359-9b82-7b3095896855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7826315f559b47bf9cd280a40a0f576d, user_id=af30b5fefb4e458ea4fbd7dfad725bbe, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:52.958497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3eba8e-8ffe-4359-9b82-7b3095896855 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:52.987611 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2341/4712] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:52 2022] PUT /identity/v3/projects/7826315f559b47bf9cd280a40a0f576d/users/af30b5fefb4e458ea4fbd7dfad725bbe/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:52.991570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3eba8e-8ffe-4359-9b82-7b3095896855 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:53.055563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '549f9ced9a7741e2a5888093586fe8c6', 'request_id': 'req-191dc0cd-bf80-4b59-95c8-589175e6bb25'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:53.071304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '549f9ced9a7741e2a5888093586fe8c6', 'request_id': 'req-191dc0cd-bf80-4b59-95c8-589175e6bb25'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:53.084627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '549f9ced9a7741e2a5888093586fe8c6', 'request_id': 'req-191dc0cd-bf80-4b59-95c8-589175e6bb25'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:53.098947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3eba8e-8ffe-4359-9b82-7b3095896855 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:53.122189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] The token cache is being invalidate because project 549f9ced9a7741e2a5888093586fe8c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:53.122765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:53.123594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-191dc0cd-bf80-4b59-95c8-589175e6bb25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:53.125993 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2372/4713] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:52 2022] DELETE /identity/v3/projects/549f9ced9a7741e2a5888093586fe8c6 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:53.140592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da35435-6e69-4723-99bf-6f92ce34f8da None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.142127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da35435-6e69-4723-99bf-6f92ce34f8da None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.142477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da35435-6e69-4723-99bf-6f92ce34f8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.179160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6da35435-6e69-4723-99bf-6f92ce34f8da None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:53.180718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6da35435-6e69-4723-99bf-6f92ce34f8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:53.193014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca218a44-16c3-41e5-8df7-005eb35b7237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:53.193979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca218a44-16c3-41e5-8df7-005eb35b7237 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.194352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca218a44-16c3-41e5-8df7-005eb35b7237 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.194705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca218a44-16c3-41e5-8df7-005eb35b7237 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.195424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca218a44-16c3-41e5-8df7-005eb35b7237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:53.197303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca218a44-16c3-41e5-8df7-005eb35b7237 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:53.208206 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2373/4714] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:53.215020 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2342/4715] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:53.216035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7db39-de1a-4973-9b6f-3d40d6308b24 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.216388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7db39-de1a-4973-9b6f-3d40d6308b24 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.216730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7db39-de1a-4973-9b6f-3d40d6308b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.220967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca218a44-16c3-41e5-8df7-005eb35b7237 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:53.223390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ca218a44-16c3-41e5-8df7-005eb35b7237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:53.243772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca218a44-16c3-41e5-8df7-005eb35b7237 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:53.250161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a4a7db39-de1a-4973-9b6f-3d40d6308b24 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:53.251600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a4a7db39-de1a-4973-9b6f-3d40d6308b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:53.253684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dc21b-0532-49b1-a8fb-6bc1fb72dc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:53.254549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dc21b-0532-49b1-a8fb-6bc1fb72dc25 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.255063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dc21b-0532-49b1-a8fb-6bc1fb72dc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.255694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dc21b-0532-49b1-a8fb-6bc1fb72dc25 admin admin] PATH_INFO: `/v3/projects/7826315f559b47bf9cd280a40a0f576d/users/af30b5fefb4e458ea4fbd7dfad725bbe/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.259071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dc21b-0532-49b1-a8fb-6bc1fb72dc25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7826315f559b47bf9cd280a40a0f576d, user_id=af30b5fefb4e458ea4fbd7dfad725bbe, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:53.265494 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2374/4716] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:53.265494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7db39-de1a-4973-9b6f-3d40d6308b24 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:53.266420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dc21b-0532-49b1-a8fb-6bc1fb72dc25 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:53.267513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a4a7db39-de1a-4973-9b6f-3d40d6308b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:53.289518 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2343/4717] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:53 2022] PUT /identity/v3/projects/7826315f559b47bf9cd280a40a0f576d/users/af30b5fefb4e458ea4fbd7dfad725bbe/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:09:53.290970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dc21b-0532-49b1-a8fb-6bc1fb72dc25 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:53.298649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7db39-de1a-4973-9b6f-3d40d6308b24 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:53.328168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:53.386037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:53.392848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:53.394515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.395052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.395545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] PATH_INFO: `/v3/users/461a5c1e334a47eb9d70908227cca0b2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.397351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] RBAC: Authorizing `identity:delete_user(user_id=461a5c1e334a47eb9d70908227cca0b2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:53.399532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:53.409336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dc21b-0532-49b1-a8fb-6bc1fb72dc25 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:53.415185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f1dde-c43e-4742-abe7-989fc6f87b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:53.417013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f1dde-c43e-4742-abe7-989fc6f87b23 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.417323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f1dde-c43e-4742-abe7-989fc6f87b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.417591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f1dde-c43e-4742-abe7-989fc6f87b23 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.422007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f1dde-c43e-4742-abe7-989fc6f87b23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:53.422419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f1dde-c43e-4742-abe7-989fc6f87b23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:53.424277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f1dde-c43e-4742-abe7-989fc6f87b23 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:53.442042 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2344/4718] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:53.445053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd2c96-c687-462d-94df-a202e400ba08 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.445300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd2c96-c687-462d-94df-a202e400ba08 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.445617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd2c96-c687-462d-94df-a202e400ba08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.469571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461a5c1e334a47eb9d70908227cca0b2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:53.484131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d3cd2c96-c687-462d-94df-a202e400ba08 None None] MFA Rules not processed for user `af30b5fefb4e458ea4fbd7dfad725bbe`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:53.484800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461a5c1e334a47eb9d70908227cca0b2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:53.485373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d3cd2c96-c687-462d-94df-a202e400ba08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:53.497637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461a5c1e334a47eb9d70908227cca0b2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:53.508535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461a5c1e334a47eb9d70908227cca0b2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:53.509321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461a5c1e334a47eb9d70908227cca0b2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:53.521978 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2375/4719] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:53 2022] DELETE /identity/v3/users/461a5c1e334a47eb9d70908227cca0b2 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:53.522652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:53.524651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:53.592964 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2345/4720] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:53.594702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd2c96-c687-462d-94df-a202e400ba08 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:53.611980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd2c96-c687-462d-94df-a202e400ba08 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:53.619682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001c53b-aad3-4b02-b0a0-b6b12af8bedf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:53.625790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0b0cd-3b53-41f4-b6e7-50dce385606f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:53.627025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0b0cd-3b53-41f4-b6e7-50dce385606f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.627450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0b0cd-3b53-41f4-b6e7-50dce385606f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.628119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0b0cd-3b53-41f4-b6e7-50dce385606f admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.642597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0b0cd-3b53-41f4-b6e7-50dce385606f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:53.646949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0b0cd-3b53-41f4-b6e7-50dce385606f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:53.696012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0c76e-525e-49cb-80df-22862f6c4893 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:53.696626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0c76e-525e-49cb-80df-22862f6c4893 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.696900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0c76e-525e-49cb-80df-22862f6c4893 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.698086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0c76e-525e-49cb-80df-22862f6c4893 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.698332 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2376/4721] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:53 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:53.699066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0c76e-525e-49cb-80df-22862f6c4893 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:53.700925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0c76e-525e-49cb-80df-22862f6c4893 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:53.706228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0b0cd-3b53-41f4-b6e7-50dce385606f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:53.755450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:53.757737 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2346/4722] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:53 2022] POST /identity/v3/projects => generated 360 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:53.761987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0c76e-525e-49cb-80df-22862f6c4893 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:53.841404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0b0cd-3b53-41f4-b6e7-50dce385606f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:53.873619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:53.895710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0c76e-525e-49cb-80df-22862f6c4893 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:53.910451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc048fd-3dba-4bc4-9d94-9578a0df0df9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:53.911181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc048fd-3dba-4bc4-9d94-9578a0df0df9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.911656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc048fd-3dba-4bc4-9d94-9578a0df0df9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.912008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc048fd-3dba-4bc4-9d94-9578a0df0df9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.912729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc048fd-3dba-4bc4-9d94-9578a0df0df9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:53.915065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc048fd-3dba-4bc4-9d94-9578a0df0df9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:53.941146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a456456-b76b-43df-88cd-14aba0622a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:53.941846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a456456-b76b-43df-88cd-14aba0622a98 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:53.942327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a456456-b76b-43df-88cd-14aba0622a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:53.942617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a456456-b76b-43df-88cd-14aba0622a98 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:53.944269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a456456-b76b-43df-88cd-14aba0622a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:53.948247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a456456-b76b-43df-88cd-14aba0622a98 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:53.949784 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2347/4723] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:53 2022] POST /identity/v3/users => generated 416 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:53.950970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc048fd-3dba-4bc4-9d94-9578a0df0df9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:53.953519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5dc048fd-3dba-4bc4-9d94-9578a0df0df9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:53.997781 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2377/4724] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:53 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:53.998640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a456456-b76b-43df-88cd-14aba0622a98 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:54.005214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4a456456-b76b-43df-88cd-14aba0622a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:54.054244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc048fd-3dba-4bc4-9d94-9578a0df0df9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:54.060271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985436e6-f3b3-48a5-a013-10670b29fa3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:54.062021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985436e6-f3b3-48a5-a013-10670b29fa3d service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.062543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985436e6-f3b3-48a5-a013-10670b29fa3d service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.063021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985436e6-f3b3-48a5-a013-10670b29fa3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.065906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-985436e6-f3b3-48a5-a013-10670b29fa3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:54.083855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985436e6-f3b3-48a5-a013-10670b29fa3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:54.085723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985436e6-f3b3-48a5-a013-10670b29fa3d service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:54.100496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a456456-b76b-43df-88cd-14aba0622a98 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:54.105812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e042f-34f1-458c-85ad-63908b67fb4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:54.106621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e042f-34f1-458c-85ad-63908b67fb4f service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.107074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e042f-34f1-458c-85ad-63908b67fb4f service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.107510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e042f-34f1-458c-85ad-63908b67fb4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.110479 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2348/4725] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:54.110800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985436e6-f3b3-48a5-a013-10670b29fa3d service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:54.111482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9a1e042f-34f1-458c-85ad-63908b67fb4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:54.136598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985436e6-f3b3-48a5-a013-10670b29fa3d service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:54.142746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384d9ae-bb32-4ea2-977c-287391e2fd7d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:54.143470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384d9ae-bb32-4ea2-977c-287391e2fd7d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.143961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384d9ae-bb32-4ea2-977c-287391e2fd7d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.144302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384d9ae-bb32-4ea2-977c-287391e2fd7d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/users/c34cc50697cc49389542e28f9a50d43b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.165597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384d9ae-bb32-4ea2-977c-287391e2fd7d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:get_user(user_id=c34cc50697cc49389542e28f9a50d43b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:54.170334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384d9ae-bb32-4ea2-977c-287391e2fd7d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:54.173010 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2349/4726] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:09:54 2022] GET /identity/v3/users/c34cc50697cc49389542e28f9a50d43b => generated 416 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:54.174556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384d9ae-bb32-4ea2-977c-287391e2fd7d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:54.178543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6384d9ae-bb32-4ea2-977c-287391e2fd7d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:54.189683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e042f-34f1-458c-85ad-63908b67fb4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:54.191162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e042f-34f1-458c-85ad-63908b67fb4f service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:54.202690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384d9ae-bb32-4ea2-977c-287391e2fd7d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:54.210455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:54.211491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.211831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.212169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] PATH_INFO: `/v3/projects/5572b130042c482384a1a06e6a91c784` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.214478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5572b130042c482384a1a06e6a91c784)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:54.217099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:54.217389 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2378/4727] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:54.218332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e042f-34f1-458c-85ad-63908b67fb4f service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:54.242065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e042f-34f1-458c-85ad-63908b67fb4f service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:54.248361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c790c1f-811c-4fac-9069-76ea4294db61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:54.249325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c790c1f-811c-4fac-9069-76ea4294db61 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.249938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c790c1f-811c-4fac-9069-76ea4294db61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.249938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c790c1f-811c-4fac-9069-76ea4294db61 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.250565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c790c1f-811c-4fac-9069-76ea4294db61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:54.252428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c790c1f-811c-4fac-9069-76ea4294db61 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:54.263428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:54.272771 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2379/4728] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:54.273925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c790c1f-811c-4fac-9069-76ea4294db61 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:54.298860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c790c1f-811c-4fac-9069-76ea4294db61 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:54.327574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5572b130042c482384a1a06e6a91c784', 'request_id': 'req-775a3d6e-0335-40dd-9299-34a526696703'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.341544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5572b130042c482384a1a06e6a91c784', 'request_id': 'req-775a3d6e-0335-40dd-9299-34a526696703'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.354600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5572b130042c482384a1a06e6a91c784', 'request_id': 'req-775a3d6e-0335-40dd-9299-34a526696703'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.376856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] The token cache is being invalidate because project 5572b130042c482384a1a06e6a91c784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:54.377240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.378117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.383682 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2350/4729] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:54 2022] DELETE /identity/v3/projects/5572b130042c482384a1a06e6a91c784 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:54.386139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:54.387291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:54.396976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:54.398127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.398568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.399120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/users/c34cc50697cc49389542e28f9a50d43b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.400799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_user(user_id=c34cc50697cc49389542e28f9a50d43b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:54.403157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:54.461555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:54.495290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a3d6e-0335-40dd-9299-34a526696703 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:54.500925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34cc50697cc49389542e28f9a50d43b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.503552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bb280-abbe-4af4-9cbe-d1f7ac9da90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:54.504173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bb280-abbe-4af4-9cbe-d1f7ac9da90f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.504453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bb280-abbe-4af4-9cbe-d1f7ac9da90f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.504723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bb280-abbe-4af4-9cbe-d1f7ac9da90f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.505935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bb280-abbe-4af4-9cbe-d1f7ac9da90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:54.507807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bb280-abbe-4af4-9cbe-d1f7ac9da90f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:54.520800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34cc50697cc49389542e28f9a50d43b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.522323 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2351/4730] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:54.524273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e134c-310d-4630-a398-0445f8cbc868 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.524273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e134c-310d-4630-a398-0445f8cbc868 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.524554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e134c-310d-4630-a398-0445f8cbc868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.533769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34cc50697cc49389542e28f9a50d43b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.544405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34cc50697cc49389542e28f9a50d43b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.545228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34cc50697cc49389542e28f9a50d43b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.552383 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2380/4731] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:09:54 2022] DELETE /identity/v3/users/c34cc50697cc49389542e28f9a50d43b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:54.553456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:54.557955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:54.563221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7d0e134c-310d-4630-a398-0445f8cbc868 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:54.565395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7d0e134c-310d-4630-a398-0445f8cbc868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:54.655976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196ea6e-288d-4be4-952a-5a6017708faf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:54.667413 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2352/4732] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:54.669318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2a6cc-dcd1-41d2-8c9d-29287b598fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:54.670009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2a6cc-dcd1-41d2-8c9d-29287b598fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.670288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2a6cc-dcd1-41d2-8c9d-29287b598fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.670734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2a6cc-dcd1-41d2-8c9d-29287b598fd3 admin admin] PATH_INFO: `/v3/projects/f66301fa61f649fc8016a81d6700dfa7/users/a83cacaed7aa4d87a5fef3a7cee1a77b/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.671251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e134c-310d-4630-a398-0445f8cbc868 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:54.674255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7d0e134c-310d-4630-a398-0445f8cbc868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:54.695991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2a6cc-dcd1-41d2-8c9d-29287b598fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66301fa61f649fc8016a81d6700dfa7, user_id=a83cacaed7aa4d87a5fef3a7cee1a77b, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:54.699102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2a6cc-dcd1-41d2-8c9d-29287b598fd3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:54.733091 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2381/4733] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:54 2022] PUT /identity/v3/projects/f66301fa61f649fc8016a81d6700dfa7/users/a83cacaed7aa4d87a5fef3a7cee1a77b/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:54.733091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368891cf-716f-4073-a911-b86a8eb0bb73 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.733091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368891cf-716f-4073-a911-b86a8eb0bb73 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.733091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368891cf-716f-4073-a911-b86a8eb0bb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.738967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:54.760636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-368891cf-716f-4073-a911-b86a8eb0bb73 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:54.761910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-368891cf-716f-4073-a911-b86a8eb0bb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:54.772921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e134c-310d-4630-a398-0445f8cbc868 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:54.844109 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2382/4734] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:54.846143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bbdd0-3efa-4c5c-95c0-06c70871beea None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.847100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bbdd0-3efa-4c5c-95c0-06c70871beea None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.847683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bbdd0-3efa-4c5c-95c0-06c70871beea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.883307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:54.884134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:54.884660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:54.885452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/09f468afd1bc41988b5a16fc7f4a8d05` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:54.886630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=09f468afd1bc41988b5a16fc7f4a8d05)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:54.888890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:54.889928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4b8bbdd0-3efa-4c5c-95c0-06c70871beea None None] MFA Rules not processed for user `a83cacaed7aa4d87a5fef3a7cee1a77b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:54.891448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4b8bbdd0-3efa-4c5c-95c0-06c70871beea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:54.933790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:54.970344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f468afd1bc41988b5a16fc7f4a8d05', 'request_id': 'req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.981410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f468afd1bc41988b5a16fc7f4a8d05', 'request_id': 'req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:54.988850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f468afd1bc41988b5a16fc7f4a8d05', 'request_id': 'req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:55.002834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project 09f468afd1bc41988b5a16fc7f4a8d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:55.003196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:55.004125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:55.005688 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2353/4735] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:54 2022] DELETE /identity/v3/projects/09f468afd1bc41988b5a16fc7f4a8d05 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:55.011749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.013175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:55.036534 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2383/4736] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:55.039152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8bbdd0-3efa-4c5c-95c0-06c70871beea None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.044459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4b8bbdd0-3efa-4c5c-95c0-06c70871beea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:55.069067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8bbdd0-3efa-4c5c-95c0-06c70871beea None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:55.124146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb00cc-12d1-4a8c-b4fd-6217f5c7e96a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:55.137507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaca3f-87d8-4f9e-ac45-2f5a834d92fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:55.138116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaca3f-87d8-4f9e-ac45-2f5a834d92fe admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:55.138383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaca3f-87d8-4f9e-ac45-2f5a834d92fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:55.138644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaca3f-87d8-4f9e-ac45-2f5a834d92fe admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:55.139283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaca3f-87d8-4f9e-ac45-2f5a834d92fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:55.139578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaca3f-87d8-4f9e-ac45-2f5a834d92fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:55.141200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaca3f-87d8-4f9e-ac45-2f5a834d92fe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:55.164358 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2354/4737] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:55.165239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaca3f-87d8-4f9e-ac45-2f5a834d92fe admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.169909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-55aaca3f-87d8-4f9e-ac45-2f5a834d92fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:55.170866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d29510-276a-4dbd-af47-74863e28c8d5 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:55.171476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d29510-276a-4dbd-af47-74863e28c8d5 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:55.171755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d29510-276a-4dbd-af47-74863e28c8d5 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:55.172022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d29510-276a-4dbd-af47-74863e28c8d5 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:55.172659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d29510-276a-4dbd-af47-74863e28c8d5 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:55.174403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d29510-276a-4dbd-af47-74863e28c8d5 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:55.196069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaca3f-87d8-4f9e-ac45-2f5a834d92fe admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:55.202405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b76a2-b20b-46cf-90ca-bc6e47ce2758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:55.203298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b76a2-b20b-46cf-90ca-bc6e47ce2758 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:55.203667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b76a2-b20b-46cf-90ca-bc6e47ce2758 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:55.204083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b76a2-b20b-46cf-90ca-bc6e47ce2758 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:55.204922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b76a2-b20b-46cf-90ca-bc6e47ce2758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:55.207057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b76a2-b20b-46cf-90ca-bc6e47ce2758 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:55.209507 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2384/4738] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:55 2022] POST /identity/v3/projects => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:55.210436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d29510-276a-4dbd-af47-74863e28c8d5 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.256762 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2355/4739] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:55 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:55.257432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b76a2-b20b-46cf-90ca-bc6e47ce2758 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.264964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:55.315704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d29510-276a-4dbd-af47-74863e28c8d5 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:55.339616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b76a2-b20b-46cf-90ca-bc6e47ce2758 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:55.346569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf6c91-f16c-4d9b-9bb7-1e1760e11deb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:55.347533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf6c91-f16c-4d9b-9bb7-1e1760e11deb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:55.347909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf6c91-f16c-4d9b-9bb7-1e1760e11deb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:55.348277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf6c91-f16c-4d9b-9bb7-1e1760e11deb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:55.349058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf6c91-f16c-4d9b-9bb7-1e1760e11deb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Adding query filter params (is_domain=True) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:09:55.349462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf6c91-f16c-4d9b-9bb7-1e1760e11deb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:55.351426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf6c91-f16c-4d9b-9bb7-1e1760e11deb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:55.372491 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2356/4740] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Oct 15 21:09:55 2022] GET /identity/v3/projects?is_domain=True => generated 1018 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:55.373471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf6c91-f16c-4d9b-9bb7-1e1760e11deb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.403061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385600f-cc5f-4668-b378-4ae19cd47081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:55.403786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385600f-cc5f-4668-b378-4ae19cd47081 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:55.404256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385600f-cc5f-4668-b378-4ae19cd47081 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:55.404596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385600f-cc5f-4668-b378-4ae19cd47081 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:55.405314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385600f-cc5f-4668-b378-4ae19cd47081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:55.408152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385600f-cc5f-4668-b378-4ae19cd47081 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:55.415195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:55.448442 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2385/4741] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:55 2022] POST /identity/v3/projects => generated 369 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:55.449587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385600f-cc5f-4668-b378-4ae19cd47081 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.488190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf6c91-f16c-4d9b-9bb7-1e1760e11deb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:55.546129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385600f-cc5f-4668-b378-4ae19cd47081 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:55.553182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889d06f-79c6-4772-ac75-b337bba06c3b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:55.553814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889d06f-79c6-4772-ac75-b337bba06c3b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:55.554042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889d06f-79c6-4772-ac75-b337bba06c3b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:55.554566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889d06f-79c6-4772-ac75-b337bba06c3b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:55.555333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889d06f-79c6-4772-ac75-b337bba06c3b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:55.584150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889d06f-79c6-4772-ac75-b337bba06c3b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:55.617600 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2386/4742] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 21:09:55 2022] GET /identity/v3/domains => generated 827 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:55.622209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889d06f-79c6-4772-ac75-b337bba06c3b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.648503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42537a9a-c641-4b3f-840a-9d54d1c4f33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:55.649152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42537a9a-c641-4b3f-840a-9d54d1c4f33b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:55.649428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42537a9a-c641-4b3f-840a-9d54d1c4f33b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:55.649694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42537a9a-c641-4b3f-840a-9d54d1c4f33b admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:55.651100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42537a9a-c641-4b3f-840a-9d54d1c4f33b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:55.653866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42537a9a-c641-4b3f-840a-9d54d1c4f33b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:55.694092 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2357/4743] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:55 2022] POST /identity/v3/users => generated 298 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:55.695640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42537a9a-c641-4b3f-840a-9d54d1c4f33b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.712905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42537a9a-c641-4b3f-840a-9d54d1c4f33b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:55.720304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:55.720304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:55.720634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:55.720985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/a0a0713a477046829a6d6b4c77d0a1f8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:55.722436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:update_project(project_id=a0a0713a477046829a6d6b4c77d0a1f8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:55.723435 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:09:55.723435 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:09:55.723435 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:09:55.725491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:55.743167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:55.744567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:55.770322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:55.797289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:55.798711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:55.799713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889d06f-79c6-4772-ac75-b337bba06c3b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:55.803099 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2358/4744] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Oct 15 21:09:55 2022] PATCH /identity/v3/projects/a0a0713a477046829a6d6b4c77d0a1f8 => generated 360 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:55.804274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.805563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:55.912342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca601e7b-616d-4ab9-a4d1-30f8cb1d37bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:55.912957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca601e7b-616d-4ab9-a4d1-30f8cb1d37bb admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:55.913272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca601e7b-616d-4ab9-a4d1-30f8cb1d37bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:55.913539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca601e7b-616d-4ab9-a4d1-30f8cb1d37bb admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:55.914228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca601e7b-616d-4ab9-a4d1-30f8cb1d37bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:55.916723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca601e7b-616d-4ab9-a4d1-30f8cb1d37bb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:55.958806 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2387/4745] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:09:55 2022] POST /identity/v3/users => generated 285 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:55.959573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca601e7b-616d-4ab9-a4d1-30f8cb1d37bb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:55.963556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ca601e7b-616d-4ab9-a4d1-30f8cb1d37bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:55.975512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857521d8-2711-4905-a885-5ec3504e05e1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:55.983753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff2c08-e023-4808-9226-2f16fbc62044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:55.985943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff2c08-e023-4808-9226-2f16fbc62044 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:55.986438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff2c08-e023-4808-9226-2f16fbc62044 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:55.986880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff2c08-e023-4808-9226-2f16fbc62044 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:55.987705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff2c08-e023-4808-9226-2f16fbc62044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:55.989853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff2c08-e023-4808-9226-2f16fbc62044 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:56.007389 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2359/4746] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:56.008539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff2c08-e023-4808-9226-2f16fbc62044 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:56.012195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c8ff2c08-e023-4808-9226-2f16fbc62044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:56.036529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff2c08-e023-4808-9226-2f16fbc62044 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:56.043260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a470b-a016-4a79-a790-54d9bf5dd158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:56.044470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a470b-a016-4a79-a790-54d9bf5dd158 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:56.044897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a470b-a016-4a79-a790-54d9bf5dd158 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:56.045349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a470b-a016-4a79-a790-54d9bf5dd158 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:56.046147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a470b-a016-4a79-a790-54d9bf5dd158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:56.048033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a470b-a016-4a79-a790-54d9bf5dd158 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:56.062348 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2360/4747] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:56.063258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a470b-a016-4a79-a790-54d9bf5dd158 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:56.080123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a470b-a016-4a79-a790-54d9bf5dd158 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:56.085468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436c7b3-7d53-43e7-87b7-6abea598aaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:56.086128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436c7b3-7d53-43e7-87b7-6abea598aaad admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:56.086401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436c7b3-7d53-43e7-87b7-6abea598aaad admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:56.086668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436c7b3-7d53-43e7-87b7-6abea598aaad admin admin] PATH_INFO: `/v3/projects/1beeea097efa4380b96e6d8328f994e9/users/126881bf487d4e0783a5bdd86cbad261/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:56.101147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:56.106249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436c7b3-7d53-43e7-87b7-6abea598aaad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1beeea097efa4380b96e6d8328f994e9, user_id=126881bf487d4e0783a5bdd86cbad261, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:56.110868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436c7b3-7d53-43e7-87b7-6abea598aaad admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:56.121295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca601e7b-616d-4ab9-a4d1-30f8cb1d37bb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:56.130032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:56.130883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:56.131489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:56.131876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/a0a0713a477046829a6d6b4c77d0a1f8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:56.138233 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2361/4748] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:56 2022] PUT /identity/v3/projects/1beeea097efa4380b96e6d8328f994e9/users/126881bf487d4e0783a5bdd86cbad261/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:56.138803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436c7b3-7d53-43e7-87b7-6abea598aaad admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:56.149713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a0a0713a477046829a6d6b4c77d0a1f8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:56.152579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:56.236806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:56.244658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:56.261389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436c7b3-7d53-43e7-87b7-6abea598aaad admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:56.268164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d62e54-501d-4c24-8f4d-efb771689899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:56.269656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d62e54-501d-4c24-8f4d-efb771689899 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:56.270092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d62e54-501d-4c24-8f4d-efb771689899 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:56.270516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d62e54-501d-4c24-8f4d-efb771689899 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:56.271367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d62e54-501d-4c24-8f4d-efb771689899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:56.273399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d62e54-501d-4c24-8f4d-efb771689899 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:56.288615 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2362/4749] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:56.290259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d62e54-501d-4c24-8f4d-efb771689899 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:56.304501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d62e54-501d-4c24-8f4d-efb771689899 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:56.312633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bf6e5-b845-489c-96e1-5904cae3d72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:56.317290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bf6e5-b845-489c-96e1-5904cae3d72a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:56.317290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bf6e5-b845-489c-96e1-5904cae3d72a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:56.317290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bf6e5-b845-489c-96e1-5904cae3d72a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:56.317290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bf6e5-b845-489c-96e1-5904cae3d72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:56.322610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bf6e5-b845-489c-96e1-5904cae3d72a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:56.350928 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2363/4750] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:09:56.350928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bf6e5-b845-489c-96e1-5904cae3d72a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:56.359717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8', 'request_id': 'req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:56.364975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bf6e5-b845-489c-96e1-5904cae3d72a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:56.376571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8', 'request_id': 'req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:56.380215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:56.386369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8', 'request_id': 'req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:56.399358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project a0a0713a477046829a6d6b4c77d0a1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:56.400462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:56.401583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:56.411244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8', 'request_id': 'req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:56.412138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8', 'request_id': 'req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:56.413572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a0a0713a477046829a6d6b4c77d0a1f8', 'request_id': 'req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:56.423170 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2388/4751] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:55 2022] DELETE /identity/v3/projects/a0a0713a477046829a6d6b4c77d0a1f8 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:56.423940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:56.426985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:56.506678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432741a5-f8c4-4be4-8cb9-a63b552ad302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:56.507513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432741a5-f8c4-4be4-8cb9-a63b552ad302 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:56.507818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432741a5-f8c4-4be4-8cb9-a63b552ad302 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:56.508094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432741a5-f8c4-4be4-8cb9-a63b552ad302 admin admin] PATH_INFO: `/v3/projects/4e6b708b260348cf904e392e1329e1d2/users/44af21476ed8446d98640881d322eea3/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:56.533666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664a6ca-bb0c-40b1-ba23-f2b3792d3083 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:56.542066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d329c1-5f36-4c9a-b91d-7d7cee0b5128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:56.542928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d329c1-5f36-4c9a-b91d-7d7cee0b5128 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:56.543210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d329c1-5f36-4c9a-b91d-7d7cee0b5128 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:56.543567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d329c1-5f36-4c9a-b91d-7d7cee0b5128 admin admin] PATH_INFO: `/v3/projects/1beeea097efa4380b96e6d8328f994e9/users/126881bf487d4e0783a5bdd86cbad261/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:56.546290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d329c1-5f36-4c9a-b91d-7d7cee0b5128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1beeea097efa4380b96e6d8328f994e9, user_id=126881bf487d4e0783a5bdd86cbad261, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:56.549470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d329c1-5f36-4c9a-b91d-7d7cee0b5128 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:56.560371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432741a5-f8c4-4be4-8cb9-a63b552ad302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6b708b260348cf904e392e1329e1d2, user_id=44af21476ed8446d98640881d322eea3, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:56.562478 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2389/4752] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:56 2022] PUT /identity/v3/projects/1beeea097efa4380b96e6d8328f994e9/users/126881bf487d4e0783a5bdd86cbad261/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:56.573354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432741a5-f8c4-4be4-8cb9-a63b552ad302 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:56.573494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d329c1-5f36-4c9a-b91d-7d7cee0b5128 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:56.573494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-45d329c1-5f36-4c9a-b91d-7d7cee0b5128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:56.600935 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2364/4753] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:09:56 2022] PUT /identity/v3/projects/4e6b708b260348cf904e392e1329e1d2/users/44af21476ed8446d98640881d322eea3/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:56.602046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a173e-4dbd-423b-aef1-f008cc462ec5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:56.602329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a173e-4dbd-423b-aef1-f008cc462ec5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:56.619432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a173e-4dbd-423b-aef1-f008cc462ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:56.705937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d329c1-5f36-4c9a-b91d-7d7cee0b5128 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:56.708254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e68a173e-4dbd-423b-aef1-f008cc462ec5 None None] MFA Rules not processed for user `126881bf487d4e0783a5bdd86cbad261`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:56.709513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e68a173e-4dbd-423b-aef1-f008cc462ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:56.939976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:56.948414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436e56-3691-448e-8dc2-a7fcef68f18c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:56.958447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436e56-3691-448e-8dc2-a7fcef68f18c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:56.958743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436e56-3691-448e-8dc2-a7fcef68f18c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:56.959136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436e56-3691-448e-8dc2-a7fcef68f18c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:56.959844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45436e56-3691-448e-8dc2-a7fcef68f18c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:56.965105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45436e56-3691-448e-8dc2-a7fcef68f18c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:57.030499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:57.036552 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2365/4754] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:57.043485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b39c4f-04fd-4729-9660-4a4c1f004799 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:57.044088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b39c4f-04fd-4729-9660-4a4c1f004799 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:57.045907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b39c4f-04fd-4729-9660-4a4c1f004799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:57.084009 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2390/4755] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:56 2022] POST /identity/v3/projects => generated 360 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:57.087046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436e56-3691-448e-8dc2-a7fcef68f18c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:57.110337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b9b39c4f-04fd-4729-9660-4a4c1f004799 None None] MFA Rules not processed for user `44af21476ed8446d98640881d322eea3`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:09:57.111971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b9b39c4f-04fd-4729-9660-4a4c1f004799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:57.228898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436e56-3691-448e-8dc2-a7fcef68f18c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:57.229713 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2366/4756] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:09:57.230959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b39c4f-04fd-4729-9660-4a4c1f004799 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:57.233339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b9b39c4f-04fd-4729-9660-4a4c1f004799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:57.245349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de66bce-4e2a-47ec-a1a5-766e41afbce9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:57.246454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de66bce-4e2a-47ec-a1a5-766e41afbce9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:57.247855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de66bce-4e2a-47ec-a1a5-766e41afbce9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:57.248297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de66bce-4e2a-47ec-a1a5-766e41afbce9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/cc36efe228aa448e89136d0ae3286226` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:57.251258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de66bce-4e2a-47ec-a1a5-766e41afbce9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:get_project(project_id=cc36efe228aa448e89136d0ae3286226)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:57.254641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de66bce-4e2a-47ec-a1a5-766e41afbce9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:57.261088 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2391/4757] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:57 2022] GET /identity/v3/projects/cc36efe228aa448e89136d0ae3286226 => generated 360 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:09:57.266350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de66bce-4e2a-47ec-a1a5-766e41afbce9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:57.292640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de66bce-4e2a-47ec-a1a5-766e41afbce9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:57.321178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:57.324278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:57.324957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:57.326114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/cc36efe228aa448e89136d0ae3286226` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:57.330496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cc36efe228aa448e89136d0ae3286226)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:57.331071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b39c4f-04fd-4729-9660-4a4c1f004799 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:57.334943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aa376-e010-434f-a70c-3924b91b3f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:57.335560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aa376-e010-434f-a70c-3924b91b3f41 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:57.335835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aa376-e010-434f-a70c-3924b91b3f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:57.336104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aa376-e010-434f-a70c-3924b91b3f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:57.339179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-770aa376-e010-434f-a70c-3924b91b3f41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:57.346256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:57.483568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc36efe228aa448e89136d0ae3286226', 'request_id': 'req-4efa68ae-a136-47eb-b5b1-05109b972928'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:57.495063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc36efe228aa448e89136d0ae3286226', 'request_id': 'req-4efa68ae-a136-47eb-b5b1-05109b972928'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:57.508380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc36efe228aa448e89136d0ae3286226', 'request_id': 'req-4efa68ae-a136-47eb-b5b1-05109b972928'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:57.510592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770aa376-e010-434f-a70c-3924b91b3f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:57.517843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770aa376-e010-434f-a70c-3924b91b3f41 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:57.528508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project cc36efe228aa448e89136d0ae3286226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:57.529069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:57.530044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:57.539289 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2392/4758] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:57 2022] DELETE /identity/v3/projects/cc36efe228aa448e89136d0ae3286226 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:57.545072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:57.547527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:57.551433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2367/4759] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:57.656180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa68ae-a136-47eb-b5b1-05109b972928 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:57.663463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e15bb0-07cb-4fa2-b893-fb75f2755410 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:57.664303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15bb0-07cb-4fa2-b893-fb75f2755410 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:57.664870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15bb0-07cb-4fa2-b893-fb75f2755410 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:57.665615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15bb0-07cb-4fa2-b893-fb75f2755410 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:57.667215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e15bb0-07cb-4fa2-b893-fb75f2755410 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:57.691954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e15bb0-07cb-4fa2-b893-fb75f2755410 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:57.759552 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2393/4760] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:57 2022] POST /identity/v3/projects => generated 361 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:57.764054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aa376-e010-434f-a70c-3924b91b3f41 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:57.777632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aa376-e010-434f-a70c-3924b91b3f41 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:57.823325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:57.871480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e15bb0-07cb-4fa2-b893-fb75f2755410 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:57.873814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-25e15bb0-07cb-4fa2-b893-fb75f2755410 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:57.889483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:57.918778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9368cf4-225c-48a6-8915-d4769aa0d380 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:57.919559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9368cf4-225c-48a6-8915-d4769aa0d380 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:57.920202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9368cf4-225c-48a6-8915-d4769aa0d380 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:57.920486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9368cf4-225c-48a6-8915-d4769aa0d380 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/94b8ccab139d43b4a9cafbed2604f0b9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:57.924672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9368cf4-225c-48a6-8915-d4769aa0d380 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:get_project(project_id=94b8ccab139d43b4a9cafbed2604f0b9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:57.932575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9368cf4-225c-48a6-8915-d4769aa0d380 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:57.937793 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2368/4761] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:57 2022] GET /identity/v3/projects/94b8ccab139d43b4a9cafbed2604f0b9 => generated 361 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:57.939043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9368cf4-225c-48a6-8915-d4769aa0d380 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:57.972865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9368cf4-225c-48a6-8915-d4769aa0d380 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:57.990680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:57.991573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:57.992033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:57.992462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/94b8ccab139d43b4a9cafbed2604f0b9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:57.995286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=94b8ccab139d43b4a9cafbed2604f0b9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:57.998925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:58.083753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e15bb0-07cb-4fa2-b893-fb75f2755410 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:58.100815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94b8ccab139d43b4a9cafbed2604f0b9', 'request_id': 'req-d300a64b-7bb6-452e-97bc-bfbd44376a2b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:58.114781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94b8ccab139d43b4a9cafbed2604f0b9', 'request_id': 'req-d300a64b-7bb6-452e-97bc-bfbd44376a2b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:58.125813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94b8ccab139d43b4a9cafbed2604f0b9', 'request_id': 'req-d300a64b-7bb6-452e-97bc-bfbd44376a2b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:58.141911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project 94b8ccab139d43b4a9cafbed2604f0b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:58.142863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:58.143920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:58.146401 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2369/4762] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:57 2022] DELETE /identity/v3/projects/94b8ccab139d43b4a9cafbed2604f0b9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:58.151843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:58.155252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:58.251852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6fe8-fcc6-494d-8672-0cdcb040f514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:58.254675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6fe8-fcc6-494d-8672-0cdcb040f514 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:58.255075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6fe8-fcc6-494d-8672-0cdcb040f514 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:58.255451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6fe8-fcc6-494d-8672-0cdcb040f514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:58.265879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-695d6fe8-fcc6-494d-8672-0cdcb040f514 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:58.267810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300a64b-7bb6-452e-97bc-bfbd44376a2b tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:58.273198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068916c-3934-4725-b009-486b2472fbbe tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:58.273863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068916c-3934-4725-b009-486b2472fbbe tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:58.274242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068916c-3934-4725-b009-486b2472fbbe tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:58.274902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068916c-3934-4725-b009-486b2472fbbe tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:58.276359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068916c-3934-4725-b009-486b2472fbbe tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:58.281295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068916c-3934-4725-b009-486b2472fbbe tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:58.341290 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2370/4763] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:09:58 2022] POST /identity/v3/projects => generated 348 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:09:58.342388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068916c-3934-4725-b009-486b2472fbbe tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:58.374483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d6fe8-fcc6-494d-8672-0cdcb040f514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:58.376850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d6fe8-fcc6-494d-8672-0cdcb040f514 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:58.395323 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2394/4764] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:09:58.404989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:58.437689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068916c-3934-4725-b009-486b2472fbbe tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:58.443477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0fd70-e361-4853-85e2-ec0ec1ebbb0c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:58.444231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0fd70-e361-4853-85e2-ec0ec1ebbb0c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:58.444580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0fd70-e361-4853-85e2-ec0ec1ebbb0c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:58.444919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0fd70-e361-4853-85e2-ec0ec1ebbb0c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/0f81b75f9acb44ee8bfca3ad304c3175` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:58.446180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0fd70-e361-4853-85e2-ec0ec1ebbb0c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:get_project(project_id=0f81b75f9acb44ee8bfca3ad304c3175)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:58.449244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0fd70-e361-4853-85e2-ec0ec1ebbb0c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:58.451000 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2371/4765] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:58 2022] GET /identity/v3/projects/0f81b75f9acb44ee8bfca3ad304c3175 => generated 348 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:58.455365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6fe8-fcc6-494d-8672-0cdcb040f514 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:58.468544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6fe8-fcc6-494d-8672-0cdcb040f514 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:58.473586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:58.474455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:58.475024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:58.475374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/0f81b75f9acb44ee8bfca3ad304c3175` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:58.476772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0f81b75f9acb44ee8bfca3ad304c3175)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:58.479731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:58.562283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:58.612805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f81b75f9acb44ee8bfca3ad304c3175', 'request_id': 'req-a670c095-4fee-4085-9b4a-0131b0c7b6bd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:58.631757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f81b75f9acb44ee8bfca3ad304c3175', 'request_id': 'req-a670c095-4fee-4085-9b4a-0131b0c7b6bd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:58.641794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f81b75f9acb44ee8bfca3ad304c3175', 'request_id': 'req-a670c095-4fee-4085-9b4a-0131b0c7b6bd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:58.658702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project 0f81b75f9acb44ee8bfca3ad304c3175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:58.659454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:58.660766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:58.669547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0fd70-e361-4853-85e2-ec0ec1ebbb0c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:58.670855 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2395/4766] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:58 2022] DELETE /identity/v3/projects/0f81b75f9acb44ee8bfca3ad304c3175 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:58.672283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b0e0fd70-e361-4853-85e2-ec0ec1ebbb0c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:58.690873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0fd70-e361-4853-85e2-ec0ec1ebbb0c tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:58.759017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:58.763902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:58.896346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a95a50-6a0f-4e31-8e13-4cbd6731e315 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:58.897663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a95a50-6a0f-4e31-8e13-4cbd6731e315 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:58.898769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a95a50-6a0f-4e31-8e13-4cbd6731e315 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:58.900378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a95a50-6a0f-4e31-8e13-4cbd6731e315 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:58.908597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a95a50-6a0f-4e31-8e13-4cbd6731e315 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:58.913839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a95a50-6a0f-4e31-8e13-4cbd6731e315 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:58.960653 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2372/4767] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:09:58 2022] POST /identity/v3/domains => generated 276 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:58.965329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a95a50-6a0f-4e31-8e13-4cbd6731e315 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:59.064881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c095-4fee-4085-9b4a-0131b0c7b6bd tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:59.274451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d15e57-6830-4220-bf2d-535066af5640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:59.275749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a95a50-6a0f-4e31-8e13-4cbd6731e315 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:59.276191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d15e57-6830-4220-bf2d-535066af5640 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:59.276451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d15e57-6830-4220-bf2d-535066af5640 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:59.276728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d15e57-6830-4220-bf2d-535066af5640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:59.284332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6af13a-dcec-40e6-a5d8-3462aec4ba81 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:59.285062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6af13a-dcec-40e6-a5d8-3462aec4ba81 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:59.285345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6af13a-dcec-40e6-a5d8-3462aec4ba81 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:59.285860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6af13a-dcec-40e6-a5d8-3462aec4ba81 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:59.286603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6af13a-dcec-40e6-a5d8-3462aec4ba81 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:59.287655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f9d15e57-6830-4220-bf2d-535066af5640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:59.289924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6af13a-dcec-40e6-a5d8-3462aec4ba81 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:59.376184 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2373/4768] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:09:58 2022] POST /identity/v3/projects => generated 404 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:09:59.378576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6af13a-dcec-40e6-a5d8-3462aec4ba81 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:59.466599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:59.483733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d15e57-6830-4220-bf2d-535066af5640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:59.486097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d15e57-6830-4220-bf2d-535066af5640 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:59.490449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6af13a-dcec-40e6-a5d8-3462aec4ba81 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:59.506348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd880d-6539-4187-8ce2-c754fa33e556 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:59.507701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd880d-6539-4187-8ce2-c754fa33e556 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:59.511530 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2396/4769] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:09:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 753 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:09:59.511874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd880d-6539-4187-8ce2-c754fa33e556 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:59.512245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd880d-6539-4187-8ce2-c754fa33e556 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/ab25b9de389142dfb97c597f24de8daa` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:59.517638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd880d-6539-4187-8ce2-c754fa33e556 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:get_project(project_id=ab25b9de389142dfb97c597f24de8daa)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:59.519840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd880d-6539-4187-8ce2-c754fa33e556 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:59.524611 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2374/4770] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:59 2022] GET /identity/v3/projects/ab25b9de389142dfb97c597f24de8daa => generated 404 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:59.536219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d15e57-6830-4220-bf2d-535066af5640 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:59.564185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d15e57-6830-4220-bf2d-535066af5640 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:59.571334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:59.573188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:59.573440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:59.573711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/ab25b9de389142dfb97c597f24de8daa` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:59.574951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ab25b9de389142dfb97c597f24de8daa)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:59.576974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:59.618984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:59.658412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab25b9de389142dfb97c597f24de8daa', 'request_id': 'req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:59.670952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab25b9de389142dfb97c597f24de8daa', 'request_id': 'req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:59.680149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab25b9de389142dfb97c597f24de8daa', 'request_id': 'req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:59.693995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project ab25b9de389142dfb97c597f24de8daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:09:59.694313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:59.695074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:59.701088 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2397/4771] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:09:59 2022] DELETE /identity/v3/projects/ab25b9de389142dfb97c597f24de8daa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:09:59.702432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd880d-6539-4187-8ce2-c754fa33e556 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:59.704450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-46fd880d-6539-4187-8ce2-c754fa33e556 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:09:59.721105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd880d-6539-4187-8ce2-c754fa33e556 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:59.793360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:59.794082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:59.794287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:59.794660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/domains/254e1d4c97444d7a873e6804cc521967` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:59.796060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=254e1d4c97444d7a873e6804cc521967)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:59.797823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:09:59.827381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '254e1d4c97444d7a873e6804cc521967'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:59.828364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '254e1d4c97444d7a873e6804cc521967'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:59.862451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '254e1d4c97444d7a873e6804cc521967'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:59.863863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '254e1d4c97444d7a873e6804cc521967'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:09:59.870496 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2375/4772] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:09:59 2022] PATCH /identity/v3/domains/254e1d4c97444d7a873e6804cc521967 => generated 277 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:09:59.871055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:09:59.882251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4826f5a-1f0c-48ef-8668-d8f30c2d58f6 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:09:59.960994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:09:59.960994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:09:59.960994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:09:59.962198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/domains/254e1d4c97444d7a873e6804cc521967` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:09:59.978780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=254e1d4c97444d7a873e6804cc521967)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:09:59.980613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:00.016519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '254e1d4c97444d7a873e6804cc521967'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:00.020331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '254e1d4c97444d7a873e6804cc521967'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:00.137189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '254e1d4c97444d7a873e6804cc521967', 'request_id': 'req-f70e01b4-82c3-4552-8960-ce30bc97c868'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:00.151698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '254e1d4c97444d7a873e6804cc521967', 'request_id': 'req-f70e01b4-82c3-4552-8960-ce30bc97c868'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:00.161867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '254e1d4c97444d7a873e6804cc521967', 'request_id': 'req-f70e01b4-82c3-4552-8960-ce30bc97c868'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:00.196416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project 254e1d4c97444d7a873e6804cc521967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:00.198400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:00.201170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:00.217539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:00.220798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '254e1d4c97444d7a873e6804cc521967', 'request_id': 'req-f70e01b4-82c3-4552-8960-ce30bc97c868'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:00.221670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '254e1d4c97444d7a873e6804cc521967', 'request_id': 'req-f70e01b4-82c3-4552-8960-ce30bc97c868'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:00.224481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '254e1d4c97444d7a873e6804cc521967', 'request_id': 'req-f70e01b4-82c3-4552-8960-ce30bc97c868'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:00.280446 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2398/4773] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:09:59 2022] DELETE /identity/v3/domains/254e1d4c97444d7a873e6804cc521967 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:00.286575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:00.288446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:00.306064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3be4e2-6fdd-49b7-9685-d7fdf7f8bbaf tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:00.392957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e12bb-fc58-46c5-a82b-7b4fd0883e0e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:00.394157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e12bb-fc58-46c5-a82b-7b4fd0883e0e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:00.395197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e12bb-fc58-46c5-a82b-7b4fd0883e0e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:00.395703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e12bb-fc58-46c5-a82b-7b4fd0883e0e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:00.396629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e12bb-fc58-46c5-a82b-7b4fd0883e0e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:00.400284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e12bb-fc58-46c5-a82b-7b4fd0883e0e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:00.431214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2376/4774] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:10:00 2022] POST /identity/v3/domains => generated 275 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:00.432149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:00.433859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:00.450630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e01b4-82c3-4552-8960-ce30bc97c868 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:00.585671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f9587-f8bc-4b1c-9909-9cb53df411cb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:00.586485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f9587-f8bc-4b1c-9909-9cb53df411cb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:00.587052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f9587-f8bc-4b1c-9909-9cb53df411cb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:00.588019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f9587-f8bc-4b1c-9909-9cb53df411cb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:00.589756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f9587-f8bc-4b1c-9909-9cb53df411cb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:00.591489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f9587-f8bc-4b1c-9909-9cb53df411cb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:00.710462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e12bb-fc58-46c5-a82b-7b4fd0883e0e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:00.711458 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2399/4775] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:00 2022] POST /identity/v3/projects => generated 410 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:10:00.731157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e12bb-fc58-46c5-a82b-7b4fd0883e0e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:00.975599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:00.989243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a31e0-cc66-4e42-9f44-9b74dd5cbe37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:00.990412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a31e0-cc66-4e42-9f44-9b74dd5cbe37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:00.993143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a31e0-cc66-4e42-9f44-9b74dd5cbe37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:00.994707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a31e0-cc66-4e42-9f44-9b74dd5cbe37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:00.996112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a31e0-cc66-4e42-9f44-9b74dd5cbe37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:00.999712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a31e0-cc66-4e42-9f44-9b74dd5cbe37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:01.035003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f9587-f8bc-4b1c-9909-9cb53df411cb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:01.037146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-164f9587-f8bc-4b1c-9909-9cb53df411cb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:01.120497 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2377/4776] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:00 2022] POST /identity/v3/projects => generated 404 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:01.128951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a31e0-cc66-4e42-9f44-9b74dd5cbe37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:01.297562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f9587-f8bc-4b1c-9909-9cb53df411cb tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:01.358712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a31e0-cc66-4e42-9f44-9b74dd5cbe37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:01.377261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:01.377930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:01.378209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:01.378480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/8fc3be06f58b4f5d8327d7d9339c96ac` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:01.380072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8fc3be06f58b4f5d8327d7d9339c96ac)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:01.383309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:01.459974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:01.469431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc3be06f58b4f5d8327d7d9339c96ac', 'request_id': 'req-0baab734-22f9-4566-93e2-6750b492837d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.475296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7838585-18a5-4dd4-b8d1-a86c04c9f9e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:01.475846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7838585-18a5-4dd4-b8d1-a86c04c9f9e8 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:01.476116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7838585-18a5-4dd4-b8d1-a86c04c9f9e8 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:01.476384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7838585-18a5-4dd4-b8d1-a86c04c9f9e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:01.481991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b7838585-18a5-4dd4-b8d1-a86c04c9f9e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:01.497372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc3be06f58b4f5d8327d7d9339c96ac', 'request_id': 'req-0baab734-22f9-4566-93e2-6750b492837d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.501353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7838585-18a5-4dd4-b8d1-a86c04c9f9e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:01.502972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7838585-18a5-4dd4-b8d1-a86c04c9f9e8 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:01.512995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc3be06f58b4f5d8327d7d9339c96ac', 'request_id': 'req-0baab734-22f9-4566-93e2-6750b492837d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.519574 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2400/4777] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:01.520475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7838585-18a5-4dd4-b8d1-a86c04c9f9e8 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:01.522090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b7838585-18a5-4dd4-b8d1-a86c04c9f9e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:01.569311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project 8fc3be06f58b4f5d8327d7d9339c96ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:01.569972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.582203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.590906 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2378/4778] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:01 2022] DELETE /identity/v3/projects/8fc3be06f58b4f5d8327d7d9339c96ac => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:01.592193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:01.594062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:01.660353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7838585-18a5-4dd4-b8d1-a86c04c9f9e8 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:01.725432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baab734-22f9-4566-93e2-6750b492837d tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:01.738721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:01.740052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:01.740555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:01.741125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/b0365c618ea945d08467282bfbeeb6c9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:01.743037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b0365c618ea945d08467282bfbeeb6c9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:01.746175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:01.754577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:01.802800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:01.803247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:01.803602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:01.803918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] PATH_INFO: `/v3/users/2f4e4fc2f97f4c1a832c6348a97fd155` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:01.806557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f4e4fc2f97f4c1a832c6348a97fd155)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:01.811597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:01.864970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0365c618ea945d08467282bfbeeb6c9', 'request_id': 'req-b1c99a4c-d458-466f-ba60-84aca5ece355'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.885870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0365c618ea945d08467282bfbeeb6c9', 'request_id': 'req-b1c99a4c-d458-466f-ba60-84aca5ece355'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.904331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0365c618ea945d08467282bfbeeb6c9', 'request_id': 'req-b1c99a4c-d458-466f-ba60-84aca5ece355'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.909653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f4e4fc2f97f4c1a832c6348a97fd155'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.929601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f4e4fc2f97f4c1a832c6348a97fd155'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.936538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project b0365c618ea945d08467282bfbeeb6c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:01.937521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.938649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.943205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2379/4779] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:01 2022] DELETE /identity/v3/projects/b0365c618ea945d08467282bfbeeb6c9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:01.947596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f4e4fc2f97f4c1a832c6348a97fd155'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.961444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:01.961886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f4e4fc2f97f4c1a832c6348a97fd155'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.963800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f4e4fc2f97f4c1a832c6348a97fd155'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:01.964353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:01.973623 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2401/4780] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:01 2022] DELETE /identity/v3/users/2f4e4fc2f97f4c1a832c6348a97fd155 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:01.973623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:01.973623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:02.000060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:02.070811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c99a4c-d458-466f-ba60-84aca5ece355 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:02.096136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f314658-fba7-4c20-836a-e2ed8910a622 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:02.108603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:02.109292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:02.109570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:02.109847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/domains/85152a07a2ef40d59e5ccc1783a27919` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:02.112219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=85152a07a2ef40d59e5ccc1783a27919)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:02.113911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:02.162376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.165187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.184262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:02.184262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:02.184262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:02.184262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] PATH_INFO: `/v3/users/44af21476ed8446d98640881d322eea3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:02.185881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=44af21476ed8446d98640881d322eea3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:02.187846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:02.218809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.219610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.225092 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2402/4781] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:10:01 2022] PATCH /identity/v3/domains/85152a07a2ef40d59e5ccc1783a27919 => generated 276 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:02.227479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:02.230478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:02.281918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:02.285616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44af21476ed8446d98640881d322eea3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.297010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44af21476ed8446d98640881d322eea3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.315687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44af21476ed8446d98640881d322eea3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.327759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44af21476ed8446d98640881d322eea3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.328677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-92b40fd9-b6c0-480a-a40e-7a628aaea5ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44af21476ed8446d98640881d322eea3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.338450 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2380/4782] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:01 2022] DELETE /identity/v3/users/44af21476ed8446d98640881d322eea3 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:02.343809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bf6a4-2a52-46e4-aadb-8500f231348e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:02.344099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bf6a4-2a52-46e4-aadb-8500f231348e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:02.344370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bf6a4-2a52-46e4-aadb-8500f231348e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:02.347079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c52c9d-d97d-4733-b4c0-bfb0487f30ed tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:02.371198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6c8bf6a4-2a52-46e4-aadb-8500f231348e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:02.372947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6c8bf6a4-2a52-46e4-aadb-8500f231348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:02.501513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:02.502246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:02.504406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:02.504708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/domains/85152a07a2ef40d59e5ccc1783a27919` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:02.525508 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2381/4783] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:10:02.532978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=85152a07a2ef40d59e5ccc1783a27919)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:02.536551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:02.538827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bf6a4-2a52-46e4-aadb-8500f231348e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:02.545980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6c8bf6a4-2a52-46e4-aadb-8500f231348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:02.576412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.579419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:02.583467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.661639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919', 'request_id': 'req-273da2f0-7017-4a83-a73e-4204286d5946'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.666475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bf6a4-2a52-46e4-aadb-8500f231348e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:02.673915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919', 'request_id': 'req-273da2f0-7017-4a83-a73e-4204286d5946'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.683891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919', 'request_id': 'req-273da2f0-7017-4a83-a73e-4204286d5946'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.699844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project 85152a07a2ef40d59e5ccc1783a27919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:02.700394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.701230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.711008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919', 'request_id': 'req-273da2f0-7017-4a83-a73e-4204286d5946'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.712966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919', 'request_id': 'req-273da2f0-7017-4a83-a73e-4204286d5946'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.713753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '85152a07a2ef40d59e5ccc1783a27919', 'request_id': 'req-273da2f0-7017-4a83-a73e-4204286d5946'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:02.733130 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2403/4784] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:10:02 2022] DELETE /identity/v3/domains/85152a07a2ef40d59e5ccc1783a27919 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:02.736866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:02.740407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:02.777130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d79b49-3600-46f3-a8b8-61286f8cfd9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:02.781670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d79b49-3600-46f3-a8b8-61286f8cfd9b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:02.782243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d79b49-3600-46f3-a8b8-61286f8cfd9b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:02.782751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d79b49-3600-46f3-a8b8-61286f8cfd9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:02.787694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c1d79b49-3600-46f3-a8b8-61286f8cfd9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:02.860492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273da2f0-7017-4a83-a73e-4204286d5946 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:02.867733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:02.869789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:02.869789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:02.869789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] PATH_INFO: `/v3/projects/cfe55a7ec21b4e6ebbf53684a480e535` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:02.871295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe55a7ec21b4e6ebbf53684a480e535)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:02.874595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:02.887822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:02.934124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d79b49-3600-46f3-a8b8-61286f8cfd9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:02.939878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d79b49-3600-46f3-a8b8-61286f8cfd9b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:02.973557 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2382/4785] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:02.974642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d79b49-3600-46f3-a8b8-61286f8cfd9b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:02.976586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c1d79b49-3600-46f3-a8b8-61286f8cfd9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:03.030870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe55a7ec21b4e6ebbf53684a480e535', 'request_id': 'req-4e8dba52-d859-4c3e-9c89-be63a7508256'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:03.054089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe55a7ec21b4e6ebbf53684a480e535', 'request_id': 'req-4e8dba52-d859-4c3e-9c89-be63a7508256'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:03.067223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe55a7ec21b4e6ebbf53684a480e535', 'request_id': 'req-4e8dba52-d859-4c3e-9c89-be63a7508256'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:03.077231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:03.090893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] The token cache is being invalidate because project cfe55a7ec21b4e6ebbf53684a480e535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:03.091514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:03.092480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4e8dba52-d859-4c3e-9c89-be63a7508256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:03.100461 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2404/4786] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:02 2022] DELETE /identity/v3/projects/cfe55a7ec21b4e6ebbf53684a480e535 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:03.100669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d79b49-3600-46f3-a8b8-61286f8cfd9b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:03.103497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b9658-567a-470d-9df1-9ef05891343b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:03.104835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b9658-567a-470d-9df1-9ef05891343b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:03.105590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b9658-567a-470d-9df1-9ef05891343b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:03.137030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bc1b9658-567a-470d-9df1-9ef05891343b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:03.138549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bc1b9658-567a-470d-9df1-9ef05891343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:03.194328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473838a1-4f5f-44ad-98b8-1e5c7c33d5ba tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:03.197830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473838a1-4f5f-44ad-98b8-1e5c7c33d5ba tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:03.198481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473838a1-4f5f-44ad-98b8-1e5c7c33d5ba tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:03.199001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473838a1-4f5f-44ad-98b8-1e5c7c33d5ba tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:03.200179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473838a1-4f5f-44ad-98b8-1e5c7c33d5ba tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:03.204999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473838a1-4f5f-44ad-98b8-1e5c7c33d5ba tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:03.295754 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2383/4787] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:02 2022] POST /identity/v3/projects => generated 377 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:03.300495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473838a1-4f5f-44ad-98b8-1e5c7c33d5ba tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:03.302883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-473838a1-4f5f-44ad-98b8-1e5c7c33d5ba tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:03.309918 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2405/4788] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:03.398305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0d8f9-c2df-4c4f-a673-5fdf403ed9a2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:03.398890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0d8f9-c2df-4c4f-a673-5fdf403ed9a2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:03.399310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0d8f9-c2df-4c4f-a673-5fdf403ed9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:03.429487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:03.435434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473838a1-4f5f-44ad-98b8-1e5c7c33d5ba tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:03.443232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-82c0d8f9-c2df-4c4f-a673-5fdf403ed9a2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:03.444825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-82c0d8f9-c2df-4c4f-a673-5fdf403ed9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:03.452291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621561c-d94c-4b1b-aa4d-fff3d55d4501 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:03.452813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621561c-d94c-4b1b-aa4d-fff3d55d4501 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:03.453116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621561c-d94c-4b1b-aa4d-fff3d55d4501 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:03.453388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621561c-d94c-4b1b-aa4d-fff3d55d4501 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/7efb4cd2376e48f7b4317fdb3b46d02a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:03.459806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621561c-d94c-4b1b-aa4d-fff3d55d4501 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:get_project(project_id=7efb4cd2376e48f7b4317fdb3b46d02a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:03.470773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621561c-d94c-4b1b-aa4d-fff3d55d4501 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:03.476234 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2384/4789] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:10:03 2022] GET /identity/v3/projects/7efb4cd2376e48f7b4317fdb3b46d02a => generated 377 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:10:03.477439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621561c-d94c-4b1b-aa4d-fff3d55d4501 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:03.498359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621561c-d94c-4b1b-aa4d-fff3d55d4501 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:03.520377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627dee5-49a8-4718-a0a5-6c0fc469c8b1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:03.521290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627dee5-49a8-4718-a0a5-6c0fc469c8b1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:03.521683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627dee5-49a8-4718-a0a5-6c0fc469c8b1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:03.522109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627dee5-49a8-4718-a0a5-6c0fc469c8b1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:03.522886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627dee5-49a8-4718-a0a5-6c0fc469c8b1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:03.527226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627dee5-49a8-4718-a0a5-6c0fc469c8b1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:03.563070 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2385/4790] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Oct 15 21:10:03 2022] GET /identity/v3/projects => generated 6955 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:03.567643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627dee5-49a8-4718-a0a5-6c0fc469c8b1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:03.580749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627dee5-49a8-4718-a0a5-6c0fc469c8b1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:03.586356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:03.587288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:03.587571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:03.587834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/7efb4cd2376e48f7b4317fdb3b46d02a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:03.589886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7efb4cd2376e48f7b4317fdb3b46d02a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:03.591995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:03.595995 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2406/4791] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:10:03.600553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0d8f9-c2df-4c4f-a673-5fdf403ed9a2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:03.617495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0d8f9-c2df-4c4f-a673-5fdf403ed9a2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:03.624321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7167a2-d5f1-4bc3-8bc1-0941ac494225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:03.625281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7167a2-d5f1-4bc3-8bc1-0941ac494225 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:03.625664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7167a2-d5f1-4bc3-8bc1-0941ac494225 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:03.626037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7167a2-d5f1-4bc3-8bc1-0941ac494225 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:03.626950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7167a2-d5f1-4bc3-8bc1-0941ac494225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:03.627360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7167a2-d5f1-4bc3-8bc1-0941ac494225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:03.629127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7167a2-d5f1-4bc3-8bc1-0941ac494225 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:03.665162 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2407/4792] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:10:03.668843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7167a2-d5f1-4bc3-8bc1-0941ac494225 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:03.676827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7efb4cd2376e48f7b4317fdb3b46d02a', 'request_id': 'req-c9576be8-9b2e-4775-8604-48a00f382426'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:03.691762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7167a2-d5f1-4bc3-8bc1-0941ac494225 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:03.702858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7efb4cd2376e48f7b4317fdb3b46d02a', 'request_id': 'req-c9576be8-9b2e-4775-8604-48a00f382426'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:03.710049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7efb4cd2376e48f7b4317fdb3b46d02a', 'request_id': 'req-c9576be8-9b2e-4775-8604-48a00f382426'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:03.742883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project 7efb4cd2376e48f7b4317fdb3b46d02a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:03.745844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:03.748820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:03.755537 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2386/4793] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:03 2022] DELETE /identity/v3/projects/7efb4cd2376e48f7b4317fdb3b46d02a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:03.762606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:03.765646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:03.811150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51902ed6-a3c2-473f-a58b-120603c809bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:03.811783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51902ed6-a3c2-473f-a58b-120603c809bb admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:03.812053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51902ed6-a3c2-473f-a58b-120603c809bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:03.812331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51902ed6-a3c2-473f-a58b-120603c809bb admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:03.813183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51902ed6-a3c2-473f-a58b-120603c809bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:03.817112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51902ed6-a3c2-473f-a58b-120603c809bb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:03.872584 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2408/4794] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:03 2022] POST /identity/v3/projects => generated 371 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:03.876664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51902ed6-a3c2-473f-a58b-120603c809bb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:03.879457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-51902ed6-a3c2-473f-a58b-120603c809bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:03.897079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9576be8-9b2e-4775-8604-48a00f382426 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:03.901448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:04.017997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51902ed6-a3c2-473f-a58b-120603c809bb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:04.032194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14c0fb-fb2f-4c1e-b515-a02cad1f34a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:04.033606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c0fb-fb2f-4c1e-b515-a02cad1f34a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.034111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c0fb-fb2f-4c1e-b515-a02cad1f34a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.036231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c0fb-fb2f-4c1e-b515-a02cad1f34a7 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.037287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14c0fb-fb2f-4c1e-b515-a02cad1f34a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:04.043543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14c0fb-fb2f-4c1e-b515-a02cad1f34a7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:04.077720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f74006-ccc1-41a4-9790-d543c87caaf8 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:04.078517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f74006-ccc1-41a4-9790-d543c87caaf8 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.078793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f74006-ccc1-41a4-9790-d543c87caaf8 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.079197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f74006-ccc1-41a4-9790-d543c87caaf8 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.079834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f74006-ccc1-41a4-9790-d543c87caaf8 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:04.082316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f74006-ccc1-41a4-9790-d543c87caaf8 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:04.094732 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2409/4795] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:03 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:04.096200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14c0fb-fb2f-4c1e-b515-a02cad1f34a7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:04.099835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f14c0fb-fb2f-4c1e-b515-a02cad1f34a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:04.118018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:04.143432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14c0fb-fb2f-4c1e-b515-a02cad1f34a7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:04.147205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2387/4796] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:10:03 2022] POST /identity/v3/projects => generated 347 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:04.150173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f74006-ccc1-41a4-9790-d543c87caaf8 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:04.281344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:04.282054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.282487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.282825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] PATH_INFO: `/v3/projects/4e6b708b260348cf904e392e1329e1d2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.284943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6b708b260348cf904e392e1329e1d2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:04.286938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:04.296226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f74006-ccc1-41a4-9790-d543c87caaf8 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:04.301887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34824ec8-ca7b-4dcd-b4eb-f3b86326703d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:04.302784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34824ec8-ca7b-4dcd-b4eb-f3b86326703d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.303382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34824ec8-ca7b-4dcd-b4eb-f3b86326703d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.303810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34824ec8-ca7b-4dcd-b4eb-f3b86326703d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.304976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34824ec8-ca7b-4dcd-b4eb-f3b86326703d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:04.307225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34824ec8-ca7b-4dcd-b4eb-f3b86326703d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:04.328043 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2388/4797] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:04.329450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34824ec8-ca7b-4dcd-b4eb-f3b86326703d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:04.370803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6b708b260348cf904e392e1329e1d2', 'request_id': 'req-527c47f9-dde4-44ab-a485-a0199f73e621'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:04.390330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6b708b260348cf904e392e1329e1d2', 'request_id': 'req-527c47f9-dde4-44ab-a485-a0199f73e621'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:04.401823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6b708b260348cf904e392e1329e1d2', 'request_id': 'req-527c47f9-dde4-44ab-a485-a0199f73e621'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:04.426803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] The token cache is being invalidate because project 4e6b708b260348cf904e392e1329e1d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:04.427080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:04.428643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:04.430368 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2410/4798] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:04 2022] DELETE /identity/v3/projects/4e6b708b260348cf904e392e1329e1d2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:04.435412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:04.442301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:04.457150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34824ec8-ca7b-4dcd-b4eb-f3b86326703d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:04.518130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:04.592233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f810ee-d2a7-4964-93d9-3797cfdb5c58 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:04.592886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f810ee-d2a7-4964-93d9-3797cfdb5c58 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.593364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f810ee-d2a7-4964-93d9-3797cfdb5c58 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.593624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f810ee-d2a7-4964-93d9-3797cfdb5c58 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/46e4604a76d94756a52455592564e3c3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.595139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f810ee-d2a7-4964-93d9-3797cfdb5c58 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:update_project(project_id=46e4604a76d94756a52455592564e3c3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:04.597179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f810ee-d2a7-4964-93d9-3797cfdb5c58 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:04.601133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c47f9-dde4-44ab-a485-a0199f73e621 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:04.613672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75522aff-79b7-49b3-abb0-c0e7eccef793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:04.614553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75522aff-79b7-49b3-abb0-c0e7eccef793 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.614832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75522aff-79b7-49b3-abb0-c0e7eccef793 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.615104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75522aff-79b7-49b3-abb0-c0e7eccef793 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.615751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75522aff-79b7-49b3-abb0-c0e7eccef793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:04.619556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75522aff-79b7-49b3-abb0-c0e7eccef793 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:04.640713 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2411/4799] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:04.642372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070b5d-d30f-470c-873e-64b6614e2864 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.642653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070b5d-d30f-470c-873e-64b6614e2864 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.642923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070b5d-d30f-470c-873e-64b6614e2864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.656393 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2389/4800] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Oct 15 21:10:04 2022] PATCH /identity/v3/projects/46e4604a76d94756a52455592564e3c3 => generated 361 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:04.657109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f810ee-d2a7-4964-93d9-3797cfdb5c58 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:04.688202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f810ee-d2a7-4964-93d9-3797cfdb5c58 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:04.690565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-89070b5d-d30f-470c-873e-64b6614e2864 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:04.692014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-89070b5d-d30f-470c-873e-64b6614e2864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:04.700707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59022e-fa5c-46d5-b389-20b3ee9140bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:04.701611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59022e-fa5c-46d5-b389-20b3ee9140bb admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.701873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59022e-fa5c-46d5-b389-20b3ee9140bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.702139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59022e-fa5c-46d5-b389-20b3ee9140bb admin admin] PATH_INFO: `/v3/projects/ffe59f75f502409eb26737b70fd58536/users/da9fa6201a6641c1a4d42d3bad77214d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.703803 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2412/4801] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:04.704827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89070b5d-d30f-470c-873e-64b6614e2864 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:04.706295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-89070b5d-d30f-470c-873e-64b6614e2864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:04.735960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89070b5d-d30f-470c-873e-64b6614e2864 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:04.740733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59022e-fa5c-46d5-b389-20b3ee9140bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffe59f75f502409eb26737b70fd58536, user_id=da9fa6201a6641c1a4d42d3bad77214d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:04.744773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59022e-fa5c-46d5-b389-20b3ee9140bb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:04.749739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f04eba-f813-45fb-aa37-6fcd18754e55 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:04.750457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f04eba-f813-45fb-aa37-6fcd18754e55 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.750785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f04eba-f813-45fb-aa37-6fcd18754e55 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.751243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f04eba-f813-45fb-aa37-6fcd18754e55 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/46e4604a76d94756a52455592564e3c3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.764507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:04.778056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f04eba-f813-45fb-aa37-6fcd18754e55 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:get_project(project_id=46e4604a76d94756a52455592564e3c3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:04.780070 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2390/4802] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:04 2022] PUT /identity/v3/projects/ffe59f75f502409eb26737b70fd58536/users/da9fa6201a6641c1a4d42d3bad77214d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:04.780584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f04eba-f813-45fb-aa37-6fcd18754e55 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:04.783557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed869d-c2e2-4eca-b5bf-27474d58812b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.784019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed869d-c2e2-4eca-b5bf-27474d58812b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.784392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed869d-c2e2-4eca-b5bf-27474d58812b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.792689 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2413/4803] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:10:04 2022] GET /identity/v3/projects/46e4604a76d94756a52455592564e3c3 => generated 348 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:04.794625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b154d5-4189-4700-ae50-1a594bc5291d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:04.794937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b154d5-4189-4700-ae50-1a594bc5291d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:04.795213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b154d5-4189-4700-ae50-1a594bc5291d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:04.857524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-64b154d5-4189-4700-ae50-1a594bc5291d None None] MFA Rules not processed for user `da9fa6201a6641c1a4d42d3bad77214d`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:04.858761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-64b154d5-4189-4700-ae50-1a594bc5291d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:04.865942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c6ed869d-c2e2-4eca-b5bf-27474d58812b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:04.867708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c6ed869d-c2e2-4eca-b5bf-27474d58812b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:05.015651 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2391/4804] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:05.016574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed869d-c2e2-4eca-b5bf-27474d58812b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:05.047907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2414/4805] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:05.049699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b154d5-4189-4700-ae50-1a594bc5291d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:05.064096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b154d5-4189-4700-ae50-1a594bc5291d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:05.075189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59493049-e3f6-4344-ac47-0e0c42cb1c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:05.075814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493049-e3f6-4344-ac47-0e0c42cb1c75 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:05.076093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493049-e3f6-4344-ac47-0e0c42cb1c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:05.076365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493049-e3f6-4344-ac47-0e0c42cb1c75 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:05.081160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59493049-e3f6-4344-ac47-0e0c42cb1c75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:05.081985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59493049-e3f6-4344-ac47-0e0c42cb1c75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:05.083292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59493049-e3f6-4344-ac47-0e0c42cb1c75 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:05.111007 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2415/4806] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:05.114753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59493049-e3f6-4344-ac47-0e0c42cb1c75 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:05.134927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59493049-e3f6-4344-ac47-0e0c42cb1c75 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:05.144007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0882e-bc8f-4784-880c-ae385a92c420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:05.144642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0882e-bc8f-4784-880c-ae385a92c420 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:05.144916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0882e-bc8f-4784-880c-ae385a92c420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:05.145309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0882e-bc8f-4784-880c-ae385a92c420 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:05.145972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0882e-bc8f-4784-880c-ae385a92c420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:05.148009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0882e-bc8f-4784-880c-ae385a92c420 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:05.161628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed869d-c2e2-4eca-b5bf-27474d58812b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:05.167383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:05.168794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:05.170272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:05.170777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/46e4604a76d94756a52455592564e3c3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:05.172557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=46e4604a76d94756a52455592564e3c3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:05.174757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:05.202081 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2416/4807] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:05 2022] POST /identity/v3/projects => generated 369 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:05.204962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0882e-bc8f-4784-880c-ae385a92c420 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:05.288610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e4604a76d94756a52455592564e3c3', 'request_id': 'req-85af357a-9fee-4377-8354-ff2a901dfaf1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:05.307285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e4604a76d94756a52455592564e3c3', 'request_id': 'req-85af357a-9fee-4377-8354-ff2a901dfaf1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:05.317781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e4604a76d94756a52455592564e3c3', 'request_id': 'req-85af357a-9fee-4377-8354-ff2a901dfaf1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:05.346167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project 46e4604a76d94756a52455592564e3c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:05.346630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:05.347527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:05.349201 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2392/4808] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:05 2022] DELETE /identity/v3/projects/46e4604a76d94756a52455592564e3c3 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:05.350058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:05.357365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:05.412622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0882e-bc8f-4784-880c-ae385a92c420 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:05.418284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:05.434906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:05.503006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95b636-5b8e-40e3-97cb-0d7258270cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:05.504711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95b636-5b8e-40e3-97cb-0d7258270cbd admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:05.504994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95b636-5b8e-40e3-97cb-0d7258270cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:05.505758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95b636-5b8e-40e3-97cb-0d7258270cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:05.507054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af357a-9fee-4377-8354-ff2a901dfaf1 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:05.507546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95b636-5b8e-40e3-97cb-0d7258270cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:05.512708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa5d83-4fd9-40f2-8f7b-5326a748f9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:05.513498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa5d83-4fd9-40f2-8f7b-5326a748f9db admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:05.513727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95b636-5b8e-40e3-97cb-0d7258270cbd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:05.514465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa5d83-4fd9-40f2-8f7b-5326a748f9db admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:05.514465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa5d83-4fd9-40f2-8f7b-5326a748f9db admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:05.515114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa5d83-4fd9-40f2-8f7b-5326a748f9db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:05.518200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa5d83-4fd9-40f2-8f7b-5326a748f9db admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:05.552205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2393/4809] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:05 2022] POST /identity/v3/users => generated 291 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:05.554097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa5d83-4fd9-40f2-8f7b-5326a748f9db admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:05.559711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8fa5d83-4fd9-40f2-8f7b-5326a748f9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:05.592172 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2417/4810] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:05 2022] POST /identity/v3/projects => generated 369 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:05.592769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95b636-5b8e-40e3-97cb-0d7258270cbd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:05.698554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa5d83-4fd9-40f2-8f7b-5326a748f9db admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:05.744504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95b636-5b8e-40e3-97cb-0d7258270cbd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:05.764702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b525d-1bf2-41f3-97c7-d359ad9f1d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:05.767265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b525d-1bf2-41f3-97c7-d359ad9f1d0e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:05.768172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b525d-1bf2-41f3-97c7-d359ad9f1d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:05.770365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b525d-1bf2-41f3-97c7-d359ad9f1d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:05.771281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b525d-1bf2-41f3-97c7-d359ad9f1d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:05.773203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b525d-1bf2-41f3-97c7-d359ad9f1d0e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:05.783049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8ead3-ffe7-4dfe-b857-a611bd207325 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:05.783680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8ead3-ffe7-4dfe-b857-a611bd207325 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:05.783956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8ead3-ffe7-4dfe-b857-a611bd207325 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:05.784371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8ead3-ffe7-4dfe-b857-a611bd207325 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:05.785322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8ead3-ffe7-4dfe-b857-a611bd207325 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:05.787169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8ead3-ffe7-4dfe-b857-a611bd207325 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:05.791635 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2418/4811] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:05.792773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b525d-1bf2-41f3-97c7-d359ad9f1d0e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:05.794974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b35b525d-1bf2-41f3-97c7-d359ad9f1d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:05.813497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b525d-1bf2-41f3-97c7-d359ad9f1d0e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:05.831385 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2394/4812] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:05 2022] POST /identity/v3/projects => generated 358 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:05.833846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8ead3-ffe7-4dfe-b857-a611bd207325 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:05.836623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84e4df-217e-40bb-b959-a59890c5a55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:05.838391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84e4df-217e-40bb-b959-a59890c5a55d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:05.839030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84e4df-217e-40bb-b959-a59890c5a55d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:05.839318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84e4df-217e-40bb-b959-a59890c5a55d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:05.840113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84e4df-217e-40bb-b959-a59890c5a55d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:05.842814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84e4df-217e-40bb-b959-a59890c5a55d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:05.879720 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2419/4813] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:05 2022] POST /identity/v3/users => generated 285 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:05.882318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84e4df-217e-40bb-b959-a59890c5a55d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:05.886993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7e84e4df-217e-40bb-b959-a59890c5a55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:05.968020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:05.977313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84e4df-217e-40bb-b959-a59890c5a55d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.016497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8ead3-ffe7-4dfe-b857-a611bd207325 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.018743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:06.025063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d07f5-966a-4c61-9c4f-0cd6811d538a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.025770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d07f5-966a-4c61-9c4f-0cd6811d538a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.026188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d07f5-966a-4c61-9c4f-0cd6811d538a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.026467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d07f5-966a-4c61-9c4f-0cd6811d538a admin admin] PATH_INFO: `/v3/projects/6789f3b886e5405caa7ef105cbc32f29/users/d80fd4a265644d69a566edde911aefea/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.050115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d07f5-966a-4c61-9c4f-0cd6811d538a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6789f3b886e5405caa7ef105cbc32f29, user_id=d80fd4a265644d69a566edde911aefea, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.053463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d07f5-966a-4c61-9c4f-0cd6811d538a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:06.060482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09995c-fcd2-4fb6-824e-1721f36b5ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.061748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09995c-fcd2-4fb6-824e-1721f36b5ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.062542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09995c-fcd2-4fb6-824e-1721f36b5ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.063167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09995c-fcd2-4fb6-824e-1721f36b5ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.067975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af09995c-fcd2-4fb6-824e-1721f36b5ec9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:06.068895 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2395/4814] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:05 2022] PUT /identity/v3/projects/6789f3b886e5405caa7ef105cbc32f29/users/d80fd4a265644d69a566edde911aefea/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:06.069825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d07f5-966a-4c61-9c4f-0cd6811d538a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:06.154058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09995c-fcd2-4fb6-824e-1721f36b5ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.156158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09995c-fcd2-4fb6-824e-1721f36b5ec9 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:06.159490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d07f5-966a-4c61-9c4f-0cd6811d538a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.166075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa0d69-3440-4cec-9dee-538aab06c4a7 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.166804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa0d69-3440-4cec-9dee-538aab06c4a7 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.167238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa0d69-3440-4cec-9dee-538aab06c4a7 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.167814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa0d69-3440-4cec-9dee-538aab06c4a7 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/a7a17c8feaae4ff59182dafed4c1d3e0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.167931 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2420/4815] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:06.168619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09995c-fcd2-4fb6-824e-1721f36b5ec9 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:06.169828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa0d69-3440-4cec-9dee-538aab06c4a7 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:update_project(project_id=a7a17c8feaae4ff59182dafed4c1d3e0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.172092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa0d69-3440-4cec-9dee-538aab06c4a7 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:06.181653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09995c-fcd2-4fb6-824e-1721f36b5ec9 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.268675 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2396/4816] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Oct 15 21:10:06 2022] PATCH /identity/v3/projects/a7a17c8feaae4ff59182dafed4c1d3e0 => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:06.269653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa0d69-3440-4cec-9dee-538aab06c4a7 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:06.277942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ba5a8-c839-4d12-8431-248dde6d3315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.278783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ba5a8-c839-4d12-8431-248dde6d3315 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.279214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ba5a8-c839-4d12-8431-248dde6d3315 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.279673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ba5a8-c839-4d12-8431-248dde6d3315 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.280313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ba5a8-c839-4d12-8431-248dde6d3315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.281951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ba5a8-c839-4d12-8431-248dde6d3315 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:06.297015 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2421/4817] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:06.297015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ba5a8-c839-4d12-8431-248dde6d3315 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:06.308054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ba5a8-c839-4d12-8431-248dde6d3315 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.314944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afe4bc-7732-4542-81f8-95b956eb4e37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.315651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afe4bc-7732-4542-81f8-95b956eb4e37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.315924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afe4bc-7732-4542-81f8-95b956eb4e37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.316743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afe4bc-7732-4542-81f8-95b956eb4e37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/a7a17c8feaae4ff59182dafed4c1d3e0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.331948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10afe4bc-7732-4542-81f8-95b956eb4e37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:get_project(project_id=a7a17c8feaae4ff59182dafed4c1d3e0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.334813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10afe4bc-7732-4542-81f8-95b956eb4e37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:06.340388 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2422/4818] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:10:06 2022] GET /identity/v3/projects/a7a17c8feaae4ff59182dafed4c1d3e0 => generated 357 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:06.340388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afe4bc-7732-4542-81f8-95b956eb4e37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:06.349463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afe4bc-7732-4542-81f8-95b956eb4e37 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.354236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa0d69-3440-4cec-9dee-538aab06c4a7 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.355102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e708083-0102-4550-9ecb-1984610e5046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.355814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e708083-0102-4550-9ecb-1984610e5046 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.356148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e708083-0102-4550-9ecb-1984610e5046 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.356342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e708083-0102-4550-9ecb-1984610e5046 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.357017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e708083-0102-4550-9ecb-1984610e5046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.358636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e708083-0102-4550-9ecb-1984610e5046 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:06.359509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f5953-4471-442f-87c2-4b55659bff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.360271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f5953-4471-442f-87c2-4b55659bff18 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.360555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f5953-4471-442f-87c2-4b55659bff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.360847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f5953-4471-442f-87c2-4b55659bff18 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.361517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f5953-4471-442f-87c2-4b55659bff18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.363098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f5953-4471-442f-87c2-4b55659bff18 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:06.372638 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2423/4819] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:06.372638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e708083-0102-4550-9ecb-1984610e5046 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:06.377818 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2397/4820] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:06.377818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f5953-4471-442f-87c2-4b55659bff18 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:06.388956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e708083-0102-4550-9ecb-1984610e5046 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.389417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f5953-4471-442f-87c2-4b55659bff18 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.394387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d83571-7f06-499c-afb4-c844d580a858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.395147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d83571-7f06-499c-afb4-c844d580a858 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.395446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d83571-7f06-499c-afb4-c844d580a858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.395791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d83571-7f06-499c-afb4-c844d580a858 admin admin] PATH_INFO: `/v3/projects/d258d5a5a3a74ce180e9d41ab4b81d71/users/b5dbd6ee072842dca3fc7ee0b40917e5/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.397818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.398777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.399260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.399821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/a7a17c8feaae4ff59182dafed4c1d3e0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.401872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a7a17c8feaae4ff59182dafed4c1d3e0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.403883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:06.427868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d83571-7f06-499c-afb4-c844d580a858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d258d5a5a3a74ce180e9d41ab4b81d71, user_id=b5dbd6ee072842dca3fc7ee0b40917e5, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.427868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d83571-7f06-499c-afb4-c844d580a858 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:06.450185 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2398/4821] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:06 2022] PUT /identity/v3/projects/d258d5a5a3a74ce180e9d41ab4b81d71/users/b5dbd6ee072842dca3fc7ee0b40917e5/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:06.451039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d83571-7f06-499c-afb4-c844d580a858 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:06.473503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:06.481528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:06.484755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a17c8feaae4ff59182dafed4c1d3e0', 'request_id': 'req-7bb78098-af6b-4058-bded-a4f9f099869a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:06.506218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a17c8feaae4ff59182dafed4c1d3e0', 'request_id': 'req-7bb78098-af6b-4058-bded-a4f9f099869a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:06.521542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a17c8feaae4ff59182dafed4c1d3e0', 'request_id': 'req-7bb78098-af6b-4058-bded-a4f9f099869a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:06.545286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project a7a17c8feaae4ff59182dafed4c1d3e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:06.545286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:06.546259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bb78098-af6b-4058-bded-a4f9f099869a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:06.550966 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2424/4822] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:06 2022] DELETE /identity/v3/projects/a7a17c8feaae4ff59182dafed4c1d3e0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:06.552585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765e418-22b5-4a22-90a0-e0cef9e22f6c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.553095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765e418-22b5-4a22-90a0-e0cef9e22f6c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.553516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765e418-22b5-4a22-90a0-e0cef9e22f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.583250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d83571-7f06-499c-afb4-c844d580a858 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.610474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d765e418-22b5-4a22-90a0-e0cef9e22f6c None None] MFA Rules not processed for user `b5dbd6ee072842dca3fc7ee0b40917e5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:06.612239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d765e418-22b5-4a22-90a0-e0cef9e22f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:06.706496 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2425/4823] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:06.706672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a23404-21e9-49dc-8b6c-16c172f44b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.706796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765e418-22b5-4a22-90a0-e0cef9e22f6c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:06.706888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a23404-21e9-49dc-8b6c-16c172f44b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.706888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a23404-21e9-49dc-8b6c-16c172f44b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.707039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d765e418-22b5-4a22-90a0-e0cef9e22f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:06.707246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a23404-21e9-49dc-8b6c-16c172f44b9c admin admin] PATH_INFO: `/v3/projects/6789f3b886e5405caa7ef105cbc32f29/users/d80fd4a265644d69a566edde911aefea/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.709864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a23404-21e9-49dc-8b6c-16c172f44b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6789f3b886e5405caa7ef105cbc32f29, user_id=d80fd4a265644d69a566edde911aefea, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.713555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a23404-21e9-49dc-8b6c-16c172f44b9c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:06.737290 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2399/4824] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:06 2022] PUT /identity/v3/projects/6789f3b886e5405caa7ef105cbc32f29/users/d80fd4a265644d69a566edde911aefea/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:06.738792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a23404-21e9-49dc-8b6c-16c172f44b9c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:06.754615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a1a23404-21e9-49dc-8b6c-16c172f44b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:06.884886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765e418-22b5-4a22-90a0-e0cef9e22f6c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.944712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a23404-21e9-49dc-8b6c-16c172f44b9c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:06.955051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582d2e4-7745-42b2-9069-17a91d450fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:06.973419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582d2e4-7745-42b2-9069-17a91d450fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:06.973743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582d2e4-7745-42b2-9069-17a91d450fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:06.974135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582d2e4-7745-42b2-9069-17a91d450fd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:06.975119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582d2e4-7745-42b2-9069-17a91d450fd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:06.980120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582d2e4-7745-42b2-9069-17a91d450fd5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:07.042936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841a006-a734-4619-a950-9e08d260f45a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:07.045352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841a006-a734-4619-a950-9e08d260f45a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:07.045649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841a006-a734-4619-a950-9e08d260f45a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:07.046267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841a006-a734-4619-a950-9e08d260f45a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:07.050924 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2400/4825] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:06 2022] POST /identity/v3/projects => generated 369 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:07.051109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841a006-a734-4619-a950-9e08d260f45a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:07.051200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0160c8-b4f5-4b50-881e-823606152207 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:07.051200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0160c8-b4f5-4b50-881e-823606152207 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:07.051200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0160c8-b4f5-4b50-881e-823606152207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:07.053005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841a006-a734-4619-a950-9e08d260f45a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:07.128552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:07.129161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fb0160c8-b4f5-4b50-881e-823606152207 None None] MFA Rules not processed for user `d80fd4a265644d69a566edde911aefea`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:07.130611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fb0160c8-b4f5-4b50-881e-823606152207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:07.171006 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2426/4826] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:06 2022] POST /identity/v3/projects => generated 353 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:07.171942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841a006-a734-4619-a950-9e08d260f45a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:07.217934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:07.364652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841a006-a734-4619-a950-9e08d260f45a tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:07.383823 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2401/4827] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:07.386231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0160c8-b4f5-4b50-881e-823606152207 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:07.392872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1c4d5-4f22-4aca-8acc-401a38d2c224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:07.394352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1c4d5-4f22-4aca-8acc-401a38d2c224 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:07.394864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1c4d5-4f22-4aca-8acc-401a38d2c224 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:07.395364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1c4d5-4f22-4aca-8acc-401a38d2c224 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:07.396107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1c4d5-4f22-4aca-8acc-401a38d2c224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:07.401151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1c4d5-4f22-4aca-8acc-401a38d2c224 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:07.442539 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2427/4828] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:07 2022] POST /identity/v3/users => generated 291 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:07.446334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1c4d5-4f22-4aca-8acc-401a38d2c224 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:07.479443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0160c8-b4f5-4b50-881e-823606152207 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:07.566675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503a96d-9222-405a-89a5-5e960dca1d1e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:07.567488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a96d-9222-405a-89a5-5e960dca1d1e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:07.567774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a96d-9222-405a-89a5-5e960dca1d1e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:07.568047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a96d-9222-405a-89a5-5e960dca1d1e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/3305e1cbc1784ad480afb635adcc9cbb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:07.569439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503a96d-9222-405a-89a5-5e960dca1d1e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:update_project(project_id=3305e1cbc1784ad480afb635adcc9cbb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:07.571688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503a96d-9222-405a-89a5-5e960dca1d1e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:07.607685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1c4d5-4f22-4aca-8acc-401a38d2c224 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:07.615400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a2f96-55b2-474d-ac15-66f2a49c1f34 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:07.616288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a2f96-55b2-474d-ac15-66f2a49c1f34 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:07.616726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a2f96-55b2-474d-ac15-66f2a49c1f34 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:07.617133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a2f96-55b2-474d-ac15-66f2a49c1f34 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:07.617854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a2f96-55b2-474d-ac15-66f2a49c1f34 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:07.619896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a2f96-55b2-474d-ac15-66f2a49c1f34 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:07.627932 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2402/4829] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Oct 15 21:10:07 2022] PATCH /identity/v3/projects/3305e1cbc1784ad480afb635adcc9cbb => generated 367 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:07.629014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503a96d-9222-405a-89a5-5e960dca1d1e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:07.644929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503a96d-9222-405a-89a5-5e960dca1d1e tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:07.654653 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2428/4830] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:07 2022] POST /identity/v3/services => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:07.654653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a2f96-55b2-474d-ac15-66f2a49c1f34 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:07.685796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13617f7-3938-480d-8447-fd919dff5b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:07.686487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13617f7-3938-480d-8447-fd919dff5b67 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:07.686785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13617f7-3938-480d-8447-fd919dff5b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:07.687001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13617f7-3938-480d-8447-fd919dff5b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:07.687800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13617f7-3938-480d-8447-fd919dff5b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:07.690033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13617f7-3938-480d-8447-fd919dff5b67 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:07.719384 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2403/4831] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:07.719384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13617f7-3938-480d-8447-fd919dff5b67 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:07.740916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a2f96-55b2-474d-ac15-66f2a49c1f34 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:07.752851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771f87e-1592-4b41-9b5f-b067e14c8f79 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:07.753792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771f87e-1592-4b41-9b5f-b067e14c8f79 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:07.754083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771f87e-1592-4b41-9b5f-b067e14c8f79 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:07.755516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771f87e-1592-4b41-9b5f-b067e14c8f79 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/3305e1cbc1784ad480afb635adcc9cbb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:07.762498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13617f7-3938-480d-8447-fd919dff5b67 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:07.768162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844521cb-0fce-4eec-9c34-5a8baeac80f0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:07.768928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844521cb-0fce-4eec-9c34-5a8baeac80f0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:07.769352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844521cb-0fce-4eec-9c34-5a8baeac80f0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:07.769722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844521cb-0fce-4eec-9c34-5a8baeac80f0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:07.770535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844521cb-0fce-4eec-9c34-5a8baeac80f0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:07.772359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844521cb-0fce-4eec-9c34-5a8baeac80f0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:07.776171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771f87e-1592-4b41-9b5f-b067e14c8f79 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:get_project(project_id=3305e1cbc1784ad480afb635adcc9cbb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:07.781704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771f87e-1592-4b41-9b5f-b067e14c8f79 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:07.784403 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2429/4832] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:10:07 2022] GET /identity/v3/projects/3305e1cbc1784ad480afb635adcc9cbb => generated 354 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:07.785412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771f87e-1592-4b41-9b5f-b067e14c8f79 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:07.794364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:07.802621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771f87e-1592-4b41-9b5f-b067e14c8f79 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:07.808769 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-844521cb-0fce-4eec-9c34-5a8baeac80f0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1758111011. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:10:07.808769 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:10:07.808769 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:07.809673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360d875-4b6a-4c3b-a6b1-28e728a9e197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:07.810632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360d875-4b6a-4c3b-a6b1-28e728a9e197 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:07.811888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360d875-4b6a-4c3b-a6b1-28e728a9e197 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:07.812407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360d875-4b6a-4c3b-a6b1-28e728a9e197 admin admin] PATH_INFO: `/v3/projects/c55b6a20bb2642dfb1d2395efd19653a/users/cb7ac1180bd346dcbaa146a4f776da3f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:07.836689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360d875-4b6a-4c3b-a6b1-28e728a9e197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55b6a20bb2642dfb1d2395efd19653a, user_id=cb7ac1180bd346dcbaa146a4f776da3f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:07.840442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360d875-4b6a-4c3b-a6b1-28e728a9e197 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:07.851598 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2404/4833] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:10:07 2022] POST /identity/v3/endpoints => generated 361 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:07.853472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844521cb-0fce-4eec-9c34-5a8baeac80f0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:07.865345 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2430/4834] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:07 2022] PUT /identity/v3/projects/c55b6a20bb2642dfb1d2395efd19653a/users/cb7ac1180bd346dcbaa146a4f776da3f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:07.868559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360d875-4b6a-4c3b-a6b1-28e728a9e197 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:07.890120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844521cb-0fce-4eec-9c34-5a8baeac80f0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:07.936554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:07.976785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:07.977466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:07.978432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:07.978757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/projects/3305e1cbc1784ad480afb635adcc9cbb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:07.982511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360d875-4b6a-4c3b-a6b1-28e728a9e197 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:07.983800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3305e1cbc1784ad480afb635adcc9cbb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:07.986334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:08.002764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b5bfa-ac3e-49c4-826e-fd352d74d0ac tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:08.003641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b5bfa-ac3e-49c4-826e-fd352d74d0ac tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.004035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b5bfa-ac3e-49c4-826e-fd352d74d0ac tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.004432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b5bfa-ac3e-49c4-826e-fd352d74d0ac tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/regions/tempest-region-1758111011` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.005187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b5bfa-ac3e-49c4-826e-fd352d74d0ac tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1758111011)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:08.006399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b5bfa-ac3e-49c4-826e-fd352d74d0ac tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:08.013283 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2431/4835] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 21:10:07 2022] GET /identity/v3/regions/tempest-region-1758111011 => generated 180 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:08.014297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b5bfa-ac3e-49c4-826e-fd352d74d0ac tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:08.060683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3305e1cbc1784ad480afb635adcc9cbb', 'request_id': 'req-76af9822-a10c-42bf-9f78-80ac43610e00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.075628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3305e1cbc1784ad480afb635adcc9cbb', 'request_id': 'req-76af9822-a10c-42bf-9f78-80ac43610e00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.098311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3305e1cbc1784ad480afb635adcc9cbb', 'request_id': 'req-76af9822-a10c-42bf-9f78-80ac43610e00'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.112656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] The token cache is being invalidate because project 3305e1cbc1784ad480afb635adcc9cbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:08.112947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.113748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.115780 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2405/4836] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:07 2022] DELETE /identity/v3/projects/3305e1cbc1784ad480afb635adcc9cbb => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:08.116367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:08.120237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:08.141082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b5bfa-ac3e-49c4-826e-fd352d74d0ac tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:08.226056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af9822-a10c-42bf-9f78-80ac43610e00 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:08.227661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6786c8-7af0-49cf-a626-3b8ec561ed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:08.229083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6786c8-7af0-49cf-a626-3b8ec561ed49 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.229528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6786c8-7af0-49cf-a626-3b8ec561ed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.229919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6786c8-7af0-49cf-a626-3b8ec561ed49 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.230783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6786c8-7af0-49cf-a626-3b8ec561ed49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:08.232866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6786c8-7af0-49cf-a626-3b8ec561ed49 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:08.233481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e917e61-ed8c-48f7-9ef2-ee2354d73b5d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:08.234163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e917e61-ed8c-48f7-9ef2-ee2354d73b5d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.234572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e917e61-ed8c-48f7-9ef2-ee2354d73b5d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.234961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e917e61-ed8c-48f7-9ef2-ee2354d73b5d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.235777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e917e61-ed8c-48f7-9ef2-ee2354d73b5d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:08.236811 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-3e917e61-ed8c-48f7-9ef2-ee2354d73b5d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:08.236811 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:08.236811 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:08.237926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e917e61-ed8c-48f7-9ef2-ee2354d73b5d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:08.242145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:08.251662 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2432/4837] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:08.252597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6786c8-7af0-49cf-a626-3b8ec561ed49 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:08.260115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bc6786c8-7af0-49cf-a626-3b8ec561ed49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:08.267641 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2406/4838] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:08 2022] POST /identity/v3/services => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:08.269417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e917e61-ed8c-48f7-9ef2-ee2354d73b5d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:08.271375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3e917e61-ed8c-48f7-9ef2-ee2354d73b5d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:08.281701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6786c8-7af0-49cf-a626-3b8ec561ed49 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:08.300778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:08.301440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.301719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.301990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] PATH_INFO: `/v3/users/dd615ca74887465b8e21b7f4011fb53f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.303137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd615ca74887465b8e21b7f4011fb53f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:08.305173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:08.359360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd615ca74887465b8e21b7f4011fb53f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.362203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:08.364999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e917e61-ed8c-48f7-9ef2-ee2354d73b5d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:08.372260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953eebcc-2e70-4a48-a3dd-1d8cd6f25622 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:08.372889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eebcc-2e70-4a48-a3dd-1d8cd6f25622 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.373292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eebcc-2e70-4a48-a3dd-1d8cd6f25622 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.373654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eebcc-2e70-4a48-a3dd-1d8cd6f25622 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/domains/85152a07a2ef40d59e5ccc1783a27919` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.376994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd615ca74887465b8e21b7f4011fb53f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.381212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953eebcc-2e70-4a48-a3dd-1d8cd6f25622 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:10:08.386853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd615ca74887465b8e21b7f4011fb53f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-953eebcc-2e70-4a48-a3dd-1d8cd6f25622 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Could not find domain: 85152a07a2ef40d59e5ccc1783a27919.: keystone.exception.DomainNotFound: Could not find domain: 85152a07a2ef40d59e5ccc1783a27919. Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 85152a07a2ef40d59e5ccc1783a27919. Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:08.387698 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:08.389176 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 85152a07a2ef40d59e5ccc1783a27919. Oct 15 21:10:08.390542 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:08.393327 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2407/4839] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:10:08 2022] PATCH /identity/v3/domains/85152a07a2ef40d59e5ccc1783a27919 => generated 112 bytes in 125 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:10:08.394094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953eebcc-2e70-4a48-a3dd-1d8cd6f25622 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:08.395204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd615ca74887465b8e21b7f4011fb53f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.396246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd615ca74887465b8e21b7f4011fb53f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.396936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-953eebcc-2e70-4a48-a3dd-1d8cd6f25622 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:08.406975 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2433/4840] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:08 2022] DELETE /identity/v3/users/dd615ca74887465b8e21b7f4011fb53f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:08.406975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:08.409611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:08.427024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953eebcc-2e70-4a48-a3dd-1d8cd6f25622 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:08.512649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2baeb-0d85-4613-84be-0ece0c3879d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:08.513616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2baeb-0d85-4613-84be-0ece0c3879d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.513909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2baeb-0d85-4613-84be-0ece0c3879d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.514201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2baeb-0d85-4613-84be-0ece0c3879d0 admin admin] PATH_INFO: `/v3/projects/c55b6a20bb2642dfb1d2395efd19653a/users/cb7ac1180bd346dcbaa146a4f776da3f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.522060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2baeb-0d85-4613-84be-0ece0c3879d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55b6a20bb2642dfb1d2395efd19653a, user_id=cb7ac1180bd346dcbaa146a4f776da3f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:08.525218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2baeb-0d85-4613-84be-0ece0c3879d0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:08.535572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c83e3-0cf1-4aeb-add8-3537f5ec5034 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:08.542424 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2408/4841] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:08 2022] PUT /identity/v3/projects/c55b6a20bb2642dfb1d2395efd19653a/users/cb7ac1180bd346dcbaa146a4f776da3f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:08.543250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2baeb-0d85-4613-84be-0ece0c3879d0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:08.545594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-79f2baeb-0d85-4613-84be-0ece0c3879d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:08.637069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a5e9e-ecbf-4ef6-9209-23bd2aeca9b0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:08.637805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a5e9e-ecbf-4ef6-9209-23bd2aeca9b0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.638021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a5e9e-ecbf-4ef6-9209-23bd2aeca9b0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.638291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a5e9e-ecbf-4ef6-9209-23bd2aeca9b0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.638953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a5e9e-ecbf-4ef6-9209-23bd2aeca9b0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:08.640565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a5e9e-ecbf-4ef6-9209-23bd2aeca9b0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:08.664977 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-e59a5e9e-ecbf-4ef6-9209-23bd2aeca9b0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-836246187. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:10:08.664977 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:10:08.664977 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:08.669382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2baeb-0d85-4613-84be-0ece0c3879d0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:08.675242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaccca-2dac-4be3-b291-ecc0af19c9b9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af30b5fefb4e458ea4fbd7dfad725bbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7826315f559b47bf9cd280a40a0f576d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:08.676040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaccca-2dac-4be3-b291-ecc0af19c9b9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.676467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaccca-2dac-4be3-b291-ecc0af19c9b9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.676829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaccca-2dac-4be3-b291-ecc0af19c9b9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] PATH_INFO: `/v3/domains/254e1d4c97444d7a873e6804cc521967` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.695047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaccca-2dac-4be3-b291-ecc0af19c9b9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-dfdaccca-2dac-4be3-b291-ecc0af19c9b9 tempest-ProjectsTestJSON-618604353 tempest-ProjectsTestJSON-618604353-project-admin] Could not find domain: 254e1d4c97444d7a873e6804cc521967.: keystone.exception.DomainNotFound: Could not find domain: 254e1d4c97444d7a873e6804cc521967. Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 254e1d4c97444d7a873e6804cc521967. Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:08.702521 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:08.703811 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 254e1d4c97444d7a873e6804cc521967. Oct 15 21:10:08.705242 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:08.706257 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2409/4842] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:10:08 2022] PATCH /identity/v3/domains/254e1d4c97444d7a873e6804cc521967 => generated 112 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:10:08.707182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77ba19-62ce-4aaa-880a-f6ef8aee9db1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.715610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77ba19-62ce-4aaa-880a-f6ef8aee9db1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.716190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77ba19-62ce-4aaa-880a-f6ef8aee9db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.728316 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2434/4843] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:10:08 2022] POST /identity/v3/endpoints => generated 361 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:08.729433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a5e9e-ecbf-4ef6-9209-23bd2aeca9b0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:08.735221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e59a5e9e-ecbf-4ef6-9209-23bd2aeca9b0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:08.747556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7c77ba19-62ce-4aaa-880a-f6ef8aee9db1 None None] MFA Rules not processed for user `cb7ac1180bd346dcbaa146a4f776da3f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:08.748848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7c77ba19-62ce-4aaa-880a-f6ef8aee9db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:08.753488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:08.782236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:08.849972 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2410/4844] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:08 2022] POST /identity/v3/auth/tokens => generated 3945 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:08.851188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77ba19-62ce-4aaa-880a-f6ef8aee9db1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:08.856028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7c77ba19-62ce-4aaa-880a-f6ef8aee9db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:08.857729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a5e9e-ecbf-4ef6-9209-23bd2aeca9b0 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:08.866047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:08.867177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.867554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.867888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] PATH_INFO: `/v3/projects/96091c3dac03410dbb4f59faa1482781` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.869981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96091c3dac03410dbb4f59faa1482781)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:08.872348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:08.875935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77ba19-62ce-4aaa-880a-f6ef8aee9db1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:08.886076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:08.887033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:08.887451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:08.887725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] PATH_INFO: `/v3/users/54ab885de090462ebc55dde75589ba63` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:08.889316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ab885de090462ebc55dde75589ba63)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:08.891490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:08.943701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96091c3dac03410dbb4f59faa1482781', 'request_id': 'req-076d06c1-39c6-4794-b3bc-4094f25c99d0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.956065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96091c3dac03410dbb4f59faa1482781', 'request_id': 'req-076d06c1-39c6-4794-b3bc-4094f25c99d0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.967385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96091c3dac03410dbb4f59faa1482781', 'request_id': 'req-076d06c1-39c6-4794-b3bc-4094f25c99d0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.976988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ab885de090462ebc55dde75589ba63'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.983334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] The token cache is being invalidate because project 96091c3dac03410dbb4f59faa1482781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:08.983628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.985145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.985562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ab885de090462ebc55dde75589ba63'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:08.989461 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2435/4845] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:08 2022] DELETE /identity/v3/projects/96091c3dac03410dbb4f59faa1482781 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:08.989461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:08.990535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:09.003737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ab885de090462ebc55dde75589ba63'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:09.014382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ab885de090462ebc55dde75589ba63'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:09.015007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ab885de090462ebc55dde75589ba63'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:09.025084 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2411/4846] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:08 2022] DELETE /identity/v3/users/54ab885de090462ebc55dde75589ba63 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:09.027000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.028840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:09.122635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d06c1-39c6-4794-b3bc-4094f25c99d0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:09.149151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a692a9-88b6-4a75-85bb-d84e5482e157 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:09.157338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede3384-2b94-469e-ad41-40636af794f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:09.158533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede3384-2b94-469e-ad41-40636af794f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.159706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede3384-2b94-469e-ad41-40636af794f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.160259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede3384-2b94-469e-ad41-40636af794f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.163307 ncdv-master-1 devstack@keystone.service[99099]: WARNING oslo_log.versionutils [None req-9ede3384-2b94-469e-ad41-40636af794f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 15 21:10:09.163818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede3384-2b94-469e-ad41-40636af794f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:09.167698 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-9ede3384-2b94-469e-ad41-40636af794f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:09.167698 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:09.167698 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:09.170877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede3384-2b94-469e-ad41-40636af794f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:09.236900 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2412/4847] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:10:09 2022] POST /identity/v3/policies => generated 227 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:09.238783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468056-f341-443c-be4f-d2ae1a43d424 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.239338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468056-f341-443c-be4f-d2ae1a43d424 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.239612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468056-f341-443c-be4f-d2ae1a43d424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.253334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:09.272698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df25a9-cc0d-4efa-bf8d-1203838d04f7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:09.273434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df25a9-cc0d-4efa-bf8d-1203838d04f7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.273718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df25a9-cc0d-4efa-bf8d-1203838d04f7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.273986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df25a9-cc0d-4efa-bf8d-1203838d04f7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/regions/tempest-region-836246187` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.274897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df25a9-cc0d-4efa-bf8d-1203838d04f7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-836246187)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:09.276042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df25a9-cc0d-4efa-bf8d-1203838d04f7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:09.281734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:09.282829 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2436/4848] 10.222.0.16 () {58 vars in 1203 bytes} [Sat Oct 15 21:10:08 2022] GET /identity/v3/regions/tempest-region-836246187 => generated 178 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:09.285545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86c3eb-aab9-4d0e-984d-98e1c4a8f840 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.286678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86c3eb-aab9-4d0e-984d-98e1c4a8f840 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.286950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86c3eb-aab9-4d0e-984d-98e1c4a8f840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.294988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6e468056-f341-443c-be4f-d2ae1a43d424 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:09.298972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e468056-f341-443c-be4f-d2ae1a43d424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:09.318296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1a86c3eb-aab9-4d0e-984d-98e1c4a8f840 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:09.319857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1a86c3eb-aab9-4d0e-984d-98e1c4a8f840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:09.405113 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2413/4849] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:09 2022] POST /identity/v3/auth/tokens => generated 3871 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:09.405113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e468056-f341-443c-be4f-d2ae1a43d424 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.419148 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2437/4850] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:09 2022] POST /identity/v3/auth/tokens => generated 3871 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:09.422075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86c3eb-aab9-4d0e-984d-98e1c4a8f840 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.424090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e468056-f341-443c-be4f-d2ae1a43d424 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:09.426167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1a86c3eb-aab9-4d0e-984d-98e1c4a8f840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:09.439328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc765dc-62e4-40df-bd3a-e85d7c2abc3f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:09.439723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc765dc-62e4-40df-bd3a-e85d7c2abc3f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.440220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc765dc-62e4-40df-bd3a-e85d7c2abc3f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.440500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc765dc-62e4-40df-bd3a-e85d7c2abc3f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies/e08af7f77cc74c35959224d186e1a787` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.441915 ncdv-master-1 devstack@keystone.service[99099]: WARNING oslo_log.versionutils [None req-dfc765dc-62e4-40df-bd3a-e85d7c2abc3f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Deprecated: identity:update_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 15 21:10:09.442231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc765dc-62e4-40df-bd3a-e85d7c2abc3f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:update_policy(policy_id=e08af7f77cc74c35959224d186e1a787)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:09.443479 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-dfc765dc-62e4-40df-bd3a-e85d7c2abc3f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:09.443479 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:09.443479 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:09.444902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc765dc-62e4-40df-bd3a-e85d7c2abc3f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:09.457745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86c3eb-aab9-4d0e-984d-98e1c4a8f840 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:09.463346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbeb7f-17ae-4583-b97e-d832bf7f5bc3 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:09.463961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbeb7f-17ae-4583-b97e-d832bf7f5bc3 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.464238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbeb7f-17ae-4583-b97e-d832bf7f5bc3 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.464712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbeb7f-17ae-4583-b97e-d832bf7f5bc3 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.465518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbeb7f-17ae-4583-b97e-d832bf7f5bc3 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:09.467346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbeb7f-17ae-4583-b97e-d832bf7f5bc3 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:09.494025 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2414/4851] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Oct 15 21:10:09 2022] PATCH /identity/v3/policies/e08af7f77cc74c35959224d186e1a787 => generated 235 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:09.494025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cb9ab-06eb-41ee-8c87-befac04bcb0e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.494025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cb9ab-06eb-41ee-8c87-befac04bcb0e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.494762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cb9ab-06eb-41ee-8c87-befac04bcb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.502095 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-8bdbeb7f-17ae-4583-b97e-d832bf7f5bc3 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1160072763. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:10:09.502095 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:10:09.502095 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:09.539046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5e9cb9ab-06eb-41ee-8c87-befac04bcb0e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:09.543307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5e9cb9ab-06eb-41ee-8c87-befac04bcb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:09.550711 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2438/4852] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:10:09 2022] POST /identity/v3/endpoints => generated 360 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:09.559495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbeb7f-17ae-4583-b97e-d832bf7f5bc3 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.562609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8bdbeb7f-17ae-4583-b97e-d832bf7f5bc3 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:09.576067 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2415/4853] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:09 2022] POST /identity/v3/auth/tokens => generated 4055 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:09.576067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cb9ab-06eb-41ee-8c87-befac04bcb0e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.608392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cb9ab-06eb-41ee-8c87-befac04bcb0e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:09.620177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf446db-9e56-45b2-a028-dae35f7975f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:09.621452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf446db-9e56-45b2-a028-dae35f7975f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.622951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf446db-9e56-45b2-a028-dae35f7975f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.623294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf446db-9e56-45b2-a028-dae35f7975f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies/e08af7f77cc74c35959224d186e1a787` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.624099 ncdv-master-1 devstack@keystone.service[99099]: WARNING oslo_log.versionutils [None req-3bf446db-9e56-45b2-a028-dae35f7975f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Deprecated: identity:get_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 15 21:10:09.625023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf446db-9e56-45b2-a028-dae35f7975f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:get_policy(policy_id=e08af7f77cc74c35959224d186e1a787)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:09.628957 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-3bf446db-9e56-45b2-a028-dae35f7975f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_policy": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:09.628957 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:09.628957 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:09.631468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf446db-9e56-45b2-a028-dae35f7975f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:09.648109 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2416/4854] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:10:09 2022] GET /identity/v3/policies/e08af7f77cc74c35959224d186e1a787 => generated 235 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:09.648109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf446db-9e56-45b2-a028-dae35f7975f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.708606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf446db-9e56-45b2-a028-dae35f7975f4 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:09.723596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564c51a-6316-4d27-a93e-4d3f12f16485 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:09.724249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564c51a-6316-4d27-a93e-4d3f12f16485 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.724529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564c51a-6316-4d27-a93e-4d3f12f16485 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.724969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564c51a-6316-4d27-a93e-4d3f12f16485 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/regions/tempest-region-1160072763` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.725584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564c51a-6316-4d27-a93e-4d3f12f16485 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1160072763)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:09.726685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564c51a-6316-4d27-a93e-4d3f12f16485 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:09.730205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbeb7f-17ae-4583-b97e-d832bf7f5bc3 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:09.731228 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2417/4855] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 21:10:09 2022] GET /identity/v3/regions/tempest-region-1160072763 => generated 180 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:09.732720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564c51a-6316-4d27-a93e-4d3f12f16485 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.739260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbdd1b-4a5f-43d1-80ab-9abeee1b7061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:09.739887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbdd1b-4a5f-43d1-80ab-9abeee1b7061 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.740166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbdd1b-4a5f-43d1-80ab-9abeee1b7061 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.740433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbdd1b-4a5f-43d1-80ab-9abeee1b7061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.757360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564c51a-6316-4d27-a93e-4d3f12f16485 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:09.762970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbdd1b-4a5f-43d1-80ab-9abeee1b7061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:09.766661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbdd1b-4a5f-43d1-80ab-9abeee1b7061 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:09.771487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3eae7-6731-4ced-a33b-2060e34a43e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:09.772185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3eae7-6731-4ced-a33b-2060e34a43e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.772469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3eae7-6731-4ced-a33b-2060e34a43e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.772781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3eae7-6731-4ced-a33b-2060e34a43e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.773458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3eae7-6731-4ced-a33b-2060e34a43e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:09.773772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3eae7-6731-4ced-a33b-2060e34a43e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:09.777587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3eae7-6731-4ced-a33b-2060e34a43e8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:09.792849 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2439/4856] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:09 2022] GET /identity/v3/auth/tokens => generated 4055 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:09.794085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c704e-191f-442a-a1b1-a49d491b8b0e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.794358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c704e-191f-442a-a1b1-a49d491b8b0e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.794629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c704e-191f-442a-a1b1-a49d491b8b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.805303 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2418/4857] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:09.806528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3eae7-6731-4ced-a33b-2060e34a43e8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.826622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3eae7-6731-4ced-a33b-2060e34a43e8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:09.829567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-479c704e-191f-442a-a1b1-a49d491b8b0e None None] MFA Rules not processed for user `1aadd87b97c247cb95321664ba0c32a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:09.830874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-479c704e-191f-442a-a1b1-a49d491b8b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:09.837882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b319-dd37-41e3-b47a-bfeb63caea0e tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:09.838538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b319-dd37-41e3-b47a-bfeb63caea0e tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.838817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b319-dd37-41e3-b47a-bfeb63caea0e tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.839090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b319-dd37-41e3-b47a-bfeb63caea0e tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies/e08af7f77cc74c35959224d186e1a787` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.839941 ncdv-master-1 devstack@keystone.service[99099]: WARNING oslo_log.versionutils [None req-0c00b319-dd37-41e3-b47a-bfeb63caea0e tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 15 21:10:09.840508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00b319-dd37-41e3-b47a-bfeb63caea0e tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=e08af7f77cc74c35959224d186e1a787)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:09.841802 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-0c00b319-dd37-41e3-b47a-bfeb63caea0e tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:09.841802 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:09.841802 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:09.843045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00b319-dd37-41e3-b47a-bfeb63caea0e tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:09.886940 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2419/4858] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:09 2022] DELETE /identity/v3/policies/e08af7f77cc74c35959224d186e1a787 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:09.889701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b319-dd37-41e3-b47a-bfeb63caea0e tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.914625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b319-dd37-41e3-b47a-bfeb63caea0e tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:09.929683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7978e-37b6-4879-b0d0-8903e4d9da27 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:09.930402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7978e-37b6-4879-b0d0-8903e4d9da27 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:09.930950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7978e-37b6-4879-b0d0-8903e4d9da27 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:09.931552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7978e-37b6-4879-b0d0-8903e4d9da27 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:09.932539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7978e-37b6-4879-b0d0-8903e4d9da27 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:09.933327 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-88c7978e-37b6-4879-b0d0-8903e4d9da27 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:09.933327 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:09.933327 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:09.936182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7978e-37b6-4879-b0d0-8903e4d9da27 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:09.939814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:09.968121 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2420/4859] 10.222.0.16 () {58 vars in 1134 bytes} [Sat Oct 15 21:10:09 2022] GET /identity/v3/endpoints => generated 4396 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:09.969048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7978e-37b6-4879-b0d0-8903e4d9da27 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.984615 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2440/4860] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:09 2022] POST /identity/v3/auth/tokens => generated 4070 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:09.985526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c704e-191f-442a-a1b1-a49d491b8b0e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:09.996915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7978e-37b6-4879-b0d0-8903e4d9da27 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.014589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e69210-2de1-46ef-b36a-d191743f429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.015519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e69210-2de1-46ef-b36a-d191743f429f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.015519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e69210-2de1-46ef-b36a-d191743f429f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.015727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e69210-2de1-46ef-b36a-d191743f429f admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.016334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e69210-2de1-46ef-b36a-d191743f429f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.018246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e69210-2de1-46ef-b36a-d191743f429f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.023387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c704e-191f-442a-a1b1-a49d491b8b0e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.041902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a53d4f-f881-4ea1-8d03-bdc98b8b7f2d tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.042541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a53d4f-f881-4ea1-8d03-bdc98b8b7f2d tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.042816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a53d4f-f881-4ea1-8d03-bdc98b8b7f2d tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.043091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a53d4f-f881-4ea1-8d03-bdc98b8b7f2d tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.050006 ncdv-master-1 devstack@keystone.service[99100]: WARNING oslo_log.versionutils [None req-a2a53d4f-f881-4ea1-8d03-bdc98b8b7f2d tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 15 21:10:10.050615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a53d4f-f881-4ea1-8d03-bdc98b8b7f2d tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.052714 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-a2a53d4f-f881-4ea1-8d03-bdc98b8b7f2d tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:10.052714 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:10.052714 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:10.056272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a53d4f-f881-4ea1-8d03-bdc98b8b7f2d tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.091230 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2441/4861] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:10:09 2022] POST /identity/v3/policies => generated 230 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:10.092242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a53d4f-f881-4ea1-8d03-bdc98b8b7f2d tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.098342 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2421/4862] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:09 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:10.101257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e69210-2de1-46ef-b36a-d191743f429f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.114293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a53d4f-f881-4ea1-8d03-bdc98b8b7f2d tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.209959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:DELETE=1,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:10.214145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9315d-8bc6-4835-a9f2-3507618a98a9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.215119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9315d-8bc6-4835-a9f2-3507618a98a9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.215556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9315d-8bc6-4835-a9f2-3507618a98a9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.216439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9315d-8bc6-4835-a9f2-3507618a98a9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints/1b799503c1b14594a3f6be98c0f47ca1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.226061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9315d-8bc6-4835-a9f2-3507618a98a9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:get_endpoint(endpoint_id=1b799503c1b14594a3f6be98c0f47ca1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.231565 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-c7e9315d-8bc6-4835-a9f2-3507618a98a9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:10.231565 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:10.231565 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:10.234399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e69210-2de1-46ef-b36a-d191743f429f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.235438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9315d-8bc6-4835-a9f2-3507618a98a9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.244199 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2442/4863] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:10 2022] GET /identity/v3/endpoints/1b799503c1b14594a3f6be98c0f47ca1 => generated 360 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:10.246772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ed2c-5022-47c8-8096-3d4834957244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.247245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9315d-8bc6-4835-a9f2-3507618a98a9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.247560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ed2c-5022-47c8-8096-3d4834957244 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.248093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ed2c-5022-47c8-8096-3d4834957244 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.248459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ed2c-5022-47c8-8096-3d4834957244 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.249991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06ed2c-5022-47c8-8096-3d4834957244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.252269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06ed2c-5022-47c8-8096-3d4834957244 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.299680 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2422/4864] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:10 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:10.300691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ed2c-5022-47c8-8096-3d4834957244 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.328170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ed2c-5022-47c8-8096-3d4834957244 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.343342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2039cc91-f45a-4814-86b4-9942117098e2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.344100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039cc91-f45a-4814-86b4-9942117098e2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.344495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039cc91-f45a-4814-86b4-9942117098e2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.344867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039cc91-f45a-4814-86b4-9942117098e2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints/1b799503c1b14594a3f6be98c0f47ca1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.346196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039cc91-f45a-4814-86b4-9942117098e2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=1b799503c1b14594a3f6be98c0f47ca1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.347443 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-2039cc91-f45a-4814-86b4-9942117098e2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:10.347443 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:10.347443 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:10.348588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039cc91-f45a-4814-86b4-9942117098e2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.371252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2039cc91-f45a-4814-86b4-9942117098e2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '1b799503c1b14594a3f6be98c0f47ca1', 'request_id': 'req-2039cc91-f45a-4814-86b4-9942117098e2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:10.376490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9315d-8bc6-4835-a9f2-3507618a98a9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.383891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03100a98-fe77-4aed-87e8-edba513633ff tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.384433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2423/4865] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:10 2022] DELETE /identity/v3/endpoints/1b799503c1b14594a3f6be98c0f47ca1 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:10.384593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100a98-fe77-4aed-87e8-edba513633ff tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.384919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100a98-fe77-4aed-87e8-edba513633ff tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.385223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100a98-fe77-4aed-87e8-edba513633ff tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.385890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2039cc91-f45a-4814-86b4-9942117098e2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.386479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03100a98-fe77-4aed-87e8-edba513633ff tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.393853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03100a98-fe77-4aed-87e8-edba513633ff tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.415418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2039cc91-f45a-4814-86b4-9942117098e2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.421612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a0d6c-5354-4f58-a872-1ba8ef5fd4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.422522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0d6c-5354-4f58-a872-1ba8ef5fd4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.423000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0d6c-5354-4f58-a872-1ba8ef5fd4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.423373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0d6c-5354-4f58-a872-1ba8ef5fd4a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.424268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a0d6c-5354-4f58-a872-1ba8ef5fd4a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.426027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a0d6c-5354-4f58-a872-1ba8ef5fd4a0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.432572 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2443/4866] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:10:10 2022] POST /identity/v3/policies => generated 230 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:10.434527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03100a98-fe77-4aed-87e8-edba513633ff tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.442603 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2424/4867] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:10.442603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a0d6c-5354-4f58-a872-1ba8ef5fd4a0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.450715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03100a98-fe77-4aed-87e8-edba513633ff tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.471419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c205dba-4149-4a92-8335-a76387c6b3e8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.472303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c205dba-4149-4a92-8335-a76387c6b3e8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.472736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c205dba-4149-4a92-8335-a76387c6b3e8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.473255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c205dba-4149-4a92-8335-a76387c6b3e8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.474070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c205dba-4149-4a92-8335-a76387c6b3e8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.475302 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-6c205dba-4149-4a92-8335-a76387c6b3e8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:10.475302 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:10.475302 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:10.476505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c205dba-4149-4a92-8335-a76387c6b3e8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.482563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a0d6c-5354-4f58-a872-1ba8ef5fd4a0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.491207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec328b2c-dcd6-494a-9bf7-3913e77ae640 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.492378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec328b2c-dcd6-494a-9bf7-3913e77ae640 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.492958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec328b2c-dcd6-494a-9bf7-3913e77ae640 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.494192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec328b2c-dcd6-494a-9bf7-3913e77ae640 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.495603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec328b2c-dcd6-494a-9bf7-3913e77ae640 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.497717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec328b2c-dcd6-494a-9bf7-3913e77ae640 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.506094 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2444/4868] 10.222.0.16 () {58 vars in 1134 bytes} [Sat Oct 15 21:10:10 2022] GET /identity/v3/endpoints => generated 4049 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:10.507372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c205dba-4149-4a92-8335-a76387c6b3e8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.533203 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2425/4869] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:10:10 2022] POST /identity/v3/policies => generated 229 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:10.534843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec328b2c-dcd6-494a-9bf7-3913e77ae640 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.536823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ec328b2c-dcd6-494a-9bf7-3913e77ae640 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:10.538623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c205dba-4149-4a92-8335-a76387c6b3e8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.548171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fcb93-c2ad-4c56-a13b-1354b16f95bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.549162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fcb93-c2ad-4c56-a13b-1354b16f95bf admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.550100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fcb93-c2ad-4c56-a13b-1354b16f95bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.550595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fcb93-c2ad-4c56-a13b-1354b16f95bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.551570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fcb93-c2ad-4c56-a13b-1354b16f95bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.552564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec328b2c-dcd6-494a-9bf7-3913e77ae640 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.554976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fcb93-c2ad-4c56-a13b-1354b16f95bf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.560238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.560912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.561281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.561557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] PATH_INFO: `/v3/projects/0010fa073e44461a919b682d04f077da` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.563541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0010fa073e44461a919b682d04f077da)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.565663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.572010 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2445/4870] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:10.573605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fcb93-c2ad-4c56-a13b-1354b16f95bf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.593710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fcb93-c2ad-4c56-a13b-1354b16f95bf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.604365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1418e1-a9b6-4aac-a3dd-7c61e1f8ca10 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.605084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1418e1-a9b6-4aac-a3dd-7c61e1f8ca10 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.605368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1418e1-a9b6-4aac-a3dd-7c61e1f8ca10 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.605639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1418e1-a9b6-4aac-a3dd-7c61e1f8ca10 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints/1b799503c1b14594a3f6be98c0f47ca1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.611947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1418e1-a9b6-4aac-a3dd-7c61e1f8ca10 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-bb1418e1-a9b6-4aac-a3dd-7c61e1f8ca10 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Could not find endpoint: 1b799503c1b14594a3f6be98c0f47ca1.: keystone.exception.EndpointNotFound: Could not find endpoint: 1b799503c1b14594a3f6be98c0f47ca1. Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 236, in _get_endpoint Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return session.query(Endpoint).filter_by(id=endpoint_id).one() Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._iter().one() Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._only_one_row( Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exc.NoResultFound( Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:10.631280 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 228, in get_endpoint Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_endpoint(endpoint_id) Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 242, in get_endpoint Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_endpoint(session, endpoint_id).to_dict() Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 238, in _get_endpoint Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 1b799503c1b14594a3f6be98c0f47ca1. Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:10.632473 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 113, in delete Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_endpoint') Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:10:10.633538 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 230, in get_endpoint Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 1b799503c1b14594a3f6be98c0f47ca1. Oct 15 21:10:10.635239 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:10.637150 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2446/4871] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:10 2022] DELETE /identity/v3/endpoints/1b799503c1b14594a3f6be98c0f47ca1 => generated 114 bytes in 64 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:10:10.638129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1418e1-a9b6-4aac-a3dd-7c61e1f8ca10 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.654732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1418e1-a9b6-4aac-a3dd-7c61e1f8ca10 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.658057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0010fa073e44461a919b682d04f077da', 'request_id': 'req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:10.676767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0010fa073e44461a919b682d04f077da', 'request_id': 'req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:10.692218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0010fa073e44461a919b682d04f077da', 'request_id': 'req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:10.704356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:10.709095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] The token cache is being invalidate because project 0010fa073e44461a919b682d04f077da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:10.709941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:10.711476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:10.715109 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2426/4872] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:10 2022] DELETE /identity/v3/projects/0010fa073e44461a919b682d04f077da => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:10.717056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.719619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:10.771431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.774580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.775182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.775846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.778736 ncdv-master-1 devstack@keystone.service[99100]: WARNING oslo_log.versionutils [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Deprecated: identity:list_policies of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 15 21:10:10.780762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:list_policies()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.784008 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_policies": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:10.784008 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:10.784008 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:10.787647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.819696 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2447/4873] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Oct 15 21:10:10 2022] GET /identity/v3/policies => generated 765 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:10.819696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.824751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:10.837509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:10.845256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fd6e2-b33a-44c6-8e60-582a2b66c61e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.855100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b9e4f-4d46-4af7-b879-a62edcb45e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.855888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b9e4f-4d46-4af7-b879-a62edcb45e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.856572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b9e4f-4d46-4af7-b879-a62edcb45e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.856572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b9e4f-4d46-4af7-b879-a62edcb45e32 admin admin] PATH_INFO: `/v3/projects/cd6d218d2e75459eae3548b0a9ba3000/users/3d2974383cae4b5485ce72b2e24a8dc6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:10.874930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b9e4f-4d46-4af7-b879-a62edcb45e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd6d218d2e75459eae3548b0a9ba3000, user_id=3d2974383cae4b5485ce72b2e24a8dc6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:10.877906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b9e4f-4d46-4af7-b879-a62edcb45e32 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:10.899380 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2427/4874] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:10 2022] PUT /identity/v3/projects/cd6d218d2e75459eae3548b0a9ba3000/users/3d2974383cae4b5485ce72b2e24a8dc6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:10.908912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b9e4f-4d46-4af7-b879-a62edcb45e32 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:10.911231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0d7b9e4f-4d46-4af7-b879-a62edcb45e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:10.942004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ec69-02f4-441f-8e21-9658441c616f tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.991504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b9e4f-4d46-4af7-b879-a62edcb45e32 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:10.997332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:10.998069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:10.998760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:10.999519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] PATH_INFO: `/v3/users/af30b5fefb4e458ea4fbd7dfad725bbe` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.001308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=af30b5fefb4e458ea4fbd7dfad725bbe)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.004006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.025825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15091527-0346-4cea-835a-db0e504b8ec7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.027512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15091527-0346-4cea-835a-db0e504b8ec7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.028102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15091527-0346-4cea-835a-db0e504b8ec7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.028583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15091527-0346-4cea-835a-db0e504b8ec7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/regions/tempest-region-1160072763` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.029685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15091527-0346-4cea-835a-db0e504b8ec7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1160072763)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.030861 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-15091527-0346-4cea-835a-db0e504b8ec7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:11.030861 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:11.030861 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:11.033570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15091527-0346-4cea-835a-db0e504b8ec7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.068806 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2448/4875] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Oct 15 21:10:10 2022] DELETE /identity/v3/regions/tempest-region-1160072763 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:11.074209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15091527-0346-4cea-835a-db0e504b8ec7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.075908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-15091527-0346-4cea-835a-db0e504b8ec7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:11.091366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af30b5fefb4e458ea4fbd7dfad725bbe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:11.112711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af30b5fefb4e458ea4fbd7dfad725bbe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:11.131086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af30b5fefb4e458ea4fbd7dfad725bbe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:11.144283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af30b5fefb4e458ea4fbd7dfad725bbe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:11.145076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1192197e-7c8c-46a6-9ebf-5f4550653d8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af30b5fefb4e458ea4fbd7dfad725bbe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:11.161492 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2428/4876] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:10 2022] DELETE /identity/v3/users/af30b5fefb4e458ea4fbd7dfad725bbe => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:11.163420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26b43b-b8d9-4f9e-9b7b-881c567fc860 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.163721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26b43b-b8d9-4f9e-9b7b-881c567fc860 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.163997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26b43b-b8d9-4f9e-9b7b-881c567fc860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.174959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:11.189072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15091527-0346-4cea-835a-db0e504b8ec7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.196419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6e26b43b-b8d9-4f9e-9b7b-881c567fc860 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:11.197781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e26b43b-b8d9-4f9e-9b7b-881c567fc860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:11.271517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:11.301494 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2429/4877] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:11 2022] POST /identity/v3/auth/tokens => generated 3871 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:11.302856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26b43b-b8d9-4f9e-9b7b-881c567fc860 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.305070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e26b43b-b8d9-4f9e-9b7b-881c567fc860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:11.320359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.321314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.321845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.322257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies/00b0b81bad044d098bbf39cffac7056f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.324309 ncdv-master-1 devstack@keystone.service[99100]: WARNING oslo_log.versionutils [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 15 21:10:11.325745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=00b0b81bad044d098bbf39cffac7056f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.328049 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:11.328049 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:11.328049 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:11.329436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.362769 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2449/4878] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:11 2022] DELETE /identity/v3/policies/00b0b81bad044d098bbf39cffac7056f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:11.362769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.365374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:11.406851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26b43b-b8d9-4f9e-9b7b-881c567fc860 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.413625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089270e-700d-4a44-84c7-e9b34ea93df7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.414243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089270e-700d-4a44-84c7-e9b34ea93df7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.414516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089270e-700d-4a44-84c7-e9b34ea93df7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.414968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089270e-700d-4a44-84c7-e9b34ea93df7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.415612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089270e-700d-4a44-84c7-e9b34ea93df7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.418879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089270e-700d-4a44-84c7-e9b34ea93df7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.433171 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2430/4879] 10.222.0.16 () {58 vars in 1134 bytes} [Sat Oct 15 21:10:11 2022] GET /identity/v3/endpoints => generated 4049 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:11.433171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089270e-700d-4a44-84c7-e9b34ea93df7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.435439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5089270e-700d-4a44-84c7-e9b34ea93df7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:11.453208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089270e-700d-4a44-84c7-e9b34ea93df7 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.458994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a3e67-56eb-49cd-ae09-939b63c5c049 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.459910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a3e67-56eb-49cd-ae09-939b63c5c049 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.460615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a3e67-56eb-49cd-ae09-939b63c5c049 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.461253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a3e67-56eb-49cd-ae09-939b63c5c049 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies/24b85d192a0743778564399873ef1d6b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.463122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a3e67-56eb-49cd-ae09-939b63c5c049 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=24b85d192a0743778564399873ef1d6b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.465357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a3e67-56eb-49cd-ae09-939b63c5c049 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.490431 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2431/4880] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:11 2022] DELETE /identity/v3/policies/24b85d192a0743778564399873ef1d6b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:11.492000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a3e67-56eb-49cd-ae09-939b63c5c049 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.494533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09510082-5f46-41d8-bff8-8c3a1381367a tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.511999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa51384-a218-486e-9619-0622c41d5e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.513672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa51384-a218-486e-9619-0622c41d5e93 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.513943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa51384-a218-486e-9619-0622c41d5e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.514309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa51384-a218-486e-9619-0622c41d5e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.526672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa51384-a218-486e-9619-0622c41d5e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.528274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a3e67-56eb-49cd-ae09-939b63c5c049 tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.528690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa51384-a218-486e-9619-0622c41d5e93 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.541750 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2450/4881] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:11 2022] GET /identity/v3/auth/tokens => generated 3871 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:11.542601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab783164-c492-441f-8b59-63054d76bd39 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.542836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa51384-a218-486e-9619-0622c41d5e93 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.543573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab783164-c492-441f-8b59-63054d76bd39 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.544313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab783164-c492-441f-8b59-63054d76bd39 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.544742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab783164-c492-441f-8b59-63054d76bd39 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.545568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab783164-c492-441f-8b59-63054d76bd39 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Adding query filter params (service_id=d7fedc92953840ddbb9418e34ce791b2) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:11.546028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab783164-c492-441f-8b59-63054d76bd39 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.550262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab783164-c492-441f-8b59-63054d76bd39 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.555693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa51384-a218-486e-9619-0622c41d5e93 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.579151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fb93f-d87f-455e-ad9c-24a536dce5ce tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb7ac1180bd346dcbaa146a4f776da3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c55b6a20bb2642dfb1d2395efd19653a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.580068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fb93f-d87f-455e-ad9c-24a536dce5ce tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.580453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fb93f-d87f-455e-ad9c-24a536dce5ce tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.580824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fb93f-d87f-455e-ad9c-24a536dce5ce tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] PATH_INFO: `/v3/policies/59086239b6d04dfb98b7a8bea3a94344` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.582584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fb93f-d87f-455e-ad9c-24a536dce5ce tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=59086239b6d04dfb98b7a8bea3a94344)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.584551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fb93f-d87f-455e-ad9c-24a536dce5ce tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.586937 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2432/4882] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:10:11 2022] GET /identity/v3/endpoints?service_id=d7fedc92953840ddbb9418e34ce791b2 => generated 503 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:11.594435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab783164-c492-441f-8b59-63054d76bd39 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.614967 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2451/4883] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:11 2022] DELETE /identity/v3/policies/59086239b6d04dfb98b7a8bea3a94344 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:11.618697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab783164-c492-441f-8b59-63054d76bd39 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.621522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fb93f-d87f-455e-ad9c-24a536dce5ce tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.624362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-705fb93f-d87f-455e-ad9c-24a536dce5ce tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:11.640011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2e7fa-898b-4a61-ab11-eb9009393a22 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.641686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2e7fa-898b-4a61-ab11-eb9009393a22 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.651505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fb93f-d87f-455e-ad9c-24a536dce5ce tempest-PoliciesTestJSON-1713628254 tempest-PoliciesTestJSON-1713628254-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.656540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2e7fa-898b-4a61-ab11-eb9009393a22 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.657915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.658560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.658901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.659239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] PATH_INFO: `/v3/users/b5dbd6ee072842dca3fc7ee0b40917e5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.659503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2e7fa-898b-4a61-ab11-eb9009393a22 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.661760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5dbd6ee072842dca3fc7ee0b40917e5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.662066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2e7fa-898b-4a61-ab11-eb9009393a22 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Adding query filter params (service_id=c597dac8263e441cbdead2de79350a8d) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:11.662386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2e7fa-898b-4a61-ab11-eb9009393a22 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.665998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.667686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2e7fa-898b-4a61-ab11-eb9009393a22 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.679782 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2433/4884] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:10:11 2022] GET /identity/v3/endpoints?service_id=c597dac8263e441cbdead2de79350a8d => generated 503 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:11.681839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2e7fa-898b-4a61-ab11-eb9009393a22 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.698843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2e7fa-898b-4a61-ab11-eb9009393a22 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.706542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c533-2df5-4308-a0f5-d61f0e1ee7ce tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.707617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c533-2df5-4308-a0f5-d61f0e1ee7ce tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.708565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c533-2df5-4308-a0f5-d61f0e1ee7ce tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.708565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c533-2df5-4308-a0f5-d61f0e1ee7ce tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.708992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c533-2df5-4308-a0f5-d61f0e1ee7ce tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Adding query filter params (interface=public) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:11.709686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c533-2df5-4308-a0f5-d61f0e1ee7ce tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.711907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c533-2df5-4308-a0f5-d61f0e1ee7ce tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.726285 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2434/4885] 10.222.0.16 () {58 vars in 1167 bytes} [Sat Oct 15 21:10:11 2022] GET /identity/v3/endpoints?interface=public => generated 3406 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:11.732901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c533-2df5-4308-a0f5-d61f0e1ee7ce tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.753027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c533-2df5-4308-a0f5-d61f0e1ee7ce tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.759495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5dbd6ee072842dca3fc7ee0b40917e5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:11.762880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ec017-3907-434f-b729-cd78df2bbc21 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.763501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ec017-3907-434f-b729-cd78df2bbc21 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.763794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ec017-3907-434f-b729-cd78df2bbc21 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.764196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ec017-3907-434f-b729-cd78df2bbc21 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.764741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ec017-3907-434f-b729-cd78df2bbc21 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Adding query filter params (interface=internal) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:11.765204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ec017-3907-434f-b729-cd78df2bbc21 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.771816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ec017-3907-434f-b729-cd78df2bbc21 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.785148 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2435/4886] 10.222.0.16 () {58 vars in 1171 bytes} [Sat Oct 15 21:10:11 2022] GET /identity/v3/endpoints?interface=internal => generated 478 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:11.787969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ec017-3907-434f-b729-cd78df2bbc21 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.794655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5dbd6ee072842dca3fc7ee0b40917e5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:11.805314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ec017-3907-434f-b729-cd78df2bbc21 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.805431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:11.813881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5dbd6ee072842dca3fc7ee0b40917e5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:11.814417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664a360-6650-4e25-91b5-9cbca60f3123 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:11.815433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664a360-6650-4e25-91b5-9cbca60f3123 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.815837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664a360-6650-4e25-91b5-9cbca60f3123 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.816275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664a360-6650-4e25-91b5-9cbca60f3123 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.817509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664a360-6650-4e25-91b5-9cbca60f3123 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:11.819337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664a360-6650-4e25-91b5-9cbca60f3123 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:11.823841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5dbd6ee072842dca3fc7ee0b40917e5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:11.828462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a45b0d4-bce5-41a5-8e9b-a5d79278f242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5dbd6ee072842dca3fc7ee0b40917e5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:11.842118 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2452/4887] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:11 2022] DELETE /identity/v3/users/b5dbd6ee072842dca3fc7ee0b40917e5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:11.846237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55569be6-ba7e-4239-a20a-120565b3b083 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:11.846520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55569be6-ba7e-4239-a20a-120565b3b083 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:11.846786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55569be6-ba7e-4239-a20a-120565b3b083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:11.862533 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2436/4888] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:11 2022] POST /identity/v3/services => generated 287 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:11.865139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664a360-6650-4e25-91b5-9cbca60f3123 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:11.870741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1664a360-6650-4e25-91b5-9cbca60f3123 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:11.911627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664a360-6650-4e25-91b5-9cbca60f3123 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:11.912873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-55569be6-ba7e-4239-a20a-120565b3b083 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:11.914143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-55569be6-ba7e-4239-a20a-120565b3b083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:12.064935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa81e28-e88e-47cf-aabd-120696a966c1 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:12.065703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa81e28-e88e-47cf-aabd-120696a966c1 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:12.066012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa81e28-e88e-47cf-aabd-120696a966c1 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:12.066288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa81e28-e88e-47cf-aabd-120696a966c1 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:12.066991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa81e28-e88e-47cf-aabd-120696a966c1 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:12.068584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa81e28-e88e-47cf-aabd-120696a966c1 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:12.098552 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2453/4889] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:11 2022] POST /identity/v3/auth/tokens => generated 4002 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:10:12.101283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:12.109545 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-cfa81e28-e88e-47cf-aabd-120696a966c1 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-671526751. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:10:12.109545 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:10:12.109545 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:12.133235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55569be6-ba7e-4239-a20a-120565b3b083 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:12.136353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-55569be6-ba7e-4239-a20a-120565b3b083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:12.161433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2437/4890] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:10:11 2022] POST /identity/v3/endpoints => generated 358 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:12.161587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa81e28-e88e-47cf-aabd-120696a966c1 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:12.230146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa81e28-e88e-47cf-aabd-120696a966c1 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:12.272402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870e5a2-d056-4af9-b16c-55771ec1858f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:12.277469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870e5a2-d056-4af9-b16c-55771ec1858f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:12.286776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870e5a2-d056-4af9-b16c-55771ec1858f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:12.287106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870e5a2-d056-4af9-b16c-55771ec1858f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/regions/tempest-region-671526751` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:12.287745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870e5a2-d056-4af9-b16c-55771ec1858f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-671526751)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:12.288994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870e5a2-d056-4af9-b16c-55771ec1858f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:12.295731 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2438/4891] 10.222.0.16 () {58 vars in 1203 bytes} [Sat Oct 15 21:10:12 2022] GET /identity/v3/regions/tempest-region-671526751 => generated 178 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:12.297230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870e5a2-d056-4af9-b16c-55771ec1858f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:12.301622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55569be6-ba7e-4239-a20a-120565b3b083 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:12.308998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a870e5a2-d056-4af9-b16c-55771ec1858f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:12.326054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f71ee-2f9c-49af-aa4f-76e9d3c05812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:12.326913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f71ee-2f9c-49af-aa4f-76e9d3c05812 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:12.327429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f71ee-2f9c-49af-aa4f-76e9d3c05812 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:12.327889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f71ee-2f9c-49af-aa4f-76e9d3c05812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:12.344061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f71ee-2f9c-49af-aa4f-76e9d3c05812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:12.347957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f71ee-2f9c-49af-aa4f-76e9d3c05812 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:12.380811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870e5a2-d056-4af9-b16c-55771ec1858f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:12.386524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:12.387159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:12.387439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:12.387757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] PATH_INFO: `/v3/projects/7826315f559b47bf9cd280a40a0f576d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:12.389296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7826315f559b47bf9cd280a40a0f576d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:12.391727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:12.403261 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2454/4892] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:12 2022] GET /identity/v3/auth/tokens => generated 4184 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:12.404121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f71ee-2f9c-49af-aa4f-76e9d3c05812 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:12.428757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f71ee-2f9c-49af-aa4f-76e9d3c05812 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:12.441210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:12.441849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:12.442390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:12.442675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints/5ca835d60ccb402a9f73cd2808a0b114` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:12.456016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=5ca835d60ccb402a9f73cd2808a0b114)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:12.456934 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:12.456934 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:12.456934 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:12.458585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:12.479848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7826315f559b47bf9cd280a40a0f576d', 'request_id': 'req-da64933a-af22-43d8-ad6d-5881052541b8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:12.496941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7826315f559b47bf9cd280a40a0f576d', 'request_id': 'req-da64933a-af22-43d8-ad6d-5881052541b8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:12.497472 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-776739926. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:10:12.497472 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:10:12.497472 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:12.515058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7826315f559b47bf9cd280a40a0f576d', 'request_id': 'req-da64933a-af22-43d8-ad6d-5881052541b8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:12.544238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] The token cache is being invalidate because project 7826315f559b47bf9cd280a40a0f576d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:12.544873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:12.547186 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2455/4893] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:10:12 2022] PATCH /identity/v3/endpoints/5ca835d60ccb402a9f73cd2808a0b114 => generated 362 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:12.547667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-da64933a-af22-43d8-ad6d-5881052541b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:12.548070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:12.550699 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2439/4894] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:12 2022] DELETE /identity/v3/projects/7826315f559b47bf9cd280a40a0f576d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:12.550924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:12.555555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7d308-8647-42ab-a82d-24b80bcfdd06 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:12.555992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7d308-8647-42ab-a82d-24b80bcfdd06 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:12.556350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7d308-8647-42ab-a82d-24b80bcfdd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:12.578982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:12.614311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cff33-db93-46ff-b642-c77abfa0cf2f tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:12.629729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-84b7d308-8647-42ab-a82d-24b80bcfdd06 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:12.631114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-84b7d308-8647-42ab-a82d-24b80bcfdd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:12.817602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3696bf9-5d03-4ce7-97ce-cedd9a71d5f8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:12.820044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3696bf9-5d03-4ce7-97ce-cedd9a71d5f8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:12.820269 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2440/4895] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:12 2022] POST /identity/v3/auth/tokens => generated 4002 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:10:12.821219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3696bf9-5d03-4ce7-97ce-cedd9a71d5f8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:12.822467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3696bf9-5d03-4ce7-97ce-cedd9a71d5f8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/regions/tempest-region-776739926` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:12.823230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3696bf9-5d03-4ce7-97ce-cedd9a71d5f8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-776739926)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:12.825221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3696bf9-5d03-4ce7-97ce-cedd9a71d5f8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:12.828451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b73918-760e-448f-a24f-29edf11f9e10 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:12.829083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b73918-760e-448f-a24f-29edf11f9e10 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:12.829588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b73918-760e-448f-a24f-29edf11f9e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:12.848705 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2456/4896] 10.222.0.16 () {58 vars in 1203 bytes} [Sat Oct 15 21:10:12 2022] GET /identity/v3/regions/tempest-region-776739926 => generated 178 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:12.852823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3696bf9-5d03-4ce7-97ce-cedd9a71d5f8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:12.880316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-57b73918-760e-448f-a24f-29edf11f9e10 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:12.880781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3696bf9-5d03-4ce7-97ce-cedd9a71d5f8 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:12.882445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-57b73918-760e-448f-a24f-29edf11f9e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:12.891975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf979ef-5d6b-4b6f-8d3e-500ff5d11229 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:12.892873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf979ef-5d6b-4b6f-8d3e-500ff5d11229 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:12.893175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf979ef-5d6b-4b6f-8d3e-500ff5d11229 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:12.893441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf979ef-5d6b-4b6f-8d3e-500ff5d11229 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints/5ca835d60ccb402a9f73cd2808a0b114` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:12.897086 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2441/4897] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:12 2022] POST /identity/v3/auth/tokens => generated 4002 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:12.900772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b73918-760e-448f-a24f-29edf11f9e10 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:12.907553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf979ef-5d6b-4b6f-8d3e-500ff5d11229 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=5ca835d60ccb402a9f73cd2808a0b114)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:12.909223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf979ef-5d6b-4b6f-8d3e-500ff5d11229 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:12.926419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b73918-760e-448f-a24f-29edf11f9e10 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:12.933879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2bf979ef-5d6b-4b6f-8d3e-500ff5d11229 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '5ca835d60ccb402a9f73cd2808a0b114', 'request_id': 'req-2bf979ef-5d6b-4b6f-8d3e-500ff5d11229'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:12.946474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446644d5-d86b-43d9-ab2d-e353c93d1ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:12.949324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446644d5-d86b-43d9-ab2d-e353c93d1ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:12.951263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446644d5-d86b-43d9-ab2d-e353c93d1ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:12.951793 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2457/4898] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:12 2022] DELETE /identity/v3/endpoints/5ca835d60ccb402a9f73cd2808a0b114 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:12.952316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446644d5-d86b-43d9-ab2d-e353c93d1ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:12.953372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446644d5-d86b-43d9-ab2d-e353c93d1ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:12.953874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446644d5-d86b-43d9-ab2d-e353c93d1ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:12.956525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf979ef-5d6b-4b6f-8d3e-500ff5d11229 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:12.958478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446644d5-d86b-43d9-ab2d-e353c93d1ea5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:12.998587 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2442/4899] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:13.003720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446644d5-d86b-43d9-ab2d-e353c93d1ea5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.016226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf979ef-5d6b-4b6f-8d3e-500ff5d11229 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.025823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446644d5-d86b-43d9-ab2d-e353c93d1ea5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.028688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149c8a9-89b4-4ab1-a66d-18c479a47a9c tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:13.030061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149c8a9-89b4-4ab1-a66d-18c479a47a9c tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:13.030934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149c8a9-89b4-4ab1-a66d-18c479a47a9c tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:13.031923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149c8a9-89b4-4ab1-a66d-18c479a47a9c tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/regions/tempest-region-776739926` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:13.037653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149c8a9-89b4-4ab1-a66d-18c479a47a9c tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-776739926)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:13.043277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149c8a9-89b4-4ab1-a66d-18c479a47a9c tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:13.083352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:13.090077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3224098-9ceb-49b6-a641-ecb856a86ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:13.117814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3224098-9ceb-49b6-a641-ecb856a86ceb admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:13.126917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3224098-9ceb-49b6-a641-ecb856a86ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:13.127447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3224098-9ceb-49b6-a641-ecb856a86ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:13.128308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3224098-9ceb-49b6-a641-ecb856a86ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:13.130063 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2458/4900] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 21:10:12 2022] DELETE /identity/v3/regions/tempest-region-776739926 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:13.132654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3224098-9ceb-49b6-a641-ecb856a86ceb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:13.135457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149c8a9-89b4-4ab1-a66d-18c479a47a9c tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.185179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149c8a9-89b4-4ab1-a66d-18c479a47a9c tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.194975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2834a-202f-4080-98c4-9065988bb720 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:13.195812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2834a-202f-4080-98c4-9065988bb720 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:13.195939 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2443/4901] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:13 2022] POST /identity/v3/projects => generated 397 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:10:13.196780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2834a-202f-4080-98c4-9065988bb720 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:13.197372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2834a-202f-4080-98c4-9065988bb720 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/regions/tempest-region-671526751` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:13.198137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3224098-9ceb-49b6-a641-ecb856a86ceb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.200667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2834a-202f-4080-98c4-9065988bb720 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-671526751)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:13.204727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2834a-202f-4080-98c4-9065988bb720 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:13.251304 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-b5d2834a-202f-4080-98c4-9065988bb720 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-671526751. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:10:13.251304 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:10:13.251304 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:13.268693 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2459/4902] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 21:10:13 2022] DELETE /identity/v3/regions/tempest-region-671526751 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:13.280138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2834a-202f-4080-98c4-9065988bb720 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.337484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2834a-202f-4080-98c4-9065988bb720 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.448622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3224098-9ceb-49b6-a641-ecb856a86ceb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.466392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29295c7-efb5-40ff-94d1-34720eb16d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:13.467054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29295c7-efb5-40ff-94d1-34720eb16d69 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:13.467334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29295c7-efb5-40ff-94d1-34720eb16d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:13.467651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29295c7-efb5-40ff-94d1-34720eb16d69 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:13.468376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29295c7-efb5-40ff-94d1-34720eb16d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:13.470559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29295c7-efb5-40ff-94d1-34720eb16d69 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:13.481446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21884eb2-9ed4-4ef4-a007-60f694385f5a tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:13.482395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884eb2-9ed4-4ef4-a007-60f694385f5a tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:13.482395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884eb2-9ed4-4ef4-a007-60f694385f5a tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:13.482653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884eb2-9ed4-4ef4-a007-60f694385f5a tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/services/d05566cf9b63443ea980b85bd7ce5597` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:13.483241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21884eb2-9ed4-4ef4-a007-60f694385f5a tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_service(service_id=d05566cf9b63443ea980b85bd7ce5597)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:13.484854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21884eb2-9ed4-4ef4-a007-60f694385f5a tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:13.505954 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2444/4903] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:13 2022] POST /identity/v3/users => generated 299 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:13.509671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:13.512153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29295c7-efb5-40ff-94d1-34720eb16d69 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.528775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29295c7-efb5-40ff-94d1-34720eb16d69 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.536267 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2460/4904] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:13 2022] DELETE /identity/v3/services/d05566cf9b63443ea980b85bd7ce5597 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:13.541419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21884eb2-9ed4-4ef4-a007-60f694385f5a tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.576105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a6ee6-c416-470f-abd9-756b8e815dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:13.576796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a6ee6-c416-470f-abd9-756b8e815dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:13.577224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a6ee6-c416-470f-abd9-756b8e815dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:13.577590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a6ee6-c416-470f-abd9-756b8e815dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:13.578953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a6ee6-c416-470f-abd9-756b8e815dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:13.580655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a6ee6-c416-470f-abd9-756b8e815dc2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:13.586293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21884eb2-9ed4-4ef4-a007-60f694385f5a tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.593181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e84752-1dbf-417c-8dc5-97a61f6a1ef2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:13.594068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e84752-1dbf-417c-8dc5-97a61f6a1ef2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:13.605676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e84752-1dbf-417c-8dc5-97a61f6a1ef2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:13.606200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e84752-1dbf-417c-8dc5-97a61f6a1ef2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints/37484e72858c4ee6bd5652e20eeee25d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:13.621624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e84752-1dbf-417c-8dc5-97a61f6a1ef2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=37484e72858c4ee6bd5652e20eeee25d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:13.624496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e84752-1dbf-417c-8dc5-97a61f6a1ef2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:13.625001 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2445/4905] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:13.626402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a6ee6-c416-470f-abd9-756b8e815dc2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.640294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a6ee6-c416-470f-abd9-756b8e815dc2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.642997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f6e84752-1dbf-417c-8dc5-97a61f6a1ef2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '37484e72858c4ee6bd5652e20eeee25d', 'request_id': 'req-f6e84752-1dbf-417c-8dc5-97a61f6a1ef2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:13.659657 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2461/4906] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:13 2022] DELETE /identity/v3/endpoints/37484e72858c4ee6bd5652e20eeee25d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:13.663311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e84752-1dbf-417c-8dc5-97a61f6a1ef2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.684263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90166a-ebe0-4074-98ff-5fee365d1ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:13.685791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90166a-ebe0-4074-98ff-5fee365d1ffa admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:13.686132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90166a-ebe0-4074-98ff-5fee365d1ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:13.686469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90166a-ebe0-4074-98ff-5fee365d1ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:13.688092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90166a-ebe0-4074-98ff-5fee365d1ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:13.689833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90166a-ebe0-4074-98ff-5fee365d1ffa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:13.708118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e84752-1dbf-417c-8dc5-97a61f6a1ef2 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.713076 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2446/4907] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:13.717457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90166a-ebe0-4074-98ff-5fee365d1ffa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.719571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779d12d-4c49-4303-9129-2f22a52f6d3d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:13.722311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779d12d-4c49-4303-9129-2f22a52f6d3d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:13.722666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779d12d-4c49-4303-9129-2f22a52f6d3d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:13.723125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779d12d-4c49-4303-9129-2f22a52f6d3d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/regions/tempest-region-836246187` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:13.723789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779d12d-4c49-4303-9129-2f22a52f6d3d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-836246187)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:13.729027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779d12d-4c49-4303-9129-2f22a52f6d3d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:13.743830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90166a-ebe0-4074-98ff-5fee365d1ffa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.753202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b214d34-3b23-43ba-83db-05218f06912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:13.756050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b214d34-3b23-43ba-83db-05218f06912d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:13.757218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b214d34-3b23-43ba-83db-05218f06912d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:13.759907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b214d34-3b23-43ba-83db-05218f06912d admin admin] PATH_INFO: `/v3/projects/0fd36b06a1a34e2d947594ec96d96a6c/users/b217b13af4af41639cd058d438ce2988/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:13.772438 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2462/4908] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 21:10:13 2022] DELETE /identity/v3/regions/tempest-region-836246187 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:13.799323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779d12d-4c49-4303-9129-2f22a52f6d3d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.859478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b214d34-3b23-43ba-83db-05218f06912d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd36b06a1a34e2d947594ec96d96a6c, user_id=b217b13af4af41639cd058d438ce2988, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:13.870976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b214d34-3b23-43ba-83db-05218f06912d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:13.892747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779d12d-4c49-4303-9129-2f22a52f6d3d tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:13.902569 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2447/4909] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:13 2022] PUT /identity/v3/projects/0fd36b06a1a34e2d947594ec96d96a6c/users/b217b13af4af41639cd058d438ce2988/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:13.903664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b214d34-3b23-43ba-83db-05218f06912d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:13.909644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5b214d34-3b23-43ba-83db-05218f06912d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:14.018745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75284f9-1ca6-4c6e-b62d-e8b731d6e1ef tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:14.019416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75284f9-1ca6-4c6e-b62d-e8b731d6e1ef tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.019715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75284f9-1ca6-4c6e-b62d-e8b731d6e1ef tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.019985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75284f9-1ca6-4c6e-b62d-e8b731d6e1ef tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/services/c597dac8263e441cbdead2de79350a8d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.020594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75284f9-1ca6-4c6e-b62d-e8b731d6e1ef tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_service(service_id=c597dac8263e441cbdead2de79350a8d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:14.022234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75284f9-1ca6-4c6e-b62d-e8b731d6e1ef tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:14.047987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b214d34-3b23-43ba-83db-05218f06912d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:14.056527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:14.057570 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2463/4910] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:13 2022] DELETE /identity/v3/services/c597dac8263e441cbdead2de79350a8d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:14.059688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f509a-c91b-4b14-abf6-0f4b15c1fd32 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.060150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f509a-c91b-4b14-abf6-0f4b15c1fd32 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.060637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f509a-c91b-4b14-abf6-0f4b15c1fd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.070835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:14.072626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.073104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.073481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] PATH_INFO: `/v3/projects/d258d5a5a3a74ce180e9d41ab4b81d71` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.075090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d258d5a5a3a74ce180e9d41ab4b81d71)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:14.077450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:14.095133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4b2f509a-c91b-4b14-abf6-0f4b15c1fd32 None None] MFA Rules not processed for user `b217b13af4af41639cd058d438ce2988`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:14.096591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4b2f509a-c91b-4b14-abf6-0f4b15c1fd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:14.154258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd258d5a5a3a74ce180e9d41ab4b81d71', 'request_id': 'req-eceabd10-ccdb-4097-966a-9638295daa98'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.154897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=7,keystone:SELECT=93 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:14.172596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd258d5a5a3a74ce180e9d41ab4b81d71', 'request_id': 'req-eceabd10-ccdb-4097-966a-9638295daa98'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.180197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd258d5a5a3a74ce180e9d41ab4b81d71', 'request_id': 'req-eceabd10-ccdb-4097-966a-9638295daa98'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.186452 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2464/4911] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3590 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:14.187556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f509a-c91b-4b14-abf6-0f4b15c1fd32 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:14.195875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] The token cache is being invalidate because project d258d5a5a3a74ce180e9d41ab4b81d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:14.197066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.198444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.200365 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2448/4912] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:13 2022] DELETE /identity/v3/projects/d258d5a5a3a74ce180e9d41ab4b81d71 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:14.201378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:14.203675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:14.238817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f509a-c91b-4b14-abf6-0f4b15c1fd32 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:14.315511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceabd10-ccdb-4097-966a-9638295daa98 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:14.341960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa870c2-e5b4-41c6-a313-6c6da0e0952e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:14.342509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa870c2-e5b4-41c6-a313-6c6da0e0952e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.342779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa870c2-e5b4-41c6-a313-6c6da0e0952e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.343042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa870c2-e5b4-41c6-a313-6c6da0e0952e admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.343696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa870c2-e5b4-41c6-a313-6c6da0e0952e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:14.345675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa870c2-e5b4-41c6-a313-6c6da0e0952e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:14.350992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590d95e-2658-4da0-988c-bd4c1a5d64e9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:14.351647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590d95e-2658-4da0-988c-bd4c1a5d64e9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.351928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590d95e-2658-4da0-988c-bd4c1a5d64e9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.352480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590d95e-2658-4da0-988c-bd4c1a5d64e9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/endpoints/e5c338b253c94050ac63168360c43deb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.359006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590d95e-2658-4da0-988c-bd4c1a5d64e9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=e5c338b253c94050ac63168360c43deb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:14.361163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590d95e-2658-4da0-988c-bd4c1a5d64e9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:14.374574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8590d95e-2658-4da0-988c-bd4c1a5d64e9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'e5c338b253c94050ac63168360c43deb', 'request_id': 'req-8590d95e-2658-4da0-988c-bd4c1a5d64e9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.380768 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2449/4913] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:14 2022] POST /identity/v3/projects => generated 397 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:14.381691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa870c2-e5b4-41c6-a313-6c6da0e0952e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:14.385615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1fa870c2-e5b4-41c6-a313-6c6da0e0952e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:14.401136 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2465/4914] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:14 2022] DELETE /identity/v3/endpoints/e5c338b253c94050ac63168360c43deb => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:14.403097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590d95e-2658-4da0-988c-bd4c1a5d64e9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:14.440574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590d95e-2658-4da0-988c-bd4c1a5d64e9 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:14.462916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:14.467986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa870c2-e5b4-41c6-a313-6c6da0e0952e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:14.476216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:14.477943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.478368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.478659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] PATH_INFO: `/v3/users/cb7ac1180bd346dcbaa146a4f776da3f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.480834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb7ac1180bd346dcbaa146a4f776da3f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:14.485312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:14.538394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a01e0d-e385-4057-a28c-de1b022c8913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:14.539181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a01e0d-e385-4057-a28c-de1b022c8913 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.539519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a01e0d-e385-4057-a28c-de1b022c8913 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.539804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a01e0d-e385-4057-a28c-de1b022c8913 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.540765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a01e0d-e385-4057-a28c-de1b022c8913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:14.543278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a01e0d-e385-4057-a28c-de1b022c8913 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:14.558872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb7ac1180bd346dcbaa146a4f776da3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.572783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb7ac1180bd346dcbaa146a4f776da3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.582054 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2466/4915] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:14 2022] POST /identity/v3/users => generated 305 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:14.582054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a01e0d-e385-4057-a28c-de1b022c8913 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:14.583936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c9a01e0d-e385-4057-a28c-de1b022c8913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:14.594346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb7ac1180bd346dcbaa146a4f776da3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.606304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb7ac1180bd346dcbaa146a4f776da3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.607543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb7ac1180bd346dcbaa146a4f776da3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:14.618989 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2450/4916] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:14 2022] DELETE /identity/v3/users/cb7ac1180bd346dcbaa146a4f776da3f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:14.620343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:14.622163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:14.657380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:14.716237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a01e0d-e385-4057-a28c-de1b022c8913 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:14.729883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cc778-c22c-4564-9a8b-64be93649a94 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:14.740682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7265de-f0ea-44ab-9bd1-6d5be8d7de11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:14.741655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7265de-f0ea-44ab-9bd1-6d5be8d7de11 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.743398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7265de-f0ea-44ab-9bd1-6d5be8d7de11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.743859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7265de-f0ea-44ab-9bd1-6d5be8d7de11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.744767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7265de-f0ea-44ab-9bd1-6d5be8d7de11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:14.747396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7265de-f0ea-44ab-9bd1-6d5be8d7de11 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:14.780481 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2451/4917] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:14.781869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7265de-f0ea-44ab-9bd1-6d5be8d7de11 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:14.790000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c1ff3-5722-40f2-a4c3-bd1b5c91ef26 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:14.790766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c1ff3-5722-40f2-a4c3-bd1b5c91ef26 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.791013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c1ff3-5722-40f2-a4c3-bd1b5c91ef26 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.791285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c1ff3-5722-40f2-a4c3-bd1b5c91ef26 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/regions/tempest-region-1758111011` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.791866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c1ff3-5722-40f2-a4c3-bd1b5c91ef26 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1758111011)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:14.793516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c1ff3-5722-40f2-a4c3-bd1b5c91ef26 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:14.810318 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-ed0c1ff3-5722-40f2-a4c3-bd1b5c91ef26 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1758111011. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:10:14.810318 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:10:14.810318 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:14.815393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7265de-f0ea-44ab-9bd1-6d5be8d7de11 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:14.820338 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2467/4918] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Oct 15 21:10:14 2022] DELETE /identity/v3/regions/tempest-region-1758111011 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:14.831114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c1ff3-5722-40f2-a4c3-bd1b5c91ef26 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:14.834934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ed0c1ff3-5722-40f2-a4c3-bd1b5c91ef26 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:14.853498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48744900-e9cb-43c0-ba33-f7531d1136e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:14.854350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48744900-e9cb-43c0-ba33-f7531d1136e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.855810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48744900-e9cb-43c0-ba33-f7531d1136e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.856603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48744900-e9cb-43c0-ba33-f7531d1136e3 admin admin] PATH_INFO: `/v3/projects/c13c7a93d1914c388b7b189464539b94/users/6a84efb7a2624c82b88a3d176cce5533/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.889961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48744900-e9cb-43c0-ba33-f7531d1136e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13c7a93d1914c388b7b189464539b94, user_id=6a84efb7a2624c82b88a3d176cce5533, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:14.894034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48744900-e9cb-43c0-ba33-f7531d1136e3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:14.900081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c1ff3-5722-40f2-a4c3-bd1b5c91ef26 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:14.921383 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2452/4919] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:14 2022] PUT /identity/v3/projects/c13c7a93d1914c388b7b189464539b94/users/6a84efb7a2624c82b88a3d176cce5533/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:14.922675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48744900-e9cb-43c0-ba33-f7531d1136e3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:14.923010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4420a-11e6-44ff-9c9b-e07ba491cd89 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80fd4a265644d69a566edde911aefea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6789f3b886e5405caa7ef105cbc32f29', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:14.923649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4420a-11e6-44ff-9c9b-e07ba491cd89 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:14.923923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4420a-11e6-44ff-9c9b-e07ba491cd89 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:14.924190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4420a-11e6-44ff-9c9b-e07ba491cd89 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] PATH_INFO: `/v3/services/d7fedc92953840ddbb9418e34ce791b2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:14.924914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4420a-11e6-44ff-9c9b-e07ba491cd89 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorizing `identity:delete_service(service_id=d7fedc92953840ddbb9418e34ce791b2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:14.926532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4420a-11e6-44ff-9c9b-e07ba491cd89 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:14.966311 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2468/4920] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:14 2022] DELETE /identity/v3/services/d7fedc92953840ddbb9418e34ce791b2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:14.970706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4420a-11e6-44ff-9c9b-e07ba491cd89 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:14.974478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f3b4420a-11e6-44ff-9c9b-e07ba491cd89 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:14.989812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:15.005767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4420a-11e6-44ff-9c9b-e07ba491cd89 tempest-EndPointsTestJSON-888434532 tempest-EndPointsTestJSON-888434532-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:15.026505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48744900-e9cb-43c0-ba33-f7531d1136e3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:15.046000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0c7c4-17bd-4967-8b4c-38d640cce99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:15.048071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0c7c4-17bd-4967-8b4c-38d640cce99d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:15.048536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0c7c4-17bd-4967-8b4c-38d640cce99d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:15.049253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0c7c4-17bd-4967-8b4c-38d640cce99d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:15.051303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0c7c4-17bd-4967-8b4c-38d640cce99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:15.055126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0c7c4-17bd-4967-8b4c-38d640cce99d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:15.094858 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2453/4921] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:15.096552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0c7c4-17bd-4967-8b4c-38d640cce99d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:15.114476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0c7c4-17bd-4967-8b4c-38d640cce99d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:15.121861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bfa0a-c8a9-405d-b149-caba52fb2c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:15.122653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfa0a-c8a9-405d-b149-caba52fb2c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:15.123083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfa0a-c8a9-405d-b149-caba52fb2c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:15.123215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfa0a-c8a9-405d-b149-caba52fb2c68 admin admin] PATH_INFO: `/v3/projects/c13c7a93d1914c388b7b189464539b94/users/6a84efb7a2624c82b88a3d176cce5533/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:15.125345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bfa0a-c8a9-405d-b149-caba52fb2c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13c7a93d1914c388b7b189464539b94, user_id=6a84efb7a2624c82b88a3d176cce5533, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:15.134916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bfa0a-c8a9-405d-b149-caba52fb2c68 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:15.161748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:15.162405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:15.165131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:15.165438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] PATH_INFO: `/v3/users/da9fa6201a6641c1a4d42d3bad77214d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:15.169181 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2454/4922] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:15 2022] PUT /identity/v3/projects/c13c7a93d1914c388b7b189464539b94/users/6a84efb7a2624c82b88a3d176cce5533/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:15.169367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] RBAC: Authorizing `identity:delete_user(user_id=da9fa6201a6641c1a4d42d3bad77214d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:15.218744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1a324-3cec-4971-b746-1a8d4c75eef6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:15.221659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1a324-3cec-4971-b746-1a8d4c75eef6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:15.222032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:15.222477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1a324-3cec-4971-b746-1a8d4c75eef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:15.296019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fbd1a324-3cec-4971-b746-1a8d4c75eef6 None None] MFA Rules not processed for user `6a84efb7a2624c82b88a3d176cce5533`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:15.297968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fbd1a324-3cec-4971-b746-1a8d4c75eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:15.341106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:15.382554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da9fa6201a6641c1a4d42d3bad77214d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:15.409202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da9fa6201a6641c1a4d42d3bad77214d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:15.435974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da9fa6201a6641c1a4d42d3bad77214d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:15.446745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da9fa6201a6641c1a4d42d3bad77214d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:15.448199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d14ab8a7-e842-4d89-86ed-637e6af0814a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da9fa6201a6641c1a4d42d3bad77214d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:15.460393 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2469/4923] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:14 2022] DELETE /identity/v3/users/da9fa6201a6641c1a4d42d3bad77214d => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:15.468274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a5e02-6a57-4c26-a98c-9bccdf179a4d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:15.468547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a5e02-6a57-4c26-a98c-9bccdf179a4d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:15.468817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a5e02-6a57-4c26-a98c-9bccdf179a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:15.482577 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2455/4924] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:15.487915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1a324-3cec-4971-b746-1a8d4c75eef6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:15.492490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fbd1a324-3cec-4971-b746-1a8d4c75eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:15.515099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8e6a5e02-6a57-4c26-a98c-9bccdf179a4d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:15.518072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8e6a5e02-6a57-4c26-a98c-9bccdf179a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:15.704220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1a324-3cec-4971-b746-1a8d4c75eef6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:15.715923 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2470/4925] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:15.745260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a5e02-6a57-4c26-a98c-9bccdf179a4d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:15.747208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8e6a5e02-6a57-4c26-a98c-9bccdf179a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:15.835341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:15.849198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd46a7c-56ae-4ec8-b237-4b7f172f0dfa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b217b13af4af41639cd058d438ce2988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0fd36b06a1a34e2d947594ec96d96a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:15.850023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd46a7c-56ae-4ec8-b237-4b7f172f0dfa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:15.850182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd46a7c-56ae-4ec8-b237-4b7f172f0dfa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:15.850369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd46a7c-56ae-4ec8-b237-4b7f172f0dfa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:15.851228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd46a7c-56ae-4ec8-b237-4b7f172f0dfa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-4bd46a7c-56ae-4ec8-b237-4b7f172f0dfa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] You are not authorized to perform the requested action: identity:create_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 162, in post Oct 15 21:10:15.870030 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Oct 15 21:10:15.871042 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Oct 15 21:10:15.871042 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application enforcer_obj._enforce( Oct 15 21:10:15.871042 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Oct 15 21:10:15.871042 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Oct 15 21:10:15.871042 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 21:10:15.871042 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Oct 15 21:10:15.871042 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Oct 15 21:10:15.871042 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:15.889333 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2456/4926] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:10:15 2022] POST /identity/v3/projects => generated 136 bytes in 391 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Oct 15 21:10:15.889333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd46a7c-56ae-4ec8-b237-4b7f172f0dfa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:15.889333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4bd46a7c-56ae-4ec8-b237-4b7f172f0dfa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:15.916054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a5e02-6a57-4c26-a98c-9bccdf179a4d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:15.920986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd46a7c-56ae-4ec8-b237-4b7f172f0dfa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:15.924394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e4bbd-3386-477a-8c92-cb1e06f903c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:15.925010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e4bbd-3386-477a-8c92-cb1e06f903c6 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:15.925836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e4bbd-3386-477a-8c92-cb1e06f903c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:15.927660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e4bbd-3386-477a-8c92-cb1e06f903c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:15.932229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:15.932864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:15.933660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:15.933929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] PATH_INFO: `/v3/projects/c55b6a20bb2642dfb1d2395efd19653a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:15.935615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c55b6a20bb2642dfb1d2395efd19653a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:15.941205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:15.949311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e4bbd-3386-477a-8c92-cb1e06f903c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:15.953254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e4bbd-3386-477a-8c92-cb1e06f903c6 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:15.978604 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2471/4927] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:15.980648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e4bbd-3386-477a-8c92-cb1e06f903c6 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:16.069286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c55b6a20bb2642dfb1d2395efd19653a', 'request_id': 'req-5961e090-7bcd-4a43-ae75-754a0c709a77'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:16.090215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c55b6a20bb2642dfb1d2395efd19653a', 'request_id': 'req-5961e090-7bcd-4a43-ae75-754a0c709a77'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:16.099650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c55b6a20bb2642dfb1d2395efd19653a', 'request_id': 'req-5961e090-7bcd-4a43-ae75-754a0c709a77'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:16.106809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:16.130465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] The token cache is being invalidate because project c55b6a20bb2642dfb1d2395efd19653a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:16.130762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:16.134533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5961e090-7bcd-4a43-ae75-754a0c709a77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:16.144954 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2457/4928] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:15 2022] DELETE /identity/v3/projects/c55b6a20bb2642dfb1d2395efd19653a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:16.154866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eac13e-f9b4-4e7d-8f41-c85003dd0030 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:16.155291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eac13e-f9b4-4e7d-8f41-c85003dd0030 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:16.155655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eac13e-f9b4-4e7d-8f41-c85003dd0030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:16.169645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e4bbd-3386-477a-8c92-cb1e06f903c6 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:16.226645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-11eac13e-f9b4-4e7d-8f41-c85003dd0030 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:16.231175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-11eac13e-f9b4-4e7d-8f41-c85003dd0030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:16.337779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5951483-9e56-4498-9a32-39998cb054a5 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a84efb7a2624c82b88a3d176cce5533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13c7a93d1914c388b7b189464539b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:16.338640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5951483-9e56-4498-9a32-39998cb054a5 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:16.338989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5951483-9e56-4498-9a32-39998cb054a5 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:16.339329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5951483-9e56-4498-9a32-39998cb054a5 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:16.340610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5951483-9e56-4498-9a32-39998cb054a5 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:16.344353 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2458/4929] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:16.345524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5951483-9e56-4498-9a32-39998cb054a5 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:16.354641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4995765-c1a7-475a-9ef4-387112df1a2b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:16.355267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4995765-c1a7-475a-9ef4-387112df1a2b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:16.355720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4995765-c1a7-475a-9ef4-387112df1a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-f5951483-9e56-4498-9a32-39998cb054a5 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Invalid input for field 'name': '' is too short Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: Failed validating 'minLength' in schema['properties']['name']: Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: On instance['name']: Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: Failed validating 'minLength' in schema['properties']['name']: Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: On instance['name']: Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: '' Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise error Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application On instance['name']: Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application '' Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.366091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 15 21:10:16.368342 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 15 21:10:16.369576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 15 21:10:16.369576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Oct 15 21:10:16.369576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.369576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Oct 15 21:10:16.369576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:10:16.369576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.369576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application On instance['name']: Oct 15 21:10:16.369576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application '' Oct 15 21:10:16.369576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.372951 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2472/4930] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:10:15 2022] POST /identity/v3/projects => generated 283 bytes in 394 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 15 21:10:16.380502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5951483-9e56-4498-9a32-39998cb054a5 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:16.384306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f5951483-9e56-4498-9a32-39998cb054a5 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:16.396419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b4995765-c1a7-475a-9ef4-387112df1a2b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:16.397889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b4995765-c1a7-475a-9ef4-387112df1a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:16.416921 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2459/4931] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:16.421190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5951483-9e56-4498-9a32-39998cb054a5 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:16.421751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4995765-c1a7-475a-9ef4-387112df1a2b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:16.434387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866818f9-112b-4668-8d60-37216b351604 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a84efb7a2624c82b88a3d176cce5533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13c7a93d1914c388b7b189464539b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:16.435107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866818f9-112b-4668-8d60-37216b351604 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:16.436089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866818f9-112b-4668-8d60-37216b351604 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:16.436089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866818f9-112b-4668-8d60-37216b351604 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:16.436538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866818f9-112b-4668-8d60-37216b351604 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:16.439680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866818f9-112b-4668-8d60-37216b351604 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:16.443402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4995765-c1a7-475a-9ef4-387112df1a2b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-866818f9-112b-4668-8d60-37216b351604 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: Failed validating 'maxLength' in schema['properties']['name']: Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: On instance['name']: Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: Failed validating 'maxLength' in schema['properties']['name']: Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: On instance['name']: Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise error Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Oct 15 21:10:16.454747 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application On instance['name']: Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:16.456141 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application On instance['name']: Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 15 21:10:16.457493 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.459804 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2473/4932] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:10:16 2022] POST /identity/v3/projects => generated 412 bytes in 80 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Oct 15 21:10:16.466101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf74f84-6bd4-4ce1-8a0f-bcbda3dbbae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:16.467577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866818f9-112b-4668-8d60-37216b351604 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:16.468663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf74f84-6bd4-4ce1-8a0f-bcbda3dbbae5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:16.469157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf74f84-6bd4-4ce1-8a0f-bcbda3dbbae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:16.469631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf74f84-6bd4-4ce1-8a0f-bcbda3dbbae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:16.470786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf74f84-6bd4-4ce1-8a0f-bcbda3dbbae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:16.471237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf74f84-6bd4-4ce1-8a0f-bcbda3dbbae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:16.487331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf74f84-6bd4-4ce1-8a0f-bcbda3dbbae5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:16.490484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866818f9-112b-4668-8d60-37216b351604 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:16.505688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3aac9e-7559-4426-96f6-0dc2340f86a6 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a84efb7a2624c82b88a3d176cce5533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13c7a93d1914c388b7b189464539b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:16.506489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3aac9e-7559-4426-96f6-0dc2340f86a6 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:16.506867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3aac9e-7559-4426-96f6-0dc2340f86a6 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:16.507235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3aac9e-7559-4426-96f6-0dc2340f86a6 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] PATH_INFO: `/v3/projects/2c36a7a229034fe7a8d20e1476ed3bbb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:16.514048 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2460/4933] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:16.517797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf74f84-6bd4-4ce1-8a0f-bcbda3dbbae5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:16.522086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3aac9e-7559-4426-96f6-0dc2340f86a6 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2c36a7a229034fe7a8d20e1476ed3bbb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:16.524634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3aac9e-7559-4426-96f6-0dc2340f86a6 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-5d3aac9e-7559-4426-96f6-0dc2340f86a6 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Could not find project: 2c36a7a229034fe7a8d20e1476ed3bbb.: keystone.exception.ProjectNotFound: Could not find project: 2c36a7a229034fe7a8d20e1476ed3bbb. Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Oct 15 21:10:16.551463 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Oct 15 21:10:16.552614 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:16.552614 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:16.552614 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Oct 15 21:10:16.552614 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Oct 15 21:10:16.552614 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:10:16.552614 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:10:16.552614 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:10:16.552614 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:10:16.552614 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 2c36a7a229034fe7a8d20e1476ed3bbb. Oct 15 21:10:16.552614 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.565644 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2474/4934] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:16 2022] DELETE /identity/v3/projects/2c36a7a229034fe7a8d20e1476ed3bbb => generated 113 bytes in 98 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 15 21:10:16.566817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3aac9e-7559-4426-96f6-0dc2340f86a6 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:16.575397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5d3aac9e-7559-4426-96f6-0dc2340f86a6 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:16.596433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf74f84-6bd4-4ce1-8a0f-bcbda3dbbae5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:16.611687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ac750-bc3c-4b53-bb98-7f1a2eb56c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:16.613613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ac750-bc3c-4b53-bb98-7f1a2eb56c15 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:16.617011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ac750-bc3c-4b53-bb98-7f1a2eb56c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:16.620710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ac750-bc3c-4b53-bb98-7f1a2eb56c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:16.626588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ac750-bc3c-4b53-bb98-7f1a2eb56c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:16.631005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ac750-bc3c-4b53-bb98-7f1a2eb56c15 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:16.695702 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2461/4935] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:16 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:16.700866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ac750-bc3c-4b53-bb98-7f1a2eb56c15 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:16.763187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3aac9e-7559-4426-96f6-0dc2340f86a6 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:16.816498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:16.847736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f957ebd-dd0b-49e4-b906-fe308e4404aa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b217b13af4af41639cd058d438ce2988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0fd36b06a1a34e2d947594ec96d96a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:16.848458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957ebd-dd0b-49e4-b906-fe308e4404aa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:16.848799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957ebd-dd0b-49e4-b906-fe308e4404aa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:16.850547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957ebd-dd0b-49e4-b906-fe308e4404aa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:16.851515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f957ebd-dd0b-49e4-b906-fe308e4404aa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] RBAC: Authorizing `identity:list_projects()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-2f957ebd-dd0b-49e4-b906-fe308e4404aa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] You are not authorized to perform the requested action: identity:list_projects.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 153, in get Oct 15 21:10:16.867258 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._list_projects() Oct 15 21:10:16.868448 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 120, in _list_projects Oct 15 21:10:16.868448 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:list_projects', Oct 15 21:10:16.868448 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Oct 15 21:10:16.868448 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application enforcer_obj._enforce( Oct 15 21:10:16.868448 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Oct 15 21:10:16.868448 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Oct 15 21:10:16.868448 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 21:10:16.868448 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Oct 15 21:10:16.868448 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Oct 15 21:10:16.868448 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:16.870034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ac750-bc3c-4b53-bb98-7f1a2eb56c15 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:16.870212 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2475/4936] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Oct 15 21:10:16 2022] GET /identity/v3/projects => generated 135 bytes in 304 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:10:16.877419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f957ebd-dd0b-49e4-b906-fe308e4404aa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:16.890153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd726c8-9847-4455-993e-80801b4d4ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:16.890772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd726c8-9847-4455-993e-80801b4d4ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:16.891048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd726c8-9847-4455-993e-80801b4d4ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:16.891316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd726c8-9847-4455-993e-80801b4d4ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:16.891950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd726c8-9847-4455-993e-80801b4d4ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:16.894089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd726c8-9847-4455-993e-80801b4d4ed4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:16.960067 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2462/4937] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:16 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:16.964788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd726c8-9847-4455-993e-80801b4d4ed4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:16.977143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:16.980120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd726c8-9847-4455-993e-80801b4d4ed4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:16.985817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633d32a-a81c-4097-ac7b-45500d221e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:16.986597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633d32a-a81c-4097-ac7b-45500d221e0c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:16.986875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633d32a-a81c-4097-ac7b-45500d221e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:16.987138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633d32a-a81c-4097-ac7b-45500d221e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:16.987820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633d32a-a81c-4097-ac7b-45500d221e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:16.990985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633d32a-a81c-4097-ac7b-45500d221e0c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:17.007303 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2463/4938] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:17.008236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633d32a-a81c-4097-ac7b-45500d221e0c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:17.027426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633d32a-a81c-4097-ac7b-45500d221e0c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:17.037607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a502-9a4c-4b28-835a-39dd0cd78e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:17.038284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a502-9a4c-4b28-835a-39dd0cd78e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:17.038564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a502-9a4c-4b28-835a-39dd0cd78e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:17.041815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f957ebd-dd0b-49e4-b906-fe308e4404aa tempest-ProjectsNegativeStaticTestJSON-1181284743 tempest-ProjectsNegativeStaticTestJSON-1181284743-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:17.043471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a502-9a4c-4b28-835a-39dd0cd78e2d admin admin] PATH_INFO: `/v3/projects/5aacf5263bbb4c0e8f97c65cb3d7122f/users/9ab13f854bf64104be49788196d20f68/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:17.048289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4635b-f4c0-45e6-ad57-f5dd66a641b3 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a84efb7a2624c82b88a3d176cce5533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13c7a93d1914c388b7b189464539b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:17.049322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4635b-f4c0-45e6-ad57-f5dd66a641b3 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:17.049598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4635b-f4c0-45e6-ad57-f5dd66a641b3 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:17.050025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4635b-f4c0-45e6-ad57-f5dd66a641b3 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:17.051751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4635b-f4c0-45e6-ad57-f5dd66a641b3 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:17.054418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4635b-f4c0-45e6-ad57-f5dd66a641b3 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:17.100553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91a502-9a4c-4b28-835a-39dd0cd78e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aacf5263bbb4c0e8f97c65cb3d7122f, user_id=9ab13f854bf64104be49788196d20f68, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:17.111302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91a502-9a4c-4b28-835a-39dd0cd78e2d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:17.124521 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2476/4939] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:16 2022] POST /identity/v3/projects => generated 356 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:10:17.130519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4635b-f4c0-45e6-ad57-f5dd66a641b3 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:17.142274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4ec4635b-f4c0-45e6-ad57-f5dd66a641b3 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:17.150956 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2464/4940] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:17 2022] PUT /identity/v3/projects/5aacf5263bbb4c0e8f97c65cb3d7122f/users/9ab13f854bf64104be49788196d20f68/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:17.153635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a502-9a4c-4b28-835a-39dd0cd78e2d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:17.306028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a502-9a4c-4b28-835a-39dd0cd78e2d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:17.312852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4635b-f4c0-45e6-ad57-f5dd66a641b3 tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:17.318177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b646c8-db48-4766-b66b-e2e8028ed97f tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a84efb7a2624c82b88a3d176cce5533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13c7a93d1914c388b7b189464539b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:17.318911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b646c8-db48-4766-b66b-e2e8028ed97f tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:17.320529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b646c8-db48-4766-b66b-e2e8028ed97f tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:17.320529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b646c8-db48-4766-b66b-e2e8028ed97f tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:17.322331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b646c8-db48-4766-b66b-e2e8028ed97f tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:17.327409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b646c8-db48-4766-b66b-e2e8028ed97f tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:17.377869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.sql.core [None req-c6b646c8-db48-4766-b66b-e2e8028ed97f tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-88536114' for key 'project.ixu_project_name_domain_id'") Oct 15 21:10:17.377869 ncdv-master-1 devstack@keystone.service[99099]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 21:10:17.377869 ncdv-master-1 devstack@keystone.service[99099]: [parameters: {'id': '2e955acae6eb41d8a5fdf34212e16de4', 'name': 'tempest-project-dup-88536114', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 15 21:10:17.377869 ncdv-master-1 devstack@keystone.service[99099]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99099) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-c6b646c8-db48-4766-b66b-e2e8028ed97f tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-88536114, project id 2e955acae6eb41d8a5fdf34212e16de4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-88536114, project id 2e955acae6eb41d8a5fdf34212e16de4. Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application conn.query(q) Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 21:10:17.388872 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application result.read() Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-tempest-project-dup-88536114' for key 'project.ixu_project_name_domain_id'") Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 21:10:17.390019 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application next(self.gen) Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application yield resource Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application next(self.gen) Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise self.value Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 21:10:17.394398 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application session.commit() Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.session.flush() Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application compat.raise_( Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 21:10:17.395735 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 21:10:17.397102 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application conn.query(q) Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application result.read() Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 21:10:17.398603 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-88536114' for key 'project.ixu_project_name_domain_id'") Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [parameters: {'id': '2e955acae6eb41d8a5fdf34212e16de4', 'name': 'tempest-project-dup-88536114', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:17.400287 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-project-dup-88536114 at domain ID default. Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:17.401271 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-88536114, project id 2e955acae6eb41d8a5fdf34212e16de4. Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2465/4941] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:10:17 2022] POST /identity/v3/projects => generated 283 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 21:10:17.402549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b646c8-db48-4766-b66b-e2e8028ed97f tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:17.440189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:17.440754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:17.441178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:17.441463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] PATH_INFO: `/v3/projects/ffe59f75f502409eb26737b70fd58536` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:17.443021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffe59f75f502409eb26737b70fd58536)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:17.445733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:17.507496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b646c8-db48-4766-b66b-e2e8028ed97f tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:17.519271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffe59f75f502409eb26737b70fd58536', 'request_id': 'req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.530980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffe59f75f502409eb26737b70fd58536', 'request_id': 'req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.542721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffe59f75f502409eb26737b70fd58536', 'request_id': 'req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.554644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:17.558470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:17.577102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] The token cache is being invalidate because project ffe59f75f502409eb26737b70fd58536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:17.578148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.579069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.584626 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2477/4942] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:17 2022] DELETE /identity/v3/projects/ffe59f75f502409eb26737b70fd58536 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:17.585948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:17.588055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:17.640832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3bd0d-e70a-4d8d-9319-71699eed9151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:17.644071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3bd0d-e70a-4d8d-9319-71699eed9151 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:17.644877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3bd0d-e70a-4d8d-9319-71699eed9151 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:17.645444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3bd0d-e70a-4d8d-9319-71699eed9151 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:17.646579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3bd0d-e70a-4d8d-9319-71699eed9151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:17.648825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3bd0d-e70a-4d8d-9319-71699eed9151 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:17.683054 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2466/4943] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:17.684400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3bd0d-e70a-4d8d-9319-71699eed9151 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:17.687553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6be3bd0d-e70a-4d8d-9319-71699eed9151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:17.695461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ae2a0-89db-4530-b511-fa7f7396b8ce admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:17.701246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a84efb7a2624c82b88a3d176cce5533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13c7a93d1914c388b7b189464539b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:17.701969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:17.702253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:17.702528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] PATH_INFO: `/v3/projects/e885d84161fe489c8d62717af1ad13af` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:17.704034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e885d84161fe489c8d62717af1ad13af)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:17.706460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:17.711355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3bd0d-e70a-4d8d-9319-71699eed9151 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:17.717737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:17.718779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:17.719263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:17.719760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] PATH_INFO: `/v3/users/d80fd4a265644d69a566edde911aefea` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:17.721365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d80fd4a265644d69a566edde911aefea)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:17.723928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:17.792661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e885d84161fe489c8d62717af1ad13af', 'request_id': 'req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.804319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e885d84161fe489c8d62717af1ad13af', 'request_id': 'req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.814288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e885d84161fe489c8d62717af1ad13af', 'request_id': 'req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.815749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd80fd4a265644d69a566edde911aefea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.827238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd80fd4a265644d69a566edde911aefea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.829462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] The token cache is being invalidate because project e885d84161fe489c8d62717af1ad13af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:17.829912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.830774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.832812 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2478/4944] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:17 2022] DELETE /identity/v3/projects/e885d84161fe489c8d62717af1ad13af => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:17.833880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:17.835451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:17.852174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd80fd4a265644d69a566edde911aefea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.865235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd80fd4a265644d69a566edde911aefea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.866522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd80fd4a265644d69a566edde911aefea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:17.878059 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2467/4945] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:17 2022] DELETE /identity/v3/users/d80fd4a265644d69a566edde911aefea => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:17.879688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:17.882256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:17.957668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd22fb-1834-4bba-ac5d-fd450b78e13e tempest-ProjectsNegativeStaticTestJSON-1534652385 tempest-ProjectsNegativeStaticTestJSON-1534652385-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:17.996339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db2e0b-5642-4f9e-953b-baa072cb0d20 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:18.004197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:18.005799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.006441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.006837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] PATH_INFO: `/v3/users/b217b13af4af41639cd058d438ce2988` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.008737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b217b13af4af41639cd058d438ce2988)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:18.011100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:18.012071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:18.100136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5474f-529a-4a92-9736-7143e1c08b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:18.100809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5474f-529a-4a92-9736-7143e1c08b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.101101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5474f-529a-4a92-9736-7143e1c08b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.101372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5474f-529a-4a92-9736-7143e1c08b25 admin admin] PATH_INFO: `/v3/projects/5aacf5263bbb4c0e8f97c65cb3d7122f/users/9ab13f854bf64104be49788196d20f68/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.107117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:18.109583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5474f-529a-4a92-9736-7143e1c08b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aacf5263bbb4c0e8f97c65cb3d7122f, user_id=9ab13f854bf64104be49788196d20f68, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:18.117407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5474f-529a-4a92-9736-7143e1c08b25 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:18.120925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b217b13af4af41639cd058d438ce2988'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:18.139407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b217b13af4af41639cd058d438ce2988'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:18.144338 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2479/4946] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:17 2022] PUT /identity/v3/projects/5aacf5263bbb4c0e8f97c65cb3d7122f/users/9ab13f854bf64104be49788196d20f68/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:18.158151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b217b13af4af41639cd058d438ce2988'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:18.162543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441f28b-0223-4935-8351-ab3e268a4d1c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.163096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441f28b-0223-4935-8351-ab3e268a4d1c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.163562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441f28b-0223-4935-8351-ab3e268a4d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.170027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b217b13af4af41639cd058d438ce2988'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:18.171083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-10194225-6e22-4c8a-9246-24cecd69c57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b217b13af4af41639cd058d438ce2988'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:18.193558 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2468/4947] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:17 2022] DELETE /identity/v3/users/b217b13af4af41639cd058d438ce2988 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:18.196464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fa415-b6f8-4a1e-ac7b-dc32bc27b4ca None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.197400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fa415-b6f8-4a1e-ac7b-dc32bc27b4ca None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.198194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fa415-b6f8-4a1e-ac7b-dc32bc27b4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.226982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-702fa415-b6f8-4a1e-ac7b-dc32bc27b4ca None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:18.234467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9441f28b-0223-4935-8351-ab3e268a4d1c None None] MFA Rules not processed for user `9ab13f854bf64104be49788196d20f68`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:18.235962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9441f28b-0223-4935-8351-ab3e268a4d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:18.237056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-702fa415-b6f8-4a1e-ac7b-dc32bc27b4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:18.349759 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2469/4948] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:18.353530 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2480/4949] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:18.358558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fa415-b6f8-4a1e-ac7b-dc32bc27b4ca None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:18.361268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-702fa415-b6f8-4a1e-ac7b-dc32bc27b4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:18.378154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fa415-b6f8-4a1e-ac7b-dc32bc27b4ca None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:18.384012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4dafe-be3d-440c-ba12-f4b62b575b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:18.384837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4dafe-be3d-440c-ba12-f4b62b575b7c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.385194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4dafe-be3d-440c-ba12-f4b62b575b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.386138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4dafe-be3d-440c-ba12-f4b62b575b7c admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.386806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4dafe-be3d-440c-ba12-f4b62b575b7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:18.388723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4dafe-be3d-440c-ba12-f4b62b575b7c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:18.394054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441f28b-0223-4935-8351-ab3e268a4d1c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:18.395525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9441f28b-0223-4935-8351-ab3e268a4d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:18.454430 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2470/4950] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:18 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:18.459292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4dafe-be3d-440c-ba12-f4b62b575b7c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:18.515529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441f28b-0223-4935-8351-ab3e268a4d1c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:18.588065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4dafe-be3d-440c-ba12-f4b62b575b7c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:18.596369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f85d28-af4c-48b4-bb08-bd975084581a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:18.596989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f85d28-af4c-48b4-bb08-bd975084581a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.597291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f85d28-af4c-48b4-bb08-bd975084581a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.597561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f85d28-af4c-48b4-bb08-bd975084581a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.598192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f85d28-af4c-48b4-bb08-bd975084581a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:18.600675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f85d28-af4c-48b4-bb08-bd975084581a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:18.645468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc367f-bc8d-4e6a-a1c3-2e7483b0fe6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:18.646085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc367f-bc8d-4e6a-a1c3-2e7483b0fe6a service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.646360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc367f-bc8d-4e6a-a1c3-2e7483b0fe6a service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.646766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc367f-bc8d-4e6a-a1c3-2e7483b0fe6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.653083 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2471/4951] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:18 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:18.660647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f85d28-af4c-48b4-bb08-bd975084581a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:18.674652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f85d28-af4c-48b4-bb08-bd975084581a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:18.678068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc367f-bc8d-4e6a-a1c3-2e7483b0fe6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:18.680840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc367f-bc8d-4e6a-a1c3-2e7483b0fe6a service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:18.684252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b6c2f-cff8-4fcf-b3b9-fd12265c77a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:18.684923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b6c2f-cff8-4fcf-b3b9-fd12265c77a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.685347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b6c2f-cff8-4fcf-b3b9-fd12265c77a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.685971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b6c2f-cff8-4fcf-b3b9-fd12265c77a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.691304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b6c2f-cff8-4fcf-b3b9-fd12265c77a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:18.693264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b6c2f-cff8-4fcf-b3b9-fd12265c77a8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:18.701622 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2481/4952] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:10:18.707827 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2472/4953] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:18.713047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc367f-bc8d-4e6a-a1c3-2e7483b0fe6a service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:18.740890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc367f-bc8d-4e6a-a1c3-2e7483b0fe6a service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:18.747060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147ed28-648e-44ce-980d-e56abfa62436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:18.747738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147ed28-648e-44ce-980d-e56abfa62436 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.748285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147ed28-648e-44ce-980d-e56abfa62436 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.748624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147ed28-648e-44ce-980d-e56abfa62436 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.749525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147ed28-648e-44ce-980d-e56abfa62436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:18.751210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147ed28-648e-44ce-980d-e56abfa62436 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:18.754567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:18.769526 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2482/4954] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:18.770494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b6c2f-cff8-4fcf-b3b9-fd12265c77a8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:18.793239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b6c2f-cff8-4fcf-b3b9-fd12265c77a8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:18.798908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbd2f5-80c6-42b4-9c37-91967e7822b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:18.799939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbd2f5-80c6-42b4-9c37-91967e7822b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.799939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbd2f5-80c6-42b4-9c37-91967e7822b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.800393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbd2f5-80c6-42b4-9c37-91967e7822b6 admin admin] PATH_INFO: `/v3/projects/8975c29b118845949d8e00a329378290/users/bbddce540e5947328062f83b93d1a077/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.823417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:18.827617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbd2f5-80c6-42b4-9c37-91967e7822b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8975c29b118845949d8e00a329378290, user_id=bbddce540e5947328062f83b93d1a077, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:18.831842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbd2f5-80c6-42b4-9c37-91967e7822b6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:18.866411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb700d7d-b76e-46bb-8a08-70f339911701 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:18.866692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb700d7d-b76e-46bb-8a08-70f339911701 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:18.866965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb700d7d-b76e-46bb-8a08-70f339911701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:18.870103 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2473/4955] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:18 2022] PUT /identity/v3/projects/8975c29b118845949d8e00a329378290/users/bbddce540e5947328062f83b93d1a077/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:18.909441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fb700d7d-b76e-46bb-8a08-70f339911701 None None] MFA Rules not processed for user `bbddce540e5947328062f83b93d1a077`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:18.911026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fb700d7d-b76e-46bb-8a08-70f339911701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:19.138484 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2483/4956] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:19.145164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbd2f5-80c6-42b4-9c37-91967e7822b6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:19.146982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-90fbd2f5-80c6-42b4-9c37-91967e7822b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:19.326958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbd2f5-80c6-42b4-9c37-91967e7822b6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:19.330942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da9ea3-f2b3-4e3d-a2c4-40222db17215 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:19.331646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da9ea3-f2b3-4e3d-a2c4-40222db17215 service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:19.331923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da9ea3-f2b3-4e3d-a2c4-40222db17215 service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:19.332192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da9ea3-f2b3-4e3d-a2c4-40222db17215 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:19.471098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da9ea3-f2b3-4e3d-a2c4-40222db17215 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:19.472617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da9ea3-f2b3-4e3d-a2c4-40222db17215 service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:19.485167 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2474/4957] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:10:19.746645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb700d7d-b76e-46bb-8a08-70f339911701 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:19.748776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fb700d7d-b76e-46bb-8a08-70f339911701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:19.935368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb700d7d-b76e-46bb-8a08-70f339911701 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:19.946440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:19.947555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:19.948031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:19.948502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] PATH_INFO: `/v3/projects/6789f3b886e5405caa7ef105cbc32f29` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:19.950418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6789f3b886e5405caa7ef105cbc32f29)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:19.952636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:20.063324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6789f3b886e5405caa7ef105cbc32f29', 'request_id': 'req-86f76a9b-7c49-499f-8f4e-177474d2831b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:20.078397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6789f3b886e5405caa7ef105cbc32f29', 'request_id': 'req-86f76a9b-7c49-499f-8f4e-177474d2831b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:20.089464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6789f3b886e5405caa7ef105cbc32f29', 'request_id': 'req-86f76a9b-7c49-499f-8f4e-177474d2831b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:20.107296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] The token cache is being invalidate because project 6789f3b886e5405caa7ef105cbc32f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:20.107756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:20.108503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:20.110446 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2484/4958] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:19 2022] DELETE /identity/v3/projects/6789f3b886e5405caa7ef105cbc32f29 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:20.117450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cac71-c46a-4236-93ad-b9b411770eb7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:20.118266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cac71-c46a-4236-93ad-b9b411770eb7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:20.118542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cac71-c46a-4236-93ad-b9b411770eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:20.154161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-171cac71-c46a-4236-93ad-b9b411770eb7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:20.154900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-171cac71-c46a-4236-93ad-b9b411770eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:20.253171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:20.254787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:20.304620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:20.345804 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2475/4959] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:20.354543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3d59a-78e3-4b4c-bf01-eda801325daf None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:20.354910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3d59a-78e3-4b4c-bf01-eda801325daf None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:20.355241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3d59a-78e3-4b4c-bf01-eda801325daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:20.431647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8bd3d59a-78e3-4b4c-bf01-eda801325daf None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:20.433154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8bd3d59a-78e3-4b4c-bf01-eda801325daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:20.437976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f76a9b-7c49-499f-8f4e-177474d2831b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:20.448672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:20.449891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:20.449891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:20.450292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] PATH_INFO: `/v3/users/a83cacaed7aa4d87a5fef3a7cee1a77b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:20.452204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83cacaed7aa4d87a5fef3a7cee1a77b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:20.456118 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2476/4960] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:20.456275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:20.457726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3d59a-78e3-4b4c-bf01-eda801325daf None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:20.478312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3d59a-78e3-4b4c-bf01-eda801325daf None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:20.490583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328642e-6d8a-4099-b317-b4b135c22552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:20.490810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328642e-6d8a-4099-b317-b4b135c22552 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:20.490913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328642e-6d8a-4099-b317-b4b135c22552 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:20.491340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328642e-6d8a-4099-b317-b4b135c22552 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:20.492008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328642e-6d8a-4099-b317-b4b135c22552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:20.492292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328642e-6d8a-4099-b317-b4b135c22552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:20.495636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328642e-6d8a-4099-b317-b4b135c22552 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:20.525356 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2477/4961] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:20.530186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328642e-6d8a-4099-b317-b4b135c22552 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:20.549447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328642e-6d8a-4099-b317-b4b135c22552 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:20.561577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83cacaed7aa4d87a5fef3a7cee1a77b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:20.562121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b690bcd-1e29-4abf-a872-3145138bc175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:20.562882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b690bcd-1e29-4abf-a872-3145138bc175 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:20.563161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b690bcd-1e29-4abf-a872-3145138bc175 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:20.563426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b690bcd-1e29-4abf-a872-3145138bc175 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:20.564052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b690bcd-1e29-4abf-a872-3145138bc175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:20.565992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b690bcd-1e29-4abf-a872-3145138bc175 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:20.592094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83cacaed7aa4d87a5fef3a7cee1a77b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:20.593935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:20.622673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83cacaed7aa4d87a5fef3a7cee1a77b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:20.639308 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2478/4962] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:20 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:20.642041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b690bcd-1e29-4abf-a872-3145138bc175 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:20.652250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83cacaed7aa4d87a5fef3a7cee1a77b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:20.653389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-275313a9-0462-4cb8-b8eb-62cd07e996be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83cacaed7aa4d87a5fef3a7cee1a77b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:20.670257 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2485/4963] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:20 2022] DELETE /identity/v3/users/a83cacaed7aa4d87a5fef3a7cee1a77b => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:20.697552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa808d69-5e6a-41e8-b214-40d4ef78ba3f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:20.697916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa808d69-5e6a-41e8-b214-40d4ef78ba3f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:20.699212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa808d69-5e6a-41e8-b214-40d4ef78ba3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:20.745637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fa808d69-5e6a-41e8-b214-40d4ef78ba3f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:20.747869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fa808d69-5e6a-41e8-b214-40d4ef78ba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:20.787440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b690bcd-1e29-4abf-a872-3145138bc175 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:20.922047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae79596-25d2-4cee-97e2-e8acec6db634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:20.924486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae79596-25d2-4cee-97e2-e8acec6db634 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:20.925831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae79596-25d2-4cee-97e2-e8acec6db634 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:20.926290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae79596-25d2-4cee-97e2-e8acec6db634 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:20.927889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae79596-25d2-4cee-97e2-e8acec6db634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:20.930085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae79596-25d2-4cee-97e2-e8acec6db634 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:20.944169 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2486/4964] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:20.944169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa808d69-5e6a-41e8-b214-40d4ef78ba3f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:20.948206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fa808d69-5e6a-41e8-b214-40d4ef78ba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:20.973878 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2479/4965] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:20 2022] POST /identity/v3/users => generated 285 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:10:20.974696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae79596-25d2-4cee-97e2-e8acec6db634 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:20.978830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cae79596-25d2-4cee-97e2-e8acec6db634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:21.005221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae79596-25d2-4cee-97e2-e8acec6db634 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:21.006659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa808d69-5e6a-41e8-b214-40d4ef78ba3f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:21.016487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:21.017684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:21.017963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:21.018676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] PATH_INFO: `/v3/projects/0fd36b06a1a34e2d947594ec96d96a6c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:21.019393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd36b06a1a34e2d947594ec96d96a6c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:21.021565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:21.027478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:21.031551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:21.031833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:21.032106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] PATH_INFO: `/v3/users/9ab13f854bf64104be49788196d20f68` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:21.033472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab13f854bf64104be49788196d20f68)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:21.036417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:21.114183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd36b06a1a34e2d947594ec96d96a6c', 'request_id': 'req-9c7c09f2-c607-4f03-891c-90ca28dd815e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.115687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:21.134825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd36b06a1a34e2d947594ec96d96a6c', 'request_id': 'req-9c7c09f2-c607-4f03-891c-90ca28dd815e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.155882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd36b06a1a34e2d947594ec96d96a6c', 'request_id': 'req-9c7c09f2-c607-4f03-891c-90ca28dd815e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.158754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab13f854bf64104be49788196d20f68'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.177656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] The token cache is being invalidate because project 0fd36b06a1a34e2d947594ec96d96a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:21.178074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.181694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.185681 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2480/4966] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:20 2022] DELETE /identity/v3/projects/0fd36b06a1a34e2d947594ec96d96a6c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:21.187995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:21.207237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab13f854bf64104be49788196d20f68'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.210030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:21.222289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab13f854bf64104be49788196d20f68'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.235893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:21.240678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab13f854bf64104be49788196d20f68'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.245262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab13f854bf64104be49788196d20f68'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.276936 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2487/4967] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:20 2022] DELETE /identity/v3/users/9ab13f854bf64104be49788196d20f68 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:21.279990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:21.284339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:21.395664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c09f2-c607-4f03-891c-90ca28dd815e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:21.474463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751bdf-2577-4de2-8914-e4340bc4642e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:21.490457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79281c00-03a8-4b08-a76f-2826febe64e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:21.496629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79281c00-03a8-4b08-a76f-2826febe64e8 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:21.496955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79281c00-03a8-4b08-a76f-2826febe64e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:21.497209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79281c00-03a8-4b08-a76f-2826febe64e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:21.516205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-79281c00-03a8-4b08-a76f-2826febe64e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:21.641366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed081d3-793c-4bf3-8ef9-3426ba73a374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:21.644722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed081d3-793c-4bf3-8ef9-3426ba73a374 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:21.645389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed081d3-793c-4bf3-8ef9-3426ba73a374 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:21.645661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed081d3-793c-4bf3-8ef9-3426ba73a374 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:21.646371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed081d3-793c-4bf3-8ef9-3426ba73a374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:21.653546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed081d3-793c-4bf3-8ef9-3426ba73a374 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:21.685677 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2481/4968] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:21.686807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed081d3-793c-4bf3-8ef9-3426ba73a374 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:21.693108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0ed081d3-793c-4bf3-8ef9-3426ba73a374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:21.712141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed081d3-793c-4bf3-8ef9-3426ba73a374 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:21.723784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:21.727816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:21.729801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:21.730247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] PATH_INFO: `/v3/users/6a84efb7a2624c82b88a3d176cce5533` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:21.735764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a84efb7a2624c82b88a3d176cce5533)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:21.742852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:21.778798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79281c00-03a8-4b08-a76f-2826febe64e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:21.781360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79281c00-03a8-4b08-a76f-2826febe64e8 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:21.808261 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2488/4969] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:21.811959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c34fe7-aa1f-41f8-b4ea-d2f2ec304e12 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:21.812291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c34fe7-aa1f-41f8-b4ea-d2f2ec304e12 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:21.812509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c34fe7-aa1f-41f8-b4ea-d2f2ec304e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:21.847190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b6c34fe7-aa1f-41f8-b4ea-d2f2ec304e12 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:21.847499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a84efb7a2624c82b88a3d176cce5533'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.849491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b6c34fe7-aa1f-41f8-b4ea-d2f2ec304e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:21.864112 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2489/4970] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:21.864112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c34fe7-aa1f-41f8-b4ea-d2f2ec304e12 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:21.864401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b6c34fe7-aa1f-41f8-b4ea-d2f2ec304e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:21.868302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a84efb7a2624c82b88a3d176cce5533'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.889062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c34fe7-aa1f-41f8-b4ea-d2f2ec304e12 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:21.889523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a84efb7a2624c82b88a3d176cce5533'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.897961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189dba-9ef2-4238-a70d-064316d8e341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:21.898635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189dba-9ef2-4238-a70d-064316d8e341 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:21.899556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189dba-9ef2-4238-a70d-064316d8e341 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:21.899905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189dba-9ef2-4238-a70d-064316d8e341 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:21.900699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba189dba-9ef2-4238-a70d-064316d8e341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:21.902383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba189dba-9ef2-4238-a70d-064316d8e341 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:21.904143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a84efb7a2624c82b88a3d176cce5533'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.905390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a84efb7a2624c82b88a3d176cce5533'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:21.919843 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2482/4971] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:21 2022] DELETE /identity/v3/users/6a84efb7a2624c82b88a3d176cce5533 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:21.920773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:21.926367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:21.931290 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2490/4972] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:21.932393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189dba-9ef2-4238-a70d-064316d8e341 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:21.936121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ba189dba-9ef2-4238-a70d-064316d8e341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:21.963333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189dba-9ef2-4238-a70d-064316d8e341 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:22.014009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:22.022550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:22.068575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17804e89-1c71-472e-8f78-6960123ec168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:22.069596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17804e89-1c71-472e-8f78-6960123ec168 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:22.071630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17804e89-1c71-472e-8f78-6960123ec168 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:22.072589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17804e89-1c71-472e-8f78-6960123ec168 admin admin] PATH_INFO: `/v3/projects/9caf8063f9b74cf887f85e59923e7522/users/f02f11d363764208bb0c0a7be5a7e705/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:22.085786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29458e00-19a3-4876-8d20-c523013e7735 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:22.098913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa36f4-c46e-4971-86f7-fffe810b23f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:22.099964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa36f4-c46e-4971-86f7-fffe810b23f9 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:22.101501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa36f4-c46e-4971-86f7-fffe810b23f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:22.102741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa36f4-c46e-4971-86f7-fffe810b23f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:22.108750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b5aa36f4-c46e-4971-86f7-fffe810b23f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:22.115538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17804e89-1c71-472e-8f78-6960123ec168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9caf8063f9b74cf887f85e59923e7522, user_id=f02f11d363764208bb0c0a7be5a7e705, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:22.121058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17804e89-1c71-472e-8f78-6960123ec168 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:22.140483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa36f4-c46e-4971-86f7-fffe810b23f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:22.146118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa36f4-c46e-4971-86f7-fffe810b23f9 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:22.163780 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2491/4973] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:21 2022] PUT /identity/v3/projects/9caf8063f9b74cf887f85e59923e7522/users/f02f11d363764208bb0c0a7be5a7e705/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:22.166285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12fe1a-7c8d-4088-89fb-0ba1da7c90e2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:22.168826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12fe1a-7c8d-4088-89fb-0ba1da7c90e2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:22.169313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12fe1a-7c8d-4088-89fb-0ba1da7c90e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:22.183674 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2483/4974] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:22.217898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4f12fe1a-7c8d-4088-89fb-0ba1da7c90e2 None None] MFA Rules not processed for user `f02f11d363764208bb0c0a7be5a7e705`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:22.219340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4f12fe1a-7c8d-4088-89fb-0ba1da7c90e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:22.328865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa36f4-c46e-4971-86f7-fffe810b23f9 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:22.337136 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2492/4975] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:22.343875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa36f4-c46e-4971-86f7-fffe810b23f9 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:22.520496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97e828-6201-4b0f-9076-95d91c6cdff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:22.522066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97e828-6201-4b0f-9076-95d91c6cdff0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:22.523301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97e828-6201-4b0f-9076-95d91c6cdff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:22.523727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97e828-6201-4b0f-9076-95d91c6cdff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:22.526370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97e828-6201-4b0f-9076-95d91c6cdff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:22.530950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97e828-6201-4b0f-9076-95d91c6cdff0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:22.580495 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2484/4976] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:22 2022] POST /identity/v3/projects => generated 369 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:22.584726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12fe1a-7c8d-4088-89fb-0ba1da7c90e2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:22.605957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12fe1a-7c8d-4088-89fb-0ba1da7c90e2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:22.748562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873265f-e88f-47c7-8f65-e9cfce198dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:22.749209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873265f-e88f-47c7-8f65-e9cfce198dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:22.749484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873265f-e88f-47c7-8f65-e9cfce198dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:22.749750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873265f-e88f-47c7-8f65-e9cfce198dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:22.750394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873265f-e88f-47c7-8f65-e9cfce198dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:22.752341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873265f-e88f-47c7-8f65-e9cfce198dd0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:22.803047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97e828-6201-4b0f-9076-95d91c6cdff0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:22.804028 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2493/4977] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:22 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:10:22.817627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97e828-6201-4b0f-9076-95d91c6cdff0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:22.833710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96fb68-f999-4d77-9954-f9b87200d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:22.834321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96fb68-f999-4d77-9954-f9b87200d8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:22.834646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96fb68-f999-4d77-9954-f9b87200d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:22.834919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96fb68-f999-4d77-9954-f9b87200d8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:22.835633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96fb68-f999-4d77-9954-f9b87200d8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:22.837562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96fb68-f999-4d77-9954-f9b87200d8e6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:22.867860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873265f-e88f-47c7-8f65-e9cfce198dd0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:22.868943 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2485/4978] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:10:22.884927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873265f-e88f-47c7-8f65-e9cfce198dd0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:22.895269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0faf2-b9fc-4f5f-a26f-d6f165180524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:22.896191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0faf2-b9fc-4f5f-a26f-d6f165180524 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:22.896461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0faf2-b9fc-4f5f-a26f-d6f165180524 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:22.896738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0faf2-b9fc-4f5f-a26f-d6f165180524 admin admin] PATH_INFO: `/v3/projects/d8380fe6b3fe41bd8a65b8ff0fbf5928/users/af7551dad082418f9cbfba3107a743cf/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:22.923100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0faf2-b9fc-4f5f-a26f-d6f165180524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8380fe6b3fe41bd8a65b8ff0fbf5928, user_id=af7551dad082418f9cbfba3107a743cf, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:22.929137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0faf2-b9fc-4f5f-a26f-d6f165180524 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:22.960744 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2494/4979] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:22 2022] PUT /identity/v3/projects/d8380fe6b3fe41bd8a65b8ff0fbf5928/users/af7551dad082418f9cbfba3107a743cf/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:22.967083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96fb68-f999-4d77-9954-f9b87200d8e6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:23.003764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96fb68-f999-4d77-9954-f9b87200d8e6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:23.014342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0faf2-b9fc-4f5f-a26f-d6f165180524 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:23.016689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-59f0faf2-b9fc-4f5f-a26f-d6f165180524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:23.067413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:23.291339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ce747-b80c-4510-a4d7-e42b0acd89c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:23.292564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ce747-b80c-4510-a4d7-e42b0acd89c2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:23.292996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ce747-b80c-4510-a4d7-e42b0acd89c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:23.294547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ce747-b80c-4510-a4d7-e42b0acd89c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:23.295732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ce747-b80c-4510-a4d7-e42b0acd89c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:23.297715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ce747-b80c-4510-a4d7-e42b0acd89c2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:23.298597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0faf2-b9fc-4f5f-a26f-d6f165180524 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:23.314417 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2486/4980] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:10:23.315895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ce747-b80c-4510-a4d7-e42b0acd89c2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:23.317812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:23.319918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:23.320215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:23.320493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] PATH_INFO: `/v3/projects/f66301fa61f649fc8016a81d6700dfa7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:23.323430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f66301fa61f649fc8016a81d6700dfa7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:23.329026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ce747-b80c-4510-a4d7-e42b0acd89c2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:23.329396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:23.335711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c9c6e-43bd-4bae-9775-58e44d59a725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:23.337550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c9c6e-43bd-4bae-9775-58e44d59a725 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:23.338235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c9c6e-43bd-4bae-9775-58e44d59a725 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:23.338656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c9c6e-43bd-4bae-9775-58e44d59a725 admin admin] PATH_INFO: `/v3/projects/d8380fe6b3fe41bd8a65b8ff0fbf5928/users/af7551dad082418f9cbfba3107a743cf/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:23.341020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c9c6e-43bd-4bae-9775-58e44d59a725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8380fe6b3fe41bd8a65b8ff0fbf5928, user_id=af7551dad082418f9cbfba3107a743cf, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:23.343961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c9c6e-43bd-4bae-9775-58e44d59a725 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:23.362504 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2487/4981] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:23 2022] PUT /identity/v3/projects/d8380fe6b3fe41bd8a65b8ff0fbf5928/users/af7551dad082418f9cbfba3107a743cf/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:23.375983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391724a0-c77c-43e3-9cbe-962140d2c30c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:23.377259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391724a0-c77c-43e3-9cbe-962140d2c30c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:23.378305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391724a0-c77c-43e3-9cbe-962140d2c30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:23.447442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-391724a0-c77c-43e3-9cbe-962140d2c30c None None] MFA Rules not processed for user `af7551dad082418f9cbfba3107a743cf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:23.449536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-391724a0-c77c-43e3-9cbe-962140d2c30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:23.467194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:23.503001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f66301fa61f649fc8016a81d6700dfa7', 'request_id': 'req-6699a465-c1a1-4098-b589-849693c61705'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:23.521962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f66301fa61f649fc8016a81d6700dfa7', 'request_id': 'req-6699a465-c1a1-4098-b589-849693c61705'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:23.536902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f66301fa61f649fc8016a81d6700dfa7', 'request_id': 'req-6699a465-c1a1-4098-b589-849693c61705'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:23.554039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] The token cache is being invalidate because project f66301fa61f649fc8016a81d6700dfa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:23.554659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:23.556486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:23.562906 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2495/4982] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:23 2022] DELETE /identity/v3/projects/f66301fa61f649fc8016a81d6700dfa7 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:23.563910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:23.565572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:23.576177 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2488/4983] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:23.577938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391724a0-c77c-43e3-9cbe-962140d2c30c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:23.601488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391724a0-c77c-43e3-9cbe-962140d2c30c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:23.610549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f54a7-85a0-4e59-80f7-8f82ac79df87 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:23.611516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f54a7-85a0-4e59-80f7-8f82ac79df87 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:23.611953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f54a7-85a0-4e59-80f7-8f82ac79df87 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:23.612967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f54a7-85a0-4e59-80f7-8f82ac79df87 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:23.614490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f54a7-85a0-4e59-80f7-8f82ac79df87 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:23.616659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f54a7-85a0-4e59-80f7-8f82ac79df87 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:23.646775 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2489/4984] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:10:23 2022] POST /identity/v3/domains => generated 275 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:23.651493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f54a7-85a0-4e59-80f7-8f82ac79df87 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:23.654808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-439f54a7-85a0-4e59-80f7-8f82ac79df87 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:23.827446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699a465-c1a1-4098-b589-849693c61705 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:23.855465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f54a7-85a0-4e59-80f7-8f82ac79df87 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:23.872882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfe6b9-d3ec-46f8-8019-9f9263d41920 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:23.873545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfe6b9-d3ec-46f8-8019-9f9263d41920 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:23.873824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfe6b9-d3ec-46f8-8019-9f9263d41920 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:23.874100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfe6b9-d3ec-46f8-8019-9f9263d41920 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:23.874741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfe6b9-d3ec-46f8-8019-9f9263d41920 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:23.876657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfe6b9-d3ec-46f8-8019-9f9263d41920 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:23.963957 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2490/4985] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:23 2022] POST /identity/v3/groups => generated 279 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:10:23.963957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfe6b9-d3ec-46f8-8019-9f9263d41920 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:23.970498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7dcfe6b9-d3ec-46f8-8019-9f9263d41920 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:23.982643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:23.990678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:23.997876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:23.998341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:23.998618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] PATH_INFO: `/v3/users/126881bf487d4e0783a5bdd86cbad261` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:24.004425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126881bf487d4e0783a5bdd86cbad261)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:24.007211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:24.129068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126881bf487d4e0783a5bdd86cbad261'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.133211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfe6b9-d3ec-46f8-8019-9f9263d41920 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:24.138900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126881bf487d4e0783a5bdd86cbad261'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.139882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:24.140586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:24.140863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:24.141252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] PATH_INFO: `/v3/projects/5aacf5263bbb4c0e8f97c65cb3d7122f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:24.146140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aacf5263bbb4c0e8f97c65cb3d7122f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:24.148075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:24.166694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126881bf487d4e0783a5bdd86cbad261'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.179652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126881bf487d4e0783a5bdd86cbad261'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.180631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126881bf487d4e0783a5bdd86cbad261'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.196312 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2496/4986] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:23 2022] DELETE /identity/v3/users/126881bf487d4e0783a5bdd86cbad261 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:24.196939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:24.215232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:24.262221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:24.328360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aacf5263bbb4c0e8f97c65cb3d7122f', 'request_id': 'req-a7737f2a-afb9-4052-bb18-0d5480251287'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.357376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aacf5263bbb4c0e8f97c65cb3d7122f', 'request_id': 'req-a7737f2a-afb9-4052-bb18-0d5480251287'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.373190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aacf5263bbb4c0e8f97c65cb3d7122f', 'request_id': 'req-a7737f2a-afb9-4052-bb18-0d5480251287'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.400472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] The token cache is being invalidate because project 5aacf5263bbb4c0e8f97c65cb3d7122f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:24.401252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.404305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.410869 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2491/4987] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:23 2022] DELETE /identity/v3/projects/5aacf5263bbb4c0e8f97c65cb3d7122f => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:24.414136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:24.419138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:24.461539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3ce42-06ec-4ca5-a7ca-f6ec49aa4791 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:24.591453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7737f2a-afb9-4052-bb18-0d5480251287 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:24.605233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:24.606794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:24.608277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:24.608659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] PATH_INFO: `/v3/projects/c13c7a93d1914c388b7b189464539b94` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:24.611720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c13c7a93d1914c388b7b189464539b94)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:24.612327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d1bc1-2eac-4c86-929f-ab59a07a0585 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:24.615206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:24.615687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d1bc1-2eac-4c86-929f-ab59a07a0585 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:24.616222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d1bc1-2eac-4c86-929f-ab59a07a0585 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:24.616622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d1bc1-2eac-4c86-929f-ab59a07a0585 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/3b91d2d21f7d4ce295d57e76db4fb2af` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:24.634136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d1bc1-2eac-4c86-929f-ab59a07a0585 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:update_group(group_id=3b91d2d21f7d4ce295d57e76db4fb2af)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:24.635371 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-180d1bc1-2eac-4c86-929f-ab59a07a0585 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:24.635371 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:24.635371 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:24.638908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d1bc1-2eac-4c86-929f-ab59a07a0585 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:24.689572 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2497/4988] 10.222.0.16 () {60 vars in 1247 bytes} [Sat Oct 15 21:10:24 2022] PATCH /identity/v3/groups/3b91d2d21f7d4ce295d57e76db4fb2af => generated 293 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:24.692432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d1bc1-2eac-4c86-929f-ab59a07a0585 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:24.695607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-180d1bc1-2eac-4c86-929f-ab59a07a0585 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:24.719922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d1bc1-2eac-4c86-929f-ab59a07a0585 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:24.740615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:24.751150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c13c7a93d1914c388b7b189464539b94', 'request_id': 'req-f750680b-08b4-41ce-ad97-31b01a835568'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.762287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c13c7a93d1914c388b7b189464539b94', 'request_id': 'req-f750680b-08b4-41ce-ad97-31b01a835568'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.771981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c13c7a93d1914c388b7b189464539b94', 'request_id': 'req-f750680b-08b4-41ce-ad97-31b01a835568'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.788736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] The token cache is being invalidate because project c13c7a93d1914c388b7b189464539b94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:24.789879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.790887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.793698 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2492/4989] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:24 2022] DELETE /identity/v3/projects/c13c7a93d1914c388b7b189464539b94 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:24.794895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:24.798142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:24.841486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:24.842912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:24.843389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:24.843812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] PATH_INFO: `/v3/users/bbddce540e5947328062f83b93d1a077` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:24.847521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbddce540e5947328062f83b93d1a077)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:24.851556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:24.927336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:24.949171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbddce540e5947328062f83b93d1a077'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.966533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbddce540e5947328062f83b93d1a077'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.967003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750680b-08b4-41ce-ad97-31b01a835568 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:24.976627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b27fb-12ff-4cc6-99fe-ef46158dddcf tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:24.979442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbddce540e5947328062f83b93d1a077'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.982538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b27fb-12ff-4cc6-99fe-ef46158dddcf tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:24.982787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b27fb-12ff-4cc6-99fe-ef46158dddcf tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:24.983086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b27fb-12ff-4cc6-99fe-ef46158dddcf tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/3b91d2d21f7d4ce295d57e76db4fb2af` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:24.991910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbddce540e5947328062f83b93d1a077'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:24.993383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-628c0e81-41a9-4ff3-9265-c3c04a3d2347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbddce540e5947328062f83b93d1a077'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:25.003184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b27fb-12ff-4cc6-99fe-ef46158dddcf tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:get_group(group_id=3b91d2d21f7d4ce295d57e76db4fb2af)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:25.003184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b27fb-12ff-4cc6-99fe-ef46158dddcf tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:25.003491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2498/4990] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:24 2022] DELETE /identity/v3/users/bbddce540e5947328062f83b93d1a077 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:25.004560 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2493/4991] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:24 2022] GET /identity/v3/groups/3b91d2d21f7d4ce295d57e76db4fb2af => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:25.008647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b27fb-12ff-4cc6-99fe-ef46158dddcf tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:25.010817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650111a8-a7f9-4dfd-ac55-bd197516c5f8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:25.011102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650111a8-a7f9-4dfd-ac55-bd197516c5f8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:25.011368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650111a8-a7f9-4dfd-ac55-bd197516c5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:25.014646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0f2b27fb-12ff-4cc6-99fe-ef46158dddcf tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:25.039821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b27fb-12ff-4cc6-99fe-ef46158dddcf tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:25.061193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-650111a8-a7f9-4dfd-ac55-bd197516c5f8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:25.063103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-650111a8-a7f9-4dfd-ac55-bd197516c5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:25.176619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299a4c9-4227-4d2e-87f7-3838e5927bd3 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:25.176619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299a4c9-4227-4d2e-87f7-3838e5927bd3 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:25.176619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299a4c9-4227-4d2e-87f7-3838e5927bd3 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:25.177539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299a4c9-4227-4d2e-87f7-3838e5927bd3 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/3b91d2d21f7d4ce295d57e76db4fb2af` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:25.177539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299a4c9-4227-4d2e-87f7-3838e5927bd3 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:update_group(group_id=3b91d2d21f7d4ce295d57e76db4fb2af)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:25.178900 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-d299a4c9-4227-4d2e-87f7-3838e5927bd3 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:25.178900 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:25.178900 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:25.182535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299a4c9-4227-4d2e-87f7-3838e5927bd3 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:25.247317 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2494/4992] 10.222.0.16 () {60 vars in 1246 bytes} [Sat Oct 15 21:10:25 2022] PATCH /identity/v3/groups/3b91d2d21f7d4ce295d57e76db4fb2af => generated 309 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:10:25.253430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1809e-c864-4191-9223-63d605ffd68f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:25.255277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1809e-c864-4191-9223-63d605ffd68f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:25.256940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1809e-c864-4191-9223-63d605ffd68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:25.281760 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2499/4993] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:25.281760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650111a8-a7f9-4dfd-ac55-bd197516c5f8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:25.295873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650111a8-a7f9-4dfd-ac55-bd197516c5f8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:25.305302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb668a63-e0a6-4355-ace0-5ff68c376345 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:25.306103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb668a63-e0a6-4355-ace0-5ff68c376345 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:25.306524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb668a63-e0a6-4355-ace0-5ff68c376345 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:25.307308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb668a63-e0a6-4355-ace0-5ff68c376345 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/3b91d2d21f7d4ce295d57e76db4fb2af` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:25.315966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb668a63-e0a6-4355-ace0-5ff68c376345 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_group(group_id=3b91d2d21f7d4ce295d57e76db4fb2af)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:25.317792 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-eb668a63-e0a6-4355-ace0-5ff68c376345 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:25.317792 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:25.317792 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:25.322195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb668a63-e0a6-4355-ace0-5ff68c376345 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:25.340701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-35e1809e-c864-4191-9223-63d605ffd68f None None] MFA Rules not processed for user `3d2974383cae4b5485ce72b2e24a8dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:25.343160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-35e1809e-c864-4191-9223-63d605ffd68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:25.400952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:25.467124 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2500/4994] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:10:25 2022] DELETE /identity/v3/groups/3b91d2d21f7d4ce295d57e76db4fb2af => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:25.468882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26774-f201-4aa4-b403-119ec9cfa2fb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:25.469344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26774-f201-4aa4-b403-119ec9cfa2fb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:25.469712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26774-f201-4aa4-b403-119ec9cfa2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:25.507362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f0f26774-f201-4aa4-b403-119ec9cfa2fb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:25.510840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f0f26774-f201-4aa4-b403-119ec9cfa2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:25.518856 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2495/4995] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:25.523382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1809e-c864-4191-9223-63d605ffd68f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:25.523382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-35e1809e-c864-4191-9223-63d605ffd68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:25.541029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1809e-c864-4191-9223-63d605ffd68f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:25.636575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:25.637269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:25.639564 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2501/4996] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:25.640822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:25.641147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] PATH_INFO: `/v3/projects/1beeea097efa4380b96e6d8328f994e9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:25.641341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f26774-f201-4aa4-b403-119ec9cfa2fb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:25.642553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1beeea097efa4380b96e6d8328f994e9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:25.645156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:25.661206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:25.756709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1beeea097efa4380b96e6d8328f994e9', 'request_id': 'req-c707310c-32e9-47ca-8e0c-0571969a4879'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:25.775971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1beeea097efa4380b96e6d8328f994e9', 'request_id': 'req-c707310c-32e9-47ca-8e0c-0571969a4879'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:25.783785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f26774-f201-4aa4-b403-119ec9cfa2fb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:25.787711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1beeea097efa4380b96e6d8328f994e9', 'request_id': 'req-c707310c-32e9-47ca-8e0c-0571969a4879'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:25.804027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] The token cache is being invalidate because project 1beeea097efa4380b96e6d8328f994e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:25.804880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:25.805818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:25.807806 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2496/4997] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:25 2022] DELETE /identity/v3/projects/1beeea097efa4380b96e6d8328f994e9 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:25.809176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:25.811287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:25.900193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef6f38-571b-4c54-a560-5cb3c6d06fe2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:25.901628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef6f38-571b-4c54-a560-5cb3c6d06fe2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:25.902305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef6f38-571b-4c54-a560-5cb3c6d06fe2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:25.902927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef6f38-571b-4c54-a560-5cb3c6d06fe2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:25.903789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef6f38-571b-4c54-a560-5cb3c6d06fe2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:25.906703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef6f38-571b-4c54-a560-5cb3c6d06fe2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:25.932491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2502/4998] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:25 2022] POST /identity/v3/groups => generated 325 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:25.933991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8607ea-192b-4bdd-8304-456aadf0f002 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:25.934248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8607ea-192b-4bdd-8304-456aadf0f002 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:25.934518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8607ea-192b-4bdd-8304-456aadf0f002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:25.972981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7e8607ea-192b-4bdd-8304-456aadf0f002 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:25.974666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7e8607ea-192b-4bdd-8304-456aadf0f002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:26.002119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:26.012731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707310c-32e9-47ca-8e0c-0571969a4879 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:26.023360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4a601-59dd-4b47-a5b5-d563f7d758f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:26.024040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4a601-59dd-4b47-a5b5-d563f7d758f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.024315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4a601-59dd-4b47-a5b5-d563f7d758f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.024590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4a601-59dd-4b47-a5b5-d563f7d758f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.026234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf4a601-59dd-4b47-a5b5-d563f7d758f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:26.027287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf4a601-59dd-4b47-a5b5-d563f7d758f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:26.029469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf4a601-59dd-4b47-a5b5-d563f7d758f9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:26.053319 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2497/4999] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:26.056309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4a601-59dd-4b47-a5b5-d563f7d758f9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:26.058647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4bf4a601-59dd-4b47-a5b5-d563f7d758f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:26.080749 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2503/5000] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:26.082052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8607ea-192b-4bdd-8304-456aadf0f002 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:26.083593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7e8607ea-192b-4bdd-8304-456aadf0f002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:26.100540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8607ea-192b-4bdd-8304-456aadf0f002 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:26.107372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284e095-5870-424a-8972-e6078a698c09 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:26.108468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284e095-5870-424a-8972-e6078a698c09 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.108847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284e095-5870-424a-8972-e6078a698c09 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.109276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284e095-5870-424a-8972-e6078a698c09 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.110014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284e095-5870-424a-8972-e6078a698c09 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:26.111987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284e095-5870-424a-8972-e6078a698c09 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:26.161022 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2504/5001] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:26 2022] POST /identity/v3/users => generated 323 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:26.164679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284e095-5870-424a-8972-e6078a698c09 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:26.195089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284e095-5870-424a-8972-e6078a698c09 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:26.209316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87844ea4-d5f9-404d-8edc-2539b5e6114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:26.210010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87844ea4-d5f9-404d-8edc-2539b5e6114d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.211313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87844ea4-d5f9-404d-8edc-2539b5e6114d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.211652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87844ea4-d5f9-404d-8edc-2539b5e6114d admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.212386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87844ea4-d5f9-404d-8edc-2539b5e6114d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:26.219366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87844ea4-d5f9-404d-8edc-2539b5e6114d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:26.262061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4a601-59dd-4b47-a5b5-d563f7d758f9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:26.275308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5546768-365e-41a6-ad4c-4d0817a2a04b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:26.275944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5546768-365e-41a6-ad4c-4d0817a2a04b service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.276207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5546768-365e-41a6-ad4c-4d0817a2a04b service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.276558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5546768-365e-41a6-ad4c-4d0817a2a04b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.278376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b5546768-365e-41a6-ad4c-4d0817a2a04b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:26.281954 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2505/5002] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:26 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:26.283829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59fbc1-9b68-4acb-a05a-b37f6416484c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.284348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59fbc1-9b68-4acb-a05a-b37f6416484c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.284967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59fbc1-9b68-4acb-a05a-b37f6416484c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.297812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:26.310126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-eb59fbc1-9b68-4acb-a05a-b37f6416484c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:26.312396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eb59fbc1-9b68-4acb-a05a-b37f6416484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:26.369726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5546768-365e-41a6-ad4c-4d0817a2a04b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:26.371462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5546768-365e-41a6-ad4c-4d0817a2a04b service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:26.381491 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2498/5003] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:26.382428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5546768-365e-41a6-ad4c-4d0817a2a04b service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:26.394096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5546768-365e-41a6-ad4c-4d0817a2a04b service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:26.449074 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2506/5004] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:26.450679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59fbc1-9b68-4acb-a05a-b37f6416484c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:26.453731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eb59fbc1-9b68-4acb-a05a-b37f6416484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:26.482521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59fbc1-9b68-4acb-a05a-b37f6416484c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:26.496454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:26.497617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.498965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.499924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] PATH_INFO: `/v3/projects/8975c29b118845949d8e00a329378290` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.503114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8975c29b118845949d8e00a329378290)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:26.505573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:26.518538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af1ba5-4496-4998-a0fe-bd8bed1047aa tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:26.519353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af1ba5-4496-4998-a0fe-bd8bed1047aa tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.520884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af1ba5-4496-4998-a0fe-bd8bed1047aa tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.521996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af1ba5-4496-4998-a0fe-bd8bed1047aa tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users/129fccee43e042238e61afb4ff5cb3b2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.568521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af1ba5-4496-4998-a0fe-bd8bed1047aa tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=0c51ebebb5e446ab92340d029519714d, user_id=129fccee43e042238e61afb4ff5cb3b2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:26.569702 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-10af1ba5-4496-4998-a0fe-bd8bed1047aa tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:26.569702 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:26.569702 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:26.571917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af1ba5-4496-4998-a0fe-bd8bed1047aa tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:26.626244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8975c29b118845949d8e00a329378290', 'request_id': 'req-0566ce92-a10d-419f-b763-939924fd019d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:26.635013 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2499/5005] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Oct 15 21:10:26 2022] PUT /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users/129fccee43e042238e61afb4ff5cb3b2 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:26.636708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af1ba5-4496-4998-a0fe-bd8bed1047aa tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:26.644950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8975c29b118845949d8e00a329378290', 'request_id': 'req-0566ce92-a10d-419f-b763-939924fd019d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:26.654785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8975c29b118845949d8e00a329378290', 'request_id': 'req-0566ce92-a10d-419f-b763-939924fd019d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:26.685766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] The token cache is being invalidate because project 8975c29b118845949d8e00a329378290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:26.686044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:26.686803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:26.689194 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2507/5006] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:26 2022] DELETE /identity/v3/projects/8975c29b118845949d8e00a329378290 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:26.693259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:26.694658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:26.703528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:26.735709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af1ba5-4496-4998-a0fe-bd8bed1047aa tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:26.741505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad992e4-1cd1-4f45-a867-52d30fdf1ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:26.742302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad992e4-1cd1-4f45-a867-52d30fdf1ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.743413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad992e4-1cd1-4f45-a867-52d30fdf1ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.743601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad992e4-1cd1-4f45-a867-52d30fdf1ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.744414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad992e4-1cd1-4f45-a867-52d30fdf1ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:26.746576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad992e4-1cd1-4f45-a867-52d30fdf1ef3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:26.787095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2500/5007] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:26 2022] POST /identity/v3/users => generated 283 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:26.788594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad992e4-1cd1-4f45-a867-52d30fdf1ef3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:26.791378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4ad992e4-1cd1-4f45-a867-52d30fdf1ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:26.812280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566ce92-a10d-419f-b763-939924fd019d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:26.815631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:26.822410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf3b19-036f-42ce-b90f-81bbd1047761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:26.823571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf3b19-036f-42ce-b90f-81bbd1047761 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.824046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf3b19-036f-42ce-b90f-81bbd1047761 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.824416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf3b19-036f-42ce-b90f-81bbd1047761 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.825783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf3b19-036f-42ce-b90f-81bbd1047761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:26.826178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf3b19-036f-42ce-b90f-81bbd1047761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:26.829265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf3b19-036f-42ce-b90f-81bbd1047761 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:26.857105 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2508/5008] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:26.857105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf3b19-036f-42ce-b90f-81bbd1047761 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:26.860686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-75cf3b19-036f-42ce-b90f-81bbd1047761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:26.894589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad992e4-1cd1-4f45-a867-52d30fdf1ef3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:26.904797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf3b19-036f-42ce-b90f-81bbd1047761 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:26.905786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acf8e6-0525-435c-bc68-67bae7b9729a tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:26.906730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acf8e6-0525-435c-bc68-67bae7b9729a tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.907108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acf8e6-0525-435c-bc68-67bae7b9729a tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.907558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acf8e6-0525-435c-bc68-67bae7b9729a tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.908301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acf8e6-0525-435c-bc68-67bae7b9729a tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:26.910288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acf8e6-0525-435c-bc68-67bae7b9729a tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:26.910619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:26.911257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.911536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.911805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] PATH_INFO: `/v3/users/f275e67f5dd14885bfed369d4a4fba65` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:26.913115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f275e67f5dd14885bfed369d4a4fba65)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:26.915198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:26.975753 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2501/5009] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:26 2022] POST /identity/v3/users => generated 323 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:26.977609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f088b1-8163-4154-a94c-2a2a1c4df20a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:26.977991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f088b1-8163-4154-a94c-2a2a1c4df20a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:26.978374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f088b1-8163-4154-a94c-2a2a1c4df20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.016885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-67f088b1-8163-4154-a94c-2a2a1c4df20a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:27.018569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-67f088b1-8163-4154-a94c-2a2a1c4df20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:27.024848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f275e67f5dd14885bfed369d4a4fba65'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:27.041357 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2502/5010] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:27.041357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f088b1-8163-4154-a94c-2a2a1c4df20a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.041357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-67f088b1-8163-4154-a94c-2a2a1c4df20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:27.064090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f088b1-8163-4154-a94c-2a2a1c4df20a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.079817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2b807-5e5a-42a9-9489-0b7db5f56bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.082641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f275e67f5dd14885bfed369d4a4fba65'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:27.082820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2b807-5e5a-42a9-9489-0b7db5f56bdb admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.083113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2b807-5e5a-42a9-9489-0b7db5f56bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.083908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2b807-5e5a-42a9-9489-0b7db5f56bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.086073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2b807-5e5a-42a9-9489-0b7db5f56bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.093420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2b807-5e5a-42a9-9489-0b7db5f56bdb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.101533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f275e67f5dd14885bfed369d4a4fba65'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:27.112282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f275e67f5dd14885bfed369d4a4fba65'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:27.113435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f275e67f5dd14885bfed369d4a4fba65'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:27.123532 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2503/5011] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:27.125404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2b807-5e5a-42a9-9489-0b7db5f56bdb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.128496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1aa2b807-5e5a-42a9-9489-0b7db5f56bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:27.131054 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2509/5012] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:26 2022] DELETE /identity/v3/users/f275e67f5dd14885bfed369d4a4fba65 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:27.136976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.141643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:27.151439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2b807-5e5a-42a9-9489-0b7db5f56bdb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.268277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3cf68-9755-45f0-8a9b-070020de6277 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.282309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea668d9e-34f4-4f3e-9509-947e56f3f894 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.282866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea668d9e-34f4-4f3e-9509-947e56f3f894 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.283149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea668d9e-34f4-4f3e-9509-947e56f3f894 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.283420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea668d9e-34f4-4f3e-9509-947e56f3f894 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users/13ab1ff5c9b347d893b6face660e0b03` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.300661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89dec5-f4e5-4a29-a5de-b32e87b1b543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.301542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89dec5-f4e5-4a29-a5de-b32e87b1b543 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.302314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89dec5-f4e5-4a29-a5de-b32e87b1b543 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.302615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89dec5-f4e5-4a29-a5de-b32e87b1b543 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.303308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89dec5-f4e5-4a29-a5de-b32e87b1b543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.307176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89dec5-f4e5-4a29-a5de-b32e87b1b543 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.307815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea668d9e-34f4-4f3e-9509-947e56f3f894 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=0c51ebebb5e446ab92340d029519714d, user_id=13ab1ff5c9b347d893b6face660e0b03)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.309960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea668d9e-34f4-4f3e-9509-947e56f3f894 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.312988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:27.355064 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2510/5013] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Oct 15 21:10:27 2022] PUT /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users/13ab1ff5c9b347d893b6face660e0b03 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:27.356538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e88db5-0811-45fe-aa18-5ca1ac7fb788 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.357122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e88db5-0811-45fe-aa18-5ca1ac7fb788 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.357319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e88db5-0811-45fe-aa18-5ca1ac7fb788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.366612 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2504/5014] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:27 2022] POST /identity/v3/projects => generated 373 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:27.368008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89dec5-f4e5-4a29-a5de-b32e87b1b543 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.369940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ab89dec5-f4e5-4a29-a5de-b32e87b1b543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:27.385079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-47e88db5-0811-45fe-aa18-5ca1ac7fb788 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:27.388029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-47e88db5-0811-45fe-aa18-5ca1ac7fb788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:27.472715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89dec5-f4e5-4a29-a5de-b32e87b1b543 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.478436 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2511/5015] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:27.479253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7c3ab-c62b-4644-8201-2bc9a595a82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.480002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc79a-5d62-465e-b071-1ce5dc5f1994 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.480344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc79a-5d62-465e-b071-1ce5dc5f1994 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.480682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc79a-5d62-465e-b071-1ce5dc5f1994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.480904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7c3ab-c62b-4644-8201-2bc9a595a82c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.481517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7c3ab-c62b-4644-8201-2bc9a595a82c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.481970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7c3ab-c62b-4644-8201-2bc9a595a82c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.482811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7c3ab-c62b-4644-8201-2bc9a595a82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.484840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7c3ab-c62b-4644-8201-2bc9a595a82c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.501975 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2505/5016] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:27.502831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7c3ab-c62b-4644-8201-2bc9a595a82c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.510722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d31dc79a-5d62-465e-b071-1ce5dc5f1994 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:27.512226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d31dc79a-5d62-465e-b071-1ce5dc5f1994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:27.520718 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2512/5017] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:27.524753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc79a-5d62-465e-b071-1ce5dc5f1994 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.534448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:27.535144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc79a-5d62-465e-b071-1ce5dc5f1994 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.541000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e0273-2ae8-466a-b7f7-5a9268fbd4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.541984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e0273-2ae8-466a-b7f7-5a9268fbd4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.542266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e0273-2ae8-466a-b7f7-5a9268fbd4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.542534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e0273-2ae8-466a-b7f7-5a9268fbd4e0 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.543162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e0273-2ae8-466a-b7f7-5a9268fbd4e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.545867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e0273-2ae8-466a-b7f7-5a9268fbd4e0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.574558 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2513/5018] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:27 2022] POST /identity/v3/users => generated 287 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:27.575832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e0273-2ae8-466a-b7f7-5a9268fbd4e0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.594795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e0273-2ae8-466a-b7f7-5a9268fbd4e0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.600719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ef6da-257e-480a-a5eb-e32b90bf3820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.601455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ef6da-257e-480a-a5eb-e32b90bf3820 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.601789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ef6da-257e-480a-a5eb-e32b90bf3820 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.602584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ef6da-257e-480a-a5eb-e32b90bf3820 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.603329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ef6da-257e-480a-a5eb-e32b90bf3820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:27.603816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ef6da-257e-480a-a5eb-e32b90bf3820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.606674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ef6da-257e-480a-a5eb-e32b90bf3820 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.616100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7c3ab-c62b-4644-8201-2bc9a595a82c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.623848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2bc68-ceff-419f-9fca-7f3935bd2c80 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.624485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2bc68-ceff-419f-9fca-7f3935bd2c80 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.624991 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2514/5019] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:27.625977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ef6da-257e-480a-a5eb-e32b90bf3820 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.628016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2bc68-ceff-419f-9fca-7f3935bd2c80 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.628305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2bc68-ceff-419f-9fca-7f3935bd2c80 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.629174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2bc68-ceff-419f-9fca-7f3935bd2c80 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.631024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2bc68-ceff-419f-9fca-7f3935bd2c80 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.641527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ef6da-257e-480a-a5eb-e32b90bf3820 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.651668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f527e-5aed-4698-8ccf-d9655b45bb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.652340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f527e-5aed-4698-8ccf-d9655b45bb12 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.652599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f527e-5aed-4698-8ccf-d9655b45bb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.652834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f527e-5aed-4698-8ccf-d9655b45bb12 admin admin] PATH_INFO: `/v3/projects/eb7796fef9524c4598dd52fb450ad036/users/dec8cd5eb5674682aad90fd42db118e8/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.674507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f527e-5aed-4698-8ccf-d9655b45bb12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb7796fef9524c4598dd52fb450ad036, user_id=dec8cd5eb5674682aad90fd42db118e8, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.674731 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2506/5020] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:27 2022] POST /identity/v3/users => generated 321 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:27.674731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2bc68-ceff-419f-9fca-7f3935bd2c80 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.674930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f527e-5aed-4698-8ccf-d9655b45bb12 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.678068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b5d2bc68-ceff-419f-9fca-7f3935bd2c80 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:27.692293 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2515/5021] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:27 2022] PUT /identity/v3/projects/eb7796fef9524c4598dd52fb450ad036/users/dec8cd5eb5674682aad90fd42db118e8/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:27.693196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f527e-5aed-4698-8ccf-d9655b45bb12 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.777182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2bc68-ceff-419f-9fca-7f3935bd2c80 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.790023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f527e-5aed-4698-8ccf-d9655b45bb12 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.796054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31058d-0b1a-44a1-b619-d7744d4b2d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.796863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31058d-0b1a-44a1-b619-d7744d4b2d15 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.797369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31058d-0b1a-44a1-b619-d7744d4b2d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.797736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31058d-0b1a-44a1-b619-d7744d4b2d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.798464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31058d-0b1a-44a1-b619-d7744d4b2d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.800276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31058d-0b1a-44a1-b619-d7744d4b2d15 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.813144 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2516/5022] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:27.814358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31058d-0b1a-44a1-b619-d7744d4b2d15 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.828106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31058d-0b1a-44a1-b619-d7744d4b2d15 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:27.833900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfdf00-b3c3-4035-80aa-65277dd937e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.834892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfdf00-b3c3-4035-80aa-65277dd937e2 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.835284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfdf00-b3c3-4035-80aa-65277dd937e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.835649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfdf00-b3c3-4035-80aa-65277dd937e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.837708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfdf00-b3c3-4035-80aa-65277dd937e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.839486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b754e55-0d39-450f-bd00-c5706ba8430e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:27.839903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfdf00-b3c3-4035-80aa-65277dd937e2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.840510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b754e55-0d39-450f-bd00-c5706ba8430e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.841001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b754e55-0d39-450f-bd00-c5706ba8430e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.841466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b754e55-0d39-450f-bd00-c5706ba8430e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.855724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b754e55-0d39-450f-bd00-c5706ba8430e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:27.857745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b754e55-0d39-450f-bd00-c5706ba8430e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:27.871609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:27.873072 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2507/5023] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:27.874683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b754e55-0d39-450f-bd00-c5706ba8430e service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:27.893432 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2517/5024] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:27 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:27.895532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d55f7b-c53b-41f5-b644-075cba06d6f7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:27.895532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d55f7b-c53b-41f5-b644-075cba06d6f7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:27.895966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d55f7b-c53b-41f5-b644-075cba06d6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:27.935028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-97d55f7b-c53b-41f5-b644-075cba06d6f7 None None] MFA Rules not processed for user `dec8cd5eb5674682aad90fd42db118e8`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:27.938288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-97d55f7b-c53b-41f5-b644-075cba06d6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:27.969109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:27.976093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b754e55-0d39-450f-bd00-c5706ba8430e service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:28.061446 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2518/5025] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:28.062564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d55f7b-c53b-41f5-b644-075cba06d6f7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:28.106897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd4c5-cef0-4a02-9c53-e669a483ffa5 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:28.107553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd4c5-cef0-4a02-9c53-e669a483ffa5 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:28.107827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd4c5-cef0-4a02-9c53-e669a483ffa5 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:28.111097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd4c5-cef0-4a02-9c53-e669a483ffa5 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users/fb79176288b847d4a65e7403ae72b7b3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:28.140941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cd4c5-cef0-4a02-9c53-e669a483ffa5 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=0c51ebebb5e446ab92340d029519714d, user_id=fb79176288b847d4a65e7403ae72b7b3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:28.143568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cd4c5-cef0-4a02-9c53-e669a483ffa5 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:28.206121 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2508/5026] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Oct 15 21:10:27 2022] PUT /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users/fb79176288b847d4a65e7403ae72b7b3 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:28.206121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd4c5-cef0-4a02-9c53-e669a483ffa5 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:28.223878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d55f7b-c53b-41f5-b644-075cba06d6f7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:28.377362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83bc9f-83e8-4f23-acaa-ffe600da716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:28.377962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83bc9f-83e8-4f23-acaa-ffe600da716a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:28.378226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83bc9f-83e8-4f23-acaa-ffe600da716a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:28.378490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83bc9f-83e8-4f23-acaa-ffe600da716a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:28.379133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83bc9f-83e8-4f23-acaa-ffe600da716a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:28.381558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83bc9f-83e8-4f23-acaa-ffe600da716a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:28.392288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd4c5-cef0-4a02-9c53-e669a483ffa5 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:28.411245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1330ad-5000-4d26-8888-a5d1039090ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:28.412084 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2519/5027] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:28.413265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83bc9f-83e8-4f23-acaa-ffe600da716a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:28.414012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1330ad-5000-4d26-8888-a5d1039090ea admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:28.415145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1330ad-5000-4d26-8888-a5d1039090ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:28.415761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1330ad-5000-4d26-8888-a5d1039090ea admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:28.417390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1330ad-5000-4d26-8888-a5d1039090ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:28.422877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1330ad-5000-4d26-8888-a5d1039090ea admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:28.428900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83bc9f-83e8-4f23-acaa-ffe600da716a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:28.434888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5362-467f-487d-a02f-2f33da019cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:28.435739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5362-467f-487d-a02f-2f33da019cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:28.436191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5362-467f-487d-a02f-2f33da019cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:28.436642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5362-467f-487d-a02f-2f33da019cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:28.437576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d5362-467f-487d-a02f-2f33da019cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:28.439833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d5362-467f-487d-a02f-2f33da019cc5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:28.459514 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2509/5028] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:28 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:28.460901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1330ad-5000-4d26-8888-a5d1039090ea admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:28.504137 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2520/5029] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:28 2022] POST /identity/v3/projects => generated 367 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:28.506017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5362-467f-487d-a02f-2f33da019cc5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:28.532395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:28.602254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1330ad-5000-4d26-8888-a5d1039090ea admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:28.631408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:28.683306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5362-467f-487d-a02f-2f33da019cc5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:28.697785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad1fc9-35cf-46f9-8573-732910e6ccb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:28.698349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad1fc9-35cf-46f9-8573-732910e6ccb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:28.698622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad1fc9-35cf-46f9-8573-732910e6ccb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:28.698891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad1fc9-35cf-46f9-8573-732910e6ccb9 admin admin] PATH_INFO: `/v3/projects/014c5e68788f4050ba767e9cee80889d/users/22d5409ee66f42fea28e7c208b4a08b4/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:28.742318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad1fc9-35cf-46f9-8573-732910e6ccb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014c5e68788f4050ba767e9cee80889d, user_id=22d5409ee66f42fea28e7c208b4a08b4, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:28.754669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad1fc9-35cf-46f9-8573-732910e6ccb9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:28.773294 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2521/5030] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:28 2022] PUT /identity/v3/projects/014c5e68788f4050ba767e9cee80889d/users/22d5409ee66f42fea28e7c208b4a08b4/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:28.775296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad1fc9-35cf-46f9-8573-732910e6ccb9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:28.775929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5fc32-c47a-443b-b57d-91dc3417a4e0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:28.776740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5fc32-c47a-443b-b57d-91dc3417a4e0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:28.777061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5fc32-c47a-443b-b57d-91dc3417a4e0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:28.777339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5fc32-c47a-443b-b57d-91dc3417a4e0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:28.778704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5fc32-c47a-443b-b57d-91dc3417a4e0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=0c51ebebb5e446ab92340d029519714d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:28.779831 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-bfb5fc32-c47a-443b-b57d-91dc3417a4e0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:28.779831 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:28.779831 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:28.781700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5fc32-c47a-443b-b57d-91dc3417a4e0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:28.819517 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2510/5031] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:10:28 2022] GET /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users => generated 1083 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:28.822405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5fc32-c47a-443b-b57d-91dc3417a4e0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:28.846444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5fc32-c47a-443b-b57d-91dc3417a4e0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:28.922402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad1fc9-35cf-46f9-8573-732910e6ccb9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:28.936565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d956f0-0955-47d4-aedb-62577527254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:28.937216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d956f0-0955-47d4-aedb-62577527254b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:28.937493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d956f0-0955-47d4-aedb-62577527254b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:28.937763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d956f0-0955-47d4-aedb-62577527254b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:28.938413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d956f0-0955-47d4-aedb-62577527254b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:28.940986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d956f0-0955-47d4-aedb-62577527254b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:28.960520 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2522/5032] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:28.961516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d956f0-0955-47d4-aedb-62577527254b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:28.966531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d3d956f0-0955-47d4-aedb-62577527254b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:28.984749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d956f0-0955-47d4-aedb-62577527254b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:28.987282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418eaf26-58f6-444f-8ced-664cfcd3a888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:28.987994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418eaf26-58f6-444f-8ced-664cfcd3a888 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:28.988556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418eaf26-58f6-444f-8ced-664cfcd3a888 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:28.989119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418eaf26-58f6-444f-8ced-664cfcd3a888 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:28.990117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418eaf26-58f6-444f-8ced-664cfcd3a888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:28.991719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:28.992582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418eaf26-58f6-444f-8ced-664cfcd3a888 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:28.993584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:28.994055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:28.994531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] PATH_INFO: `/v3/projects/d5ce316f5cf94936b501265f6a8ac463` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:28.996380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ce316f5cf94936b501265f6a8ac463)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:28.998792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:29.048615 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2511/5033] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:28 2022] POST /identity/v3/users => generated 290 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:29.051484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30cc39-b0a9-4e65-96e7-a0ae9efec8cc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:29.052081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30cc39-b0a9-4e65-96e7-a0ae9efec8cc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:29.052793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30cc39-b0a9-4e65-96e7-a0ae9efec8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:29.090388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ce316f5cf94936b501265f6a8ac463', 'request_id': 'req-a323231f-4741-4667-8247-bc44fd444137'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.105905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ce316f5cf94936b501265f6a8ac463', 'request_id': 'req-a323231f-4741-4667-8247-bc44fd444137'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.115101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-aa30cc39-b0a9-4e65-96e7-a0ae9efec8cc None None] MFA Rules not processed for user `22d5409ee66f42fea28e7c208b4a08b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:29.116507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ce316f5cf94936b501265f6a8ac463', 'request_id': 'req-a323231f-4741-4667-8247-bc44fd444137'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.118660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aa30cc39-b0a9-4e65-96e7-a0ae9efec8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:29.132545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] The token cache is being invalidate because project d5ce316f5cf94936b501265f6a8ac463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:29.132911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.133737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.139404 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2523/5034] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:28 2022] DELETE /identity/v3/projects/d5ce316f5cf94936b501265f6a8ac463 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:29.140367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:29.142558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:29.162686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:29.265752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323231f-4741-4667-8247-bc44fd444137 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:29.271923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08f28b-2488-4e82-aab3-a497c6d9ae81 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:29.272588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08f28b-2488-4e82-aab3-a497c6d9ae81 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:29.272870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08f28b-2488-4e82-aab3-a497c6d9ae81 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:29.273180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08f28b-2488-4e82-aab3-a497c6d9ae81 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users/129fccee43e042238e61afb4ff5cb3b2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:29.276152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08f28b-2488-4e82-aab3-a497c6d9ae81 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=0c51ebebb5e446ab92340d029519714d, user_id=129fccee43e042238e61afb4ff5cb3b2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:29.277374 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-fd08f28b-2488-4e82-aab3-a497c6d9ae81 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:29.277374 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:29.277374 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:29.278783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08f28b-2488-4e82-aab3-a497c6d9ae81 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:29.304569 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2512/5035] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:29.308799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30cc39-b0a9-4e65-96e7-a0ae9efec8cc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:29.311939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aa30cc39-b0a9-4e65-96e7-a0ae9efec8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:29.331134 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2524/5036] 10.222.0.16 () {58 vars in 1342 bytes} [Sat Oct 15 21:10:29 2022] HEAD /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users/129fccee43e042238e61afb4ff5cb3b2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:29.332094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08f28b-2488-4e82-aab3-a497c6d9ae81 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:29.334593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fd08f28b-2488-4e82-aab3-a497c6d9ae81 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:29.406783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:29.452861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30cc39-b0a9-4e65-96e7-a0ae9efec8cc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:29.455364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08f28b-2488-4e82-aab3-a497c6d9ae81 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:29.463163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47352ad6-0fce-4d91-a490-93acac1db2bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:29.463617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47352ad6-0fce-4d91-a490-93acac1db2bf service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:29.463859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47352ad6-0fce-4d91-a490-93acac1db2bf service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:29.464120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47352ad6-0fce-4d91-a490-93acac1db2bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:29.466040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436826a8-828e-4535-8bc5-87aa1bb45f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:29.466191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-47352ad6-0fce-4d91-a490-93acac1db2bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:29.466658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436826a8-828e-4535-8bc5-87aa1bb45f7e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:29.466941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436826a8-828e-4535-8bc5-87aa1bb45f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:29.467212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436826a8-828e-4535-8bc5-87aa1bb45f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:29.467858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436826a8-828e-4535-8bc5-87aa1bb45f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:29.471656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436826a8-828e-4535-8bc5-87aa1bb45f7e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:29.493426 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2525/5037] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:29.501137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436826a8-828e-4535-8bc5-87aa1bb45f7e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:29.503553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-436826a8-828e-4535-8bc5-87aa1bb45f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:29.529347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436826a8-828e-4535-8bc5-87aa1bb45f7e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:29.536705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424b63-db1f-4b38-b9cd-a0cf3e4a9088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:29.537409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424b63-db1f-4b38-b9cd-a0cf3e4a9088 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:29.537692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424b63-db1f-4b38-b9cd-a0cf3e4a9088 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:29.537960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424b63-db1f-4b38-b9cd-a0cf3e4a9088 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:29.538600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf424b63-db1f-4b38-b9cd-a0cf3e4a9088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:29.540985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf424b63-db1f-4b38-b9cd-a0cf3e4a9088 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:29.563070 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2526/5038] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:29.563944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424b63-db1f-4b38-b9cd-a0cf3e4a9088 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:29.567450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bf424b63-db1f-4b38-b9cd-a0cf3e4a9088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:29.588068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424b63-db1f-4b38-b9cd-a0cf3e4a9088 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:29.599449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:29.600331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:29.600706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:29.601109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] PATH_INFO: `/v3/users/d4614972e267487db45505f2ae0a91e0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:29.614419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4614972e267487db45505f2ae0a91e0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:29.621940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:29.622416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47352ad6-0fce-4d91-a490-93acac1db2bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:29.624194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47352ad6-0fce-4d91-a490-93acac1db2bf service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:29.656330 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2513/5039] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:29.657228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47352ad6-0fce-4d91-a490-93acac1db2bf service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:29.677566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47352ad6-0fce-4d91-a490-93acac1db2bf service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:29.684833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c09d48-e91f-4e7c-b000-3dd8184eb71d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:29.685593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c09d48-e91f-4e7c-b000-3dd8184eb71d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:29.685881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c09d48-e91f-4e7c-b000-3dd8184eb71d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:29.686443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c09d48-e91f-4e7c-b000-3dd8184eb71d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users/129fccee43e042238e61afb4ff5cb3b2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:29.688379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c09d48-e91f-4e7c-b000-3dd8184eb71d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=0c51ebebb5e446ab92340d029519714d, user_id=129fccee43e042238e61afb4ff5cb3b2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:29.693499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c09d48-e91f-4e7c-b000-3dd8184eb71d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:29.708300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40c09d48-e91f-4e7c-b000-3dd8184eb71d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '129fccee43e042238e61afb4ff5cb3b2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.731733 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2514/5040] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Oct 15 21:10:29 2022] DELETE /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users/129fccee43e042238e61afb4ff5cb3b2 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:29.733638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c09d48-e91f-4e7c-b000-3dd8184eb71d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:29.735627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-40c09d48-e91f-4e7c-b000-3dd8184eb71d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:29.750430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4614972e267487db45505f2ae0a91e0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.768205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4614972e267487db45505f2ae0a91e0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.787802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4614972e267487db45505f2ae0a91e0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.808493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4614972e267487db45505f2ae0a91e0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.809842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4614972e267487db45505f2ae0a91e0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:29.815852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:29.834745 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2527/5041] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:29 2022] DELETE /identity/v3/users/d4614972e267487db45505f2ae0a91e0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:29.837528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:29.840218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:29.886154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c09d48-e91f-4e7c-b000-3dd8184eb71d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:29.963411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11cb0f-b011-4373-be24-bd87eb633d2a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:29.974168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8165827-9c0a-43f3-8eb9-66335b901731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:29.974905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8165827-9c0a-43f3-8eb9-66335b901731 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:29.975280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8165827-9c0a-43f3-8eb9-66335b901731 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:29.975602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8165827-9c0a-43f3-8eb9-66335b901731 admin admin] PATH_INFO: `/v3/projects/7581df74c9854212a83bae756249a009/users/fdf5a33dacf94c2684bca52f9d5ff8a0/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:29.995860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7120d-1160-49b6-b0e1-d163557ff7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:29.997792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7120d-1160-49b6-b0e1-d163557ff7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:29.999205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7120d-1160-49b6-b0e1-d163557ff7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:29.999607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7120d-1160-49b6-b0e1-d163557ff7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:30.002641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8165827-9c0a-43f3-8eb9-66335b901731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7581df74c9854212a83bae756249a009, user_id=fdf5a33dacf94c2684bca52f9d5ff8a0, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:30.006456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8165827-9c0a-43f3-8eb9-66335b901731 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:30.009526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1dc7120d-1160-49b6-b0e1-d163557ff7f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:30.052775 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2528/5042] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:29 2022] PUT /identity/v3/projects/7581df74c9854212a83bae756249a009/users/fdf5a33dacf94c2684bca52f9d5ff8a0/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:30.052775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8165827-9c0a-43f3-8eb9-66335b901731 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:30.052775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e8165827-9c0a-43f3-8eb9-66335b901731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:30.059181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:30.154282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7120d-1160-49b6-b0e1-d163557ff7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:30.156621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7120d-1160-49b6-b0e1-d163557ff7f8 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:30.180468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8165827-9c0a-43f3-8eb9-66335b901731 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:30.181998 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2515/5043] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:30.182927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7120d-1160-49b6-b0e1-d163557ff7f8 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:30.185469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1dc7120d-1160-49b6-b0e1-d163557ff7f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:30.196793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13195732-28a5-45c1-bbf5-3bbd2e64b35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:30.199041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13195732-28a5-45c1-bbf5-3bbd2e64b35d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:30.199547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13195732-28a5-45c1-bbf5-3bbd2e64b35d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:30.200224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13195732-28a5-45c1-bbf5-3bbd2e64b35d admin admin] PATH_INFO: `/v3/projects/5228810b5b2a41249b055cbcd0705515/users/7bd5e0fd5b6a456e879af06396442556/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:30.205948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7120d-1160-49b6-b0e1-d163557ff7f8 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:30.241817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13195732-28a5-45c1-bbf5-3bbd2e64b35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5228810b5b2a41249b055cbcd0705515, user_id=7bd5e0fd5b6a456e879af06396442556, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:30.249933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13195732-28a5-45c1-bbf5-3bbd2e64b35d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:30.280695 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2529/5044] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:30 2022] PUT /identity/v3/projects/5228810b5b2a41249b055cbcd0705515/users/7bd5e0fd5b6a456e879af06396442556/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:30.282058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13195732-28a5-45c1-bbf5-3bbd2e64b35d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:30.286291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-13195732-28a5-45c1-bbf5-3bbd2e64b35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:30.331113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f39af6-755b-4668-a602-a8a8e6b09614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:30.332799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39af6-755b-4668-a602-a8a8e6b09614 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:30.343271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39af6-755b-4668-a602-a8a8e6b09614 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:30.343271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39af6-755b-4668-a602-a8a8e6b09614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:30.346412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b3f39af6-755b-4668-a602-a8a8e6b09614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-b3f39af6-755b-4668-a602-a8a8e6b09614 service neutron] Could not find project: f400a79cb647438f968a895142c5410d.: keystone.exception.ProjectNotFound: Could not find project: f400a79cb647438f968a895142c5410d. Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 15 21:10:30.413828 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:10:30.416081 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:10:30.417375 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:10:30.418599 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:10:30.419767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f400a79cb647438f968a895142c5410d. Oct 15 21:10:30.419767 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:30.419767 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2516/5045] 10.222.0.16 () {60 vars in 1396 bytes} [Sat Oct 15 21:10:30 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 113 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:10:30.420025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec08229-3762-4aa9-b7a0-5ef7d4f9f6d9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:30.421744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec08229-3762-4aa9-b7a0-5ef7d4f9f6d9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:30.421892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec08229-3762-4aa9-b7a0-5ef7d4f9f6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:30.444818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13195732-28a5-45c1-bbf5-3bbd2e64b35d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:30.457058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204947c8-3d78-4b7f-b11b-aa41ba603be0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:30.457970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204947c8-3d78-4b7f-b11b-aa41ba603be0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:30.458251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204947c8-3d78-4b7f-b11b-aa41ba603be0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:30.458524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204947c8-3d78-4b7f-b11b-aa41ba603be0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users/13ab1ff5c9b347d893b6face660e0b03` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:30.460860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204947c8-3d78-4b7f-b11b-aa41ba603be0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=0c51ebebb5e446ab92340d029519714d, user_id=13ab1ff5c9b347d893b6face660e0b03)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:30.465645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204947c8-3d78-4b7f-b11b-aa41ba603be0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:30.496305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3ec08229-3762-4aa9-b7a0-5ef7d4f9f6d9 None None] MFA Rules not processed for user `fdf5a33dacf94c2684bca52f9d5ff8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:30.500162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3ec08229-3762-4aa9-b7a0-5ef7d4f9f6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:30.507652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:30.517294 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2530/5046] 10.222.0.16 () {58 vars in 1342 bytes} [Sat Oct 15 21:10:30 2022] HEAD /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users/13ab1ff5c9b347d893b6face660e0b03 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:30.518784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204947c8-3d78-4b7f-b11b-aa41ba603be0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:30.697577 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2517/5047] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:30.699415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec08229-3762-4aa9-b7a0-5ef7d4f9f6d9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:30.710062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204947c8-3d78-4b7f-b11b-aa41ba603be0 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:30.728572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb295a5e-2ea1-4c14-b2f3-8bada9e9a248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:30.729340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb295a5e-2ea1-4c14-b2f3-8bada9e9a248 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:30.729620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb295a5e-2ea1-4c14-b2f3-8bada9e9a248 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:30.729883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb295a5e-2ea1-4c14-b2f3-8bada9e9a248 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:30.730604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb295a5e-2ea1-4c14-b2f3-8bada9e9a248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:30.732141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb295a5e-2ea1-4c14-b2f3-8bada9e9a248 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:30.734152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec08229-3762-4aa9-b7a0-5ef7d4f9f6d9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:30.751943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2595cc-603c-4a46-a394-37e01b8037eb tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:30.752866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2595cc-603c-4a46-a394-37e01b8037eb tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:30.755728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2595cc-603c-4a46-a394-37e01b8037eb tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:30.756472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2595cc-603c-4a46-a394-37e01b8037eb tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users/13ab1ff5c9b347d893b6face660e0b03` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:30.756682 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2531/5048] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:30.758243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6c674-ac9e-45a6-89a5-0e4d40c5eb12 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:30.758553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6c674-ac9e-45a6-89a5-0e4d40c5eb12 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:30.758975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6c674-ac9e-45a6-89a5-0e4d40c5eb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:30.765950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2595cc-603c-4a46-a394-37e01b8037eb tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=0c51ebebb5e446ab92340d029519714d, user_id=13ab1ff5c9b347d893b6face660e0b03)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:30.784103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2595cc-603c-4a46-a394-37e01b8037eb tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:30.822952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ee2595cc-603c-4a46-a394-37e01b8037eb tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '13ab1ff5c9b347d893b6face660e0b03'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:30.855299 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2518/5049] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Oct 15 21:10:30 2022] DELETE /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users/13ab1ff5c9b347d893b6face660e0b03 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:30.865635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2595cc-603c-4a46-a394-37e01b8037eb tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:30.877409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ee2595cc-603c-4a46-a394-37e01b8037eb tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:30.885729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c6e6c674-ac9e-45a6-89a5-0e4d40c5eb12 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:30.886977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c6e6c674-ac9e-45a6-89a5-0e4d40c5eb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:30.995575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:31.047467 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2532/5050] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:10:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:31.048614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6c674-ac9e-45a6-89a5-0e4d40c5eb12 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:31.062436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2595cc-603c-4a46-a394-37e01b8037eb tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:31.073294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0d65-be04-4553-b8ff-f21c59b0201b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:31.073925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0d65-be04-4553-b8ff-f21c59b0201b service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:31.074202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0d65-be04-4553-b8ff-f21c59b0201b service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:31.074471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0d65-be04-4553-b8ff-f21c59b0201b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:31.140932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6c674-ac9e-45a6-89a5-0e4d40c5eb12 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:31.147847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f5911-8807-44db-8e8a-f295e47ffe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:31.148664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f5911-8807-44db-8e8a-f295e47ffe0b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:31.149283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f5911-8807-44db-8e8a-f295e47ffe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:31.149716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f5911-8807-44db-8e8a-f295e47ffe0b admin admin] PATH_INFO: `/v3/projects/5228810b5b2a41249b055cbcd0705515/users/7bd5e0fd5b6a456e879af06396442556/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:31.166268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f5911-8807-44db-8e8a-f295e47ffe0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5228810b5b2a41249b055cbcd0705515, user_id=7bd5e0fd5b6a456e879af06396442556, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:31.177794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f5911-8807-44db-8e8a-f295e47ffe0b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:31.178004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea0d65-be04-4553-b8ff-f21c59b0201b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:31.178004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea0d65-be04-4553-b8ff-f21c59b0201b service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:31.188695 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2519/5051] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:31.189600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0d65-be04-4553-b8ff-f21c59b0201b service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:31.212960 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2533/5052] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:31 2022] PUT /identity/v3/projects/5228810b5b2a41249b055cbcd0705515/users/7bd5e0fd5b6a456e879af06396442556/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:31.212960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f5911-8807-44db-8e8a-f295e47ffe0b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:31.214112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-203f5911-8807-44db-8e8a-f295e47ffe0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:31.227058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:31.309228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0d65-be04-4553-b8ff-f21c59b0201b service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:31.323334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f5911-8807-44db-8e8a-f295e47ffe0b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:31.329302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:31.329946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:31.330222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:31.330492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] PATH_INFO: `/v3/projects/60b889e1c32345f2aeba98ef12933717` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:31.334406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b889e1c32345f2aeba98ef12933717)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:31.337850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:31.393742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf97b47-c005-475b-8716-91471c123420 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:31.394510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf97b47-c005-475b-8716-91471c123420 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:31.394908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf97b47-c005-475b-8716-91471c123420 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:31.395324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf97b47-c005-475b-8716-91471c123420 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users/fb79176288b847d4a65e7403ae72b7b3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:31.397815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf97b47-c005-475b-8716-91471c123420 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=0c51ebebb5e446ab92340d029519714d, user_id=fb79176288b847d4a65e7403ae72b7b3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:31.399134 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-acf97b47-c005-475b-8716-91471c123420 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:31.399134 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:31.399134 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:31.401188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf97b47-c005-475b-8716-91471c123420 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:31.425752 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2520/5053] 10.222.0.16 () {58 vars in 1342 bytes} [Sat Oct 15 21:10:31 2022] HEAD /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users/fb79176288b847d4a65e7403ae72b7b3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:31.425752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf97b47-c005-475b-8716-91471c123420 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:31.438555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b889e1c32345f2aeba98ef12933717', 'request_id': 'req-fcc5120b-31e7-4367-af81-49f55ace604c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:31.459260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b889e1c32345f2aeba98ef12933717', 'request_id': 'req-fcc5120b-31e7-4367-af81-49f55ace604c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:31.474968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b889e1c32345f2aeba98ef12933717', 'request_id': 'req-fcc5120b-31e7-4367-af81-49f55ace604c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:31.500254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] The token cache is being invalidate because project 60b889e1c32345f2aeba98ef12933717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:31.501243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:31.502736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcc5120b-31e7-4367-af81-49f55ace604c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:31.506564 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2534/5054] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:31 2022] DELETE /identity/v3/projects/60b889e1c32345f2aeba98ef12933717 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:31.507623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:31.510679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676335f-1708-40b6-a9ef-76e32a088abc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:31.511811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676335f-1708-40b6-a9ef-76e32a088abc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:31.512567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676335f-1708-40b6-a9ef-76e32a088abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:31.562698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf97b47-c005-475b-8716-91471c123420 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:31.581533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0676335f-1708-40b6-a9ef-76e32a088abc None None] MFA Rules not processed for user `7bd5e0fd5b6a456e879af06396442556`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:31.585205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0676335f-1708-40b6-a9ef-76e32a088abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:31.657966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3cd4f-aae7-4cde-8176-d75e8befcad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:31.658697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3cd4f-aae7-4cde-8176-d75e8befcad7 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:31.658973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3cd4f-aae7-4cde-8176-d75e8befcad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:31.659235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3cd4f-aae7-4cde-8176-d75e8befcad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:31.661001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-57a3cd4f-aae7-4cde-8176-d75e8befcad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:31.742348 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2535/5055] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:31.743457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676335f-1708-40b6-a9ef-76e32a088abc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:31.745483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0676335f-1708-40b6-a9ef-76e32a088abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:31.766709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3cd4f-aae7-4cde-8176-d75e8befcad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:31.768175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3cd4f-aae7-4cde-8176-d75e8befcad7 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:31.776796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:31.782510 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2521/5056] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:31.783588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3cd4f-aae7-4cde-8176-d75e8befcad7 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:31.785282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-57a3cd4f-aae7-4cde-8176-d75e8befcad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:31.826011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676335f-1708-40b6-a9ef-76e32a088abc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:31.830903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:31.831603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:31.832037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:31.832337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users/fb79176288b847d4a65e7403ae72b7b3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:31.834175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=0c51ebebb5e446ab92340d029519714d, user_id=fb79176288b847d4a65e7403ae72b7b3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:31.835246 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:31.835246 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:31.835246 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:31.837076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:31.846421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'fb79176288b847d4a65e7403ae72b7b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:31.868986 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2536/5057] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Oct 15 21:10:31 2022] DELETE /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users/fb79176288b847d4a65e7403ae72b7b3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:31.872526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:31.874240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:31.879060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3cd4f-aae7-4cde-8176-d75e8befcad7 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:31.916178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:31.972142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba8d3a-83fb-42d0-bfc0-43222badcc37 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:31.980725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:31.981511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:31.981859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:31.982202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] PATH_INFO: `/v3/users/22d5409ee66f42fea28e7c208b4a08b4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:31.983673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d5409ee66f42fea28e7c208b4a08b4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:31.985895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:31.986504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:31.987328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:31.987751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:31.988165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] PATH_INFO: `/v3/users/9e6a3bbcd47d40d08a8e0650de3380cb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:31.989978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e6a3bbcd47d40d08a8e0650de3380cb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:31.992237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:32.098722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d5409ee66f42fea28e7c208b4a08b4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.100919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e6a3bbcd47d40d08a8e0650de3380cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.114545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e6a3bbcd47d40d08a8e0650de3380cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.119682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d5409ee66f42fea28e7c208b4a08b4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.134535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d5409ee66f42fea28e7c208b4a08b4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.143934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d5409ee66f42fea28e7c208b4a08b4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.144704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d5409ee66f42fea28e7c208b4a08b4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.153873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e6a3bbcd47d40d08a8e0650de3380cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.171538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e6a3bbcd47d40d08a8e0650de3380cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.172401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e6a3bbcd47d40d08a8e0650de3380cb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.172576 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2537/5058] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:31 2022] DELETE /identity/v3/users/22d5409ee66f42fea28e7c208b4a08b4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:32.173617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:32.175315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:32.183727 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2522/5059] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:31 2022] DELETE /identity/v3/users/9e6a3bbcd47d40d08a8e0650de3380cb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:32.185102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:32.186850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:32.237848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:32.310791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba36f43-bbe0-4733-b69b-fa6580bba00c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:32.316756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f50b4f-138e-4174-9fc2-c53c9f5917a2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:32.325236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08df67-3766-4214-9ed0-845eae242961 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:32.326311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08df67-3766-4214-9ed0-845eae242961 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:32.326880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08df67-3766-4214-9ed0-845eae242961 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:32.327384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08df67-3766-4214-9ed0-845eae242961 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:32.328695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08df67-3766-4214-9ed0-845eae242961 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:32.332822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08df67-3766-4214-9ed0-845eae242961 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:32.368866 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2523/5060] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:32 2022] POST /identity/v3/roles => generated 235 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:32.370755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08df67-3766-4214-9ed0-845eae242961 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:32.374164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ef08df67-3766-4214-9ed0-845eae242961 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:32.434217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:32.450273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:32.451398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:32.451398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] PATH_INFO: `/v3/users/fdf5a33dacf94c2684bca52f9d5ff8a0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:32.453056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdf5a33dacf94c2684bca52f9d5ff8a0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:32.455143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:32.499955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:32.552041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08df67-3766-4214-9ed0-845eae242961 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:32.573518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdf5a33dacf94c2684bca52f9d5ff8a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.575023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a6cb3-2360-47ea-bd1f-719bb3c59ee9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:32.589727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a6cb3-2360-47ea-bd1f-719bb3c59ee9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:32.589727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a6cb3-2360-47ea-bd1f-719bb3c59ee9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:32.589727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a6cb3-2360-47ea-bd1f-719bb3c59ee9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:32.589727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a6cb3-2360-47ea-bd1f-719bb3c59ee9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=0c51ebebb5e446ab92340d029519714d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:32.589727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a6cb3-2360-47ea-bd1f-719bb3c59ee9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:32.614786 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2524/5061] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:10:32 2022] GET /identity/v3/groups/0c51ebebb5e446ab92340d029519714d/users => generated 145 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:32.616748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0c08f-e110-4da7-a2dc-b487855e430b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:32.618303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0c08f-e110-4da7-a2dc-b487855e430b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:32.618700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0c08f-e110-4da7-a2dc-b487855e430b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:32.627280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdf5a33dacf94c2684bca52f9d5ff8a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.645162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdf5a33dacf94c2684bca52f9d5ff8a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.651096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2ad0c08f-e110-4da7-a2dc-b487855e430b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:32.652387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2ad0c08f-e110-4da7-a2dc-b487855e430b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:32.660247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdf5a33dacf94c2684bca52f9d5ff8a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.660762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdf5a33dacf94c2684bca52f9d5ff8a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.664885 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2525/5062] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:32.665585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0c08f-e110-4da7-a2dc-b487855e430b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:32.674621 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2538/5063] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:32 2022] DELETE /identity/v3/users/fdf5a33dacf94c2684bca52f9d5ff8a0 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:32.675718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:32.677380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:32.694501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0c08f-e110-4da7-a2dc-b487855e430b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:32.776460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:32.814194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab56f6e-5df7-4a06-a634-c259fb7c026a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:32.823002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:32.824028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:32.824363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:32.824693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/users/fb79176288b847d4a65e7403ae72b7b3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:32.826181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_user(user_id=fb79176288b847d4a65e7403ae72b7b3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:32.828458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:32.843303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c63a59-a74f-4b4e-9c8c-aefe6390e596 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:32.844606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c63a59-a74f-4b4e-9c8c-aefe6390e596 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:32.844891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c63a59-a74f-4b4e-9c8c-aefe6390e596 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:32.845396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c63a59-a74f-4b4e-9c8c-aefe6390e596 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:32.846705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c63a59-a74f-4b4e-9c8c-aefe6390e596 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:32.850169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c63a59-a74f-4b4e-9c8c-aefe6390e596 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:32.889322 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2526/5064] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:32 2022] POST /identity/v3/roles => generated 235 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:32.890475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c63a59-a74f-4b4e-9c8c-aefe6390e596 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:32.894637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e8c63a59-a74f-4b4e-9c8c-aefe6390e596 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:32.931596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb79176288b847d4a65e7403ae72b7b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.957788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb79176288b847d4a65e7403ae72b7b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.974034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb79176288b847d4a65e7403ae72b7b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.988249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb79176288b847d4a65e7403ae72b7b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:32.989880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52d3dc02-325f-451c-a175-880c86027431 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb79176288b847d4a65e7403ae72b7b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.008902 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2539/5065] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:32 2022] DELETE /identity/v3/users/fb79176288b847d4a65e7403ae72b7b3 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:33.010372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349b713-7707-454d-98fe-9f4e9531baae None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:33.010624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349b713-7707-454d-98fe-9f4e9531baae None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:33.010994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349b713-7707-454d-98fe-9f4e9531baae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:33.054080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c63a59-a74f-4b4e-9c8c-aefe6390e596 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:33.055545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9349b713-7707-454d-98fe-9f4e9531baae None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:33.057396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9349b713-7707-454d-98fe-9f4e9531baae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.165558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c8c6c-62cd-4c84-9e4c-0f3bebb1dd1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:33.165997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c8c6c-62cd-4c84-9e4c-0f3bebb1dd1a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:33.166323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c8c6c-62cd-4c84-9e4c-0f3bebb1dd1a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:33.166600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c8c6c-62cd-4c84-9e4c-0f3bebb1dd1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:33.170053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ad3c8c6c-62cd-4c84-9e4c-0f3bebb1dd1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.196258 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2540/5066] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:33.197344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349b713-7707-454d-98fe-9f4e9531baae None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:33.199543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9349b713-7707-454d-98fe-9f4e9531baae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.238018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:33.271814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c8c6c-62cd-4c84-9e4c-0f3bebb1dd1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:33.273743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c8c6c-62cd-4c84-9e4c-0f3bebb1dd1a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:33.276706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349b713-7707-454d-98fe-9f4e9531baae None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:33.286124 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2527/5067] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:33.287334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c8c6c-62cd-4c84-9e4c-0f3bebb1dd1a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:33.289815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e2a3b-ed0b-4382-b773-acd50610dca6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:33.290230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ad3c8c6c-62cd-4c84-9e4c-0f3bebb1dd1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.290638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e2a3b-ed0b-4382-b773-acd50610dca6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:33.291009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e2a3b-ed0b-4382-b773-acd50610dca6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:33.291540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e2a3b-ed0b-4382-b773-acd50610dca6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:33.292376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e2a3b-ed0b-4382-b773-acd50610dca6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:33.294411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e2a3b-ed0b-4382-b773-acd50610dca6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:33.305837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c8c6c-62cd-4c84-9e4c-0f3bebb1dd1a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:33.312271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:33.312713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:33.312990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:33.313276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] PATH_INFO: `/v3/projects/d1dc8e5c6e72468f95736e23ef28f86b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:33.318734 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2541/5068] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:33 2022] POST /identity/v3/roles => generated 234 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:33.318905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1dc8e5c6e72468f95736e23ef28f86b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:33.319022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e2a3b-ed0b-4382-b773-acd50610dca6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:33.319121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:33.321655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-511e2a3b-ed0b-4382-b773-acd50610dca6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.346873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:33.405122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1dc8e5c6e72468f95736e23ef28f86b', 'request_id': 'req-bf4f5e6c-02cd-411c-8273-cfad7f99b111'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.417094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1dc8e5c6e72468f95736e23ef28f86b', 'request_id': 'req-bf4f5e6c-02cd-411c-8273-cfad7f99b111'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.432484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1dc8e5c6e72468f95736e23ef28f86b', 'request_id': 'req-bf4f5e6c-02cd-411c-8273-cfad7f99b111'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.451064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] The token cache is being invalidate because project d1dc8e5c6e72468f95736e23ef28f86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:33.451503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e2a3b-ed0b-4382-b773-acd50610dca6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:33.451972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.453429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.455572 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2528/5069] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:33 2022] DELETE /identity/v3/projects/d1dc8e5c6e72468f95736e23ef28f86b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:33.457083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:33.460016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.524464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:33.525481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:33.525917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:33.526170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/users/13ab1ff5c9b347d893b6face660e0b03` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:33.527396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_user(user_id=13ab1ff5c9b347d893b6face660e0b03)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:33.529638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:33.550863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f5e6c-02cd-411c-8273-cfad7f99b111 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:33.555978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:33.556620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:33.556889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:33.557195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] PATH_INFO: `/v3/users/3d2974383cae4b5485ce72b2e24a8dc6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:33.558346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d2974383cae4b5485ce72b2e24a8dc6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:33.561202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:33.592345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ab1ff5c9b347d893b6face660e0b03'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.614621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ab1ff5c9b347d893b6face660e0b03'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.621536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d2974383cae4b5485ce72b2e24a8dc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.626133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ab1ff5c9b347d893b6face660e0b03'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.629959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d2974383cae4b5485ce72b2e24a8dc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.631610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:33.637187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ab1ff5c9b347d893b6face660e0b03'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.638083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ab1ff5c9b347d893b6face660e0b03'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.644569 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2542/5070] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:33 2022] DELETE /identity/v3/users/13ab1ff5c9b347d893b6face660e0b03 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:33.645551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:33.646869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.652272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d2974383cae4b5485ce72b2e24a8dc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.666669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d2974383cae4b5485ce72b2e24a8dc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.668189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d2974383cae4b5485ce72b2e24a8dc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:33.679780 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2529/5071] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:33 2022] DELETE /identity/v3/users/3d2974383cae4b5485ce72b2e24a8dc6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:33.682853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:33.682853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.685874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:33.715658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4057b32-5d66-42d8-abd8-acb44f0d4573 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:33.758295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fe927-f4db-4f97-84d9-b94229eb248f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:33.764069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90752b55-7044-4a8f-be5f-9d882a536f24 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:33.764789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90752b55-7044-4a8f-be5f-9d882a536f24 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:33.765208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90752b55-7044-4a8f-be5f-9d882a536f24 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:33.765584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90752b55-7044-4a8f-be5f-9d882a536f24 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:33.766345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90752b55-7044-4a8f-be5f-9d882a536f24 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:33.768128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90752b55-7044-4a8f-be5f-9d882a536f24 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:33.780449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8446-ee10-49d5-8cc5-e67067c7bb04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:33.781401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8446-ee10-49d5-8cc5-e67067c7bb04 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:33.781676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8446-ee10-49d5-8cc5-e67067c7bb04 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:33.781941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8446-ee10-49d5-8cc5-e67067c7bb04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:33.783735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b47e8446-ee10-49d5-8cc5-e67067c7bb04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.791053 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2530/5072] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:10:33 2022] POST /identity/v3/domains => generated 276 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:33.792355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3261e8-c556-46f6-8b24-3c06061e0819 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:33.792768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3261e8-c556-46f6-8b24-3c06061e0819 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:33.793174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3261e8-c556-46f6-8b24-3c06061e0819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:33.819750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ea3261e8-c556-46f6-8b24-3c06061e0819 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:33.821653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ea3261e8-c556-46f6-8b24-3c06061e0819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.858808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e8446-ee10-49d5-8cc5-e67067c7bb04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:33.860301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e8446-ee10-49d5-8cc5-e67067c7bb04 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:33.872111 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2543/5073] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:33.873552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8446-ee10-49d5-8cc5-e67067c7bb04 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:33.875091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b47e8446-ee10-49d5-8cc5-e67067c7bb04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.888244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8446-ee10-49d5-8cc5-e67067c7bb04 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:33.917200 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2531/5074] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:33.918130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3261e8-c556-46f6-8b24-3c06061e0819 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:33.919779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ea3261e8-c556-46f6-8b24-3c06061e0819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:33.986316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:33.986980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:33.987211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:33.987474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] PATH_INFO: `/v3/projects/014c5e68788f4050ba767e9cee80889d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:33.988594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=014c5e68788f4050ba767e9cee80889d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:33.990843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:34.000333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3261e8-c556-46f6-8b24-3c06061e0819 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:34.005809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:34.006649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.006957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.007256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/users/129fccee43e042238e61afb4ff5cb3b2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.009255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_user(user_id=129fccee43e042238e61afb4ff5cb3b2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:34.015512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:34.032729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:34.041616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:34.051846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '014c5e68788f4050ba767e9cee80889d', 'request_id': 'req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.078472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '129fccee43e042238e61afb4ff5cb3b2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.083120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '014c5e68788f4050ba767e9cee80889d', 'request_id': 'req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.091820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '014c5e68788f4050ba767e9cee80889d', 'request_id': 'req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.097356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '129fccee43e042238e61afb4ff5cb3b2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.115722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '129fccee43e042238e61afb4ff5cb3b2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.123029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] The token cache is being invalidate because project 014c5e68788f4050ba767e9cee80889d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:34.123552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.124548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e9f7abf7-f06c-4e80-9d8a-3ce6a20c6ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.127756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '129fccee43e042238e61afb4ff5cb3b2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.128658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '129fccee43e042238e61afb4ff5cb3b2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.141231 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2532/5075] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:33 2022] DELETE /identity/v3/users/129fccee43e042238e61afb4ff5cb3b2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:34.142296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:34.143308 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2544/5076] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:33 2022] DELETE /identity/v3/projects/014c5e68788f4050ba767e9cee80889d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:34.145764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2c3f5-3e63-46e3-98ad-0503adf1cfa0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.146276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2c3f5-3e63-46e3-98ad-0503adf1cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.146535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.147158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2c3f5-3e63-46e3-98ad-0503adf1cfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.170810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8fd2c3f5-3e63-46e3-98ad-0503adf1cfa0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:34.172217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8fd2c3f5-3e63-46e3-98ad-0503adf1cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.250843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9447324-26fe-4141-a357-2adf3d8825dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:34.255771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fe5b5-489a-4473-b59b-a6de710eb1d1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:34.257013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fe5b5-489a-4473-b59b-a6de710eb1d1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.257335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fe5b5-489a-4473-b59b-a6de710eb1d1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.257649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fe5b5-489a-4473-b59b-a6de710eb1d1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.259215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fe5b5-489a-4473-b59b-a6de710eb1d1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:34.266224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fe5b5-489a-4473-b59b-a6de710eb1d1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:34.266494 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2545/5077] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:34.267845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2c3f5-3e63-46e3-98ad-0503adf1cfa0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:34.269913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8fd2c3f5-3e63-46e3-98ad-0503adf1cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.285952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2c3f5-3e63-46e3-98ad-0503adf1cfa0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:34.291988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:34.292704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.293639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.293921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] PATH_INFO: `/v3/projects/cd6d218d2e75459eae3548b0a9ba3000` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.295080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd6d218d2e75459eae3548b0a9ba3000)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:34.297080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:34.319963 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2533/5078] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/projects => generated 401 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:34.319963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fe5b5-489a-4473-b59b-a6de710eb1d1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:34.321404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a18fe5b5-489a-4473-b59b-a6de710eb1d1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.355932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd6d218d2e75459eae3548b0a9ba3000', 'request_id': 'req-0819b4bb-8156-42d7-95ea-c707ba704565'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.365243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd6d218d2e75459eae3548b0a9ba3000', 'request_id': 'req-0819b4bb-8156-42d7-95ea-c707ba704565'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.373703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd6d218d2e75459eae3548b0a9ba3000', 'request_id': 'req-0819b4bb-8156-42d7-95ea-c707ba704565'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.386960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] The token cache is being invalidate because project cd6d218d2e75459eae3548b0a9ba3000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:34.387246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.388748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0819b4bb-8156-42d7-95ea-c707ba704565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.394949 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2546/5079] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:34 2022] DELETE /identity/v3/projects/cd6d218d2e75459eae3548b0a9ba3000 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:34.394949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cfaaf-1541-4aaa-8670-a0cf4963abdf None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.394949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cfaaf-1541-4aaa-8670-a0cf4963abdf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.394949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cfaaf-1541-4aaa-8670-a0cf4963abdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.417922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d31cfaaf-1541-4aaa-8670-a0cf4963abdf None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:34.419335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d31cfaaf-1541-4aaa-8670-a0cf4963abdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.420293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fe5b5-489a-4473-b59b-a6de710eb1d1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:34.431617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:34.482280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:34.489166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ff036-9f57-414e-96d2-80ded2dd5900 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:34.489894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff036-9f57-414e-96d2-80ded2dd5900 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.490172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff036-9f57-414e-96d2-80ded2dd5900 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.490445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff036-9f57-414e-96d2-80ded2dd5900 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/0c51ebebb5e446ab92340d029519714d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.491861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ff036-9f57-414e-96d2-80ded2dd5900 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_group(group_id=0c51ebebb5e446ab92340d029519714d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:34.495413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ff036-9f57-414e-96d2-80ded2dd5900 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:34.506575 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2547/5080] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:34.506575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31cfaaf-1541-4aaa-8670-a0cf4963abdf None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:34.506575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d31cfaaf-1541-4aaa-8670-a0cf4963abdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.517533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31cfaaf-1541-4aaa-8670-a0cf4963abdf None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:34.523993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:34.524858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.525305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.525645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] PATH_INFO: `/v3/projects/7581df74c9854212a83bae756249a009` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.528225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7581df74c9854212a83bae756249a009)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:34.531578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:34.569772 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2534/5081] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:10:34 2022] DELETE /identity/v3/groups/0c51ebebb5e446ab92340d029519714d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:34.571104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ff036-9f57-414e-96d2-80ded2dd5900 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:34.573996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-594ff036-9f57-414e-96d2-80ded2dd5900 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.604915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7581df74c9854212a83bae756249a009', 'request_id': 'req-caba1974-5183-4535-b30c-ba99ca393073'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.613960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7581df74c9854212a83bae756249a009', 'request_id': 'req-caba1974-5183-4535-b30c-ba99ca393073'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.622270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7581df74c9854212a83bae756249a009', 'request_id': 'req-caba1974-5183-4535-b30c-ba99ca393073'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.636697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] The token cache is being invalidate because project 7581df74c9854212a83bae756249a009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:34.637088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.637917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:34.640072 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2548/5082] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:34 2022] DELETE /identity/v3/projects/7581df74c9854212a83bae756249a009 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:34.640765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:34.644321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.651582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ff036-9f57-414e-96d2-80ded2dd5900 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:34.729965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e1301-b5e2-41c5-b9c4-5efbc081727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:34.730402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba1974-5183-4535-b30c-ba99ca393073 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:34.730740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1301-b5e2-41c5-b9c4-5efbc081727e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.731221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1301-b5e2-41c5-b9c4-5efbc081727e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.731500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1301-b5e2-41c5-b9c4-5efbc081727e admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.732241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e1301-b5e2-41c5-b9c4-5efbc081727e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:34.732672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e1301-b5e2-41c5-b9c4-5efbc081727e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:34.734561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e1301-b5e2-41c5-b9c4-5efbc081727e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:34.738344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac878565-ea71-4cd0-81fb-3285f4e9acef tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:34.739296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac878565-ea71-4cd0-81fb-3285f4e9acef tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.739585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac878565-ea71-4cd0-81fb-3285f4e9acef tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.739851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac878565-ea71-4cd0-81fb-3285f4e9acef tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.740557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac878565-ea71-4cd0-81fb-3285f4e9acef tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:34.742852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac878565-ea71-4cd0-81fb-3285f4e9acef tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:34.750161 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2535/5083] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:34.752047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bef27f-705e-4f10-bcf5-0debeb8f5047 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.752485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bef27f-705e-4f10-bcf5-0debeb8f5047 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.752909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bef27f-705e-4f10-bcf5-0debeb8f5047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.766940 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2549/5084] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/groups => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:34.766940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366e09c-68b5-4e21-a148-072e78828c93 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.766940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366e09c-68b5-4e21-a148-072e78828c93 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.766940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366e09c-68b5-4e21-a148-072e78828c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.778912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-08bef27f-705e-4f10-bcf5-0debeb8f5047 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:34.780117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-08bef27f-705e-4f10-bcf5-0debeb8f5047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.788012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7366e09c-68b5-4e21-a148-072e78828c93 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:34.788275 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2536/5085] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:34.789675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7366e09c-68b5-4e21-a148-072e78828c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.794034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bef27f-705e-4f10-bcf5-0debeb8f5047 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:34.795823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-08bef27f-705e-4f10-bcf5-0debeb8f5047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.801469 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2550/5086] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:34.801469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4016680-2dda-45b3-8bbd-83ff4f1f84a3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.801469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4016680-2dda-45b3-8bbd-83ff4f1f84a3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.801469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4016680-2dda-45b3-8bbd-83ff4f1f84a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.809471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:34.823582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d4016680-2dda-45b3-8bbd-83ff4f1f84a3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:34.824795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4016680-2dda-45b3-8bbd-83ff4f1f84a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.832981 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2551/5087] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:34.833765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4016680-2dda-45b3-8bbd-83ff4f1f84a3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:34.835227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4016680-2dda-45b3-8bbd-83ff4f1f84a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.849053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4016680-2dda-45b3-8bbd-83ff4f1f84a3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:34.856046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f566767-6695-40b8-ad93-bbffdbdf49ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:34.856742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f566767-6695-40b8-ad93-bbffdbdf49ed admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.858665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f566767-6695-40b8-ad93-bbffdbdf49ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.858665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f566767-6695-40b8-ad93-bbffdbdf49ed admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.858665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f566767-6695-40b8-ad93-bbffdbdf49ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:34.862016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f566767-6695-40b8-ad93-bbffdbdf49ed admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:34.889021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bef27f-705e-4f10-bcf5-0debeb8f5047 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:34.895224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e1327-b5a0-4f58-8963-7d7a9482a68c tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:34.895834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e1327-b5a0-4f58-8963-7d7a9482a68c tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.896110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e1327-b5a0-4f58-8963-7d7a9482a68c tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.896379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e1327-b5a0-4f58-8963-7d7a9482a68c tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.897003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e1327-b5a0-4f58-8963-7d7a9482a68c tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:34.899043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e1327-b5a0-4f58-8963-7d7a9482a68c tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:34.907121 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2552/5088] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:34.908124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f566767-6695-40b8-ad93-bbffdbdf49ed admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:34.925782 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2537/5089] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/groups => generated 323 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:34.927231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fae28-9c12-4c82-b8b0-f34d1b2b463c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:34.927604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fae28-9c12-4c82-b8b0-f34d1b2b463c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:34.928236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fae28-9c12-4c82-b8b0-f34d1b2b463c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:34.955511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d60fae28-9c12-4c82-b8b0-f34d1b2b463c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:34.956721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d60fae28-9c12-4c82-b8b0-f34d1b2b463c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:34.969140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:35.002562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f566767-6695-40b8-ad93-bbffdbdf49ed admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.007491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a3cc9-8081-4f69-b5c0-ebed9ce94637 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.008901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a3cc9-8081-4f69-b5c0-ebed9ce94637 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.009397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a3cc9-8081-4f69-b5c0-ebed9ce94637 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.009744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a3cc9-8081-4f69-b5c0-ebed9ce94637 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.010492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a3cc9-8081-4f69-b5c0-ebed9ce94637 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.012182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a3cc9-8081-4f69-b5c0-ebed9ce94637 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.032216 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2553/5090] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/roles => generated 235 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.032216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a3cc9-8081-4f69-b5c0-ebed9ce94637 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.032483 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2538/5091] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:35.032483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f3fb-bf71-4b13-a086-8acae5c47fde None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.032483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f3fb-bf71-4b13-a086-8acae5c47fde None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.032483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f3fb-bf71-4b13-a086-8acae5c47fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.045870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a3cc9-8081-4f69-b5c0-ebed9ce94637 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.051081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d99f7-b2c1-4c15-9989-566daa2316d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.051680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d99f7-b2c1-4c15-9989-566daa2316d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.051949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d99f7-b2c1-4c15-9989-566daa2316d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.052213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d99f7-b2c1-4c15-9989-566daa2316d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.052848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d99f7-b2c1-4c15-9989-566daa2316d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:35.053175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d99f7-b2c1-4c15-9989-566daa2316d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.054890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d99f7-b2c1-4c15-9989-566daa2316d2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.057267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-caf4f3fb-bf71-4b13-a086-8acae5c47fde None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:35.058500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-caf4f3fb-bf71-4b13-a086-8acae5c47fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:35.067389 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2539/5092] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:35.067650 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2554/5093] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:35.068255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4f3fb-bf71-4b13-a086-8acae5c47fde None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.072352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d99f7-b2c1-4c15-9989-566daa2316d2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.080482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4f3fb-bf71-4b13-a086-8acae5c47fde None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.085663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5329d-611c-40fc-93d3-1900ebe8c78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.087304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5329d-611c-40fc-93d3-1900ebe8c78c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.087587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5329d-611c-40fc-93d3-1900ebe8c78c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.087861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5329d-611c-40fc-93d3-1900ebe8c78c admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.088492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5329d-611c-40fc-93d3-1900ebe8c78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.090498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5329d-611c-40fc-93d3-1900ebe8c78c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.117812 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2540/5094] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/users => generated 284 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.119198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5329d-611c-40fc-93d3-1900ebe8c78c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.132399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5329d-611c-40fc-93d3-1900ebe8c78c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.138103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f42f79-154f-4506-9b8a-fdc29eb67dba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.138709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f42f79-154f-4506-9b8a-fdc29eb67dba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.138982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f42f79-154f-4506-9b8a-fdc29eb67dba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.139250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f42f79-154f-4506-9b8a-fdc29eb67dba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.140011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f42f79-154f-4506-9b8a-fdc29eb67dba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.141982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f42f79-154f-4506-9b8a-fdc29eb67dba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.151356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d99f7-b2c1-4c15-9989-566daa2316d2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.157634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72400b39-32e9-4238-b2fd-d0b5c4803444 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.158467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400b39-32e9-4238-b2fd-d0b5c4803444 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.158978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400b39-32e9-4238-b2fd-d0b5c4803444 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.159283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400b39-32e9-4238-b2fd-d0b5c4803444 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.159903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72400b39-32e9-4238-b2fd-d0b5c4803444 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.161835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72400b39-32e9-4238-b2fd-d0b5c4803444 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.168671 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2541/5095] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/users => generated 439 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.170081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f42f79-154f-4506-9b8a-fdc29eb67dba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.184490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f42f79-154f-4506-9b8a-fdc29eb67dba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.186907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2555/5096] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/groups => generated 326 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.187931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72400b39-32e9-4238-b2fd-d0b5c4803444 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.192497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c28190-8f30-4e2a-9abf-31952b5d0c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.193130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28190-8f30-4e2a-9abf-31952b5d0c74 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.194139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28190-8f30-4e2a-9abf-31952b5d0c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.194415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28190-8f30-4e2a-9abf-31952b5d0c74 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.195225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c28190-8f30-4e2a-9abf-31952b5d0c74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:35.196157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c28190-8f30-4e2a-9abf-31952b5d0c74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.198023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c28190-8f30-4e2a-9abf-31952b5d0c74 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.216240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72400b39-32e9-4238-b2fd-d0b5c4803444 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.220850 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2542/5097] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:35.220850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c28190-8f30-4e2a-9abf-31952b5d0c74 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.225280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af432a-99bb-42a3-b1c7-d50f1f961bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.226003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af432a-99bb-42a3-b1c7-d50f1f961bba admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.226351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af432a-99bb-42a3-b1c7-d50f1f961bba admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.226652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af432a-99bb-42a3-b1c7-d50f1f961bba admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.227373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af432a-99bb-42a3-b1c7-d50f1f961bba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:35.227735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af432a-99bb-42a3-b1c7-d50f1f961bba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.229533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af432a-99bb-42a3-b1c7-d50f1f961bba admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.232804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c28190-8f30-4e2a-9abf-31952b5d0c74 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.239520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285d829-0a4e-4a54-bfa7-bf3f1b67de64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.239926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285d829-0a4e-4a54-bfa7-bf3f1b67de64 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.240273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285d829-0a4e-4a54-bfa7-bf3f1b67de64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.240596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285d829-0a4e-4a54-bfa7-bf3f1b67de64 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.241404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285d829-0a4e-4a54-bfa7-bf3f1b67de64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.244513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285d829-0a4e-4a54-bfa7-bf3f1b67de64 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.248391 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2556/5098] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:35.249744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af432a-99bb-42a3-b1c7-d50f1f961bba admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.262765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af432a-99bb-42a3-b1c7-d50f1f961bba admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.268316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8879ac-f28c-4a99-a2d4-0fe8f88cec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.269067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8879ac-f28c-4a99-a2d4-0fe8f88cec56 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.269839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8879ac-f28c-4a99-a2d4-0fe8f88cec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.269839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8879ac-f28c-4a99-a2d4-0fe8f88cec56 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.270534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8879ac-f28c-4a99-a2d4-0fe8f88cec56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.272441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8879ac-f28c-4a99-a2d4-0fe8f88cec56 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.274835 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2543/5099] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.278151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285d829-0a4e-4a54-bfa7-bf3f1b67de64 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.289575 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2557/5100] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:35 2022] GET /identity/v3/roles => generated 2323 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:35.289575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8879ac-f28c-4a99-a2d4-0fe8f88cec56 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.299804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8879ac-f28c-4a99-a2d4-0fe8f88cec56 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.311676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:35.355288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285d829-0a4e-4a54-bfa7-bf3f1b67de64 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.360162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9484e-f189-4585-b6c3-bca38c81f038 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.361323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9484e-f189-4585-b6c3-bca38c81f038 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.361658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9484e-f189-4585-b6c3-bca38c81f038 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.361993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9484e-f189-4585-b6c3-bca38c81f038 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.363091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9484e-f189-4585-b6c3-bca38c81f038 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.364759 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-5be9484e-f189-4585-b6c3-bca38c81f038 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:35.364759 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:35.364759 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:35.365672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9484e-f189-4585-b6c3-bca38c81f038 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.368592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7f760-40cd-453f-a3dd-6ee0111533a8 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.369316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7f760-40cd-453f-a3dd-6ee0111533a8 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.369674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7f760-40cd-453f-a3dd-6ee0111533a8 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.370008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7f760-40cd-453f-a3dd-6ee0111533a8 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.370804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7f760-40cd-453f-a3dd-6ee0111533a8 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.373079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7f760-40cd-453f-a3dd-6ee0111533a8 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.384925 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2544/5101] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/roles => generated 270 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.388279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9484e-f189-4585-b6c3-bca38c81f038 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.394082 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2558/5102] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/groups => generated 325 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.394947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7f760-40cd-453f-a3dd-6ee0111533a8 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.421962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:35.468224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9484e-f189-4585-b6c3-bca38c81f038 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.468512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7f760-40cd-453f-a3dd-6ee0111533a8 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.473611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b06966-6d89-448c-af4d-aeb0980db35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.473783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee45dfa-1cbe-40f0-82d4-2575762c8d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.473925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b06966-6d89-448c-af4d-aeb0980db35b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.474171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b06966-6d89-448c-af4d-aeb0980db35b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.474390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee45dfa-1cbe-40f0-82d4-2575762c8d29 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.474502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b06966-6d89-448c-af4d-aeb0980db35b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.474670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee45dfa-1cbe-40f0-82d4-2575762c8d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.474943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee45dfa-1cbe-40f0-82d4-2575762c8d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.475067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b06966-6d89-448c-af4d-aeb0980db35b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.475564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee45dfa-1cbe-40f0-82d4-2575762c8d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.477463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b06966-6d89-448c-af4d-aeb0980db35b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.478351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee45dfa-1cbe-40f0-82d4-2575762c8d29 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.508474 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2545/5103] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/projects => generated 395 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.509763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b06966-6d89-448c-af4d-aeb0980db35b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.513364 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2559/5104] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.514625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee45dfa-1cbe-40f0-82d4-2575762c8d29 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.598555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee45dfa-1cbe-40f0-82d4-2575762c8d29 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.606222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b65b70-d1e4-46a8-b2b4-8396c1d42294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.607456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b65b70-d1e4-46a8-b2b4-8396c1d42294 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.607660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b65b70-d1e4-46a8-b2b4-8396c1d42294 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.607907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b65b70-d1e4-46a8-b2b4-8396c1d42294 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.608709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b65b70-d1e4-46a8-b2b4-8396c1d42294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.609697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b06966-6d89-448c-af4d-aeb0980db35b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.611845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b65b70-d1e4-46a8-b2b4-8396c1d42294 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.617309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e492a0-1e2b-427a-b3c4-59b3afaceb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.618300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e492a0-1e2b-427a-b3c4-59b3afaceb5b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.618300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e492a0-1e2b-427a-b3c4-59b3afaceb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.618565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e492a0-1e2b-427a-b3c4-59b3afaceb5b admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.619067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e492a0-1e2b-427a-b3c4-59b3afaceb5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.621483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e492a0-1e2b-427a-b3c4-59b3afaceb5b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.636517 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2560/5105] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:35 2022] GET /identity/v3/roles => generated 2323 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:35.636517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b65b70-d1e4-46a8-b2b4-8396c1d42294 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.649683 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2546/5106] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/users => generated 284 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.650522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e492a0-1e2b-427a-b3c4-59b3afaceb5b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.686390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:35.705623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b65b70-d1e4-46a8-b2b4-8396c1d42294 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.710197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6a4a-1da1-42bc-a4c2-b75f7f5c86e4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.710891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6a4a-1da1-42bc-a4c2-b75f7f5c86e4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.711166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6a4a-1da1-42bc-a4c2-b75f7f5c86e4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.711436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6a4a-1da1-42bc-a4c2-b75f7f5c86e4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/54ddcc4a2701438694789d1eba5c2ac5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.716640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e492a0-1e2b-427a-b3c4-59b3afaceb5b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.721384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22f37c-2e90-4c0b-94a5-27781fee73ed tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.721981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22f37c-2e90-4c0b-94a5-27781fee73ed tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.722254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22f37c-2e90-4c0b-94a5-27781fee73ed tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.722518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22f37c-2e90-4c0b-94a5-27781fee73ed tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.723163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22f37c-2e90-4c0b-94a5-27781fee73ed tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:list_groups()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.724162 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-6e22f37c-2e90-4c0b-94a5-27781fee73ed tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_groups": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:35.724162 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:35.724162 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:35.725294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22f37c-2e90-4c0b-94a5-27781fee73ed tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.728061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a6a4a-1da1-42bc-a4c2-b75f7f5c86e4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_grant(project_id=2335de37d6144200bbdef5af1a5cbab2, user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=54ddcc4a2701438694789d1eba5c2ac5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.730924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a6a4a-1da1-42bc-a4c2-b75f7f5c86e4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.732985 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2547/5107] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 21:10:35 2022] GET /identity/v3/groups => generated 1774 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:35.733620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22f37c-2e90-4c0b-94a5-27781fee73ed tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.748869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22f37c-2e90-4c0b-94a5-27781fee73ed tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.749122 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2561/5108] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:35 2022] PUT /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/54ddcc4a2701438694789d1eba5c2ac5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:35.749122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6a4a-1da1-42bc-a4c2-b75f7f5c86e4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.775667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:35.811051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09395969-9543-432a-91d3-7e92609b86d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.811646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09395969-9543-432a-91d3-7e92609b86d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.811924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09395969-9543-432a-91d3-7e92609b86d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.812188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09395969-9543-432a-91d3-7e92609b86d0 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.812799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09395969-9543-432a-91d3-7e92609b86d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.813246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6a4a-1da1-42bc-a4c2-b75f7f5c86e4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.814773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09395969-9543-432a-91d3-7e92609b86d0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.818045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a5cb8-a138-4509-b129-8f80709f6b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.818662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a5cb8-a138-4509-b129-8f80709f6b23 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.818931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a5cb8-a138-4509-b129-8f80709f6b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.819193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a5cb8-a138-4509-b129-8f80709f6b23 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.819796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a5cb8-a138-4509-b129-8f80709f6b23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.821806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a5cb8-a138-4509-b129-8f80709f6b23 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.836499 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2548/5109] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.837313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09395969-9543-432a-91d3-7e92609b86d0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.847919 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2562/5110] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:35.847919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa47525-32a0-467b-813e-57fa656f2e5b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.847919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa47525-32a0-467b-813e-57fa656f2e5b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.848306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09395969-9543-432a-91d3-7e92609b86d0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.848404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa47525-32a0-467b-813e-57fa656f2e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.852062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5abdfb-c9e4-40df-8b5f-e7e952ebf78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.852710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5abdfb-c9e4-40df-8b5f-e7e952ebf78f admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.852966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5abdfb-c9e4-40df-8b5f-e7e952ebf78f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.853259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5abdfb-c9e4-40df-8b5f-e7e952ebf78f admin admin] PATH_INFO: `/v3/projects/e7b6ea65ad2e405a94727beb4bbeb68d/users/8aa4174b94564057a4ed7e15c3b73672/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.868336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5abdfb-c9e4-40df-8b5f-e7e952ebf78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b6ea65ad2e405a94727beb4bbeb68d, user_id=8aa4174b94564057a4ed7e15c3b73672, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.868525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dfa47525-32a0-467b-813e-57fa656f2e5b None None] MFA Rules not processed for user `c48346e56c47481cab7f248b67369a27`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:35.869840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dfa47525-32a0-467b-813e-57fa656f2e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:35.870979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5abdfb-c9e4-40df-8b5f-e7e952ebf78f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.885234 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2549/5111] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:35 2022] PUT /identity/v3/projects/e7b6ea65ad2e405a94727beb4bbeb68d/users/8aa4174b94564057a4ed7e15c3b73672/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:35.886025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5abdfb-c9e4-40df-8b5f-e7e952ebf78f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.937938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2563/5112] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:35.939051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa47525-32a0-467b-813e-57fa656f2e5b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:35.955176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5abdfb-c9e4-40df-8b5f-e7e952ebf78f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:35.961366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d980cd-11da-44e9-a02a-a7e3f0ac7b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:35.961979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d980cd-11da-44e9-a02a-a7e3f0ac7b7f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:35.962252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d980cd-11da-44e9-a02a-a7e3f0ac7b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:35.962518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d980cd-11da-44e9-a02a-a7e3f0ac7b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:35.963284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d980cd-11da-44e9-a02a-a7e3f0ac7b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:35.964869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d980cd-11da-44e9-a02a-a7e3f0ac7b7f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:35.982036 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2550/5113] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:35 2022] GET /identity/v3/roles => generated 2323 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:35.982036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d980cd-11da-44e9-a02a-a7e3f0ac7b7f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.017857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa47525-32a0-467b-813e-57fa656f2e5b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.023848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:36.065776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d980cd-11da-44e9-a02a-a7e3f0ac7b7f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.071514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08de0a-6d7b-4c03-bbdf-8d89bbb8d934 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.072114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08de0a-6d7b-4c03-bbdf-8d89bbb8d934 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.072558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08de0a-6d7b-4c03-bbdf-8d89bbb8d934 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.072832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08de0a-6d7b-4c03-bbdf-8d89bbb8d934 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.073909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08de0a-6d7b-4c03-bbdf-8d89bbb8d934 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Adding query filter params (scope.project.id=2335de37d6144200bbdef5af1a5cbab2, user.id=4d18588ffdd8497ab8903f4a5a81fe87) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:36.074220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08de0a-6d7b-4c03-bbdf-8d89bbb8d934 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.075383 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-6e08de0a-6d7b-4c03-bbdf-8d89bbb8d934 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:36.075383 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:36.075383 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:36.076539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08de0a-6d7b-4c03-bbdf-8d89bbb8d934 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.077246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:36.084334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aa772-130e-41fe-a133-6914f01d8f10 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.085488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aa772-130e-41fe-a133-6914f01d8f10 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.086491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aa772-130e-41fe-a133-6914f01d8f10 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.087649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aa772-130e-41fe-a133-6914f01d8f10 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/41f35f7d471c4abeab584d485255907c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.093610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81aa772-130e-41fe-a133-6914f01d8f10 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_group(group_id=41f35f7d471c4abeab584d485255907c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.095710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81aa772-130e-41fe-a133-6914f01d8f10 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.134296 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2551/5114] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Oct 15 21:10:35 2022] GET /identity/v3/role_assignments?scope.project.id=2335de37d6144200bbdef5af1a5cbab2&user.id=4d18588ffdd8497ab8903f4a5a81fe87&effective => generated 228 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:36.135532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08de0a-6d7b-4c03-bbdf-8d89bbb8d934 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.150991 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2564/5115] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:10:35 2022] DELETE /identity/v3/groups/41f35f7d471c4abeab584d485255907c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:36.152212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aa772-130e-41fe-a133-6914f01d8f10 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.157976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08de0a-6d7b-4c03-bbdf-8d89bbb8d934 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.227893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aa772-130e-41fe-a133-6914f01d8f10 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.235247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87221ef5-aeb9-4949-958f-25765b427d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.236023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87221ef5-aeb9-4949-958f-25765b427d47 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.236447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87221ef5-aeb9-4949-958f-25765b427d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.236939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87221ef5-aeb9-4949-958f-25765b427d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.237901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87221ef5-aeb9-4949-958f-25765b427d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.240580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87221ef5-aeb9-4949-958f-25765b427d47 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.247631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762bf92-a442-486a-9e0b-789309eb011f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.248553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762bf92-a442-486a-9e0b-789309eb011f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.248832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762bf92-a442-486a-9e0b-789309eb011f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.249239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762bf92-a442-486a-9e0b-789309eb011f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.249768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762bf92-a442-486a-9e0b-789309eb011f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.251370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762bf92-a442-486a-9e0b-789309eb011f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.258549 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2565/5116] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:36 2022] GET /identity/v3/roles => generated 2323 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:36.260016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33904eb-97be-44bf-9b35-065b6aea68d9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.260371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33904eb-97be-44bf-9b35-065b6aea68d9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.261346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33904eb-97be-44bf-9b35-065b6aea68d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.267490 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2552/5117] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:36 2022] GET /identity/v3/roles => generated 2323 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:36.268938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762bf92-a442-486a-9e0b-789309eb011f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.281412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762bf92-a442-486a-9e0b-789309eb011f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.288165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b09bee-2848-40d8-a2d9-3c3bc395203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.288791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b09bee-2848-40d8-a2d9-3c3bc395203d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.289143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b09bee-2848-40d8-a2d9-3c3bc395203d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.289371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b09bee-2848-40d8-a2d9-3c3bc395203d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.290002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b09bee-2848-40d8-a2d9-3c3bc395203d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.292060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b09bee-2848-40d8-a2d9-3c3bc395203d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.294138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a33904eb-97be-44bf-9b35-065b6aea68d9 None None] MFA Rules not processed for user `8aa4174b94564057a4ed7e15c3b73672`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:36.295497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a33904eb-97be-44bf-9b35-065b6aea68d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:36.304047 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2553/5118] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:36 2022] GET /identity/v3/roles => generated 2323 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:36.307401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b09bee-2848-40d8-a2d9-3c3bc395203d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.365668 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2566/5119] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:36.370026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33904eb-97be-44bf-9b35-065b6aea68d9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.376408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b09bee-2848-40d8-a2d9-3c3bc395203d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.381107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.381887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.382168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.382447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/54ddcc4a2701438694789d1eba5c2ac5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.384771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=2335de37d6144200bbdef5af1a5cbab2, user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=54ddcc4a2701438694789d1eba5c2ac5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.387438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.392893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role 54ddcc4a2701438694789d1eba5c2ac5 was removed from user 4d18588ffdd8497ab8903f4a5a81fe87 on project 2335de37d6144200bbdef5af1a5cbab2. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:36.393213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:36.393904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:36.408349 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2554/5120] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Oct 15 21:10:36 2022] DELETE /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/54ddcc4a2701438694789d1eba5c2ac5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:36.409303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.410709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:36.414753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:36.422135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:36.440268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33904eb-97be-44bf-9b35-065b6aea68d9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.481196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8492d-5ba8-411b-a96f-b4a9dfc970dd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.485946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febaad9-bc26-4c9d-8f42-1f009b7eee04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.486535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febaad9-bc26-4c9d-8f42-1f009b7eee04 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.486806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febaad9-bc26-4c9d-8f42-1f009b7eee04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.487334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febaad9-bc26-4c9d-8f42-1f009b7eee04 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.487689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febaad9-bc26-4c9d-8f42-1f009b7eee04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.489571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febaad9-bc26-4c9d-8f42-1f009b7eee04 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.498943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6f6fa-7461-4bfd-9592-5e2fe472eda9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.499570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6f6fa-7461-4bfd-9592-5e2fe472eda9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.499849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6f6fa-7461-4bfd-9592-5e2fe472eda9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.500120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6f6fa-7461-4bfd-9592-5e2fe472eda9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/25a19962f3b74ffe93921fa38227e78a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.500818 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2555/5121] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:36 2022] GET /identity/v3/roles => generated 2323 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:36.501471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febaad9-bc26-4c9d-8f42-1f009b7eee04 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.504909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1febaad9-bc26-4c9d-8f42-1f009b7eee04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:36.508925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6f6fa-7461-4bfd-9592-5e2fe472eda9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_group(group_id=25a19962f3b74ffe93921fa38227e78a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.510980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6f6fa-7461-4bfd-9592-5e2fe472eda9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.517763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febaad9-bc26-4c9d-8f42-1f009b7eee04 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.522902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196652ff-8210-48a8-971f-d2567e2d95bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.524182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196652ff-8210-48a8-971f-d2567e2d95bc admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.524573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196652ff-8210-48a8-971f-d2567e2d95bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.524841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196652ff-8210-48a8-971f-d2567e2d95bc admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.525488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196652ff-8210-48a8-971f-d2567e2d95bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.527035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196652ff-8210-48a8-971f-d2567e2d95bc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.541937 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2556/5122] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:36 2022] GET /identity/v3/roles => generated 2323 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:36.541937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196652ff-8210-48a8-971f-d2567e2d95bc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.541937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-196652ff-8210-48a8-971f-d2567e2d95bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:36.554070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196652ff-8210-48a8-971f-d2567e2d95bc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.557410 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2567/5123] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:10:36 2022] DELETE /identity/v3/groups/25a19962f3b74ffe93921fa38227e78a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:36.558245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6f6fa-7461-4bfd-9592-5e2fe472eda9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.559805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-afd6f6fa-7461-4bfd-9592-5e2fe472eda9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:36.561939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845846ea-3995-4a6a-a16e-a3f8e4c82981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.562590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845846ea-3995-4a6a-a16e-a3f8e4c82981 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.562869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845846ea-3995-4a6a-a16e-a3f8e4c82981 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.563145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845846ea-3995-4a6a-a16e-a3f8e4c82981 admin admin] PATH_INFO: `/v3/projects/f9083719cf3f4408970021efc83e3d54/users/b9a65c6e684a4385ba56269757641b88/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.580036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845846ea-3995-4a6a-a16e-a3f8e4c82981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9083719cf3f4408970021efc83e3d54, user_id=b9a65c6e684a4385ba56269757641b88, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.582784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845846ea-3995-4a6a-a16e-a3f8e4c82981 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.598005 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2557/5124] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:36 2022] PUT /identity/v3/projects/f9083719cf3f4408970021efc83e3d54/users/b9a65c6e684a4385ba56269757641b88/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:36.598620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845846ea-3995-4a6a-a16e-a3f8e4c82981 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.600255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-845846ea-3995-4a6a-a16e-a3f8e4c82981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:36.624568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6f6fa-7461-4bfd-9592-5e2fe472eda9 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.670868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845846ea-3995-4a6a-a16e-a3f8e4c82981 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.675686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845af1-b886-49a5-a4eb-6d19ecd736e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.676286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845af1-b886-49a5-a4eb-6d19ecd736e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.676562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845af1-b886-49a5-a4eb-6d19ecd736e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.676727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfe8cf-95e4-4145-8506-79bd85ea2650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.676853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845af1-b886-49a5-a4eb-6d19ecd736e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.677360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfe8cf-95e4-4145-8506-79bd85ea2650 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.677462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845af1-b886-49a5-a4eb-6d19ecd736e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.677638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfe8cf-95e4-4145-8506-79bd85ea2650 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.677907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfe8cf-95e4-4145-8506-79bd85ea2650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.679938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845af1-b886-49a5-a4eb-6d19ecd736e8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.680232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4bfe8cf-95e4-4145-8506-79bd85ea2650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:36.691880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfe8cf-95e4-4145-8506-79bd85ea2650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.693390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfe8cf-95e4-4145-8506-79bd85ea2650 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.697356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:36.707515 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2568/5125] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:36.708530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfe8cf-95e4-4145-8506-79bd85ea2650 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.709114 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2558/5126] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:36 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:36.709936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845af1-b886-49a5-a4eb-6d19ecd736e8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.713092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4bfe8cf-95e4-4145-8506-79bd85ea2650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:36.768862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:36.782593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845af1-b886-49a5-a4eb-6d19ecd736e8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.787522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f575e73-1880-4b4e-8955-6e78d99963a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.788177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575e73-1880-4b4e-8955-6e78d99963a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.788457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575e73-1880-4b4e-8955-6e78d99963a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.789245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575e73-1880-4b4e-8955-6e78d99963a5 admin admin] PATH_INFO: `/v3/projects/605f7455125b45f9a9ae4de530afcf9b/users/6085f73e836f4be8becf2182de6ebaa0/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.800272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfe8cf-95e4-4145-8506-79bd85ea2650 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.806002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f575e73-1880-4b4e-8955-6e78d99963a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=605f7455125b45f9a9ae4de530afcf9b, user_id=6085f73e836f4be8becf2182de6ebaa0, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.808992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f575e73-1880-4b4e-8955-6e78d99963a5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.829113 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2559/5127] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:36 2022] PUT /identity/v3/projects/605f7455125b45f9a9ae4de530afcf9b/users/6085f73e836f4be8becf2182de6ebaa0/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:36.830130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f575e73-1880-4b4e-8955-6e78d99963a5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.869658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.870297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.870568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.870843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/54ddcc4a2701438694789d1eba5c2ac5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.871989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=54ddcc4a2701438694789d1eba5c2ac5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.873206 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:36.873206 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:36.873206 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:36.874684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.903861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f575e73-1880-4b4e-8955-6e78d99963a5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:36.909819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b9fed-26fb-4811-910c-5237db06259a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:36.910556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b9fed-26fb-4811-910c-5237db06259a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.910831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b9fed-26fb-4811-910c-5237db06259a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.911142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b9fed-26fb-4811-910c-5237db06259a admin admin] PATH_INFO: `/v3/projects/16eb29186a7b4964a49882162a35cf06/users/a7eae9dc0a1a47d9910f49bb33316d91/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.917430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role 54ddcc4a2701438694789d1eba5c2ac5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:36.917714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:36.918475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:36.920809 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2569/5128] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:36 2022] DELETE /identity/v3/roles/54ddcc4a2701438694789d1eba5c2ac5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:36.921593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:36.922918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:36.931723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b9fed-26fb-4811-910c-5237db06259a admin admin] RBAC: Authorizing `identity:create_grant(project_id=16eb29186a7b4964a49882162a35cf06, user_id=a7eae9dc0a1a47d9910f49bb33316d91, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:36.934593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b9fed-26fb-4811-910c-5237db06259a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:36.948522 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2560/5129] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:36 2022] PUT /identity/v3/projects/16eb29186a7b4964a49882162a35cf06/users/a7eae9dc0a1a47d9910f49bb33316d91/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:36.949904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca48a-8482-400d-95ec-55e82d574cd5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:36.950184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca48a-8482-400d-95ec-55e82d574cd5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:36.950452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca48a-8482-400d-95ec-55e82d574cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:36.980057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6d5ca48a-8482-400d-95ec-55e82d574cd5 None None] MFA Rules not processed for user `b9a65c6e684a4385ba56269757641b88`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:36.981349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6d5ca48a-8482-400d-95ec-55e82d574cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:37.006566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1bcac-49df-4bc8-ac95-995ac37054e8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:37.023973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:37.059751 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2561/5130] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:37.059751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ca48a-8482-400d-95ec-55e82d574cd5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:37.059751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6d5ca48a-8482-400d-95ec-55e82d574cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:37.114989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855067d-aa0e-49bf-8bcc-8d9d4a4163b4 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:37.117771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855067d-aa0e-49bf-8bcc-8d9d4a4163b4 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.118270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855067d-aa0e-49bf-8bcc-8d9d4a4163b4 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.120890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855067d-aa0e-49bf-8bcc-8d9d4a4163b4 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/2ddd5072e41b41eeaa69eef6073ef157` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.127051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8855067d-aa0e-49bf-8bcc-8d9d4a4163b4 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_group(group_id=2ddd5072e41b41eeaa69eef6073ef157)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:37.129544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8855067d-aa0e-49bf-8bcc-8d9d4a4163b4 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:37.165065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:37.183304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ca48a-8482-400d-95ec-55e82d574cd5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:37.199342 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2570/5131] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:10:36 2022] DELETE /identity/v3/groups/2ddd5072e41b41eeaa69eef6073ef157 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:37.199342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52c30f-cc23-47b8-9545-0f0a3b4b8a94 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.199342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52c30f-cc23-47b8-9545-0f0a3b4b8a94 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.199342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52c30f-cc23-47b8-9545-0f0a3b4b8a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.201894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fc2f2-33ed-46c3-9528-746ad00a2eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:37.202521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fc2f2-33ed-46c3-9528-746ad00a2eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.202855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fc2f2-33ed-46c3-9528-746ad00a2eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.203124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fc2f2-33ed-46c3-9528-746ad00a2eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.204744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fc2f2-33ed-46c3-9528-746ad00a2eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:37.208546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fc2f2-33ed-46c3-9528-746ad00a2eb6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:37.232160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3f52c30f-cc23-47b8-9545-0f0a3b4b8a94 None None] MFA Rules not processed for user `6085f73e836f4be8becf2182de6ebaa0`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:37.233611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3f52c30f-cc23-47b8-9545-0f0a3b4b8a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:37.236520 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2562/5132] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:37 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:37.237645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fc2f2-33ed-46c3-9528-746ad00a2eb6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:37.239783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-863fc2f2-33ed-46c3-9528-746ad00a2eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:37.314223 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2571/5133] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:37.315513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81acb86-89d9-497f-a65d-49b93d039524 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.315796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81acb86-89d9-497f-a65d-49b93d039524 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.316059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81acb86-89d9-497f-a65d-49b93d039524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.322621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fc2f2-33ed-46c3-9528-746ad00a2eb6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:37.337295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a81acb86-89d9-497f-a65d-49b93d039524 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:37.338504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a81acb86-89d9-497f-a65d-49b93d039524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:37.393462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221585c-b3fa-4b42-92f6-90698d69fb04 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:37.394128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221585c-b3fa-4b42-92f6-90698d69fb04 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.394405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221585c-b3fa-4b42-92f6-90698d69fb04 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.394702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221585c-b3fa-4b42-92f6-90698d69fb04 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.398280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221585c-b3fa-4b42-92f6-90698d69fb04 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_grant(project_id=2335de37d6144200bbdef5af1a5cbab2, user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:37.401573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221585c-b3fa-4b42-92f6-90698d69fb04 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:37.415790 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2572/5134] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:37.416420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81acb86-89d9-497f-a65d-49b93d039524 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:37.421217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a81acb86-89d9-497f-a65d-49b93d039524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:37.426740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:37.428131 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2563/5135] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:37 2022] PUT /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:37.430036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221585c-b3fa-4b42-92f6-90698d69fb04 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:37.495310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81acb86-89d9-497f-a65d-49b93d039524 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:37.514963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221585c-b3fa-4b42-92f6-90698d69fb04 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:37.521564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc4dfb-4b88-45ac-b088-ba7aeff93140 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:37.522394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4dfb-4b88-45ac-b088-ba7aeff93140 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.522837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4dfb-4b88-45ac-b088-ba7aeff93140 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.523730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4dfb-4b88-45ac-b088-ba7aeff93140 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.524552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc4dfb-4b88-45ac-b088-ba7aeff93140 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:37.526474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc4dfb-4b88-45ac-b088-ba7aeff93140 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:37.552365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde09e8-f0c8-4060-9692-ff150ace7b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:37.556539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde09e8-f0c8-4060-9692-ff150ace7b93 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.556539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde09e8-f0c8-4060-9692-ff150ace7b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.556846 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2564/5136] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:37 2022] POST /identity/v3/users => generated 323 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:37.556956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde09e8-f0c8-4060-9692-ff150ace7b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.557080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc4dfb-4b88-45ac-b088-ba7aeff93140 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:37.567522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:37.624590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde09e8-f0c8-4060-9692-ff150ace7b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:37.626206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde09e8-f0c8-4060-9692-ff150ace7b93 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:37.636506 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2573/5137] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:37.637313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde09e8-f0c8-4060-9692-ff150ace7b93 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:37.638812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9dde09e8-f0c8-4060-9692-ff150ace7b93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:37.657309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc4dfb-4b88-45ac-b088-ba7aeff93140 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:37.663239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fbde6-9eac-482a-8b9e-c05e265582de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:37.664577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fbde6-9eac-482a-8b9e-c05e265582de admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.664934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fbde6-9eac-482a-8b9e-c05e265582de admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.665387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fbde6-9eac-482a-8b9e-c05e265582de admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.666729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fbde6-9eac-482a-8b9e-c05e265582de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:37.668849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fbde6-9eac-482a-8b9e-c05e265582de admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:37.683967 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2565/5138] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:37 2022] GET /identity/v3/roles => generated 2323 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:37.688288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fbde6-9eac-482a-8b9e-c05e265582de admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:37.704375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fbde6-9eac-482a-8b9e-c05e265582de admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:37.709471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371776-2298-4dce-85ce-860650d55c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:37.710186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371776-2298-4dce-85ce-860650d55c89 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.711223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371776-2298-4dce-85ce-860650d55c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.711815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371776-2298-4dce-85ce-860650d55c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.724080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80371776-2298-4dce-85ce-860650d55c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:37.724786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde09e8-f0c8-4060-9692-ff150ace7b93 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:37.725902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80371776-2298-4dce-85ce-860650d55c89 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:37.731142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64245725-8587-42e9-9977-b138e3060666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:37.731902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64245725-8587-42e9-9977-b138e3060666 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.732179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64245725-8587-42e9-9977-b138e3060666 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.732691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64245725-8587-42e9-9977-b138e3060666 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.734097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64245725-8587-42e9-9977-b138e3060666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:37.736029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64245725-8587-42e9-9977-b138e3060666 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:37.741090 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2566/5139] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:37.742111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371776-2298-4dce-85ce-860650d55c89 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:37.765352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:37.785448 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2574/5140] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:37 2022] POST /identity/v3/projects => generated 389 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:37.786772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64245725-8587-42e9-9977-b138e3060666 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:37.827913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371776-2298-4dce-85ce-860650d55c89 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:37.871481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64245725-8587-42e9-9977-b138e3060666 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:37.876574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec38737-c90b-4346-ace0-151db8114f21 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:37.877358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec38737-c90b-4346-ace0-151db8114f21 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.877901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec38737-c90b-4346-ace0-151db8114f21 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.878190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec38737-c90b-4346-ace0-151db8114f21 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.879001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec38737-c90b-4346-ace0-151db8114f21 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:37.887145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec38737-c90b-4346-ace0-151db8114f21 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:37.912183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88b90-52ea-4c14-aaca-a324852238bf tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:37.913005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88b90-52ea-4c14-aaca-a324852238bf tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:37.913381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88b90-52ea-4c14-aaca-a324852238bf tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:37.914315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88b90-52ea-4c14-aaca-a324852238bf tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:37.916665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c88b90-52ea-4c14-aaca-a324852238bf tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:37.917884 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-87c88b90-52ea-4c14-aaca-a324852238bf tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:37.917884 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:37.917884 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:37.918935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c88b90-52ea-4c14-aaca-a324852238bf tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:37.924017 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2575/5141] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:37 2022] POST /identity/v3/groups => generated 327 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:37.925102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec38737-c90b-4346-ace0-151db8114f21 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:37.970438 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2567/5142] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:37 2022] PUT /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 521 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:37.971384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88b90-52ea-4c14-aaca-a324852238bf tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:38.004064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:38.027449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec38737-c90b-4346-ace0-151db8114f21 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:38.065581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88b90-52ea-4c14-aaca-a324852238bf tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:38.071687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822a584-18ce-403e-b16a-0a285f4054ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:38.072323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822a584-18ce-403e-b16a-0a285f4054ee admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:38.072573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822a584-18ce-403e-b16a-0a285f4054ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:38.072825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822a584-18ce-403e-b16a-0a285f4054ee admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:38.073555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822a584-18ce-403e-b16a-0a285f4054ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:38.075456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822a584-18ce-403e-b16a-0a285f4054ee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:38.098660 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2568/5143] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:37 2022] POST /identity/v3/users => generated 301 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:38.099408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff854cb-d746-4271-b7fd-ae510257e75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:38.099661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822a584-18ce-403e-b16a-0a285f4054ee admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:38.100651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff854cb-d746-4271-b7fd-ae510257e75e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:38.100961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff854cb-d746-4271-b7fd-ae510257e75e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:38.101420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff854cb-d746-4271-b7fd-ae510257e75e admin admin] PATH_INFO: `/v3/projects/5c60521f47ad4003b2c6507621c0d448/users/db0b24a87eac48beb5293d777077e64f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:38.129867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff854cb-d746-4271-b7fd-ae510257e75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c60521f47ad4003b2c6507621c0d448, user_id=db0b24a87eac48beb5293d777077e64f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:38.135393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff854cb-d746-4271-b7fd-ae510257e75e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:38.158943 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2576/5144] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:37 2022] PUT /identity/v3/projects/5c60521f47ad4003b2c6507621c0d448/users/db0b24a87eac48beb5293d777077e64f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:38.159945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff854cb-d746-4271-b7fd-ae510257e75e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:38.181395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822a584-18ce-403e-b16a-0a285f4054ee admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:38.220681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:38.307407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485688b1-a96b-44e1-9dae-6eaed76c64dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:38.308203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485688b1-a96b-44e1-9dae-6eaed76c64dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:38.308486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485688b1-a96b-44e1-9dae-6eaed76c64dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:38.308759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485688b1-a96b-44e1-9dae-6eaed76c64dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/3b57a5d08eb94f9fbb11d797cd81e69d/users/670596e667024649b450c11737c66d8f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:38.316730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff854cb-d746-4271-b7fd-ae510257e75e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:38.344024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96adbd-9ccc-47c5-9a1c-dd9b90abc260 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:38.344887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96adbd-9ccc-47c5-9a1c-dd9b90abc260 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:38.345357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96adbd-9ccc-47c5-9a1c-dd9b90abc260 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:38.346116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96adbd-9ccc-47c5-9a1c-dd9b90abc260 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:38.347794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96adbd-9ccc-47c5-9a1c-dd9b90abc260 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Adding query filter params (scope.project.id=2335de37d6144200bbdef5af1a5cbab2, user.id=4d18588ffdd8497ab8903f4a5a81fe87) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:38.351334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96adbd-9ccc-47c5-9a1c-dd9b90abc260 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:38.354796 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-4c96adbd-9ccc-47c5-9a1c-dd9b90abc260 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:38.354796 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:38.354796 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:38.371848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96adbd-9ccc-47c5-9a1c-dd9b90abc260 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:38.383007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485688b1-a96b-44e1-9dae-6eaed76c64dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=3b57a5d08eb94f9fbb11d797cd81e69d, user_id=670596e667024649b450c11737c66d8f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:38.389689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485688b1-a96b-44e1-9dae-6eaed76c64dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:38.466727 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2569/5145] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Oct 15 21:10:38 2022] PUT /identity/v3/groups/3b57a5d08eb94f9fbb11d797cd81e69d/users/670596e667024649b450c11737c66d8f => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:38.473671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485688b1-a96b-44e1-9dae-6eaed76c64dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:38.591036 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2577/5146] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Oct 15 21:10:38 2022] GET /identity/v3/role_assignments?scope.project.id=2335de37d6144200bbdef5af1a5cbab2&user.id=4d18588ffdd8497ab8903f4a5a81fe87&effective => generated 1061 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:38.592371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96adbd-9ccc-47c5-9a1c-dd9b90abc260 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:38.605308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:38.684930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485688b1-a96b-44e1-9dae-6eaed76c64dc tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:38.713321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6581006-6098-462b-bea4-5d53108280a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:38.713963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6581006-6098-462b-bea4-5d53108280a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:38.714669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6581006-6098-462b-bea4-5d53108280a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:38.714965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6581006-6098-462b-bea4-5d53108280a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:38.715614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6581006-6098-462b-bea4-5d53108280a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:38.719621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6581006-6098-462b-bea4-5d53108280a1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:38.739873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96adbd-9ccc-47c5-9a1c-dd9b90abc260 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:38.743148 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2570/5147] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:38 2022] GET /identity/v3/roles => generated 2323 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:38.744181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6581006-6098-462b-bea4-5d53108280a1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:38.752802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f91db-c1e2-4ca0-98a2-f144ab46a5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:38.753565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f91db-c1e2-4ca0-98a2-f144ab46a5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:38.753912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f91db-c1e2-4ca0-98a2-f144ab46a5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:38.754249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f91db-c1e2-4ca0-98a2-f144ab46a5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:38.755393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f91db-c1e2-4ca0-98a2-f144ab46a5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:38.758051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f91db-c1e2-4ca0-98a2-f144ab46a5c0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:38.777564 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2578/5148] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:38 2022] GET /identity/v3/roles => generated 2323 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:38.778837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f91db-c1e2-4ca0-98a2-f144ab46a5c0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:38.866913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6581006-6098-462b-bea4-5d53108280a1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:38.874021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfef021-929c-4832-bd5e-3a7a48b78f06 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:38.874946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfef021-929c-4832-bd5e-3a7a48b78f06 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:38.875916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfef021-929c-4832-bd5e-3a7a48b78f06 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:38.876411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfef021-929c-4832-bd5e-3a7a48b78f06 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:38.877904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfef021-929c-4832-bd5e-3a7a48b78f06 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:38.880867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfef021-929c-4832-bd5e-3a7a48b78f06 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:38.906174 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2571/5149] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:38 2022] POST /identity/v3/groups => generated 326 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:38.907354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfef021-929c-4832-bd5e-3a7a48b78f06 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:38.907478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f91db-c1e2-4ca0-98a2-f144ab46a5c0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:38.927256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c0736-b489-4917-9ae8-0807c8769fbd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:38.927256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0736-b489-4917-9ae8-0807c8769fbd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:38.927256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0736-b489-4917-9ae8-0807c8769fbd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:38.927692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0736-b489-4917-9ae8-0807c8769fbd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:38.927692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c0736-b489-4917-9ae8-0807c8769fbd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:38.927692 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-146c0736-b489-4917-9ae8-0807c8769fbd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:38.927692 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:38.927692 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:38.927692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c0736-b489-4917-9ae8-0807c8769fbd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:38.951739 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2579/5150] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:38 2022] DELETE /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:38.954751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c0736-b489-4917-9ae8-0807c8769fbd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:38.959996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfef021-929c-4832-bd5e-3a7a48b78f06 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:39.021512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:39.105353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c0736-b489-4917-9ae8-0807c8769fbd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:39.105549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121c72a-1e52-4f31-98f5-3bd49abd5a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:39.106328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121c72a-1e52-4f31-98f5-3bd49abd5a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:39.108881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121c72a-1e52-4f31-98f5-3bd49abd5a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:39.109217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121c72a-1e52-4f31-98f5-3bd49abd5a63 admin admin] PATH_INFO: `/v3/projects/4c1dd434c4d749b48b131f48655cd76f/users/cdaa336c6ff2489c8a8d360fbd418177/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:39.117376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffa5cb-cf20-4b94-8551-8f362d15edbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:39.118057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffa5cb-cf20-4b94-8551-8f362d15edbd admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:39.118470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffa5cb-cf20-4b94-8551-8f362d15edbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:39.118746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffa5cb-cf20-4b94-8551-8f362d15edbd admin admin] PATH_INFO: `/v3/projects/5c60521f47ad4003b2c6507621c0d448/users/db0b24a87eac48beb5293d777077e64f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:39.125158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffa5cb-cf20-4b94-8551-8f362d15edbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c60521f47ad4003b2c6507621c0d448, user_id=db0b24a87eac48beb5293d777077e64f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:39.127831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffa5cb-cf20-4b94-8551-8f362d15edbd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:39.152682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121c72a-1e52-4f31-98f5-3bd49abd5a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1dd434c4d749b48b131f48655cd76f, user_id=cdaa336c6ff2489c8a8d360fbd418177, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:39.157415 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2580/5151] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:38 2022] PUT /identity/v3/projects/5c60521f47ad4003b2c6507621c0d448/users/db0b24a87eac48beb5293d777077e64f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:39.160768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121c72a-1e52-4f31-98f5-3bd49abd5a63 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:39.167494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffa5cb-cf20-4b94-8551-8f362d15edbd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:39.222990 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2572/5152] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:38 2022] PUT /identity/v3/projects/4c1dd434c4d749b48b131f48655cd76f/users/cdaa336c6ff2489c8a8d360fbd418177/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:39.223903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121c72a-1e52-4f31-98f5-3bd49abd5a63 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:39.326126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:39.361585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121c72a-1e52-4f31-98f5-3bd49abd5a63 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:39.371682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49411f9c-d1ba-4019-a6ea-cefcf47be766 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:39.376021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49411f9c-d1ba-4019-a6ea-cefcf47be766 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:39.377993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49411f9c-d1ba-4019-a6ea-cefcf47be766 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:39.378625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49411f9c-d1ba-4019-a6ea-cefcf47be766 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:39.381836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49411f9c-d1ba-4019-a6ea-cefcf47be766 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Adding query filter params (scope.project.id=2335de37d6144200bbdef5af1a5cbab2, user.id=4d18588ffdd8497ab8903f4a5a81fe87) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:39.384230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49411f9c-d1ba-4019-a6ea-cefcf47be766 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:39.387125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49411f9c-d1ba-4019-a6ea-cefcf47be766 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:39.420199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffa5cb-cf20-4b94-8551-8f362d15edbd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:39.516881 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2573/5153] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Oct 15 21:10:39 2022] GET /identity/v3/role_assignments?scope.project.id=2335de37d6144200bbdef5af1a5cbab2&user.id=4d18588ffdd8497ab8903f4a5a81fe87&effective => generated 577 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:39.517838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49411f9c-d1ba-4019-a6ea-cefcf47be766 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:39.574770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbb9f4-7e15-4cb8-953c-501bbcc9c035 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:39.575490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbb9f4-7e15-4cb8-953c-501bbcc9c035 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:39.577110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbb9f4-7e15-4cb8-953c-501bbcc9c035 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:39.577110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbb9f4-7e15-4cb8-953c-501bbcc9c035 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/e53763386e96446daaa27dae73739c05/users/670596e667024649b450c11737c66d8f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:39.584819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbb9f4-7e15-4cb8-953c-501bbcc9c035 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=e53763386e96446daaa27dae73739c05, user_id=670596e667024649b450c11737c66d8f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:39.588114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbb9f4-7e15-4cb8-953c-501bbcc9c035 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:39.615755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49411f9c-d1ba-4019-a6ea-cefcf47be766 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:39.626243 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2581/5154] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Oct 15 21:10:39 2022] PUT /identity/v3/groups/e53763386e96446daaa27dae73739c05/users/670596e667024649b450c11737c66d8f => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:39.628100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d583b-6f28-49af-94c1-d4f73ca06ba7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:39.628390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d583b-6f28-49af-94c1-d4f73ca06ba7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:39.628793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d583b-6f28-49af-94c1-d4f73ca06ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:39.633934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7754a3d-f920-4455-8cfe-7f35462cf4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:39.634548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7754a3d-f920-4455-8cfe-7f35462cf4ad admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:39.634820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7754a3d-f920-4455-8cfe-7f35462cf4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:39.635087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7754a3d-f920-4455-8cfe-7f35462cf4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:39.635972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7754a3d-f920-4455-8cfe-7f35462cf4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:39.639760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7754a3d-f920-4455-8cfe-7f35462cf4ad admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:39.672515 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2574/5155] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:39 2022] GET /identity/v3/roles => generated 2323 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:39.673739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7754a3d-f920-4455-8cfe-7f35462cf4ad admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:39.689885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d45d583b-6f28-49af-94c1-d4f73ca06ba7 None None] MFA Rules not processed for user `db0b24a87eac48beb5293d777077e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:39.691239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d45d583b-6f28-49af-94c1-d4f73ca06ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:39.700242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7754a3d-f920-4455-8cfe-7f35462cf4ad admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:39.732259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:39.818211 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2582/5156] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:39.818211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d583b-6f28-49af-94c1-d4f73ca06ba7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:39.823957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e107d7-00db-4e60-9ff7-29cf933dc1ec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:39.824616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e107d7-00db-4e60-9ff7-29cf933dc1ec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:39.824893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e107d7-00db-4e60-9ff7-29cf933dc1ec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:39.825215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e107d7-00db-4e60-9ff7-29cf933dc1ec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:39.827044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e107d7-00db-4e60-9ff7-29cf933dc1ec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:39.828238 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-a1e107d7-00db-4e60-9ff7-29cf933dc1ec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:39.828238 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:39.828238 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:39.828940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e107d7-00db-4e60-9ff7-29cf933dc1ec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-a1e107d7-00db-4e60-9ff7-29cf933dc1ec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] 6e3634c6ea4d4c6eb96fe49f863b4e4e does not imply 4fb9d9a02ac04634bbc8bf4eaa9188ce.: keystone.exception.ImpliedRoleNotFound: 6e3634c6ea4d4c6eb96fe49f863b4e4e does not imply 4fb9d9a02ac04634bbc8bf4eaa9188ce. Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = query.one() Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._iter().one() Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._only_one_row( Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exc.NoResultFound( Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:39.841309 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Oct 15 21:10:39.842483 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Oct 15 21:10:39.843420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Oct 15 21:10:39.843420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Oct 15 21:10:39.843420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Oct 15 21:10:39.843420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 6e3634c6ea4d4c6eb96fe49f863b4e4e does not imply 4fb9d9a02ac04634bbc8bf4eaa9188ce. Oct 15 21:10:39.843420 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:39.845875 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2575/5157] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:39 2022] DELETE /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 137 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:10:39.846750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e107d7-00db-4e60-9ff7-29cf933dc1ec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:39.909614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:39.949682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e107d7-00db-4e60-9ff7-29cf933dc1ec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:39.954697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e330f-92ad-4070-a518-a5ac210a0218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:39.955710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e330f-92ad-4070-a518-a5ac210a0218 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:39.956161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e330f-92ad-4070-a518-a5ac210a0218 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:39.956536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e330f-92ad-4070-a518-a5ac210a0218 admin admin] PATH_INFO: `/v3/projects/4c1dd434c4d749b48b131f48655cd76f/users/cdaa336c6ff2489c8a8d360fbd418177/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:39.959177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e330f-92ad-4070-a518-a5ac210a0218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1dd434c4d749b48b131f48655cd76f, user_id=cdaa336c6ff2489c8a8d360fbd418177, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:39.962186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e330f-92ad-4070-a518-a5ac210a0218 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:39.977338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d583b-6f28-49af-94c1-d4f73ca06ba7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:39.989433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2576/5158] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:39 2022] PUT /identity/v3/projects/4c1dd434c4d749b48b131f48655cd76f/users/cdaa336c6ff2489c8a8d360fbd418177/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:39.990646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e330f-92ad-4070-a518-a5ac210a0218 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:40.069566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e330f-92ad-4070-a518-a5ac210a0218 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:40.075366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:40.076243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:40.076721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:40.077130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:40.081419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=2335de37d6144200bbdef5af1a5cbab2, user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:40.084441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:40.091421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role 6e3634c6ea4d4c6eb96fe49f863b4e4e was removed from user 4d18588ffdd8497ab8903f4a5a81fe87 on project 2335de37d6144200bbdef5af1a5cbab2. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:40.091973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:40.093348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:40.121448 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2577/5159] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Oct 15 21:10:39 2022] DELETE /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:40.123764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:40.126416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174460ac-c9cf-4858-928c-547486f3d2b2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:40.127844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174460ac-c9cf-4858-928c-547486f3d2b2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:40.128792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:40.129317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174460ac-c9cf-4858-928c-547486f3d2b2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:40.130493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174460ac-c9cf-4858-928c-547486f3d2b2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/users/670596e667024649b450c11737c66d8f/groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:40.135046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174460ac-c9cf-4858-928c-547486f3d2b2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:list_groups_for_user(user_id=670596e667024649b450c11737c66d8f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:40.142651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174460ac-c9cf-4858-928c-547486f3d2b2 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:40.186852 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2583/5160] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:10:39 2022] GET /identity/v3/users/670596e667024649b450c11737c66d8f/groups => generated 839 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:40.207689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c59d7f-fe39-45ca-8866-1848d8373687 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:40.208022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c59d7f-fe39-45ca-8866-1848d8373687 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:40.208285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c59d7f-fe39-45ca-8866-1848d8373687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:40.223677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2e350-7be4-438b-9492-3afa0b27f86e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:40.239571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e48bf6-55f8-4350-8a1a-6f1aaa146a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:40.240472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e48bf6-55f8-4350-8a1a-6f1aaa146a9c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:40.240755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e48bf6-55f8-4350-8a1a-6f1aaa146a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:40.241083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e48bf6-55f8-4350-8a1a-6f1aaa146a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:40.243732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-35e48bf6-55f8-4350-8a1a-6f1aaa146a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:40.275437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-50c59d7f-fe39-45ca-8866-1848d8373687 None None] MFA Rules not processed for user `cdaa336c6ff2489c8a8d360fbd418177`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:40.276664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-50c59d7f-fe39-45ca-8866-1848d8373687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:40.371009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:40.400278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e48bf6-55f8-4350-8a1a-6f1aaa146a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:40.402348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e48bf6-55f8-4350-8a1a-6f1aaa146a9c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:40.427061 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2578/5161] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:40.427948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e48bf6-55f8-4350-8a1a-6f1aaa146a9c service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:40.434974 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2584/5162] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:40.435108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-35e48bf6-55f8-4350-8a1a-6f1aaa146a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:40.436032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c59d7f-fe39-45ca-8866-1848d8373687 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:40.437697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-50c59d7f-fe39-45ca-8866-1848d8373687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:40.458474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:40.547135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c59d7f-fe39-45ca-8866-1848d8373687 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:40.564049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d723e-9bb6-46c7-8825-fa6311d38be6 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:40.564690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d723e-9bb6-46c7-8825-fa6311d38be6 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:40.564967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d723e-9bb6-46c7-8825-fa6311d38be6 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:40.565280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d723e-9bb6-46c7-8825-fa6311d38be6 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/e53763386e96446daaa27dae73739c05` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:40.566740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d723e-9bb6-46c7-8825-fa6311d38be6 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_group(group_id=e53763386e96446daaa27dae73739c05)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:40.571118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d723e-9bb6-46c7-8825-fa6311d38be6 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:40.574955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e48bf6-55f8-4350-8a1a-6f1aaa146a9c service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:40.589945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71091dc8-e77b-4ea9-8e13-bd2488d758b4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:40.590579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71091dc8-e77b-4ea9-8e13-bd2488d758b4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:40.590860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71091dc8-e77b-4ea9-8e13-bd2488d758b4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:40.591126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71091dc8-e77b-4ea9-8e13-bd2488d758b4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:40.594339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71091dc8-e77b-4ea9-8e13-bd2488d758b4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:40.595957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71091dc8-e77b-4ea9-8e13-bd2488d758b4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:40.631126 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2579/5163] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:40 2022] POST /identity/v3/roles => generated 271 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:40.632668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71091dc8-e77b-4ea9-8e13-bd2488d758b4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:40.650143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71091dc8-e77b-4ea9-8e13-bd2488d758b4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:40.655518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b4a55-4ad5-4c34-abe1-680bee5d77ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:40.656578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b4a55-4ad5-4c34-abe1-680bee5d77ca service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:40.657069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b4a55-4ad5-4c34-abe1-680bee5d77ca service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:40.658170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b4a55-4ad5-4c34-abe1-680bee5d77ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:40.661180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7c5b4a55-4ad5-4c34-abe1-680bee5d77ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:40.670602 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2585/5164] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:10:40 2022] DELETE /identity/v3/groups/e53763386e96446daaa27dae73739c05 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:40.671619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d723e-9bb6-46c7-8825-fa6311d38be6 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:40.678629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b49d723e-9bb6-46c7-8825-fa6311d38be6 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:40.796292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b4a55-4ad5-4c34-abe1-680bee5d77ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:40.801289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b4a55-4ad5-4c34-abe1-680bee5d77ca service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:40.830339 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2580/5165] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:40.831080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d723e-9bb6-46c7-8825-fa6311d38be6 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:40.832011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b4a55-4ad5-4c34-abe1-680bee5d77ca service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:40.845561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d304c-b2ac-41c6-bc2a-675a08924328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:40.847240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d304c-b2ac-41c6-bc2a-675a08924328 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:40.847240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d304c-b2ac-41c6-bc2a-675a08924328 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:40.850275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d304c-b2ac-41c6-bc2a-675a08924328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:40.854932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a84d304c-b2ac-41c6-bc2a-675a08924328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:40.858202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b4a55-4ad5-4c34-abe1-680bee5d77ca service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:40.950250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:41.011938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77769b1-fa0a-45b5-9dc5-371b38d0ecc2 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:41.014264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77769b1-fa0a-45b5-9dc5-371b38d0ecc2 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:41.015389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77769b1-fa0a-45b5-9dc5-371b38d0ecc2 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:41.015734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77769b1-fa0a-45b5-9dc5-371b38d0ecc2 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:41.019454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77769b1-fa0a-45b5-9dc5-371b38d0ecc2 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Adding query filter params (domain_id=46b5964105bc4469ab876d72a935c181) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:41.019838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77769b1-fa0a-45b5-9dc5-371b38d0ecc2 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:41.022750 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-f77769b1-fa0a-45b5-9dc5-371b38d0ecc2 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domain_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:41.022750 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:41.022750 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:41.026112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77769b1-fa0a-45b5-9dc5-371b38d0ecc2 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:41.030053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d304c-b2ac-41c6-bc2a-675a08924328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:41.034540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d304c-b2ac-41c6-bc2a-675a08924328 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:41.049674 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2581/5166] 10.222.0.16 () {58 vars in 1207 bytes} [Sat Oct 15 21:10:40 2022] GET /identity/v3/roles?domain_id=46b5964105bc4469ab876d72a935c181 => generated 408 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:41.050681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77769b1-fa0a-45b5-9dc5-371b38d0ecc2 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:41.107389 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2586/5167] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:41.110222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d304c-b2ac-41c6-bc2a-675a08924328 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:41.141920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d304c-b2ac-41c6-bc2a-675a08924328 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:41.159951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:41.161107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:41.161662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:41.162234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/c176c3ebfa934e9da65b0ebb9cacf2f7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:41.165806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=c176c3ebfa934e9da65b0ebb9cacf2f7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:41.172555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:41.199607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:41.296182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77769b1-fa0a-45b5-9dc5-371b38d0ecc2 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:41.301354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role c176c3ebfa934e9da65b0ebb9cacf2f7 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:41.301638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.303463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.310899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12236-9ad2-4e8b-ad84-31b4fd903c1d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:41.315356 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2587/5168] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:41 2022] DELETE /identity/v3/roles/c176c3ebfa934e9da65b0ebb9cacf2f7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:41.316959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12236-9ad2-4e8b-ad84-31b4fd903c1d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:41.316959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12236-9ad2-4e8b-ad84-31b4fd903c1d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:41.316959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12236-9ad2-4e8b-ad84-31b4fd903c1d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/groups/3b57a5d08eb94f9fbb11d797cd81e69d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:41.317227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:41.317968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12236-9ad2-4e8b-ad84-31b4fd903c1d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_group(group_id=3b57a5d08eb94f9fbb11d797cd81e69d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:41.319299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:41.324373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12236-9ad2-4e8b-ad84-31b4fd903c1d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:41.444529 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2582/5169] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:10:41 2022] DELETE /identity/v3/groups/3b57a5d08eb94f9fbb11d797cd81e69d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:41.448775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12236-9ad2-4e8b-ad84-31b4fd903c1d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:41.452851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cba12236-9ad2-4e8b-ad84-31b4fd903c1d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:41.479583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36508ce-7a48-41ef-b658-b23a50293d8c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:41.570526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12236-9ad2-4e8b-ad84-31b4fd903c1d tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:41.599925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:41.601027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:41.601565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:41.602071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/users/670596e667024649b450c11737c66d8f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:41.603709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_user(user_id=670596e667024649b450c11737c66d8f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:41.609010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:41.621276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a286070-2ca5-4a6b-a1ee-928c27917466 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:41.622142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286070-2ca5-4a6b-a1ee-928c27917466 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:41.622636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286070-2ca5-4a6b-a1ee-928c27917466 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:41.623257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286070-2ca5-4a6b-a1ee-928c27917466 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:41.624177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a286070-2ca5-4a6b-a1ee-928c27917466 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Adding query filter params (domain_id=46b5964105bc4469ab876d72a935c181) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:41.624806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a286070-2ca5-4a6b-a1ee-928c27917466 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:41.626125 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-5a286070-2ca5-4a6b-a1ee-928c27917466 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domain_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:41.626125 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:41.626125 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:41.627462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a286070-2ca5-4a6b-a1ee-928c27917466 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:41.635830 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2588/5170] 10.222.0.16 () {58 vars in 1207 bytes} [Sat Oct 15 21:10:41 2022] GET /identity/v3/roles?domain_id=46b5964105bc4469ab876d72a935c181 => generated 148 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:41.639794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a286070-2ca5-4a6b-a1ee-928c27917466 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:41.656301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a286070-2ca5-4a6b-a1ee-928c27917466 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:41.664969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d42e0-37d9-4f27-8ef0-bb57e35562d4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:41.665686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d42e0-37d9-4f27-8ef0-bb57e35562d4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:41.666040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d42e0-37d9-4f27-8ef0-bb57e35562d4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:41.666319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d42e0-37d9-4f27-8ef0-bb57e35562d4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/c176c3ebfa934e9da65b0ebb9cacf2f7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:41.682191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d42e0-37d9-4f27-8ef0-bb57e35562d4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-e65d42e0-37d9-4f27-8ef0-bb57e35562d4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Could not find role: c176c3ebfa934e9da65b0ebb9cacf2f7.: keystone.exception.RoleNotFound: Could not find role: c176c3ebfa934e9da65b0ebb9cacf2f7. Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 165, in delete Oct 15 21:10:41.689091 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_role') Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:10:41.690282 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: c176c3ebfa934e9da65b0ebb9cacf2f7. Oct 15 21:10:41.691430 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:41.700168 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2589/5171] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:41 2022] DELETE /identity/v3/roles/c176c3ebfa934e9da65b0ebb9cacf2f7 => generated 110 bytes in 59 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:10:41.700168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d42e0-37d9-4f27-8ef0-bb57e35562d4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:41.712984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '670596e667024649b450c11737c66d8f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.716482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d42e0-37d9-4f27-8ef0-bb57e35562d4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:41.723848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa3428-7e5b-485e-9bff-a1d4075d530b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:41.724729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '670596e667024649b450c11737c66d8f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.724901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa3428-7e5b-485e-9bff-a1d4075d530b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:41.725472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa3428-7e5b-485e-9bff-a1d4075d530b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:41.726152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa3428-7e5b-485e-9bff-a1d4075d530b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/46b5964105bc4469ab876d72a935c181/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:41.735595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa3428-7e5b-485e-9bff-a1d4075d530b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=46b5964105bc4469ab876d72a935c181, group_id=5f2f341b06584412b37b181285bb7608, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:41.739579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa3428-7e5b-485e-9bff-a1d4075d530b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:41.744723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '670596e667024649b450c11737c66d8f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.756868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '670596e667024649b450c11737c66d8f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.758543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:41.759650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '670596e667024649b450c11737c66d8f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.774716 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2590/5172] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:41 2022] PUT /identity/v3/domains/46b5964105bc4469ab876d72a935c181/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:41.775908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa3428-7e5b-485e-9bff-a1d4075d530b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:41.778688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0aaa3428-7e5b-485e-9bff-a1d4075d530b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:41.782381 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2583/5173] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:41 2022] DELETE /identity/v3/users/670596e667024649b450c11737c66d8f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:41.787378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:41.790597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:41.813943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:41.887827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa3428-7e5b-485e-9bff-a1d4075d530b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:41.892140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38dcb1-2944-450d-aca2-6f454017c325 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:41.898791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:41.899899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:41.900132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:41.900401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/domains/0684a950e59f4e049b8d2df29a509648` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:41.901906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=0684a950e59f4e049b8d2df29a509648)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:41.904114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:41.934155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '0684a950e59f4e049b8d2df29a509648'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.935064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '0684a950e59f4e049b8d2df29a509648'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.969347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99649e6a-ac58-490e-aed1-06647a8e4f89 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:41.970283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99649e6a-ac58-490e-aed1-06647a8e4f89 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:41.970621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99649e6a-ac58-490e-aed1-06647a8e4f89 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:41.970998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99649e6a-ac58-490e-aed1-06647a8e4f89 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/46b5964105bc4469ab876d72a935c181/groups/5f2f341b06584412b37b181285bb7608/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:41.973185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99649e6a-ac58-490e-aed1-06647a8e4f89 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=46b5964105bc4469ab876d72a935c181, group_id=5f2f341b06584412b37b181285bb7608)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:41.974776 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-99649e6a-ac58-490e-aed1-06647a8e4f89 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:41.974776 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:41.974776 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:41.975179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '0684a950e59f4e049b8d2df29a509648'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.976025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '0684a950e59f4e049b8d2df29a509648'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:41.977356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99649e6a-ac58-490e-aed1-06647a8e4f89 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:41.981638 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2584/5174] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:10:41 2022] PATCH /identity/v3/domains/0684a950e59f4e049b8d2df29a509648 => generated 276 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:41.983968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:41.986174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:41.998474 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2591/5175] 10.222.0.16 () {58 vars in 1365 bytes} [Sat Oct 15 21:10:41 2022] GET /identity/v3/domains/46b5964105bc4469ab876d72a935c181/groups/5f2f341b06584412b37b181285bb7608/roles => generated 410 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:42.002233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99649e6a-ac58-490e-aed1-06647a8e4f89 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:42.004032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-99649e6a-ac58-490e-aed1-06647a8e4f89 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:42.022065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99649e6a-ac58-490e-aed1-06647a8e4f89 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:42.086155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929db79c-f519-42c0-9860-6b9f03efbce7 tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:42.095180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af7551dad082418f9cbfba3107a743cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:42.095963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:42.096241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:42.096573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] PATH_INFO: `/v3/domains/0684a950e59f4e049b8d2df29a509648` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:42.107397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba4f5e-b4a8-4a93-b61b-71e8c205932f tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:42.108365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba4f5e-b4a8-4a93-b61b-71e8c205932f tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:42.108654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba4f5e-b4a8-4a93-b61b-71e8c205932f tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:42.109347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba4f5e-b4a8-4a93-b61b-71e8c205932f tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/46b5964105bc4469ab876d72a935c181/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:42.112362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba4f5e-b4a8-4a93-b61b-71e8c205932f tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=46b5964105bc4469ab876d72a935c181, group_id=5f2f341b06584412b37b181285bb7608, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:42.114190 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-baba4f5e-b4a8-4a93-b61b-71e8c205932f tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:42.114190 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:42.114190 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:42.115444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=0684a950e59f4e049b8d2df29a509648)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:42.117445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba4f5e-b4a8-4a93-b61b-71e8c205932f tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:42.117800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:42.129370 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2592/5176] 10.222.0.16 () {58 vars in 1465 bytes} [Sat Oct 15 21:10:42 2022] HEAD /identity/v3/domains/46b5964105bc4469ab876d72a935c181/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:42.131899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba4f5e-b4a8-4a93-b61b-71e8c205932f tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:42.145114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba4f5e-b4a8-4a93-b61b-71e8c205932f tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:42.151422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:42.152652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:42.153791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:42.154270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/46b5964105bc4469ab876d72a935c181/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:42.155915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '0684a950e59f4e049b8d2df29a509648'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.157322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=46b5964105bc4469ab876d72a935c181, group_id=5f2f341b06584412b37b181285bb7608, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:42.161806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:42.169591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role fd1a838a0c6742c4a46797eb352eded1 was removed from group 5f2f341b06584412b37b181285bb7608 on domain 46b5964105bc4469ab876d72a935c181. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:42.169878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '0684a950e59f4e049b8d2df29a509648'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.170458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.172459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.196735 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2593/5177] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Oct 15 21:10:42 2022] DELETE /identity/v3/domains/46b5964105bc4469ab876d72a935c181/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:42.198140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:42.200326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:42.231135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:42.252094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:42.256586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0684a950e59f4e049b8d2df29a509648', 'request_id': 'req-9e4fe545-3744-4820-b390-ae297340590f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.274272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0684a950e59f4e049b8d2df29a509648', 'request_id': 'req-9e4fe545-3744-4820-b390-ae297340590f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.287381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0684a950e59f4e049b8d2df29a509648', 'request_id': 'req-9e4fe545-3744-4820-b390-ae297340590f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.311134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] The token cache is being invalidate because project 0684a950e59f4e049b8d2df29a509648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:42.311477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.312122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.315100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fbbbd-d8c7-40f4-828c-f5c628904f82 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:42.321545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '0684a950e59f4e049b8d2df29a509648', 'request_id': 'req-9e4fe545-3744-4820-b390-ae297340590f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.322327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '0684a950e59f4e049b8d2df29a509648', 'request_id': 'req-9e4fe545-3744-4820-b390-ae297340590f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.323033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '0684a950e59f4e049b8d2df29a509648', 'request_id': 'req-9e4fe545-3744-4820-b390-ae297340590f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.334888 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2585/5178] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:10:41 2022] DELETE /identity/v3/domains/0684a950e59f4e049b8d2df29a509648 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:42.340685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:42.344355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:42.446759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cec28f-09de-442b-95ae-af8b6ff619a6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:42.447445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cec28f-09de-442b-95ae-af8b6ff619a6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:42.447728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cec28f-09de-442b-95ae-af8b6ff619a6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:42.448056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cec28f-09de-442b-95ae-af8b6ff619a6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:42.450855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cec28f-09de-442b-95ae-af8b6ff619a6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_grant(project_id=2335de37d6144200bbdef5af1a5cbab2, group_id=5f2f341b06584412b37b181285bb7608, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:42.453846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cec28f-09de-442b-95ae-af8b6ff619a6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:42.465197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fe545-3744-4820-b390-ae297340590f tempest-GroupsV3TestJSON-1827080972 tempest-GroupsV3TestJSON-1827080972-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:42.479330 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2594/5179] 10.222.0.16 () {60 vars in 1486 bytes} [Sat Oct 15 21:10:42 2022] PUT /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:42.480384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cec28f-09de-442b-95ae-af8b6ff619a6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:42.482221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-12cec28f-09de-442b-95ae-af8b6ff619a6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:42.578840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:42.579488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:42.580549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:42.580815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] PATH_INFO: `/v3/users/f02f11d363764208bb0c0a7be5a7e705` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:42.582792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f02f11d363764208bb0c0a7be5a7e705)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:42.587344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:42.615463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cec28f-09de-442b-95ae-af8b6ff619a6 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:42.628932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffe735-04e9-456c-b2bb-cfe02970c0a4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:42.629642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffe735-04e9-456c-b2bb-cfe02970c0a4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:42.629923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffe735-04e9-456c-b2bb-cfe02970c0a4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:42.630201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffe735-04e9-456c-b2bb-cfe02970c0a4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/groups/5f2f341b06584412b37b181285bb7608/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:42.639926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffe735-04e9-456c-b2bb-cfe02970c0a4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_grants(project_id=2335de37d6144200bbdef5af1a5cbab2, group_id=5f2f341b06584412b37b181285bb7608)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:42.644307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffe735-04e9-456c-b2bb-cfe02970c0a4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:42.674741 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2595/5180] 10.222.0.16 () {58 vars in 1368 bytes} [Sat Oct 15 21:10:42 2022] GET /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/groups/5f2f341b06584412b37b181285bb7608/roles => generated 411 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:42.681972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffe735-04e9-456c-b2bb-cfe02970c0a4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:42.703139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffe735-04e9-456c-b2bb-cfe02970c0a4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:42.709925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefe247-e862-44e0-ab38-df928d8866c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:42.711716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefe247-e862-44e0-ab38-df928d8866c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:42.712250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefe247-e862-44e0-ab38-df928d8866c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:42.712764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefe247-e862-44e0-ab38-df928d8866c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/groups/5f2f341b06584412b37b181285bb7608/users/4d18588ffdd8497ab8903f4a5a81fe87` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:42.715006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefe247-e862-44e0-ab38-df928d8866c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=5f2f341b06584412b37b181285bb7608, user_id=4d18588ffdd8497ab8903f4a5a81fe87)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:42.715534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f02f11d363764208bb0c0a7be5a7e705'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.717523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefe247-e862-44e0-ab38-df928d8866c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:42.727312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f02f11d363764208bb0c0a7be5a7e705'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.733798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=16,keystone:SELECT=83,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:42.762463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f02f11d363764208bb0c0a7be5a7e705'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.766206 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2596/5181] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Oct 15 21:10:42 2022] PUT /identity/v3/groups/5f2f341b06584412b37b181285bb7608/users/4d18588ffdd8497ab8903f4a5a81fe87 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:42.772635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc491-9fad-4616-b5e8-5714b8603ced None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:42.773741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc491-9fad-4616-b5e8-5714b8603ced None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:42.776283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f02f11d363764208bb0c0a7be5a7e705'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.777150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-58d15992-0987-41d1-8763-b61c64e9457a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f02f11d363764208bb0c0a7be5a7e705'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:42.777599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc491-9fad-4616-b5e8-5714b8603ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:42.796836 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2586/5182] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:42 2022] DELETE /identity/v3/users/f02f11d363764208bb0c0a7be5a7e705 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:42.800671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072f560-4c85-4d0b-a091-010c915630a4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:42.801249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072f560-4c85-4d0b-a091-010c915630a4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:42.801772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072f560-4c85-4d0b-a091-010c915630a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:42.834929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5072f560-4c85-4d0b-a091-010c915630a4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:42.837189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5072f560-4c85-4d0b-a091-010c915630a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:42.841856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8a9cc491-9fad-4616-b5e8-5714b8603ced None None] MFA Rules not processed for user `4d18588ffdd8497ab8903f4a5a81fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:42.843546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a9cc491-9fad-4616-b5e8-5714b8603ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:42.863149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:42.992501 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2587/5183] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:43.005521 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2597/5184] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3265 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:43.010839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072f560-4c85-4d0b-a091-010c915630a4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:43.010987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc491-9fad-4616-b5e8-5714b8603ced None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:43.012502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a9cc491-9fad-4616-b5e8-5714b8603ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:43.031806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5072f560-4c85-4d0b-a091-010c915630a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:43.239350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc491-9fad-4616-b5e8-5714b8603ced None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:43.258775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38033075-934c-4502-8711-350398c9a180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:43.259465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38033075-934c-4502-8711-350398c9a180 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:43.259765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38033075-934c-4502-8711-350398c9a180 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:43.260039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38033075-934c-4502-8711-350398c9a180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:43.262139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072f560-4c85-4d0b-a091-010c915630a4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:43.267920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbb827-fb38-4299-bf6b-a405439fa45a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:43.268588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbb827-fb38-4299-bf6b-a405439fa45a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:43.269022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbb827-fb38-4299-bf6b-a405439fa45a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:43.269331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbb827-fb38-4299-bf6b-a405439fa45a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:43.274014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbb827-fb38-4299-bf6b-a405439fa45a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:check_grant(project_id=2335de37d6144200bbdef5af1a5cbab2, group_id=5f2f341b06584412b37b181285bb7608, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:43.275049 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-15fbb827-fb38-4299-bf6b-a405439fa45a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:43.275049 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:43.275049 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:43.276961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbb827-fb38-4299-bf6b-a405439fa45a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:43.284162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38033075-934c-4502-8711-350398c9a180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:43.284949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38033075-934c-4502-8711-350398c9a180 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:43.296137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2588/5185] 10.222.0.16 () {58 vars in 1468 bytes} [Sat Oct 15 21:10:43 2022] HEAD /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:43.300045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbb827-fb38-4299-bf6b-a405439fa45a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:43.310993 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2598/5186] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:43.315513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbb827-fb38-4299-bf6b-a405439fa45a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:43.330043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:43.331337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:43.331620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:43.331894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:43.339806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=2335de37d6144200bbdef5af1a5cbab2, group_id=5f2f341b06584412b37b181285bb7608, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:43.344865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:43.351738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role fd1a838a0c6742c4a46797eb352eded1 was removed from group 5f2f341b06584412b37b181285bb7608 on project 2335de37d6144200bbdef5af1a5cbab2. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:43.352157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:43.352971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:43.378916 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2589/5187] 10.222.0.16 () {58 vars in 1470 bytes} [Sat Oct 15 21:10:43 2022] DELETE /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:43.380498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38033075-934c-4502-8711-350398c9a180 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:43.392509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38033075-934c-4502-8711-350398c9a180 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:43.477100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb52bf6-7968-48b2-b0d4-95039c71b2db tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:43.478150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb52bf6-7968-48b2-b0d4-95039c71b2db tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:43.478624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb52bf6-7968-48b2-b0d4-95039c71b2db tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:43.479091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb52bf6-7968-48b2-b0d4-95039c71b2db tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/groups/5f2f341b06584412b37b181285bb7608/users/4d18588ffdd8497ab8903f4a5a81fe87` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:43.481359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb52bf6-7968-48b2-b0d4-95039c71b2db tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=5f2f341b06584412b37b181285bb7608, user_id=4d18588ffdd8497ab8903f4a5a81fe87)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:43.483586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb52bf6-7968-48b2-b0d4-95039c71b2db tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:43.498145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-edb52bf6-7968-48b2-b0d4-95039c71b2db tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '4d18588ffdd8497ab8903f4a5a81fe87'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:43.518730 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2599/5188] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Oct 15 21:10:43 2022] DELETE /identity/v3/groups/5f2f341b06584412b37b181285bb7608/users/4d18588ffdd8497ab8903f4a5a81fe87 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:43.525606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:43.527818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:43.548756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8d8a1-26fa-4906-a97a-dc43cc1504f1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:43.649257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca69934-8287-47a3-8c21-255964b124e3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:43.649943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca69934-8287-47a3-8c21-255964b124e3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:43.650225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca69934-8287-47a3-8c21-255964b124e3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:43.650998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca69934-8287-47a3-8c21-255964b124e3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/system/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:43.653182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca69934-8287-47a3-8c21-255964b124e3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_system_grant_for_group(group_id=5f2f341b06584412b37b181285bb7608, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:43.654330 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-cca69934-8287-47a3-8c21-255964b124e3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:43.654330 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:43.654330 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:43.655197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca69934-8287-47a3-8c21-255964b124e3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:43.661412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:43.669746 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2590/5189] 10.222.0.16 () {60 vars in 1381 bytes} [Sat Oct 15 21:10:43 2022] PUT /identity/v3/system/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:43.673700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb52bf6-7968-48b2-b0d4-95039c71b2db tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:43.699215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb52bf6-7968-48b2-b0d4-95039c71b2db tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:43.706747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95a4a6-b2ff-4d25-a08e-e55efaa5c176 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:43.707788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95a4a6-b2ff-4d25-a08e-e55efaa5c176 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:43.708810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95a4a6-b2ff-4d25-a08e-e55efaa5c176 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:43.709297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95a4a6-b2ff-4d25-a08e-e55efaa5c176 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/system/groups/5f2f341b06584412b37b181285bb7608/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:43.711094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95a4a6-b2ff-4d25-a08e-e55efaa5c176 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_system_grants_for_group(group_id=5f2f341b06584412b37b181285bb7608)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:43.712351 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-1d95a4a6-b2ff-4d25-a08e-e55efaa5c176 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:43.712351 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:43.712351 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:43.713896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95a4a6-b2ff-4d25-a08e-e55efaa5c176 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:43.721736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:43.733117 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2600/5190] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:10:43 2022] GET /identity/v3/system/groups/5f2f341b06584412b37b181285bb7608/roles => generated 376 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:43.737020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca69934-8287-47a3-8c21-255964b124e3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:43.749796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca69934-8287-47a3-8c21-255964b124e3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:43.756619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159e6ff-b2cb-412b-8a60-2aa111651945 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:43.759039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159e6ff-b2cb-412b-8a60-2aa111651945 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:43.759528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159e6ff-b2cb-412b-8a60-2aa111651945 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:43.759957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159e6ff-b2cb-412b-8a60-2aa111651945 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/system/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:43.762041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159e6ff-b2cb-412b-8a60-2aa111651945 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:check_system_grant_for_group(group_id=5f2f341b06584412b37b181285bb7608, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:43.764051 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-e159e6ff-b2cb-412b-8a60-2aa111651945 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:43.764051 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:43.764051 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:43.765373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159e6ff-b2cb-412b-8a60-2aa111651945 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:43.774407 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2591/5191] 10.222.0.16 () {58 vars in 1363 bytes} [Sat Oct 15 21:10:43 2022] HEAD /identity/v3/system/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:43.778637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95a4a6-b2ff-4d25-a08e-e55efaa5c176 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:43.797551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95a4a6-b2ff-4d25-a08e-e55efaa5c176 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:43.807292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd53ed4-ced9-448e-a4b9-46a949a9cb7b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:43.808618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd53ed4-ced9-448e-a4b9-46a949a9cb7b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:43.808618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd53ed4-ced9-448e-a4b9-46a949a9cb7b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:43.808934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd53ed4-ced9-448e-a4b9-46a949a9cb7b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/system/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:43.811099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd53ed4-ced9-448e-a4b9-46a949a9cb7b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_group(group_id=5f2f341b06584412b37b181285bb7608, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:43.812324 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-5fd53ed4-ced9-448e-a4b9-46a949a9cb7b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:43.812324 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:43.812324 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:43.814436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd53ed4-ced9-448e-a4b9-46a949a9cb7b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:43.830718 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2601/5192] 10.222.0.16 () {58 vars in 1365 bytes} [Sat Oct 15 21:10:43 2022] DELETE /identity/v3/system/groups/5f2f341b06584412b37b181285bb7608/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:43.838414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159e6ff-b2cb-412b-8a60-2aa111651945 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:43.862488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159e6ff-b2cb-412b-8a60-2aa111651945 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:43.873945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404a9ef-d395-4a0e-80ac-f29165b6636d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:43.874800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404a9ef-d395-4a0e-80ac-f29165b6636d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:43.875159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404a9ef-d395-4a0e-80ac-f29165b6636d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:43.875612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404a9ef-d395-4a0e-80ac-f29165b6636d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/46b5964105bc4469ab876d72a935c181/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:43.880218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404a9ef-d395-4a0e-80ac-f29165b6636d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=46b5964105bc4469ab876d72a935c181, user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:43.885347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404a9ef-d395-4a0e-80ac-f29165b6636d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:43.936700 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2592/5193] 10.222.0.16 () {60 vars in 1480 bytes} [Sat Oct 15 21:10:43 2022] PUT /identity/v3/domains/46b5964105bc4469ab876d72a935c181/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:43.936836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd53ed4-ced9-448e-a4b9-46a949a9cb7b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:43.953846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd53ed4-ced9-448e-a4b9-46a949a9cb7b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.048192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c151c-31e6-4898-87a1-4d50f4b647b1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.049998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c151c-31e6-4898-87a1-4d50f4b647b1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.050472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c151c-31e6-4898-87a1-4d50f4b647b1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.051065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c151c-31e6-4898-87a1-4d50f4b647b1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/46b5964105bc4469ab876d72a935c181/users/4d18588ffdd8497ab8903f4a5a81fe87/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.055831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c151c-31e6-4898-87a1-4d50f4b647b1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=46b5964105bc4469ab876d72a935c181, user_id=4d18588ffdd8497ab8903f4a5a81fe87)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.059243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c151c-31e6-4898-87a1-4d50f4b647b1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.083668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404a9ef-d395-4a0e-80ac-f29165b6636d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:44.096749 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2602/5194] 10.222.0.16 () {58 vars in 1362 bytes} [Sat Oct 15 21:10:43 2022] GET /identity/v3/domains/46b5964105bc4469ab876d72a935c181/users/4d18588ffdd8497ab8903f4a5a81fe87/roles => generated 409 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:44.101067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404a9ef-d395-4a0e-80ac-f29165b6636d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.108110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89923834-771c-494e-b9e3-bc7ca596b7ee tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.108946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923834-771c-494e-b9e3-bc7ca596b7ee tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.109821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923834-771c-494e-b9e3-bc7ca596b7ee tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.109821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923834-771c-494e-b9e3-bc7ca596b7ee tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/46b5964105bc4469ab876d72a935c181/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.132222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89923834-771c-494e-b9e3-bc7ca596b7ee tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=46b5964105bc4469ab876d72a935c181, user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.138584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89923834-771c-494e-b9e3-bc7ca596b7ee tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.158494 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2593/5195] 10.222.0.16 () {58 vars in 1462 bytes} [Sat Oct 15 21:10:44 2022] HEAD /identity/v3/domains/46b5964105bc4469ab876d72a935c181/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:44.162737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c151c-31e6-4898-87a1-4d50f4b647b1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:44.176469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c151c-31e6-4898-87a1-4d50f4b647b1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.182840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.183513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.183794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.184216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/46b5964105bc4469ab876d72a935c181/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.187534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=46b5964105bc4469ab876d72a935c181, user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.208888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.219356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role fd1a838a0c6742c4a46797eb352eded1 was removed from user 4d18588ffdd8497ab8903f4a5a81fe87 on domain 46b5964105bc4469ab876d72a935c181. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:44.219875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:44.220694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:44.250310 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2603/5196] 10.222.0.16 () {58 vars in 1464 bytes} [Sat Oct 15 21:10:44 2022] DELETE /identity/v3/domains/46b5964105bc4469ab876d72a935c181/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:44.251806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89923834-771c-494e-b9e3-bc7ca596b7ee tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:44.255042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-89923834-771c-494e-b9e3-bc7ca596b7ee tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:44.280381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89923834-771c-494e-b9e3-bc7ca596b7ee tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.362320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f6def-5992-4c52-b45c-4b88ec673888 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.363114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f6def-5992-4c52-b45c-4b88ec673888 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.363465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f6def-5992-4c52-b45c-4b88ec673888 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.364232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f6def-5992-4c52-b45c-4b88ec673888 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.368056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f6def-5992-4c52-b45c-4b88ec673888 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_grant(project_id=2335de37d6144200bbdef5af1a5cbab2, user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.371726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f6def-5992-4c52-b45c-4b88ec673888 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.396542 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2594/5197] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:44 2022] PUT /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:44.403326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:44.417920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb2973-c31b-4ad4-8460-878f775eea95 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.498533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc6c27-a553-4a08-93d4-3a05efa02e1e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.499204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc6c27-a553-4a08-93d4-3a05efa02e1e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.499546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc6c27-a553-4a08-93d4-3a05efa02e1e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.499826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc6c27-a553-4a08-93d4-3a05efa02e1e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.501897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc6c27-a553-4a08-93d4-3a05efa02e1e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_grants(project_id=2335de37d6144200bbdef5af1a5cbab2, user_id=4d18588ffdd8497ab8903f4a5a81fe87)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.506699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc6c27-a553-4a08-93d4-3a05efa02e1e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.535536 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2604/5198] 10.222.0.16 () {58 vars in 1365 bytes} [Sat Oct 15 21:10:44 2022] GET /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles => generated 410 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:44.540935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f6def-5992-4c52-b45c-4b88ec673888 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:44.562242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f6def-5992-4c52-b45c-4b88ec673888 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.568612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1df0e2-e04c-4285-853c-c7c55867c4ad tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.570264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1df0e2-e04c-4285-853c-c7c55867c4ad tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.570802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1df0e2-e04c-4285-853c-c7c55867c4ad tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.571217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1df0e2-e04c-4285-853c-c7c55867c4ad tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.573565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1df0e2-e04c-4285-853c-c7c55867c4ad tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:check_grant(project_id=2335de37d6144200bbdef5af1a5cbab2, user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.577259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1df0e2-e04c-4285-853c-c7c55867c4ad tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.587605 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2595/5199] 10.222.0.16 () {58 vars in 1465 bytes} [Sat Oct 15 21:10:44 2022] HEAD /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:44.591600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc6c27-a553-4a08-93d4-3a05efa02e1e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:44.605951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc6c27-a553-4a08-93d4-3a05efa02e1e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.612745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.614012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.614012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.614429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.617168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=2335de37d6144200bbdef5af1a5cbab2, user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.621794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.645131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role fd1a838a0c6742c4a46797eb352eded1 was removed from user 4d18588ffdd8497ab8903f4a5a81fe87 on project 2335de37d6144200bbdef5af1a5cbab2. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:44.645421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:44.646497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:44.676102 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2605/5200] 10.222.0.16 () {58 vars in 1467 bytes} [Sat Oct 15 21:10:44 2022] DELETE /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:44.678169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1df0e2-e04c-4285-853c-c7c55867c4ad tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:44.681479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5b1df0e2-e04c-4285-853c-c7c55867c4ad tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:44.699614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1df0e2-e04c-4285-853c-c7c55867c4ad tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.794258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2539298-43b4-4e49-971d-5f6f3fd04dc8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.795149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2539298-43b4-4e49-971d-5f6f3fd04dc8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.795467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2539298-43b4-4e49-971d-5f6f3fd04dc8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.795789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2539298-43b4-4e49-971d-5f6f3fd04dc8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/system/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.797938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2539298-43b4-4e49-971d-5f6f3fd04dc8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.799192 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-b2539298-43b4-4e49-971d-5f6f3fd04dc8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:44.799192 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:44.799192 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:44.800911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2539298-43b4-4e49-971d-5f6f3fd04dc8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.813811 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2596/5201] 10.222.0.16 () {60 vars in 1378 bytes} [Sat Oct 15 21:10:44 2022] PUT /identity/v3/system/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:44.815059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:44.829960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446924f-9b89-47cf-a45f-0c1cd468d3a1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.836486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295e747-bc5f-4f43-a62b-ac96bbc649dc tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.837192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295e747-bc5f-4f43-a62b-ac96bbc649dc tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.837519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295e747-bc5f-4f43-a62b-ac96bbc649dc tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.837854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295e747-bc5f-4f43-a62b-ac96bbc649dc tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/system/users/4d18588ffdd8497ab8903f4a5a81fe87/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.839258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295e747-bc5f-4f43-a62b-ac96bbc649dc tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_system_grants_for_user(user_id=4d18588ffdd8497ab8903f4a5a81fe87)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.841409 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-1295e747-bc5f-4f43-a62b-ac96bbc649dc tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:44.841409 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:44.841409 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:44.842516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295e747-bc5f-4f43-a62b-ac96bbc649dc tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.875626 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2606/5202] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 21:10:44 2022] GET /identity/v3/system/users/4d18588ffdd8497ab8903f4a5a81fe87/roles => generated 375 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:10:44.879618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2539298-43b4-4e49-971d-5f6f3fd04dc8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:44.893396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2539298-43b4-4e49-971d-5f6f3fd04dc8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.901560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34c03c-e9c7-497d-b257-a91af9b13c80 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.904024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34c03c-e9c7-497d-b257-a91af9b13c80 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.904922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34c03c-e9c7-497d-b257-a91af9b13c80 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.906059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34c03c-e9c7-497d-b257-a91af9b13c80 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/system/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.908424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34c03c-e9c7-497d-b257-a91af9b13c80 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:check_system_grant_for_user(user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.910045 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-be34c03c-e9c7-497d-b257-a91af9b13c80 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:44.910045 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:44.910045 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:44.911746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34c03c-e9c7-497d-b257-a91af9b13c80 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.920376 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2597/5203] 10.222.0.16 () {58 vars in 1360 bytes} [Sat Oct 15 21:10:44 2022] HEAD /identity/v3/system/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:44.924632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295e747-bc5f-4f43-a62b-ac96bbc649dc tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:44.940445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295e747-bc5f-4f43-a62b-ac96bbc649dc tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:44.949728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0aaf0-7281-482c-9887-49b70cc1992e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:44.950864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0aaf0-7281-482c-9887-49b70cc1992e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:44.951345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0aaf0-7281-482c-9887-49b70cc1992e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:44.952044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0aaf0-7281-482c-9887-49b70cc1992e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/system/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:44.954855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0aaf0-7281-482c-9887-49b70cc1992e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_user(user_id=4d18588ffdd8497ab8903f4a5a81fe87, role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:44.956610 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-e8a0aaf0-7281-482c-9887-49b70cc1992e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:44.956610 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:44.956610 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:44.958554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0aaf0-7281-482c-9887-49b70cc1992e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:44.999672 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2607/5204] 10.222.0.16 () {58 vars in 1362 bytes} [Sat Oct 15 21:10:44 2022] DELETE /identity/v3/system/users/4d18588ffdd8497ab8903f4a5a81fe87/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:45.004462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34c03c-e9c7-497d-b257-a91af9b13c80 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:45.024480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34c03c-e9c7-497d-b257-a91af9b13c80 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:45.035516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd8273-ba91-4cd3-a5d5-28c2c70d9e46 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:45.036201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd8273-ba91-4cd3-a5d5-28c2c70d9e46 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:45.036800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd8273-ba91-4cd3-a5d5-28c2c70d9e46 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:45.037135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd8273-ba91-4cd3-a5d5-28c2c70d9e46 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:45.037786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd8273-ba91-4cd3-a5d5-28c2c70d9e46 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:45.040245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd8273-ba91-4cd3-a5d5-28c2c70d9e46 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:45.069775 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2598/5205] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:45 2022] POST /identity/v3/roles => generated 270 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:45.073993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0aaf0-7281-482c-9887-49b70cc1992e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:45.096310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0aaf0-7281-482c-9887-49b70cc1992e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:45.113931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43be092-c5d0-4258-a6f2-8427de86ceeb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:45.116933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43be092-c5d0-4258-a6f2-8427de86ceeb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:45.126792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43be092-c5d0-4258-a6f2-8427de86ceeb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:45.127268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43be092-c5d0-4258-a6f2-8427de86ceeb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:45.132912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43be092-c5d0-4258-a6f2-8427de86ceeb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:45.136564 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-f43be092-c5d0-4258-a6f2-8427de86ceeb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:45.136564 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:45.136564 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:45.138450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43be092-c5d0-4258-a6f2-8427de86ceeb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:45.171825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:45.183142 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2608/5206] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:45 2022] POST /identity/v3/roles => generated 269 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:45.187041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd8273-ba91-4cd3-a5d5-28c2c70d9e46 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:45.198002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd8273-ba91-4cd3-a5d5-28c2c70d9e46 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:45.206515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7e916-6569-4b8a-b9cf-aef86b33cb55 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:45.208905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7e916-6569-4b8a-b9cf-aef86b33cb55 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:45.212696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7e916-6569-4b8a-b9cf-aef86b33cb55 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:45.213181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7e916-6569-4b8a-b9cf-aef86b33cb55 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/219b3a5b5982444984ad59c4a11bcf7e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:45.215339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7e916-6569-4b8a-b9cf-aef86b33cb55 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=0b56b3f4bc834d48a728cc2cc4526aa5, implied_role_id=219b3a5b5982444984ad59c4a11bcf7e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:45.217090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7e916-6569-4b8a-b9cf-aef86b33cb55 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:45.288958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43be092-c5d0-4258-a6f2-8427de86ceeb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:45.297132 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2599/5207] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:45 2022] PUT /identity/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/219b3a5b5982444984ad59c4a11bcf7e => generated 530 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:45.306850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43be092-c5d0-4258-a6f2-8427de86ceeb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:45.402784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51f785-c083-4f57-8606-7374602c36c1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:45.403878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51f785-c083-4f57-8606-7374602c36c1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:45.404371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51f785-c083-4f57-8606-7374602c36c1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:45.404855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51f785-c083-4f57-8606-7374602c36c1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:45.405823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51f785-c083-4f57-8606-7374602c36c1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:45.408704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51f785-c083-4f57-8606-7374602c36c1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:45.454992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7e916-6569-4b8a-b9cf-aef86b33cb55 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:45.466265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:45.468314 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2609/5208] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:10:45 2022] POST /identity/v3/domains => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:45.470091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51f785-c083-4f57-8606-7374602c36c1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:45.476471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4e51f785-c083-4f57-8606-7374602c36c1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:45.478187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7e916-6569-4b8a-b9cf-aef86b33cb55 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:45.689528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5de35a-1794-4f01-a5da-6b304ad78563 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:45.701676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5de35a-1794-4f01-a5da-6b304ad78563 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:45.709313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5de35a-1794-4f01-a5da-6b304ad78563 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:45.709616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5de35a-1794-4f01-a5da-6b304ad78563 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:45.710924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5de35a-1794-4f01-a5da-6b304ad78563 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:45.712672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5de35a-1794-4f01-a5da-6b304ad78563 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:45.747540 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2600/5209] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:45 2022] POST /identity/v3/roles => generated 269 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:45.748576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5de35a-1794-4f01-a5da-6b304ad78563 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:45.768103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5de35a-1794-4f01-a5da-6b304ad78563 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:45.772022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51f785-c083-4f57-8606-7374602c36c1 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:45.778646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e09040-c713-4584-b997-8437ca751e57 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:45.779460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e09040-c713-4584-b997-8437ca751e57 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:45.779841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e09040-c713-4584-b997-8437ca751e57 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:45.780206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e09040-c713-4584-b997-8437ca751e57 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/8c3263b983674287bfca4f817a2413e3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:45.786013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e09040-c713-4584-b997-8437ca751e57 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=0b56b3f4bc834d48a728cc2cc4526aa5, implied_role_id=8c3263b983674287bfca4f817a2413e3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:45.788149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e09040-c713-4584-b997-8437ca751e57 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:45.809925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:45.812474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:45.813138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:45.813862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] PATH_INFO: `/v3/projects/9caf8063f9b74cf887f85e59923e7522` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:45.819253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9caf8063f9b74cf887f85e59923e7522)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:45.829438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:45.844445 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2601/5210] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:45 2022] PUT /identity/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/8c3263b983674287bfca4f817a2413e3 => generated 530 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:10:45.845611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e09040-c713-4584-b997-8437ca751e57 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:45.985359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9caf8063f9b74cf887f85e59923e7522', 'request_id': 'req-f4435b6f-e8c8-4b80-8046-d7de227ab398'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:45.998176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9caf8063f9b74cf887f85e59923e7522', 'request_id': 'req-f4435b6f-e8c8-4b80-8046-d7de227ab398'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:46.011384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9caf8063f9b74cf887f85e59923e7522', 'request_id': 'req-f4435b6f-e8c8-4b80-8046-d7de227ab398'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:46.033063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e09040-c713-4584-b997-8437ca751e57 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:46.037409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] The token cache is being invalidate because project 9caf8063f9b74cf887f85e59923e7522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:46.037570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:46.039941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:46.042687 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2610/5211] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:45 2022] DELETE /identity/v3/projects/9caf8063f9b74cf887f85e59923e7522 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:46.046727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:46.050808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:46.096922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:46.146035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4435b6f-e8c8-4b80-8046-d7de227ab398 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:46.153730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:46.155005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:46.155515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:46.155996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] PATH_INFO: `/v3/users/af7551dad082418f9cbfba3107a743cf` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:46.162230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=af7551dad082418f9cbfba3107a743cf)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:46.165149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:46.175510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a134b-a3dc-496a-b7ec-e6ed05caa24b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:46.176173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a134b-a3dc-496a-b7ec-e6ed05caa24b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:46.176685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a134b-a3dc-496a-b7ec-e6ed05caa24b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:46.176685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a134b-a3dc-496a-b7ec-e6ed05caa24b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:46.184241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a134b-a3dc-496a-b7ec-e6ed05caa24b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=0b56b3f4bc834d48a728cc2cc4526aa5, implied_role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:46.185891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a134b-a3dc-496a-b7ec-e6ed05caa24b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:46.251957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:46.252433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2602/5212] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:45 2022] PUT /identity/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/fd1a838a0c6742c4a46797eb352eded1 => generated 526 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:10:46.253926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af7551dad082418f9cbfba3107a743cf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:46.256177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a134b-a3dc-496a-b7ec-e6ed05caa24b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:46.259152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-677a134b-a3dc-496a-b7ec-e6ed05caa24b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:46.281114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af7551dad082418f9cbfba3107a743cf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:46.295479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af7551dad082418f9cbfba3107a743cf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:46.310060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af7551dad082418f9cbfba3107a743cf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:46.311358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af7551dad082418f9cbfba3107a743cf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:46.318255 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2611/5213] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:46 2022] DELETE /identity/v3/users/af7551dad082418f9cbfba3107a743cf => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:46.410348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a134b-a3dc-496a-b7ec-e6ed05caa24b tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:46.423620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:46.425399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:46.546890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fe7a6-5a91-4673-af58-908fdcbce918 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:46.548544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fe7a6-5a91-4673-af58-908fdcbce918 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:46.548828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fe7a6-5a91-4673-af58-908fdcbce918 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:46.549128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fe7a6-5a91-4673-af58-908fdcbce918 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/fd1a838a0c6742c4a46797eb352eded1/implies/0b56b3f4bc834d48a728cc2cc4526aa5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:46.552830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fe7a6-5a91-4673-af58-908fdcbce918 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=fd1a838a0c6742c4a46797eb352eded1, implied_role_id=0b56b3f4bc834d48a728cc2cc4526aa5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:46.555233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fe7a6-5a91-4673-af58-908fdcbce918 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:46.598735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e8298-e76b-444b-b609-777bf8ec17ad admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-b38fe7a6-5a91-4673-af58-908fdcbce918 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] 0b56b3f4bc834d48a728cc2cc4526aa5 cannot be an implied roles.: keystone.exception.InvalidImpliedRole: 0b56b3f4bc834d48a728cc2cc4526aa5 cannot be an implied roles. Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 259, in put Oct 15 21:10:46.606597 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application PROVIDERS.role_api.create_implied_role(prior_role_id, implied_role_id) Oct 15 21:10:46.607749 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:46.607749 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:46.607749 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1361, in create_implied_role Oct 15 21:10:46.607749 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.InvalidImpliedRole(msg, Oct 15 21:10:46.607749 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.InvalidImpliedRole: 0b56b3f4bc834d48a728cc2cc4526aa5 cannot be an implied roles. Oct 15 21:10:46.607749 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:10:46.608634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a01365-5afc-464b-a23d-204c2166b054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:46.609333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01365-5afc-464b-a23d-204c2166b054 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:46.609657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01365-5afc-464b-a23d-204c2166b054 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:46.609976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01365-5afc-464b-a23d-204c2166b054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:46.610979 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2603/5214] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:46 2022] PUT /identity/v3/roles/fd1a838a0c6742c4a46797eb352eded1/implies/0b56b3f4bc834d48a728cc2cc4526aa5 => generated 116 bytes in 355 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Oct 15 21:10:46.615980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-21a01365-5afc-464b-a23d-204c2166b054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:46.619807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fe7a6-5a91-4673-af58-908fdcbce918 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:46.621562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b38fe7a6-5a91-4673-af58-908fdcbce918 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:46.646130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fe7a6-5a91-4673-af58-908fdcbce918 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:46.658714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb499ba2-234d-48eb-9ec0-59c6d4a3fda3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:46.659400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb499ba2-234d-48eb-9ec0-59c6d4a3fda3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:46.660098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb499ba2-234d-48eb-9ec0-59c6d4a3fda3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:46.660098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb499ba2-234d-48eb-9ec0-59c6d4a3fda3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:46.667481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb499ba2-234d-48eb-9ec0-59c6d4a3fda3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=0b56b3f4bc834d48a728cc2cc4526aa5, implied_role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:46.669245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb499ba2-234d-48eb-9ec0-59c6d4a3fda3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:46.694549 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2604/5215] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:46 2022] DELETE /identity/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:46.699608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb499ba2-234d-48eb-9ec0-59c6d4a3fda3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:46.800102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:46.810162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a01365-5afc-464b-a23d-204c2166b054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:46.811880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a01365-5afc-464b-a23d-204c2166b054 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:46.833881 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2612/5216] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:46 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:10:46.846817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb499ba2-234d-48eb-9ec0-59c6d4a3fda3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:46.869912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83751f-1778-4485-8758-07c2000bab76 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:46.870684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83751f-1778-4485-8758-07c2000bab76 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:46.871541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83751f-1778-4485-8758-07c2000bab76 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:46.873353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83751f-1778-4485-8758-07c2000bab76 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/8c3263b983674287bfca4f817a2413e3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:46.879418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83751f-1778-4485-8758-07c2000bab76 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=0b56b3f4bc834d48a728cc2cc4526aa5, implied_role_id=8c3263b983674287bfca4f817a2413e3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:46.885592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83751f-1778-4485-8758-07c2000bab76 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:46.907254 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2605/5217] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:46 2022] DELETE /identity/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/8c3263b983674287bfca4f817a2413e3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:46.911822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a01365-5afc-464b-a23d-204c2166b054 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:46.972333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a01365-5afc-464b-a23d-204c2166b054 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:47.135168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:47.136995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:47.141183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:47.141467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/8c3263b983674287bfca4f817a2413e3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:47.145813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=8c3263b983674287bfca4f817a2413e3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:47.148136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:47.265067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role 8c3263b983674287bfca4f817a2413e3 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:47.265762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:47.266278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:47.275011 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2613/5218] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:46 2022] DELETE /identity/v3/roles/8c3263b983674287bfca4f817a2413e3 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:47.278920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83751f-1778-4485-8758-07c2000bab76 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:47.284135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8a83751f-1778-4485-8758-07c2000bab76 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:47.305864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83751f-1778-4485-8758-07c2000bab76 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:47.380774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:47.489276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:47.489973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:47.490258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:47.490533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/4e83b07dc2f64291a9067b3b0a0d5124` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:47.522402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=4e83b07dc2f64291a9067b3b0a0d5124)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:47.528685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:47.582832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:47.586414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:47.648761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:47.650272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:47.656156 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2606/5219] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:10:47 2022] PATCH /identity/v3/domains/4e83b07dc2f64291a9067b3b0a0d5124 => generated 276 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:47.660197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:47.674365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82114a-af5a-429d-8296-4abc0ffbe777 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:47.773747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:47.774801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:47.775540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:47.776173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/4e83b07dc2f64291a9067b3b0a0d5124` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:47.795125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=4e83b07dc2f64291a9067b3b0a0d5124)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:47.802325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:47.914721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:47.926232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.004205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:48.040914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124', 'request_id': 'req-5a0080b1-2164-4995-a91f-72885c6474a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.060665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124', 'request_id': 'req-5a0080b1-2164-4995-a91f-72885c6474a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.068875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124', 'request_id': 'req-5a0080b1-2164-4995-a91f-72885c6474a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.106552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] The token cache is being invalidate because project 4e83b07dc2f64291a9067b3b0a0d5124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:48.106861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.107674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.134398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124', 'request_id': 'req-5a0080b1-2164-4995-a91f-72885c6474a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.136085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124', 'request_id': 'req-5a0080b1-2164-4995-a91f-72885c6474a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.137802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '4e83b07dc2f64291a9067b3b0a0d5124', 'request_id': 'req-5a0080b1-2164-4995-a91f-72885c6474a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.158104 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2614/5220] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:10:47 2022] DELETE /identity/v3/domains/4e83b07dc2f64291a9067b3b0a0d5124 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:48.159662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:48.161404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:48.187121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82961bea-dbf1-4d88-b722-0eb5b191a8c7 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:48.196750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:48.198593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:48.375865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0080b1-2164-4995-a91f-72885c6474a9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:48.390808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:48.391776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:48.392387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:48.393010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] PATH_INFO: `/v3/projects/d8380fe6b3fe41bd8a65b8ff0fbf5928` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:48.395343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8380fe6b3fe41bd8a65b8ff0fbf5928)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:48.398916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:48.416616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e088c6b-830c-4505-a2b9-14c72faada85 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:48.417419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e088c6b-830c-4505-a2b9-14c72faada85 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:48.418029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e088c6b-830c-4505-a2b9-14c72faada85 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:48.418315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e088c6b-830c-4505-a2b9-14c72faada85 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/219b3a5b5982444984ad59c4a11bcf7e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:48.423693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e088c6b-830c-4505-a2b9-14c72faada85 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=0b56b3f4bc834d48a728cc2cc4526aa5, implied_role_id=219b3a5b5982444984ad59c4a11bcf7e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:48.425881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e088c6b-830c-4505-a2b9-14c72faada85 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:48.438364 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2607/5221] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:48 2022] DELETE /identity/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5/implies/219b3a5b5982444984ad59c4a11bcf7e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:48.442293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e088c6b-830c-4505-a2b9-14c72faada85 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:48.444166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3e088c6b-830c-4505-a2b9-14c72faada85 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:48.518566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'request_id': 'req-949e463a-164a-4ec5-b254-284f11c0eb90'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.534393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'request_id': 'req-949e463a-164a-4ec5-b254-284f11c0eb90'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.548817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8380fe6b3fe41bd8a65b8ff0fbf5928', 'request_id': 'req-949e463a-164a-4ec5-b254-284f11c0eb90'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.586653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] The token cache is being invalidate because project d8380fe6b3fe41bd8a65b8ff0fbf5928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:48.589544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.590519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-949e463a-164a-4ec5-b254-284f11c0eb90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.593187 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2615/5222] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:48 2022] DELETE /identity/v3/projects/d8380fe6b3fe41bd8a65b8ff0fbf5928 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:48.599402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12555fdc-5eea-499b-9e1f-668db3cc6efa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:48.599881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12555fdc-5eea-499b-9e1f-668db3cc6efa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:48.600307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12555fdc-5eea-499b-9e1f-668db3cc6efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:48.617837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e088c6b-830c-4505-a2b9-14c72faada85 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:48.642786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-12555fdc-5eea-499b-9e1f-668db3cc6efa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:48.645736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-12555fdc-5eea-499b-9e1f-668db3cc6efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:48.684053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:48.694065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:48.782580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:48.783693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:48.784215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:48.785812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/219b3a5b5982444984ad59c4a11bcf7e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:48.788538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=219b3a5b5982444984ad59c4a11bcf7e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:48.789955 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:48.789955 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:48.789955 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:48.791326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:48.802070 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2616/5223] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:48.808335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed9a5d-2d39-4fe9-b9be-45ff653871c2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:48.809745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed9a5d-2d39-4fe9-b9be-45ff653871c2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:48.810350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed9a5d-2d39-4fe9-b9be-45ff653871c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:48.877168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9aed9a5d-2d39-4fe9-b9be-45ff653871c2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:48.883135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9aed9a5d-2d39-4fe9-b9be-45ff653871c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:48.886940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role 219b3a5b5982444984ad59c4a11bcf7e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:48.887324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.889293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:48.897097 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2608/5224] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:48 2022] DELETE /identity/v3/roles/219b3a5b5982444984ad59c4a11bcf7e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:48.899783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:48.901506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:48.908453 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2617/5225] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:48.911238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed9a5d-2d39-4fe9-b9be-45ff653871c2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:48.949637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed9a5d-2d39-4fe9-b9be-45ff653871c2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:49.091625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e918e1-1a7c-4961-b8e7-d045f031fe27 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:49.106199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:49.108598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:49.108878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:49.109176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:49.112151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=0b56b3f4bc834d48a728cc2cc4526aa5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:49.113005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759bad5-5ecb-4c8e-b531-d1751283dfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:49.113913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759bad5-5ecb-4c8e-b531-d1751283dfb0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:49.114219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759bad5-5ecb-4c8e-b531-d1751283dfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:49.114438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759bad5-5ecb-4c8e-b531-d1751283dfb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:49.115083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759bad5-5ecb-4c8e-b531-d1751283dfb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:10:49.115378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759bad5-5ecb-4c8e-b531-d1751283dfb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:49.118480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:49.122236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759bad5-5ecb-4c8e-b531-d1751283dfb0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:49.159859 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2618/5226] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:49.161955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759bad5-5ecb-4c8e-b531-d1751283dfb0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:49.190000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759bad5-5ecb-4c8e-b531-d1751283dfb0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:49.213451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role 0b56b3f4bc834d48a728cc2cc4526aa5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:49.213677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:49.214756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35978c8-9539-49ce-a4fb-9bf6fabcd908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:49.215522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:49.216614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35978c8-9539-49ce-a4fb-9bf6fabcd908 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:49.216935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35978c8-9539-49ce-a4fb-9bf6fabcd908 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:49.218201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35978c8-9539-49ce-a4fb-9bf6fabcd908 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:49.219708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35978c8-9539-49ce-a4fb-9bf6fabcd908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:49.220728 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2609/5227] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:48 2022] DELETE /identity/v3/roles/0b56b3f4bc834d48a728cc2cc4526aa5 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:49.222459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35978c8-9539-49ce-a4fb-9bf6fabcd908 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:49.231029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:49.234938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:49.323223 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2619/5228] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:49 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:49.326746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35978c8-9539-49ce-a4fb-9bf6fabcd908 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:49.335194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c35978c8-9539-49ce-a4fb-9bf6fabcd908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:49.382420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:49.403593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d79855-2fc8-40a0-b802-f6cdacd972c3 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:49.456465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35978c8-9539-49ce-a4fb-9bf6fabcd908 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:49.465344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76148787-fce3-4c6c-b5b0-a12b563cb388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:49.466259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76148787-fce3-4c6c-b5b0-a12b563cb388 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:49.466699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76148787-fce3-4c6c-b5b0-a12b563cb388 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:49.467112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76148787-fce3-4c6c-b5b0-a12b563cb388 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:49.467892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76148787-fce3-4c6c-b5b0-a12b563cb388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:49.470457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76148787-fce3-4c6c-b5b0-a12b563cb388 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:49.515191 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2620/5229] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:49 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:49.516747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76148787-fce3-4c6c-b5b0-a12b563cb388 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:49.527864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7f987-f052-4ad6-833d-6389f3fba825 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:49.529199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7f987-f052-4ad6-833d-6389f3fba825 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:49.530410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7f987-f052-4ad6-833d-6389f3fba825 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:49.530691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7f987-f052-4ad6-833d-6389f3fba825 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:49.532792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76148787-fce3-4c6c-b5b0-a12b563cb388 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:49.539858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe77816-379b-4d49-a185-bccfe131b712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:49.540113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7f987-f052-4ad6-833d-6389f3fba825 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:49.540981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe77816-379b-4d49-a185-bccfe131b712 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:49.541464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe77816-379b-4d49-a185-bccfe131b712 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:49.541705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7f987-f052-4ad6-833d-6389f3fba825 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:49.542036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe77816-379b-4d49-a185-bccfe131b712 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:49.542831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe77816-379b-4d49-a185-bccfe131b712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:49.545667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe77816-379b-4d49-a185-bccfe131b712 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:49.567232 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2621/5230] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:49 2022] GET /identity/v3/roles => generated 2323 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:49.569178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe77816-379b-4d49-a185-bccfe131b712 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:49.600553 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2610/5231] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:49 2022] PUT /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 521 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:49.603082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe77816-379b-4d49-a185-bccfe131b712 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:49.604296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27d9dc-4ce1-4e47-838f-c290e98d8c63 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:49.604630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27d9dc-4ce1-4e47-838f-c290e98d8c63 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:49.604876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27d9dc-4ce1-4e47-838f-c290e98d8c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:49.623458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:49.690522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bb27d9dc-4ce1-4e47-838f-c290e98d8c63 None None] MFA Rules not processed for user `a7eae9dc0a1a47d9910f49bb33316d91`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:49.692042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bb27d9dc-4ce1-4e47-838f-c290e98d8c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:49.785851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a0552-6862-4462-be51-0ae5ab7a3968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:49.786464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a0552-6862-4462-be51-0ae5ab7a3968 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:49.786891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a0552-6862-4462-be51-0ae5ab7a3968 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:49.787013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a0552-6862-4462-be51-0ae5ab7a3968 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:49.787655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a0552-6862-4462-be51-0ae5ab7a3968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:49.789348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a0552-6862-4462-be51-0ae5ab7a3968 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:49.816494 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2622/5232] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:49 2022] GET /identity/v3/roles => generated 2323 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:49.823003 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2611/5233] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:49 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:49.823896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a0552-6862-4462-be51-0ae5ab7a3968 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:49.824917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27d9dc-4ce1-4e47-838f-c290e98d8c63 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:49.880791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27d9dc-4ce1-4e47-838f-c290e98d8c63 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:49.893609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558b0d6-8fe8-4bb4-96d2-24ee988450bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:49.894621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558b0d6-8fe8-4bb4-96d2-24ee988450bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:49.895053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558b0d6-8fe8-4bb4-96d2-24ee988450bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:49.895485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558b0d6-8fe8-4bb4-96d2-24ee988450bf admin admin] PATH_INFO: `/v3/projects/5360eb36b9c64153975f4a2e8c315feb/users/75aea18e943d4417a54b2eac95f0ab98/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:49.935923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558b0d6-8fe8-4bb4-96d2-24ee988450bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5360eb36b9c64153975f4a2e8c315feb, user_id=75aea18e943d4417a54b2eac95f0ab98, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:49.943737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558b0d6-8fe8-4bb4-96d2-24ee988450bf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:50.000390 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2612/5234] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:49 2022] PUT /identity/v3/projects/5360eb36b9c64153975f4a2e8c315feb/users/75aea18e943d4417a54b2eac95f0ab98/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:50.008711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd1a59-f6bc-469e-be59-5191930488d6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:50.009199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd1a59-f6bc-469e-be59-5191930488d6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:50.009704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd1a59-f6bc-469e-be59-5191930488d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:50.070221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-35cd1a59-f6bc-469e-be59-5191930488d6 None None] MFA Rules not processed for user `a7eae9dc0a1a47d9910f49bb33316d91`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:50.071537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-35cd1a59-f6bc-469e-be59-5191930488d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:50.072344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a0552-6862-4462-be51-0ae5ab7a3968 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:50.162549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:50.188440 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2613/5235] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:50 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:50.215642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfe9df-cbb3-4b84-ab51-30cce415ac55 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:50.215973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfe9df-cbb3-4b84-ab51-30cce415ac55 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:50.216255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfe9df-cbb3-4b84-ab51-30cce415ac55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:50.271697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:50.273303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:50.273764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:50.274195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:50.278711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:check_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:50.293303 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:50.293303 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:50.293303 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:50.294269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:50.295149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a4dfe9df-cbb3-4b84-ab51-30cce415ac55 None None] MFA Rules not processed for user `75aea18e943d4417a54b2eac95f0ab98`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:50.295283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:50.296164 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:50.296164 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:50.296164 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:50.296406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a4dfe9df-cbb3-4b84-ab51-30cce415ac55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:50.296992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:50.324893 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2623/5236] 10.222.0.16 () {58 vars in 1345 bytes} [Sat Oct 15 21:10:49 2022] HEAD /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:50.324893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:50.328447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:50.465735 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2614/5237] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:50.466620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfe9df-cbb3-4b84-ab51-30cce415ac55 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:50.494402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfe9df-cbb3-4b84-ab51-30cce415ac55 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:50.500120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:50.505529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956ee8a-4644-4f80-962b-7f44720d4fb9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:50.506433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956ee8a-4644-4f80-962b-7f44720d4fb9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:50.507737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956ee8a-4644-4f80-962b-7f44720d4fb9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:50.508021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956ee8a-4644-4f80-962b-7f44720d4fb9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:50.514615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956ee8a-4644-4f80-962b-7f44720d4fb9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:50.515801 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-2956ee8a-4644-4f80-962b-7f44720d4fb9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:50.515801 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:50.515801 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:50.520017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956ee8a-4644-4f80-962b-7f44720d4fb9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:50.536050 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2615/5238] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Oct 15 21:10:50 2022] GET /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 521 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:50.537255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956ee8a-4644-4f80-962b-7f44720d4fb9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:50.579384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305adc37-bd0d-4b84-9442-ac81c3469743 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:50.610855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300223e0-68b1-4bf9-b68b-d2a80cd5ad4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:50.611501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300223e0-68b1-4bf9-b68b-d2a80cd5ad4b service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:50.611797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300223e0-68b1-4bf9-b68b-d2a80cd5ad4b service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:50.612041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300223e0-68b1-4bf9-b68b-d2a80cd5ad4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:50.640039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300223e0-68b1-4bf9-b68b-d2a80cd5ad4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:50.642915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300223e0-68b1-4bf9-b68b-d2a80cd5ad4b service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:50.671314 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2624/5239] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:50 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:50.694483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300223e0-68b1-4bf9-b68b-d2a80cd5ad4b service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:50.713557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956ee8a-4644-4f80-962b-7f44720d4fb9 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:50.722249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd18ac-b965-43b9-b05f-d101b34d99fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:50.722866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd18ac-b965-43b9-b05f-d101b34d99fa admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:50.723138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd18ac-b965-43b9-b05f-d101b34d99fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:50.723401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd18ac-b965-43b9-b05f-d101b34d99fa admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:50.724024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd18ac-b965-43b9-b05f-d101b34d99fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:50.725143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300223e0-68b1-4bf9-b68b-d2a80cd5ad4b service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:50.726838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd18ac-b965-43b9-b05f-d101b34d99fa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:50.740307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac193245-d524-44e4-9dbf-86acaaac0166 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:50.742717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac193245-d524-44e4-9dbf-86acaaac0166 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:50.743006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac193245-d524-44e4-9dbf-86acaaac0166 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:50.743286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac193245-d524-44e4-9dbf-86acaaac0166 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:50.749331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac193245-d524-44e4-9dbf-86acaaac0166 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:50.751756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac193245-d524-44e4-9dbf-86acaaac0166 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:50.773085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2616/5240] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:50 2022] POST /identity/v3/projects => generated 373 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:50.775364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd18ac-b965-43b9-b05f-d101b34d99fa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:50.781546 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2625/5241] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:50 2022] DELETE /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:50.786124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac193245-d524-44e4-9dbf-86acaaac0166 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:50.917961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:50.924268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd18ac-b965-43b9-b05f-d101b34d99fa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:50.934080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac193245-d524-44e4-9dbf-86acaaac0166 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:50.945030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ae90b-e8df-4abe-a2f1-3410e4756fcd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:50.945760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ae90b-e8df-4abe-a2f1-3410e4756fcd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:50.946032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ae90b-e8df-4abe-a2f1-3410e4756fcd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:50.946303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ae90b-e8df-4abe-a2f1-3410e4756fcd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:50.948016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ae90b-e8df-4abe-a2f1-3410e4756fcd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:50.949708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ae90b-e8df-4abe-a2f1-3410e4756fcd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-f50ae90b-e8df-4abe-a2f1-3410e4756fcd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] 6e3634c6ea4d4c6eb96fe49f863b4e4e does not imply 4fb9d9a02ac04634bbc8bf4eaa9188ce.: keystone.exception.ImpliedRoleNotFound: 6e3634c6ea4d4c6eb96fe49f863b4e4e does not imply 4fb9d9a02ac04634bbc8bf4eaa9188ce. Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = query.one() Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._iter().one() Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._only_one_row( Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exc.NoResultFound( Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:50.966546 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 235, in get Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_implied_role(prior_role_id, implied_role_id) Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 239, in _get_implied_role Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.role_api.get_implied_role( Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 149, in get_implied_role Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Oct 15 21:10:50.967718 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Oct 15 21:10:50.968856 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Oct 15 21:10:50.968856 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 6e3634c6ea4d4c6eb96fe49f863b4e4e does not imply 4fb9d9a02ac04634bbc8bf4eaa9188ce. Oct 15 21:10:50.968856 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:50.977810 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2626/5242] 10.222.0.16 () {58 vars in 1344 bytes} [Sat Oct 15 21:10:50 2022] GET /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 137 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:10:50.982837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ae90b-e8df-4abe-a2f1-3410e4756fcd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:51.001394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ae90b-e8df-4abe-a2f1-3410e4756fcd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:51.008034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b33cdd-b871-41ac-9bff-cb9e8d4a6784 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:51.008844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b33cdd-b871-41ac-9bff-cb9e8d4a6784 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.009274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b33cdd-b871-41ac-9bff-cb9e8d4a6784 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.009738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b33cdd-b871-41ac-9bff-cb9e8d4a6784 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.011710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b33cdd-b871-41ac-9bff-cb9e8d4a6784 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:51.013659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b33cdd-b871-41ac-9bff-cb9e8d4a6784 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-99b33cdd-b871-41ac-9bff-cb9e8d4a6784 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] 6e3634c6ea4d4c6eb96fe49f863b4e4e does not imply 4fb9d9a02ac04634bbc8bf4eaa9188ce.: keystone.exception.ImpliedRoleNotFound: 6e3634c6ea4d4c6eb96fe49f863b4e4e does not imply 4fb9d9a02ac04634bbc8bf4eaa9188ce. Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = query.one() Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._iter().one() Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._only_one_row( Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exc.NoResultFound( Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:51.027576 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Oct 15 21:10:51.028938 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Oct 15 21:10:51.030318 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Oct 15 21:10:51.030318 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Oct 15 21:10:51.030318 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Oct 15 21:10:51.030318 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 6e3634c6ea4d4c6eb96fe49f863b4e4e does not imply 4fb9d9a02ac04634bbc8bf4eaa9188ce. Oct 15 21:10:51.030318 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:51.036491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2627/5243] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:50 2022] DELETE /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 137 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:10:51.038585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b33cdd-b871-41ac-9bff-cb9e8d4a6784 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:51.062485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef6e99-ac03-43df-ae9c-02a380310b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:51.063225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef6e99-ac03-43df-ae9c-02a380310b1e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.063605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef6e99-ac03-43df-ae9c-02a380310b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.063991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef6e99-ac03-43df-ae9c-02a380310b1e admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.064720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef6e99-ac03-43df-ae9c-02a380310b1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:51.066832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef6e99-ac03-43df-ae9c-02a380310b1e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:51.068722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b33cdd-b871-41ac-9bff-cb9e8d4a6784 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:51.080106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cd34f-db6b-4cd6-a0ec-16ff8bac83ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:51.080920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cd34f-db6b-4cd6-a0ec-16ff8bac83ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.081097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cd34f-db6b-4cd6-a0ec-16ff8bac83ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.081374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cd34f-db6b-4cd6-a0ec-16ff8bac83ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.085702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cd34f-db6b-4cd6-a0ec-16ff8bac83ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:51.086859 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-9d6cd34f-db6b-4cd6-a0ec-16ff8bac83ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:51.086859 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:51.086859 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:51.087963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cd34f-db6b-4cd6-a0ec-16ff8bac83ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:51.118353 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2617/5244] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:50 2022] POST /identity/v3/users => generated 293 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:51.119469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef6e99-ac03-43df-ae9c-02a380310b1e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:51.136892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef6e99-ac03-43df-ae9c-02a380310b1e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:51.146412 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2628/5245] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:51 2022] PUT /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 521 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:51.151315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cd34f-db6b-4cd6-a0ec-16ff8bac83ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:51.155904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89fc35-43c7-406a-97e5-577cb756b486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:51.156681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89fc35-43c7-406a-97e5-577cb756b486 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.160256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89fc35-43c7-406a-97e5-577cb756b486 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.160632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89fc35-43c7-406a-97e5-577cb756b486 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.162799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89fc35-43c7-406a-97e5-577cb756b486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:51.165476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89fc35-43c7-406a-97e5-577cb756b486 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:51.193905 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2618/5246] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:51 2022] GET /identity/v3/roles => generated 2323 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:51.201111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89fc35-43c7-406a-97e5-577cb756b486 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:51.223390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89fc35-43c7-406a-97e5-577cb756b486 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:51.339399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cd34f-db6b-4cd6-a0ec-16ff8bac83ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:51.354266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5abbd-1dbc-4ef4-8f3a-b7f2a8c41835 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:51.354849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5abbd-1dbc-4ef4-8f3a-b7f2a8c41835 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.355124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5abbd-1dbc-4ef4-8f3a-b7f2a8c41835 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.355476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5abbd-1dbc-4ef4-8f3a-b7f2a8c41835 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/e085cf0d1c31436fb78e1a41eca71b39` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.357288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5abbd-1dbc-4ef4-8f3a-b7f2a8c41835 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=e085cf0d1c31436fb78e1a41eca71b39)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:51.358909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5abbd-1dbc-4ef4-8f3a-b7f2a8c41835 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:51.370300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d40f0f-c43a-4f05-a20c-a8213c7e86f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:51.372100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d40f0f-c43a-4f05-a20c-a8213c7e86f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.373305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d40f0f-c43a-4f05-a20c-a8213c7e86f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.373692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d40f0f-c43a-4f05-a20c-a8213c7e86f0 admin admin] PATH_INFO: `/v3/projects/841626c2473f42d394e2a1f988cc4994/users/97d6b8c0746d4793b55171f48bd9b3eb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.400492 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2629/5247] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:51 2022] PUT /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/e085cf0d1c31436fb78e1a41eca71b39 => generated 520 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:51.401903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5abbd-1dbc-4ef4-8f3a-b7f2a8c41835 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:51.411956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d40f0f-c43a-4f05-a20c-a8213c7e86f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841626c2473f42d394e2a1f988cc4994, user_id=97d6b8c0746d4793b55171f48bd9b3eb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:51.418907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d40f0f-c43a-4f05-a20c-a8213c7e86f0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:51.421656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:51.455100 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2619/5248] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:51 2022] PUT /identity/v3/projects/841626c2473f42d394e2a1f988cc4994/users/97d6b8c0746d4793b55171f48bd9b3eb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:51.464819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d40f0f-c43a-4f05-a20c-a8213c7e86f0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:51.546037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5abbd-1dbc-4ef4-8f3a-b7f2a8c41835 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:51.639492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d40f0f-c43a-4f05-a20c-a8213c7e86f0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:51.650904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036da78c-8ba3-4036-bd61-60a22c4b7faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:51.651707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036da78c-8ba3-4036-bd61-60a22c4b7faa admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.652256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036da78c-8ba3-4036-bd61-60a22c4b7faa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.653722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036da78c-8ba3-4036-bd61-60a22c4b7faa admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.655735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036da78c-8ba3-4036-bd61-60a22c4b7faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:51.657001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff2f75-7c9e-43ee-9518-268d4611dec4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:51.657726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff2f75-7c9e-43ee-9518-268d4611dec4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.658009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff2f75-7c9e-43ee-9518-268d4611dec4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.658276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff2f75-7c9e-43ee-9518-268d4611dec4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/e085cf0d1c31436fb78e1a41eca71b39/implies/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.659945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff2f75-7c9e-43ee-9518-268d4611dec4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=e085cf0d1c31436fb78e1a41eca71b39, implied_role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:51.661628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff2f75-7c9e-43ee-9518-268d4611dec4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:51.663810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036da78c-8ba3-4036-bd61-60a22c4b7faa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:51.704895 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2620/5249] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:51 2022] GET /identity/v3/roles => generated 2323 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:51.709227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036da78c-8ba3-4036-bd61-60a22c4b7faa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:51.725789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:51.727832 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2630/5250] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:51 2022] PUT /identity/v3/roles/e085cf0d1c31436fb78e1a41eca71b39/implies/fd1a838a0c6742c4a46797eb352eded1 => generated 520 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:51.732749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff2f75-7c9e-43ee-9518-268d4611dec4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:51.756799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036da78c-8ba3-4036-bd61-60a22c4b7faa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:51.892994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df0159-1694-44a6-b4ea-f8c8d95d8ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:51.892994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df0159-1694-44a6-b4ea-f8c8d95d8ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.894292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df0159-1694-44a6-b4ea-f8c8d95d8ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.895432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df0159-1694-44a6-b4ea-f8c8d95d8ba0 admin admin] PATH_INFO: `/v3/projects/841626c2473f42d394e2a1f988cc4994/users/97d6b8c0746d4793b55171f48bd9b3eb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.896969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df0159-1694-44a6-b4ea-f8c8d95d8ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841626c2473f42d394e2a1f988cc4994, user_id=97d6b8c0746d4793b55171f48bd9b3eb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:51.901818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff2f75-7c9e-43ee-9518-268d4611dec4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:51.902806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df0159-1694-44a6-b4ea-f8c8d95d8ba0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:51.908932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abf96b-b3a9-4c35-8b16-8725d64177da tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:51.909958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abf96b-b3a9-4c35-8b16-8725d64177da tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.915884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abf96b-b3a9-4c35-8b16-8725d64177da tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.916454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abf96b-b3a9-4c35-8b16-8725d64177da tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/role_inferences` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.917250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abf96b-b3a9-4c35-8b16-8725d64177da tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_role_inference_rules()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:51.918631 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-53abf96b-b3a9-4c35-8b16-8725d64177da tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_inference_rules": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:51.918631 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:51.918631 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:51.920248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abf96b-b3a9-4c35-8b16-8725d64177da tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:51.925577 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2621/5251] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:10:51 2022] PUT /identity/v3/projects/841626c2473f42d394e2a1f988cc4994/users/97d6b8c0746d4793b55171f48bd9b3eb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:51.935893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a814e9-7ee8-4103-af1b-b61a09c1b5f7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:51.936353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a814e9-7ee8-4103-af1b-b61a09c1b5f7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:51.936489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a814e9-7ee8-4103-af1b-b61a09c1b5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:51.964258 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2631/5252] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 21:10:51 2022] GET /identity/v3/role_inferences => generated 1606 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:51.965795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abf96b-b3a9-4c35-8b16-8725d64177da tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:51.984553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abf96b-b3a9-4c35-8b16-8725d64177da tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:52.004127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-77a814e9-7ee8-4103-af1b-b61a09c1b5f7 None None] MFA Rules not processed for user `97d6b8c0746d4793b55171f48bd9b3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:52.005682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-77a814e9-7ee8-4103-af1b-b61a09c1b5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:52.031921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:52.104494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2439c8-f42f-43be-bee6-072ce80ddbec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:52.106928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2439c8-f42f-43be-bee6-072ce80ddbec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:52.107219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2439c8-f42f-43be-bee6-072ce80ddbec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:52.108121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2439c8-f42f-43be-bee6-072ce80ddbec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/e085cf0d1c31436fb78e1a41eca71b39/implies/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:52.112684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2439c8-f42f-43be-bee6-072ce80ddbec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=e085cf0d1c31436fb78e1a41eca71b39, implied_role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:52.118048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2439c8-f42f-43be-bee6-072ce80ddbec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:52.142897 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2632/5253] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:51 2022] DELETE /identity/v3/roles/e085cf0d1c31436fb78e1a41eca71b39/implies/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:52.148267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2439c8-f42f-43be-bee6-072ce80ddbec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:52.197085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2622/5254] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:10:52.200038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a814e9-7ee8-4103-af1b-b61a09c1b5f7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:52.230896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a814e9-7ee8-4103-af1b-b61a09c1b5f7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:52.380834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2439c8-f42f-43be-bee6-072ce80ddbec tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:52.397176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5953d-8768-48e8-b193-59a7ec0c25ba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:52.398048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5953d-8768-48e8-b193-59a7ec0c25ba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:52.400109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5953d-8768-48e8-b193-59a7ec0c25ba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:52.400431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5953d-8768-48e8-b193-59a7ec0c25ba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/e085cf0d1c31436fb78e1a41eca71b39` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:52.405593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5953d-8768-48e8-b193-59a7ec0c25ba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=e085cf0d1c31436fb78e1a41eca71b39)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:52.408949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5953d-8768-48e8-b193-59a7ec0c25ba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:52.447062 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2633/5255] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:52 2022] DELETE /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/e085cf0d1c31436fb78e1a41eca71b39 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:52.452871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5953d-8768-48e8-b193-59a7ec0c25ba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:52.470150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee20f9-ead4-4e1f-99e6-b97aabcd68fe tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:52.470782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee20f9-ead4-4e1f-99e6-b97aabcd68fe tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:52.471064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee20f9-ead4-4e1f-99e6-b97aabcd68fe tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:52.471339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee20f9-ead4-4e1f-99e6-b97aabcd68fe tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:52.473880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee20f9-ead4-4e1f-99e6-b97aabcd68fe tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:52.475637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee20f9-ead4-4e1f-99e6-b97aabcd68fe tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:52.544316 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2623/5256] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:10:52 2022] POST /identity/v3/domains => generated 276 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:52.548105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee20f9-ead4-4e1f-99e6-b97aabcd68fe tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:52.606813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:52.679415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5953d-8768-48e8-b193-59a7ec0c25ba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:52.740094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee20f9-ead4-4e1f-99e6-b97aabcd68fe tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:52.755962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72a67c-9267-40ea-b70e-c2c972c9b772 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:52.756599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72a67c-9267-40ea-b70e-c2c972c9b772 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:52.758878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72a67c-9267-40ea-b70e-c2c972c9b772 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:52.759157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72a67c-9267-40ea-b70e-c2c972c9b772 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:52.761244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72a67c-9267-40ea-b70e-c2c972c9b772 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:52.763191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72a67c-9267-40ea-b70e-c2c972c9b772 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:52.846782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbeb78-c21a-4fe4-81f1-1650960b8f18 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:52.847446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbeb78-c21a-4fe4-81f1-1650960b8f18 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:52.849064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbeb78-c21a-4fe4-81f1-1650960b8f18 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:52.851180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbeb78-c21a-4fe4-81f1-1650960b8f18 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:52.857584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbeb78-c21a-4fe4-81f1-1650960b8f18 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:52.859377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbeb78-c21a-4fe4-81f1-1650960b8f18 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:52.878668 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2634/5257] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:52 2022] DELETE /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:52.880100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbeb78-c21a-4fe4-81f1-1650960b8f18 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:52.883350 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2624/5258] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:10:52 2022] POST /identity/v3/projects => generated 403 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:52.886201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72a67c-9267-40ea-b70e-c2c972c9b772 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:52.894319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:53.032585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbeb78-c21a-4fe4-81f1-1650960b8f18 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.039735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72a67c-9267-40ea-b70e-c2c972c9b772 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.059875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877248dd-1d85-4a9b-9be8-fd7d3f5fde6f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:53.061150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877248dd-1d85-4a9b-9be8-fd7d3f5fde6f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:53.062368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877248dd-1d85-4a9b-9be8-fd7d3f5fde6f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:53.063280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877248dd-1d85-4a9b-9be8-fd7d3f5fde6f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/groups` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:53.066554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877248dd-1d85-4a9b-9be8-fd7d3f5fde6f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:53.073733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877248dd-1d85-4a9b-9be8-fd7d3f5fde6f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:53.115992 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2625/5259] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:52 2022] POST /identity/v3/groups => generated 302 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:53.121093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877248dd-1d85-4a9b-9be8-fd7d3f5fde6f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:53.168920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877248dd-1d85-4a9b-9be8-fd7d3f5fde6f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.176662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b404b7c-748f-444a-b7d3-bae638cf072b tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:53.177316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b404b7c-748f-444a-b7d3-bae638cf072b tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:53.177707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b404b7c-748f-444a-b7d3-bae638cf072b tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:53.177863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b404b7c-748f-444a-b7d3-bae638cf072b tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:53.178489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b404b7c-748f-444a-b7d3-bae638cf072b tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:53.181506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b404b7c-748f-444a-b7d3-bae638cf072b tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:53.214820 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2626/5260] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:10:53 2022] POST /identity/v3/users => generated 445 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:53.222783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b404b7c-748f-444a-b7d3-bae638cf072b tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:53.254630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b404b7c-748f-444a-b7d3-bae638cf072b tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.258979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ea744-a8bb-413f-a19d-7e1eb8cb3225 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:53.260491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ea744-a8bb-413f-a19d-7e1eb8cb3225 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:53.262948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ea744-a8bb-413f-a19d-7e1eb8cb3225 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:53.265125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ea744-a8bb-413f-a19d-7e1eb8cb3225 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:53.266138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ea744-a8bb-413f-a19d-7e1eb8cb3225 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:53.269872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ea744-a8bb-413f-a19d-7e1eb8cb3225 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:53.271188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d030c-38e1-4321-9c9d-40a72a973243 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:53.271568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d030c-38e1-4321-9c9d-40a72a973243 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:53.271847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d030c-38e1-4321-9c9d-40a72a973243 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:53.272173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d030c-38e1-4321-9c9d-40a72a973243 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:53.273552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d030c-38e1-4321-9c9d-40a72a973243 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:53.275161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d030c-38e1-4321-9c9d-40a72a973243 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:53.318873 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2627/5261] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:53 2022] POST /identity/v3/roles => generated 240 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:53.320249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d030c-38e1-4321-9c9d-40a72a973243 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:53.331943 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2635/5262] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:52 2022] GET /identity/v3/roles => generated 2323 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:53.336473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ea744-a8bb-413f-a19d-7e1eb8cb3225 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:53.368884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ea744-a8bb-413f-a19d-7e1eb8cb3225 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.372253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d030c-38e1-4321-9c9d-40a72a973243 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.380548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d484790-b187-4de0-9dae-c978117c8319 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:53.381563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484790-b187-4de0-9dae-c978117c8319 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:53.381821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484790-b187-4de0-9dae-c978117c8319 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:53.382001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484790-b187-4de0-9dae-c978117c8319 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/1b01f83a26d44b37bf246dfc33621a47/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:53.391553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f614a-9f42-4593-b6ab-58f3a6ebc803 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:53.393704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f614a-9f42-4593-b6ab-58f3a6ebc803 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:53.395401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f614a-9f42-4593-b6ab-58f3a6ebc803 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:53.396791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f614a-9f42-4593-b6ab-58f3a6ebc803 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:53.398710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f614a-9f42-4593-b6ab-58f3a6ebc803 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:53.408862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f614a-9f42-4593-b6ab-58f3a6ebc803 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:53.434215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d484790-b187-4de0-9dae-c978117c8319 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7d5517e8193c409a8d77d52269b338f2, group_id=cd4e7ec7048d42d0aa0cac5adebdde51, role_id=1b01f83a26d44b37bf246dfc33621a47)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:53.437204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d484790-b187-4de0-9dae-c978117c8319 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:53.464833 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2628/5263] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:53 2022] POST /identity/v3/roles => generated 235 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:53.467701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f614a-9f42-4593-b6ab-58f3a6ebc803 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:53.485799 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2636/5264] 10.222.0.16 () {60 vars in 1585 bytes} [Sat Oct 15 21:10:53 2022] PUT /identity/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/1b01f83a26d44b37bf246dfc33621a47/inherited_to_projects => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:53.495465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d484790-b187-4de0-9dae-c978117c8319 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:53.496773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f614a-9f42-4593-b6ab-58f3a6ebc803 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.633237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:53.672763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d484790-b187-4de0-9dae-c978117c8319 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.676336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd66a3d-d526-4b15-9fbe-c5fa7911d670 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:53.678244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd66a3d-d526-4b15-9fbe-c5fa7911d670 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:53.679248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd66a3d-d526-4b15-9fbe-c5fa7911d670 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:53.679695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd66a3d-d526-4b15-9fbe-c5fa7911d670 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/29947cdd539c4c2ab8574651f52d562b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:53.682679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd66a3d-d526-4b15-9fbe-c5fa7911d670 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:update_role(role_id=29947cdd539c4c2ab8574651f52d562b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:53.685177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17b907-359e-48ad-a6f5-bc484c17798a tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:53.686321 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-6bd66a3d-d526-4b15-9fbe-c5fa7911d670 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:53.686321 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:53.686321 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:53.687230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17b907-359e-48ad-a6f5-bc484c17798a tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:53.688710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17b907-359e-48ad-a6f5-bc484c17798a tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:53.689284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd66a3d-d526-4b15-9fbe-c5fa7911d670 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:53.690200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17b907-359e-48ad-a6f5-bc484c17798a tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/1b01f83a26d44b37bf246dfc33621a47/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:53.695838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17b907-359e-48ad-a6f5-bc484c17798a tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:check_grant(project_id=7d5517e8193c409a8d77d52269b338f2, group_id=cd4e7ec7048d42d0aa0cac5adebdde51, role_id=1b01f83a26d44b37bf246dfc33621a47)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:53.701941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17b907-359e-48ad-a6f5-bc484c17798a tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:53.715947 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2637/5265] 10.222.0.16 () {58 vars in 1567 bytes} [Sat Oct 15 21:10:53 2022] HEAD /identity/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/1b01f83a26d44b37bf246dfc33621a47/inherited_to_projects => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:53.715947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17b907-359e-48ad-a6f5-bc484c17798a tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:53.719975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1f17b907-359e-48ad-a6f5-bc484c17798a tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:53.783897 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2629/5266] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Oct 15 21:10:53 2022] PATCH /identity/v3/roles/29947cdd539c4c2ab8574651f52d562b => generated 237 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:53.784876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd66a3d-d526-4b15-9fbe-c5fa7911d670 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:53.815190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd66a3d-d526-4b15-9fbe-c5fa7911d670 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.821424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:53.822172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:53.822444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:53.823528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/1b01f83a26d44b37bf246dfc33621a47/inherited_to_projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:53.827092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=7d5517e8193c409a8d77d52269b338f2, group_id=cd4e7ec7048d42d0aa0cac5adebdde51, role_id=1b01f83a26d44b37bf246dfc33621a47)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:53.830562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:53.839330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role 1b01f83a26d44b37bf246dfc33621a47 was removed from group cd4e7ec7048d42d0aa0cac5adebdde51 on project 7d5517e8193c409a8d77d52269b338f2. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:53.839616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:53.840683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:53.848192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:53.872255 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2630/5267] 10.222.0.16 () {58 vars in 1569 bytes} [Sat Oct 15 21:10:53 2022] DELETE /identity/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/1b01f83a26d44b37bf246dfc33621a47/inherited_to_projects => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:53.872255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:53.873938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:53.953340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17b907-359e-48ad-a6f5-bc484c17798a tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.977998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f7abb-d885-45f9-923f-67c90ea647bc tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:53.996800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c50cd-dd2a-47da-acf8-c0b001d9abcb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:53.997881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c50cd-dd2a-47da-acf8-c0b001d9abcb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:53.998598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c50cd-dd2a-47da-acf8-c0b001d9abcb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:53.999103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c50cd-dd2a-47da-acf8-c0b001d9abcb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/29947cdd539c4c2ab8574651f52d562b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:54.019675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c50cd-dd2a-47da-acf8-c0b001d9abcb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:get_role(role_id=29947cdd539c4c2ab8574651f52d562b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:54.023727 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-6d7c50cd-dd2a-47da-acf8-c0b001d9abcb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:54.023727 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:10:54.023727 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:10:54.025114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c50cd-dd2a-47da-acf8-c0b001d9abcb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:54.034746 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2631/5268] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:10:53 2022] GET /identity/v3/roles/29947cdd539c4c2ab8574651f52d562b => generated 237 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:54.036001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c50cd-dd2a-47da-acf8-c0b001d9abcb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:54.041300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6d7c50cd-dd2a-47da-acf8-c0b001d9abcb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:54.093561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6d40d-c04e-4c3e-8d89-8de25e3f86a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:54.094202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6d40d-c04e-4c3e-8d89-8de25e3f86a1 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:54.094499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6d40d-c04e-4c3e-8d89-8de25e3f86a1 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:54.094948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6d40d-c04e-4c3e-8d89-8de25e3f86a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:54.097634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83f6d40d-c04e-4c3e-8d89-8de25e3f86a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:54.115947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6d40d-c04e-4c3e-8d89-8de25e3f86a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:54.120331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6d40d-c04e-4c3e-8d89-8de25e3f86a1 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:54.168155 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2638/5269] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:54.168953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6d40d-c04e-4c3e-8d89-8de25e3f86a1 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:54.202281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6d40d-c04e-4c3e-8d89-8de25e3f86a1 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:54.222423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475f6cb-8cac-4cd2-b482-2b4d278aed9c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:54.223468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475f6cb-8cac-4cd2-b482-2b4d278aed9c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:54.223879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475f6cb-8cac-4cd2-b482-2b4d278aed9c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:54.224288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475f6cb-8cac-4cd2-b482-2b4d278aed9c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:54.226955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475f6cb-8cac-4cd2-b482-2b4d278aed9c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:54.230490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475f6cb-8cac-4cd2-b482-2b4d278aed9c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:54.266385 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2639/5270] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:10:54 2022] GET /identity/v3/roles => generated 2782 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:10:54.269129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475f6cb-8cac-4cd2-b482-2b4d278aed9c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:54.293992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475f6cb-8cac-4cd2-b482-2b4d278aed9c tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:54.302642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c50cd-dd2a-47da-acf8-c0b001d9abcb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:54.307718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:54.311016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:54.312267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:54.313293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/29947cdd539c4c2ab8574651f52d562b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:54.314353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:54.315815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:54.316253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:54.316676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles/1b01f83a26d44b37bf246dfc33621a47` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:54.317975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_role(role_id=29947cdd539c4c2ab8574651f52d562b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:54.325860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_role(role_id=1b01f83a26d44b37bf246dfc33621a47)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:54.327484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:54.332434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:54.412409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role 1b01f83a26d44b37bf246dfc33621a47 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:54.412409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:54.414499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:54.419394 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2632/5271] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:54 2022] DELETE /identity/v3/roles/1b01f83a26d44b37bf246dfc33621a47 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:54.427013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:54.429767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:54.437193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role 29947cdd539c4c2ab8574651f52d562b has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:54.437850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:54.441319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:54.445116 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2640/5272] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:54 2022] DELETE /identity/v3/roles/29947cdd539c4c2ab8574651f52d562b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:54.451734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:54.452773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:54.467374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:54.617976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78510f16-e4d2-4f45-a405-67cb6405cde9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:54.629727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b58372-139c-416c-ba6b-fa9d08df8cd8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:54.643691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e701-fb3c-449e-8488-75f8c049a004 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:54.645196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e701-fb3c-449e-8488-75f8c049a004 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:54.645486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e701-fb3c-449e-8488-75f8c049a004 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:54.647151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e701-fb3c-449e-8488-75f8c049a004 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:54.652599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68e701-fb3c-449e-8488-75f8c049a004 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:54.657826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68e701-fb3c-449e-8488-75f8c049a004 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:54.668204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:54.711851 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2641/5273] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:54 2022] PUT /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 521 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:54.716120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e701-fb3c-449e-8488-75f8c049a004 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:54.815744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d454d0d-865e-4dfe-b302-180d1310b0de tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:54.818397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d454d0d-865e-4dfe-b302-180d1310b0de tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:54.820494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d454d0d-865e-4dfe-b302-180d1310b0de tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:54.820817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d454d0d-865e-4dfe-b302-180d1310b0de tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:54.828589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d454d0d-865e-4dfe-b302-180d1310b0de tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:54.834467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d454d0d-865e-4dfe-b302-180d1310b0de tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:54.879073 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2633/5274] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:54 2022] POST /identity/v3/roles => generated 240 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:54.883123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e701-fb3c-449e-8488-75f8c049a004 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:54.886055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d454d0d-865e-4dfe-b302-180d1310b0de tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:54.892414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4d454d0d-865e-4dfe-b302-180d1310b0de tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:54.900067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e16e0d-137d-4e99-9c71-ca9080db4ee8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:54.901630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e16e0d-137d-4e99-9c71-ca9080db4ee8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:54.901920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e16e0d-137d-4e99-9c71-ca9080db4ee8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:54.902814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e16e0d-137d-4e99-9c71-ca9080db4ee8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/e085cf0d1c31436fb78e1a41eca71b39` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:54.907918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e16e0d-137d-4e99-9c71-ca9080db4ee8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=e085cf0d1c31436fb78e1a41eca71b39)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:54.911673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e16e0d-137d-4e99-9c71-ca9080db4ee8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:54.925008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d454d0d-865e-4dfe-b302-180d1310b0de tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:54.978972 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2642/5275] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:54 2022] PUT /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/e085cf0d1c31436fb78e1a41eca71b39 => generated 520 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:54.983764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e16e0d-137d-4e99-9c71-ca9080db4ee8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:55.082590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b87a-e017-4ff1-96f5-52fac4e4227d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:55.083282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b87a-e017-4ff1-96f5-52fac4e4227d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:55.083567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b87a-e017-4ff1-96f5-52fac4e4227d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:55.083845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b87a-e017-4ff1-96f5-52fac4e4227d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/2e47a1cfa1e9411d8cb01b507abc7d31/inherited_to_projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:55.131125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0b87a-e017-4ff1-96f5-52fac4e4227d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7d5517e8193c409a8d77d52269b338f2, user_id=116af5d85ed04da6ba2b2bb1007aaef9, role_id=2e47a1cfa1e9411d8cb01b507abc7d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:55.137212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0b87a-e017-4ff1-96f5-52fac4e4227d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:55.139422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e16e0d-137d-4e99-9c71-ca9080db4ee8 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:55.146596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3bf31-7229-40c9-bdf0-39e430795414 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:55.147670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3bf31-7229-40c9-bdf0-39e430795414 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:55.148432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3bf31-7229-40c9-bdf0-39e430795414 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:55.149693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3bf31-7229-40c9-bdf0-39e430795414 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/e085cf0d1c31436fb78e1a41eca71b39/implies/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:55.153606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3bf31-7229-40c9-bdf0-39e430795414 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=e085cf0d1c31436fb78e1a41eca71b39, implied_role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:55.155675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3bf31-7229-40c9-bdf0-39e430795414 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:55.172008 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2634/5276] 10.222.0.16 () {60 vars in 1582 bytes} [Sat Oct 15 21:10:54 2022] PUT /identity/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/2e47a1cfa1e9411d8cb01b507abc7d31/inherited_to_projects => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:55.177377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b87a-e017-4ff1-96f5-52fac4e4227d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:55.206613 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2643/5277] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Oct 15 21:10:54 2022] PUT /identity/v3/roles/e085cf0d1c31436fb78e1a41eca71b39/implies/fd1a838a0c6742c4a46797eb352eded1 => generated 520 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:10:55.207892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3bf31-7229-40c9-bdf0-39e430795414 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:55.315217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:55.329251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:55.435694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3bf31-7229-40c9-bdf0-39e430795414 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:55.444634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d5a8e-aac6-4399-9ed3-26c8290548be tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:55.446607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d5a8e-aac6-4399-9ed3-26c8290548be tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:55.447077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d5a8e-aac6-4399-9ed3-26c8290548be tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:55.448558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d5a8e-aac6-4399-9ed3-26c8290548be tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/e085cf0d1c31436fb78e1a41eca71b39/implies` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:55.450094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d5a8e-aac6-4399-9ed3-26c8290548be tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=e085cf0d1c31436fb78e1a41eca71b39)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:55.453440 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-863d5a8e-aac6-4399-9ed3-26c8290548be tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:10:55.453440 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:10:55.453440 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:10:55.454549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b87a-e017-4ff1-96f5-52fac4e4227d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:55.458168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d5a8e-aac6-4399-9ed3-26c8290548be tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:55.483175 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2644/5278] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:10:55 2022] GET /identity/v3/roles/e085cf0d1c31436fb78e1a41eca71b39/implies => generated 489 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:55.487360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d5a8e-aac6-4399-9ed3-26c8290548be tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:55.512007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d5a8e-aac6-4399-9ed3-26c8290548be tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:55.524470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db869794-b3a3-4dce-8a1d-d73d6a318cba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:55.526462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db869794-b3a3-4dce-8a1d-d73d6a318cba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:55.526778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db869794-b3a3-4dce-8a1d-d73d6a318cba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:55.527051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db869794-b3a3-4dce-8a1d-d73d6a318cba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:55.530132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db869794-b3a3-4dce-8a1d-d73d6a318cba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:55.536531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db869794-b3a3-4dce-8a1d-d73d6a318cba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:55.558267 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2645/5279] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:10:55 2022] GET /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies => generated 660 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:55.560464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db869794-b3a3-4dce-8a1d-d73d6a318cba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:55.604732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db869794-b3a3-4dce-8a1d-d73d6a318cba tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:55.619509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5271b9-4e3b-478f-ac57-266148fadd67 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:55.620344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5271b9-4e3b-478f-ac57-266148fadd67 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:55.620682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5271b9-4e3b-478f-ac57-266148fadd67 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:55.620956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5271b9-4e3b-478f-ac57-266148fadd67 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/e085cf0d1c31436fb78e1a41eca71b39/implies/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:55.624420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5271b9-4e3b-478f-ac57-266148fadd67 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=e085cf0d1c31436fb78e1a41eca71b39, implied_role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:55.628945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5271b9-4e3b-478f-ac57-266148fadd67 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:55.635395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad218d-cd73-470d-8f9b-69fb6892e62c tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:55.636823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad218d-cd73-470d-8f9b-69fb6892e62c tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:55.637370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad218d-cd73-470d-8f9b-69fb6892e62c tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:55.638326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad218d-cd73-470d-8f9b-69fb6892e62c tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/2e47a1cfa1e9411d8cb01b507abc7d31/inherited_to_projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:55.641709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad218d-cd73-470d-8f9b-69fb6892e62c tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:check_grant(project_id=7d5517e8193c409a8d77d52269b338f2, user_id=116af5d85ed04da6ba2b2bb1007aaef9, role_id=2e47a1cfa1e9411d8cb01b507abc7d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:55.645868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad218d-cd73-470d-8f9b-69fb6892e62c tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:55.655622 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2635/5280] 10.222.0.16 () {58 vars in 1564 bytes} [Sat Oct 15 21:10:55 2022] HEAD /identity/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/2e47a1cfa1e9411d8cb01b507abc7d31/inherited_to_projects => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:55.658888 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2646/5281] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:55 2022] DELETE /identity/v3/roles/e085cf0d1c31436fb78e1a41eca71b39/implies/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:55.660554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5271b9-4e3b-478f-ac57-266148fadd67 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:55.661976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad218d-cd73-470d-8f9b-69fb6892e62c tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:55.689300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad218d-cd73-470d-8f9b-69fb6892e62c tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:55.892190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5271b9-4e3b-478f-ac57-266148fadd67 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:55.904891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:55.905729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:55.905974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:55.906359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/2e47a1cfa1e9411d8cb01b507abc7d31/inherited_to_projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:55.913931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cfa2d-3551-489b-86a3-d2911a79c6ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:55.914921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=7d5517e8193c409a8d77d52269b338f2, user_id=116af5d85ed04da6ba2b2bb1007aaef9, role_id=2e47a1cfa1e9411d8cb01b507abc7d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:55.915320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cfa2d-3551-489b-86a3-d2911a79c6ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:55.916654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cfa2d-3551-489b-86a3-d2911a79c6ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:55.917176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cfa2d-3551-489b-86a3-d2911a79c6ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/e085cf0d1c31436fb78e1a41eca71b39` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:55.917900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:55.919967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108cfa2d-3551-489b-86a3-d2911a79c6ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=e085cf0d1c31436fb78e1a41eca71b39)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:55.922556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108cfa2d-3551-489b-86a3-d2911a79c6ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:55.924557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role 2e47a1cfa1e9411d8cb01b507abc7d31 was removed from user 116af5d85ed04da6ba2b2bb1007aaef9 on project 7d5517e8193c409a8d77d52269b338f2. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:55.925199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:55.926086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:55.946216 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2647/5282] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:55 2022] DELETE /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/e085cf0d1c31436fb78e1a41eca71b39 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:55.948909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cfa2d-3551-489b-86a3-d2911a79c6ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:55.951602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-108cfa2d-3551-489b-86a3-d2911a79c6ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:55.957694 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2636/5283] 10.222.0.16 () {58 vars in 1566 bytes} [Sat Oct 15 21:10:55 2022] DELETE /identity/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/2e47a1cfa1e9411d8cb01b507abc7d31/inherited_to_projects => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:55.958689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:55.960860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:56.060559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cfa2d-3551-489b-86a3-d2911a79c6ed tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:56.123868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15e552-a3d5-46b6-a883-693c61abb12f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:56.130113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a962-d10e-48bf-aab3-b075881999ff tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:56.130991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a962-d10e-48bf-aab3-b075881999ff tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:56.131270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a962-d10e-48bf-aab3-b075881999ff tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:56.131796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a962-d10e-48bf-aab3-b075881999ff tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:56.133614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333a962-d10e-48bf-aab3-b075881999ff tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e, implied_role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:56.136463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333a962-d10e-48bf-aab3-b075881999ff tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:56.159423 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2637/5284] 10.222.0.16 () {58 vars in 1347 bytes} [Sat Oct 15 21:10:55 2022] DELETE /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e/implies/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:56.161763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a962-d10e-48bf-aab3-b075881999ff tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:56.164638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:56.165461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:56.165582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0333a962-d10e-48bf-aab3-b075881999ff tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:56.165865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:56.166205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] PATH_INFO: `/v3/users/6085f73e836f4be8becf2182de6ebaa0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:56.168385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6085f73e836f4be8becf2182de6ebaa0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:56.171832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:56.174705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:56.200762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:56.298211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6085f73e836f4be8becf2182de6ebaa0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:56.350261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6085f73e836f4be8becf2182de6ebaa0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:56.360505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a962-d10e-48bf-aab3-b075881999ff tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:56.372965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6085f73e836f4be8becf2182de6ebaa0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:56.379405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b2855-58a7-4945-8ba7-31c2b6964bab tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:56.380077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b2855-58a7-4945-8ba7-31c2b6964bab tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:56.380359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b2855-58a7-4945-8ba7-31c2b6964bab tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:56.380633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b2855-58a7-4945-8ba7-31c2b6964bab tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles/2e47a1cfa1e9411d8cb01b507abc7d31` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:56.388308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b2855-58a7-4945-8ba7-31c2b6964bab tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2e47a1cfa1e9411d8cb01b507abc7d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:56.389973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b2855-58a7-4945-8ba7-31c2b6964bab tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:56.398248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6085f73e836f4be8becf2182de6ebaa0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:56.400093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6085f73e836f4be8becf2182de6ebaa0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:56.410644 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2648/5285] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:55 2022] DELETE /identity/v3/users/6085f73e836f4be8becf2182de6ebaa0 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:56.417030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:56.419330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:56.476210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a56b2855-58a7-4945-8ba7-31c2b6964bab tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role 2e47a1cfa1e9411d8cb01b507abc7d31 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:56.476771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a56b2855-58a7-4945-8ba7-31c2b6964bab tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:56.479876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a56b2855-58a7-4945-8ba7-31c2b6964bab tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:56.483433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2638/5286] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:56 2022] DELETE /identity/v3/roles/2e47a1cfa1e9411d8cb01b507abc7d31 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:56.484743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b0c6d-7455-4aa2-9b67-2a9a4f6b4b71 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:56.485023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b0c6d-7455-4aa2-9b67-2a9a4f6b4b71 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:56.490509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b0c6d-7455-4aa2-9b67-2a9a4f6b4b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:56.535666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0b7b0c6d-7455-4aa2-9b67-2a9a4f6b4b71 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:10:56.536910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0b7b0c6d-7455-4aa2-9b67-2a9a4f6b4b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:56.621471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd5df7-a3ea-4362-b605-fa916e725b88 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:56.633610 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2639/5287] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:10:56.636069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b0c6d-7455-4aa2-9b67-2a9a4f6b4b71 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:56.638291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0b7b0c6d-7455-4aa2-9b67-2a9a4f6b4b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:56.739766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41857e31-a3ad-40e4-a990-cfe6dec47a5d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:56.740459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41857e31-a3ad-40e4-a990-cfe6dec47a5d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:56.740724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41857e31-a3ad-40e4-a990-cfe6dec47a5d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:56.741638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41857e31-a3ad-40e4-a990-cfe6dec47a5d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/4e83b07dc2f64291a9067b3b0a0d5124` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:56.797003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41857e31-a3ad-40e4-a990-cfe6dec47a5d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 21:10:56.810291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b0c6d-7455-4aa2-9b67-2a9a4f6b4b71 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-41857e31-a3ad-40e4-a990-cfe6dec47a5d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Could not find domain: 4e83b07dc2f64291a9067b3b0a0d5124.: keystone.exception.DomainNotFound: Could not find domain: 4e83b07dc2f64291a9067b3b0a0d5124. Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 4e83b07dc2f64291a9067b3b0a0d5124. Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:10:56.824169 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:10:56.825582 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 4e83b07dc2f64291a9067b3b0a0d5124. Oct 15 21:10:56.826822 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:10:56.829050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d98eab-9fd2-40fd-8e98-3e4d8f45c549 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:56.829762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d98eab-9fd2-40fd-8e98-3e4d8f45c549 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:56.830842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d98eab-9fd2-40fd-8e98-3e4d8f45c549 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:56.831468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d98eab-9fd2-40fd-8e98-3e4d8f45c549 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:56.832337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d98eab-9fd2-40fd-8e98-3e4d8f45c549 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:56.834240 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2649/5288] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:10:56 2022] PATCH /identity/v3/domains/4e83b07dc2f64291a9067b3b0a0d5124 => generated 112 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:10:56.834362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d98eab-9fd2-40fd-8e98-3e4d8f45c549 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:56.842080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41857e31-a3ad-40e4-a990-cfe6dec47a5d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:56.847600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-41857e31-a3ad-40e4-a990-cfe6dec47a5d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:56.910180 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2640/5289] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:56 2022] POST /identity/v3/roles => generated 240 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:56.914192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d98eab-9fd2-40fd-8e98-3e4d8f45c549 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:56.926014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-17d98eab-9fd2-40fd-8e98-3e4d8f45c549 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:56.951816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:56.959084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d98eab-9fd2-40fd-8e98-3e4d8f45c549 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:56.971956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:56.974056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:56.975401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:56.977434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:56.977889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/users/4d18588ffdd8497ab8903f4a5a81fe87` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:56.982669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_user(user_id=4d18588ffdd8497ab8903f4a5a81fe87)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:56.986030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:57.031393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41857e31-a3ad-40e4-a990-cfe6dec47a5d tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:57.035536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10586cf9-f740-47c3-9c52-3f1407af1908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:57.036296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10586cf9-f740-47c3-9c52-3f1407af1908 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:57.036869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10586cf9-f740-47c3-9c52-3f1407af1908 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:57.037388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10586cf9-f740-47c3-9c52-3f1407af1908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:57.063217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10586cf9-f740-47c3-9c52-3f1407af1908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:57.064661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10586cf9-f740-47c3-9c52-3f1407af1908 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:57.076136 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2650/5290] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:10:57.077167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10586cf9-f740-47c3-9c52-3f1407af1908 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:57.089867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10586cf9-f740-47c3-9c52-3f1407af1908 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:57.102926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e83364-b2e6-41f1-abd7-0a4e29e283f1 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:57.103611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e83364-b2e6-41f1-abd7-0a4e29e283f1 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:57.103888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e83364-b2e6-41f1-abd7-0a4e29e283f1 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:57.106894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e83364-b2e6-41f1-abd7-0a4e29e283f1 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/dba4166bad2a4effa61627cd50ba3f51/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:57.109467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e83364-b2e6-41f1-abd7-0a4e29e283f1 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=f5765fae0b04466d9bcf450f38daefcb, group_id=cd4e7ec7048d42d0aa0cac5adebdde51, role_id=dba4166bad2a4effa61627cd50ba3f51)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:57.112045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d18588ffdd8497ab8903f4a5a81fe87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:57.113769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e83364-b2e6-41f1-abd7-0a4e29e283f1 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:57.133986 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2651/5291] 10.222.0.16 () {60 vars in 1582 bytes} [Sat Oct 15 21:10:57 2022] PUT /identity/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/dba4166bad2a4effa61627cd50ba3f51/inherited_to_projects => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:57.136761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d18588ffdd8497ab8903f4a5a81fe87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:57.137801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e83364-b2e6-41f1-abd7-0a4e29e283f1 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:57.163308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d18588ffdd8497ab8903f4a5a81fe87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:57.183376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d18588ffdd8497ab8903f4a5a81fe87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:57.187283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d18588ffdd8497ab8903f4a5a81fe87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:57.199825 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2641/5292] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:56 2022] DELETE /identity/v3/users/4d18588ffdd8497ab8903f4a5a81fe87 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:57.203621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:57.206306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:57.337142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e83364-b2e6-41f1-abd7-0a4e29e283f1 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:57.431022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a61d18-4e3c-4867-838c-fb58534cc5df tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:57.437107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:57.437581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:57.437858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:57.438131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/fd1a838a0c6742c4a46797eb352eded1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:57.439594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_role(role_id=fd1a838a0c6742c4a46797eb352eded1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:57.441832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:57.477257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45126d9e-8996-45d4-a950-9c2c5b3d4aae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:57.478079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45126d9e-8996-45d4-a950-9c2c5b3d4aae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:57.478409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45126d9e-8996-45d4-a950-9c2c5b3d4aae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:57.479074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45126d9e-8996-45d4-a950-9c2c5b3d4aae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:57.483277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45126d9e-8996-45d4-a950-9c2c5b3d4aae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=f5765fae0b04466d9bcf450f38daefcb, group_id=cd4e7ec7048d42d0aa0cac5adebdde51)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:57.488324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45126d9e-8996-45d4-a950-9c2c5b3d4aae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:57.543172 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2652/5293] 10.222.0.16 () {58 vars in 1464 bytes} [Sat Oct 15 21:10:57 2022] GET /identity/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/inherited_to_projects => generated 448 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:10:57.546938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45126d9e-8996-45d4-a950-9c2c5b3d4aae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:57.552055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-45126d9e-8996-45d4-a950-9c2c5b3d4aae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:57.578887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role fd1a838a0c6742c4a46797eb352eded1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:57.580373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:57.583498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:57.590797 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2642/5294] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:57 2022] DELETE /identity/v3/roles/fd1a838a0c6742c4a46797eb352eded1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:57.590797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:57.594063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:57.662159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45126d9e-8996-45d4-a950-9c2c5b3d4aae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:57.669879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:57.684045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:57.721629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299045f-c6c4-4621-afaf-3ee2a589dceb tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:57.728952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50e1b3-37b9-40c3-87c7-3f6029f6a648 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:57.729786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50e1b3-37b9-40c3-87c7-3f6029f6a648 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:57.730066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50e1b3-37b9-40c3-87c7-3f6029f6a648 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:57.730342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50e1b3-37b9-40c3-87c7-3f6029f6a648 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/dba4166bad2a4effa61627cd50ba3f51/inherited_to_projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:57.733929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50e1b3-37b9-40c3-87c7-3f6029f6a648 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=f5765fae0b04466d9bcf450f38daefcb, group_id=cd4e7ec7048d42d0aa0cac5adebdde51, role_id=dba4166bad2a4effa61627cd50ba3f51)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:57.744622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50e1b3-37b9-40c3-87c7-3f6029f6a648 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:57.752989 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2643/5295] 10.222.0.16 () {58 vars in 1564 bytes} [Sat Oct 15 21:10:57 2022] HEAD /identity/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/dba4166bad2a4effa61627cd50ba3f51/inherited_to_projects => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:57.755117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50e1b3-37b9-40c3-87c7-3f6029f6a648 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:57.759137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4c50e1b3-37b9-40c3-87c7-3f6029f6a648 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:57.812514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:57.813310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:57.813592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:57.813867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] PATH_INFO: `/v3/projects/605f7455125b45f9a9ae4de530afcf9b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:57.820456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=605f7455125b45f9a9ae4de530afcf9b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:57.825850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:57.935241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '605f7455125b45f9a9ae4de530afcf9b', 'request_id': 'req-557328fb-1d3a-4a00-bd5d-224438fd4c44'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:57.949574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '605f7455125b45f9a9ae4de530afcf9b', 'request_id': 'req-557328fb-1d3a-4a00-bd5d-224438fd4c44'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:57.974036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '605f7455125b45f9a9ae4de530afcf9b', 'request_id': 'req-557328fb-1d3a-4a00-bd5d-224438fd4c44'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:57.998972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50e1b3-37b9-40c3-87c7-3f6029f6a648 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:58.016132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] The token cache is being invalidate because project 605f7455125b45f9a9ae4de530afcf9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:58.016556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.026579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.031018 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2653/5296] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:57 2022] DELETE /identity/v3/projects/605f7455125b45f9a9ae4de530afcf9b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:58.032677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:58.036805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:58.165448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557328fb-1d3a-4a00-bd5d-224438fd4c44 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:58.179090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1034d-eb8b-4f85-a034-a781fb20be4e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:58.187890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:58.188264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1034d-eb8b-4f85-a034-a781fb20be4e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:58.188794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1034d-eb8b-4f85-a034-a781fb20be4e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:58.188936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:58.189314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:58.189932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1034d-eb8b-4f85-a034-a781fb20be4e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/groups/5f2f341b06584412b37b181285bb7608` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:58.190072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/dba4166bad2a4effa61627cd50ba3f51/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:58.191943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1034d-eb8b-4f85-a034-a781fb20be4e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_group(group_id=5f2f341b06584412b37b181285bb7608)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:58.195320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1034d-eb8b-4f85-a034-a781fb20be4e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:58.195466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=f5765fae0b04466d9bcf450f38daefcb, group_id=cd4e7ec7048d42d0aa0cac5adebdde51, role_id=dba4166bad2a4effa61627cd50ba3f51)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:58.198239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:58.211888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role dba4166bad2a4effa61627cd50ba3f51 was removed from group cd4e7ec7048d42d0aa0cac5adebdde51 on domain f5765fae0b04466d9bcf450f38daefcb. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:58.212237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.213458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.241960 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2654/5297] 10.222.0.16 () {58 vars in 1566 bytes} [Sat Oct 15 21:10:58 2022] DELETE /identity/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/groups/cd4e7ec7048d42d0aa0cac5adebdde51/roles/dba4166bad2a4effa61627cd50ba3f51/inherited_to_projects => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:58.242920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:58.249738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:58.314271 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2644/5298] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:10:57 2022] DELETE /identity/v3/groups/5f2f341b06584412b37b181285bb7608 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:58.317387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1034d-eb8b-4f85-a034-a781fb20be4e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:58.317387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-50a1034d-eb8b-4f85-a034-a781fb20be4e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:58.348312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:58.458403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24e897-5c72-4d79-af05-6f140ec912ae tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:58.461150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:58.527857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1034d-eb8b-4f85-a034-a781fb20be4e tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:58.535359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:58.536148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:58.536559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:58.536920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles/dba4166bad2a4effa61627cd50ba3f51` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:58.542575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_role(role_id=dba4166bad2a4effa61627cd50ba3f51)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:58.544680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:58.597656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role dba4166bad2a4effa61627cd50ba3f51 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:58.598679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.600374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.601377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:58.602613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:58.603416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:58.604571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] PATH_INFO: `/v3/users/cdaa336c6ff2489c8a8d360fbd418177` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:58.610782 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2645/5299] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:58 2022] DELETE /identity/v3/roles/dba4166bad2a4effa61627cd50ba3f51 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:58.611559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdaa336c6ff2489c8a8d360fbd418177)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:58.612405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:58.615443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:58.616625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:58.771362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdaa336c6ff2489c8a8d360fbd418177'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.802475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecd049-5b08-4afd-8497-9332768aa49d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:58.814751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:58.816388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:58.818019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:58.818019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/projects/2335de37d6144200bbdef5af1a5cbab2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:58.820935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2335de37d6144200bbdef5af1a5cbab2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:58.824576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:58.843849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdaa336c6ff2489c8a8d360fbd418177'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.880373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdaa336c6ff2489c8a8d360fbd418177'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.912799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdaa336c6ff2489c8a8d360fbd418177'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.917460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdaa336c6ff2489c8a8d360fbd418177'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:58.932833 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2655/5300] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:58 2022] DELETE /identity/v3/users/cdaa336c6ff2489c8a8d360fbd418177 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:58.937220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:58.942629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:59.010007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2335de37d6144200bbdef5af1a5cbab2', 'request_id': 'req-a0755451-71e9-4066-a1f9-cf558e97a0c4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.034085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2335de37d6144200bbdef5af1a5cbab2', 'request_id': 'req-a0755451-71e9-4066-a1f9-cf558e97a0c4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.046421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2335de37d6144200bbdef5af1a5cbab2', 'request_id': 'req-a0755451-71e9-4066-a1f9-cf558e97a0c4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.054515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:59.069517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] The token cache is being invalidate because project 2335de37d6144200bbdef5af1a5cbab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:59.070617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.071696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.074254 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2646/5301] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:10:58 2022] DELETE /identity/v3/projects/2335de37d6144200bbdef5af1a5cbab2 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:59.076647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70464a55-c04b-403d-8d58-26cfe56f0773 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:59.080093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:59.081583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:59.181759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f77a1-22e0-4d70-8488-0fc3b5cb03cb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:59.182692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f77a1-22e0-4d70-8488-0fc3b5cb03cb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:59.183302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f77a1-22e0-4d70-8488-0fc3b5cb03cb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:59.183638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f77a1-22e0-4d70-8488-0fc3b5cb03cb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:59.185655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f77a1-22e0-4d70-8488-0fc3b5cb03cb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:59.185806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:59.187996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f77a1-22e0-4d70-8488-0fc3b5cb03cb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:59.238250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0755451-71e9-4066-a1f9-cf558e97a0c4 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:59.255725 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2656/5302] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:58 2022] POST /identity/v3/roles => generated 239 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:10:59.257455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f77a1-22e0-4d70-8488-0fc3b5cb03cb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:59.260485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cb5f77a1-22e0-4d70-8488-0fc3b5cb03cb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:59.266312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:59.267165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:59.269377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:59.269857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/46b5964105bc4469ab876d72a935c181` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:59.275309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=46b5964105bc4469ab876d72a935c181)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:59.276365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f77a1-22e0-4d70-8488-0fc3b5cb03cb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:59.278214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:59.305742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115e674-b8e9-4f57-b515-2eb42bfdb3bd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:59.306908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115e674-b8e9-4f57-b515-2eb42bfdb3bd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:59.307289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115e674-b8e9-4f57-b515-2eb42bfdb3bd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:59.307661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115e674-b8e9-4f57-b515-2eb42bfdb3bd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/8f2c75dae4124736aabe554fe3ba7d9f/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:59.313691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115e674-b8e9-4f57-b515-2eb42bfdb3bd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=f5765fae0b04466d9bcf450f38daefcb, user_id=116af5d85ed04da6ba2b2bb1007aaef9, role_id=8f2c75dae4124736aabe554fe3ba7d9f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:59.316525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115e674-b8e9-4f57-b515-2eb42bfdb3bd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:59.348857 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2657/5303] 10.222.0.16 () {60 vars in 1579 bytes} [Sat Oct 15 21:10:59 2022] PUT /identity/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/8f2c75dae4124736aabe554fe3ba7d9f/inherited_to_projects => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:10:59.352090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '46b5964105bc4469ab876d72a935c181'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.352979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115e674-b8e9-4f57-b515-2eb42bfdb3bd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:59.353708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '46b5964105bc4469ab876d72a935c181'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.410832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '46b5964105bc4469ab876d72a935c181'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.412062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '46b5964105bc4469ab876d72a935c181'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.417533 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2647/5304] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:10:59 2022] PATCH /identity/v3/domains/46b5964105bc4469ab876d72a935c181 => generated 277 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:59.422257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:59.424290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:59.495303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115e674-b8e9-4f57-b515-2eb42bfdb3bd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:59.556692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0d80c-57b5-446b-8b4b-58f511c191bd tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:59.563180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:59.563952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:59.564367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:59.564729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/domains/46b5964105bc4469ab876d72a935c181` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:59.578587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=46b5964105bc4469ab876d72a935c181)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:59.579691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fc645-be1d-43f8-b0b2-d055d18fcf0d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:59.580696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fc645-be1d-43f8-b0b2-d055d18fcf0d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:59.580833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:59.583052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fc645-be1d-43f8-b0b2-d055d18fcf0d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:59.583582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fc645-be1d-43f8-b0b2-d055d18fcf0d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:59.586395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fc645-be1d-43f8-b0b2-d055d18fcf0d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=f5765fae0b04466d9bcf450f38daefcb, user_id=116af5d85ed04da6ba2b2bb1007aaef9)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:59.589484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fc645-be1d-43f8-b0b2-d055d18fcf0d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:59.607525 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2658/5305] 10.222.0.16 () {58 vars in 1461 bytes} [Sat Oct 15 21:10:59 2022] GET /identity/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/inherited_to_projects => generated 446 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:10:59.611630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '46b5964105bc4469ab876d72a935c181'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.611924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fc645-be1d-43f8-b0b2-d055d18fcf0d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:59.614221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3c5fc645-be1d-43f8-b0b2-d055d18fcf0d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:59.623992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '46b5964105bc4469ab876d72a935c181'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.628758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fc645-be1d-43f8-b0b2-d055d18fcf0d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:59.636548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26694ee6-afaf-4834-a7fa-8613605a0fad tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:59.638306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694ee6-afaf-4834-a7fa-8613605a0fad tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:59.638729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694ee6-afaf-4834-a7fa-8613605a0fad tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:59.639187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694ee6-afaf-4834-a7fa-8613605a0fad tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/8f2c75dae4124736aabe554fe3ba7d9f/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:59.641992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26694ee6-afaf-4834-a7fa-8613605a0fad tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=f5765fae0b04466d9bcf450f38daefcb, user_id=116af5d85ed04da6ba2b2bb1007aaef9, role_id=8f2c75dae4124736aabe554fe3ba7d9f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:59.645845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26694ee6-afaf-4834-a7fa-8613605a0fad tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:59.658631 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2659/5306] 10.222.0.16 () {58 vars in 1561 bytes} [Sat Oct 15 21:10:59 2022] HEAD /identity/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/8f2c75dae4124736aabe554fe3ba7d9f/inherited_to_projects => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:59.660805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26694ee6-afaf-4834-a7fa-8613605a0fad tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:59.677756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26694ee6-afaf-4834-a7fa-8613605a0fad tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:59.685212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:59.685980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:59.686440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:59.686784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/8f2c75dae4124736aabe554fe3ba7d9f/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:59.690171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=f5765fae0b04466d9bcf450f38daefcb, user_id=116af5d85ed04da6ba2b2bb1007aaef9, role_id=8f2c75dae4124736aabe554fe3ba7d9f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:59.694792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:59.701802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role 8f2c75dae4124736aabe554fe3ba7d9f was removed from user 116af5d85ed04da6ba2b2bb1007aaef9 on domain f5765fae0b04466d9bcf450f38daefcb. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:59.702169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.702959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.706398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:59.713171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46b5964105bc4469ab876d72a935c181', 'request_id': 'req-5f11964d-6899-4497-bce1-35e771a1de60'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.728126 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2660/5307] 10.222.0.16 () {58 vars in 1563 bytes} [Sat Oct 15 21:10:59 2022] DELETE /identity/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/8f2c75dae4124736aabe554fe3ba7d9f/inherited_to_projects => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:59.732686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:59.733987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46b5964105bc4469ab876d72a935c181', 'request_id': 'req-5f11964d-6899-4497-bce1-35e771a1de60'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.734993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:59.744083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46b5964105bc4469ab876d72a935c181', 'request_id': 'req-5f11964d-6899-4497-bce1-35e771a1de60'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.767096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] The token cache is being invalidate because project 46b5964105bc4469ab876d72a935c181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:59.768367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.769326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.777266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:UPDATE=1,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:59.791126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '46b5964105bc4469ab876d72a935c181', 'request_id': 'req-5f11964d-6899-4497-bce1-35e771a1de60'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.791986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '46b5964105bc4469ab876d72a935c181', 'request_id': 'req-5f11964d-6899-4497-bce1-35e771a1de60'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.795574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '46b5964105bc4469ab876d72a935c181', 'request_id': 'req-5f11964d-6899-4497-bce1-35e771a1de60'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.809061 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2648/5308] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:10:59 2022] DELETE /identity/v3/domains/46b5964105bc4469ab876d72a935c181 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:59.814462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:59.816086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:59.889565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e5183-f402-4050-90c9-856b6413d9c2 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:10:59.908851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:10:59.910411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:10:59.910927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:10:59.911414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles/8f2c75dae4124736aabe554fe3ba7d9f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:10:59.913897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8f2c75dae4124736aabe554fe3ba7d9f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:10:59.916298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:10:59.976415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role 8f2c75dae4124736aabe554fe3ba7d9f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:10:59.976803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.978167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:10:59.982029 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2661/5309] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:59 2022] DELETE /identity/v3/roles/8f2c75dae4124736aabe554fe3ba7d9f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:10:59.987444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:10:59.990469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:10:59.999030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11964d-6899-4497-bce1-35e771a1de60 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:00.103421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88f98e-64d3-4efa-b7b2-5f4e2770ff70 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:00.124629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ceb3-ad04-4bf7-be96-0719050e6b94 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:00.125775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ceb3-ad04-4bf7-be96-0719050e6b94 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:00.126209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ceb3-ad04-4bf7-be96-0719050e6b94 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:00.126653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ceb3-ad04-4bf7-be96-0719050e6b94 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:00.127778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613ceb3-ad04-4bf7-be96-0719050e6b94 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:00.129611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613ceb3-ad04-4bf7-be96-0719050e6b94 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:00.134509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:00.135094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:00.135371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:00.135633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/e085cf0d1c31436fb78e1a41eca71b39` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:00.138295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_role(role_id=e085cf0d1c31436fb78e1a41eca71b39)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:00.144131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:00.200694 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2662/5310] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:10:59 2022] POST /identity/v3/roles => generated 240 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:00.215701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ceb3-ad04-4bf7-be96-0719050e6b94 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:00.234625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ceb3-ad04-4bf7-be96-0719050e6b94 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:00.257845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role e085cf0d1c31436fb78e1a41eca71b39 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:00.258182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:00.261173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:00.266419 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2649/5311] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:10:59 2022] DELETE /identity/v3/roles/e085cf0d1c31436fb78e1a41eca71b39 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:00.268244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc5389-8b56-4161-bd6b-37532b3e2ffd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:00.268869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc5389-8b56-4161-bd6b-37532b3e2ffd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:00.269201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc5389-8b56-4161-bd6b-37532b3e2ffd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:00.269475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc5389-8b56-4161-bd6b-37532b3e2ffd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:00.270127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc5389-8b56-4161-bd6b-37532b3e2ffd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:00.272404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:00.279474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc5389-8b56-4161-bd6b-37532b3e2ffd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:00.282302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:00.353382 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2663/5312] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:00 2022] POST /identity/v3/projects => generated 409 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:00.358030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc5389-8b56-4161-bd6b-37532b3e2ffd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:00.367295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b1bc5389-8b56-4161-bd6b-37532b3e2ffd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:00.396413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:00.401457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37761772-d481-471f-9ad5-4c1b19424e56 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:00.437450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:00.507766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:00.508435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:00.508708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:00.508972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/4fb9d9a02ac04634bbc8bf4eaa9188ce` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:00.511196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4fb9d9a02ac04634bbc8bf4eaa9188ce)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:00.512783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:00.561240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role 4fb9d9a02ac04634bbc8bf4eaa9188ce has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:00.561515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:00.562182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:00.564784 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2650/5313] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:00 2022] DELETE /identity/v3/roles/4fb9d9a02ac04634bbc8bf4eaa9188ce => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:00.565583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:00.568193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:00.574787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc5389-8b56-4161-bd6b-37532b3e2ffd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:00.703563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b8b1c-8d27-4600-84b1-a3630367762a tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:00.709434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269e73c-bd38-456c-ae24-632c48159c49 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db0b24a87eac48beb5293d777077e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c60521f47ad4003b2c6507621c0d448', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:00.710964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269e73c-bd38-456c-ae24-632c48159c49 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:00.711242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269e73c-bd38-456c-ae24-632c48159c49 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:00.711507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269e73c-bd38-456c-ae24-632c48159c49 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:00.719457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269e73c-bd38-456c-ae24-632c48159c49 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:00.719866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269e73c-bd38-456c-ae24-632c48159c49 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:00.721867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269e73c-bd38-456c-ae24-632c48159c49 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:00.752166 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2651/5314] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:00.752166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269e73c-bd38-456c-ae24-632c48159c49 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:00.755751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6269e73c-bd38-456c-ae24-632c48159c49 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:00.790811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac82d7-8119-4f25-9ee5-e1b50074bfcf tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:00.791851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac82d7-8119-4f25-9ee5-e1b50074bfcf tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:00.792124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac82d7-8119-4f25-9ee5-e1b50074bfcf tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:00.792407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac82d7-8119-4f25-9ee5-e1b50074bfcf tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/37ead7a102804589b8319ed8f0444e64/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:00.797573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac82d7-8119-4f25-9ee5-e1b50074bfcf tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=f5765fae0b04466d9bcf450f38daefcb, user_id=116af5d85ed04da6ba2b2bb1007aaef9, role_id=37ead7a102804589b8319ed8f0444e64)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:00.801519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac82d7-8119-4f25-9ee5-e1b50074bfcf tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:00.823557 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2664/5315] 10.222.0.16 () {60 vars in 1579 bytes} [Sat Oct 15 21:11:00 2022] PUT /identity/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/37ead7a102804589b8319ed8f0444e64/inherited_to_projects => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:00.827025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac82d7-8119-4f25-9ee5-e1b50074bfcf tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:00.970712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac82d7-8119-4f25-9ee5-e1b50074bfcf tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:00.971141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269e73c-bd38-456c-ae24-632c48159c49 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:00.979271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda3bf-6454-47a9-9f01-88df3fbbf097 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db0b24a87eac48beb5293d777077e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c60521f47ad4003b2c6507621c0d448', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:00.980201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda3bf-6454-47a9-9f01-88df3fbbf097 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:00.980636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda3bf-6454-47a9-9f01-88df3fbbf097 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:00.980914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda3bf-6454-47a9-9f01-88df3fbbf097 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:00.981708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbda3bf-6454-47a9-9f01-88df3fbbf097 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:00.985953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbda3bf-6454-47a9-9f01-88df3fbbf097 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:00.990426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:01.052834 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2665/5316] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:00 2022] POST /identity/v3/projects => generated 349 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:01.057004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda3bf-6454-47a9-9f01-88df3fbbf097 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:01.059620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6fbda3bf-6454-47a9-9f01-88df3fbbf097 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:01.150225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd5e0fd5b6a456e879af06396442556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5228810b5b2a41249b055cbcd0705515', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:01.150952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:01.151354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:01.151628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] PATH_INFO: `/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:01.153677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorizing `identity:delete_role(role_id=6e3634c6ea4d4c6eb96fe49f863b4e4e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:01.155289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:01.161128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:01.191195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda3bf-6454-47a9-9f01-88df3fbbf097 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:01.216820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab539a-0229-4e34-8d0b-2767f564de28 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:01.217489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab539a-0229-4e34-8d0b-2767f564de28 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:01.217769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab539a-0229-4e34-8d0b-2767f564de28 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:01.218041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab539a-0229-4e34-8d0b-2767f564de28 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:01.219250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab539a-0229-4e34-8d0b-2767f564de28 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Adding query filter params (scope.project.id=7d5517e8193c409a8d77d52269b338f2, user.id=116af5d85ed04da6ba2b2bb1007aaef9) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:01.219571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab539a-0229-4e34-8d0b-2767f564de28 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:01.221616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab539a-0229-4e34-8d0b-2767f564de28 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:01.252025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invalidating the token cache because role 6e3634c6ea4d4c6eb96fe49f863b4e4e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:01.254639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:01.256469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:01.261550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2652/5317] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:00 2022] DELETE /identity/v3/roles/6e3634c6ea4d4c6eb96fe49f863b4e4e => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:01.262855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:01.271182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:01.341364 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2666/5318] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Oct 15 21:11:01 2022] GET /identity/v3/role_assignments?scope.project.id=7d5517e8193c409a8d77d52269b338f2&user.id=116af5d85ed04da6ba2b2bb1007aaef9&effective => generated 648 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:01.342096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab539a-0229-4e34-8d0b-2767f564de28 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:01.344104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eeab539a-0229-4e34-8d0b-2767f564de28 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:01.471511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab539a-0229-4e34-8d0b-2767f564de28 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:01.499656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb2aed-e50f-4d6c-879a-a3eb20ebf068 tempest-RolesV3TestJSON-1882656179 tempest-RolesV3TestJSON-1882656179-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:01.507012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab08ad-f4ee-4a74-a5e4-e9fefb7f2e56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:01.508227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab08ad-f4ee-4a74-a5e4-e9fefb7f2e56 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:01.508915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab08ad-f4ee-4a74-a5e4-e9fefb7f2e56 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:01.509381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab08ad-f4ee-4a74-a5e4-e9fefb7f2e56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:01.515330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c2ab08ad-f4ee-4a74-a5e4-e9fefb7f2e56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:01.550978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab08ad-f4ee-4a74-a5e4-e9fefb7f2e56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:01.555137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab08ad-f4ee-4a74-a5e4-e9fefb7f2e56 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:01.568160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:01.569436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:01.570893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:01.570893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] PATH_INFO: `/v3/projects/4c1dd434c4d749b48b131f48655cd76f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:01.572374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1dd434c4d749b48b131f48655cd76f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:01.579175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:01.585309 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2653/5319] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:01.586189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab08ad-f4ee-4a74-a5e4-e9fefb7f2e56 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:01.594065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c2ab08ad-f4ee-4a74-a5e4-e9fefb7f2e56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:01.677195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1dd434c4d749b48b131f48655cd76f', 'request_id': 'req-d1850431-2dfc-438a-b1bb-46fb706531b7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:01.693446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1dd434c4d749b48b131f48655cd76f', 'request_id': 'req-d1850431-2dfc-438a-b1bb-46fb706531b7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:01.697654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:01.711276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1dd434c4d749b48b131f48655cd76f', 'request_id': 'req-d1850431-2dfc-438a-b1bb-46fb706531b7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:01.724008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab08ad-f4ee-4a74-a5e4-e9fefb7f2e56 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:01.734438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] The token cache is being invalidate because project 4c1dd434c4d749b48b131f48655cd76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:01.734735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:01.735468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:01.738372 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2667/5320] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:01 2022] DELETE /identity/v3/projects/4c1dd434c4d749b48b131f48655cd76f => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:01.740219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:01.743210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:01.769164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:01.876883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1850431-2dfc-438a-b1bb-46fb706531b7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:01.889543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:01.890318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:01.890722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:01.891762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] PATH_INFO: `/v3/users/dec8cd5eb5674682aad90fd42db118e8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:01.894071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dec8cd5eb5674682aad90fd42db118e8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:01.896941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:01.904747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767a107-0e5d-4656-a368-b3610199f09d tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db0b24a87eac48beb5293d777077e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c60521f47ad4003b2c6507621c0d448', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:01.905578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767a107-0e5d-4656-a368-b3610199f09d tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:01.906068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767a107-0e5d-4656-a368-b3610199f09d tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:01.906521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767a107-0e5d-4656-a368-b3610199f09d tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:01.907272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767a107-0e5d-4656-a368-b3610199f09d tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:01.908254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767a107-0e5d-4656-a368-b3610199f09d tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:01.912009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767a107-0e5d-4656-a368-b3610199f09d tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:01.930787 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2654/5321] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:01.931884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767a107-0e5d-4656-a368-b3610199f09d tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:01.937935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a767a107-0e5d-4656-a368-b3610199f09d tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:01.997830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dec8cd5eb5674682aad90fd42db118e8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:02.016018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dec8cd5eb5674682aad90fd42db118e8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:02.030946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dec8cd5eb5674682aad90fd42db118e8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:02.051295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dec8cd5eb5674682aad90fd42db118e8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:02.054162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-39fc66fc-1928-4acf-bb9f-ca694b31c64a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dec8cd5eb5674682aad90fd42db118e8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:02.070381 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2668/5322] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:01 2022] DELETE /identity/v3/users/dec8cd5eb5674682aad90fd42db118e8 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:02.070888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d19e2-22c8-4672-b6a7-a9743f96ff2d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:02.072816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d19e2-22c8-4672-b6a7-a9743f96ff2d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:02.073121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d19e2-22c8-4672-b6a7-a9743f96ff2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:02.109243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767a107-0e5d-4656-a368-b3610199f09d tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:02.125726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-373d19e2-22c8-4672-b6a7-a9743f96ff2d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:02.127312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-373d19e2-22c8-4672-b6a7-a9743f96ff2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:02.216451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec91d8c-6a8b-47db-a5ef-6de6c53d7ca9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:02.217114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec91d8c-6a8b-47db-a5ef-6de6c53d7ca9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:02.218784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec91d8c-6a8b-47db-a5ef-6de6c53d7ca9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:02.225108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec91d8c-6a8b-47db-a5ef-6de6c53d7ca9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:02.226363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec91d8c-6a8b-47db-a5ef-6de6c53d7ca9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Adding query filter params (scope.project.id=8718b2948ce449fb8a8858a39e1a135c, user.id=116af5d85ed04da6ba2b2bb1007aaef9) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:02.226688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec91d8c-6a8b-47db-a5ef-6de6c53d7ca9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:02.229133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec91d8c-6a8b-47db-a5ef-6de6c53d7ca9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:02.296386 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2669/5323] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:02.297965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a18571-6606-46ba-9778-f32d98635b39 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:02.298481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a18571-6606-46ba-9778-f32d98635b39 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:02.298825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a18571-6606-46ba-9778-f32d98635b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:02.323214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:02.332169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-23a18571-6606-46ba-9778-f32d98635b39 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:02.333449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-23a18571-6606-46ba-9778-f32d98635b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:02.353155 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2670/5324] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:02.358776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee3614-ba0f-4004-9365-3c82aa8fb41d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:02.359268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee3614-ba0f-4004-9365-3c82aa8fb41d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:02.359750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee3614-ba0f-4004-9365-3c82aa8fb41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:02.375074 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2655/5325] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Oct 15 21:11:01 2022] GET /identity/v3/role_assignments?scope.project.id=8718b2948ce449fb8a8858a39e1a135c&user.id=116af5d85ed04da6ba2b2bb1007aaef9&effective => generated 648 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:02.376521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec91d8c-6a8b-47db-a5ef-6de6c53d7ca9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:02.379586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2ec91d8c-6a8b-47db-a5ef-6de6c53d7ca9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:02.395941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c0ee3614-ba0f-4004-9365-3c82aa8fb41d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:02.396718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:02.398314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c0ee3614-ba0f-4004-9365-3c82aa8fb41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:02.416388 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2671/5326] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:02.417719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee3614-ba0f-4004-9365-3c82aa8fb41d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:02.420342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c0ee3614-ba0f-4004-9365-3c82aa8fb41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:02.436907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee3614-ba0f-4004-9365-3c82aa8fb41d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:02.443430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:02.444884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:02.445409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:02.445695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/37ead7a102804589b8319ed8f0444e64/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:02.448456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=f5765fae0b04466d9bcf450f38daefcb, user_id=116af5d85ed04da6ba2b2bb1007aaef9, role_id=37ead7a102804589b8319ed8f0444e64)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:02.451316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:02.457654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role 37ead7a102804589b8319ed8f0444e64 was removed from user 116af5d85ed04da6ba2b2bb1007aaef9 on domain f5765fae0b04466d9bcf450f38daefcb. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:02.458017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:02.458831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:02.477187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec91d8c-6a8b-47db-a5ef-6de6c53d7ca9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:02.481154 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2672/5327] 10.222.0.16 () {58 vars in 1563 bytes} [Sat Oct 15 21:11:02 2022] DELETE /identity/v3/OS-INHERIT/domains/f5765fae0b04466d9bcf450f38daefcb/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/37ead7a102804589b8319ed8f0444e64/inherited_to_projects => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:02.483486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:02.488115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:02.559271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e6449-3e8a-47bc-bc41-9415518ba4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:02.559981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e6449-3e8a-47bc-bc41-9415518ba4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:02.560311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e6449-3e8a-47bc-bc41-9415518ba4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:02.561310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e6449-3e8a-47bc-bc41-9415518ba4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:02.564165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8a1e6449-3e8a-47bc-bc41-9415518ba4c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:02.598119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac38ad-115b-4a49-ada1-ff7647e8b4b5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:02.604159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68250b-eaf7-4aa7-adb2-ce7d0330fb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:02.604899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68250b-eaf7-4aa7-adb2-ce7d0330fb58 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:02.605307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68250b-eaf7-4aa7-adb2-ce7d0330fb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:02.605818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68250b-eaf7-4aa7-adb2-ce7d0330fb58 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:02.606705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68250b-eaf7-4aa7-adb2-ce7d0330fb58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:02.607110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68250b-eaf7-4aa7-adb2-ce7d0330fb58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:02.609582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68250b-eaf7-4aa7-adb2-ce7d0330fb58 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:02.625586 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2673/5328] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:02.630671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68250b-eaf7-4aa7-adb2-ce7d0330fb58 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:02.635499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8c68250b-eaf7-4aa7-adb2-ce7d0330fb58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:02.651682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e6449-3e8a-47bc-bc41-9415518ba4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:02.653716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e6449-3e8a-47bc-bc41-9415518ba4c8 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:02.667341 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2656/5329] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:11:02.668457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e6449-3e8a-47bc-bc41-9415518ba4c8 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:02.690263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e6449-3e8a-47bc-bc41-9415518ba4c8 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:02.697264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8337cc9-4713-4828-bba7-5aa92ef18784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:02.697880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8337cc9-4713-4828-bba7-5aa92ef18784 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:02.698155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8337cc9-4713-4828-bba7-5aa92ef18784 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:02.698853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8337cc9-4713-4828-bba7-5aa92ef18784 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:02.699488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8337cc9-4713-4828-bba7-5aa92ef18784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:02.702157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8337cc9-4713-4828-bba7-5aa92ef18784 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:02.747468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68250b-eaf7-4aa7-adb2-ce7d0330fb58 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:02.755247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5385a-9aed-43c2-aac3-efde2f748979 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:02.758538 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2657/5330] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:02 2022] POST /identity/v3/projects => generated 397 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:02.758819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5385a-9aed-43c2-aac3-efde2f748979 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:02.759283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8337cc9-4713-4828-bba7-5aa92ef18784 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:02.759582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5385a-9aed-43c2-aac3-efde2f748979 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:02.760380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5385a-9aed-43c2-aac3-efde2f748979 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:02.761786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5385a-9aed-43c2-aac3-efde2f748979 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Adding query filter params (scope.project.id=7d5517e8193c409a8d77d52269b338f2, user.id=116af5d85ed04da6ba2b2bb1007aaef9) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:02.762336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a8337cc9-4713-4828-bba7-5aa92ef18784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:02.763183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5385a-9aed-43c2-aac3-efde2f748979 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:02.772742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5385a-9aed-43c2-aac3-efde2f748979 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:02.853179 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2674/5331] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Oct 15 21:11:02 2022] GET /identity/v3/role_assignments?scope.project.id=7d5517e8193c409a8d77d52269b338f2&user.id=116af5d85ed04da6ba2b2bb1007aaef9&effective => generated 228 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:02.854668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5385a-9aed-43c2-aac3-efde2f748979 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:02.861943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:02.877430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5385a-9aed-43c2-aac3-efde2f748979 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:02.898184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8337cc9-4713-4828-bba7-5aa92ef18784 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:02.914084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db0b24a87eac48beb5293d777077e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c60521f47ad4003b2c6507621c0d448', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:02.915600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:02.915882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:02.916428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] PATH_INFO: `/v3/projects/65fbf0b1784041b5a3c28801ff2f4467` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:02.918413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: Authorizing `identity:delete_project(project_id=65fbf0b1784041b5a3c28801ff2f4467)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:02.920966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:02.945731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:02.986381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65fbf0b1784041b5a3c28801ff2f4467', 'request_id': 'req-4ca714f3-1fd4-4798-8198-bfff01577a77'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.000230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65fbf0b1784041b5a3c28801ff2f4467', 'request_id': 'req-4ca714f3-1fd4-4798-8198-bfff01577a77'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.006170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95540e88-9491-4005-9af3-6440160546df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:03.006822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95540e88-9491-4005-9af3-6440160546df admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:03.007145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95540e88-9491-4005-9af3-6440160546df admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:03.007469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95540e88-9491-4005-9af3-6440160546df admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:03.009596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95540e88-9491-4005-9af3-6440160546df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:03.013157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95540e88-9491-4005-9af3-6440160546df admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:03.024465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65fbf0b1784041b5a3c28801ff2f4467', 'request_id': 'req-4ca714f3-1fd4-4798-8198-bfff01577a77'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.057675 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2675/5332] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:02 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:03.057675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95540e88-9491-4005-9af3-6440160546df admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:03.071852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] The token cache is being invalidate because project 65fbf0b1784041b5a3c28801ff2f4467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:03.072219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.077420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.079964 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2658/5333] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:02 2022] DELETE /identity/v3/projects/65fbf0b1784041b5a3c28801ff2f4467 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:03.087967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:03.098289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:03.211269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95540e88-9491-4005-9af3-6440160546df admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:03.251092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca714f3-1fd4-4798-8198-bfff01577a77 tempest-RoutersAdminTest-1494689308 tempest-RoutersAdminTest-1494689308-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:03.263340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753bf1d-09fc-427f-97cd-2440da13bd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:03.263955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753bf1d-09fc-427f-97cd-2440da13bd1c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:03.264255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753bf1d-09fc-427f-97cd-2440da13bd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:03.264634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753bf1d-09fc-427f-97cd-2440da13bd1c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:03.265176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753bf1d-09fc-427f-97cd-2440da13bd1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:03.266791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753bf1d-09fc-427f-97cd-2440da13bd1c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:03.287358 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2659/5334] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:03 2022] GET /identity/v3/roles => generated 1651 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:03.288826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753bf1d-09fc-427f-97cd-2440da13bd1c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:03.302004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e753bf1d-09fc-427f-97cd-2440da13bd1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:03.406111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bc3a9-0a85-46ca-a4e9-73b2cd470e6e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:03.407657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bc3a9-0a85-46ca-a4e9-73b2cd470e6e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:03.409673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bc3a9-0a85-46ca-a4e9-73b2cd470e6e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:03.410164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bc3a9-0a85-46ca-a4e9-73b2cd470e6e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:03.411225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bc3a9-0a85-46ca-a4e9-73b2cd470e6e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Adding query filter params (scope.project.id=8718b2948ce449fb8a8858a39e1a135c, user.id=116af5d85ed04da6ba2b2bb1007aaef9) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:03.412042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bc3a9-0a85-46ca-a4e9-73b2cd470e6e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:03.415513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bc3a9-0a85-46ca-a4e9-73b2cd470e6e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:03.431555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753bf1d-09fc-427f-97cd-2440da13bd1c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:03.436898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e7970-cb1c-405e-ab4f-8ef895e3908b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:03.437576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e7970-cb1c-405e-ab4f-8ef895e3908b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:03.437852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e7970-cb1c-405e-ab4f-8ef895e3908b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:03.438121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e7970-cb1c-405e-ab4f-8ef895e3908b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:03.440223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-db6e7970-cb1c-405e-ab4f-8ef895e3908b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:03.505323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:03.530078 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2676/5335] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Oct 15 21:11:03 2022] GET /identity/v3/role_assignments?scope.project.id=8718b2948ce449fb8a8858a39e1a135c&user.id=116af5d85ed04da6ba2b2bb1007aaef9&effective => generated 228 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:03.532030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bc3a9-0a85-46ca-a4e9-73b2cd470e6e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:03.566790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bc3a9-0a85-46ca-a4e9-73b2cd470e6e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:03.588512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8be9f1-5600-4878-988d-0f56f048ea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:03.589220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8be9f1-5600-4878-988d-0f56f048ea31 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:03.589509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8be9f1-5600-4878-988d-0f56f048ea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:03.590300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8be9f1-5600-4878-988d-0f56f048ea31 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:03.590447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8be9f1-5600-4878-988d-0f56f048ea31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:03.598036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8be9f1-5600-4878-988d-0f56f048ea31 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:03.604591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e7970-cb1c-405e-ab4f-8ef895e3908b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:03.606070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e7970-cb1c-405e-ab4f-8ef895e3908b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:03.629820 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2677/5336] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:03 2022] GET /identity/v3/roles => generated 1651 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:03.630813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8be9f1-5600-4878-988d-0f56f048ea31 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:03.642186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7b8be9f1-5600-4878-988d-0f56f048ea31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:03.646100 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2660/5337] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:03.651319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e7970-cb1c-405e-ab4f-8ef895e3908b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:03.654458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-db6e7970-cb1c-405e-ab4f-8ef895e3908b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:03.676015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8be9f1-5600-4878-988d-0f56f048ea31 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:03.691362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:03.692904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e7970-cb1c-405e-ab4f-8ef895e3908b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:03.694139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:03.694462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:03.694739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] PATH_INFO: `/v3/projects/eb7796fef9524c4598dd52fb450ad036` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:03.696994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb7796fef9524c4598dd52fb450ad036)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:03.702927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:03.708614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:03.711055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:03.711356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:03.711607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/projects/8718b2948ce449fb8a8858a39e1a135c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:03.716780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8718b2948ce449fb8a8858a39e1a135c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:03.719322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:03.749981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:03.869819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb7796fef9524c4598dd52fb450ad036', 'request_id': 'req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.885071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8718b2948ce449fb8a8858a39e1a135c', 'request_id': 'req-7db0a185-e84c-48ac-bd78-156e61c2c99f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.892887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb7796fef9524c4598dd52fb450ad036', 'request_id': 'req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.907544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb7796fef9524c4598dd52fb450ad036', 'request_id': 'req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.915518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8718b2948ce449fb8a8858a39e1a135c', 'request_id': 'req-7db0a185-e84c-48ac-bd78-156e61c2c99f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.926794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8718b2948ce449fb8a8858a39e1a135c', 'request_id': 'req-7db0a185-e84c-48ac-bd78-156e61c2c99f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.955595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] The token cache is being invalidate because project eb7796fef9524c4598dd52fb450ad036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:03.957559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.958720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.966453 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2678/5338] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:03 2022] DELETE /identity/v3/projects/eb7796fef9524c4598dd52fb450ad036 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:03.968109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:03.970880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] The token cache is being invalidate because project 8718b2948ce449fb8a8858a39e1a135c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:03.972722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:03.974159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.975481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:03.983256 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2661/5339] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:03 2022] DELETE /identity/v3/projects/8718b2948ce449fb8a8858a39e1a135c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:03.984131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:03.989465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:04.138638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ea2a-a127-4ee8-ae2a-686ee8643712 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:04.188447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0a185-e84c-48ac-bd78-156e61c2c99f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:04.212964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:04.214456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:04.214823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:04.214988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] PATH_INFO: `/v3/users/7bd5e0fd5b6a456e879af06396442556` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:04.222291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd5e0fd5b6a456e879af06396442556)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:04.237553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:04.311922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51f1a-79e4-47eb-816b-74b2b044f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:04.313778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51f1a-79e4-47eb-816b-74b2b044f5a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:04.315660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51f1a-79e4-47eb-816b-74b2b044f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:04.316393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51f1a-79e4-47eb-816b-74b2b044f5a7 admin admin] PATH_INFO: `/v3/projects/1b381f47153c4515b05b26989824c9df/users/0081064aa9b5484cb8048b675750dfef/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:04.355033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f51f1a-79e4-47eb-816b-74b2b044f5a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b381f47153c4515b05b26989824c9df, user_id=0081064aa9b5484cb8048b675750dfef, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:04.360448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f51f1a-79e4-47eb-816b-74b2b044f5a7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:04.384008 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2679/5340] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:03 2022] PUT /identity/v3/projects/1b381f47153c4515b05b26989824c9df/users/0081064aa9b5484cb8048b675750dfef/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:04.385449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51f1a-79e4-47eb-816b-74b2b044f5a7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:04.406720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-71f51f1a-79e4-47eb-816b-74b2b044f5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:04.427324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd5e0fd5b6a456e879af06396442556'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:04.447025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd5e0fd5b6a456e879af06396442556'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:04.448443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:04.484581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd5e0fd5b6a456e879af06396442556'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:04.503022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd5e0fd5b6a456e879af06396442556'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:04.503836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-355187b9-4774-4680-b07f-d4b0f829a20c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd5e0fd5b6a456e879af06396442556'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:04.514870 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2662/5341] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:03 2022] DELETE /identity/v3/users/7bd5e0fd5b6a456e879af06396442556 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:04.517775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53660f44-612d-47db-8156-1c67a05f65b1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:04.518095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53660f44-612d-47db-8156-1c67a05f65b1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:04.518363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53660f44-612d-47db-8156-1c67a05f65b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:04.522889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:04.568174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-53660f44-612d-47db-8156-1c67a05f65b1 None None] MFA Rules not processed for user `0081064aa9b5484cb8048b675750dfef`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:04.570115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-53660f44-612d-47db-8156-1c67a05f65b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:04.589100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51f1a-79e4-47eb-816b-74b2b044f5a7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:04.673723 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2663/5342] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:04.675104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:04.676036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:04.676408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53660f44-612d-47db-8156-1c67a05f65b1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:04.676545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:04.676872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles/37ead7a102804589b8319ed8f0444e64` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:04.679483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-53660f44-612d-47db-8156-1c67a05f65b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:04.679705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_role(role_id=37ead7a102804589b8319ed8f0444e64)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:04.681537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:04.756369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role 37ead7a102804589b8319ed8f0444e64 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:04.756930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:04.757779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:04.762861 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2680/5343] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:04 2022] DELETE /identity/v3/roles/37ead7a102804589b8319ed8f0444e64 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:04.767950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:04.770140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:04.853217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53660f44-612d-47db-8156-1c67a05f65b1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:04.921977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a21550-d728-48fc-9cc1-e05f2d3eaac4 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:04.927706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bebfd-7ecc-49e3-bec3-8ca84b2468b9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:04.929476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bebfd-7ecc-49e3-bec3-8ca84b2468b9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:04.930044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bebfd-7ecc-49e3-bec3-8ca84b2468b9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:04.930327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bebfd-7ecc-49e3-bec3-8ca84b2468b9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:04.931011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bebfd-7ecc-49e3-bec3-8ca84b2468b9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:04.932896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bebfd-7ecc-49e3-bec3-8ca84b2468b9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:04.962617 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2681/5344] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:11:04 2022] POST /identity/v3/roles => generated 239 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:04.966232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bebfd-7ecc-49e3-bec3-8ca84b2468b9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:04.974564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39736c-84ad-49fe-baf4-b7e29b0e1b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:04.976696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39736c-84ad-49fe-baf4-b7e29b0e1b3a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:04.977389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39736c-84ad-49fe-baf4-b7e29b0e1b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:04.977676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39736c-84ad-49fe-baf4-b7e29b0e1b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:04.978461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39736c-84ad-49fe-baf4-b7e29b0e1b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:04.980959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39736c-84ad-49fe-baf4-b7e29b0e1b3a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:04.995449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bebfd-7ecc-49e3-bec3-8ca84b2468b9 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:05.003336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:05.008306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402054a1-43cd-4dc6-b724-af93fe975ef5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:05.011500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402054a1-43cd-4dc6-b724-af93fe975ef5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:05.012516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402054a1-43cd-4dc6-b724-af93fe975ef5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:05.013016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402054a1-43cd-4dc6-b724-af93fe975ef5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:05.014997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402054a1-43cd-4dc6-b724-af93fe975ef5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:05.024586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402054a1-43cd-4dc6-b724-af93fe975ef5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:05.033838 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2664/5345] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:04 2022] POST /identity/v3/projects => generated 397 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:05.034986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39736c-84ad-49fe-baf4-b7e29b0e1b3a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:05.039519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4a39736c-84ad-49fe-baf4-b7e29b0e1b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:05.108514 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2682/5346] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:04 2022] POST /identity/v3/projects => generated 408 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:05.113730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402054a1-43cd-4dc6-b724-af93fe975ef5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:05.134695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:05.218985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39736c-84ad-49fe-baf4-b7e29b0e1b3a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:05.282732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402054a1-43cd-4dc6-b724-af93fe975ef5 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:05.297020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e855d4-1d80-4a74-8d3c-e1b25f52f39d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:05.298817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e855d4-1d80-4a74-8d3c-e1b25f52f39d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:05.300161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e855d4-1d80-4a74-8d3c-e1b25f52f39d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:05.300443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e855d4-1d80-4a74-8d3c-e1b25f52f39d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/0986658977524ac1b92e518dc6daa8c5/inherited_to_projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:05.305834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e855d4-1d80-4a74-8d3c-e1b25f52f39d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7d5517e8193c409a8d77d52269b338f2, user_id=116af5d85ed04da6ba2b2bb1007aaef9, role_id=0986658977524ac1b92e518dc6daa8c5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:05.314076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e855d4-1d80-4a74-8d3c-e1b25f52f39d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:05.345999 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2683/5347] 10.222.0.16 () {60 vars in 1582 bytes} [Sat Oct 15 21:11:05 2022] PUT /identity/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/0986658977524ac1b92e518dc6daa8c5/inherited_to_projects => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:05.353276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e855d4-1d80-4a74-8d3c-e1b25f52f39d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:05.373015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13722e-596c-4af2-b935-bceac18c2740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:05.375044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13722e-596c-4af2-b935-bceac18c2740 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:05.375942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13722e-596c-4af2-b935-bceac18c2740 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:05.376610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13722e-596c-4af2-b935-bceac18c2740 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:05.377409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13722e-596c-4af2-b935-bceac18c2740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:05.380647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13722e-596c-4af2-b935-bceac18c2740 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:05.436488 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2665/5348] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:05 2022] POST /identity/v3/users => generated 305 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:05.438725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13722e-596c-4af2-b935-bceac18c2740 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:05.470995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13722e-596c-4af2-b935-bceac18c2740 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:05.532314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e855d4-1d80-4a74-8d3c-e1b25f52f39d tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:05.544531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b198f6f-6f8b-40c2-8728-7bb2cccaa90e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:05.545174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b198f6f-6f8b-40c2-8728-7bb2cccaa90e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:05.545456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b198f6f-6f8b-40c2-8728-7bb2cccaa90e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:05.545727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b198f6f-6f8b-40c2-8728-7bb2cccaa90e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:05.550986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b198f6f-6f8b-40c2-8728-7bb2cccaa90e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Adding query filter params (scope.project.id=15f06e06e18c4d54bf12d43cc3d15d0b, user.id=116af5d85ed04da6ba2b2bb1007aaef9) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:05.552416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b198f6f-6f8b-40c2-8728-7bb2cccaa90e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:05.557879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b198f6f-6f8b-40c2-8728-7bb2cccaa90e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:05.604942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c9cbe-c10c-42a2-90b1-66f05048d8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:05.607553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c9cbe-c10c-42a2-90b1-66f05048d8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:05.608320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c9cbe-c10c-42a2-90b1-66f05048d8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:05.608815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c9cbe-c10c-42a2-90b1-66f05048d8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:05.609896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c9cbe-c10c-42a2-90b1-66f05048d8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:05.613693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c9cbe-c10c-42a2-90b1-66f05048d8d0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:05.628681 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2666/5349] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:05 2022] GET /identity/v3/roles => generated 1650 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:05.633276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c9cbe-c10c-42a2-90b1-66f05048d8d0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:05.649018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c9cbe-c10c-42a2-90b1-66f05048d8d0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:05.660737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d59368-2291-4207-a4ae-3d78fe85fbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:05.665148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d59368-2291-4207-a4ae-3d78fe85fbb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:05.665671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d59368-2291-4207-a4ae-3d78fe85fbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:05.667875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d59368-2291-4207-a4ae-3d78fe85fbb8 admin admin] PATH_INFO: `/v3/projects/6d22e36542a64de18e6445ee19bd9990/users/582d27fd5cd447e5ad9d50b06cd0aca8/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:05.731702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d59368-2291-4207-a4ae-3d78fe85fbb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d22e36542a64de18e6445ee19bd9990, user_id=582d27fd5cd447e5ad9d50b06cd0aca8, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:05.734968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d59368-2291-4207-a4ae-3d78fe85fbb8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:05.739194 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2684/5350] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Oct 15 21:11:05 2022] GET /identity/v3/role_assignments?scope.project.id=15f06e06e18c4d54bf12d43cc3d15d0b&user.id=116af5d85ed04da6ba2b2bb1007aaef9&effective => generated 649 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:05.741065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b198f6f-6f8b-40c2-8728-7bb2cccaa90e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:05.745706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5b198f6f-6f8b-40c2-8728-7bb2cccaa90e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:05.761934 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2667/5351] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:05 2022] PUT /identity/v3/projects/6d22e36542a64de18e6445ee19bd9990/users/582d27fd5cd447e5ad9d50b06cd0aca8/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:05.761934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d59368-2291-4207-a4ae-3d78fe85fbb8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:05.769397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:05.855685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:05.872226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d59368-2291-4207-a4ae-3d78fe85fbb8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:05.878205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:05.878897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:05.879179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:05.879459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/0986658977524ac1b92e518dc6daa8c5/inherited_to_projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:05.882600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=7d5517e8193c409a8d77d52269b338f2, user_id=116af5d85ed04da6ba2b2bb1007aaef9, role_id=0986658977524ac1b92e518dc6daa8c5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:05.886059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:05.892995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role 0986658977524ac1b92e518dc6daa8c5 was removed from user 116af5d85ed04da6ba2b2bb1007aaef9 on project 7d5517e8193c409a8d77d52269b338f2. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:05.892995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:05.893694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:05.920884 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2668/5352] 10.222.0.16 () {58 vars in 1566 bytes} [Sat Oct 15 21:11:05 2022] DELETE /identity/v3/OS-INHERIT/projects/7d5517e8193c409a8d77d52269b338f2/users/116af5d85ed04da6ba2b2bb1007aaef9/roles/0986658977524ac1b92e518dc6daa8c5/inherited_to_projects => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:05.921137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b198f6f-6f8b-40c2-8728-7bb2cccaa90e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:05.921248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:05.923708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:06.069239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e058202-be11-4cb7-be61-40ced99bee19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:06.071018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e058202-be11-4cb7-be61-40ced99bee19 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:06.071495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e058202-be11-4cb7-be61-40ced99bee19 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:06.072075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e058202-be11-4cb7-be61-40ced99bee19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:06.075388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4e058202-be11-4cb7-be61-40ced99bee19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:06.097768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e058202-be11-4cb7-be61-40ced99bee19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:06.111702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e058202-be11-4cb7-be61-40ced99bee19 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:06.112905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b015e-777c-40cc-9ad4-ece1171df7cd tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:06.122654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddd4b9-4584-40f2-9381-b62d35c07a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:06.123479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddd4b9-4584-40f2-9381-b62d35c07a28 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:06.124103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddd4b9-4584-40f2-9381-b62d35c07a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:06.124482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddd4b9-4584-40f2-9381-b62d35c07a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:06.125257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddd4b9-4584-40f2-9381-b62d35c07a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:06.127378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddd4b9-4584-40f2-9381-b62d35c07a28 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:06.143451 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2669/5353] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:05 2022] GET /identity/v3/roles => generated 1650 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:06.144854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddd4b9-4584-40f2-9381-b62d35c07a28 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:06.147252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-faddd4b9-4584-40f2-9381-b62d35c07a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:06.149474 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2685/5354] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:06.150963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e058202-be11-4cb7-be61-40ced99bee19 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:06.164883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e058202-be11-4cb7-be61-40ced99bee19 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:06.170892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a90a6-77f5-4886-91e9-113a4be77617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:06.172131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a90a6-77f5-4886-91e9-113a4be77617 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:06.172564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a90a6-77f5-4886-91e9-113a4be77617 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:06.173274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a90a6-77f5-4886-91e9-113a4be77617 admin admin] PATH_INFO: `/v3/projects/6d22e36542a64de18e6445ee19bd9990/users/582d27fd5cd447e5ad9d50b06cd0aca8/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:06.176128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a90a6-77f5-4886-91e9-113a4be77617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d22e36542a64de18e6445ee19bd9990, user_id=582d27fd5cd447e5ad9d50b06cd0aca8, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:06.182130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a90a6-77f5-4886-91e9-113a4be77617 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:06.202257 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2686/5355] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:06 2022] PUT /identity/v3/projects/6d22e36542a64de18e6445ee19bd9990/users/582d27fd5cd447e5ad9d50b06cd0aca8/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:06.212614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ac443-d167-422f-b682-0a4348461121 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:06.213323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ac443-d167-422f-b682-0a4348461121 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:06.213894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ac443-d167-422f-b682-0a4348461121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:06.272753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fa8ac443-d167-422f-b682-0a4348461121 None None] MFA Rules not processed for user `582d27fd5cd447e5ad9d50b06cd0aca8`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:06.274580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fa8ac443-d167-422f-b682-0a4348461121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:06.356891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddd4b9-4584-40f2-9381-b62d35c07a28 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:06.418527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:06.435782 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2687/5356] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:11:06.436693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ac443-d167-422f-b682-0a4348461121 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:06.438290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fa8ac443-d167-422f-b682-0a4348461121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:06.452877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2afd0-9982-44f3-b1f4-e087f1715ebb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:06.453609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2afd0-9982-44f3-b1f4-e087f1715ebb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:06.453895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2afd0-9982-44f3-b1f4-e087f1715ebb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:06.454075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2afd0-9982-44f3-b1f4-e087f1715ebb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:06.454794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2afd0-9982-44f3-b1f4-e087f1715ebb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Adding query filter params (scope.project.id=15f06e06e18c4d54bf12d43cc3d15d0b, user.id=116af5d85ed04da6ba2b2bb1007aaef9) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:06.455105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2afd0-9982-44f3-b1f4-e087f1715ebb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:06.457140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2afd0-9982-44f3-b1f4-e087f1715ebb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:06.513905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:06.544402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ac443-d167-422f-b682-0a4348461121 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:06.550866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:06.552562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:06.552562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:06.552562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] PATH_INFO: `/v3/projects/5228810b5b2a41249b055cbcd0705515` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:06.553690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5228810b5b2a41249b055cbcd0705515)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:06.555886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:06.573785 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2670/5357] 10.222.0.16 () {58 vars in 1356 bytes} [Sat Oct 15 21:11:06 2022] GET /identity/v3/role_assignments?scope.project.id=15f06e06e18c4d54bf12d43cc3d15d0b&user.id=116af5d85ed04da6ba2b2bb1007aaef9&effective => generated 228 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:06.573785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2afd0-9982-44f3-b1f4-e087f1715ebb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:06.578674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-53e2afd0-9982-44f3-b1f4-e087f1715ebb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:06.674400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5228810b5b2a41249b055cbcd0705515', 'request_id': 'req-6a1edb9f-deed-4b86-80a2-a5af6729be6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:06.686179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5228810b5b2a41249b055cbcd0705515', 'request_id': 'req-6a1edb9f-deed-4b86-80a2-a5af6729be6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:06.701711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5228810b5b2a41249b055cbcd0705515', 'request_id': 'req-6a1edb9f-deed-4b86-80a2-a5af6729be6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:06.724393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] The token cache is being invalidate because project 5228810b5b2a41249b055cbcd0705515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:06.724670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:06.728635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:06.735026 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2688/5358] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:06 2022] DELETE /identity/v3/projects/5228810b5b2a41249b055cbcd0705515 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:06.736070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:06.737748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2afd0-9982-44f3-b1f4-e087f1715ebb tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:06.739805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:06.847629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ecdfd-4fd8-465c-a02b-106cc7f3859c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:06.851568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ecdfd-4fd8-465c-a02b-106cc7f3859c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:06.852263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ecdfd-4fd8-465c-a02b-106cc7f3859c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:06.853396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ecdfd-4fd8-465c-a02b-106cc7f3859c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:06.856626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1edb9f-deed-4b86-80a2-a5af6729be6b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:06.861458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7e7ecdfd-4fd8-465c-a02b-106cc7f3859c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:06.862348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:06.863136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:06.863674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:06.863953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/projects/15f06e06e18c4d54bf12d43cc3d15d0b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:06.865192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_project(project_id=15f06e06e18c4d54bf12d43cc3d15d0b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:06.867385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:07.001500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15f06e06e18c4d54bf12d43cc3d15d0b', 'request_id': 'req-240c902d-f2ba-4799-9981-dcab1d84d17f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.016273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15f06e06e18c4d54bf12d43cc3d15d0b', 'request_id': 'req-240c902d-f2ba-4799-9981-dcab1d84d17f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.026178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ecdfd-4fd8-465c-a02b-106cc7f3859c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:07.027917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ecdfd-4fd8-465c-a02b-106cc7f3859c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:07.029129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15f06e06e18c4d54bf12d43cc3d15d0b', 'request_id': 'req-240c902d-f2ba-4799-9981-dcab1d84d17f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.050001 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2671/5359] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:06 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:07.052340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab55ad6-d0b7-4bd5-ad14-86a24a90d0d9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.053161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] The token cache is being invalidate because project 15f06e06e18c4d54bf12d43cc3d15d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:07.053965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.054263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab55ad6-d0b7-4bd5-ad14-86a24a90d0d9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.054702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.055136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab55ad6-d0b7-4bd5-ad14-86a24a90d0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.058688 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2689/5360] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:06 2022] DELETE /identity/v3/projects/15f06e06e18c4d54bf12d43cc3d15d0b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:07.062737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:07.067106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:07.077458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:07.092089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:07.098590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0ab55ad6-d0b7-4bd5-ad14-86a24a90d0d9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:07.100027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0ab55ad6-d0b7-4bd5-ad14-86a24a90d0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:07.213451 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2672/5361] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:07.215543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d2954-2e84-4acf-b642-1aeb233ff59f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.215833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d2954-2e84-4acf-b642-1aeb233ff59f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.216153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d2954-2e84-4acf-b642-1aeb233ff59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.249891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c902d-f2ba-4799-9981-dcab1d84d17f tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:07.262232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f36d2954-2e84-4acf-b642-1aeb233ff59f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:07.267097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f36d2954-2e84-4acf-b642-1aeb233ff59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:07.273863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:07.274524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.275348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.275629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/roles/0986658977524ac1b92e518dc6daa8c5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.279677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0986658977524ac1b92e518dc6daa8c5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:07.281744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:07.285281 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2673/5362] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:07.290738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d2954-2e84-4acf-b642-1aeb233ff59f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:07.304447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d2954-2e84-4acf-b642-1aeb233ff59f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:07.311292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf7593-946a-45cc-9515-b7fe7d5e9fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:07.312076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf7593-946a-45cc-9515-b7fe7d5e9fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.312446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf7593-946a-45cc-9515-b7fe7d5e9fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.312844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf7593-946a-45cc-9515-b7fe7d5e9fa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.313695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf7593-946a-45cc-9515-b7fe7d5e9fa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:07.314227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf7593-946a-45cc-9515-b7fe7d5e9fa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:07.317125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf7593-946a-45cc-9515-b7fe7d5e9fa3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:07.331598 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2674/5363] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:07.336178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf7593-946a-45cc-9515-b7fe7d5e9fa3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:07.348788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf7593-946a-45cc-9515-b7fe7d5e9fa3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:07.355254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb978a81-4fe6-4a03-9bdd-18429e90407f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:07.356158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb978a81-4fe6-4a03-9bdd-18429e90407f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.356538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb978a81-4fe6-4a03-9bdd-18429e90407f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.356944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb978a81-4fe6-4a03-9bdd-18429e90407f admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.357912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb978a81-4fe6-4a03-9bdd-18429e90407f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:07.360230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb978a81-4fe6-4a03-9bdd-18429e90407f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:07.376852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invalidating the token cache because role 0986658977524ac1b92e518dc6daa8c5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:07.377319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.378020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.379938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2690/5364] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:07 2022] DELETE /identity/v3/roles/0986658977524ac1b92e518dc6daa8c5 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:07.386261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:07.388730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:07.441483 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2675/5365] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:07 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:07.442634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb978a81-4fe6-4a03-9bdd-18429e90407f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:07.451121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bb978a81-4fe6-4a03-9bdd-18429e90407f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:07.492823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfa38b-fd4d-4589-adc7-40267b5fada7 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:07.550029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb978a81-4fe6-4a03-9bdd-18429e90407f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:07.555723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecebbb-992c-4307-af09-d9aa070fa7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:07.556481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecebbb-992c-4307-af09-d9aa070fa7ee admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.557547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecebbb-992c-4307-af09-d9aa070fa7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.557547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecebbb-992c-4307-af09-d9aa070fa7ee admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.558093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eecebbb-992c-4307-af09-d9aa070fa7ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:07.560890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eecebbb-992c-4307-af09-d9aa070fa7ee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:07.593885 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2676/5366] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:07 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:07.595362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecebbb-992c-4307-af09-d9aa070fa7ee admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:07.603068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:07.604650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.605100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.605483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/users/116af5d85ed04da6ba2b2bb1007aaef9` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.607566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_user(user_id=116af5d85ed04da6ba2b2bb1007aaef9)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:07.608903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecebbb-992c-4307-af09-d9aa070fa7ee admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:07.610877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:07.616190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f2efd-2b8d-48f2-a0e6-393831d8af47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:07.617636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f2efd-2b8d-48f2-a0e6-393831d8af47 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.618100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f2efd-2b8d-48f2-a0e6-393831d8af47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.619805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f2efd-2b8d-48f2-a0e6-393831d8af47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.622847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f2efd-2b8d-48f2-a0e6-393831d8af47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:07.624852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f2efd-2b8d-48f2-a0e6-393831d8af47 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:07.644616 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2677/5367] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:11:07.649027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f2efd-2b8d-48f2-a0e6-393831d8af47 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:07.662696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:07.669663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f2efd-2b8d-48f2-a0e6-393831d8af47 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:07.683182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49a039-f8a9-4d77-9082-c98ca81f072d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:07.684092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49a039-f8a9-4d77-9082-c98ca81f072d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.686184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49a039-f8a9-4d77-9082-c98ca81f072d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.687884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49a039-f8a9-4d77-9082-c98ca81f072d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.688851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49a039-f8a9-4d77-9082-c98ca81f072d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:07.691176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49a039-f8a9-4d77-9082-c98ca81f072d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:07.704275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '116af5d85ed04da6ba2b2bb1007aaef9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.707770 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2678/5368] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:11:07.711417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49a039-f8a9-4d77-9082-c98ca81f072d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:07.728471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49a039-f8a9-4d77-9082-c98ca81f072d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:07.736301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf86a3-d72b-4680-82d7-cfb91302adec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:07.737378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf86a3-d72b-4680-82d7-cfb91302adec admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.738045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf86a3-d72b-4680-82d7-cfb91302adec admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.738878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf86a3-d72b-4680-82d7-cfb91302adec admin admin] PATH_INFO: `/v3/projects/f48303fa2cb94246a8bc8abe3c3fc58d/users/0902a3cd5c044bd0af8a9a5436299c85/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.742564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '116af5d85ed04da6ba2b2bb1007aaef9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.765417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '116af5d85ed04da6ba2b2bb1007aaef9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.772550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf86a3-d72b-4680-82d7-cfb91302adec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48303fa2cb94246a8bc8abe3c3fc58d, user_id=0902a3cd5c044bd0af8a9a5436299c85, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:07.775746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf86a3-d72b-4680-82d7-cfb91302adec admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:07.776348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '116af5d85ed04da6ba2b2bb1007aaef9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.777099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '116af5d85ed04da6ba2b2bb1007aaef9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:07.785939 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2691/5369] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:07 2022] DELETE /identity/v3/users/116af5d85ed04da6ba2b2bb1007aaef9 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:07.791204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:07.792657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:07.795336 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2679/5370] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:07 2022] PUT /identity/v3/projects/f48303fa2cb94246a8bc8abe3c3fc58d/users/0902a3cd5c044bd0af8a9a5436299c85/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:07.808112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fbfde-4ef3-4faf-bd40-c6cd2b5efbd5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:07.808637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fbfde-4ef3-4faf-bd40-c6cd2b5efbd5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:07.809267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fbfde-4ef3-4faf-bd40-c6cd2b5efbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:07.852167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:07.896743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0f6fbfde-4ef3-4faf-bd40-c6cd2b5efbd5 None None] MFA Rules not processed for user `0902a3cd5c044bd0af8a9a5436299c85`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:07.901390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0f6fbfde-4ef3-4faf-bd40-c6cd2b5efbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:07.908201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47421bd0-3589-4f72-ad34-c26ccb48dc8e tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:08.058552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470c185-5259-4451-b59a-01498fd5d472 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:08.060056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470c185-5259-4451-b59a-01498fd5d472 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:08.063623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470c185-5259-4451-b59a-01498fd5d472 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:08.064039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470c185-5259-4451-b59a-01498fd5d472 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/groups/cd4e7ec7048d42d0aa0cac5adebdde51` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:08.066536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470c185-5259-4451-b59a-01498fd5d472 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_group(group_id=cd4e7ec7048d42d0aa0cac5adebdde51)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:08.070802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470c185-5259-4451-b59a-01498fd5d472 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:08.084869 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2680/5371] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:08.089297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fbfde-4ef3-4faf-bd40-c6cd2b5efbd5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:08.093184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0f6fbfde-4ef3-4faf-bd40-c6cd2b5efbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:08.177260 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2692/5372] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:11:07 2022] DELETE /identity/v3/groups/cd4e7ec7048d42d0aa0cac5adebdde51 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:08.179500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470c185-5259-4451-b59a-01498fd5d472 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:08.193049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:08.262750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fbfde-4ef3-4faf-bd40-c6cd2b5efbd5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:08.309647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470c185-5259-4451-b59a-01498fd5d472 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:08.320867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:08.321828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:08.324461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:08.325306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/projects/7d5517e8193c409a8d77d52269b338f2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:08.327225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7d5517e8193c409a8d77d52269b338f2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:08.332206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:08.347499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb2e97-f630-4c60-9abb-5fdc3ecc9e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:08.348132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2e97-f630-4c60-9abb-5fdc3ecc9e28 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:08.348413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2e97-f630-4c60-9abb-5fdc3ecc9e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:08.348683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2e97-f630-4c60-9abb-5fdc3ecc9e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:08.349407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb2e97-f630-4c60-9abb-5fdc3ecc9e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:08.352585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb2e97-f630-4c60-9abb-5fdc3ecc9e28 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:08.401224 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2681/5373] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:08 2022] POST /identity/v3/projects => generated 365 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:08.405493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb2e97-f630-4c60-9abb-5fdc3ecc9e28 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:08.413356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5517e8193c409a8d77d52269b338f2', 'request_id': 'req-a0d79787-73cb-43ae-89bf-ad196e041d48'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:08.425613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5517e8193c409a8d77d52269b338f2', 'request_id': 'req-a0d79787-73cb-43ae-89bf-ad196e041d48'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:08.435134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5517e8193c409a8d77d52269b338f2', 'request_id': 'req-a0d79787-73cb-43ae-89bf-ad196e041d48'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:08.463889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] The token cache is being invalidate because project 7d5517e8193c409a8d77d52269b338f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:08.464341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:08.465729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:08.467837 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2693/5374] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:08 2022] DELETE /identity/v3/projects/7d5517e8193c409a8d77d52269b338f2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:08.468157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:08.471882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:08.476228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:08.540398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb2e97-f630-4c60-9abb-5fdc3ecc9e28 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:08.553921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19242e44-79e1-4ea3-a592-cbd65d66a467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:08.554592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19242e44-79e1-4ea3-a592-cbd65d66a467 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:08.554840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19242e44-79e1-4ea3-a592-cbd65d66a467 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:08.555065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19242e44-79e1-4ea3-a592-cbd65d66a467 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:08.556109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19242e44-79e1-4ea3-a592-cbd65d66a467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:08.558115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19242e44-79e1-4ea3-a592-cbd65d66a467 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:08.603360 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2682/5375] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:08 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:08.604911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19242e44-79e1-4ea3-a592-cbd65d66a467 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:08.608084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-19242e44-79e1-4ea3-a592-cbd65d66a467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:08.614037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d79787-73cb-43ae-89bf-ad196e041d48 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:08.686347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:08.716269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:08.717262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:08.717763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:08.718339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/domains/f5765fae0b04466d9bcf450f38daefcb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:08.720108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f5765fae0b04466d9bcf450f38daefcb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:08.722282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:08.746402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19242e44-79e1-4ea3-a592-cbd65d66a467 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:08.755805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f55a0c-1a3d-4d82-bb91-5b839f34a02f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:08.756964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:08.757509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f55a0c-1a3d-4d82-bb91-5b839f34a02f service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:08.757805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f55a0c-1a3d-4d82-bb91-5b839f34a02f service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:08.758776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:08.758928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f55a0c-1a3d-4d82-bb91-5b839f34a02f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:08.762590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-18f55a0c-1a3d-4d82-bb91-5b839f34a02f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:08.794987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:08.795778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:08.803185 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2694/5376] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:11:08 2022] PATCH /identity/v3/domains/f5765fae0b04466d9bcf450f38daefcb => generated 277 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:11:08.804447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:08.805881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:08.886712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f55a0c-1a3d-4d82-bb91-5b839f34a02f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:08.889266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f55a0c-1a3d-4d82-bb91-5b839f34a02f service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:08.894156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85d41d-dc76-4b85-985f-b07361445a17 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:08.901780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f235cd-bb25-4164-a7cb-01f4226a89c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:08.902445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f235cd-bb25-4164-a7cb-01f4226a89c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:08.902670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f235cd-bb25-4164-a7cb-01f4226a89c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:08.903076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f235cd-bb25-4164-a7cb-01f4226a89c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:08.903976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f235cd-bb25-4164-a7cb-01f4226a89c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:08.905806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f235cd-bb25-4164-a7cb-01f4226a89c8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:08.906067 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2683/5377] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:08 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:08.907517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f55a0c-1a3d-4d82-bb91-5b839f34a02f service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:08.909606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-18f55a0c-1a3d-4d82-bb91-5b839f34a02f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:08.923792 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2695/5378] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:08.925225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f235cd-bb25-4164-a7cb-01f4226a89c8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:08.932263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f55a0c-1a3d-4d82-bb91-5b839f34a02f service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:08.937687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f235cd-bb25-4164-a7cb-01f4226a89c8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:08.949719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e8d74-e67b-4936-acdf-6071101d5e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:08.951548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e8d74-e67b-4936-acdf-6071101d5e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:08.951548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e8d74-e67b-4936-acdf-6071101d5e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:08.951829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e8d74-e67b-4936-acdf-6071101d5e6c admin admin] PATH_INFO: `/v3/projects/dba4298b957040af9a1c9b2ac6cd9ff7/users/a87f4de367854e459c8f438edb5abb4a/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:08.994844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e8d74-e67b-4936-acdf-6071101d5e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba4298b957040af9a1c9b2ac6cd9ff7, user_id=a87f4de367854e459c8f438edb5abb4a, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:08.994844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e8d74-e67b-4936-acdf-6071101d5e6c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:09.033160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:09.058045 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2696/5379] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:08 2022] PUT /identity/v3/projects/dba4298b957040af9a1c9b2ac6cd9ff7/users/a87f4de367854e459c8f438edb5abb4a/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:09.062322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e8d74-e67b-4936-acdf-6071101d5e6c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:09.078483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d6b8c0746d4793b55171f48bd9b3eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841626c2473f42d394e2a1f988cc4994', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:09.079591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:09.080920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:09.081769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] PATH_INFO: `/v3/domains/f5765fae0b04466d9bcf450f38daefcb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:09.110535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f5765fae0b04466d9bcf450f38daefcb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:09.114149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:09.166238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.180876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.206916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e8d74-e67b-4936-acdf-6071101d5e6c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:09.212945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fcdf3-5733-4e55-9898-9148571482e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:09.213588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fcdf3-5733-4e55-9898-9148571482e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:09.213871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fcdf3-5733-4e55-9898-9148571482e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:09.214140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fcdf3-5733-4e55-9898-9148571482e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:09.214991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fcdf3-5733-4e55-9898-9148571482e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:09.216645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fcdf3-5733-4e55-9898-9148571482e9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:09.246936 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2697/5380] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:09.248566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fcdf3-5733-4e55-9898-9148571482e9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:09.267385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fcdf3-5733-4e55-9898-9148571482e9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:09.292184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb', 'request_id': 'req-52d7bd1b-62ee-481b-8fed-494331178615'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.297691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:09.308372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb', 'request_id': 'req-52d7bd1b-62ee-481b-8fed-494331178615'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.324311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb', 'request_id': 'req-52d7bd1b-62ee-481b-8fed-494331178615'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.355991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] The token cache is being invalidate because project f5765fae0b04466d9bcf450f38daefcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:09.356645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.359802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.373697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb', 'request_id': 'req-52d7bd1b-62ee-481b-8fed-494331178615'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.374745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb', 'request_id': 'req-52d7bd1b-62ee-481b-8fed-494331178615'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.375675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f5765fae0b04466d9bcf450f38daefcb', 'request_id': 'req-52d7bd1b-62ee-481b-8fed-494331178615'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.376040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ef354-869a-41fa-b60f-6d7246a3841b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:09.376799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ef354-869a-41fa-b60f-6d7246a3841b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:09.377243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ef354-869a-41fa-b60f-6d7246a3841b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:09.377524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ef354-869a-41fa-b60f-6d7246a3841b admin admin] PATH_INFO: `/v3/projects/dba4298b957040af9a1c9b2ac6cd9ff7/users/a87f4de367854e459c8f438edb5abb4a/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:09.383887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ef354-869a-41fa-b60f-6d7246a3841b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba4298b957040af9a1c9b2ac6cd9ff7, user_id=a87f4de367854e459c8f438edb5abb4a, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:09.394029 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2684/5381] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:11:08 2022] DELETE /identity/v3/domains/f5765fae0b04466d9bcf450f38daefcb => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:09.397952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:09.400032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:09.403229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ef354-869a-41fa-b60f-6d7246a3841b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:09.433400 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2698/5382] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:09 2022] PUT /identity/v3/projects/dba4298b957040af9a1c9b2ac6cd9ff7/users/a87f4de367854e459c8f438edb5abb4a/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:09.434295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7bd1b-62ee-481b-8fed-494331178615 tempest-InheritsV3TestJSON-1268603275 tempest-InheritsV3TestJSON-1268603275-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:09.444237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a856afc-6dd2-43b1-98ce-c2f06336cd2e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:09.444555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a856afc-6dd2-43b1-98ce-c2f06336cd2e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:09.444779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a856afc-6dd2-43b1-98ce-c2f06336cd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:09.518369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6a856afc-6dd2-43b1-98ce-c2f06336cd2e None None] MFA Rules not processed for user `a87f4de367854e459c8f438edb5abb4a`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:09.520712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6a856afc-6dd2-43b1-98ce-c2f06336cd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:09.539898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:09.541144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:09.541758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:09.542228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] PATH_INFO: `/v3/users/75aea18e943d4417a54b2eac95f0ab98` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:09.543657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75aea18e943d4417a54b2eac95f0ab98)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:09.546916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:09.614263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:09.651716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75aea18e943d4417a54b2eac95f0ab98'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.662996 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2699/5383] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:09.664931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a856afc-6dd2-43b1-98ce-c2f06336cd2e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:09.667181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75aea18e943d4417a54b2eac95f0ab98'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.682894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a856afc-6dd2-43b1-98ce-c2f06336cd2e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:09.689177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75aea18e943d4417a54b2eac95f0ab98'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.692108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5b17d-9f52-46cd-ba4f-e700f19f2e4a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:09.693099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5b17d-9f52-46cd-ba4f-e700f19f2e4a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:09.694277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5b17d-9f52-46cd-ba4f-e700f19f2e4a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:09.694808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5b17d-9f52-46cd-ba4f-e700f19f2e4a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:09.696179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5b17d-9f52-46cd-ba4f-e700f19f2e4a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:09.699186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5b17d-9f52-46cd-ba4f-e700f19f2e4a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:09.700952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75aea18e943d4417a54b2eac95f0ab98'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.702742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6e4d9b66-a997-4f58-8ad7-f8e01b527221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75aea18e943d4417a54b2eac95f0ab98'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:09.713428 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2685/5384] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:09 2022] DELETE /identity/v3/users/75aea18e943d4417a54b2eac95f0ab98 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:09.717924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ff41e-6fa1-4928-bf94-078479373e59 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:09.718306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ff41e-6fa1-4928-bf94-078479373e59 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:09.718649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ff41e-6fa1-4928-bf94-078479373e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:09.745153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-967ff41e-6fa1-4928-bf94-078479373e59 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:09.746469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-967ff41e-6fa1-4928-bf94-078479373e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:09.763723 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2700/5385] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:09 2022] POST /identity/v3/projects => generated 359 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:09.766842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5b17d-9f52-46cd-ba4f-e700f19f2e4a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:09.768901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e5b5b17d-9f52-46cd-ba4f-e700f19f2e4a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:09.850741 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2686/5386] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:09.860743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ff41e-6fa1-4928-bf94-078479373e59 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:09.862178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-967ff41e-6fa1-4928-bf94-078479373e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:09.877528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5b17d-9f52-46cd-ba4f-e700f19f2e4a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:09.883512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75263ac2-fd5c-49b1-89ed-a826b2dc378d tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:09.884111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75263ac2-fd5c-49b1-89ed-a826b2dc378d tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:09.884391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75263ac2-fd5c-49b1-89ed-a826b2dc378d tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:09.884815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75263ac2-fd5c-49b1-89ed-a826b2dc378d tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:09.885963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75263ac2-fd5c-49b1-89ed-a826b2dc378d tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:09.887535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75263ac2-fd5c-49b1-89ed-a826b2dc378d tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:09.921916 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2701/5387] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:09 2022] POST /identity/v3/users => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:09.923116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75263ac2-fd5c-49b1-89ed-a826b2dc378d tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:09.930541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:09.935594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75263ac2-fd5c-49b1-89ed-a826b2dc378d tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:09.945010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ff41e-6fa1-4928-bf94-078479373e59 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:09.945544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17606ed5-c5c7-4953-85d7-6ed766cafa27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:09.946195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606ed5-c5c7-4953-85d7-6ed766cafa27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:09.947176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606ed5-c5c7-4953-85d7-6ed766cafa27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:09.947176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606ed5-c5c7-4953-85d7-6ed766cafa27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/21b6ece30d0b4a99858102a95d43675e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:09.963772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17606ed5-c5c7-4953-85d7-6ed766cafa27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:get_user(user_id=21b6ece30d0b4a99858102a95d43675e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:09.966362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17606ed5-c5c7-4953-85d7-6ed766cafa27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:09.968272 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2702/5388] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:11:09 2022] GET /identity/v3/users/21b6ece30d0b4a99858102a95d43675e => generated 373 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:09.972231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17606ed5-c5c7-4953-85d7-6ed766cafa27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:09.985272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17606ed5-c5c7-4953-85d7-6ed766cafa27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:09.992946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:09.993681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:09.994634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:09.994634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/21b6ece30d0b4a99858102a95d43675e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:09.995620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:delete_user(user_id=21b6ece30d0b4a99858102a95d43675e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:09.998658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:10.009064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0473f44-f1dc-4761-ba25-803772cc271b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:10.009969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0473f44-f1dc-4761-ba25-803772cc271b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:10.010238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0473f44-f1dc-4761-ba25-803772cc271b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:10.010510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0473f44-f1dc-4761-ba25-803772cc271b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:10.032684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:10.051598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21b6ece30d0b4a99858102a95d43675e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.068217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21b6ece30d0b4a99858102a95d43675e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.079773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21b6ece30d0b4a99858102a95d43675e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.087481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0473f44-f1dc-4761-ba25-803772cc271b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:10.088412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21b6ece30d0b4a99858102a95d43675e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.089132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0473f44-f1dc-4761-ba25-803772cc271b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:10.089421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21b6ece30d0b4a99858102a95d43675e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.097099 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2703/5389] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:09 2022] DELETE /identity/v3/users/21b6ece30d0b4a99858102a95d43675e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:10.105081 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2687/5390] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:10.109946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:10.111790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:10.200087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d82bfc-6e58-405a-9678-981416565c35 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:10.210109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:10.210885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:10.211293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:10.211658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects/ca34ef04119c40ed8d3b4f265bf71168` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:10.213184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ca34ef04119c40ed8d3b4f265bf71168)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:10.215200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:10.292498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca34ef04119c40ed8d3b4f265bf71168', 'request_id': 'req-43a0c327-ff2a-4f4a-beda-bed4349d500a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.338333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca34ef04119c40ed8d3b4f265bf71168', 'request_id': 'req-43a0c327-ff2a-4f4a-beda-bed4349d500a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.351513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca34ef04119c40ed8d3b4f265bf71168', 'request_id': 'req-43a0c327-ff2a-4f4a-beda-bed4349d500a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.358095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:10.376055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] The token cache is being invalidate because project ca34ef04119c40ed8d3b4f265bf71168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:10.376930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.380833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.388136 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2704/5391] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:10 2022] DELETE /identity/v3/projects/ca34ef04119c40ed8d3b4f265bf71168 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:10.390192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0473f44-f1dc-4761-ba25-803772cc271b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:10.401706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0473f44-f1dc-4761-ba25-803772cc271b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:10.458781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:10.461181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:10.510266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e2b9-b127-4806-8980-518638abb03b tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:10.510883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e2b9-b127-4806-8980-518638abb03b tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:10.511205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e2b9-b127-4806-8980-518638abb03b tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:10.511566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e2b9-b127-4806-8980-518638abb03b tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:10.512443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181e2b9-b127-4806-8980-518638abb03b tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:10.514080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181e2b9-b127-4806-8980-518638abb03b tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:10.585519 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2688/5392] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:10 2022] POST /identity/v3/projects => generated 359 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:10.587556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e2b9-b127-4806-8980-518638abb03b tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:10.590161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8181e2b9-b127-4806-8980-518638abb03b tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:10.663552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c327-ff2a-4f4a-beda-bed4349d500a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:10.733966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e2b9-b127-4806-8980-518638abb03b tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:10.742222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84a367-50d4-49b9-abb7-d0dfe8004780 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:10.742904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a367-50d4-49b9-abb7-d0dfe8004780 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:10.743192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a367-50d4-49b9-abb7-d0dfe8004780 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:10.743467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a367-50d4-49b9-abb7-d0dfe8004780 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:10.744286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84a367-50d4-49b9-abb7-d0dfe8004780 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:10.753981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84a367-50d4-49b9-abb7-d0dfe8004780 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:10.806192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:10.807037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:10.807411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:10.812220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] PATH_INFO: `/v3/projects/5360eb36b9c64153975f4a2e8c315feb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:10.814543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5360eb36b9c64153975f4a2e8c315feb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:10.816687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:10.837738 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2689/5393] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:10 2022] POST /identity/v3/users => generated 417 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:10.841773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84a367-50d4-49b9-abb7-d0dfe8004780 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:10.871907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84a367-50d4-49b9-abb7-d0dfe8004780 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:10.891212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ac36e-1e47-458c-a2a6-905573d0caf1 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:10.891858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ac36e-1e47-458c-a2a6-905573d0caf1 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:10.892140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ac36e-1e47-458c-a2a6-905573d0caf1 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:10.892416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ac36e-1e47-458c-a2a6-905573d0caf1 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:10.893124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ac36e-1e47-458c-a2a6-905573d0caf1 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:10.897691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ac36e-1e47-458c-a2a6-905573d0caf1 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:10.931806 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2690/5394] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:11:10 2022] POST /identity/v3/roles => generated 238 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:10.935758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ac36e-1e47-458c-a2a6-905573d0caf1 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:10.961564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5360eb36b9c64153975f4a2e8c315feb', 'request_id': 'req-f32f107d-bd95-4bdd-955e-4deec29c0764'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.977800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ac36e-1e47-458c-a2a6-905573d0caf1 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:10.978893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5360eb36b9c64153975f4a2e8c315feb', 'request_id': 'req-f32f107d-bd95-4bdd-955e-4deec29c0764'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:10.988079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5360eb36b9c64153975f4a2e8c315feb', 'request_id': 'req-f32f107d-bd95-4bdd-955e-4deec29c0764'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:11.001180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:11.012289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] The token cache is being invalidate because project 5360eb36b9c64153975f4a2e8c315feb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:11.012579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:11.013300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:11.017997 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2705/5395] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:10 2022] DELETE /identity/v3/projects/5360eb36b9c64153975f4a2e8c315feb => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:11.018900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:11.022318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:11.081214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d5f76-8387-453e-ac42-86481a7f16bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:11.082345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d5f76-8387-453e-ac42-86481a7f16bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:11.082729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d5f76-8387-453e-ac42-86481a7f16bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:11.083267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d5f76-8387-453e-ac42-86481a7f16bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/1d73ba9e79f14e828e70219e28d63e3d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:11.098336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:11.177014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f107d-bd95-4bdd-955e-4deec29c0764 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:11.217200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:11.217868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:11.218140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:11.218406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] PATH_INFO: `/v3/users/97d6b8c0746d4793b55171f48bd9b3eb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:11.222859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97d6b8c0746d4793b55171f48bd9b3eb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:11.225424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:11.325873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d5f76-8387-453e-ac42-86481a7f16bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:get_user(user_id=1d73ba9e79f14e828e70219e28d63e3d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:11.331055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97d6b8c0746d4793b55171f48bd9b3eb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:11.335375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d5f76-8387-453e-ac42-86481a7f16bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:11.349498 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2691/5396] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:11:10 2022] GET /identity/v3/users/1d73ba9e79f14e828e70219e28d63e3d => generated 417 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:11:11.349498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d5f76-8387-453e-ac42-86481a7f16bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:11.356384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e4d5f76-8387-453e-ac42-86481a7f16bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:11.357652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97d6b8c0746d4793b55171f48bd9b3eb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:11.368367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97d6b8c0746d4793b55171f48bd9b3eb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:11.378557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97d6b8c0746d4793b55171f48bd9b3eb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:11.379342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97d6b8c0746d4793b55171f48bd9b3eb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:11.387708 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2706/5397] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:11 2022] DELETE /identity/v3/users/97d6b8c0746d4793b55171f48bd9b3eb => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:11.390444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d5f76-8387-453e-ac42-86481a7f16bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:11.499864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64c16d-a943-441a-8314-f1ed157e6dc5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:11.500894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64c16d-a943-441a-8314-f1ed157e6dc5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:11.501501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64c16d-a943-441a-8314-f1ed157e6dc5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:11.501971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64c16d-a943-441a-8314-f1ed157e6dc5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/roles/4a1c398cfc204ba8ba786ac911e49691` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:11.504394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64c16d-a943-441a-8314-f1ed157e6dc5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:get_role(role_id=4a1c398cfc204ba8ba786ac911e49691)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:11.507828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64c16d-a943-441a-8314-f1ed157e6dc5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:11.509556 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2692/5398] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:11:11 2022] GET /identity/v3/roles/4a1c398cfc204ba8ba786ac911e49691 => generated 238 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:11.513684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:11.515225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:11.536988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40866a-823d-44eb-8373-60f59753ed39 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:11.546016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec7321-83fe-48e6-bc1b-b87b67d27132 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:11.547299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec7321-83fe-48e6-bc1b-b87b67d27132 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:11.547856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec7321-83fe-48e6-bc1b-b87b67d27132 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:11.548876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec7321-83fe-48e6-bc1b-b87b67d27132 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:11.550344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec7321-83fe-48e6-bc1b-b87b67d27132 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:11.553230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec7321-83fe-48e6-bc1b-b87b67d27132 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:11.670642 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2707/5399] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:11 2022] POST /identity/v3/projects => generated 359 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:11.675634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64c16d-a943-441a-8314-f1ed157e6dc5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:11.690341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64c16d-a943-441a-8314-f1ed157e6dc5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:11.838567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd005-e57e-4ba4-b2b0-43387b2b182e tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:11.839706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd005-e57e-4ba4-b2b0-43387b2b182e tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:11.840261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd005-e57e-4ba4-b2b0-43387b2b182e tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:11.840650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd005-e57e-4ba4-b2b0-43387b2b182e tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects/539a54c757074f30887b7b7c8991e226` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:11.842351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffd005-e57e-4ba4-b2b0-43387b2b182e tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:get_project(project_id=539a54c757074f30887b7b7c8991e226)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:11.845698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffd005-e57e-4ba4-b2b0-43387b2b182e tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:11.851923 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2693/5400] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:11:11 2022] GET /identity/v3/projects/539a54c757074f30887b7b7c8991e226 => generated 359 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:11.856253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec7321-83fe-48e6-bc1b-b87b67d27132 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:11.877099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec7321-83fe-48e6-bc1b-b87b67d27132 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:11.885880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6bc36-31ce-43ad-85d8-3162ab733f6c tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:11.886966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6bc36-31ce-43ad-85d8-3162ab733f6c tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:11.887479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6bc36-31ce-43ad-85d8-3162ab733f6c tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:11.888020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6bc36-31ce-43ad-85d8-3162ab733f6c tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects/539a54c757074f30887b7b7c8991e226/users/1d73ba9e79f14e828e70219e28d63e3d/roles/4a1c398cfc204ba8ba786ac911e49691` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:11.892468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6bc36-31ce-43ad-85d8-3162ab733f6c tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_grant(project_id=539a54c757074f30887b7b7c8991e226, user_id=1d73ba9e79f14e828e70219e28d63e3d, role_id=4a1c398cfc204ba8ba786ac911e49691)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:11.901954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6bc36-31ce-43ad-85d8-3162ab733f6c tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:11.936987 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2708/5401] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:11 2022] PUT /identity/v3/projects/539a54c757074f30887b7b7c8991e226/users/1d73ba9e79f14e828e70219e28d63e3d/roles/4a1c398cfc204ba8ba786ac911e49691 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:11.937970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd005-e57e-4ba4-b2b0-43387b2b182e tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:11.955545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd005-e57e-4ba4-b2b0-43387b2b182e tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:12.063578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:12.079239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a4f1-08a0-40f5-b1ac-0d5b428d7dd0 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:12.079953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a4f1-08a0-40f5-b1ac-0d5b428d7dd0 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:12.080400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a4f1-08a0-40f5-b1ac-0d5b428d7dd0 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:12.080795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a4f1-08a0-40f5-b1ac-0d5b428d7dd0 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:12.081699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4a4f1-08a0-40f5-b1ac-0d5b428d7dd0 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:12.084097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4a4f1-08a0-40f5-b1ac-0d5b428d7dd0 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:12.126219 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2694/5402] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:11 2022] POST /identity/v3/projects => generated 359 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:12.129855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6bc36-31ce-43ad-85d8-3162ab733f6c tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:12.142031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6bc36-31ce-43ad-85d8-3162ab733f6c tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:12.223403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a4f1-08a0-40f5-b1ac-0d5b428d7dd0 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:12.227456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dab4a4f1-08a0-40f5-b1ac-0d5b428d7dd0 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:12.310111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e6740-1e04-4fc3-9bb0-2ca4571eb0c7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:12.310766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e6740-1e04-4fc3-9bb0-2ca4571eb0c7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:12.311042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e6740-1e04-4fc3-9bb0-2ca4571eb0c7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:12.311309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e6740-1e04-4fc3-9bb0-2ca4571eb0c7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects/3b32eabb33f04cd1a1a8b2e1cb86257e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:12.315558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e6740-1e04-4fc3-9bb0-2ca4571eb0c7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:get_project(project_id=3b32eabb33f04cd1a1a8b2e1cb86257e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:12.319243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e6740-1e04-4fc3-9bb0-2ca4571eb0c7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:12.325782 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2709/5403] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:11:12 2022] GET /identity/v3/projects/3b32eabb33f04cd1a1a8b2e1cb86257e => generated 359 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:12.327678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e6740-1e04-4fc3-9bb0-2ca4571eb0c7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:12.350451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e6740-1e04-4fc3-9bb0-2ca4571eb0c7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:12.352448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a4f1-08a0-40f5-b1ac-0d5b428d7dd0 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:12.359501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:12.360045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24e60c-f1c3-4596-9e8d-06a83204b913 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:12.361025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:12.361263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24e60c-f1c3-4596-9e8d-06a83204b913 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:12.361514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24e60c-f1c3-4596-9e8d-06a83204b913 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:12.361801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24e60c-f1c3-4596-9e8d-06a83204b913 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects/3b32eabb33f04cd1a1a8b2e1cb86257e/users/1d73ba9e79f14e828e70219e28d63e3d/roles/4a1c398cfc204ba8ba786ac911e49691` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:12.362489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:12.363253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] PATH_INFO: `/v3/projects/841626c2473f42d394e2a1f988cc4994` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:12.365722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=841626c2473f42d394e2a1f988cc4994)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:12.369007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:12.372848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24e60c-f1c3-4596-9e8d-06a83204b913 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3b32eabb33f04cd1a1a8b2e1cb86257e, user_id=1d73ba9e79f14e828e70219e28d63e3d, role_id=4a1c398cfc204ba8ba786ac911e49691)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:12.375724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24e60c-f1c3-4596-9e8d-06a83204b913 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:12.412636 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2710/5404] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:12 2022] PUT /identity/v3/projects/3b32eabb33f04cd1a1a8b2e1cb86257e/users/1d73ba9e79f14e828e70219e28d63e3d/roles/4a1c398cfc204ba8ba786ac911e49691 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:12.414336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24e60c-f1c3-4596-9e8d-06a83204b913 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:12.421122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:12.468025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841626c2473f42d394e2a1f988cc4994', 'request_id': 'req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:12.482182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841626c2473f42d394e2a1f988cc4994', 'request_id': 'req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:12.500692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841626c2473f42d394e2a1f988cc4994', 'request_id': 'req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:12.519123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] The token cache is being invalidate because project 841626c2473f42d394e2a1f988cc4994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:12.519552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:12.520430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-31c9baca-7a5d-49f4-8e24-aaa97cf4f762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:12.526056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24e60c-f1c3-4596-9e8d-06a83204b913 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:12.526272 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2695/5405] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:12 2022] DELETE /identity/v3/projects/841626c2473f42d394e2a1f988cc4994 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:12.532055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39ba90-9a83-4247-822b-0b589ee55223 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:12.532549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39ba90-9a83-4247-822b-0b589ee55223 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:12.532950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39ba90-9a83-4247-822b-0b589ee55223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:12.582982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7a39ba90-9a83-4247-822b-0b589ee55223 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:12.586448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7a39ba90-9a83-4247-822b-0b589ee55223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:12.635206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413657a-aa01-465e-9fda-ff73cbd15ec7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:12.637763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413657a-aa01-465e-9fda-ff73cbd15ec7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:12.638481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413657a-aa01-465e-9fda-ff73cbd15ec7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:12.640370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413657a-aa01-465e-9fda-ff73cbd15ec7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/1d73ba9e79f14e828e70219e28d63e3d/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:12.643299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413657a-aa01-465e-9fda-ff73cbd15ec7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:list_user_projects(user_id=1d73ba9e79f14e828e70219e28d63e3d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:12.649101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413657a-aa01-465e-9fda-ff73cbd15ec7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:12.727163 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2711/5406] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:11:12 2022] GET /identity/v3/users/1d73ba9e79f14e828e70219e28d63e3d/projects => generated 842 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:12.757335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413657a-aa01-465e-9fda-ff73cbd15ec7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:12.757995 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2696/5407] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:12.763681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff878772-b0c4-4491-a17a-18c48231d952 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:12.764025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff878772-b0c4-4491-a17a-18c48231d952 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:12.764298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff878772-b0c4-4491-a17a-18c48231d952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:12.765491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0413657a-aa01-465e-9fda-ff73cbd15ec7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:12.802016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413657a-aa01-465e-9fda-ff73cbd15ec7 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:12.812403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ff878772-b0c4-4491-a17a-18c48231d952 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:12.815601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:12.816598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:12.816946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:12.817314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects/3b32eabb33f04cd1a1a8b2e1cb86257e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:12.819565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ff878772-b0c4-4491-a17a-18c48231d952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:12.821712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3b32eabb33f04cd1a1a8b2e1cb86257e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:12.824822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:12.840916 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2697/5408] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:12.843198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff878772-b0c4-4491-a17a-18c48231d952 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:12.853621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:12.877421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff878772-b0c4-4491-a17a-18c48231d952 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:12.895725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46078aec-e117-475c-8e87-9100b80a4d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:12.898731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46078aec-e117-475c-8e87-9100b80a4d43 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:12.899005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46078aec-e117-475c-8e87-9100b80a4d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:12.899269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46078aec-e117-475c-8e87-9100b80a4d43 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:12.899924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46078aec-e117-475c-8e87-9100b80a4d43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:12.900217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46078aec-e117-475c-8e87-9100b80a4d43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:12.901840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46078aec-e117-475c-8e87-9100b80a4d43 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:12.928154 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2698/5409] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:12.932697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46078aec-e117-475c-8e87-9100b80a4d43 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:12.949638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46078aec-e117-475c-8e87-9100b80a4d43 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:12.960909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b32eabb33f04cd1a1a8b2e1cb86257e', 'request_id': 'req-fe2fa12d-d95f-48ac-b09a-cdd73691c492'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:12.981577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b32eabb33f04cd1a1a8b2e1cb86257e', 'request_id': 'req-fe2fa12d-d95f-48ac-b09a-cdd73691c492'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:12.996719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b32eabb33f04cd1a1a8b2e1cb86257e', 'request_id': 'req-fe2fa12d-d95f-48ac-b09a-cdd73691c492'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:13.025085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] The token cache is being invalidate because project 3b32eabb33f04cd1a1a8b2e1cb86257e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:13.025539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:13.026445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:13.028732 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2712/5410] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:12 2022] DELETE /identity/v3/projects/3b32eabb33f04cd1a1a8b2e1cb86257e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:13.035280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:13.036865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:13.045545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:13.112452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f5a74-0a97-48d3-8737-2d9075a7251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:13.113242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f5a74-0a97-48d3-8737-2d9075a7251a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:13.113866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f5a74-0a97-48d3-8737-2d9075a7251a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:13.114516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f5a74-0a97-48d3-8737-2d9075a7251a admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:13.117247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f5a74-0a97-48d3-8737-2d9075a7251a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:13.121362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f5a74-0a97-48d3-8737-2d9075a7251a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:13.181463 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2699/5411] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:12 2022] POST /identity/v3/projects => generated 377 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:13.186130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f5a74-0a97-48d3-8737-2d9075a7251a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:13.189762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6d4f5a74-0a97-48d3-8737-2d9075a7251a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:13.190876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fa12d-d95f-48ac-b09a-cdd73691c492 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:13.326088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f5a74-0a97-48d3-8737-2d9075a7251a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:13.328970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:13.330147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:13.330646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:13.331126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects/539a54c757074f30887b7b7c8991e226` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:13.333908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:delete_project(project_id=539a54c757074f30887b7b7c8991e226)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:13.336465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53dac0-aa0d-47d6-b948-9bef69fdf45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:13.337281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53dac0-aa0d-47d6-b948-9bef69fdf45c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:13.337615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53dac0-aa0d-47d6-b948-9bef69fdf45c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:13.337889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53dac0-aa0d-47d6-b948-9bef69fdf45c admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:13.338525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53dac0-aa0d-47d6-b948-9bef69fdf45c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:13.340332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:13.341535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53dac0-aa0d-47d6-b948-9bef69fdf45c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:13.379577 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2700/5412] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:13 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:13.384760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53dac0-aa0d-47d6-b948-9bef69fdf45c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:13.416469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53dac0-aa0d-47d6-b948-9bef69fdf45c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:13.437867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5a0eb-bacb-4e9f-8833-2ef9ff281f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:13.438653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5a0eb-bacb-4e9f-8833-2ef9ff281f9b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:13.442189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5a0eb-bacb-4e9f-8833-2ef9ff281f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:13.443795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5a0eb-bacb-4e9f-8833-2ef9ff281f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:13.447112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5a0eb-bacb-4e9f-8833-2ef9ff281f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:13.451279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5a0eb-bacb-4e9f-8833-2ef9ff281f9b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:13.476059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '539a54c757074f30887b7b7c8991e226', 'request_id': 'req-899b96ea-fd46-4237-af11-0de1d46ccb30'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:13.488096 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2701/5413] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:13 2022] GET /identity/v3/roles => generated 1649 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:13.495334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '539a54c757074f30887b7b7c8991e226', 'request_id': 'req-899b96ea-fd46-4237-af11-0de1d46ccb30'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:13.500944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5a0eb-bacb-4e9f-8833-2ef9ff281f9b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:13.508321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '539a54c757074f30887b7b7c8991e226', 'request_id': 'req-899b96ea-fd46-4237-af11-0de1d46ccb30'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:13.526137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] The token cache is being invalidate because project 539a54c757074f30887b7b7c8991e226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:13.526433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:13.527294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:13.528070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5a0eb-bacb-4e9f-8833-2ef9ff281f9b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:13.531782 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2713/5414] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:13 2022] DELETE /identity/v3/projects/539a54c757074f30887b7b7c8991e226 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:13.533143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:13.534663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:13.626002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:13.643159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:13.655480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6deb1-6315-4155-99cb-a52d6ab66b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:13.656124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6deb1-6315-4155-99cb-a52d6ab66b95 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:13.656329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6deb1-6315-4155-99cb-a52d6ab66b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:13.656597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6deb1-6315-4155-99cb-a52d6ab66b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:13.657265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6deb1-6315-4155-99cb-a52d6ab66b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:13.658842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6deb1-6315-4155-99cb-a52d6ab66b95 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:13.676005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b96ea-fd46-4237-af11-0de1d46ccb30 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:13.676254 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2702/5415] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:13 2022] GET /identity/v3/roles => generated 1649 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:13.677059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6deb1-6315-4155-99cb-a52d6ab66b95 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:13.679539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-36f6deb1-6315-4155-99cb-a52d6ab66b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:13.695985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:13.696710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:13.697070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:13.697418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/roles/4a1c398cfc204ba8ba786ac911e49691` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:13.697811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6deb1-6315-4155-99cb-a52d6ab66b95 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:13.709661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4a1c398cfc204ba8ba786ac911e49691)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:13.710417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7cf5b-4388-4e2d-995e-2efbd10b7422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:13.711513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7cf5b-4388-4e2d-995e-2efbd10b7422 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:13.711728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:13.711849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7cf5b-4388-4e2d-995e-2efbd10b7422 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:13.712468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7cf5b-4388-4e2d-995e-2efbd10b7422 admin admin] PATH_INFO: `/v3/projects/545df810b69b43a28229f7cbe38ff5cd/users/9afb60498a284007a90f928ff8098f9c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:13.732783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7cf5b-4388-4e2d-995e-2efbd10b7422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=545df810b69b43a28229f7cbe38ff5cd, user_id=9afb60498a284007a90f928ff8098f9c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:13.735936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7cf5b-4388-4e2d-995e-2efbd10b7422 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:13.777362 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2703/5416] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:13 2022] PUT /identity/v3/projects/545df810b69b43a28229f7cbe38ff5cd/users/9afb60498a284007a90f928ff8098f9c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:13.798340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invalidating the token cache because role 4a1c398cfc204ba8ba786ac911e49691 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:13.798645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:13.799477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:13.800103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92000-93f1-4799-a060-d57cda4e722c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:13.804971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92000-93f1-4799-a060-d57cda4e722c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:13.805854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92000-93f1-4799-a060-d57cda4e722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:13.807533 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2714/5417] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:13 2022] DELETE /identity/v3/roles/4a1c398cfc204ba8ba786ac911e49691 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:13.812686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:13.817029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:13.849988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-25f92000-93f1-4799-a060-d57cda4e722c None None] MFA Rules not processed for user `9afb60498a284007a90f928ff8098f9c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:13.852389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-25f92000-93f1-4799-a060-d57cda4e722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:13.980255 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2704/5418] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:13.981353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec112d-e7dc-4b4a-ac9c-ae3274ba0e27 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:13.985591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f92000-93f1-4799-a060-d57cda4e722c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:13.989771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-25f92000-93f1-4799-a060-d57cda4e722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:13.994990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:13.995703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:13.996062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:13.996389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/1d73ba9e79f14e828e70219e28d63e3d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:13.999837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1d73ba9e79f14e828e70219e28d63e3d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:14.002275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:14.101195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d73ba9e79f14e828e70219e28d63e3d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.134174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d73ba9e79f14e828e70219e28d63e3d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.151753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d73ba9e79f14e828e70219e28d63e3d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.164033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f92000-93f1-4799-a060-d57cda4e722c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:14.165077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d73ba9e79f14e828e70219e28d63e3d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.165959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d73ba9e79f14e828e70219e28d63e3d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.173887 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2715/5419] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:13 2022] DELETE /identity/v3/users/1d73ba9e79f14e828e70219e28d63e3d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:14.179186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decb45a-0e92-4452-81eb-1871d6805fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:14.180140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:14.181662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:14.183868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decb45a-0e92-4452-81eb-1871d6805fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.184160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decb45a-0e92-4452-81eb-1871d6805fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.184554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decb45a-0e92-4452-81eb-1871d6805fa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.185430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decb45a-0e92-4452-81eb-1871d6805fa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:14.192799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decb45a-0e92-4452-81eb-1871d6805fa2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:14.253878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:14.263603 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2705/5420] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:13 2022] POST /identity/v3/projects => generated 375 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:14.265383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decb45a-0e92-4452-81eb-1871d6805fa2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:14.268049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3decb45a-0e92-4452-81eb-1871d6805fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:14.307258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b1ab2-fd11-4263-a76c-7048b78878a3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:14.328062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:14.363249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decb45a-0e92-4452-81eb-1871d6805fa2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:14.369424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc56ab-488d-4df3-92a0-5b837612fc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:14.370291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc56ab-488d-4df3-92a0-5b837612fc21 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.370536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc56ab-488d-4df3-92a0-5b837612fc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.371343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc56ab-488d-4df3-92a0-5b837612fc21 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.372229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc56ab-488d-4df3-92a0-5b837612fc21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:14.374232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc56ab-488d-4df3-92a0-5b837612fc21 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:14.406671 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2706/5421] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:14 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:14.407625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc56ab-488d-4df3-92a0-5b837612fc21 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:14.418879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc56ab-488d-4df3-92a0-5b837612fc21 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:14.425067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f019175-3a23-48bf-abe3-2ff97388f2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:14.426008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f019175-3a23-48bf-abe3-2ff97388f2dc admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.426008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f019175-3a23-48bf-abe3-2ff97388f2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.426231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f019175-3a23-48bf-abe3-2ff97388f2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.426872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f019175-3a23-48bf-abe3-2ff97388f2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:14.427059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:14.428220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.429254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.429470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f019175-3a23-48bf-abe3-2ff97388f2dc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:14.429577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects/c0231ddc99ed48b29978698ac77c3ef5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.431832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c0231ddc99ed48b29978698ac77c3ef5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:14.435926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:14.446224 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2707/5422] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:14.447282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f019175-3a23-48bf-abe3-2ff97388f2dc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:14.461882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f019175-3a23-48bf-abe3-2ff97388f2dc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:14.468484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35547b86-0ae7-483c-8cae-d954699c238c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:14.469335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35547b86-0ae7-483c-8cae-d954699c238c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.469694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35547b86-0ae7-483c-8cae-d954699c238c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.469972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35547b86-0ae7-483c-8cae-d954699c238c admin admin] PATH_INFO: `/v3/projects/98f5e9d5f9cb4d678b5e0339e4fc1e95/users/047464bfd2db48328dd289edab484a5d/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.489508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35547b86-0ae7-483c-8cae-d954699c238c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f5e9d5f9cb4d678b5e0339e4fc1e95, user_id=047464bfd2db48328dd289edab484a5d, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:14.493434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35547b86-0ae7-483c-8cae-d954699c238c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:14.513207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0231ddc99ed48b29978698ac77c3ef5', 'request_id': 'req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.516662 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2708/5423] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:14 2022] PUT /identity/v3/projects/98f5e9d5f9cb4d678b5e0339e4fc1e95/users/047464bfd2db48328dd289edab484a5d/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:14.517614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35547b86-0ae7-483c-8cae-d954699c238c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:14.525223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0231ddc99ed48b29978698ac77c3ef5', 'request_id': 'req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.536618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0231ddc99ed48b29978698ac77c3ef5', 'request_id': 'req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.558351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] The token cache is being invalidate because project c0231ddc99ed48b29978698ac77c3ef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:14.558865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.562104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.566408 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2716/5424] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:14 2022] DELETE /identity/v3/projects/c0231ddc99ed48b29978698ac77c3ef5 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:14.574149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:14.578129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:14.618469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35547b86-0ae7-483c-8cae-d954699c238c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:14.628624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bb465-29e5-42a2-89d0-98f15551d79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:14.630603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bb465-29e5-42a2-89d0-98f15551d79d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.632354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bb465-29e5-42a2-89d0-98f15551d79d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.632886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bb465-29e5-42a2-89d0-98f15551d79d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.635312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bb465-29e5-42a2-89d0-98f15551d79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:14.638891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bb465-29e5-42a2-89d0-98f15551d79d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:14.663739 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2709/5425] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:14.665358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bb465-29e5-42a2-89d0-98f15551d79d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:14.668381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4c3bb465-29e5-42a2-89d0-98f15551d79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:14.681507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2aeb7b-98fb-4c04-b66f-44e8a8dd9279 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:14.684997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bb465-29e5-42a2-89d0-98f15551d79d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:14.690460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:14.696634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f40f4-5536-4b2f-b229-a60674c42234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:14.698380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f40f4-5536-4b2f-b229-a60674c42234 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.698672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f40f4-5536-4b2f-b229-a60674c42234 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.698962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f40f4-5536-4b2f-b229-a60674c42234 admin admin] PATH_INFO: `/v3/projects/98f5e9d5f9cb4d678b5e0339e4fc1e95/users/047464bfd2db48328dd289edab484a5d/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.701585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f40f4-5536-4b2f-b229-a60674c42234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f5e9d5f9cb4d678b5e0339e4fc1e95, user_id=047464bfd2db48328dd289edab484a5d, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:14.706470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f40f4-5536-4b2f-b229-a60674c42234 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:14.725796 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2710/5426] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:14 2022] PUT /identity/v3/projects/98f5e9d5f9cb4d678b5e0339e4fc1e95/users/047464bfd2db48328dd289edab484a5d/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:14.738569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88894527-3914-4f5a-99d4-4c8b28362fc3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.738872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88894527-3914-4f5a-99d4-4c8b28362fc3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.739184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88894527-3914-4f5a-99d4-4c8b28362fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.781517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-88894527-3914-4f5a-99d4-4c8b28362fc3 None None] MFA Rules not processed for user `047464bfd2db48328dd289edab484a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:14.782817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-88894527-3914-4f5a-99d4-4c8b28362fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:14.785184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:14.785879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.786164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.786436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/0902a3cd5c044bd0af8a9a5436299c85` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.789239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:update_user(user_id=0902a3cd5c044bd0af8a9a5436299c85)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:14.793445 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:11:14.793445 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:11:14.793445 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:11:14.795500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:14.919194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '0902a3cd5c044bd0af8a9a5436299c85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.932075 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2711/5427] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:14.933490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88894527-3914-4f5a-99d4-4c8b28362fc3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:14.934061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invalidating the token cache because user 0902a3cd5c044bd0af8a9a5436299c85 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:14.934500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.935162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c71e2b51-360f-4a07-99df-a98d12bc2f1a tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:14.937374 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2717/5428] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Oct 15 21:11:14 2022] PATCH /identity/v3/users/0902a3cd5c044bd0af8a9a5436299c85 => generated 297 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:14.940228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:14.943119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4f19f-bb2b-4bcb-8f8f-f45c6c6d6789 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.943496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4f19f-bb2b-4bcb-8f8f-f45c6c6d6789 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.943898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4f19f-bb2b-4bcb-8f8f-f45c6c6d6789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.951229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88894527-3914-4f5a-99d4-4c8b28362fc3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:14.967532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e4480-dab2-4d54-b0ad-ea94b499c287 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047464bfd2db48328dd289edab484a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:14.968189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e4480-dab2-4d54-b0ad-ea94b499c287 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:14.968418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e4480-dab2-4d54-b0ad-ea94b499c287 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:14.968689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e4480-dab2-4d54-b0ad-ea94b499c287 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:14.969452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e4480-dab2-4d54-b0ad-ea94b499c287 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:14.971465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e4480-dab2-4d54-b0ad-ea94b499c287 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.016557 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2712/5429] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:14 2022] POST /identity/v3/projects => generated 333 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:15.021010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-35c4f19f-bb2b-4bcb-8f8f-f45c6c6d6789 None None] MFA Rules not processed for user `0902a3cd5c044bd0af8a9a5436299c85`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:15.021681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e4480-dab2-4d54-b0ad-ea94b499c287 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.023089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-570e4480-dab2-4d54-b0ad-ea94b499c287 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:15.023397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-35c4f19f-bb2b-4bcb-8f8f-f45c6c6d6789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:15.028932 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2718/5430] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:14 2022] POST /identity/v3/auth/tokens => generated 349 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 21:11:15.033150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4f19f-bb2b-4bcb-8f8f-f45c6c6d6789 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.034702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-35c4f19f-bb2b-4bcb-8f8f-f45c6c6d6789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:15.131869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e4480-dab2-4d54-b0ad-ea94b499c287 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:15.134561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4f19f-bb2b-4bcb-8f8f-f45c6c6d6789 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:15.138869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eda1d1-80b6-425f-956d-e4d27d4b07a1 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047464bfd2db48328dd289edab484a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:15.139738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eda1d1-80b6-425f-956d-e4d27d4b07a1 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.140420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eda1d1-80b6-425f-956d-e4d27d4b07a1 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.141339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eda1d1-80b6-425f-956d-e4d27d4b07a1 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.142154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eda1d1-80b6-425f-956d-e4d27d4b07a1 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:15.144821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eda1d1-80b6-425f-956d-e4d27d4b07a1 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.145006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:15.146138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.147555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.147935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/0902a3cd5c044bd0af8a9a5436299c85` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.150056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:update_user(user_id=0902a3cd5c044bd0af8a9a5436299c85)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:15.152364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.189239 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2713/5431] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:11:15 2022] POST /identity/v3/projects => generated 333 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:15.193267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eda1d1-80b6-425f-956d-e4d27d4b07a1 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.222724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '0902a3cd5c044bd0af8a9a5436299c85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.236149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invalidating the token cache because user 0902a3cd5c044bd0af8a9a5436299c85 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:15.237328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.238579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.245584 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2719/5432] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Oct 15 21:11:15 2022] PATCH /identity/v3/users/0902a3cd5c044bd0af8a9a5436299c85 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:15.246832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.248731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:15.295241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eda1d1-80b6-425f-956d-e4d27d4b07a1 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:15.300567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ff1bc-a4a3-43d8-bc99-e72ed9eedfc5 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047464bfd2db48328dd289edab484a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:15.301380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ff1bc-a4a3-43d8-bc99-e72ed9eedfc5 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.301676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ff1bc-a4a3-43d8-bc99-e72ed9eedfc5 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.301942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ff1bc-a4a3-43d8-bc99-e72ed9eedfc5 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.302795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ff1bc-a4a3-43d8-bc99-e72ed9eedfc5 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:15.304814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ff1bc-a4a3-43d8-bc99-e72ed9eedfc5 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.308215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae50bb5-e0ed-4845-93e3-fa906bfd97e2 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:15.379050 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2714/5433] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:15 2022] POST /identity/v3/projects => generated 357 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:15.384230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ff1bc-a4a3-43d8-bc99-e72ed9eedfc5 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.386978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-520ff1bc-a4a3-43d8-bc99-e72ed9eedfc5 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:15.388791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:UPDATE=2,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:15.425667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6783c76-023b-414e-9835-2ded376108c8 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:15.426236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6783c76-023b-414e-9835-2ded376108c8 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.426569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6783c76-023b-414e-9835-2ded376108c8 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.426850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6783c76-023b-414e-9835-2ded376108c8 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/0902a3cd5c044bd0af8a9a5436299c85` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.455660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6783c76-023b-414e-9835-2ded376108c8 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:update_user(user_id=0902a3cd5c044bd0af8a9a5436299c85)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:15.457777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:15.460954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6783c76-023b-414e-9835-2ded376108c8 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.491658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ff1bc-a4a3-43d8-bc99-e72ed9eedfc5 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:15.497208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97408285-c552-4079-9251-94c5f0303a08 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047464bfd2db48328dd289edab484a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:15.497951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97408285-c552-4079-9251-94c5f0303a08 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.498177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97408285-c552-4079-9251-94c5f0303a08 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.498385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97408285-c552-4079-9251-94c5f0303a08 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.499031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97408285-c552-4079-9251-94c5f0303a08 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:15.501069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97408285-c552-4079-9251-94c5f0303a08 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.514663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b6783c76-023b-414e-9835-2ded376108c8 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '0902a3cd5c044bd0af8a9a5436299c85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.524770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b6783c76-023b-414e-9835-2ded376108c8 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invalidating the token cache because user 0902a3cd5c044bd0af8a9a5436299c85 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:15.525210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b6783c76-023b-414e-9835-2ded376108c8 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.526100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b6783c76-023b-414e-9835-2ded376108c8 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.530826 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2720/5434] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Oct 15 21:11:15 2022] PATCH /identity/v3/users/0902a3cd5c044bd0af8a9a5436299c85 => generated 297 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:15.532424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5410b-70be-4262-a108-1e16730aa3d5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.533192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5410b-70be-4262-a108-1e16730aa3d5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.533327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5410b-70be-4262-a108-1e16730aa3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.536251 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2715/5435] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Oct 15 21:11:15 2022] GET /identity/v3/projects => generated 7940 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:11:15.543016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97408285-c552-4079-9251-94c5f0303a08 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.545112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-97408285-c552-4079-9251-94c5f0303a08 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:15.571900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97408285-c552-4079-9251-94c5f0303a08 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:15.581378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bf90d-c5a8-4972-a3e8-3092d9754a93 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047464bfd2db48328dd289edab484a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:15.581897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-23b5410b-70be-4262-a108-1e16730aa3d5 None None] MFA Rules not processed for user `0902a3cd5c044bd0af8a9a5436299c85`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:15.582582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bf90d-c5a8-4972-a3e8-3092d9754a93 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.583129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-23b5410b-70be-4262-a108-1e16730aa3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:15.584134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bf90d-c5a8-4972-a3e8-3092d9754a93 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.584814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bf90d-c5a8-4972-a3e8-3092d9754a93 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.585916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63bf90d-c5a8-4972-a3e8-3092d9754a93 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:15.586600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63bf90d-c5a8-4972-a3e8-3092d9754a93 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:15.589634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63bf90d-c5a8-4972-a3e8-3092d9754a93 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.597176 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2721/5436] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:15 2022] POST /identity/v3/auth/tokens => generated 349 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:11:15.597176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5410b-70be-4262-a108-1e16730aa3d5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.597176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-23b5410b-70be-4262-a108-1e16730aa3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:15.626271 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2716/5437] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 21:11:15 2022] GET /identity/v3/projects?enabled=False => generated 444 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:15.627980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bf90d-c5a8-4972-a3e8-3092d9754a93 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.644328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bf90d-c5a8-4972-a3e8-3092d9754a93 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:15.652578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d529-d634-4c61-8dec-6d0bd4150560 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047464bfd2db48328dd289edab484a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:15.653347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d529-d634-4c61-8dec-6d0bd4150560 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.653987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d529-d634-4c61-8dec-6d0bd4150560 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.654569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d529-d634-4c61-8dec-6d0bd4150560 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.655973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5d529-d634-4c61-8dec-6d0bd4150560 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Adding query filter params (parent_id=f089503e32354605aa178f58d1d3d950) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:15.657926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5d529-d634-4c61-8dec-6d0bd4150560 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:15.660879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5d529-d634-4c61-8dec-6d0bd4150560 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.690568 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2717/5438] 10.222.0.16 () {58 vars in 1216 bytes} [Sat Oct 15 21:11:15 2022] GET /identity/v3/projects?parent_id=f089503e32354605aa178f58d1d3d950 => generated 497 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:15.696099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5410b-70be-4262-a108-1e16730aa3d5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:15.697494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d529-d634-4c61-8dec-6d0bd4150560 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.701281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:15.702000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.702354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.702750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/0902a3cd5c044bd0af8a9a5436299c85` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.704186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:update_user(user_id=0902a3cd5c044bd0af8a9a5436299c85)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:15.709381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.712139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d529-d634-4c61-8dec-6d0bd4150560 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:15.721149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047464bfd2db48328dd289edab484a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:15.721149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.721657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.722039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] PATH_INFO: `/v3/projects/8e034af5bcd24021a34142f73fe2b2dc` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.723415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8e034af5bcd24021a34142f73fe2b2dc)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:15.727449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.761505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '0902a3cd5c044bd0af8a9a5436299c85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.774098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invalidating the token cache because user 0902a3cd5c044bd0af8a9a5436299c85 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:15.774586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.775578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.778410 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2722/5439] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Oct 15 21:11:15 2022] PATCH /identity/v3/users/0902a3cd5c044bd0af8a9a5436299c85 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:15.784285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.785803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:15.807454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ed9cc-cfe0-4b8f-b27b-53fb9915c4ac tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:15.810905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e034af5bcd24021a34142f73fe2b2dc', 'request_id': 'req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.826027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e034af5bcd24021a34142f73fe2b2dc', 'request_id': 'req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.835501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e034af5bcd24021a34142f73fe2b2dc', 'request_id': 'req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.869724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] The token cache is being invalidate because project 8e034af5bcd24021a34142f73fe2b2dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:15.870062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.873346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:15.876678 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2718/5440] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:15 2022] DELETE /identity/v3/projects/8e034af5bcd24021a34142f73fe2b2dc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:15.882351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.888122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:15.903511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22646-067b-44fe-b6b9-fb9545f914e5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:15.908976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22646-067b-44fe-b6b9-fb9545f914e5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:15.909547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22646-067b-44fe-b6b9-fb9545f914e5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:15.911100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22646-067b-44fe-b6b9-fb9545f914e5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:15.911860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f22646-067b-44fe-b6b9-fb9545f914e5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:15.917073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f22646-067b-44fe-b6b9-fb9545f914e5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:15.952516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:UPDATE=2,keystone:INSERT=6,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:15.979039 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2723/5441] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:11:15 2022] POST /identity/v3/users => generated 264 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:15.983365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22646-067b-44fe-b6b9-fb9545f914e5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:15.985867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-27f22646-067b-44fe-b6b9-fb9545f914e5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:16.000798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22646-067b-44fe-b6b9-fb9545f914e5 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:16.008806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01e2a5-d93d-44b3-bd9b-65bcc5f7bf19 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:16.009643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01e2a5-d93d-44b3-bd9b-65bcc5f7bf19 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:16.010071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01e2a5-d93d-44b3-bd9b-65bcc5f7bf19 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:16.010406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01e2a5-d93d-44b3-bd9b-65bcc5f7bf19 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/620c9fff9f76419d8e09edef7aa16b86/password` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:16.023954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6a36e-0bf4-4b1c-9e9d-31bb1c73d14d tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:16.033163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047464bfd2db48328dd289edab484a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:16.033893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:16.034246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:16.034591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] PATH_INFO: `/v3/projects/f089503e32354605aa178f58d1d3d950` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:16.035882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f089503e32354605aa178f58d1d3d950)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:16.038906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:16.091957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:16.094678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3e01e2a5-d93d-44b3-bd9b-65bcc5f7bf19 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '620c9fff9f76419d8e09edef7aa16b86'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.106573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f089503e32354605aa178f58d1d3d950', 'request_id': 'req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.110011 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2724/5442] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:11:15 2022] POST /identity/v3/users/620c9fff9f76419d8e09edef7aa16b86/password => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:16.117788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f089503e32354605aa178f58d1d3d950', 'request_id': 'req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.125262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f089503e32354605aa178f58d1d3d950', 'request_id': 'req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.139188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] The token cache is being invalidate because project f089503e32354605aa178f58d1d3d950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:16.140067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.140193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.142073 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2719/5443] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:15 2022] DELETE /identity/v3/projects/f089503e32354605aa178f58d1d3d950 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:16.145775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01e2a5-d93d-44b3-bd9b-65bcc5f7bf19 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:16.148117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3e01e2a5-d93d-44b3-bd9b-65bcc5f7bf19 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:16.164670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01e2a5-d93d-44b3-bd9b-65bcc5f7bf19 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:16.243494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047464bfd2db48328dd289edab484a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:16.244182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:16.244732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:16.244897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] PATH_INFO: `/v3/projects/55d1ef91aaff45c4ab39a35e0f623aab` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:16.246726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorizing `identity:delete_project(project_id=55d1ef91aaff45c4ab39a35e0f623aab)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:16.250967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:16.335267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d1ef91aaff45c4ab39a35e0f623aab', 'request_id': 'req-760a7c70-67cd-43d5-821b-1dbc43f28610'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.347913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d1ef91aaff45c4ab39a35e0f623aab', 'request_id': 'req-760a7c70-67cd-43d5-821b-1dbc43f28610'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.370578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d1ef91aaff45c4ab39a35e0f623aab', 'request_id': 'req-760a7c70-67cd-43d5-821b-1dbc43f28610'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.385560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] The token cache is being invalidate because project 55d1ef91aaff45c4ab39a35e0f623aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:16.387989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.388886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-760a7c70-67cd-43d5-821b-1dbc43f28610 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.390870 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2725/5444] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:16 2022] DELETE /identity/v3/projects/55d1ef91aaff45c4ab39a35e0f623aab => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:16.394753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:16.396174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:16.535395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa79b7-8aad-4216-83c1-7b51bf2c4a19 tempest-ListProjectsTestJSON-937109541 tempest-ListProjectsTestJSON-937109541-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:16.655512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:16.656190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:16.656465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:16.656735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] PATH_INFO: `/v3/users/9afb60498a284007a90f928ff8098f9c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:16.659685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9afb60498a284007a90f928ff8098f9c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:16.661688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:16.742893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9afb60498a284007a90f928ff8098f9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.750699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9afb60498a284007a90f928ff8098f9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.769631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9afb60498a284007a90f928ff8098f9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.771149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:16.782826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9afb60498a284007a90f928ff8098f9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.791576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9afb60498a284007a90f928ff8098f9c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:16.801083 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2720/5445] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:16 2022] DELETE /identity/v3/users/9afb60498a284007a90f928ff8098f9c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:16.806036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc82ed-1261-4b74-afbf-3d907effd562 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:16.806451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc82ed-1261-4b74-afbf-3d907effd562 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:16.807326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc82ed-1261-4b74-afbf-3d907effd562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:16.840228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c7cc82ed-1261-4b74-afbf-3d907effd562 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:16.840228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c7cc82ed-1261-4b74-afbf-3d907effd562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:16.982618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=8 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:17.065190 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2726/5446] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:11:17.078402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:17.079920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:17.115743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136714f-a235-4ee8-8416-f3d43284db97 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:17.116317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136714f-a235-4ee8-8416-f3d43284db97 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:17.116317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136714f-a235-4ee8-8416-f3d43284db97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:17.156854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3136714f-a235-4ee8-8416-f3d43284db97 None None] MFA Rules not processed for user `620c9fff9f76419d8e09edef7aa16b86`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:17.159268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3136714f-a235-4ee8-8416-f3d43284db97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:17.169630 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2727/5447] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:17 2022] POST /identity/v3/auth/tokens => generated 329 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 21:11:17.173723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136714f-a235-4ee8-8416-f3d43284db97 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:17.175295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3136714f-a235-4ee8-8416-f3d43284db97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:17.232221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c0ad-f632-46f1-83da-dd825da0a1e5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:17.243759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecb6d9-6e6a-4e22-aeff-aed543f05d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:17.247199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecb6d9-6e6a-4e22-aeff-aed543f05d28 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:17.248953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecb6d9-6e6a-4e22-aeff-aed543f05d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:17.249482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecb6d9-6e6a-4e22-aeff-aed543f05d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:17.288289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecb6d9-6e6a-4e22-aeff-aed543f05d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:17.290793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecb6d9-6e6a-4e22-aeff-aed543f05d28 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:17.308628 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2721/5448] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:17.334598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136714f-a235-4ee8-8416-f3d43284db97 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:17.339599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3ab8c-6be0-412a-8b23-d5333abe55ef tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:17.340421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3ab8c-6be0-412a-8b23-d5333abe55ef tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:17.340707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3ab8c-6be0-412a-8b23-d5333abe55ef tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:17.341027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3ab8c-6be0-412a-8b23-d5333abe55ef tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:17.348973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3ab8c-6be0-412a-8b23-d5333abe55ef tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:17.353197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3ab8c-6be0-412a-8b23-d5333abe55ef tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:17.368066 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2728/5449] 10.222.0.16 () {56 vars in 1263 bytes} [Sat Oct 15 21:11:17 2022] GET /identity/v3/auth/tokens => generated 329 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 15 21:11:17.370768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecb6d9-6e6a-4e22-aeff-aed543f05d28 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:17.391065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecb6d9-6e6a-4e22-aeff-aed543f05d28 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:17.400235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:17.401408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:17.401948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:17.402537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/620c9fff9f76419d8e09edef7aa16b86` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:17.404380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:delete_user(user_id=620c9fff9f76419d8e09edef7aa16b86)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:17.408643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:17.490395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '620c9fff9f76419d8e09edef7aa16b86'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:17.500179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '620c9fff9f76419d8e09edef7aa16b86'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:17.533621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '620c9fff9f76419d8e09edef7aa16b86'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:17.547961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '620c9fff9f76419d8e09edef7aa16b86'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:17.549221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '620c9fff9f76419d8e09edef7aa16b86'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:17.559731 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2722/5450] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:17 2022] DELETE /identity/v3/users/620c9fff9f76419d8e09edef7aa16b86 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:17.565012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3ab8c-6be0-412a-8b23-d5333abe55ef tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:17.577596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3ab8c-6be0-412a-8b23-d5333abe55ef tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:17.687654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd6516-92ed-4a49-8316-4b53519ca489 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:17.688330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd6516-92ed-4a49-8316-4b53519ca489 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:17.688790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd6516-92ed-4a49-8316-4b53519ca489 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:17.689176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd6516-92ed-4a49-8316-4b53519ca489 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:17.689746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd6516-92ed-4a49-8316-4b53519ca489 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:17.692701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd6516-92ed-4a49-8316-4b53519ca489 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:17.734840 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2729/5451] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:17 2022] POST /identity/v3/users => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:17.736498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:17.738910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:17.758029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491c8f0-01b5-4d55-b273-c4a1837b28e3 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:17.763416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31287e-3b9d-4d59-aaba-94518d02aabc tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:17.764814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31287e-3b9d-4d59-aaba-94518d02aabc tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:17.765005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31287e-3b9d-4d59-aaba-94518d02aabc tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:17.765498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31287e-3b9d-4d59-aaba-94518d02aabc tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:17.766217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31287e-3b9d-4d59-aaba-94518d02aabc tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:17.769663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31287e-3b9d-4d59-aaba-94518d02aabc tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:17.815296 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2723/5452] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:17 2022] POST /identity/v3/projects => generated 359 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:17.822141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd6516-92ed-4a49-8316-4b53519ca489 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:17.838512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd6516-92ed-4a49-8316-4b53519ca489 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:17.840369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31287e-3b9d-4d59-aaba-94518d02aabc tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:17.843593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7f31287e-3b9d-4d59-aaba-94518d02aabc tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:17.917367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:17.941776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d0416-cd86-4586-a513-4e6feb920539 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:17.942075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=3,keystone:DELETE=11,keystone:SELECT=86 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:17.942937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d0416-cd86-4586-a513-4e6feb920539 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:17.945298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d0416-cd86-4586-a513-4e6feb920539 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:17.945811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d0416-cd86-4586-a513-4e6feb920539 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/56f82eaa27744e20863ddbf47be4aa51` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:17.967389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d0416-cd86-4586-a513-4e6feb920539 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:update_user(user_id=56f82eaa27744e20863ddbf47be4aa51)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:17.970030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d0416-cd86-4586-a513-4e6feb920539 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:17.981163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31287e-3b9d-4d59-aaba-94518d02aabc tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:18.008740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:18.009578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:18.010475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:18.010669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] PATH_INFO: `/v3/projects/545df810b69b43a28229f7cbe38ff5cd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:18.012450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=545df810b69b43a28229f7cbe38ff5cd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:18.015692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:18.032364 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2730/5453] 10.222.0.16 () {60 vars in 1244 bytes} [Sat Oct 15 21:11:17 2022] PATCH /identity/v3/users/56f82eaa27744e20863ddbf47be4aa51 => generated 524 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:18.035980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d0416-cd86-4586-a513-4e6feb920539 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:18.047672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d0416-cd86-4586-a513-4e6feb920539 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:18.054424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f485d2-92b0-4a55-a0b3-7505a7891cc9 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:18.055063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f485d2-92b0-4a55-a0b3-7505a7891cc9 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:18.055331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f485d2-92b0-4a55-a0b3-7505a7891cc9 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:18.055613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f485d2-92b0-4a55-a0b3-7505a7891cc9 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/56f82eaa27744e20863ddbf47be4aa51` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:18.072579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f485d2-92b0-4a55-a0b3-7505a7891cc9 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:get_user(user_id=56f82eaa27744e20863ddbf47be4aa51)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:18.074937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f485d2-92b0-4a55-a0b3-7505a7891cc9 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:18.076900 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2731/5454] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:11:18 2022] GET /identity/v3/users/56f82eaa27744e20863ddbf47be4aa51 => generated 390 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:11:18.083033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f485d2-92b0-4a55-a0b3-7505a7891cc9 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:18.096315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f485d2-92b0-4a55-a0b3-7505a7891cc9 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:18.109109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '545df810b69b43a28229f7cbe38ff5cd', 'request_id': 'req-3245bddf-4224-409b-b215-5501f5ab3c16'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.124112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '545df810b69b43a28229f7cbe38ff5cd', 'request_id': 'req-3245bddf-4224-409b-b215-5501f5ab3c16'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.138341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '545df810b69b43a28229f7cbe38ff5cd', 'request_id': 'req-3245bddf-4224-409b-b215-5501f5ab3c16'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.155888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] The token cache is being invalidate because project 545df810b69b43a28229f7cbe38ff5cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:18.156190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.157849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.165872 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2724/5455] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:17 2022] DELETE /identity/v3/projects/545df810b69b43a28229f7cbe38ff5cd => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:18.169416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:18.172914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:18.185415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:18.186873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:18.187954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:18.189677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/projects/7b7e9605d77246d2b4f1d67bf52e6d58` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:18.196953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7b7e9605d77246d2b4f1d67bf52e6d58)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:18.203181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:18.288219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b7e9605d77246d2b4f1d67bf52e6d58', 'request_id': 'req-0cd86846-a8a8-41be-975d-ed53b1d74883'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.302072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b7e9605d77246d2b4f1d67bf52e6d58', 'request_id': 'req-0cd86846-a8a8-41be-975d-ed53b1d74883'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.310932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b7e9605d77246d2b4f1d67bf52e6d58', 'request_id': 'req-0cd86846-a8a8-41be-975d-ed53b1d74883'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.317357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245bddf-4224-409b-b215-5501f5ab3c16 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:18.328049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] The token cache is being invalidate because project 7b7e9605d77246d2b4f1d67bf52e6d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:18.328324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.330072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.332835 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2732/5456] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:18 2022] DELETE /identity/v3/projects/7b7e9605d77246d2b4f1d67bf52e6d58 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:18.338070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:18.339840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:18.421956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:UPDATE=2,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:18.432108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:18.433395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:18.433817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:18.435663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] PATH_INFO: `/v3/users/047464bfd2db48328dd289edab484a5d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:18.437406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=047464bfd2db48328dd289edab484a5d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:18.441249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:18.459271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd86846-a8a8-41be-975d-ed53b1d74883 tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:18.470106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a87f4de367854e459c8f438edb5abb4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:18.470802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:18.471158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:18.471429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] PATH_INFO: `/v3/users/56f82eaa27744e20863ddbf47be4aa51` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:18.472936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorizing `identity:delete_user(user_id=56f82eaa27744e20863ddbf47be4aa51)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:18.473721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:18.478647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:18.526539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047464bfd2db48328dd289edab484a5d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.539884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047464bfd2db48328dd289edab484a5d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.544328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f82eaa27744e20863ddbf47be4aa51'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.560242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f82eaa27744e20863ddbf47be4aa51'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.561347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047464bfd2db48328dd289edab484a5d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.570994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f82eaa27744e20863ddbf47be4aa51'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.576013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047464bfd2db48328dd289edab484a5d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.579135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047464bfd2db48328dd289edab484a5d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.580605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f82eaa27744e20863ddbf47be4aa51'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.581345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3663b0db-3884-4bc6-93b9-28e69621c1bd tempest-UsersV3TestJSON-990749543 tempest-UsersV3TestJSON-990749543-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f82eaa27744e20863ddbf47be4aa51'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.591084 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2725/5457] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:18 2022] DELETE /identity/v3/users/047464bfd2db48328dd289edab484a5d => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:18.596757 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2733/5458] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:18 2022] DELETE /identity/v3/users/56f82eaa27744e20863ddbf47be4aa51 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:18.598275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:18.600716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:18.703244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d3a77-4442-4a03-81ea-6d409a5534cc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:18.785929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:18.786567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:18.786907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:18.787266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] PATH_INFO: `/v3/users/0902a3cd5c044bd0af8a9a5436299c85` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:18.803360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0902a3cd5c044bd0af8a9a5436299c85)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:18.805468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:18.869539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:18.877000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0902a3cd5c044bd0af8a9a5436299c85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.887300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0902a3cd5c044bd0af8a9a5436299c85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.915433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0902a3cd5c044bd0af8a9a5436299c85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.927879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0902a3cd5c044bd0af8a9a5436299c85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.928579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0902a3cd5c044bd0af8a9a5436299c85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:18.940679 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2726/5459] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:18 2022] DELETE /identity/v3/users/0902a3cd5c044bd0af8a9a5436299c85 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:18.947704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc27631-305b-4a1a-a145-4740cf527c69 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:18.948168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc27631-305b-4a1a-a145-4740cf527c69 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:18.948661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc27631-305b-4a1a-a145-4740cf527c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:18.976784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0dc27631-305b-4a1a-a145-4740cf527c69 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:18.979283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0dc27631-305b-4a1a-a145-4740cf527c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:19.092962 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2734/5460] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:19.115018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:19.117143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:19.288424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ee43-7a50-4fda-a47b-d2f5a3f1873d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:19.294561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcdff8-0657-4d43-ac16-46c8bbb2843b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:19.295824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcdff8-0657-4d43-ac16-46c8bbb2843b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:19.296195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcdff8-0657-4d43-ac16-46c8bbb2843b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:19.296545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcdff8-0657-4d43-ac16-46c8bbb2843b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:19.308071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcdff8-0657-4d43-ac16-46c8bbb2843b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:19.309529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcdff8-0657-4d43-ac16-46c8bbb2843b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:19.320821 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2727/5461] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:19.345712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc27631-305b-4a1a-a145-4740cf527c69 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:19.348773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0dc27631-305b-4a1a-a145-4740cf527c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:19.425023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc27631-305b-4a1a-a145-4740cf527c69 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:19.442472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:19.443810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:19.447313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:19.448740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] PATH_INFO: `/v3/projects/98f5e9d5f9cb4d678b5e0339e4fc1e95` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:19.453476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f5e9d5f9cb4d678b5e0339e4fc1e95)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:19.462154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:19.607728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'request_id': 'req-07be3b63-8d41-488e-90c9-b1687b548ebe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:19.612277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:19.626991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'request_id': 'req-07be3b63-8d41-488e-90c9-b1687b548ebe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:19.639571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f5e9d5f9cb4d678b5e0339e4fc1e95', 'request_id': 'req-07be3b63-8d41-488e-90c9-b1687b548ebe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:19.661475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] The token cache is being invalidate because project 98f5e9d5f9cb4d678b5e0339e4fc1e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:19.662070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:19.663206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-07be3b63-8d41-488e-90c9-b1687b548ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:19.665467 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2735/5462] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:19 2022] DELETE /identity/v3/projects/98f5e9d5f9cb4d678b5e0339e4fc1e95 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:19.672494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67d177-178c-4586-acb4-a59702ba2953 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:19.672781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67d177-178c-4586-acb4-a59702ba2953 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:19.673135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67d177-178c-4586-acb4-a59702ba2953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:19.698873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9d67d177-178c-4586-acb4-a59702ba2953 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:19.700145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9d67d177-178c-4586-acb4-a59702ba2953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:19.826517 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2728/5463] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:19.835103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc38854-d43b-4f54-aab6-57d2dba44763 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:19.835655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc38854-d43b-4f54-aab6-57d2dba44763 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:19.835941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc38854-d43b-4f54-aab6-57d2dba44763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:19.884864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8cc38854-d43b-4f54-aab6-57d2dba44763 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:19.886314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8cc38854-d43b-4f54-aab6-57d2dba44763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:19.904260 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2736/5464] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:19.908841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67d177-178c-4586-acb4-a59702ba2953 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:19.925680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67d177-178c-4586-acb4-a59702ba2953 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:19.938628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02ebf0-2128-4917-9a06-61b48bb78607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:19.938628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02ebf0-2128-4917-9a06-61b48bb78607 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:19.938951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02ebf0-2128-4917-9a06-61b48bb78607 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:19.943632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02ebf0-2128-4917-9a06-61b48bb78607 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:19.944125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02ebf0-2128-4917-9a06-61b48bb78607 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:19.944702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02ebf0-2128-4917-9a06-61b48bb78607 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:19.946383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02ebf0-2128-4917-9a06-61b48bb78607 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:19.964800 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2729/5465] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:19.969277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc38854-d43b-4f54-aab6-57d2dba44763 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:19.983972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc38854-d43b-4f54-aab6-57d2dba44763 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:19.990466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88a6df-7fc6-44b7-8609-5bd1a3a4776a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:19.991159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88a6df-7fc6-44b7-8609-5bd1a3a4776a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:19.991496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88a6df-7fc6-44b7-8609-5bd1a3a4776a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:19.993356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88a6df-7fc6-44b7-8609-5bd1a3a4776a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:19.994107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88a6df-7fc6-44b7-8609-5bd1a3a4776a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:19.996998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88a6df-7fc6-44b7-8609-5bd1a3a4776a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:20.048183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02ebf0-2128-4917-9a06-61b48bb78607 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:20.065100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2c02ebf0-2128-4917-9a06-61b48bb78607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:20.075680 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2737/5466] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:19 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:20.077307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88a6df-7fc6-44b7-8609-5bd1a3a4776a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:20.115785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02ebf0-2128-4917-9a06-61b48bb78607 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:20.217168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:20.253512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:20.254359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.254643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.254912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] PATH_INFO: `/v3/users/0081064aa9b5484cb8048b675750dfef` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.257739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0081064aa9b5484cb8048b675750dfef)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:20.259708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:20.263644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88a6df-7fc6-44b7-8609-5bd1a3a4776a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:20.275394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304fb09-290c-41e2-bde2-8dc504184117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:20.276016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304fb09-290c-41e2-bde2-8dc504184117 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.276285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304fb09-290c-41e2-bde2-8dc504184117 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.276555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304fb09-290c-41e2-bde2-8dc504184117 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.277226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304fb09-290c-41e2-bde2-8dc504184117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:20.279174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304fb09-290c-41e2-bde2-8dc504184117 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:20.316286 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2738/5467] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:20 2022] POST /identity/v3/users => generated 303 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:20.321309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304fb09-290c-41e2-bde2-8dc504184117 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:20.334275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304fb09-290c-41e2-bde2-8dc504184117 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:20.344520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0081064aa9b5484cb8048b675750dfef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:20.346496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4099fdb-1367-45df-ad4a-16fa5d698d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:20.349412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4099fdb-1367-45df-ad4a-16fa5d698d9b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.349907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4099fdb-1367-45df-ad4a-16fa5d698d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.350284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4099fdb-1367-45df-ad4a-16fa5d698d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.351431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4099fdb-1367-45df-ad4a-16fa5d698d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:20.354648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4099fdb-1367-45df-ad4a-16fa5d698d9b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:20.371640 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2739/5468] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:11:20.372828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0081064aa9b5484cb8048b675750dfef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:20.378230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4099fdb-1367-45df-ad4a-16fa5d698d9b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:20.392249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4099fdb-1367-45df-ad4a-16fa5d698d9b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:20.395666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0081064aa9b5484cb8048b675750dfef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:20.403870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dad2c2-cb86-4fa7-9cdb-1c4258fb2923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:20.403870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad2c2-cb86-4fa7-9cdb-1c4258fb2923 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.403870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad2c2-cb86-4fa7-9cdb-1c4258fb2923 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.403870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad2c2-cb86-4fa7-9cdb-1c4258fb2923 admin admin] PATH_INFO: `/v3/projects/7ff4e461d2ca44a4bd3e152a88e81d7e/users/9d2493460f524acd8c3deec95ac4e33c/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.415883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0081064aa9b5484cb8048b675750dfef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:20.416744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a5e427f5-bbd2-4b30-ac04-1abe069767d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0081064aa9b5484cb8048b675750dfef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:20.431339 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2730/5469] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:20 2022] DELETE /identity/v3/users/0081064aa9b5484cb8048b675750dfef => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:20.432851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dad2c2-cb86-4fa7-9cdb-1c4258fb2923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff4e461d2ca44a4bd3e152a88e81d7e, user_id=9d2493460f524acd8c3deec95ac4e33c, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:20.434712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1ee4a-6db7-43a5-88a8-9a04c496b417 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.435021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1ee4a-6db7-43a5-88a8-9a04c496b417 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.435298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1ee4a-6db7-43a5-88a8-9a04c496b417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.435553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dad2c2-cb86-4fa7-9cdb-1c4258fb2923 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:20.460156 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2740/5470] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:20 2022] PUT /identity/v3/projects/7ff4e461d2ca44a4bd3e152a88e81d7e/users/9d2493460f524acd8c3deec95ac4e33c/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:20.469361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dad2c2-cb86-4fa7-9cdb-1c4258fb2923 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:20.475230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d8f1ee4a-6db7-43a5-88a8-9a04c496b417 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:20.476907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8f1ee4a-6db7-43a5-88a8-9a04c496b417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:20.482354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f8dad2c2-cb86-4fa7-9cdb-1c4258fb2923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:20.571241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dad2c2-cb86-4fa7-9cdb-1c4258fb2923 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:20.576421 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2731/5471] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:20.577589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1ee4a-6db7-43a5-88a8-9a04c496b417 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:20.579255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8f1ee4a-6db7-43a5-88a8-9a04c496b417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:20.587671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499773f-ef2e-41e3-a5ec-e9da8a59b7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:20.589084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499773f-ef2e-41e3-a5ec-e9da8a59b7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.589620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499773f-ef2e-41e3-a5ec-e9da8a59b7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.590378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499773f-ef2e-41e3-a5ec-e9da8a59b7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.591105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499773f-ef2e-41e3-a5ec-e9da8a59b7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:20.594017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499773f-ef2e-41e3-a5ec-e9da8a59b7f2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:20.609650 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2741/5472] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:20.611487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499773f-ef2e-41e3-a5ec-e9da8a59b7f2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:20.618333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:20.626236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499773f-ef2e-41e3-a5ec-e9da8a59b7f2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:20.638477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b436ee-4868-4b91-bd10-874f048c20ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:20.639513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b436ee-4868-4b91-bd10-874f048c20ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.640007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b436ee-4868-4b91-bd10-874f048c20ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.640603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b436ee-4868-4b91-bd10-874f048c20ff admin admin] PATH_INFO: `/v3/projects/7ff4e461d2ca44a4bd3e152a88e81d7e/users/9d2493460f524acd8c3deec95ac4e33c/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.643024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b436ee-4868-4b91-bd10-874f048c20ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff4e461d2ca44a4bd3e152a88e81d7e, user_id=9d2493460f524acd8c3deec95ac4e33c, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:20.646071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b436ee-4868-4b91-bd10-874f048c20ff admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:20.664444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1ee4a-6db7-43a5-88a8-9a04c496b417 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:20.670139 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2742/5473] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:20 2022] PUT /identity/v3/projects/7ff4e461d2ca44a4bd3e152a88e81d7e/users/9d2493460f524acd8c3deec95ac4e33c/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:20.671997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcb573-2beb-41d8-866d-6702386823c4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.672280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcb573-2beb-41d8-866d-6702386823c4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.672553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcb573-2beb-41d8-866d-6702386823c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.693908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:20.710806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4fdcb573-2beb-41d8-866d-6702386823c4 None None] MFA Rules not processed for user `9d2493460f524acd8c3deec95ac4e33c`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:20.712475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4fdcb573-2beb-41d8-866d-6702386823c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:20.744907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb6b58-703c-4bd6-b4e4-37cffc20bedc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:20.745598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb6b58-703c-4bd6-b4e4-37cffc20bedc service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.745927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb6b58-703c-4bd6-b4e4-37cffc20bedc service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.746398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb6b58-703c-4bd6-b4e4-37cffc20bedc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.835516 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2743/5474] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:11:20.835516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcb573-2beb-41d8-866d-6702386823c4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:20.835828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb6b58-703c-4bd6-b4e4-37cffc20bedc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:20.836607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb6b58-703c-4bd6-b4e4-37cffc20bedc service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:20.836823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4fdcb573-2beb-41d8-866d-6702386823c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:20.848636 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2732/5475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:20.850293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb6b58-703c-4bd6-b4e4-37cffc20bedc service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:20.854477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcb573-2beb-41d8-866d-6702386823c4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:20.864276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:20.865486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb6b58-703c-4bd6-b4e4-37cffc20bedc service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:20.866955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.868208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.868584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] PATH_INFO: `/v3/projects/f48303fa2cb94246a8bc8abe3c3fc58d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.871034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48303fa2cb94246a8bc8abe3c3fc58d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:20.876604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:20.882596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5c8bb-690e-4318-aa14-8ac7d5e5f5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:20.885296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5c8bb-690e-4318-aa14-8ac7d5e5f5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:20.885749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5c8bb-690e-4318-aa14-8ac7d5e5f5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:20.886492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5c8bb-690e-4318-aa14-8ac7d5e5f5a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:20.887562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5c8bb-690e-4318-aa14-8ac7d5e5f5a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:20.890583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5c8bb-690e-4318-aa14-8ac7d5e5f5a4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:20.968978 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2733/5476] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:20 2022] POST /identity/v3/projects => generated 393 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:20.971512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5c8bb-690e-4318-aa14-8ac7d5e5f5a4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:21.000836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48303fa2cb94246a8bc8abe3c3fc58d', 'request_id': 'req-32677576-7cc7-4f94-99c3-e313cc4cb8fc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.024699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48303fa2cb94246a8bc8abe3c3fc58d', 'request_id': 'req-32677576-7cc7-4f94-99c3-e313cc4cb8fc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.033747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48303fa2cb94246a8bc8abe3c3fc58d', 'request_id': 'req-32677576-7cc7-4f94-99c3-e313cc4cb8fc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.058940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] The token cache is being invalidate because project f48303fa2cb94246a8bc8abe3c3fc58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:21.059337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.061138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.064512 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2744/5477] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:20 2022] DELETE /identity/v3/projects/f48303fa2cb94246a8bc8abe3c3fc58d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:21.078073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:21.084187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:21.148566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5c8bb-690e-4318-aa14-8ac7d5e5f5a4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:21.232758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:21.249625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32677576-7cc7-4f94-99c3-e313cc4cb8fc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:21.283606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:21.284403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:21.286285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:21.288680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:21.290394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] PATH_INFO: `/v3/users/a87f4de367854e459c8f438edb5abb4a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:21.298393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a87f4de367854e459c8f438edb5abb4a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:21.299512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee52cc-c3d3-426c-adb6-d68884024d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:21.300739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee52cc-c3d3-426c-adb6-d68884024d54 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:21.301536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee52cc-c3d3-426c-adb6-d68884024d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:21.304382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee52cc-c3d3-426c-adb6-d68884024d54 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:21.310479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee52cc-c3d3-426c-adb6-d68884024d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:21.313320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee52cc-c3d3-426c-adb6-d68884024d54 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:21.314085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:21.362264 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2734/5478] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:20 2022] POST /identity/v3/users => generated 297 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:21.365744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee52cc-c3d3-426c-adb6-d68884024d54 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:21.367864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bfee52cc-c3d3-426c-adb6-d68884024d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:21.392585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee52cc-c3d3-426c-adb6-d68884024d54 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:21.413098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e9f87-3270-4a9d-af0b-ec11374151d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:21.413098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9f87-3270-4a9d-af0b-ec11374151d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:21.413098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9f87-3270-4a9d-af0b-ec11374151d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:21.414123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9f87-3270-4a9d-af0b-ec11374151d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:21.416593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e9f87-3270-4a9d-af0b-ec11374151d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:21.418316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e9f87-3270-4a9d-af0b-ec11374151d8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:21.431778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a87f4de367854e459c8f438edb5abb4a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.451582 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2735/5479] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:11:21.453320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e9f87-3270-4a9d-af0b-ec11374151d8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:21.461208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a87f4de367854e459c8f438edb5abb4a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.470706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e9f87-3270-4a9d-af0b-ec11374151d8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:21.479683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a064f13-6403-459a-bfee-2619d00f873f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:21.480353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a064f13-6403-459a-bfee-2619d00f873f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:21.480637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a064f13-6403-459a-bfee-2619d00f873f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:21.480905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a064f13-6403-459a-bfee-2619d00f873f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:21.481690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a064f13-6403-459a-bfee-2619d00f873f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:21.484152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a064f13-6403-459a-bfee-2619d00f873f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:21.494893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a87f4de367854e459c8f438edb5abb4a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.513593 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2736/5480] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:21.515972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a064f13-6403-459a-bfee-2619d00f873f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:21.517658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a87f4de367854e459c8f438edb5abb4a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.520479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a87f4de367854e459c8f438edb5abb4a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.540403 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2745/5481] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:21 2022] DELETE /identity/v3/users/a87f4de367854e459c8f438edb5abb4a => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:21.540403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:21.543160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:21.546110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a064f13-6403-459a-bfee-2619d00f873f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:21.652701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5cb791-4bf9-4aa8-be0d-f2ce78dfafd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:21.653839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5cb791-4bf9-4aa8-be0d-f2ce78dfafd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:21.654576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5cb791-4bf9-4aa8-be0d-f2ce78dfafd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:21.655582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5cb791-4bf9-4aa8-be0d-f2ce78dfafd3 admin admin] PATH_INFO: `/v3/projects/e0a213976c664c1a8a409c8355c31cd8/users/fadcdce5062147249ebce68227767855/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:21.675851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5cb791-4bf9-4aa8-be0d-f2ce78dfafd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a213976c664c1a8a409c8355c31cd8, user_id=fadcdce5062147249ebce68227767855, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:21.683386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5cb791-4bf9-4aa8-be0d-f2ce78dfafd3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:21.694883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22947c84-f0f1-4a9a-97d8-f5e930018aee admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:21.707839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:21.710697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:21.712245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:21.712705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] PATH_INFO: `/v3/projects/1b381f47153c4515b05b26989824c9df` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:21.715622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b381f47153c4515b05b26989824c9df)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:21.721206 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2737/5482] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:21 2022] PUT /identity/v3/projects/e0a213976c664c1a8a409c8355c31cd8/users/fadcdce5062147249ebce68227767855/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:21.721835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:21.726302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91a934-8793-4ff7-a925-cfb53a6f5fe3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:21.726709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91a934-8793-4ff7-a925-cfb53a6f5fe3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:21.727084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91a934-8793-4ff7-a925-cfb53a6f5fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:21.814685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6e91a934-8793-4ff7-a925-cfb53a6f5fe3 None None] MFA Rules not processed for user `fadcdce5062147249ebce68227767855`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:21.820053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e91a934-8793-4ff7-a925-cfb53a6f5fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:21.919885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b381f47153c4515b05b26989824c9df', 'request_id': 'req-10857277-a459-4c89-b0e0-2dce6c3e390b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.929556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b381f47153c4515b05b26989824c9df', 'request_id': 'req-10857277-a459-4c89-b0e0-2dce6c3e390b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.939845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b381f47153c4515b05b26989824c9df', 'request_id': 'req-10857277-a459-4c89-b0e0-2dce6c3e390b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.983622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] The token cache is being invalidate because project 1b381f47153c4515b05b26989824c9df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:21.985544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.987211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:21.993563 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2746/5483] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:21 2022] DELETE /identity/v3/projects/1b381f47153c4515b05b26989824c9df => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:21.995939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:22.003356 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2738/5484] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:22.003563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:22.018058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91a934-8793-4ff7-a925-cfb53a6f5fe3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:22.019334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:22.019538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e91a934-8793-4ff7-a925-cfb53a6f5fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:22.088559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:22.113535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91a934-8793-4ff7-a925-cfb53a6f5fe3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:22.141904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10857277-a459-4c89-b0e0-2dce6c3e390b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:22.148940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:22.149708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:22.149984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:22.150249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] PATH_INFO: `/v3/users/582d27fd5cd447e5ad9d50b06cd0aca8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:22.152117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] RBAC: Authorizing `identity:delete_user(user_id=582d27fd5cd447e5ad9d50b06cd0aca8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:22.154869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:22.187485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f341a-0b52-4ac9-b3ea-72afdc34772d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:22.188103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f341a-0b52-4ac9-b3ea-72afdc34772d service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:22.188884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f341a-0b52-4ac9-b3ea-72afdc34772d service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:22.192610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f341a-0b52-4ac9-b3ea-72afdc34772d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:22.198880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bf1f341a-0b52-4ac9-b3ea-72afdc34772d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:22.275480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '582d27fd5cd447e5ad9d50b06cd0aca8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:22.304951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '582d27fd5cd447e5ad9d50b06cd0aca8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:22.322378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '582d27fd5cd447e5ad9d50b06cd0aca8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:22.330208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '582d27fd5cd447e5ad9d50b06cd0aca8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:22.331006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '582d27fd5cd447e5ad9d50b06cd0aca8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:22.350175 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2747/5485] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:21 2022] DELETE /identity/v3/users/582d27fd5cd447e5ad9d50b06cd0aca8 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:22.379671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f341a-0b52-4ac9-b3ea-72afdc34772d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:22.381631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f341a-0b52-4ac9-b3ea-72afdc34772d service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:22.394539 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2739/5486] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:22 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:22.648693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:22.651365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:22.850370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a59f6-346b-4c4d-9340-2dccdd65611c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:22.862473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:22.863091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:22.863630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:22.864506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] PATH_INFO: `/v3/projects/dba4298b957040af9a1c9b2ac6cd9ff7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:22.893179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba4298b957040af9a1c9b2ac6cd9ff7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:22.895161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:22.977270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:23.040965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'request_id': 'req-c372c449-868d-45ec-b506-da208d891033'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:23.053651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f341a-0b52-4ac9-b3ea-72afdc34772d service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:23.070326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f341a-0b52-4ac9-b3ea-72afdc34772d service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:23.081653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'request_id': 'req-c372c449-868d-45ec-b506-da208d891033'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:23.106906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba4298b957040af9a1c9b2ac6cd9ff7', 'request_id': 'req-c372c449-868d-45ec-b506-da208d891033'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:23.147853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] The token cache is being invalidate because project dba4298b957040af9a1c9b2ac6cd9ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:23.148231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:23.148996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c372c449-868d-45ec-b506-da208d891033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:23.151730 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2748/5487] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:22 2022] DELETE /identity/v3/projects/dba4298b957040af9a1c9b2ac6cd9ff7 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:23.162222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f65f-1f5d-47ec-a3f7-27bdacbb4082 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:23.162739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f65f-1f5d-47ec-a3f7-27bdacbb4082 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:23.163115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f65f-1f5d-47ec-a3f7-27bdacbb4082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:23.221629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:23.222271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:23.222551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:23.222816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] PATH_INFO: `/v3/projects/6d22e36542a64de18e6445ee19bd9990` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:23.226596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d22e36542a64de18e6445ee19bd9990)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:23.227311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8db5f65f-1f5d-47ec-a3f7-27bdacbb4082 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:23.228505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:23.231789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8db5f65f-1f5d-47ec-a3f7-27bdacbb4082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:23.273673 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2749/5488] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:11:23.278481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c0bc4-2cae-4a52-86bb-92dc0ee975cc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:23.278481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c0bc4-2cae-4a52-86bb-92dc0ee975cc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:23.278481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c0bc4-2cae-4a52-86bb-92dc0ee975cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:23.381767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3f2c0bc4-2cae-4a52-86bb-92dc0ee975cc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:23.382994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3f2c0bc4-2cae-4a52-86bb-92dc0ee975cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:23.445315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d22e36542a64de18e6445ee19bd9990', 'request_id': 'req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:23.458655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:23.473293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d22e36542a64de18e6445ee19bd9990', 'request_id': 'req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:23.506412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d22e36542a64de18e6445ee19bd9990', 'request_id': 'req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:23.568703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] The token cache is being invalidate because project 6d22e36542a64de18e6445ee19bd9990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:23.569600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:23.578272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-70cdafdb-f7e7-4c7b-99a1-0e22c51fa39b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:23.584666 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2740/5489] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:23 2022] DELETE /identity/v3/projects/6d22e36542a64de18e6445ee19bd9990 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:23.591666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba167b20-b18b-4d79-926a-884ed4958439 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:23.592079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba167b20-b18b-4d79-926a-884ed4958439 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:23.592446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba167b20-b18b-4d79-926a-884ed4958439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:23.610659 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2750/5490] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:23.615966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c0bc4-2cae-4a52-86bb-92dc0ee975cc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:23.662513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c0bc4-2cae-4a52-86bb-92dc0ee975cc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:23.683923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ba167b20-b18b-4d79-926a-884ed4958439 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:23.699171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ba167b20-b18b-4d79-926a-884ed4958439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:23.720593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2da52-e167-45fb-82ce-8f47e83a1fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:23.730122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2da52-e167-45fb-82ce-8f47e83a1fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:23.730520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2da52-e167-45fb-82ce-8f47e83a1fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:23.730843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2da52-e167-45fb-82ce-8f47e83a1fa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:23.741276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2da52-e167-45fb-82ce-8f47e83a1fa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:23.741587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2da52-e167-45fb-82ce-8f47e83a1fa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:23.743266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2da52-e167-45fb-82ce-8f47e83a1fa1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:23.760831 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2741/5491] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:23.762763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e13469-8383-4e98-93b9-6a98c0e12449 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:23.763039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e13469-8383-4e98-93b9-6a98c0e12449 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:23.763246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e13469-8383-4e98-93b9-6a98c0e12449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:23.830733 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2751/5492] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:23.836554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2da52-e167-45fb-82ce-8f47e83a1fa1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:23.841194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-62e13469-8383-4e98-93b9-6a98c0e12449 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:23.842439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-62e13469-8383-4e98-93b9-6a98c0e12449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:23.866911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2da52-e167-45fb-82ce-8f47e83a1fa1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:23.874979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d239bc4-ce9b-4b7c-a191-a0a0e98a5f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:23.876552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d239bc4-ce9b-4b7c-a191-a0a0e98a5f21 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:23.876997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d239bc4-ce9b-4b7c-a191-a0a0e98a5f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:23.877535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d239bc4-ce9b-4b7c-a191-a0a0e98a5f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:23.878992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d239bc4-ce9b-4b7c-a191-a0a0e98a5f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:23.881896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d239bc4-ce9b-4b7c-a191-a0a0e98a5f21 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:23.887645 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2742/5493] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:23.891574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e13469-8383-4e98-93b9-6a98c0e12449 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:23.915544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e13469-8383-4e98-93b9-6a98c0e12449 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:23.938437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ce4fc-6d90-40ea-828c-bec937023af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:23.939050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ce4fc-6d90-40ea-828c-bec937023af2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:23.939321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ce4fc-6d90-40ea-828c-bec937023af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:23.939582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ce4fc-6d90-40ea-828c-bec937023af2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:23.940229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ce4fc-6d90-40ea-828c-bec937023af2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:23.940523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ce4fc-6d90-40ea-828c-bec937023af2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:23.942150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ce4fc-6d90-40ea-828c-bec937023af2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:23.967215 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2752/5494] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:23 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:23.970092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d239bc4-ce9b-4b7c-a191-a0a0e98a5f21 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:23.994984 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2743/5495] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:11:24.016118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ce4fc-6d90-40ea-828c-bec937023af2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:24.052345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ce4fc-6d90-40ea-828c-bec937023af2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:24.130882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d239bc4-ce9b-4b7c-a191-a0a0e98a5f21 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:24.143894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a509a-a0a7-4cab-a202-76a125eaf4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:24.144612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a509a-a0a7-4cab-a202-76a125eaf4af admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:24.144986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a509a-a0a7-4cab-a202-76a125eaf4af admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:24.145494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a509a-a0a7-4cab-a202-76a125eaf4af admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:24.146973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a509a-a0a7-4cab-a202-76a125eaf4af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:24.152168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a509a-a0a7-4cab-a202-76a125eaf4af admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:24.175132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c6033-9c5e-471b-a99d-8486107a5084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:24.178218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c6033-9c5e-471b-a99d-8486107a5084 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:24.180062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c6033-9c5e-471b-a99d-8486107a5084 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:24.180869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c6033-9c5e-471b-a99d-8486107a5084 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:24.181852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c6033-9c5e-471b-a99d-8486107a5084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:24.186338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c6033-9c5e-471b-a99d-8486107a5084 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:24.192115 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2753/5496] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:23 2022] POST /identity/v3/users => generated 285 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:24.192390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a509a-a0a7-4cab-a202-76a125eaf4af admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:24.205904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:24.215008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a509a-a0a7-4cab-a202-76a125eaf4af admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:24.231402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edbd26-0753-4d3a-be07-22816595ec4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:24.232099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edbd26-0753-4d3a-be07-22816595ec4c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:24.232445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edbd26-0753-4d3a-be07-22816595ec4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:24.232834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edbd26-0753-4d3a-be07-22816595ec4c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:24.233583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0edbd26-0753-4d3a-be07-22816595ec4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:24.239636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0edbd26-0753-4d3a-be07-22816595ec4c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:24.263552 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2754/5497] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:24.271644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edbd26-0753-4d3a-be07-22816595ec4c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:24.289817 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2744/5498] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:24 2022] POST /identity/v3/projects => generated 393 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:24.294555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c6033-9c5e-471b-a99d-8486107a5084 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:24.296266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edbd26-0753-4d3a-be07-22816595ec4c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:24.458201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c6033-9c5e-471b-a99d-8486107a5084 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:24.471751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34d524-1b26-4056-aa22-388428dbead6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:24.472643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34d524-1b26-4056-aa22-388428dbead6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:24.472982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34d524-1b26-4056-aa22-388428dbead6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:24.473334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34d524-1b26-4056-aa22-388428dbead6 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:24.474482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34d524-1b26-4056-aa22-388428dbead6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:24.475361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f4993-a6f2-4174-a22d-4834806da725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:24.476167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f4993-a6f2-4174-a22d-4834806da725 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:24.476543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34d524-1b26-4056-aa22-388428dbead6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:24.476979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f4993-a6f2-4174-a22d-4834806da725 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:24.478069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f4993-a6f2-4174-a22d-4834806da725 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:24.478790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f4993-a6f2-4174-a22d-4834806da725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:24.483001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f4993-a6f2-4174-a22d-4834806da725 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:24.528676 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2755/5499] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:24.532692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f4993-a6f2-4174-a22d-4834806da725 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:24.540149 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2745/5500] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:24 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:24.541405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34d524-1b26-4056-aa22-388428dbead6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:24.562256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f4993-a6f2-4174-a22d-4834806da725 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:24.571197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63c267-c289-4ed7-82dd-aa17b2d3c2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:24.572143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34d524-1b26-4056-aa22-388428dbead6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:24.572515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63c267-c289-4ed7-82dd-aa17b2d3c2c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:24.573623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63c267-c289-4ed7-82dd-aa17b2d3c2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:24.574199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63c267-c289-4ed7-82dd-aa17b2d3c2c7 admin admin] PATH_INFO: `/v3/projects/6e4ccceba0f84d49a106a2b91b02ff5e/users/1c6e17500ee34dc690a6126657f85481/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:24.582793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b07d2-46f9-4cd5-8b80-072959ccdfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:24.586438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b07d2-46f9-4cd5-8b80-072959ccdfc8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:24.586668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b07d2-46f9-4cd5-8b80-072959ccdfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:24.586958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b07d2-46f9-4cd5-8b80-072959ccdfc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:24.588760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b07d2-46f9-4cd5-8b80-072959ccdfc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:24.597207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b07d2-46f9-4cd5-8b80-072959ccdfc8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:24.606246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63c267-c289-4ed7-82dd-aa17b2d3c2c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4ccceba0f84d49a106a2b91b02ff5e, user_id=1c6e17500ee34dc690a6126657f85481, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:24.611354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63c267-c289-4ed7-82dd-aa17b2d3c2c7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:24.623207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2746/5501] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:24.629867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b07d2-46f9-4cd5-8b80-072959ccdfc8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:24.636264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ac2b07d2-46f9-4cd5-8b80-072959ccdfc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:24.664250 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2756/5502] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:24 2022] PUT /identity/v3/projects/6e4ccceba0f84d49a106a2b91b02ff5e/users/1c6e17500ee34dc690a6126657f85481/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:24.664250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63c267-c289-4ed7-82dd-aa17b2d3c2c7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:24.676185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b07d2-46f9-4cd5-8b80-072959ccdfc8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:24.707165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:24.790888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63c267-c289-4ed7-82dd-aa17b2d3c2c7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:24.801641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a44ee-206e-413b-aeda-f203758695bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:24.802713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a44ee-206e-413b-aeda-f203758695bb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:24.804258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a44ee-206e-413b-aeda-f203758695bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:24.804607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a44ee-206e-413b-aeda-f203758695bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:24.805502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a44ee-206e-413b-aeda-f203758695bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:24.809096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a44ee-206e-413b-aeda-f203758695bb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:24.834233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:24.834728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:24.835072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:24.835412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] PATH_INFO: `/v3/users/9d2493460f524acd8c3deec95ac4e33c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:24.839937 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2757/5503] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:24.839937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1a6d2-b681-4a6b-82fd-9a6d913150cb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:24.839937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1a6d2-b681-4a6b-82fd-9a6d913150cb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:24.839937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1a6d2-b681-4a6b-82fd-9a6d913150cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:24.842057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d2493460f524acd8c3deec95ac4e33c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:24.843502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:24.916189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-08c1a6d2-b681-4a6b-82fd-9a6d913150cb None None] MFA Rules not processed for user `1c6e17500ee34dc690a6126657f85481`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:24.918062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-08c1a6d2-b681-4a6b-82fd-9a6d913150cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:24.982911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d2493460f524acd8c3deec95ac4e33c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:24.997603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d2493460f524acd8c3deec95ac4e33c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:25.009362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:25.038396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d2493460f524acd8c3deec95ac4e33c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:25.073125 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2758/5504] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:25.073334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d2493460f524acd8c3deec95ac4e33c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:25.073450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1a6d2-b681-4a6b-82fd-9a6d913150cb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:25.073534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d2493460f524acd8c3deec95ac4e33c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:25.077666 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2747/5505] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:24 2022] DELETE /identity/v3/users/9d2493460f524acd8c3deec95ac4e33c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:25.078509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:25.079915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:25.095633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1a6d2-b681-4a6b-82fd-9a6d913150cb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:25.159968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3004d6-e783-4fff-8bb6-1e312d94859a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:25.165177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d2562-bdd1-48dd-b7e3-bc42c41a62b3 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:25.165847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d2562-bdd1-48dd-b7e3-bc42c41a62b3 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.166188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d2562-bdd1-48dd-b7e3-bc42c41a62b3 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.166469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d2562-bdd1-48dd-b7e3-bc42c41a62b3 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.167095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d2562-bdd1-48dd-b7e3-bc42c41a62b3 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=1c6e17500ee34dc690a6126657f85481)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:25.169104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d2562-bdd1-48dd-b7e3-bc42c41a62b3 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:25.221901 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2748/5506] 10.222.0.16 () {60 vars in 1299 bytes} [Sat Oct 15 21:11:25 2022] POST /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2 => generated 367 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:25.223594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf59cb-9596-43d4-bcef-f158b64fd47c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.224541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf59cb-9596-43d4-bcef-f158b64fd47c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.225935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf59cb-9596-43d4-bcef-f158b64fd47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.234753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621df6-fbe3-4146-b892-7c2bf639414b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:25.235523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621df6-fbe3-4146-b892-7c2bf639414b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.235843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621df6-fbe3-4146-b892-7c2bf639414b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.236115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621df6-fbe3-4146-b892-7c2bf639414b admin admin] PATH_INFO: `/v3/projects/b6313360f88643df9e2a5e9dee09d91e/users/74decb4847bb474d844b484d7b1f29b5/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.266706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd621df6-fbe3-4146-b892-7c2bf639414b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6313360f88643df9e2a5e9dee09d91e, user_id=74decb4847bb474d844b484d7b1f29b5, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:25.270365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd621df6-fbe3-4146-b892-7c2bf639414b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:25.274896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-acdf59cb-9596-43d4-bcef-f158b64fd47c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:25.276134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-acdf59cb-9596-43d4-bcef-f158b64fd47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:25.293767 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2759/5507] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:25 2022] PUT /identity/v3/projects/b6313360f88643df9e2a5e9dee09d91e/users/74decb4847bb474d844b484d7b1f29b5/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:25.306317 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2749/5508] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:25.308217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621df6-fbe3-4146-b892-7c2bf639414b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:25.308348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f497540-28eb-4d58-810c-a9531eba5625 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.308590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f497540-28eb-4d58-810c-a9531eba5625 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.308862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f497540-28eb-4d58-810c-a9531eba5625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.386991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8f497540-28eb-4d58-810c-a9531eba5625 None None] MFA Rules not processed for user `74decb4847bb474d844b484d7b1f29b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:25.388216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f497540-28eb-4d58-810c-a9531eba5625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:25.408862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:25.415191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621df6-fbe3-4146-b892-7c2bf639414b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:25.421922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0bc27-1a19-4bce-b7b2-e6a2a3249f4c tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:25.422703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0bc27-1a19-4bce-b7b2-e6a2a3249f4c tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.423038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0bc27-1a19-4bce-b7b2-e6a2a3249f4c tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.423312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0bc27-1a19-4bce-b7b2-e6a2a3249f4c tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/f2ac159964574b15976cf45403e81fc1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.426188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0bc27-1a19-4bce-b7b2-e6a2a3249f4c tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=1c6e17500ee34dc690a6126657f85481, credential_id=f2ac159964574b15976cf45403e81fc1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:25.428877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0bc27-1a19-4bce-b7b2-e6a2a3249f4c tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:25.455974 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2760/5509] 10.222.0.16 () {58 vars in 1380 bytes} [Sat Oct 15 21:11:25 2022] DELETE /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/f2ac159964574b15976cf45403e81fc1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:25.455974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0bc27-1a19-4bce-b7b2-e6a2a3249f4c tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:25.455974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4ad0bc27-1a19-4bce-b7b2-e6a2a3249f4c tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:25.519177 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2750/5510] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:25.520207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f497540-28eb-4d58-810c-a9531eba5625 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:25.547600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f497540-28eb-4d58-810c-a9531eba5625 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:25.550144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0bc27-1a19-4bce-b7b2-e6a2a3249f4c tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:25.554832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cfdd5-a7d1-4b90-966d-cf8f608c026e tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:25.555518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cfdd5-a7d1-4b90-966d-cf8f608c026e tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.556748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cfdd5-a7d1-4b90-966d-cf8f608c026e tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.556892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbede3c-fc32-4296-810a-11a81f443bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:25.557021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cfdd5-a7d1-4b90-966d-cf8f608c026e tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.557863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cfdd5-a7d1-4b90-966d-cf8f608c026e tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=1c6e17500ee34dc690a6126657f85481)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:25.558144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbede3c-fc32-4296-810a-11a81f443bce service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.558605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbede3c-fc32-4296-810a-11a81f443bce service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.559450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cfdd5-a7d1-4b90-966d-cf8f608c026e tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:25.559721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbede3c-fc32-4296-810a-11a81f443bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.604679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:25.606807 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2751/5511] 10.222.0.16 () {60 vars in 1299 bytes} [Sat Oct 15 21:11:25 2022] POST /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2 => generated 367 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:25.609521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cfdd5-a7d1-4b90-966d-cf8f608c026e tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:25.634655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cfdd5-a7d1-4b90-966d-cf8f608c026e tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:25.640532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee289f-93ec-4c59-b378-bcba6e583c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:25.641654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee289f-93ec-4c59-b378-bcba6e583c2f service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.642527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee289f-93ec-4c59-b378-bcba6e583c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.642990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee289f-93ec-4c59-b378-bcba6e583c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.656313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee289f-93ec-4c59-b378-bcba6e583c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:25.658258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee289f-93ec-4c59-b378-bcba6e583c2f service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:25.665136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbede3c-fc32-4296-810a-11a81f443bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:25.666945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbede3c-fc32-4296-810a-11a81f443bce service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:25.674226 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2752/5512] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:25.675529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee289f-93ec-4c59-b378-bcba6e583c2f service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:25.691018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee289f-93ec-4c59-b378-bcba6e583c2f service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:25.696448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e82ed-a412-4987-86c2-02b885be4038 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:25.697445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e82ed-a412-4987-86c2-02b885be4038 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.698132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e82ed-a412-4987-86c2-02b885be4038 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.698528 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2761/5513] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:25.698780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e82ed-a412-4987-86c2-02b885be4038 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/c7c35cd4af78411ca675922dab621ead` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.702122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e82ed-a412-4987-86c2-02b885be4038 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=1c6e17500ee34dc690a6126657f85481, credential_id=c7c35cd4af78411ca675922dab621ead)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:25.722322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e82ed-a412-4987-86c2-02b885be4038 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:25.782593 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2753/5514] 10.222.0.16 () {58 vars in 1380 bytes} [Sat Oct 15 21:11:25 2022] DELETE /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/c7c35cd4af78411ca675922dab621ead => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:25.788543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbede3c-fc32-4296-810a-11a81f443bce service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:25.802471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbede3c-fc32-4296-810a-11a81f443bce service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:25.820086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9e9b8-50ab-4fe3-8c05-0ead4c332aae tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:25.821787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9e9b8-50ab-4fe3-8c05-0ead4c332aae tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.822260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9e9b8-50ab-4fe3-8c05-0ead4c332aae tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.822765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9e9b8-50ab-4fe3-8c05-0ead4c332aae tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/c7c35cd4af78411ca675922dab621ead` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-3ca9e9b8-50ab-4fe3-8c05-0ead4c332aae tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Could not find credential: 5ec69f77c4a82bc655c626895760db7737e082c8e3cec081bbe63b21773b23a4.: keystone.exception.CredentialNotFound: Could not find credential: 5ec69f77c4a82bc655c626895760db7737e082c8e3cec081bbe63b21773b23a4. Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 426, in get Oct 15 21:11:25.865237 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 119, in _build_enforcer_target_data_owner_and_user_id_match Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref['credential'] = PROVIDERS.credential_api.get_credential( Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 119, in get_credential Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application credential = self._get_credential(credential_id) Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:11:25.866621 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 124, in _get_credential Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.get_credential(credential_id) Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 88, in get_credential Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_credential(session, credential_id).to_dict() Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 83, in _get_credential Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.CredentialNotFound(credential_id=credential_id) Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.CredentialNotFound: Could not find credential: 5ec69f77c4a82bc655c626895760db7737e082c8e3cec081bbe63b21773b23a4. Oct 15 21:11:25.867901 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:11:25.872861 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2762/5515] 10.222.0.16 () {58 vars in 1377 bytes} [Sat Oct 15 21:11:25 2022] GET /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/c7c35cd4af78411ca675922dab621ead => generated 148 bytes in 85 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:11:25.879240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e82ed-a412-4987-86c2-02b885be4038 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:25.894511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e82ed-a412-4987-86c2-02b885be4038 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:25.903707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67a8ba-e469-440a-a2db-dc4f7068ff3b tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:25.904506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a8ba-e469-440a-a2db-dc4f7068ff3b tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.904918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a8ba-e469-440a-a2db-dc4f7068ff3b tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.905333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a8ba-e469-440a-a2db-dc4f7068ff3b tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.906570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67a8ba-e469-440a-a2db-dc4f7068ff3b tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=1c6e17500ee34dc690a6126657f85481)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:25.908728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67a8ba-e469-440a-a2db-dc4f7068ff3b tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:25.935582 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2754/5516] 10.222.0.16 () {60 vars in 1299 bytes} [Sat Oct 15 21:11:25 2022] POST /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2 => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:25.939967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9e9b8-50ab-4fe3-8c05-0ead4c332aae tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:25.955865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9e9b8-50ab-4fe3-8c05-0ead4c332aae tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:25.963431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6b64b-2f82-4eb7-8154-26d62cfa8aab tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:25.964492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6b64b-2f82-4eb7-8154-26d62cfa8aab tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:25.965323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6b64b-2f82-4eb7-8154-26d62cfa8aab tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:25.965742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6b64b-2f82-4eb7-8154-26d62cfa8aab tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:25.966834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6b64b-2f82-4eb7-8154-26d62cfa8aab tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=1c6e17500ee34dc690a6126657f85481)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:25.979447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6b64b-2f82-4eb7-8154-26d62cfa8aab tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:26.004792 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2763/5517] 10.222.0.16 () {60 vars in 1299 bytes} [Sat Oct 15 21:11:25 2022] POST /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2 => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:26.008769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67a8ba-e469-440a-a2db-dc4f7068ff3b tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:26.024851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67a8ba-e469-440a-a2db-dc4f7068ff3b tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:26.033393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902733fe-1335-4248-a595-fb05a1436cee tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:26.034325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902733fe-1335-4248-a595-fb05a1436cee tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:26.034951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902733fe-1335-4248-a595-fb05a1436cee tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:26.035338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902733fe-1335-4248-a595-fb05a1436cee tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:26.036112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902733fe-1335-4248-a595-fb05a1436cee tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_list_credentials(user_id=1c6e17500ee34dc690a6126657f85481)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:26.039333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902733fe-1335-4248-a595-fb05a1436cee tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:26.061515 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2755/5518] 10.222.0.16 () {58 vars in 1278 bytes} [Sat Oct 15 21:11:26 2022] GET /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2 => generated 865 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:11:26.065738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6b64b-2f82-4eb7-8154-26d62cfa8aab tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:26.092932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6b64b-2f82-4eb7-8154-26d62cfa8aab tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:26.101051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db45763-9a26-4780-9f24-7c2babc8ebb1 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:26.102063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db45763-9a26-4780-9f24-7c2babc8ebb1 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:26.102673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db45763-9a26-4780-9f24-7c2babc8ebb1 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:26.102842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db45763-9a26-4780-9f24-7c2babc8ebb1 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/fc1833c2fc1d45fba19f9198c13cfccf` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:26.106539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db45763-9a26-4780-9f24-7c2babc8ebb1 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=1c6e17500ee34dc690a6126657f85481, credential_id=fc1833c2fc1d45fba19f9198c13cfccf)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:26.108390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db45763-9a26-4780-9f24-7c2babc8ebb1 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:26.140476 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2764/5519] 10.222.0.16 () {58 vars in 1380 bytes} [Sat Oct 15 21:11:26 2022] DELETE /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/fc1833c2fc1d45fba19f9198c13cfccf => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:26.142354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902733fe-1335-4248-a595-fb05a1436cee tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:26.158750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902733fe-1335-4248-a595-fb05a1436cee tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:26.169664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2374-c796-4237-a0b9-77dd033e1172 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:26.170423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2374-c796-4237-a0b9-77dd033e1172 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:26.170859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2374-c796-4237-a0b9-77dd033e1172 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:26.171236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2374-c796-4237-a0b9-77dd033e1172 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/40da76bad6ab44e3ad6e670e047eef03` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:26.173415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2374-c796-4237-a0b9-77dd033e1172 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=1c6e17500ee34dc690a6126657f85481, credential_id=40da76bad6ab44e3ad6e670e047eef03)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:26.174970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2374-c796-4237-a0b9-77dd033e1172 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:26.200754 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2756/5520] 10.222.0.16 () {58 vars in 1380 bytes} [Sat Oct 15 21:11:26 2022] DELETE /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/40da76bad6ab44e3ad6e670e047eef03 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:26.202270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db45763-9a26-4780-9f24-7c2babc8ebb1 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:26.215682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db45763-9a26-4780-9f24-7c2babc8ebb1 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:26.222312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc404b47-d968-4deb-b942-3adf39f57adf tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:26.223244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404b47-d968-4deb-b942-3adf39f57adf tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:26.223655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404b47-d968-4deb-b942-3adf39f57adf tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:26.224422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404b47-d968-4deb-b942-3adf39f57adf tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:26.225231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc404b47-d968-4deb-b942-3adf39f57adf tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=1c6e17500ee34dc690a6126657f85481)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:26.227181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc404b47-d968-4deb-b942-3adf39f57adf tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:26.269985 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2765/5521] 10.222.0.16 () {60 vars in 1299 bytes} [Sat Oct 15 21:11:26 2022] POST /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2 => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:26.273571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2374-c796-4237-a0b9-77dd033e1172 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:26.295471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2374-c796-4237-a0b9-77dd033e1172 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:26.301564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0285b-371a-4b33-bfdd-80bbdfb50366 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:26.303124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0285b-371a-4b33-bfdd-80bbdfb50366 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:26.304834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0285b-371a-4b33-bfdd-80bbdfb50366 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:26.305267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0285b-371a-4b33-bfdd-80bbdfb50366 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/624ed231231344ca810cbafa22267570` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:26.315434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0285b-371a-4b33-bfdd-80bbdfb50366 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_get_credential(user_id=1c6e17500ee34dc690a6126657f85481, credential_id=624ed231231344ca810cbafa22267570)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:26.317407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0285b-371a-4b33-bfdd-80bbdfb50366 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:26.326560 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2757/5522] 10.222.0.16 () {58 vars in 1377 bytes} [Sat Oct 15 21:11:26 2022] GET /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/624ed231231344ca810cbafa22267570 => generated 367 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:11:26.331641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc404b47-d968-4deb-b942-3adf39f57adf tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:26.345979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc404b47-d968-4deb-b942-3adf39f57adf tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:26.351727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9a233-0dca-481d-abcf-2328080ea426 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6e17500ee34dc690a6126657f85481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e4ccceba0f84d49a106a2b91b02ff5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:26.352533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a233-0dca-481d-abcf-2328080ea426 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:26.353122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a233-0dca-481d-abcf-2328080ea426 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:26.353511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a233-0dca-481d-abcf-2328080ea426 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/624ed231231344ca810cbafa22267570` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:26.356043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9a233-0dca-481d-abcf-2328080ea426 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=1c6e17500ee34dc690a6126657f85481, credential_id=624ed231231344ca810cbafa22267570)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:26.357739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9a233-0dca-481d-abcf-2328080ea426 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:26.407851 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2766/5523] 10.222.0.16 () {58 vars in 1380 bytes} [Sat Oct 15 21:11:26 2022] DELETE /identity/v3/users/1c6e17500ee34dc690a6126657f85481/credentials/OS-EC2/624ed231231344ca810cbafa22267570 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:26.414317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0285b-371a-4b33-bfdd-80bbdfb50366 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:26.416625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e7f0285b-371a-4b33-bfdd-80bbdfb50366 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:26.459303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0285b-371a-4b33-bfdd-80bbdfb50366 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:26.464407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:26.465384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:26.465677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:26.465955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] PATH_INFO: `/v3/users/1c6e17500ee34dc690a6126657f85481` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:26.467235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6e17500ee34dc690a6126657f85481)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:26.470260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:26.585016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6e17500ee34dc690a6126657f85481'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:26.609778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6e17500ee34dc690a6126657f85481'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:26.639627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9a233-0dca-481d-abcf-2328080ea426 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:26.641818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-44b9a233-0dca-481d-abcf-2328080ea426 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:26.675605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9a233-0dca-481d-abcf-2328080ea426 tempest-EC2CredentialsTest-64592505 tempest-EC2CredentialsTest-64592505-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:26.685938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6e17500ee34dc690a6126657f85481'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:26.691021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:26.694520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:26.695986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:26.696264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] PATH_INFO: `/v3/projects/7ff4e461d2ca44a4bd3e152a88e81d7e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:26.700256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff4e461d2ca44a4bd3e152a88e81d7e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:26.703034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:26.707160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6e17500ee34dc690a6126657f85481'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:26.708303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3b829209-3aea-44f7-b16c-f671ab7a8425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6e17500ee34dc690a6126657f85481'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:26.712273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:26.724924 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2758/5524] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:26 2022] DELETE /identity/v3/users/1c6e17500ee34dc690a6126657f85481 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:26.730621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3ca8f-e52f-43bf-8d00-2adc55a0f558 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:26.731845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3ca8f-e52f-43bf-8d00-2adc55a0f558 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:26.732123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3ca8f-e52f-43bf-8d00-2adc55a0f558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:26.784846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-95c3ca8f-e52f-43bf-8d00-2adc55a0f558 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:26.786177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-95c3ca8f-e52f-43bf-8d00-2adc55a0f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:26.844597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff4e461d2ca44a4bd3e152a88e81d7e', 'request_id': 'req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:26.865208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff4e461d2ca44a4bd3e152a88e81d7e', 'request_id': 'req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:26.878835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff4e461d2ca44a4bd3e152a88e81d7e', 'request_id': 'req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:26.905886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] The token cache is being invalidate because project 7ff4e461d2ca44a4bd3e152a88e81d7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:26.906267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:26.912332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:26.927883 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2767/5525] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:26 2022] DELETE /identity/v3/projects/7ff4e461d2ca44a4bd3e152a88e81d7e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:26.929832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:26.932118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:26.981089 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2759/5526] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:26.993883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3ca8f-e52f-43bf-8d00-2adc55a0f558 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:26.996435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-95c3ca8f-e52f-43bf-8d00-2adc55a0f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:26.998831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:27.050385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec4e3a-bc8d-4735-8b72-c8a18095aef9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:27.057961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:27.059478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:27.059478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:27.059718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] PATH_INFO: `/v3/users/fadcdce5062147249ebce68227767855` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:27.061212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fadcdce5062147249ebce68227767855)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:27.064112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:27.165702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3ca8f-e52f-43bf-8d00-2adc55a0f558 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:27.193631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fadcdce5062147249ebce68227767855'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:27.222845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fadcdce5062147249ebce68227767855'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:27.246251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fadcdce5062147249ebce68227767855'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:27.262481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fadcdce5062147249ebce68227767855'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:27.264773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fadcdce5062147249ebce68227767855'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:27.279656 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2768/5527] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:26 2022] DELETE /identity/v3/users/fadcdce5062147249ebce68227767855 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:27.313476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcfc66-7586-406e-8c28-d0ae45f478d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:27.315004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcfc66-7586-406e-8c28-d0ae45f478d4 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:27.315288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcfc66-7586-406e-8c28-d0ae45f478d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:27.315558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcfc66-7586-406e-8c28-d0ae45f478d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:27.318220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2dfcfc66-7586-406e-8c28-d0ae45f478d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:27.334528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcfc66-7586-406e-8c28-d0ae45f478d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:27.336268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcfc66-7586-406e-8c28-d0ae45f478d4 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:27.351112 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2760/5528] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:28.570459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:28.572681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:28.604366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcfc66-7586-406e-8c28-d0ae45f478d4 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:28.606010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2dfcfc66-7586-406e-8c28-d0ae45f478d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:28.631371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcfc66-7586-406e-8c28-d0ae45f478d4 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:28.694218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ab1fd-5345-401f-9489-9937a8f8a766 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:28.701419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:28.702315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:28.702643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:28.702915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] PATH_INFO: `/v3/projects/e0a213976c664c1a8a409c8355c31cd8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:28.704203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:28.704348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a213976c664c1a8a409c8355c31cd8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:28.705249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:28.705735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:28.706199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] PATH_INFO: `/v3/projects/6e4ccceba0f84d49a106a2b91b02ff5e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:28.706664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:28.707787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4ccceba0f84d49a106a2b91b02ff5e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:28.710422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:28.712557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:28.786895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a213976c664c1a8a409c8355c31cd8', 'request_id': 'req-9df62c52-34d8-403b-9790-c859b96bf197'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:28.800773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a213976c664c1a8a409c8355c31cd8', 'request_id': 'req-9df62c52-34d8-403b-9790-c859b96bf197'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:28.806495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:28.815296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4ccceba0f84d49a106a2b91b02ff5e', 'request_id': 'req-338f3495-eee1-4a34-9918-c04aebd71ae1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:28.816048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a213976c664c1a8a409c8355c31cd8', 'request_id': 'req-9df62c52-34d8-403b-9790-c859b96bf197'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:28.834532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] The token cache is being invalidate because project e0a213976c664c1a8a409c8355c31cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:28.834804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:28.835426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9df62c52-34d8-403b-9790-c859b96bf197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:28.835792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4ccceba0f84d49a106a2b91b02ff5e', 'request_id': 'req-338f3495-eee1-4a34-9918-c04aebd71ae1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:28.841935 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2769/5529] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:28 2022] DELETE /identity/v3/projects/e0a213976c664c1a8a409c8355c31cd8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:28.843915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aff1f3-6932-416e-beef-3fb1c14af5a6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:28.844308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aff1f3-6932-416e-beef-3fb1c14af5a6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:28.845368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aff1f3-6932-416e-beef-3fb1c14af5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:28.855419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4ccceba0f84d49a106a2b91b02ff5e', 'request_id': 'req-338f3495-eee1-4a34-9918-c04aebd71ae1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:28.873345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] The token cache is being invalidate because project 6e4ccceba0f84d49a106a2b91b02ff5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:28.874320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:28.875190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-338f3495-eee1-4a34-9918-c04aebd71ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:28.877505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-79aff1f3-6932-416e-beef-3fb1c14af5a6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:28.877784 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2761/5530] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:28 2022] DELETE /identity/v3/projects/6e4ccceba0f84d49a106a2b91b02ff5e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:28.879179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-79aff1f3-6932-416e-beef-3fb1c14af5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:28.882020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e87f7-4805-41d7-88e9-6851e6c16697 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:28.882583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e87f7-4805-41d7-88e9-6851e6c16697 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:28.883041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e87f7-4805-41d7-88e9-6851e6c16697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:28.911945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f47e87f7-4805-41d7-88e9-6851e6c16697 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:28.913886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f47e87f7-4805-41d7-88e9-6851e6c16697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:28.992365 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2770/5531] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:28.994615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97d4c8-8909-454b-9ac4-6cf74b65bbbe None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:28.995007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97d4c8-8909-454b-9ac4-6cf74b65bbbe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:28.995981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97d4c8-8909-454b-9ac4-6cf74b65bbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.001408 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2762/5532] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:29.010416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadb9b5-4fcd-4e83-8df6-dd23b3dd5322 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.010705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadb9b5-4fcd-4e83-8df6-dd23b3dd5322 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.010985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadb9b5-4fcd-4e83-8df6-dd23b3dd5322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.029506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dd97d4c8-8909-454b-9ac4-6cf74b65bbbe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:29.031502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dd97d4c8-8909-454b-9ac4-6cf74b65bbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:29.040366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0aadb9b5-4fcd-4e83-8df6-dd23b3dd5322 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:29.041848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0aadb9b5-4fcd-4e83-8df6-dd23b3dd5322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:29.044160 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2771/5533] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:29.048553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97d4c8-8909-454b-9ac4-6cf74b65bbbe None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.060199 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2763/5534] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:29.064366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadb9b5-4fcd-4e83-8df6-dd23b3dd5322 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.067389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97d4c8-8909-454b-9ac4-6cf74b65bbbe None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.074376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed5e98-fe0c-4aee-b476-9017dc5670e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.075313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5e98-fe0c-4aee-b476-9017dc5670e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.075783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5e98-fe0c-4aee-b476-9017dc5670e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.076211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5e98-fe0c-4aee-b476-9017dc5670e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.076801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadb9b5-4fcd-4e83-8df6-dd23b3dd5322 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.077248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed5e98-fe0c-4aee-b476-9017dc5670e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:29.077713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed5e98-fe0c-4aee-b476-9017dc5670e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.080624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed5e98-fe0c-4aee-b476-9017dc5670e6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.084122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac487a-139e-4795-a64f-83841902273e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.085446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac487a-139e-4795-a64f-83841902273e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.085876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac487a-139e-4795-a64f-83841902273e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.086242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac487a-139e-4795-a64f-83841902273e admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.087004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac487a-139e-4795-a64f-83841902273e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:29.087420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac487a-139e-4795-a64f-83841902273e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.089704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac487a-139e-4795-a64f-83841902273e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.103799 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2772/5535] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:29.105692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed5e98-fe0c-4aee-b476-9017dc5670e6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.111965 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2764/5536] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:11:29.116952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac487a-139e-4795-a64f-83841902273e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.121941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed5e98-fe0c-4aee-b476-9017dc5670e6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.128288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2943108-e3ad-442f-967c-abeb7403ec2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.128978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2943108-e3ad-442f-967c-abeb7403ec2e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.129298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2943108-e3ad-442f-967c-abeb7403ec2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.129566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2943108-e3ad-442f-967c-abeb7403ec2e admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.130449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2943108-e3ad-442f-967c-abeb7403ec2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.132498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2943108-e3ad-442f-967c-abeb7403ec2e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.132718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac487a-139e-4795-a64f-83841902273e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.138745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ec96e-5934-4e44-b3e7-857f6de4dd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.141379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ec96e-5934-4e44-b3e7-857f6de4dd8c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.141749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ec96e-5934-4e44-b3e7-857f6de4dd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.142108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ec96e-5934-4e44-b3e7-857f6de4dd8c admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.142883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ec96e-5934-4e44-b3e7-857f6de4dd8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.146480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ec96e-5934-4e44-b3e7-857f6de4dd8c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.174147 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2773/5537] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:29 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:29.175281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2943108-e3ad-442f-967c-abeb7403ec2e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.201576 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2765/5538] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:29 2022] POST /identity/v3/projects => generated 395 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:29.206645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ec96e-5934-4e44-b3e7-857f6de4dd8c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.316549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:29.345162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ec96e-5934-4e44-b3e7-857f6de4dd8c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.353661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4702fd-f22a-447e-bb77-d9c6cc49c8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.354362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4702fd-f22a-447e-bb77-d9c6cc49c8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.354584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4702fd-f22a-447e-bb77-d9c6cc49c8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.354855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4702fd-f22a-447e-bb77-d9c6cc49c8a2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.355493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4702fd-f22a-447e-bb77-d9c6cc49c8a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.357614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4702fd-f22a-447e-bb77-d9c6cc49c8a2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.383508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2943108-e3ad-442f-967c-abeb7403ec2e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.396821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd224e7-7349-46a9-b4de-78cf53913351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.397820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd224e7-7349-46a9-b4de-78cf53913351 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.398440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd224e7-7349-46a9-b4de-78cf53913351 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.399656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd224e7-7349-46a9-b4de-78cf53913351 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.401738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd224e7-7349-46a9-b4de-78cf53913351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.406127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd224e7-7349-46a9-b4de-78cf53913351 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.427724 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2766/5539] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:29 2022] POST /identity/v3/users => generated 304 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:29.431879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4702fd-f22a-447e-bb77-d9c6cc49c8a2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.452252 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2774/5540] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:29 2022] POST /identity/v3/users => generated 283 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:29.456036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd224e7-7349-46a9-b4de-78cf53913351 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.458978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4702fd-f22a-447e-bb77-d9c6cc49c8a2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.464903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b740179-231c-4c23-8c0e-b5ea6787193b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.465563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b740179-231c-4c23-8c0e-b5ea6787193b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.466540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b740179-231c-4c23-8c0e-b5ea6787193b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.467264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b740179-231c-4c23-8c0e-b5ea6787193b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.467932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b740179-231c-4c23-8c0e-b5ea6787193b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.470091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b740179-231c-4c23-8c0e-b5ea6787193b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.479131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd224e7-7349-46a9-b4de-78cf53913351 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.483498 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2767/5541] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:29.487925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b740179-231c-4c23-8c0e-b5ea6787193b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.488423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f577-058f-49d7-a3a5-3684adfd0ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.489151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f577-058f-49d7-a3a5-3684adfd0ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.489448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f577-058f-49d7-a3a5-3684adfd0ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.489727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f577-058f-49d7-a3a5-3684adfd0ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.490312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42f577-058f-49d7-a3a5-3684adfd0ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.501230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b740179-231c-4c23-8c0e-b5ea6787193b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.502431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42f577-058f-49d7-a3a5-3684adfd0ff1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.513703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313e55e-70b5-451f-a6ac-1e858f467e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.516210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e55e-70b5-451f-a6ac-1e858f467e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.516779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e55e-70b5-451f-a6ac-1e858f467e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.517350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e55e-70b5-451f-a6ac-1e858f467e65 admin admin] PATH_INFO: `/v3/projects/65fbadd5c3504935a8c44e48fcfe8336/users/9fbc95d769d84fd0850412ae0ea3cbbd/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.525072 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2775/5542] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:11:29.528642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f577-058f-49d7-a3a5-3684adfd0ff1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.541097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f577-058f-49d7-a3a5-3684adfd0ff1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.545136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313e55e-70b5-451f-a6ac-1e858f467e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fbadd5c3504935a8c44e48fcfe8336, user_id=9fbc95d769d84fd0850412ae0ea3cbbd, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.547997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac523d-2f98-4635-aef7-52d7d8250743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.548873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313e55e-70b5-451f-a6ac-1e858f467e65 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.549296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac523d-2f98-4635-aef7-52d7d8250743 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.549934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac523d-2f98-4635-aef7-52d7d8250743 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.550384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac523d-2f98-4635-aef7-52d7d8250743 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.551341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac523d-2f98-4635-aef7-52d7d8250743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.554083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac523d-2f98-4635-aef7-52d7d8250743 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.570197 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2776/5543] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:29.571525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac523d-2f98-4635-aef7-52d7d8250743 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.575376 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2768/5544] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:29 2022] PUT /identity/v3/projects/65fbadd5c3504935a8c44e48fcfe8336/users/9fbc95d769d84fd0850412ae0ea3cbbd/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:29.578884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313e55e-70b5-451f-a6ac-1e858f467e65 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.584802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac523d-2f98-4635-aef7-52d7d8250743 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.595980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:29.675073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360293ab-9e95-4356-a6b8-190d960b461d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.675741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360293ab-9e95-4356-a6b8-190d960b461d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.676014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360293ab-9e95-4356-a6b8-190d960b461d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.676285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360293ab-9e95-4356-a6b8-190d960b461d admin admin] PATH_INFO: `/v3/projects/197ac62373aa4cc1a2de96300f66605a/users/4af95920bd034321ad367bbb68fe3b3d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.696285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313e55e-70b5-451f-a6ac-1e858f467e65 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.700612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360293ab-9e95-4356-a6b8-190d960b461d admin admin] RBAC: Authorizing `identity:create_grant(project_id=197ac62373aa4cc1a2de96300f66605a, user_id=4af95920bd034321ad367bbb68fe3b3d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.706819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360293ab-9e95-4356-a6b8-190d960b461d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.707531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c4d51-1d2b-4e2f-85fe-56bed0f3fb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.709582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4d51-1d2b-4e2f-85fe-56bed0f3fb3a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.711018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4d51-1d2b-4e2f-85fe-56bed0f3fb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.712289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4d51-1d2b-4e2f-85fe-56bed0f3fb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.714043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c4d51-1d2b-4e2f-85fe-56bed0f3fb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.716032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c4d51-1d2b-4e2f-85fe-56bed0f3fb3a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.730003 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2777/5545] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:29 2022] PUT /identity/v3/projects/197ac62373aa4cc1a2de96300f66605a/users/4af95920bd034321ad367bbb68fe3b3d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:29.733928 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2769/5546] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:11:29.740620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360293ab-9e95-4356-a6b8-190d960b461d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.743212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e7e60-641d-4d39-b860-efc1ae2f8b06 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.743889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e7e60-641d-4d39-b860-efc1ae2f8b06 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.744426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e7e60-641d-4d39-b860-efc1ae2f8b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.796584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f08e7e60-641d-4d39-b860-efc1ae2f8b06 None None] MFA Rules not processed for user `4af95920bd034321ad367bbb68fe3b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:29.798074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f08e7e60-641d-4d39-b860-efc1ae2f8b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:29.882924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360293ab-9e95-4356-a6b8-190d960b461d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:29.888720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8cf0d-3836-4f8b-b8a7-0f4c3339fc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:29.889439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8cf0d-3836-4f8b-b8a7-0f4c3339fc66 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.890025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8cf0d-3836-4f8b-b8a7-0f4c3339fc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.890025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8cf0d-3836-4f8b-b8a7-0f4c3339fc66 admin admin] PATH_INFO: `/v3/projects/65fbadd5c3504935a8c44e48fcfe8336/users/9fbc95d769d84fd0850412ae0ea3cbbd/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.892019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8cf0d-3836-4f8b-b8a7-0f4c3339fc66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fbadd5c3504935a8c44e48fcfe8336, user_id=9fbc95d769d84fd0850412ae0ea3cbbd, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:29.896545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8cf0d-3836-4f8b-b8a7-0f4c3339fc66 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:29.931329 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2778/5547] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:29 2022] PUT /identity/v3/projects/65fbadd5c3504935a8c44e48fcfe8336/users/9fbc95d769d84fd0850412ae0ea3cbbd/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:29.933069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82260739-1d60-49b3-bb03-f21c48968de3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:29.934679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82260739-1d60-49b3-bb03-f21c48968de3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:29.934956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82260739-1d60-49b3-bb03-f21c48968de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:29.967117 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2770/5548] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:29.979632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e7e60-641d-4d39-b860-efc1ae2f8b06 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:29.981331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f08e7e60-641d-4d39-b860-efc1ae2f8b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:29.992397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-82260739-1d60-49b3-bb03-f21c48968de3 None None] MFA Rules not processed for user `9fbc95d769d84fd0850412ae0ea3cbbd`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:29.993410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-82260739-1d60-49b3-bb03-f21c48968de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:30.041029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:30.077379 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2779/5549] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:30.084011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82260739-1d60-49b3-bb03-f21c48968de3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:30.106808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e7e60-641d-4d39-b860-efc1ae2f8b06 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:30.122642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:30.219955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82260739-1d60-49b3-bb03-f21c48968de3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:30.230090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba4348-454e-4aea-a76a-c39da11779f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:30.230856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba4348-454e-4aea-a76a-c39da11779f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:30.231237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba4348-454e-4aea-a76a-c39da11779f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:30.231607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba4348-454e-4aea-a76a-c39da11779f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:30.232709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba4348-454e-4aea-a76a-c39da11779f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:30.234546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba4348-454e-4aea-a76a-c39da11779f5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:30.250334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bed65b-3f99-4052-af81-7777ca7ca0ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:30.252020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bed65b-3f99-4052-af81-7777ca7ca0ef service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:30.253918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bed65b-3f99-4052-af81-7777ca7ca0ef service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:30.254780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bed65b-3f99-4052-af81-7777ca7ca0ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:30.337050 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2780/5550] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:30 2022] POST /identity/v3/projects => generated 395 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:30.340674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba4348-454e-4aea-a76a-c39da11779f5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:30.399518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bed65b-3f99-4052-af81-7777ca7ca0ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:30.401212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bed65b-3f99-4052-af81-7777ca7ca0ef service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:30.418937 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2771/5551] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:11:30.458121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba4348-454e-4aea-a76a-c39da11779f5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:30.463618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecab5a8-373a-4c20-a070-73d1ebfaea7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:30.464232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecab5a8-373a-4c20-a070-73d1ebfaea7e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:30.464565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecab5a8-373a-4c20-a070-73d1ebfaea7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:30.464836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecab5a8-373a-4c20-a070-73d1ebfaea7e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:30.465664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecab5a8-373a-4c20-a070-73d1ebfaea7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:30.467628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecab5a8-373a-4c20-a070-73d1ebfaea7e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:30.501099 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2781/5552] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:30 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:30.502988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bed65b-3f99-4052-af81-7777ca7ca0ef service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:30.527182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bed65b-3f99-4052-af81-7777ca7ca0ef service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:30.542037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6ab45-e9fd-4d07-ad34-f6b6af3d2760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:30.542657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6ab45-e9fd-4d07-ad34-f6b6af3d2760 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:30.542956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6ab45-e9fd-4d07-ad34-f6b6af3d2760 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:30.543207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6ab45-e9fd-4d07-ad34-f6b6af3d2760 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:30.543825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6ab45-e9fd-4d07-ad34-f6b6af3d2760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:30.545458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6ab45-e9fd-4d07-ad34-f6b6af3d2760 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:30.576852 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2772/5553] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:30.578134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecab5a8-373a-4c20-a070-73d1ebfaea7e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:30.596970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecab5a8-373a-4c20-a070-73d1ebfaea7e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:30.603639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9db85-56ec-446e-8191-464b8625478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:30.604573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9db85-56ec-446e-8191-464b8625478f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:30.604854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9db85-56ec-446e-8191-464b8625478f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:30.605287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9db85-56ec-446e-8191-464b8625478f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:30.605946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9db85-56ec-446e-8191-464b8625478f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:30.607750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9db85-56ec-446e-8191-464b8625478f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:30.632580 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2782/5554] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:30.634018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6ab45-e9fd-4d07-ad34-f6b6af3d2760 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:30.651818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6ab45-e9fd-4d07-ad34-f6b6af3d2760 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:30.662474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa37e3-4fda-4bcd-b100-611c7f57db5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:30.663652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa37e3-4fda-4bcd-b100-611c7f57db5f admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:30.663652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa37e3-4fda-4bcd-b100-611c7f57db5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:30.663829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa37e3-4fda-4bcd-b100-611c7f57db5f admin admin] PATH_INFO: `/v3/projects/3d8d3bdb0e8a47c7a9398cef1e4b6b05/users/2daef5b3cc2d4fcea28d841f132d4312/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:30.689609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa37e3-4fda-4bcd-b100-611c7f57db5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8d3bdb0e8a47c7a9398cef1e4b6b05, user_id=2daef5b3cc2d4fcea28d841f132d4312, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:30.694141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa37e3-4fda-4bcd-b100-611c7f57db5f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:30.713995 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2773/5555] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:30 2022] PUT /identity/v3/projects/3d8d3bdb0e8a47c7a9398cef1e4b6b05/users/2daef5b3cc2d4fcea28d841f132d4312/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:30.724660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcafdb-0b2a-4717-99c8-663f1c6a0323 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:30.724937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcafdb-0b2a-4717-99c8-663f1c6a0323 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:30.725243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcafdb-0b2a-4717-99c8-663f1c6a0323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:30.771664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-42bcafdb-0b2a-4717-99c8-663f1c6a0323 None None] MFA Rules not processed for user `2daef5b3cc2d4fcea28d841f132d4312`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:30.773136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-42bcafdb-0b2a-4717-99c8-663f1c6a0323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:30.891929 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2783/5556] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:30.901180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa37e3-4fda-4bcd-b100-611c7f57db5f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:30.903954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-38fa37e3-4fda-4bcd-b100-611c7f57db5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:31.036915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa37e3-4fda-4bcd-b100-611c7f57db5f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:31.045589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c0337-929d-42f0-96d4-f71d023d3cd9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:31.046220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c0337-929d-42f0-96d4-f71d023d3cd9 service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:31.046499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c0337-929d-42f0-96d4-f71d023d3cd9 service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:31.046771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c0337-929d-42f0-96d4-f71d023d3cd9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:31.059524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcafdb-0b2a-4717-99c8-663f1c6a0323 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:31.074097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:31.095485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:31.170598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcafdb-0b2a-4717-99c8-663f1c6a0323 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:31.178170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:31.186392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:31.187002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:31.187451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] PATH_INFO: `/v3/users/4af95920bd034321ad367bbb68fe3b3d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:31.187757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c0337-929d-42f0-96d4-f71d023d3cd9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:31.189555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4af95920bd034321ad367bbb68fe3b3d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:31.190794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c0337-929d-42f0-96d4-f71d023d3cd9 service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:31.191960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:31.220736 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2774/5557] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:30 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:31.292602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4af95920bd034321ad367bbb68fe3b3d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:31.311925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4af95920bd034321ad367bbb68fe3b3d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:31.325394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4af95920bd034321ad367bbb68fe3b3d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:31.353977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4af95920bd034321ad367bbb68fe3b3d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:31.355614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4af95920bd034321ad367bbb68fe3b3d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:31.369220 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2784/5558] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:31 2022] DELETE /identity/v3/users/4af95920bd034321ad367bbb68fe3b3d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:31.371265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528e87-19e7-4d8e-9451-8826c9fdad6a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:31.371668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528e87-19e7-4d8e-9451-8826c9fdad6a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:31.372042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528e87-19e7-4d8e-9451-8826c9fdad6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:31.412056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b0528e87-19e7-4d8e-9451-8826c9fdad6a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:31.414588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b0528e87-19e7-4d8e-9451-8826c9fdad6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:31.572202 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2775/5559] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:11:31.590949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:31.596815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:31.751990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bad0fe-61ea-49f4-af19-1fc02772a23a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:31.758679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59ec75-efef-4e50-8397-5645cd210841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:31.759282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59ec75-efef-4e50-8397-5645cd210841 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:31.759555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59ec75-efef-4e50-8397-5645cd210841 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:31.759901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59ec75-efef-4e50-8397-5645cd210841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:31.782688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59ec75-efef-4e50-8397-5645cd210841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:31.784382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59ec75-efef-4e50-8397-5645cd210841 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:31.814124 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2785/5560] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:32.139357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0528e87-19e7-4d8e-9451-8826c9fdad6a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:32.140950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b0528e87-19e7-4d8e-9451-8826c9fdad6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:32.155064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0528e87-19e7-4d8e-9451-8826c9fdad6a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:32.159455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:32.160280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:32.160654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:32.161022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] PATH_INFO: `/v3/users/9fbc95d769d84fd0850412ae0ea3cbbd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:32.162411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fbc95d769d84fd0850412ae0ea3cbbd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:32.164721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:32.265165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fbc95d769d84fd0850412ae0ea3cbbd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:32.282033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fbc95d769d84fd0850412ae0ea3cbbd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:32.306873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fbc95d769d84fd0850412ae0ea3cbbd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:32.318565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fbc95d769d84fd0850412ae0ea3cbbd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:32.319358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fbc95d769d84fd0850412ae0ea3cbbd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:32.335099 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2776/5561] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:32 2022] DELETE /identity/v3/users/9fbc95d769d84fd0850412ae0ea3cbbd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:32.336716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a492-9da1-44b3-b607-beea60846b4e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:32.337004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a492-9da1-44b3-b607-beea60846b4e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:32.337307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a492-9da1-44b3-b607-beea60846b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:32.380491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-68e5a492-9da1-44b3-b607-beea60846b4e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:32.381845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-68e5a492-9da1-44b3-b607-beea60846b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:32.424125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:32.486561 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2786/5562] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:32.515618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:32.517176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:32.559507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:32.612306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e524f-0c8e-42e9-ab7b-b7d2b088b90f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:32.617432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08859edb-8a6d-4100-93cb-8313fd74d0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:32.618099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08859edb-8a6d-4100-93cb-8313fd74d0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:32.618378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08859edb-8a6d-4100-93cb-8313fd74d0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:32.618646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08859edb-8a6d-4100-93cb-8313fd74d0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:32.633997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08859edb-8a6d-4100-93cb-8313fd74d0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:32.636651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08859edb-8a6d-4100-93cb-8313fd74d0f9 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:32.650223 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2777/5563] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:32.828607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5a492-9da1-44b3-b607-beea60846b4e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:32.829786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-68e5a492-9da1-44b3-b607-beea60846b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:32.843371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5a492-9da1-44b3-b607-beea60846b4e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:32.850136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:32.850905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:32.851280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:32.851645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] PATH_INFO: `/v3/projects/197ac62373aa4cc1a2de96300f66605a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:32.852882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] RBAC: Authorizing `identity:delete_project(project_id=197ac62373aa4cc1a2de96300f66605a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:32.854998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:32.926496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197ac62373aa4cc1a2de96300f66605a', 'request_id': 'req-1aa2256d-cb2b-418e-809c-ea1ab911774f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:32.944823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197ac62373aa4cc1a2de96300f66605a', 'request_id': 'req-1aa2256d-cb2b-418e-809c-ea1ab911774f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:32.972740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197ac62373aa4cc1a2de96300f66605a', 'request_id': 'req-1aa2256d-cb2b-418e-809c-ea1ab911774f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:32.990279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] The token cache is being invalidate because project 197ac62373aa4cc1a2de96300f66605a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:32.990527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:32.991241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1aa2256d-cb2b-418e-809c-ea1ab911774f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:32.993754 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2787/5564] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:32 2022] DELETE /identity/v3/projects/197ac62373aa4cc1a2de96300f66605a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:32.998636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88f787-76e5-4410-a92b-05e4cfa3a3de None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:32.998923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88f787-76e5-4410-a92b-05e4cfa3a3de None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:32.999194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88f787-76e5-4410-a92b-05e4cfa3a3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:33.035666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5e88f787-76e5-4410-a92b-05e4cfa3a3de None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:33.037510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5e88f787-76e5-4410-a92b-05e4cfa3a3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:33.152486 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2778/5565] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:33.160141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134e57-3577-4166-afc7-d611a472fa63 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:33.160479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134e57-3577-4166-afc7-d611a472fa63 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:33.160828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134e57-3577-4166-afc7-d611a472fa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:33.213701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7b134e57-3577-4166-afc7-d611a472fa63 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:33.216409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7b134e57-3577-4166-afc7-d611a472fa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:33.242441 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2788/5566] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:33.244374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88f787-76e5-4410-a92b-05e4cfa3a3de None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:33.263635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88f787-76e5-4410-a92b-05e4cfa3a3de None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:33.291233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4a48c-dee4-4dd0-83d2-f266e7041191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:33.291954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4a48c-dee4-4dd0-83d2-f266e7041191 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:33.292306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4a48c-dee4-4dd0-83d2-f266e7041191 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:33.292992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4a48c-dee4-4dd0-83d2-f266e7041191 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:33.300993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4a48c-dee4-4dd0-83d2-f266e7041191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:33.301905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4a48c-dee4-4dd0-83d2-f266e7041191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:33.306277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4a48c-dee4-4dd0-83d2-f266e7041191 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:33.348179 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2779/5567] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:33.353908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b134e57-3577-4166-afc7-d611a472fa63 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:33.368048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b134e57-3577-4166-afc7-d611a472fa63 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:33.377462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184f92b-289b-448f-b131-710b0d62942b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:33.378290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184f92b-289b-448f-b131-710b0d62942b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:33.378405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184f92b-289b-448f-b131-710b0d62942b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:33.381264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184f92b-289b-448f-b131-710b0d62942b admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:33.383270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184f92b-289b-448f-b131-710b0d62942b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:33.388312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184f92b-289b-448f-b131-710b0d62942b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:33.482792 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2789/5568] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:33 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:33.486738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4a48c-dee4-4dd0-83d2-f266e7041191 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:33.511299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4a48c-dee4-4dd0-83d2-f266e7041191 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:33.615613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f64533-7537-4e89-95f3-87e473173124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:33.618330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f64533-7537-4e89-95f3-87e473173124 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:33.618623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f64533-7537-4e89-95f3-87e473173124 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:33.618898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f64533-7537-4e89-95f3-87e473173124 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:33.619549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f64533-7537-4e89-95f3-87e473173124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:33.624282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f64533-7537-4e89-95f3-87e473173124 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:33.670370 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2780/5569] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:33 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:33.674596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184f92b-289b-448f-b131-710b0d62942b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:33.693774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184f92b-289b-448f-b131-710b0d62942b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:33.721338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d51f7-121c-4d3b-bb4a-393b08ae5eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:33.721963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d51f7-121c-4d3b-bb4a-393b08ae5eac admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:33.722241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d51f7-121c-4d3b-bb4a-393b08ae5eac admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:33.722510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d51f7-121c-4d3b-bb4a-393b08ae5eac admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:33.723164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d51f7-121c-4d3b-bb4a-393b08ae5eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:33.725073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d51f7-121c-4d3b-bb4a-393b08ae5eac admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:33.763633 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2790/5570] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:33.765115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f64533-7537-4e89-95f3-87e473173124 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:33.782685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f64533-7537-4e89-95f3-87e473173124 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:33.802533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969f704-b8e4-4470-9fb2-9497283362fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:33.803337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969f704-b8e4-4470-9fb2-9497283362fb admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:33.803617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969f704-b8e4-4470-9fb2-9497283362fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:33.803890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969f704-b8e4-4470-9fb2-9497283362fb admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:33.804543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969f704-b8e4-4470-9fb2-9497283362fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:33.806529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969f704-b8e4-4470-9fb2-9497283362fb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:33.837232 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2781/5571] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:33.838484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d51f7-121c-4d3b-bb4a-393b08ae5eac admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:33.850220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d51f7-121c-4d3b-bb4a-393b08ae5eac admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:33.855840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfded90-ca1b-413c-9632-98c870abdecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:33.857130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfded90-ca1b-413c-9632-98c870abdecd admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:33.857413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfded90-ca1b-413c-9632-98c870abdecd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:33.857690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfded90-ca1b-413c-9632-98c870abdecd admin admin] PATH_INFO: `/v3/projects/a89d5eb5c8d246d6a866c79c5dddbf3e/users/34d02da171db432783f630c853e35fc1/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:33.894328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfded90-ca1b-413c-9632-98c870abdecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89d5eb5c8d246d6a866c79c5dddbf3e, user_id=34d02da171db432783f630c853e35fc1, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:33.898189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfded90-ca1b-413c-9632-98c870abdecd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:33.928107 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2791/5572] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:33 2022] PUT /identity/v3/projects/a89d5eb5c8d246d6a866c79c5dddbf3e/users/34d02da171db432783f630c853e35fc1/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:33.930476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdf1eb-5ccc-44ca-81c7-22a6cf7e52a4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:33.931207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdf1eb-5ccc-44ca-81c7-22a6cf7e52a4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:33.931601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdf1eb-5ccc-44ca-81c7-22a6cf7e52a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:33.987750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-60fdf1eb-5ccc-44ca-81c7-22a6cf7e52a4 None None] MFA Rules not processed for user `34d02da171db432783f630c853e35fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:33.993546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-60fdf1eb-5ccc-44ca-81c7-22a6cf7e52a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:34.022222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfded90-ca1b-413c-9632-98c870abdecd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:34.024394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1cfded90-ca1b-413c-9632-98c870abdecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:34.041854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:34.121625 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2782/5573] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:34.140462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:34.162031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfded90-ca1b-413c-9632-98c870abdecd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:34.168443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:34.169248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:34.169641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:34.170005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] PATH_INFO: `/v3/projects/65fbadd5c3504935a8c44e48fcfe8336` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:34.171252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65fbadd5c3504935a8c44e48fcfe8336)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:34.173311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:34.228893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdf1eb-5ccc-44ca-81c7-22a6cf7e52a4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:34.230450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-60fdf1eb-5ccc-44ca-81c7-22a6cf7e52a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:34.235134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65fbadd5c3504935a8c44e48fcfe8336', 'request_id': 'req-9f8031c3-482e-4e04-b84a-1cd15ace3287'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:34.287429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65fbadd5c3504935a8c44e48fcfe8336', 'request_id': 'req-9f8031c3-482e-4e04-b84a-1cd15ace3287'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:34.307457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65fbadd5c3504935a8c44e48fcfe8336', 'request_id': 'req-9f8031c3-482e-4e04-b84a-1cd15ace3287'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:34.335063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] The token cache is being invalidate because project 65fbadd5c3504935a8c44e48fcfe8336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:34.335651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:34.336346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:34.342307 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2792/5574] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:34 2022] DELETE /identity/v3/projects/65fbadd5c3504935a8c44e48fcfe8336 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:34.343581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:34.345463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:34.346709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdf1eb-5ccc-44ca-81c7-22a6cf7e52a4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:34.468545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23943d48-0d34-4aae-b8f7-88abb687288a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:34.469374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23943d48-0d34-4aae-b8f7-88abb687288a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:34.469773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23943d48-0d34-4aae-b8f7-88abb687288a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:34.470051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23943d48-0d34-4aae-b8f7-88abb687288a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:34.471863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-23943d48-0d34-4aae-b8f7-88abb687288a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:34.481375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8031c3-482e-4e04-b84a-1cd15ace3287 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:34.487793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:34.490155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:34.490530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:34.491811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] PATH_INFO: `/v3/users/2daef5b3cc2d4fcea28d841f132d4312` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:34.493203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2daef5b3cc2d4fcea28d841f132d4312)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:34.495253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:34.582985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2daef5b3cc2d4fcea28d841f132d4312'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:34.598386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23943d48-0d34-4aae-b8f7-88abb687288a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:34.600425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23943d48-0d34-4aae-b8f7-88abb687288a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:34.603465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2daef5b3cc2d4fcea28d841f132d4312'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:34.615774 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2783/5575] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:34.619456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2daef5b3cc2d4fcea28d841f132d4312'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:34.631078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2daef5b3cc2d4fcea28d841f132d4312'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:34.631908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd592c6f-ba13-4d74-a8b0-ae78d6d13535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2daef5b3cc2d4fcea28d841f132d4312'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:34.634404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:34.640016 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2793/5576] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:34 2022] DELETE /identity/v3/users/2daef5b3cc2d4fcea28d841f132d4312 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:35.272569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23943d48-0d34-4aae-b8f7-88abb687288a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:35.305404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23943d48-0d34-4aae-b8f7-88abb687288a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:35.376275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:35.410237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:35.411129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:35.411501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:35.411971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] PATH_INFO: `/v3/projects/3d8d3bdb0e8a47c7a9398cef1e4b6b05` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:35.413146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d8d3bdb0e8a47c7a9398cef1e4b6b05)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:35.415377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:35.506486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d8d3bdb0e8a47c7a9398cef1e4b6b05', 'request_id': 'req-919cbca7-b831-445d-b428-fcc76cedad96'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:35.518676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d8d3bdb0e8a47c7a9398cef1e4b6b05', 'request_id': 'req-919cbca7-b831-445d-b428-fcc76cedad96'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:35.528095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d8d3bdb0e8a47c7a9398cef1e4b6b05', 'request_id': 'req-919cbca7-b831-445d-b428-fcc76cedad96'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:35.543285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] The token cache is being invalidate because project 3d8d3bdb0e8a47c7a9398cef1e4b6b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:35.543634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:35.544218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-919cbca7-b831-445d-b428-fcc76cedad96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:35.546809 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2784/5577] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:35 2022] DELETE /identity/v3/projects/3d8d3bdb0e8a47c7a9398cef1e4b6b05 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:35.551391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff0b36-8f8c-4a7a-ad6e-fe6cc43e3ca3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:35.552476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff0b36-8f8c-4a7a-ad6e-fe6cc43e3ca3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:35.552476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff0b36-8f8c-4a7a-ad6e-fe6cc43e3ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:35.574833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8dff0b36-8f8c-4a7a-ad6e-fe6cc43e3ca3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:35.576213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8dff0b36-8f8c-4a7a-ad6e-fe6cc43e3ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:35.653715 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2794/5578] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:35.663417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca163e-3be8-4a12-86f3-9d631fe77912 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:35.663417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca163e-3be8-4a12-86f3-9d631fe77912 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:35.663417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca163e-3be8-4a12-86f3-9d631fe77912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:35.687716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3bca163e-3be8-4a12-86f3-9d631fe77912 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:35.688865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3bca163e-3be8-4a12-86f3-9d631fe77912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:35.698452 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2785/5579] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:35.702949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff0b36-8f8c-4a7a-ad6e-fe6cc43e3ca3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:35.720614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff0b36-8f8c-4a7a-ad6e-fe6cc43e3ca3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:35.729491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9717f7-066b-4ee8-8e71-ff3a616e6d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:35.730106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9717f7-066b-4ee8-8e71-ff3a616e6d1d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:35.730382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9717f7-066b-4ee8-8e71-ff3a616e6d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:35.730703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9717f7-066b-4ee8-8e71-ff3a616e6d1d admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:35.732107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9717f7-066b-4ee8-8e71-ff3a616e6d1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:35.732529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9717f7-066b-4ee8-8e71-ff3a616e6d1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:35.734509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9717f7-066b-4ee8-8e71-ff3a616e6d1d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:35.754161 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2795/5580] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:35.755322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca163e-3be8-4a12-86f3-9d631fe77912 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:35.766589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca163e-3be8-4a12-86f3-9d631fe77912 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:35.772408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa787a-5bf1-4db1-9052-1dfe7c2b1231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:35.773196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa787a-5bf1-4db1-9052-1dfe7c2b1231 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:35.773536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa787a-5bf1-4db1-9052-1dfe7c2b1231 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:35.773874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa787a-5bf1-4db1-9052-1dfe7c2b1231 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:35.774571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa787a-5bf1-4db1-9052-1dfe7c2b1231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:35.776756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa787a-5bf1-4db1-9052-1dfe7c2b1231 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:35.816032 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2786/5581] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:35 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:35.818295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9717f7-066b-4ee8-8e71-ff3a616e6d1d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:35.830806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9717f7-066b-4ee8-8e71-ff3a616e6d1d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:35.910830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047760d-e5d3-46cd-bb6d-a17d650f5a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:35.911924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047760d-e5d3-46cd-bb6d-a17d650f5a42 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:35.912267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047760d-e5d3-46cd-bb6d-a17d650f5a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:35.912619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047760d-e5d3-46cd-bb6d-a17d650f5a42 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:35.913393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047760d-e5d3-46cd-bb6d-a17d650f5a42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:35.915390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047760d-e5d3-46cd-bb6d-a17d650f5a42 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:35.953453 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2796/5582] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:35 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:35.954386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa787a-5bf1-4db1-9052-1dfe7c2b1231 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:35.979766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa787a-5bf1-4db1-9052-1dfe7c2b1231 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:35.985464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b5749-8d86-4ac0-9103-28fb360938aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:35.987568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b5749-8d86-4ac0-9103-28fb360938aa admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:35.987933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b5749-8d86-4ac0-9103-28fb360938aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:35.988575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b5749-8d86-4ac0-9103-28fb360938aa admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:35.989715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b5749-8d86-4ac0-9103-28fb360938aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:35.991994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b5749-8d86-4ac0-9103-28fb360938aa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:36.006279 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2787/5583] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:36.011125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047760d-e5d3-46cd-bb6d-a17d650f5a42 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:36.024635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047760d-e5d3-46cd-bb6d-a17d650f5a42 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:36.030382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6fa0d-1a90-4860-8448-765f886b8636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:36.031106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6fa0d-1a90-4860-8448-765f886b8636 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:36.031488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6fa0d-1a90-4860-8448-765f886b8636 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:36.032008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6fa0d-1a90-4860-8448-765f886b8636 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:36.033123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6fa0d-1a90-4860-8448-765f886b8636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:36.034701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6fa0d-1a90-4860-8448-765f886b8636 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:36.062842 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2797/5584] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:11:36.067604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b5749-8d86-4ac0-9103-28fb360938aa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:36.081852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b5749-8d86-4ac0-9103-28fb360938aa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:36.090695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b17ea-1742-44c2-ac9e-4dcb71a33f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:36.092678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b17ea-1742-44c2-ac9e-4dcb71a33f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:36.092989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b17ea-1742-44c2-ac9e-4dcb71a33f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:36.093381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b17ea-1742-44c2-ac9e-4dcb71a33f74 admin admin] PATH_INFO: `/v3/projects/cc99f6958443492f8e87cc53ab0ab200/users/2956edfb39484e38a5f8a44997dc4055/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:36.113717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b17ea-1742-44c2-ac9e-4dcb71a33f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc99f6958443492f8e87cc53ab0ab200, user_id=2956edfb39484e38a5f8a44997dc4055, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:36.120285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b17ea-1742-44c2-ac9e-4dcb71a33f74 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:36.142663 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2788/5585] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:36 2022] PUT /identity/v3/projects/cc99f6958443492f8e87cc53ab0ab200/users/2956edfb39484e38a5f8a44997dc4055/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:36.155197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341a943-6dfc-449b-88ad-a6bf18091392 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:36.155691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341a943-6dfc-449b-88ad-a6bf18091392 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:36.156272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341a943-6dfc-449b-88ad-a6bf18091392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:36.207434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9341a943-6dfc-449b-88ad-a6bf18091392 None None] MFA Rules not processed for user `2956edfb39484e38a5f8a44997dc4055`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:36.208999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9341a943-6dfc-449b-88ad-a6bf18091392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:36.314647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:36.334324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b17ea-1742-44c2-ac9e-4dcb71a33f74 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:36.334970 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2798/5586] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:36.432269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b17ea-1742-44c2-ac9e-4dcb71a33f74 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:36.437830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbd472-9e5c-4c3c-8838-4763558f02be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:36.438897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbd472-9e5c-4c3c-8838-4763558f02be admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:36.439479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbd472-9e5c-4c3c-8838-4763558f02be admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:36.439884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbd472-9e5c-4c3c-8838-4763558f02be admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:36.441903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbd472-9e5c-4c3c-8838-4763558f02be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:36.444670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbd472-9e5c-4c3c-8838-4763558f02be admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:36.453311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:36.494929 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2789/5587] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:36 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:36.499753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341a943-6dfc-449b-88ad-a6bf18091392 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:36.514235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341a943-6dfc-449b-88ad-a6bf18091392 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:36.585935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11831e-347b-444d-8e18-fc0e870e08d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:36.585935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11831e-347b-444d-8e18-fc0e870e08d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:36.586745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11831e-347b-444d-8e18-fc0e870e08d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:36.586745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11831e-347b-444d-8e18-fc0e870e08d6 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:36.587731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11831e-347b-444d-8e18-fc0e870e08d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:36.589686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11831e-347b-444d-8e18-fc0e870e08d6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:36.615425 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2799/5588] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:36 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:36.619452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbd472-9e5c-4c3c-8838-4763558f02be admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:36.651946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbd472-9e5c-4c3c-8838-4763558f02be admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:36.657364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3838b7-de07-4aa9-8634-6d6dce096d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:36.658295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3838b7-de07-4aa9-8634-6d6dce096d8e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:36.658936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3838b7-de07-4aa9-8634-6d6dce096d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:36.659564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3838b7-de07-4aa9-8634-6d6dce096d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:36.660424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3838b7-de07-4aa9-8634-6d6dce096d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:36.662824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3838b7-de07-4aa9-8634-6d6dce096d8e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:36.674494 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2790/5589] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:36.679127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11831e-347b-444d-8e18-fc0e870e08d6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:36.691527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11831e-347b-444d-8e18-fc0e870e08d6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:36.698931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62a530-c06c-4b44-9ebf-648ad06022c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:36.699796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62a530-c06c-4b44-9ebf-648ad06022c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:36.700298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62a530-c06c-4b44-9ebf-648ad06022c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:36.700801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62a530-c06c-4b44-9ebf-648ad06022c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:36.701495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62a530-c06c-4b44-9ebf-648ad06022c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:36.703232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62a530-c06c-4b44-9ebf-648ad06022c1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:36.717815 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2800/5590] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:11:36.719066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3838b7-de07-4aa9-8634-6d6dce096d8e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:36.733750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3838b7-de07-4aa9-8634-6d6dce096d8e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:36.741968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d5ed5-d0d2-4edd-be9e-d32a065284d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:36.742639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d5ed5-d0d2-4edd-be9e-d32a065284d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:36.742917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d5ed5-d0d2-4edd-be9e-d32a065284d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:36.743188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d5ed5-d0d2-4edd-be9e-d32a065284d6 admin admin] PATH_INFO: `/v3/projects/541d28f03e7e4222a24f8814ad549d97/users/d1dac3a09d654cb7a3f5c16e731a6798/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:36.762530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d5ed5-d0d2-4edd-be9e-d32a065284d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=541d28f03e7e4222a24f8814ad549d97, user_id=d1dac3a09d654cb7a3f5c16e731a6798, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:36.765260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d5ed5-d0d2-4edd-be9e-d32a065284d6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:36.780147 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2791/5591] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:36 2022] PUT /identity/v3/projects/541d28f03e7e4222a24f8814ad549d97/users/d1dac3a09d654cb7a3f5c16e731a6798/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:36.790978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b2e29-2c6b-4b7d-9935-7098773226a0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:36.790978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b2e29-2c6b-4b7d-9935-7098773226a0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:36.790978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b2e29-2c6b-4b7d-9935-7098773226a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:36.843255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4a6b2e29-2c6b-4b7d-9935-7098773226a0 None None] MFA Rules not processed for user `d1dac3a09d654cb7a3f5c16e731a6798`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:36.845123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4a6b2e29-2c6b-4b7d-9935-7098773226a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:36.977102 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2801/5592] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:36.984774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d5ed5-d0d2-4edd-be9e-d32a065284d6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:36.987793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e90d5ed5-d0d2-4edd-be9e-d32a065284d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:37.096469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d5ed5-d0d2-4edd-be9e-d32a065284d6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:37.102301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba409f-2ce4-4090-8f34-1ec6552ef7c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:37.103156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba409f-2ce4-4090-8f34-1ec6552ef7c9 service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:37.103589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba409f-2ce4-4090-8f34-1ec6552ef7c9 service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:37.104262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba409f-2ce4-4090-8f34-1ec6552ef7c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:37.182677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba409f-2ce4-4090-8f34-1ec6552ef7c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:37.184151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba409f-2ce4-4090-8f34-1ec6552ef7c9 service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:37.197630 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2792/5593] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:37.406803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081d044-25be-4a86-97f1-7ca5ba110831 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:37.407082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081d044-25be-4a86-97f1-7ca5ba110831 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:37.407345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081d044-25be-4a86-97f1-7ca5ba110831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:37.445587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b081d044-25be-4a86-97f1-7ca5ba110831 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:37.446889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b081d044-25be-4a86-97f1-7ca5ba110831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:37.564372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:37.581830 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2802/5594] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:37.656150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba409f-2ce4-4090-8f34-1ec6552ef7c9 service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:37.657712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e1ba409f-2ce4-4090-8f34-1ec6552ef7c9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:37.810508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:37.829514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba409f-2ce4-4090-8f34-1ec6552ef7c9 service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:37.845657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d884d-8afd-4e43-85a5-d26e94acada3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:37.846522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d884d-8afd-4e43-85a5-d26e94acada3 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:37.847783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d884d-8afd-4e43-85a5-d26e94acada3 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:37.848234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d884d-8afd-4e43-85a5-d26e94acada3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:37.866178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d884d-8afd-4e43-85a5-d26e94acada3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:37.868426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d884d-8afd-4e43-85a5-d26e94acada3 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:37.880447 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2793/5595] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:11:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:11:38.082402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081d044-25be-4a86-97f1-7ca5ba110831 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:38.084731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b081d044-25be-4a86-97f1-7ca5ba110831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:38.112870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d884d-8afd-4e43-85a5-d26e94acada3 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:38.116421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-412d884d-8afd-4e43-85a5-d26e94acada3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:38.157296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d884d-8afd-4e43-85a5-d26e94acada3 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:38.164176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3352b-dead-4d6f-abb5-15bf34bf5e5e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:38.164805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3352b-dead-4d6f-abb5-15bf34bf5e5e service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:38.165092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3352b-dead-4d6f-abb5-15bf34bf5e5e service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:38.166059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3352b-dead-4d6f-abb5-15bf34bf5e5e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:38.196710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3352b-dead-4d6f-abb5-15bf34bf5e5e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:38.198260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3352b-dead-4d6f-abb5-15bf34bf5e5e service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:38.231367 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2794/5596] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:38.256463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081d044-25be-4a86-97f1-7ca5ba110831 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:38.265814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:38.266824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:38.267397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:38.268514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] PATH_INFO: `/v3/users/74decb4847bb474d844b484d7b1f29b5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:38.270890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74decb4847bb474d844b484d7b1f29b5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:38.274020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:38.344042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74decb4847bb474d844b484d7b1f29b5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:38.368024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74decb4847bb474d844b484d7b1f29b5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:38.375750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ac0f5-7dfd-4064-95ce-e70b6e31a3f3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:38.376036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ac0f5-7dfd-4064-95ce-e70b6e31a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:38.376302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ac0f5-7dfd-4064-95ce-e70b6e31a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:38.396755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74decb4847bb474d844b484d7b1f29b5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:38.406795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74decb4847bb474d844b484d7b1f29b5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:38.408170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f223c33e-d4c8-4290-b197-46679e648945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74decb4847bb474d844b484d7b1f29b5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:38.418055 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2803/5597] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:38 2022] DELETE /identity/v3/users/74decb4847bb474d844b484d7b1f29b5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:38.424015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cedbeb-4c41-4680-8511-b16a8cc4c2b7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:38.424573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cedbeb-4c41-4680-8511-b16a8cc4c2b7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:38.425363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cedbeb-4c41-4680-8511-b16a8cc4c2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:38.428946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-400ac0f5-7dfd-4064-95ce-e70b6e31a3f3 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:38.430400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-400ac0f5-7dfd-4064-95ce-e70b6e31a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:38.453215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b9cedbeb-4c41-4680-8511-b16a8cc4c2b7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:38.454954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9cedbeb-4c41-4680-8511-b16a8cc4c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:38.562036 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2795/5598] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:11:38.586501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ac0f5-7dfd-4064-95ce-e70b6e31a3f3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:38.588115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-400ac0f5-7dfd-4064-95ce-e70b6e31a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:38.612960 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2804/5599] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:38.698938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ac0f5-7dfd-4064-95ce-e70b6e31a3f3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:38.705982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec619365-0760-4310-adee-f89cb2528436 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:38.706364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cedbeb-4c41-4680-8511-b16a8cc4c2b7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:38.707170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619365-0760-4310-adee-f89cb2528436 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:38.707688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619365-0760-4310-adee-f89cb2528436 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:38.707901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9cedbeb-4c41-4680-8511-b16a8cc4c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:38.709629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619365-0760-4310-adee-f89cb2528436 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:38.723874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec619365-0760-4310-adee-f89cb2528436 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:38.726383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec619365-0760-4310-adee-f89cb2528436 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:38.743380 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2796/5600] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:11:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:11:38.784378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:38.893606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cedbeb-4c41-4680-8511-b16a8cc4c2b7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:38.903098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd349474-c274-41c7-bcba-9549ae5bb999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:38.904028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd349474-c274-41c7-bcba-9549ae5bb999 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:38.904672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd349474-c274-41c7-bcba-9549ae5bb999 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:38.905831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd349474-c274-41c7-bcba-9549ae5bb999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:38.919349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd349474-c274-41c7-bcba-9549ae5bb999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:38.923070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd349474-c274-41c7-bcba-9549ae5bb999 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:38.940698 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2805/5601] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:38.990708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec619365-0760-4310-adee-f89cb2528436 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:38.995889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ec619365-0760-4310-adee-f89cb2528436 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:39.011656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec619365-0760-4310-adee-f89cb2528436 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:39.016635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:39.017427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:39.017779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:39.018062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] PATH_INFO: `/v3/users/2956edfb39484e38a5f8a44997dc4055` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:39.019411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2956edfb39484e38a5f8a44997dc4055)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:39.021642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:39.074778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2956edfb39484e38a5f8a44997dc4055'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:39.082770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2956edfb39484e38a5f8a44997dc4055'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:39.088594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:39.097999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2956edfb39484e38a5f8a44997dc4055'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:39.106195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2956edfb39484e38a5f8a44997dc4055'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:39.106818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2956edfb39484e38a5f8a44997dc4055'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:39.113696 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2797/5602] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:38 2022] DELETE /identity/v3/users/2956edfb39484e38a5f8a44997dc4055 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:39.119603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6e77a-e974-4aa6-9bc4-764e8ca5e8d2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:39.120194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6e77a-e974-4aa6-9bc4-764e8ca5e8d2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:39.120194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6e77a-e974-4aa6-9bc4-764e8ca5e8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:39.141338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-afd6e77a-e974-4aa6-9bc4-764e8ca5e8d2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:39.142570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-afd6e77a-e974-4aa6-9bc4-764e8ca5e8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:39.214832 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2806/5603] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:39.226004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:39.227447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:39.296583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864bc87-819f-42ac-9e8f-dfcb8a6659bb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:39.300809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d7a74-4495-46ae-86a1-e8f4e82576b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:39.302040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d7a74-4495-46ae-86a1-e8f4e82576b8 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:39.302761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d7a74-4495-46ae-86a1-e8f4e82576b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:39.302761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d7a74-4495-46ae-86a1-e8f4e82576b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:39.315149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d7a74-4495-46ae-86a1-e8f4e82576b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:39.317621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d7a74-4495-46ae-86a1-e8f4e82576b8 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:39.349745 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2798/5604] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:11:39.694187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6e77a-e974-4aa6-9bc4-764e8ca5e8d2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:39.696579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-afd6e77a-e974-4aa6-9bc4-764e8ca5e8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:39.719921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6e77a-e974-4aa6-9bc4-764e8ca5e8d2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:39.737494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:39.738402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:39.738707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:39.740682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] PATH_INFO: `/v3/projects/b6313360f88643df9e2a5e9dee09d91e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:39.743510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6313360f88643df9e2a5e9dee09d91e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:39.745504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:39.825215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6313360f88643df9e2a5e9dee09d91e', 'request_id': 'req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:39.834460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d7a74-4495-46ae-86a1-e8f4e82576b8 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:39.836233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c05d7a74-4495-46ae-86a1-e8f4e82576b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:39.846765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6313360f88643df9e2a5e9dee09d91e', 'request_id': 'req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:39.870009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d7a74-4495-46ae-86a1-e8f4e82576b8 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:39.889768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6313360f88643df9e2a5e9dee09d91e', 'request_id': 'req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:39.907976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] The token cache is being invalidate because project b6313360f88643df9e2a5e9dee09d91e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:39.908959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:39.910469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5aad52d0-9a95-4067-a3bd-f51a6cf3f9a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:39.916273 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2807/5605] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:39 2022] DELETE /identity/v3/projects/b6313360f88643df9e2a5e9dee09d91e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:39.923152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02726bcf-c0e9-4e41-9920-f07be5fd7b3e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:39.924110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02726bcf-c0e9-4e41-9920-f07be5fd7b3e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:39.924755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02726bcf-c0e9-4e41-9920-f07be5fd7b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:39.956612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-02726bcf-c0e9-4e41-9920-f07be5fd7b3e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:39.959067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-02726bcf-c0e9-4e41-9920-f07be5fd7b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:39.996828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:39.997510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:39.997990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:39.998426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] PATH_INFO: `/v3/projects/cc99f6958443492f8e87cc53ab0ab200` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:39.999824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc99f6958443492f8e87cc53ab0ab200)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:40.002076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:40.050908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:40.074465 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2808/5606] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:40.084998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e905e-7bf4-42b2-8fc7-082eb7e5e045 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:40.085764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e905e-7bf4-42b2-8fc7-082eb7e5e045 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:40.086134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e905e-7bf4-42b2-8fc7-082eb7e5e045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:40.087587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc99f6958443492f8e87cc53ab0ab200', 'request_id': 'req-185267cd-1650-4f21-98be-9f0664a5c1a5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:40.100146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc99f6958443492f8e87cc53ab0ab200', 'request_id': 'req-185267cd-1650-4f21-98be-9f0664a5c1a5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:40.110477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e61e905e-7bf4-42b2-8fc7-082eb7e5e045 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:40.110704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc99f6958443492f8e87cc53ab0ab200', 'request_id': 'req-185267cd-1650-4f21-98be-9f0664a5c1a5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:40.115768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e61e905e-7bf4-42b2-8fc7-082eb7e5e045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:40.133159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] The token cache is being invalidate because project cc99f6958443492f8e87cc53ab0ab200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:40.133615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:40.134613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:40.140195 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2799/5607] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:39 2022] DELETE /identity/v3/projects/cc99f6958443492f8e87cc53ab0ab200 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:40.141607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:40.143795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:40.168684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:40.215122 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2809/5608] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:40.219273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e905e-7bf4-42b2-8fc7-082eb7e5e045 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:40.243560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e905e-7bf4-42b2-8fc7-082eb7e5e045 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:40.259151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da07dfb-f27e-4944-8980-b1083420c1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:40.263616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da07dfb-f27e-4944-8980-b1083420c1ec admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:40.263901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da07dfb-f27e-4944-8980-b1083420c1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:40.264175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da07dfb-f27e-4944-8980-b1083420c1ec admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:40.264906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da07dfb-f27e-4944-8980-b1083420c1ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:40.265235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da07dfb-f27e-4944-8980-b1083420c1ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:40.268292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da07dfb-f27e-4944-8980-b1083420c1ec admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:40.272928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185267cd-1650-4f21-98be-9f0664a5c1a5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:40.279052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:40.279829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:40.280230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:40.280601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] PATH_INFO: `/v3/users/d1dac3a09d654cb7a3f5c16e731a6798` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:40.282607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1dac3a09d654cb7a3f5c16e731a6798)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:40.287353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:40.296068 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2810/5609] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:40.300640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da07dfb-f27e-4944-8980-b1083420c1ec admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:40.324934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da07dfb-f27e-4944-8980-b1083420c1ec admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:40.329750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff0a7f-9ca8-4be1-9030-06c832a0f867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:40.330356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff0a7f-9ca8-4be1-9030-06c832a0f867 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:40.330626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff0a7f-9ca8-4be1-9030-06c832a0f867 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:40.330884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff0a7f-9ca8-4be1-9030-06c832a0f867 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:40.331650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff0a7f-9ca8-4be1-9030-06c832a0f867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:40.334647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff0a7f-9ca8-4be1-9030-06c832a0f867 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:40.406605 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2811/5610] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:40 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:40.411563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff0a7f-9ca8-4be1-9030-06c832a0f867 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:40.416239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1dac3a09d654cb7a3f5c16e731a6798'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:40.437029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1dac3a09d654cb7a3f5c16e731a6798'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:40.461464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1dac3a09d654cb7a3f5c16e731a6798'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:40.474163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1dac3a09d654cb7a3f5c16e731a6798'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:40.474777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1dac3a09d654cb7a3f5c16e731a6798'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:40.483623 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2800/5611] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:40 2022] DELETE /identity/v3/users/d1dac3a09d654cb7a3f5c16e731a6798 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:40.545506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff0a7f-9ca8-4be1-9030-06c832a0f867 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:40.674667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1b587-d92b-41c0-add1-ee100100ec9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:40.674667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1b587-d92b-41c0-add1-ee100100ec9d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:40.674996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1b587-d92b-41c0-add1-ee100100ec9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:40.675192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1b587-d92b-41c0-add1-ee100100ec9d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:40.675950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1b587-d92b-41c0-add1-ee100100ec9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:40.679077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1b587-d92b-41c0-add1-ee100100ec9d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:40.689088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:40.733456 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2812/5612] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:40 2022] POST /identity/v3/users => generated 286 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:40.737013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:40.738766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:40.754315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f76cd5-683f-4a1b-a5dd-6a83b2fc632a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:40.761395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19934a6-4a97-45cd-9fb0-58c9cbe4961c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:40.762614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19934a6-4a97-45cd-9fb0-58c9cbe4961c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:40.763085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19934a6-4a97-45cd-9fb0-58c9cbe4961c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:40.763553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19934a6-4a97-45cd-9fb0-58c9cbe4961c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:40.765204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19934a6-4a97-45cd-9fb0-58c9cbe4961c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:40.768410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19934a6-4a97-45cd-9fb0-58c9cbe4961c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:40.794953 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2801/5613] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:40.798576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1b587-d92b-41c0-add1-ee100100ec9d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:40.811735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1b587-d92b-41c0-add1-ee100100ec9d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:40.822458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623e465-ebdb-4941-8338-847c79d529b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:40.824013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623e465-ebdb-4941-8338-847c79d529b0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:40.824601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623e465-ebdb-4941-8338-847c79d529b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:40.825451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623e465-ebdb-4941-8338-847c79d529b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:40.826341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623e465-ebdb-4941-8338-847c79d529b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:40.828930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623e465-ebdb-4941-8338-847c79d529b0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:40.853417 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2813/5614] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:40.856695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19934a6-4a97-45cd-9fb0-58c9cbe4961c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:40.868455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19934a6-4a97-45cd-9fb0-58c9cbe4961c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:40.873516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35f1b5-27bf-4476-85b9-f1dc97680a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:40.874187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35f1b5-27bf-4476-85b9-f1dc97680a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:40.874462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35f1b5-27bf-4476-85b9-f1dc97680a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:40.874734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35f1b5-27bf-4476-85b9-f1dc97680a57 admin admin] PATH_INFO: `/v3/projects/bbff1010e0954b2187b8c333fe0dfe1c/users/2e580a9f5307454d86f6430ade4910ef/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:40.893688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35f1b5-27bf-4476-85b9-f1dc97680a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbff1010e0954b2187b8c333fe0dfe1c, user_id=2e580a9f5307454d86f6430ade4910ef, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:40.898454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35f1b5-27bf-4476-85b9-f1dc97680a57 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:40.939260 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2802/5615] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:40 2022] PUT /identity/v3/projects/bbff1010e0954b2187b8c333fe0dfe1c/users/2e580a9f5307454d86f6430ade4910ef/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:40.951191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174390c2-1504-424f-b4b6-fe4f359a4e97 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:40.951706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174390c2-1504-424f-b4b6-fe4f359a4e97 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:40.952184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174390c2-1504-424f-b4b6-fe4f359a4e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:40.991725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-174390c2-1504-424f-b4b6-fe4f359a4e97 None None] MFA Rules not processed for user `2e580a9f5307454d86f6430ade4910ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:40.993350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-174390c2-1504-424f-b4b6-fe4f359a4e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:41.116183 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2814/5616] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:41.129920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35f1b5-27bf-4476-85b9-f1dc97680a57 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:41.132588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8d35f1b5-27bf-4476-85b9-f1dc97680a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:41.213980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35f1b5-27bf-4476-85b9-f1dc97680a57 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:41.218847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac7abf-c5d3-4766-80a4-d950050a1cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:41.219467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac7abf-c5d3-4766-80a4-d950050a1cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:41.219743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac7abf-c5d3-4766-80a4-d950050a1cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:41.220007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac7abf-c5d3-4766-80a4-d950050a1cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:41.230850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac7abf-c5d3-4766-80a4-d950050a1cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:41.233554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac7abf-c5d3-4766-80a4-d950050a1cd8 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:41.242267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:41.251145 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2803/5617] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:41.588985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174390c2-1504-424f-b4b6-fe4f359a4e97 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:41.591927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-174390c2-1504-424f-b4b6-fe4f359a4e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:41.845192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174390c2-1504-424f-b4b6-fe4f359a4e97 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:41.852020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:41.853881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:41.854235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:41.854706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] PATH_INFO: `/v3/projects/541d28f03e7e4222a24f8814ad549d97` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:41.855935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=541d28f03e7e4222a24f8814ad549d97)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:41.859154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:41.959888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '541d28f03e7e4222a24f8814ad549d97', 'request_id': 'req-0e22ba41-0d01-4909-8c41-b304b71299b0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:41.972864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '541d28f03e7e4222a24f8814ad549d97', 'request_id': 'req-0e22ba41-0d01-4909-8c41-b304b71299b0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:41.985415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '541d28f03e7e4222a24f8814ad549d97', 'request_id': 'req-0e22ba41-0d01-4909-8c41-b304b71299b0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:42.000406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:42.012477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] The token cache is being invalidate because project 541d28f03e7e4222a24f8814ad549d97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:42.013483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:42.014916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0e22ba41-0d01-4909-8c41-b304b71299b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:42.016905 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2815/5618] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:41 2022] DELETE /identity/v3/projects/541d28f03e7e4222a24f8814ad549d97 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:42.023316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318a27a-087c-4918-88d6-3727e6796540 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:42.023850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318a27a-087c-4918-88d6-3727e6796540 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:42.023850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318a27a-087c-4918-88d6-3727e6796540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:42.052343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1318a27a-087c-4918-88d6-3727e6796540 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:42.066102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1318a27a-087c-4918-88d6-3727e6796540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:42.156274 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2804/5619] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:42.165385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716edd2e-2c79-4366-b777-0d90c410c972 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:42.165672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716edd2e-2c79-4366-b777-0d90c410c972 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:42.165950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716edd2e-2c79-4366-b777-0d90c410c972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:42.200933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-716edd2e-2c79-4366-b777-0d90c410c972 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:42.213418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-716edd2e-2c79-4366-b777-0d90c410c972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:42.237410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318a27a-087c-4918-88d6-3727e6796540 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:42.239833 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2816/5620] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:42.255720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318a27a-087c-4918-88d6-3727e6796540 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:42.261156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de91b70-6bee-46dd-bc32-80c4fedbe8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:42.261758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de91b70-6bee-46dd-bc32-80c4fedbe8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:42.262026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de91b70-6bee-46dd-bc32-80c4fedbe8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:42.262288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de91b70-6bee-46dd-bc32-80c4fedbe8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:42.262924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de91b70-6bee-46dd-bc32-80c4fedbe8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:42.263219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de91b70-6bee-46dd-bc32-80c4fedbe8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:42.264791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de91b70-6bee-46dd-bc32-80c4fedbe8d1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:42.299725 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2805/5621] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:42.300826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716edd2e-2c79-4366-b777-0d90c410c972 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:42.319281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716edd2e-2c79-4366-b777-0d90c410c972 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:42.328256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081c19d-819a-4f81-a2b6-e8791b21ceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:42.329083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081c19d-819a-4f81-a2b6-e8791b21ceb1 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:42.329472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081c19d-819a-4f81-a2b6-e8791b21ceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:42.329848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081c19d-819a-4f81-a2b6-e8791b21ceb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:42.330588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081c19d-819a-4f81-a2b6-e8791b21ceb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:42.332982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081c19d-819a-4f81-a2b6-e8791b21ceb1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:42.398341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de91b70-6bee-46dd-bc32-80c4fedbe8d1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:42.408938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2817/5622] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:42 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:42.411992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de91b70-6bee-46dd-bc32-80c4fedbe8d1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:42.506173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524955c6-a1bb-4413-81b0-d484e33f6c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:42.506952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524955c6-a1bb-4413-81b0-d484e33f6c54 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:42.507231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524955c6-a1bb-4413-81b0-d484e33f6c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:42.507497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524955c6-a1bb-4413-81b0-d484e33f6c54 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:42.508256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524955c6-a1bb-4413-81b0-d484e33f6c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:42.510236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524955c6-a1bb-4413-81b0-d484e33f6c54 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:42.539594 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2806/5623] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:42 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:42.545389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081c19d-819a-4f81-a2b6-e8791b21ceb1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:42.557475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081c19d-819a-4f81-a2b6-e8791b21ceb1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:42.562538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0f0ed-ed34-41da-a9b0-ffbfd80ff2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:42.563265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0f0ed-ed34-41da-a9b0-ffbfd80ff2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:42.563664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0f0ed-ed34-41da-a9b0-ffbfd80ff2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:42.564038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0f0ed-ed34-41da-a9b0-ffbfd80ff2f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:42.564856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0f0ed-ed34-41da-a9b0-ffbfd80ff2f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:42.569204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0f0ed-ed34-41da-a9b0-ffbfd80ff2f4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:42.583776 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2818/5624] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:42.584990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524955c6-a1bb-4413-81b0-d484e33f6c54 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:42.596832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524955c6-a1bb-4413-81b0-d484e33f6c54 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:42.602884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a64a83-eedb-41f0-b7c6-3814662cc681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:42.603498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a64a83-eedb-41f0-b7c6-3814662cc681 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:42.603779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a64a83-eedb-41f0-b7c6-3814662cc681 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:42.604786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a64a83-eedb-41f0-b7c6-3814662cc681 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:42.604786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a64a83-eedb-41f0-b7c6-3814662cc681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:42.606591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a64a83-eedb-41f0-b7c6-3814662cc681 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:42.621836 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2807/5625] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:42.624868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0f0ed-ed34-41da-a9b0-ffbfd80ff2f4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:42.638729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0f0ed-ed34-41da-a9b0-ffbfd80ff2f4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:42.652539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42df63-3863-4e83-bb9d-6cbe31a109b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:42.653478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42df63-3863-4e83-bb9d-6cbe31a109b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:42.653758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42df63-3863-4e83-bb9d-6cbe31a109b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:42.654029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42df63-3863-4e83-bb9d-6cbe31a109b3 admin admin] PATH_INFO: `/v3/projects/3a37c0fbf636487494d1aaff719972f1/users/9ae089b3d163453a8f27575c77bb3f6b/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:42.704290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42df63-3863-4e83-bb9d-6cbe31a109b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a37c0fbf636487494d1aaff719972f1, user_id=9ae089b3d163453a8f27575c77bb3f6b, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:42.711596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42df63-3863-4e83-bb9d-6cbe31a109b3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:42.747876 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2819/5626] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:42 2022] PUT /identity/v3/projects/3a37c0fbf636487494d1aaff719972f1/users/9ae089b3d163453a8f27575c77bb3f6b/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:42.848736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e9720-0e17-4b07-b920-ad991982fa4c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:42.849135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e9720-0e17-4b07-b920-ad991982fa4c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:42.849360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e9720-0e17-4b07-b920-ad991982fa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:42.885075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f58e9720-0e17-4b07-b920-ad991982fa4c None None] MFA Rules not processed for user `9ae089b3d163453a8f27575c77bb3f6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:42.886426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f58e9720-0e17-4b07-b920-ad991982fa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:42.968121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:42.984279 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2808/5627] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:42.985922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42df63-3863-4e83-bb9d-6cbe31a109b3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:43.093294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42df63-3863-4e83-bb9d-6cbe31a109b3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:43.102217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d193c-ff4f-40b8-bdbb-7987a73ad5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:43.103129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d193c-ff4f-40b8-bdbb-7987a73ad5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:43.103785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d193c-ff4f-40b8-bdbb-7987a73ad5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:43.104228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d193c-ff4f-40b8-bdbb-7987a73ad5a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:43.104986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d193c-ff4f-40b8-bdbb-7987a73ad5a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:43.107313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d193c-ff4f-40b8-bdbb-7987a73ad5a8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:43.159806 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2820/5628] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:42 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:43.164780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e9720-0e17-4b07-b920-ad991982fa4c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:43.184217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e9720-0e17-4b07-b920-ad991982fa4c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:43.274514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0993582-2731-4a1c-a143-dc3f73d0b7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:43.275435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0993582-2731-4a1c-a143-dc3f73d0b7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:43.275827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0993582-2731-4a1c-a143-dc3f73d0b7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:43.276189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0993582-2731-4a1c-a143-dc3f73d0b7c4 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:43.276916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0993582-2731-4a1c-a143-dc3f73d0b7c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:43.279106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0993582-2731-4a1c-a143-dc3f73d0b7c4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:43.310108 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2809/5629] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:43 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:43.313968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d193c-ff4f-40b8-bdbb-7987a73ad5a8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:43.348677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d193c-ff4f-40b8-bdbb-7987a73ad5a8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:43.380521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c8ec8-0221-4140-ad8e-3e8931907c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:43.384612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c8ec8-0221-4140-ad8e-3e8931907c4a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:43.385681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c8ec8-0221-4140-ad8e-3e8931907c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:43.387398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c8ec8-0221-4140-ad8e-3e8931907c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:43.387398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c8ec8-0221-4140-ad8e-3e8931907c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:43.394496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c8ec8-0221-4140-ad8e-3e8931907c4a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:43.436242 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2821/5630] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:43.442237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0993582-2731-4a1c-a143-dc3f73d0b7c4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:43.472117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0993582-2731-4a1c-a143-dc3f73d0b7c4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:43.504654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b2df6-dc38-47ea-8d54-575b5d2dd323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:43.505315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b2df6-dc38-47ea-8d54-575b5d2dd323 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:43.505774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b2df6-dc38-47ea-8d54-575b5d2dd323 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:43.506416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b2df6-dc38-47ea-8d54-575b5d2dd323 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:43.507107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b2df6-dc38-47ea-8d54-575b5d2dd323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:43.509021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b2df6-dc38-47ea-8d54-575b5d2dd323 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:43.524634 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2810/5631] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:43.528367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c8ec8-0221-4140-ad8e-3e8931907c4a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:43.544445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c8ec8-0221-4140-ad8e-3e8931907c4a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:43.555983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b99b0f-4673-42e3-b076-bc5fe03c62e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:43.556664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b99b0f-4673-42e3-b076-bc5fe03c62e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:43.556941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b99b0f-4673-42e3-b076-bc5fe03c62e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:43.557237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b99b0f-4673-42e3-b076-bc5fe03c62e1 admin admin] PATH_INFO: `/v3/projects/1fc36c992dba4cfaaf702c7304f48d4a/users/80237d50f3514e95ae4bd13c0f1dfe31/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:43.602612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b99b0f-4673-42e3-b076-bc5fe03c62e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc36c992dba4cfaaf702c7304f48d4a, user_id=80237d50f3514e95ae4bd13c0f1dfe31, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:43.605422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b99b0f-4673-42e3-b076-bc5fe03c62e1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:43.638418 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2822/5632] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:43 2022] PUT /identity/v3/projects/1fc36c992dba4cfaaf702c7304f48d4a/users/80237d50f3514e95ae4bd13c0f1dfe31/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:43.649639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8156e5-96d6-4bd1-8ba1-f94595d5e70c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:43.649981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8156e5-96d6-4bd1-8ba1-f94595d5e70c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:43.650353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8156e5-96d6-4bd1-8ba1-f94595d5e70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:43.705975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7a8156e5-96d6-4bd1-8ba1-f94595d5e70c None None] MFA Rules not processed for user `80237d50f3514e95ae4bd13c0f1dfe31`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:43.716591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7a8156e5-96d6-4bd1-8ba1-f94595d5e70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:43.909220 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2811/5633] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:43.911110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b99b0f-4673-42e3-b076-bc5fe03c62e1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:43.912855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b1b99b0f-4673-42e3-b076-bc5fe03c62e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:44.020182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:44.039348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b99b0f-4673-42e3-b076-bc5fe03c62e1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:44.049376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80e50a-8f9b-4ae2-9ce2-9de83250182f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:44.050005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80e50a-8f9b-4ae2-9ce2-9de83250182f service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:44.050286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80e50a-8f9b-4ae2-9ce2-9de83250182f service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:44.050557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80e50a-8f9b-4ae2-9ce2-9de83250182f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:44.146768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80e50a-8f9b-4ae2-9ce2-9de83250182f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:44.148744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80e50a-8f9b-4ae2-9ce2-9de83250182f service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:44.174667 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2823/5634] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:44.340182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8156e5-96d6-4bd1-8ba1-f94595d5e70c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:44.358188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8156e5-96d6-4bd1-8ba1-f94595d5e70c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:44.363344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14009e-b69c-4dff-81b9-28fc2e3f295c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:44.364586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14009e-b69c-4dff-81b9-28fc2e3f295c service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:44.364928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14009e-b69c-4dff-81b9-28fc2e3f295c service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:44.365301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14009e-b69c-4dff-81b9-28fc2e3f295c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:44.384620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14009e-b69c-4dff-81b9-28fc2e3f295c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:44.386447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14009e-b69c-4dff-81b9-28fc2e3f295c service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:44.400412 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2812/5635] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:44.929328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80e50a-8f9b-4ae2-9ce2-9de83250182f service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:45.013127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80e50a-8f9b-4ae2-9ce2-9de83250182f service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:45.022635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:45.023626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:45.023915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:45.024245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] PATH_INFO: `/v3/users/9ae089b3d163453a8f27575c77bb3f6b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:45.025431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae089b3d163453a8f27575c77bb3f6b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:45.027545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:45.104016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae089b3d163453a8f27575c77bb3f6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:45.120848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae089b3d163453a8f27575c77bb3f6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:45.132773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae089b3d163453a8f27575c77bb3f6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:45.146257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae089b3d163453a8f27575c77bb3f6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:45.148631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae089b3d163453a8f27575c77bb3f6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:45.158582 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2824/5636] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:44 2022] DELETE /identity/v3/users/9ae089b3d163453a8f27575c77bb3f6b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:45.163108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c391671-9852-4b9a-96d1-0d92c0a11746 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:45.163396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c391671-9852-4b9a-96d1-0d92c0a11746 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:45.163677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c391671-9852-4b9a-96d1-0d92c0a11746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:45.186089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1c391671-9852-4b9a-96d1-0d92c0a11746 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:45.187386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1c391671-9852-4b9a-96d1-0d92c0a11746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:45.258502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:45.282420 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2813/5637] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:45.306946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:45.309076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:45.333879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:45.395849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee83bc-825b-4b8d-a3a8-584b859a2922 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:45.400884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa831d-102b-4740-a85c-3d163ae03cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:45.401962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa831d-102b-4740-a85c-3d163ae03cab service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:45.402099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa831d-102b-4740-a85c-3d163ae03cab service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:45.402472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa831d-102b-4740-a85c-3d163ae03cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:45.418873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa831d-102b-4740-a85c-3d163ae03cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:45.424082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa831d-102b-4740-a85c-3d163ae03cab service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:45.443337 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2825/5638] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:46.684258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c391671-9852-4b9a-96d1-0d92c0a11746 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:46.685969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1c391671-9852-4b9a-96d1-0d92c0a11746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:46.708442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c391671-9852-4b9a-96d1-0d92c0a11746 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:46.715847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:46.716898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:46.717504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:46.718001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] PATH_INFO: `/v3/projects/3a37c0fbf636487494d1aaff719972f1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:46.719697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a37c0fbf636487494d1aaff719972f1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:46.722300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:46.839178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a37c0fbf636487494d1aaff719972f1', 'request_id': 'req-f72e85cc-3aa1-4af3-b77e-502ebf419666'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:46.856707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a37c0fbf636487494d1aaff719972f1', 'request_id': 'req-f72e85cc-3aa1-4af3-b77e-502ebf419666'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:46.867441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a37c0fbf636487494d1aaff719972f1', 'request_id': 'req-f72e85cc-3aa1-4af3-b77e-502ebf419666'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:46.880213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] The token cache is being invalidate because project 3a37c0fbf636487494d1aaff719972f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:46.880788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:46.881564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:46.883795 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2814/5639] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:46 2022] DELETE /identity/v3/projects/3a37c0fbf636487494d1aaff719972f1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:46.888361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa831d-102b-4740-a85c-3d163ae03cab service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:46.900016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa831d-102b-4740-a85c-3d163ae03cab service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:46.981905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:46.982512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:46.982786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:46.983052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] PATH_INFO: `/v3/users/80237d50f3514e95ae4bd13c0f1dfe31` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:46.984637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80237d50f3514e95ae4bd13c0f1dfe31)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:46.986996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:47.061742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80237d50f3514e95ae4bd13c0f1dfe31'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:47.084325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80237d50f3514e95ae4bd13c0f1dfe31'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:47.101063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80237d50f3514e95ae4bd13c0f1dfe31'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:47.112806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80237d50f3514e95ae4bd13c0f1dfe31'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:47.114985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0686605-9a39-4958-b2d6-8d6cc9119427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80237d50f3514e95ae4bd13c0f1dfe31'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:47.124806 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2826/5640] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:46 2022] DELETE /identity/v3/users/80237d50f3514e95ae4bd13c0f1dfe31 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:48.095629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:48.097132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:48.136440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e85cc-3aa1-4af3-b77e-502ebf419666 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:48.246678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:48.248091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:48.248736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:48.249840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] PATH_INFO: `/v3/projects/1fc36c992dba4cfaaf702c7304f48d4a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:48.252052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc36c992dba4cfaaf702c7304f48d4a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:48.255001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:48.316645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc36c992dba4cfaaf702c7304f48d4a', 'request_id': 'req-af9923e4-5b15-4e1e-987f-f3987183efc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:48.329306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc36c992dba4cfaaf702c7304f48d4a', 'request_id': 'req-af9923e4-5b15-4e1e-987f-f3987183efc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:48.338753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc36c992dba4cfaaf702c7304f48d4a', 'request_id': 'req-af9923e4-5b15-4e1e-987f-f3987183efc6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:48.355747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] The token cache is being invalidate because project 1fc36c992dba4cfaaf702c7304f48d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:48.356084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:48.357515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-af9923e4-5b15-4e1e-987f-f3987183efc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:48.366397 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2815/5641] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:48 2022] DELETE /identity/v3/projects/1fc36c992dba4cfaaf702c7304f48d4a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:48.369085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b93e7c-9517-4b7a-9694-c027f1c6173e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:48.369580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b93e7c-9517-4b7a-9694-c027f1c6173e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:48.369580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b93e7c-9517-4b7a-9694-c027f1c6173e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:48.395475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c8b93e7c-9517-4b7a-9694-c027f1c6173e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:48.396945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c8b93e7c-9517-4b7a-9694-c027f1c6173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:48.441027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:48.487937 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2827/5642] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:48.490566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d2ddb-3bd1-48b6-9b44-e1979d05beb3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:48.490857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d2ddb-3bd1-48b6-9b44-e1979d05beb3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:48.491176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d2ddb-3bd1-48b6-9b44-e1979d05beb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:48.502022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:48.517722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d19d2ddb-3bd1-48b6-9b44-e1979d05beb3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:48.519509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d19d2ddb-3bd1-48b6-9b44-e1979d05beb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:48.542194 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2816/5643] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:48.546249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b93e7c-9517-4b7a-9694-c027f1c6173e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:48.561870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b93e7c-9517-4b7a-9694-c027f1c6173e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:48.571886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6936-51d5-4c2b-8c09-b4d0a391e7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:48.574745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6936-51d5-4c2b-8c09-b4d0a391e7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:48.575307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6936-51d5-4c2b-8c09-b4d0a391e7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:48.575586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6936-51d5-4c2b-8c09-b4d0a391e7b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:48.576262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e6936-51d5-4c2b-8c09-b4d0a391e7b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:48.576563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e6936-51d5-4c2b-8c09-b4d0a391e7b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:48.578515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e6936-51d5-4c2b-8c09-b4d0a391e7b6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:48.592252 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2828/5644] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:48.596898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d2ddb-3bd1-48b6-9b44-e1979d05beb3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:48.607806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d2ddb-3bd1-48b6-9b44-e1979d05beb3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:48.612929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e72d1-c3ed-459a-b47f-7d3cb7344e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:48.614850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e72d1-c3ed-459a-b47f-7d3cb7344e94 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:48.615210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e72d1-c3ed-459a-b47f-7d3cb7344e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:48.615612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e72d1-c3ed-459a-b47f-7d3cb7344e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:48.616318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e72d1-c3ed-459a-b47f-7d3cb7344e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:48.618353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e72d1-c3ed-459a-b47f-7d3cb7344e94 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:48.653911 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2817/5645] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:48 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:48.658337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6936-51d5-4c2b-8c09-b4d0a391e7b6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:48.670189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6936-51d5-4c2b-8c09-b4d0a391e7b6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:48.749313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13af37-4c04-4ac2-afc6-bbf8e3830325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:48.750049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13af37-4c04-4ac2-afc6-bbf8e3830325 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:48.750386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13af37-4c04-4ac2-afc6-bbf8e3830325 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:48.750719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13af37-4c04-4ac2-afc6-bbf8e3830325 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:48.752030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13af37-4c04-4ac2-afc6-bbf8e3830325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:48.754225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13af37-4c04-4ac2-afc6-bbf8e3830325 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:48.789753 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2829/5646] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:48 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:48.791243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e72d1-c3ed-459a-b47f-7d3cb7344e94 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:48.805674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e72d1-c3ed-459a-b47f-7d3cb7344e94 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:48.810515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc857ab1-efe6-4fcd-b676-841187dde6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:48.811214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc857ab1-efe6-4fcd-b676-841187dde6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:48.811559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc857ab1-efe6-4fcd-b676-841187dde6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:48.811897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc857ab1-efe6-4fcd-b676-841187dde6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:48.812601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc857ab1-efe6-4fcd-b676-841187dde6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:48.814534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc857ab1-efe6-4fcd-b676-841187dde6a8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:48.829925 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2818/5647] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:11:48.831014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13af37-4c04-4ac2-afc6-bbf8e3830325 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:48.842848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13af37-4c04-4ac2-afc6-bbf8e3830325 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:48.847943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccc124-67b5-44e2-843b-2fdc984a91ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:48.849296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccc124-67b5-44e2-843b-2fdc984a91ec admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:48.849450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccc124-67b5-44e2-843b-2fdc984a91ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:48.849801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccc124-67b5-44e2-843b-2fdc984a91ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:48.850653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccc124-67b5-44e2-843b-2fdc984a91ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:48.852320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccc124-67b5-44e2-843b-2fdc984a91ec admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:48.868860 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2830/5648] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:48.873998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc857ab1-efe6-4fcd-b676-841187dde6a8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:48.891184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc857ab1-efe6-4fcd-b676-841187dde6a8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:48.897565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f06d56-4006-442e-b3cb-ece63a5728f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:48.898764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f06d56-4006-442e-b3cb-ece63a5728f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:48.899263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f06d56-4006-442e-b3cb-ece63a5728f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:48.899738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f06d56-4006-442e-b3cb-ece63a5728f0 admin admin] PATH_INFO: `/v3/projects/9a1c5f0c627a41b4a60f7d20ca9ff753/users/fca16fd2d377433c98ed40a24395111a/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:48.924010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f06d56-4006-442e-b3cb-ece63a5728f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1c5f0c627a41b4a60f7d20ca9ff753, user_id=fca16fd2d377433c98ed40a24395111a, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:48.928754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f06d56-4006-442e-b3cb-ece63a5728f0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:48.956415 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2819/5649] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:48 2022] PUT /identity/v3/projects/9a1c5f0c627a41b4a60f7d20ca9ff753/users/fca16fd2d377433c98ed40a24395111a/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:11:48.969547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732439af-1fb7-4b25-88d7-232c0e78b78b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:48.969547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732439af-1fb7-4b25-88d7-232c0e78b78b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:48.969807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732439af-1fb7-4b25-88d7-232c0e78b78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:49.021354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-732439af-1fb7-4b25-88d7-232c0e78b78b None None] MFA Rules not processed for user `fca16fd2d377433c98ed40a24395111a`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:49.023737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-732439af-1fb7-4b25-88d7-232c0e78b78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:49.113944 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2831/5650] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:11:49.168947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f06d56-4006-442e-b3cb-ece63a5728f0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:49.171860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f3f06d56-4006-442e-b3cb-ece63a5728f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:49.254628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f06d56-4006-442e-b3cb-ece63a5728f0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:49.258418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875aeced-fc2a-4b22-bcc4-e88fb518fcf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:49.259031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aeced-fc2a-4b22-bcc4-e88fb518fcf1 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:49.259308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aeced-fc2a-4b22-bcc4-e88fb518fcf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:49.259578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aeced-fc2a-4b22-bcc4-e88fb518fcf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:49.269152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875aeced-fc2a-4b22-bcc4-e88fb518fcf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:49.270632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875aeced-fc2a-4b22-bcc4-e88fb518fcf1 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:49.285102 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2820/5651] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:11:53.630151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732439af-1fb7-4b25-88d7-232c0e78b78b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:53.631666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-732439af-1fb7-4b25-88d7-232c0e78b78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:53.712632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:53.763901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732439af-1fb7-4b25-88d7-232c0e78b78b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:53.769014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:53.770041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:53.771577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:53.771859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] PATH_INFO: `/v3/users/b9a65c6e684a4385ba56269757641b88` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:53.773171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a65c6e684a4385ba56269757641b88)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:53.784801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:53.911684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a65c6e684a4385ba56269757641b88'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:53.945750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a65c6e684a4385ba56269757641b88'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:53.955525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a65c6e684a4385ba56269757641b88'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:53.970204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a65c6e684a4385ba56269757641b88'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:53.971186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a65c6e684a4385ba56269757641b88'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:53.983723 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2832/5652] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:11:53 2022] DELETE /identity/v3/users/b9a65c6e684a4385ba56269757641b88 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:53.989157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c191c-dd36-46ff-a412-dc95d6fbfb90 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:53.990923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c191c-dd36-46ff-a412-dc95d6fbfb90 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:53.991202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c191c-dd36-46ff-a412-dc95d6fbfb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:54.012491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e17c191c-dd36-46ff-a412-dc95d6fbfb90 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:54.014107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e17c191c-dd36-46ff-a412-dc95d6fbfb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:54.103158 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2821/5653] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:54.114083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:54.115528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:54.193526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c6fa2-20fe-44a3-9a35-74b61fd15bbd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:54.198066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413980c-bbb1-4fac-8dda-9df61c5d568a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:54.198895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413980c-bbb1-4fac-8dda-9df61c5d568a service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:54.199328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413980c-bbb1-4fac-8dda-9df61c5d568a service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:54.199781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413980c-bbb1-4fac-8dda-9df61c5d568a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:54.212561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413980c-bbb1-4fac-8dda-9df61c5d568a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:54.214308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413980c-bbb1-4fac-8dda-9df61c5d568a service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:54.227713 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2833/5654] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:11:54.642202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c191c-dd36-46ff-a412-dc95d6fbfb90 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:54.647959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e17c191c-dd36-46ff-a412-dc95d6fbfb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:54.678048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c191c-dd36-46ff-a412-dc95d6fbfb90 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:54.683642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:54.685657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:54.686955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:54.687401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] PATH_INFO: `/v3/projects/f9083719cf3f4408970021efc83e3d54` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:54.690240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9083719cf3f4408970021efc83e3d54)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:54.697872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:54.757459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:54.764170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9083719cf3f4408970021efc83e3d54', 'request_id': 'req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:54.776040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9083719cf3f4408970021efc83e3d54', 'request_id': 'req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:54.793323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9083719cf3f4408970021efc83e3d54', 'request_id': 'req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:54.807164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] The token cache is being invalidate because project f9083719cf3f4408970021efc83e3d54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:11:54.807610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:54.808289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f36936d3-d4f0-4e86-8e98-1e6241de4d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:11:54.810536 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2822/5655] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:11:54 2022] DELETE /identity/v3/projects/f9083719cf3f4408970021efc83e3d54 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:54.815250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17bb0-f1ca-46f3-9279-666c52600735 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:54.815773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17bb0-f1ca-46f3-9279-666c52600735 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:54.815773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17bb0-f1ca-46f3-9279-666c52600735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:54.847216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-edd17bb0-f1ca-46f3-9279-666c52600735 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:54.848892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-edd17bb0-f1ca-46f3-9279-666c52600735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:54.931908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:54.945479 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2834/5656] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:54.953934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f37e2-9198-40c6-8cdd-d39f34c85f6d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:54.954275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f37e2-9198-40c6-8cdd-d39f34c85f6d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:54.954629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f37e2-9198-40c6-8cdd-d39f34c85f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:54.980609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ad1f37e2-9198-40c6-8cdd-d39f34c85f6d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:54.981892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ad1f37e2-9198-40c6-8cdd-d39f34c85f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:54.997054 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2823/5657] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:54.999084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd17bb0-f1ca-46f3-9279-666c52600735 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:55.016666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd17bb0-f1ca-46f3-9279-666c52600735 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:55.026086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4ff2e-e911-4364-a352-a5470d3ba67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:55.026365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ff2e-e911-4364-a352-a5470d3ba67c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:55.026841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ff2e-e911-4364-a352-a5470d3ba67c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:55.027180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ff2e-e911-4364-a352-a5470d3ba67c admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:55.028433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4ff2e-e911-4364-a352-a5470d3ba67c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:11:55.039202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4ff2e-e911-4364-a352-a5470d3ba67c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:55.041172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4ff2e-e911-4364-a352-a5470d3ba67c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:55.065359 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2835/5658] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:11:55.069498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f37e2-9198-40c6-8cdd-d39f34c85f6d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:55.081500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f37e2-9198-40c6-8cdd-d39f34c85f6d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:55.089194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748565-cc27-419a-9207-0deaf9b7013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:55.090063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748565-cc27-419a-9207-0deaf9b7013a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:55.090917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748565-cc27-419a-9207-0deaf9b7013a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:55.091384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748565-cc27-419a-9207-0deaf9b7013a admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:55.092245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6748565-cc27-419a-9207-0deaf9b7013a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:55.094398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6748565-cc27-419a-9207-0deaf9b7013a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:55.144966 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2824/5659] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:11:55 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:11:55.149497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4ff2e-e911-4364-a352-a5470d3ba67c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:55.166432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4ff2e-e911-4364-a352-a5470d3ba67c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:55.266458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d030448-2c9d-40cd-a21e-2efdc8bfd54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:55.267159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d030448-2c9d-40cd-a21e-2efdc8bfd54d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:55.267504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d030448-2c9d-40cd-a21e-2efdc8bfd54d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:55.267836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d030448-2c9d-40cd-a21e-2efdc8bfd54d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:55.268616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d030448-2c9d-40cd-a21e-2efdc8bfd54d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:55.271293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d030448-2c9d-40cd-a21e-2efdc8bfd54d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:55.308004 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2836/5660] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:11:55 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:11:55.312655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748565-cc27-419a-9207-0deaf9b7013a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:55.346167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748565-cc27-419a-9207-0deaf9b7013a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:55.364684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ffb09-6e2b-4919-9640-412c83eaceef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:55.366237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ffb09-6e2b-4919-9640-412c83eaceef admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:55.368976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ffb09-6e2b-4919-9640-412c83eaceef admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:55.369456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ffb09-6e2b-4919-9640-412c83eaceef admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:55.370273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ffb09-6e2b-4919-9640-412c83eaceef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:55.374855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ffb09-6e2b-4919-9640-412c83eaceef admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:55.412053 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2825/5661] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:55.416094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d030448-2c9d-40cd-a21e-2efdc8bfd54d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:55.450137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d030448-2c9d-40cd-a21e-2efdc8bfd54d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:55.464578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdeeee3-16ea-46c6-b20d-54a36b34cf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:55.466369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeeee3-16ea-46c6-b20d-54a36b34cf98 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:55.468074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeeee3-16ea-46c6-b20d-54a36b34cf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:55.468818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeeee3-16ea-46c6-b20d-54a36b34cf98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:55.471213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdeeee3-16ea-46c6-b20d-54a36b34cf98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:55.480260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdeeee3-16ea-46c6-b20d-54a36b34cf98 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:55.522602 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2837/5662] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:11:55.529370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ffb09-6e2b-4919-9640-412c83eaceef admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:55.548023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ffb09-6e2b-4919-9640-412c83eaceef admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:55.556040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab5e78-8f33-4659-aaec-063aa542448c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:55.557277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5e78-8f33-4659-aaec-063aa542448c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:55.557797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5e78-8f33-4659-aaec-063aa542448c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:55.558260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5e78-8f33-4659-aaec-063aa542448c admin admin] PATH_INFO: `/v3/projects/d2fba9ff4830494ab0f4259143c4619a/users/fbb0ff2e9f3b4c08a1ced0e068f71f5f/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:55.607294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab5e78-8f33-4659-aaec-063aa542448c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2fba9ff4830494ab0f4259143c4619a, user_id=fbb0ff2e9f3b4c08a1ced0e068f71f5f, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:55.610941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab5e78-8f33-4659-aaec-063aa542448c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:55.641739 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2826/5663] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:11:55 2022] PUT /identity/v3/projects/d2fba9ff4830494ab0f4259143c4619a/users/fbb0ff2e9f3b4c08a1ced0e068f71f5f/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:11:55.645953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931c151-c90c-46e3-8f6d-4bc8770c9064 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:55.645953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931c151-c90c-46e3-8f6d-4bc8770c9064 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:55.646256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931c151-c90c-46e3-8f6d-4bc8770c9064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:55.685713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0931c151-c90c-46e3-8f6d-4bc8770c9064 None None] MFA Rules not processed for user `fbb0ff2e9f3b4c08a1ced0e068f71f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:11:55.687035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0931c151-c90c-46e3-8f6d-4bc8770c9064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:55.789192 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2838/5664] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:11:55.830484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab5e78-8f33-4659-aaec-063aa542448c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:11:55.833497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8bab5e78-8f33-4659-aaec-063aa542448c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:11:55.963223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab5e78-8f33-4659-aaec-063aa542448c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:11:55.976442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99bb17-82d4-457d-9dba-caf8af8552e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:11:55.978636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99bb17-82d4-457d-9dba-caf8af8552e1 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:11:55.980892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99bb17-82d4-457d-9dba-caf8af8552e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:11:55.981437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99bb17-82d4-457d-9dba-caf8af8552e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:11:55.995277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99bb17-82d4-457d-9dba-caf8af8552e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:11:56.013623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99bb17-82d4-457d-9dba-caf8af8552e1 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:11:56.056225 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2827/5665] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:11:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:02.332384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931c151-c90c-46e3-8f6d-4bc8770c9064 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:02.333881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0931c151-c90c-46e3-8f6d-4bc8770c9064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:02.424240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:02.444642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931c151-c90c-46e3-8f6d-4bc8770c9064 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:02.450822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:02.452164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:02.452675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:02.453204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] PATH_INFO: `/v3/users/34d02da171db432783f630c853e35fc1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:02.455318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=34d02da171db432783f630c853e35fc1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:02.458135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:02.540808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34d02da171db432783f630c853e35fc1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:02.557688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34d02da171db432783f630c853e35fc1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:02.579844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34d02da171db432783f630c853e35fc1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:02.594602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34d02da171db432783f630c853e35fc1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:02.595410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34d02da171db432783f630c853e35fc1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:02.608178 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2839/5666] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:12:02 2022] DELETE /identity/v3/users/34d02da171db432783f630c853e35fc1 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:02.612496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbe02f-8cd9-450d-84e3-4f7cac8c9d5d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:02.613451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbe02f-8cd9-450d-84e3-4f7cac8c9d5d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:02.613451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbe02f-8cd9-450d-84e3-4f7cac8c9d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:02.641839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-59fbe02f-8cd9-450d-84e3-4f7cac8c9d5d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:02.645676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-59fbe02f-8cd9-450d-84e3-4f7cac8c9d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:02.732005 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2828/5667] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:02.829005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:02.831142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:02.982432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f380c1-f64f-4b4b-886a-f772b9c089dc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:02.990861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa5f6f-b361-47ab-9a3b-c5ae3bfdc08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:02.991811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa5f6f-b361-47ab-9a3b-c5ae3bfdc08a service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:02.992259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa5f6f-b361-47ab-9a3b-c5ae3bfdc08a service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:02.992711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa5f6f-b361-47ab-9a3b-c5ae3bfdc08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:03.011625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa5f6f-b361-47ab-9a3b-c5ae3bfdc08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:03.013085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa5f6f-b361-47ab-9a3b-c5ae3bfdc08a service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:03.043265 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2840/5668] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:03.900999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbe02f-8cd9-450d-84e3-4f7cac8c9d5d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:03.902863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-59fbe02f-8cd9-450d-84e3-4f7cac8c9d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:03.915518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbe02f-8cd9-450d-84e3-4f7cac8c9d5d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:03.920971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:03.921927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:03.922495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:03.923099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] PATH_INFO: `/v3/projects/a89d5eb5c8d246d6a866c79c5dddbf3e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:03.924188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89d5eb5c8d246d6a866c79c5dddbf3e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:03.926790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:03.928674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:04.013753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89d5eb5c8d246d6a866c79c5dddbf3e', 'request_id': 'req-c8db77e0-78d6-4ba9-9d1f-c831de19d186'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:04.022365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89d5eb5c8d246d6a866c79c5dddbf3e', 'request_id': 'req-c8db77e0-78d6-4ba9-9d1f-c831de19d186'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:04.029819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89d5eb5c8d246d6a866c79c5dddbf3e', 'request_id': 'req-c8db77e0-78d6-4ba9-9d1f-c831de19d186'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:04.044546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] The token cache is being invalidate because project a89d5eb5c8d246d6a866c79c5dddbf3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:12:04.044717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:04.045922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8db77e0-78d6-4ba9-9d1f-c831de19d186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:04.051789 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2829/5669] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:12:03 2022] DELETE /identity/v3/projects/a89d5eb5c8d246d6a866c79c5dddbf3e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:04.054072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8677d5-508c-4235-8a21-0f623068a546 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:04.054577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8677d5-508c-4235-8a21-0f623068a546 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:04.054683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8677d5-508c-4235-8a21-0f623068a546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:04.077012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ab8677d5-508c-4235-8a21-0f623068a546 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:04.078435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ab8677d5-508c-4235-8a21-0f623068a546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:04.169061 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2841/5670] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:04.171509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89dd16-812f-4def-93ab-12f0b74b37c6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:04.172035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89dd16-812f-4def-93ab-12f0b74b37c6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:04.172534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89dd16-812f-4def-93ab-12f0b74b37c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:04.212222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0b89dd16-812f-4def-93ab-12f0b74b37c6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:04.212960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0b89dd16-812f-4def-93ab-12f0b74b37c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:04.225912 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2830/5671] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:04.227754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8677d5-508c-4235-8a21-0f623068a546 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:04.239873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8677d5-508c-4235-8a21-0f623068a546 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:04.246213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95818a74-13b8-422e-9839-48e2ecc1ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:04.246818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95818a74-13b8-422e-9839-48e2ecc1ecad admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:04.247394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95818a74-13b8-422e-9839-48e2ecc1ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:04.247925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95818a74-13b8-422e-9839-48e2ecc1ecad admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:04.249270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95818a74-13b8-422e-9839-48e2ecc1ecad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:12:04.249674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95818a74-13b8-422e-9839-48e2ecc1ecad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:04.251720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95818a74-13b8-422e-9839-48e2ecc1ecad admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:04.269317 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2842/5672] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:12:04.271290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89dd16-812f-4def-93ab-12f0b74b37c6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:04.281465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89dd16-812f-4def-93ab-12f0b74b37c6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:04.287854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ced2d-7e85-40b9-ba28-e3f09c6dba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:04.288643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ced2d-7e85-40b9-ba28-e3f09c6dba30 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:04.289008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ced2d-7e85-40b9-ba28-e3f09c6dba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:04.289435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ced2d-7e85-40b9-ba28-e3f09c6dba30 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:04.290178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ced2d-7e85-40b9-ba28-e3f09c6dba30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:04.292246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ced2d-7e85-40b9-ba28-e3f09c6dba30 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:04.329170 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2831/5673] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:12:04 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:04.332064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95818a74-13b8-422e-9839-48e2ecc1ecad admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:04.337845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:04.350606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95818a74-13b8-422e-9839-48e2ecc1ecad admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:04.442305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9e0a6-4a60-4039-bc2d-35dc37f126ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:04.443086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9e0a6-4a60-4039-bc2d-35dc37f126ba admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:04.443513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9e0a6-4a60-4039-bc2d-35dc37f126ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:04.443877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9e0a6-4a60-4039-bc2d-35dc37f126ba admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:04.444605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9e0a6-4a60-4039-bc2d-35dc37f126ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:04.446948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9e0a6-4a60-4039-bc2d-35dc37f126ba admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:04.477086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2843/5674] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:12:04 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:04.478930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ced2d-7e85-40b9-ba28-e3f09c6dba30 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:04.491053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ced2d-7e85-40b9-ba28-e3f09c6dba30 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:04.496895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe200d7-c118-4de5-a172-bd2c67f2602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:04.497492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe200d7-c118-4de5-a172-bd2c67f2602c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:04.498135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe200d7-c118-4de5-a172-bd2c67f2602c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:04.498135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe200d7-c118-4de5-a172-bd2c67f2602c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:04.498651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe200d7-c118-4de5-a172-bd2c67f2602c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:04.500482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe200d7-c118-4de5-a172-bd2c67f2602c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:04.517000 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2832/5675] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:04.520576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9e0a6-4a60-4039-bc2d-35dc37f126ba admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:04.536604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9e0a6-4a60-4039-bc2d-35dc37f126ba admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:04.546298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afafa13-2f8a-4b91-ac23-f914a76276d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:04.547211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afafa13-2f8a-4b91-ac23-f914a76276d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:04.547936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afafa13-2f8a-4b91-ac23-f914a76276d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:04.548562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afafa13-2f8a-4b91-ac23-f914a76276d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:04.549419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afafa13-2f8a-4b91-ac23-f914a76276d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:04.552501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afafa13-2f8a-4b91-ac23-f914a76276d0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:04.569262 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2844/5676] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:04.571334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe200d7-c118-4de5-a172-bd2c67f2602c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:04.591573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe200d7-c118-4de5-a172-bd2c67f2602c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:04.604092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6226d5-56e5-4a55-ad27-023f98653ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:04.604851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6226d5-56e5-4a55-ad27-023f98653ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:04.605175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6226d5-56e5-4a55-ad27-023f98653ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:04.605556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6226d5-56e5-4a55-ad27-023f98653ff4 admin admin] PATH_INFO: `/v3/projects/07e1f179834e407aa699cd61a710cf72/users/1989373ee2744d76b9ca4ad323356006/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:04.643506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6226d5-56e5-4a55-ad27-023f98653ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07e1f179834e407aa699cd61a710cf72, user_id=1989373ee2744d76b9ca4ad323356006, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:04.647756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6226d5-56e5-4a55-ad27-023f98653ff4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:04.677095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2833/5677] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:12:04 2022] PUT /identity/v3/projects/07e1f179834e407aa699cd61a710cf72/users/1989373ee2744d76b9ca4ad323356006/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:04.694110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d6503-0381-4b29-9b1b-516b41a26113 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:04.694743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d6503-0381-4b29-9b1b-516b41a26113 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:04.695090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d6503-0381-4b29-9b1b-516b41a26113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:04.735464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0c2d6503-0381-4b29-9b1b-516b41a26113 None None] MFA Rules not processed for user `1989373ee2744d76b9ca4ad323356006`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:04.737517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0c2d6503-0381-4b29-9b1b-516b41a26113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:04.826082 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2845/5678] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:04.850875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6226d5-56e5-4a55-ad27-023f98653ff4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:04.854396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e6226d5-56e5-4a55-ad27-023f98653ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:04.935234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6226d5-56e5-4a55-ad27-023f98653ff4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:04.939215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1dbf7-29e9-4c0e-9d3f-2e395571b447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:04.940191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1dbf7-29e9-4c0e-9d3f-2e395571b447 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:04.940553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1dbf7-29e9-4c0e-9d3f-2e395571b447 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:04.941448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1dbf7-29e9-4c0e-9d3f-2e395571b447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:04.951690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1dbf7-29e9-4c0e-9d3f-2e395571b447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:04.953401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1dbf7-29e9-4c0e-9d3f-2e395571b447 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:04.964449 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2834/5679] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:11.818639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d6503-0381-4b29-9b1b-516b41a26113 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:11.820792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0c2d6503-0381-4b29-9b1b-516b41a26113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:11.994337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d6503-0381-4b29-9b1b-516b41a26113 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:12.013101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:12.013788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:12.014067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:12.014335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] PATH_INFO: `/v3/users/8aa4174b94564057a4ed7e15c3b73672` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:12.015975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aa4174b94564057a4ed7e15c3b73672)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:12.018070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:12.048468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:12.118866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aa4174b94564057a4ed7e15c3b73672'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:12.135739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aa4174b94564057a4ed7e15c3b73672'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:12.150655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aa4174b94564057a4ed7e15c3b73672'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:12.166273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aa4174b94564057a4ed7e15c3b73672'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:12.167265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aa4174b94564057a4ed7e15c3b73672'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:12.180086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2846/5680] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:12:11 2022] DELETE /identity/v3/users/8aa4174b94564057a4ed7e15c3b73672 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:12.181738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7677-e7a9-494f-ba2f-1e5d9c93cfbe None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:12.182390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7677-e7a9-494f-ba2f-1e5d9c93cfbe None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:12.182390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7677-e7a9-494f-ba2f-1e5d9c93cfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:12.216032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f7df7677-e7a9-494f-ba2f-1e5d9c93cfbe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:12.217463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f7df7677-e7a9-494f-ba2f-1e5d9c93cfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:12.287890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:12.351758 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2835/5681] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:12:12.407565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:12.410504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:12.521191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dfb73-7276-4938-9bea-eaccf15ca535 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:12.527251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cc3d1-caf9-4fc2-9471-0349feac14a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:12.528063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cc3d1-caf9-4fc2-9471-0349feac14a1 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:12.528442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cc3d1-caf9-4fc2-9471-0349feac14a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:12.528818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cc3d1-caf9-4fc2-9471-0349feac14a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:12.541588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251cc3d1-caf9-4fc2-9471-0349feac14a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:12.543390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251cc3d1-caf9-4fc2-9471-0349feac14a1 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:12.556859 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2847/5682] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:14.108841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df7677-e7a9-494f-ba2f-1e5d9c93cfbe None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:14.110307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f7df7677-e7a9-494f-ba2f-1e5d9c93cfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:14.123368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df7677-e7a9-494f-ba2f-1e5d9c93cfbe None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:14.128495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:14.129154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:14.129434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:14.129684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] PATH_INFO: `/v3/projects/e7b6ea65ad2e405a94727beb4bbeb68d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:14.131232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b6ea65ad2e405a94727beb4bbeb68d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:14.133243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:14.211059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b6ea65ad2e405a94727beb4bbeb68d', 'request_id': 'req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:14.220622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b6ea65ad2e405a94727beb4bbeb68d', 'request_id': 'req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:14.229733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b6ea65ad2e405a94727beb4bbeb68d', 'request_id': 'req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:14.244567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] The token cache is being invalidate because project e7b6ea65ad2e405a94727beb4bbeb68d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:12:14.245665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:14.246660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:14.249104 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2836/5683] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:12:14 2022] DELETE /identity/v3/projects/e7b6ea65ad2e405a94727beb4bbeb68d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:14.253759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cc3d1-caf9-4fc2-9471-0349feac14a1 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:14.268123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cc3d1-caf9-4fc2-9471-0349feac14a1 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:14.345030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:14.346006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:14.346422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:14.346883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] PATH_INFO: `/v3/users/db0b24a87eac48beb5293d777077e64f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:14.348238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=db0b24a87eac48beb5293d777077e64f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:14.352699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:14.428079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:14.466108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db0b24a87eac48beb5293d777077e64f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:14.493992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db0b24a87eac48beb5293d777077e64f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:14.507994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db0b24a87eac48beb5293d777077e64f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:14.520425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db0b24a87eac48beb5293d777077e64f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:14.522068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b3333254-d181-43da-a7c4-3cb8b2572a8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db0b24a87eac48beb5293d777077e64f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:14.538401 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2848/5684] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:12:14 2022] DELETE /identity/v3/users/db0b24a87eac48beb5293d777077e64f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:15.193672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:15.197507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:15.232990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f7fd6-a6c6-4c40-9464-e4d35fcbab7a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:15.382240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:15.383970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:15.384355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:15.384733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] PATH_INFO: `/v3/projects/5c60521f47ad4003b2c6507621c0d448` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:15.386114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c60521f47ad4003b2c6507621c0d448)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:15.388282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:15.467457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c60521f47ad4003b2c6507621c0d448', 'request_id': 'req-231d517a-cd0d-4810-b882-f7b3fd63c046'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:15.472290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:15.479932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c60521f47ad4003b2c6507621c0d448', 'request_id': 'req-231d517a-cd0d-4810-b882-f7b3fd63c046'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:15.490329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c60521f47ad4003b2c6507621c0d448', 'request_id': 'req-231d517a-cd0d-4810-b882-f7b3fd63c046'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:15.507587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] The token cache is being invalidate because project 5c60521f47ad4003b2c6507621c0d448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:12:15.508273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:15.509757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-231d517a-cd0d-4810-b882-f7b3fd63c046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:15.515283 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2837/5685] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:12:15 2022] DELETE /identity/v3/projects/5c60521f47ad4003b2c6507621c0d448 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:15.516713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25547a3-4279-4b61-bdc4-956e65e910e7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:15.517228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25547a3-4279-4b61-bdc4-956e65e910e7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:15.517336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25547a3-4279-4b61-bdc4-956e65e910e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:15.547354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c25547a3-4279-4b61-bdc4-956e65e910e7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:15.548879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c25547a3-4279-4b61-bdc4-956e65e910e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:15.637732 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2849/5686] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:15.645602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f4db2-0a96-4e29-bc15-280104e67074 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:15.646300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f4db2-0a96-4e29-bc15-280104e67074 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:15.646644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f4db2-0a96-4e29-bc15-280104e67074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:15.668296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-701f4db2-0a96-4e29-bc15-280104e67074 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:15.669662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-701f4db2-0a96-4e29-bc15-280104e67074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:15.679384 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2838/5687] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:15.683678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25547a3-4279-4b61-bdc4-956e65e910e7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:15.696622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25547a3-4279-4b61-bdc4-956e65e910e7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:15.701982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda10bdf-3aa3-4ea9-bd66-9b2e3f0751de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:15.702524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda10bdf-3aa3-4ea9-bd66-9b2e3f0751de admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:15.702825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda10bdf-3aa3-4ea9-bd66-9b2e3f0751de admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:15.703092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda10bdf-3aa3-4ea9-bd66-9b2e3f0751de admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:15.703747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda10bdf-3aa3-4ea9-bd66-9b2e3f0751de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:12:15.704044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda10bdf-3aa3-4ea9-bd66-9b2e3f0751de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:15.705988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda10bdf-3aa3-4ea9-bd66-9b2e3f0751de admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:15.719238 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2850/5688] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:12:15.724044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f4db2-0a96-4e29-bc15-280104e67074 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:15.734318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f4db2-0a96-4e29-bc15-280104e67074 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:15.740243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca0e18-f42e-4a9d-93bc-b0e36dd44884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:15.740857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca0e18-f42e-4a9d-93bc-b0e36dd44884 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:15.741177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca0e18-f42e-4a9d-93bc-b0e36dd44884 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:15.741442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca0e18-f42e-4a9d-93bc-b0e36dd44884 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:15.742081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca0e18-f42e-4a9d-93bc-b0e36dd44884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:15.744004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca0e18-f42e-4a9d-93bc-b0e36dd44884 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:15.779026 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2839/5689] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:12:15 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:12:15.780245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda10bdf-3aa3-4ea9-bd66-9b2e3f0751de admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:15.791217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda10bdf-3aa3-4ea9-bd66-9b2e3f0751de admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:15.860933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f114f-ac0f-43d1-abb9-2d4646a9eb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:15.862000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f114f-ac0f-43d1-abb9-2d4646a9eb9c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:15.863585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f114f-ac0f-43d1-abb9-2d4646a9eb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:15.864059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f114f-ac0f-43d1-abb9-2d4646a9eb9c admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:15.864696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f114f-ac0f-43d1-abb9-2d4646a9eb9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:15.866673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f114f-ac0f-43d1-abb9-2d4646a9eb9c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:15.904721 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2851/5690] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:12:15 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:15.910005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca0e18-f42e-4a9d-93bc-b0e36dd44884 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:15.936789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca0e18-f42e-4a9d-93bc-b0e36dd44884 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:15.942746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc40aa9-7a61-4504-869f-8c18f8d559e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:15.943676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc40aa9-7a61-4504-869f-8c18f8d559e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:15.944030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc40aa9-7a61-4504-869f-8c18f8d559e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:15.944370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc40aa9-7a61-4504-869f-8c18f8d559e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:15.945161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc40aa9-7a61-4504-869f-8c18f8d559e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:15.946830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc40aa9-7a61-4504-869f-8c18f8d559e7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:15.960732 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2840/5691] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:12:15.964735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f114f-ac0f-43d1-abb9-2d4646a9eb9c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:15.975860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f114f-ac0f-43d1-abb9-2d4646a9eb9c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:15.982313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694d1c9-4c7f-4c05-a12b-4a347d75e841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:15.982918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d1c9-4c7f-4c05-a12b-4a347d75e841 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:15.983191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d1c9-4c7f-4c05-a12b-4a347d75e841 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:15.983476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d1c9-4c7f-4c05-a12b-4a347d75e841 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:15.984246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694d1c9-4c7f-4c05-a12b-4a347d75e841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:15.986193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694d1c9-4c7f-4c05-a12b-4a347d75e841 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:15.998458 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2852/5692] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:16.002879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc40aa9-7a61-4504-869f-8c18f8d559e7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:16.014581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc40aa9-7a61-4504-869f-8c18f8d559e7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:16.019961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44535a33-2c37-4e54-9f2d-385c497f3824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:16.020938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44535a33-2c37-4e54-9f2d-385c497f3824 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:16.021498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44535a33-2c37-4e54-9f2d-385c497f3824 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:16.021897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44535a33-2c37-4e54-9f2d-385c497f3824 admin admin] PATH_INFO: `/v3/projects/0205358c6de24847bff60df4cc5a57be/users/2ce795062cda4070a6d774298799b99e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:16.041525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44535a33-2c37-4e54-9f2d-385c497f3824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0205358c6de24847bff60df4cc5a57be, user_id=2ce795062cda4070a6d774298799b99e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:16.044472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44535a33-2c37-4e54-9f2d-385c497f3824 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:16.075243 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2841/5693] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:12:16 2022] PUT /identity/v3/projects/0205358c6de24847bff60df4cc5a57be/users/2ce795062cda4070a6d774298799b99e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:16.076302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bfd70-7982-4055-b451-61b0b19e9136 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:16.077481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bfd70-7982-4055-b451-61b0b19e9136 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:16.077481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bfd70-7982-4055-b451-61b0b19e9136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:16.106878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-592bfd70-7982-4055-b451-61b0b19e9136 None None] MFA Rules not processed for user `2ce795062cda4070a6d774298799b99e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:16.108257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-592bfd70-7982-4055-b451-61b0b19e9136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:16.116350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:16.206525 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2853/5694] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:16.210836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44535a33-2c37-4e54-9f2d-385c497f3824 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:16.293565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44535a33-2c37-4e54-9f2d-385c497f3824 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:16.298789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e8c0f-a50d-455b-9e74-c5290ecb1699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:16.299469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e8c0f-a50d-455b-9e74-c5290ecb1699 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:16.299798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e8c0f-a50d-455b-9e74-c5290ecb1699 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:16.300130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e8c0f-a50d-455b-9e74-c5290ecb1699 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:16.300823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e8c0f-a50d-455b-9e74-c5290ecb1699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:16.306227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e8c0f-a50d-455b-9e74-c5290ecb1699 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:16.343622 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2842/5695] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:12:16 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:16.344875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bfd70-7982-4055-b451-61b0b19e9136 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:16.369646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bfd70-7982-4055-b451-61b0b19e9136 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:16.465014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452c09-7510-4372-895b-e934ee993d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:16.465668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452c09-7510-4372-895b-e934ee993d49 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:16.466021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452c09-7510-4372-895b-e934ee993d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:16.466207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452c09-7510-4372-895b-e934ee993d49 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:16.466846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae452c09-7510-4372-895b-e934ee993d49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:16.469608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae452c09-7510-4372-895b-e934ee993d49 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:16.510347 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2854/5696] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:12:16 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:16.512515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e8c0f-a50d-455b-9e74-c5290ecb1699 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:16.529627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e8c0f-a50d-455b-9e74-c5290ecb1699 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:16.547387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cdfdbc-de4a-4d1d-b6b1-d1b50a98f3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:16.548399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdfdbc-de4a-4d1d-b6b1-d1b50a98f3cd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:16.549524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdfdbc-de4a-4d1d-b6b1-d1b50a98f3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:16.550610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdfdbc-de4a-4d1d-b6b1-d1b50a98f3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:16.553461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cdfdbc-de4a-4d1d-b6b1-d1b50a98f3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:16.557534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cdfdbc-de4a-4d1d-b6b1-d1b50a98f3cd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:16.573641 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2843/5697] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:16.574854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452c09-7510-4372-895b-e934ee993d49 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:16.587665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452c09-7510-4372-895b-e934ee993d49 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:16.605634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f2042-d9d8-4eb8-89aa-2e01bc0f1f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:16.606036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f2042-d9d8-4eb8-89aa-2e01bc0f1f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:16.606311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f2042-d9d8-4eb8-89aa-2e01bc0f1f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:16.606594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f2042-d9d8-4eb8-89aa-2e01bc0f1f27 admin admin] PATH_INFO: `/v3/projects/f17710ae718b477ea1d24487561ebfc8/users/038805bf16844e2da7cb92c9eedd4d1c/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:16.632960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f2042-d9d8-4eb8-89aa-2e01bc0f1f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f17710ae718b477ea1d24487561ebfc8, user_id=038805bf16844e2da7cb92c9eedd4d1c, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:16.635819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f2042-d9d8-4eb8-89aa-2e01bc0f1f27 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:16.662598 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2855/5698] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:12:16 2022] PUT /identity/v3/projects/f17710ae718b477ea1d24487561ebfc8/users/038805bf16844e2da7cb92c9eedd4d1c/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:16.666394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cdfdbc-de4a-4d1d-b6b1-d1b50a98f3cd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:16.682862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cdfdbc-de4a-4d1d-b6b1-d1b50a98f3cd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:16.752579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:16.784802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf808084-b948-4b80-b8f9-880f56da7cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:16.786054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808084-b948-4b80-b8f9-880f56da7cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:16.786491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808084-b948-4b80-b8f9-880f56da7cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:16.786939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808084-b948-4b80-b8f9-880f56da7cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:16.787739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf808084-b948-4b80-b8f9-880f56da7cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:16.790200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf808084-b948-4b80-b8f9-880f56da7cf6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:16.808829 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2844/5699] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:12:16.814079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f2042-d9d8-4eb8-89aa-2e01bc0f1f27 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:16.832043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f2042-d9d8-4eb8-89aa-2e01bc0f1f27 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:16.843721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b586e-18eb-43f2-b84d-eb9619eb6182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:16.844914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b586e-18eb-43f2-b84d-eb9619eb6182 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:16.845442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b586e-18eb-43f2-b84d-eb9619eb6182 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:16.850022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b586e-18eb-43f2-b84d-eb9619eb6182 admin admin] PATH_INFO: `/v3/projects/f17710ae718b477ea1d24487561ebfc8/users/038805bf16844e2da7cb92c9eedd4d1c/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:16.856035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b586e-18eb-43f2-b84d-eb9619eb6182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f17710ae718b477ea1d24487561ebfc8, user_id=038805bf16844e2da7cb92c9eedd4d1c, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:16.861109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b586e-18eb-43f2-b84d-eb9619eb6182 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:16.888075 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2856/5700] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:12:16 2022] PUT /identity/v3/projects/f17710ae718b477ea1d24487561ebfc8/users/038805bf16844e2da7cb92c9eedd4d1c/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:16.898995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fb433-68ce-4f0c-88dc-6f2ee61ed40b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:16.899485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fb433-68ce-4f0c-88dc-6f2ee61ed40b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:16.899771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fb433-68ce-4f0c-88dc-6f2ee61ed40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:16.943351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-378fb433-68ce-4f0c-88dc-6f2ee61ed40b None None] MFA Rules not processed for user `038805bf16844e2da7cb92c9eedd4d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:16.944848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-378fb433-68ce-4f0c-88dc-6f2ee61ed40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:17.089168 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2845/5701] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:17.120438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b586e-18eb-43f2-b84d-eb9619eb6182 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:17.122553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2f6b586e-18eb-43f2-b84d-eb9619eb6182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:17.222007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b586e-18eb-43f2-b84d-eb9619eb6182 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:17.228414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fad23f-6845-4567-a8b5-db13e76b69b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:17.229725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad23f-6845-4567-a8b5-db13e76b69b8 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:17.230131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad23f-6845-4567-a8b5-db13e76b69b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:17.231012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad23f-6845-4567-a8b5-db13e76b69b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:17.248798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fad23f-6845-4567-a8b5-db13e76b69b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:17.251585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fad23f-6845-4567-a8b5-db13e76b69b8 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:17.271596 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2857/5702] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:20.065423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fb433-68ce-4f0c-88dc-6f2ee61ed40b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:20.067015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-378fb433-68ce-4f0c-88dc-6f2ee61ed40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:20.225909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fb433-68ce-4f0c-88dc-6f2ee61ed40b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:20.238886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc0059-da1c-4bd7-92c5-4dfe74ae9da6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:20.239517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc0059-da1c-4bd7-92c5-4dfe74ae9da6 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:20.239785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc0059-da1c-4bd7-92c5-4dfe74ae9da6 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:20.240059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc0059-da1c-4bd7-92c5-4dfe74ae9da6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:20.244485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e5bc0059-da1c-4bd7-92c5-4dfe74ae9da6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:20.276424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc0059-da1c-4bd7-92c5-4dfe74ae9da6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:20.278086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc0059-da1c-4bd7-92c5-4dfe74ae9da6 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:20.314003 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2846/5703] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:20.421168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fad23f-6845-4567-a8b5-db13e76b69b8 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:20.423142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f9fad23f-6845-4567-a8b5-db13e76b69b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:20.430163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:20.573292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fad23f-6845-4567-a8b5-db13e76b69b8 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:20.582881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:20.583738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:20.584192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:20.584615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] PATH_INFO: `/v3/users/fca16fd2d377433c98ed40a24395111a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:20.587287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca16fd2d377433c98ed40a24395111a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:20.589447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:20.696551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca16fd2d377433c98ed40a24395111a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:20.715441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca16fd2d377433c98ed40a24395111a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:20.734949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca16fd2d377433c98ed40a24395111a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:20.751248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca16fd2d377433c98ed40a24395111a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:20.752265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca16fd2d377433c98ed40a24395111a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:20.762829 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2858/5704] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:12:20 2022] DELETE /identity/v3/users/fca16fd2d377433c98ed40a24395111a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:20.764931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580f435-d051-4b0f-8510-a87faa15ce46 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:20.765532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580f435-d051-4b0f-8510-a87faa15ce46 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:20.765968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580f435-d051-4b0f-8510-a87faa15ce46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:20.836562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d580f435-d051-4b0f-8510-a87faa15ce46 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:20.841994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d580f435-d051-4b0f-8510-a87faa15ce46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:20.974979 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2847/5705] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:20.984716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:20.990655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:21.107409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7782e89-d8b7-4a23-b434-9665fd912261 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:21.119022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944ed40-65ea-49cf-b834-13eca339fc42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:21.119852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944ed40-65ea-49cf-b834-13eca339fc42 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:21.120142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944ed40-65ea-49cf-b834-13eca339fc42 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:21.120583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944ed40-65ea-49cf-b834-13eca339fc42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:21.137994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944ed40-65ea-49cf-b834-13eca339fc42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:21.141990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944ed40-65ea-49cf-b834-13eca339fc42 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:21.167308 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2859/5706] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:21.794992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580f435-d051-4b0f-8510-a87faa15ce46 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:21.798074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d580f435-d051-4b0f-8510-a87faa15ce46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:21.814780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:21.832356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580f435-d051-4b0f-8510-a87faa15ce46 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:21.847325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:21.848479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:21.849299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:21.851289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] PATH_INFO: `/v3/projects/9a1c5f0c627a41b4a60f7d20ca9ff753` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:21.860916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1c5f0c627a41b4a60f7d20ca9ff753)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:21.867064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:22.025188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1c5f0c627a41b4a60f7d20ca9ff753', 'request_id': 'req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:22.037879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1c5f0c627a41b4a60f7d20ca9ff753', 'request_id': 'req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:22.063518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1c5f0c627a41b4a60f7d20ca9ff753', 'request_id': 'req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:22.091001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] The token cache is being invalidate because project 9a1c5f0c627a41b4a60f7d20ca9ff753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:12:22.091737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:22.092707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d303f8c7-b7b7-4e53-bced-ae1b8e83741a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:22.095198 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2848/5707] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:12:21 2022] DELETE /identity/v3/projects/9a1c5f0c627a41b4a60f7d20ca9ff753 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:12:22.101429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13dff3-85a1-4957-822a-229243670380 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:22.102486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13dff3-85a1-4957-822a-229243670380 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:22.104234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13dff3-85a1-4957-822a-229243670380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:22.167597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2c13dff3-85a1-4957-822a-229243670380 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:22.169628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2c13dff3-85a1-4957-822a-229243670380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:22.242075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:22.370613 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2860/5708] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:22.371295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bd5f4-128a-4ec3-adcb-fcaf1899138f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:22.371639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bd5f4-128a-4ec3-adcb-fcaf1899138f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:22.372004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bd5f4-128a-4ec3-adcb-fcaf1899138f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:22.449567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-801bd5f4-128a-4ec3-adcb-fcaf1899138f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:22.453158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-801bd5f4-128a-4ec3-adcb-fcaf1899138f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:22.536792 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2849/5709] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:12:22.538665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13dff3-85a1-4957-822a-229243670380 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:22.565912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13dff3-85a1-4957-822a-229243670380 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:22.586297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1dbcb-4ea9-4513-babf-51ab3743edb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:22.587222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1dbcb-4ea9-4513-babf-51ab3743edb8 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:22.588663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1dbcb-4ea9-4513-babf-51ab3743edb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:22.589428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1dbcb-4ea9-4513-babf-51ab3743edb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:22.590513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1dbcb-4ea9-4513-babf-51ab3743edb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:12:22.590918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1dbcb-4ea9-4513-babf-51ab3743edb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:22.595406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1dbcb-4ea9-4513-babf-51ab3743edb8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:22.632885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bd5f4-128a-4ec3-adcb-fcaf1899138f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:22.645286 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2861/5710] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:12:22.646595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bd5f4-128a-4ec3-adcb-fcaf1899138f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:22.660325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee97b1-6b98-4e73-bd09-4f27bc84c7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:22.664066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee97b1-6b98-4e73-bd09-4f27bc84c7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:22.664470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee97b1-6b98-4e73-bd09-4f27bc84c7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:22.665320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee97b1-6b98-4e73-bd09-4f27bc84c7e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:22.665737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee97b1-6b98-4e73-bd09-4f27bc84c7e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:22.671899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee97b1-6b98-4e73-bd09-4f27bc84c7e8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:22.774947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1dbcb-4ea9-4513-babf-51ab3743edb8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:22.794441 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2850/5711] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:12:22 2022] POST /identity/v3/projects => generated 359 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:22.807553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1dbcb-4ea9-4513-babf-51ab3743edb8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:22.946654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb60563-4ac4-46f4-9c57-344079053397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:22.947192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb60563-4ac4-46f4-9c57-344079053397 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:22.947463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb60563-4ac4-46f4-9c57-344079053397 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:22.947787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb60563-4ac4-46f4-9c57-344079053397 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:22.948417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb60563-4ac4-46f4-9c57-344079053397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:22.950588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb60563-4ac4-46f4-9c57-344079053397 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:22.991262 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2862/5712] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:12:22 2022] POST /identity/v3/users => generated 280 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:12:22.993020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee97b1-6b98-4e73-bd09-4f27bc84c7e8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:23.010157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee97b1-6b98-4e73-bd09-4f27bc84c7e8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:23.017150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d73fc-f69a-4987-945e-0de761dd96d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:23.018054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d73fc-f69a-4987-945e-0de761dd96d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:23.019003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d73fc-f69a-4987-945e-0de761dd96d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:23.019503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d73fc-f69a-4987-945e-0de761dd96d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:23.020411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d73fc-f69a-4987-945e-0de761dd96d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:23.023142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d73fc-f69a-4987-945e-0de761dd96d8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:23.056929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb60563-4ac4-46f4-9c57-344079053397 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:23.057429 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2851/5713] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:23.108759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb60563-4ac4-46f4-9c57-344079053397 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:23.164776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fba7be-10db-432b-9701-686423379dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:23.165421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba7be-10db-432b-9701-686423379dbe admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:23.165695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba7be-10db-432b-9701-686423379dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:23.165963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba7be-10db-432b-9701-686423379dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:23.166611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fba7be-10db-432b-9701-686423379dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:23.168227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fba7be-10db-432b-9701-686423379dbe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:23.224232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d73fc-f69a-4987-945e-0de761dd96d8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:23.225680 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2863/5714] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:23.249135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d73fc-f69a-4987-945e-0de761dd96d8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:23.260687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa12b0-7b39-4f68-95bc-fb1b4a1144c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:23.261739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa12b0-7b39-4f68-95bc-fb1b4a1144c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:23.262500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa12b0-7b39-4f68-95bc-fb1b4a1144c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:23.262947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa12b0-7b39-4f68-95bc-fb1b4a1144c6 admin admin] PATH_INFO: `/v3/projects/0133e7f74bc9474b8c47920563402cf1/users/de1f5c0c96d848e195426008c6840936/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:23.289458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa12b0-7b39-4f68-95bc-fb1b4a1144c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0133e7f74bc9474b8c47920563402cf1, user_id=de1f5c0c96d848e195426008c6840936, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:23.292744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa12b0-7b39-4f68-95bc-fb1b4a1144c6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:23.340905 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2852/5715] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:12:23 2022] PUT /identity/v3/projects/0133e7f74bc9474b8c47920563402cf1/users/de1f5c0c96d848e195426008c6840936/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:23.351673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50941a11-9136-44ae-a8fe-43466903865e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:23.352016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50941a11-9136-44ae-a8fe-43466903865e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:23.352230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50941a11-9136-44ae-a8fe-43466903865e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:23.400268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-50941a11-9136-44ae-a8fe-43466903865e None None] MFA Rules not processed for user `de1f5c0c96d848e195426008c6840936`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:23.401624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-50941a11-9136-44ae-a8fe-43466903865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:23.491687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:23.531297 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2864/5716] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:23.571423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa12b0-7b39-4f68-95bc-fb1b4a1144c6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:23.573347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-71fa12b0-7b39-4f68-95bc-fb1b4a1144c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:23.657063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa12b0-7b39-4f68-95bc-fb1b4a1144c6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:23.660883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a370f6-3702-4fc5-8d7f-deae466122c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:23.661516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a370f6-3702-4fc5-8d7f-deae466122c0 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:23.661788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a370f6-3702-4fc5-8d7f-deae466122c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:23.662076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a370f6-3702-4fc5-8d7f-deae466122c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:23.671605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a370f6-3702-4fc5-8d7f-deae466122c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:23.673465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a370f6-3702-4fc5-8d7f-deae466122c0 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:23.685360 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2853/5717] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:31.024748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4e9a5-9c96-47b9-be94-e2b63c5f379a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:31.025382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4e9a5-9c96-47b9-be94-e2b63c5f379a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:31.025382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4e9a5-9c96-47b9-be94-e2b63c5f379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:31.060516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-19b4e9a5-9c96-47b9-be94-e2b63c5f379a None None] MFA Rules not processed for user `c48346e56c47481cab7f248b67369a27`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:31.063188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-19b4e9a5-9c96-47b9-be94-e2b63c5f379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:31.188702 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2865/5718] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:33.681070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:35.882229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a370f6-3702-4fc5-8d7f-deae466122c0 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:35.883707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-85a370f6-3702-4fc5-8d7f-deae466122c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:36.001714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a370f6-3702-4fc5-8d7f-deae466122c0 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:36.007585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:36.009415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:36.009973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:36.010464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] PATH_INFO: `/v3/users/1989373ee2744d76b9ca4ad323356006` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:36.013013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1989373ee2744d76b9ca4ad323356006)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:36.016844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:36.101760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1989373ee2744d76b9ca4ad323356006'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:36.111410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1989373ee2744d76b9ca4ad323356006'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:36.149204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1989373ee2744d76b9ca4ad323356006'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:36.169733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1989373ee2744d76b9ca4ad323356006'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:36.170899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1989373ee2744d76b9ca4ad323356006'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:36.187937 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2854/5719] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:12:35 2022] DELETE /identity/v3/users/1989373ee2744d76b9ca4ad323356006 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:12:36.193078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ffca-2003-49da-a1f9-d640331211f1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:36.193939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ffca-2003-49da-a1f9-d640331211f1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:36.196093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ffca-2003-49da-a1f9-d640331211f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:36.263556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6559ffca-2003-49da-a1f9-d640331211f1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:36.264838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6559ffca-2003-49da-a1f9-d640331211f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:36.373716 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2866/5720] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:36.407035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:36.408520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:36.443413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ffca-2003-49da-a1f9-d640331211f1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:36.445313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6559ffca-2003-49da-a1f9-d640331211f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:36.477593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ffca-2003-49da-a1f9-d640331211f1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:36.500503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:36.503107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:36.503617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:36.504760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] PATH_INFO: `/v3/users/a7eae9dc0a1a47d9910f49bb33316d91` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:36.511507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7eae9dc0a1a47d9910f49bb33316d91)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:36.516153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:36.617630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fa794-c317-4f37-aeb2-82990792f2e4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:36.622904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7eae9dc0a1a47d9910f49bb33316d91'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:36.625213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937441a-b84e-4918-93e1-b8b5878c1f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:36.625952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937441a-b84e-4918-93e1-b8b5878c1f87 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:36.626243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937441a-b84e-4918-93e1-b8b5878c1f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:36.626645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937441a-b84e-4918-93e1-b8b5878c1f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:36.646432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7eae9dc0a1a47d9910f49bb33316d91'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:36.649204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937441a-b84e-4918-93e1-b8b5878c1f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:36.652377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937441a-b84e-4918-93e1-b8b5878c1f87 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:36.661457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7eae9dc0a1a47d9910f49bb33316d91'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:36.693701 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2855/5721] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:36.696889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7eae9dc0a1a47d9910f49bb33316d91'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:36.697820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7eae9dc0a1a47d9910f49bb33316d91'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:36.711394 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2867/5722] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:12:36 2022] DELETE /identity/v3/users/a7eae9dc0a1a47d9910f49bb33316d91 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:36.727078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937441a-b84e-4918-93e1-b8b5878c1f87 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:36.727078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2937441a-b84e-4918-93e1-b8b5878c1f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:36.740917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:36.818999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937441a-b84e-4918-93e1-b8b5878c1f87 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:36.899639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3893505-aa7f-4d21-8a79-09dc38e314c5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '038805bf16844e2da7cb92c9eedd4d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f17710ae718b477ea1d24487561ebfc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:36.901201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3893505-aa7f-4d21-8a79-09dc38e314c5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:36.901702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3893505-aa7f-4d21-8a79-09dc38e314c5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:36.902141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3893505-aa7f-4d21-8a79-09dc38e314c5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:36.903178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3893505-aa7f-4d21-8a79-09dc38e314c5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:12:36.903759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3893505-aa7f-4d21-8a79-09dc38e314c5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:36.906983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3893505-aa7f-4d21-8a79-09dc38e314c5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:36.922396 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2856/5723] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:12:36.926298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:36.927972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:36.939756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:36.944998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18533fc9-37d3-476f-bda8-f05da4c2dc48 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:36.950947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944fdbe-a982-4560-8c7b-3129ba7618d5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '038805bf16844e2da7cb92c9eedd4d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f17710ae718b477ea1d24487561ebfc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:36.952152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944fdbe-a982-4560-8c7b-3129ba7618d5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:36.952681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944fdbe-a982-4560-8c7b-3129ba7618d5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:36.953223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944fdbe-a982-4560-8c7b-3129ba7618d5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:36.954229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944fdbe-a982-4560-8c7b-3129ba7618d5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:36.957312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944fdbe-a982-4560-8c7b-3129ba7618d5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:37.001603 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2868/5724] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:12:36 2022] POST /identity/v3/projects => generated 346 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:12:37.003777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3893505-aa7f-4d21-8a79-09dc38e314c5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:37.015978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3893505-aa7f-4d21-8a79-09dc38e314c5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:37.102016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578f841-3d51-49bb-a6fb-a8916365cc9a tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '038805bf16844e2da7cb92c9eedd4d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f17710ae718b477ea1d24487561ebfc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:37.103027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578f841-3d51-49bb-a6fb-a8916365cc9a tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:37.103586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578f841-3d51-49bb-a6fb-a8916365cc9a tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:37.104021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578f841-3d51-49bb-a6fb-a8916365cc9a tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:37.105804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578f841-3d51-49bb-a6fb-a8916365cc9a tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:12:37.106761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578f841-3d51-49bb-a6fb-a8916365cc9a tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:37.108621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578f841-3d51-49bb-a6fb-a8916365cc9a tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:37.136845 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2857/5725] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:12:37.433975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944fdbe-a982-4560-8c7b-3129ba7618d5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:37.436754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d944fdbe-a982-4560-8c7b-3129ba7618d5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:37.525077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578f841-3d51-49bb-a6fb-a8916365cc9a tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:37.525830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944fdbe-a982-4560-8c7b-3129ba7618d5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:37.527452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7578f841-3d51-49bb-a6fb-a8916365cc9a tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:37.532335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:37.533426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:37.534023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:37.534518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] PATH_INFO: `/v3/projects/16eb29186a7b4964a49882162a35cf06` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:37.535986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] RBAC: Authorizing `identity:delete_project(project_id=16eb29186a7b4964a49882162a35cf06)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:37.538873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:37.553122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578f841-3d51-49bb-a6fb-a8916365cc9a tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:37.573174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:37.575489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:37.575950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:37.576462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] PATH_INFO: `/v3/projects/07e1f179834e407aa699cd61a710cf72` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:37.591926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07e1f179834e407aa699cd61a710cf72)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:37.594256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:37.622906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16eb29186a7b4964a49882162a35cf06', 'request_id': 'req-0a202939-7c1a-4d0b-8e9e-014f6b51507a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:37.641589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16eb29186a7b4964a49882162a35cf06', 'request_id': 'req-0a202939-7c1a-4d0b-8e9e-014f6b51507a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:37.653012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:37.658157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16eb29186a7b4964a49882162a35cf06', 'request_id': 'req-0a202939-7c1a-4d0b-8e9e-014f6b51507a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:37.664937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07e1f179834e407aa699cd61a710cf72', 'request_id': 'req-e98f566f-2a75-451d-a911-1cc368bf0ae4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:37.676552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07e1f179834e407aa699cd61a710cf72', 'request_id': 'req-e98f566f-2a75-451d-a911-1cc368bf0ae4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:37.681075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] The token cache is being invalidate because project 16eb29186a7b4964a49882162a35cf06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:12:37.681373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:37.682056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0a202939-7c1a-4d0b-8e9e-014f6b51507a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:37.685090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07e1f179834e407aa699cd61a710cf72', 'request_id': 'req-e98f566f-2a75-451d-a911-1cc368bf0ae4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:37.686305 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2869/5726] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:12:37 2022] DELETE /identity/v3/projects/16eb29186a7b4964a49882162a35cf06 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:37.691664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40e20-f6b9-4b17-931e-cd30403ca166 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:37.691959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40e20-f6b9-4b17-931e-cd30403ca166 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:37.692210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40e20-f6b9-4b17-931e-cd30403ca166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:37.703709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] The token cache is being invalidate because project 07e1f179834e407aa699cd61a710cf72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:12:37.704051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:37.704853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e98f566f-2a75-451d-a911-1cc368bf0ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:37.707255 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2858/5727] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:12:37 2022] DELETE /identity/v3/projects/07e1f179834e407aa699cd61a710cf72 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:12:37.713101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f53d1-1ea0-42e6-ae77-3f2b0cfe9ef5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:37.713543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f53d1-1ea0-42e6-ae77-3f2b0cfe9ef5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:37.713914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f53d1-1ea0-42e6-ae77-3f2b0cfe9ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:37.716392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-09d40e20-f6b9-4b17-931e-cd30403ca166 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:37.717670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-09d40e20-f6b9-4b17-931e-cd30403ca166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:37.747674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a52f53d1-1ea0-42e6-ae77-3f2b0cfe9ef5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:37.749159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a52f53d1-1ea0-42e6-ae77-3f2b0cfe9ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:37.826523 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2870/5728] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:37.835277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea92f4-a95a-4325-a510-457907c9be3b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:37.835563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea92f4-a95a-4325-a510-457907c9be3b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:37.835838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea92f4-a95a-4325-a510-457907c9be3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:37.863819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-93ea92f4-a95a-4325-a510-457907c9be3b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:37.865361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-93ea92f4-a95a-4325-a510-457907c9be3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:37.866728 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2859/5729] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:37.875601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf60267-2bed-491f-a377-49abaa0d5f81 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:37.876491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf60267-2bed-491f-a377-49abaa0d5f81 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:37.877780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf60267-2bed-491f-a377-49abaa0d5f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:37.889444 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2871/5730] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:12:37.890974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea92f4-a95a-4325-a510-457907c9be3b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:37.913696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea92f4-a95a-4325-a510-457907c9be3b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:37.920584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3e01b-0c58-4247-9ceb-2a993a858737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:37.921235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3e01b-0c58-4247-9ceb-2a993a858737 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:37.922119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3e01b-0c58-4247-9ceb-2a993a858737 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:37.922462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3e01b-0c58-4247-9ceb-2a993a858737 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:37.923132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3e01b-0c58-4247-9ceb-2a993a858737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:12:37.924326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3e01b-0c58-4247-9ceb-2a993a858737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:37.926714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3e01b-0c58-4247-9ceb-2a993a858737 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:37.932493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:37.940623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bcf60267-2bed-491f-a377-49abaa0d5f81 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:37.942870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bcf60267-2bed-491f-a377-49abaa0d5f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:37.952593 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2872/5731] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:12:37.957339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3e01b-0c58-4247-9ceb-2a993a858737 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:37.959196 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2860/5732] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:12:37.962999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf60267-2bed-491f-a377-49abaa0d5f81 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:37.972093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3e01b-0c58-4247-9ceb-2a993a858737 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:37.977362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51820a-25b9-4d74-a028-8011d308e095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:37.978081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51820a-25b9-4d74-a028-8011d308e095 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:37.978364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51820a-25b9-4d74-a028-8011d308e095 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:37.978636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51820a-25b9-4d74-a028-8011d308e095 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:37.979261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51820a-25b9-4d74-a028-8011d308e095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:37.981215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51820a-25b9-4d74-a028-8011d308e095 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:37.996002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf60267-2bed-491f-a377-49abaa0d5f81 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:38.010164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430cf5-e5e6-48ce-b56c-24ac6b99f89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:38.011159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430cf5-e5e6-48ce-b56c-24ac6b99f89d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.012689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430cf5-e5e6-48ce-b56c-24ac6b99f89d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.013194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430cf5-e5e6-48ce-b56c-24ac6b99f89d admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.014236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33430cf5-e5e6-48ce-b56c-24ac6b99f89d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:12:38.014764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33430cf5-e5e6-48ce-b56c-24ac6b99f89d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:38.017171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33430cf5-e5e6-48ce-b56c-24ac6b99f89d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:38.035654 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2861/5733] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:12:38.041299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430cf5-e5e6-48ce-b56c-24ac6b99f89d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.042095 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2873/5734] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:12:37 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:38.043343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51820a-25b9-4d74-a028-8011d308e095 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.071086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430cf5-e5e6-48ce-b56c-24ac6b99f89d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:38.145749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51820a-25b9-4d74-a028-8011d308e095 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:38.156182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ad286-e04c-4e76-88a4-d2b188fd6c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:38.156779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ad286-e04c-4e76-88a4-d2b188fd6c29 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.157231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ad286-e04c-4e76-88a4-d2b188fd6c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.157505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ad286-e04c-4e76-88a4-d2b188fd6c29 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.158290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ad286-e04c-4e76-88a4-d2b188fd6c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:38.160692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ad286-e04c-4e76-88a4-d2b188fd6c29 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:38.218965 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2874/5735] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:12:38 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:38.224592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ad286-e04c-4e76-88a4-d2b188fd6c29 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.253026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ad286-e04c-4e76-88a4-d2b188fd6c29 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:38.253442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afe3dc-957f-4f9b-84bc-064a0787ce38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:38.254284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afe3dc-957f-4f9b-84bc-064a0787ce38 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.254773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afe3dc-957f-4f9b-84bc-064a0787ce38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.255249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afe3dc-957f-4f9b-84bc-064a0787ce38 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.256199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afe3dc-957f-4f9b-84bc-064a0787ce38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:38.262448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afe3dc-957f-4f9b-84bc-064a0787ce38 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:38.266634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93beed0-f08b-401a-af26-9496004ffc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:38.272209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93beed0-f08b-401a-af26-9496004ffc8f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.272792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93beed0-f08b-401a-af26-9496004ffc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.273137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93beed0-f08b-401a-af26-9496004ffc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.274390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93beed0-f08b-401a-af26-9496004ffc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:38.275865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93beed0-f08b-401a-af26-9496004ffc8f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:38.305457 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2875/5736] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:38.306467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93beed0-f08b-401a-af26-9496004ffc8f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.321010 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2862/5737] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:12:38 2022] POST /identity/v3/projects => generated 373 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:38.321415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93beed0-f08b-401a-af26-9496004ffc8f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:38.326340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afe3dc-957f-4f9b-84bc-064a0787ce38 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.381227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:38.435559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afe3dc-957f-4f9b-84bc-064a0787ce38 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:38.449083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3b2a5-3870-472f-ba35-f58a57e96818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:38.450578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6536b-6470-469e-a5b6-2556825bc404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:38.451503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3b2a5-3870-472f-ba35-f58a57e96818 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.451727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3b2a5-3870-472f-ba35-f58a57e96818 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.451958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3b2a5-3870-472f-ba35-f58a57e96818 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.457132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6536b-6470-469e-a5b6-2556825bc404 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.457132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6536b-6470-469e-a5b6-2556825bc404 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.457132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6536b-6470-469e-a5b6-2556825bc404 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.457501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3b2a5-3870-472f-ba35-f58a57e96818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:38.458721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3b2a5-3870-472f-ba35-f58a57e96818 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:38.459349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6536b-6470-469e-a5b6-2556825bc404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:38.461067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6536b-6470-469e-a5b6-2556825bc404 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:38.498892 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2876/5738] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:38.500228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6536b-6470-469e-a5b6-2556825bc404 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.517996 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2863/5739] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:12:38 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:38.521989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3b2a5-3870-472f-ba35-f58a57e96818 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.531877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6536b-6470-469e-a5b6-2556825bc404 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:38.544859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a52be-0a4c-4694-b18d-26744878096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:38.545910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a52be-0a4c-4694-b18d-26744878096c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.545910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a52be-0a4c-4694-b18d-26744878096c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.546121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a52be-0a4c-4694-b18d-26744878096c admin admin] PATH_INFO: `/v3/projects/0be4ac2274794d4392c4f08bc0d2395e/users/ca0dba28facd46128c0015cbdf7edd67/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.547212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3b2a5-3870-472f-ba35-f58a57e96818 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:38.563421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8909b5-fe13-4d51-85c3-dc2b9d7e926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:38.564072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8909b5-fe13-4d51-85c3-dc2b9d7e926c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.565208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8909b5-fe13-4d51-85c3-dc2b9d7e926c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.565682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8909b5-fe13-4d51-85c3-dc2b9d7e926c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.567261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8909b5-fe13-4d51-85c3-dc2b9d7e926c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:38.570283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8909b5-fe13-4d51-85c3-dc2b9d7e926c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:38.593785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a52be-0a4c-4694-b18d-26744878096c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be4ac2274794d4392c4f08bc0d2395e, user_id=ca0dba28facd46128c0015cbdf7edd67, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:38.599448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a52be-0a4c-4694-b18d-26744878096c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:38.603959 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2864/5740] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:12:38.607856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8909b5-fe13-4d51-85c3-dc2b9d7e926c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.628985 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2877/5741] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:12:38 2022] PUT /identity/v3/projects/0be4ac2274794d4392c4f08bc0d2395e/users/ca0dba28facd46128c0015cbdf7edd67/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:12:38.638232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4d47c-eff0-44b5-b2d1-7ce7b5ecde76 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.638538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4d47c-eff0-44b5-b2d1-7ce7b5ecde76 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.638811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4d47c-eff0-44b5-b2d1-7ce7b5ecde76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.641231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8909b5-fe13-4d51-85c3-dc2b9d7e926c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:38.697349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-20f4d47c-eff0-44b5-b2d1-7ce7b5ecde76 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:38.700814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-20f4d47c-eff0-44b5-b2d1-7ce7b5ecde76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:38.748547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:38.834402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934ffbb-56d3-4c3f-b961-df5ff4984e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:38.835164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934ffbb-56d3-4c3f-b961-df5ff4984e9c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.835708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934ffbb-56d3-4c3f-b961-df5ff4984e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.836496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934ffbb-56d3-4c3f-b961-df5ff4984e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.838634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934ffbb-56d3-4c3f-b961-df5ff4984e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:38.844983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934ffbb-56d3-4c3f-b961-df5ff4984e9c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:38.875534 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2865/5742] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:38.875534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934ffbb-56d3-4c3f-b961-df5ff4984e9c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.875534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2934ffbb-56d3-4c3f-b961-df5ff4984e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:38.879922 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2878/5743] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:12:38.880747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4d47c-eff0-44b5-b2d1-7ce7b5ecde76 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.895091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4d47c-eff0-44b5-b2d1-7ce7b5ecde76 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:38.902638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c02d8-4eb6-429c-9838-246b75aa7ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:38.903685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c02d8-4eb6-429c-9838-246b75aa7ca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:38.905168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c02d8-4eb6-429c-9838-246b75aa7ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:38.905462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c02d8-4eb6-429c-9838-246b75aa7ca4 admin admin] PATH_INFO: `/v3/projects/73a0248de84b42c6b7a4537942c72954/users/9fa8c518febe491ab7c4ff883fb33c4c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:38.938160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c02d8-4eb6-429c-9838-246b75aa7ca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73a0248de84b42c6b7a4537942c72954, user_id=9fa8c518febe491ab7c4ff883fb33c4c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:38.942870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c02d8-4eb6-429c-9838-246b75aa7ca4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:38.965793 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2879/5744] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:12:38 2022] PUT /identity/v3/projects/73a0248de84b42c6b7a4537942c72954/users/9fa8c518febe491ab7c4ff883fb33c4c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:38.966968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c02d8-4eb6-429c-9838-246b75aa7ca4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:38.968726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5a2c02d8-4eb6-429c-9838-246b75aa7ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:38.996774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934ffbb-56d3-4c3f-b961-df5ff4984e9c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:39.041887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c02d8-4eb6-429c-9838-246b75aa7ca4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:39.046069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691abd0f-3276-41f1-9830-9a92c838d2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:39.046660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691abd0f-3276-41f1-9830-9a92c838d2fe service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:39.046944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691abd0f-3276-41f1-9830-9a92c838d2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:39.047214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691abd0f-3276-41f1-9830-9a92c838d2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:39.063080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:39.074877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '038805bf16844e2da7cb92c9eedd4d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f17710ae718b477ea1d24487561ebfc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:39.075663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:39.075942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:39.076208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] PATH_INFO: `/v3/projects/1499132af49f459faf4ab31d0cd6529c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:39.077833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1499132af49f459faf4ab31d0cd6529c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:39.081461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:39.122280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691abd0f-3276-41f1-9830-9a92c838d2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:39.124004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691abd0f-3276-41f1-9830-9a92c838d2fe service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:39.138720 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2880/5745] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:39.138956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1499132af49f459faf4ab31d0cd6529c', 'request_id': 'req-25511076-ff1d-4385-a40e-32e0cf465ae5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:39.140003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691abd0f-3276-41f1-9830-9a92c838d2fe service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:39.143974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-691abd0f-3276-41f1-9830-9a92c838d2fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:39.149073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1499132af49f459faf4ab31d0cd6529c', 'request_id': 'req-25511076-ff1d-4385-a40e-32e0cf465ae5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:39.157299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1499132af49f459faf4ab31d0cd6529c', 'request_id': 'req-25511076-ff1d-4385-a40e-32e0cf465ae5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:39.162442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691abd0f-3276-41f1-9830-9a92c838d2fe service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:39.172162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] The token cache is being invalidate because project 1499132af49f459faf4ab31d0cd6529c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:12:39.172977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:39.174267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-25511076-ff1d-4385-a40e-32e0cf465ae5 tempest-RoutersIpV6AdminTest-1127358172 tempest-RoutersIpV6AdminTest-1127358172-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:39.180555 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2866/5746] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:12:38 2022] DELETE /identity/v3/projects/1499132af49f459faf4ab31d0cd6529c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:39.180555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a60fa-6517-4d1c-8462-241c7d55ea14 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:39.180555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a60fa-6517-4d1c-8462-241c7d55ea14 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:39.180555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a60fa-6517-4d1c-8462-241c7d55ea14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:39.238841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-687a60fa-6517-4d1c-8462-241c7d55ea14 None None] MFA Rules not processed for user `9fa8c518febe491ab7c4ff883fb33c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:39.241593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-687a60fa-6517-4d1c-8462-241c7d55ea14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:39.244681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:39.264945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:39.265969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:39.266383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:39.266727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] PATH_INFO: `/v3/users/fbb0ff2e9f3b4c08a1ced0e068f71f5f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:39.270336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbb0ff2e9f3b4c08a1ced0e068f71f5f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:39.272671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:39.386775 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2867/5747] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:39.386775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a60fa-6517-4d1c-8462-241c7d55ea14 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:39.387436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-687a60fa-6517-4d1c-8462-241c7d55ea14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:39.400969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbb0ff2e9f3b4c08a1ced0e068f71f5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:39.430737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbb0ff2e9f3b4c08a1ced0e068f71f5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:39.444986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbb0ff2e9f3b4c08a1ced0e068f71f5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:39.477484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbb0ff2e9f3b4c08a1ced0e068f71f5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:39.482246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbb0ff2e9f3b4c08a1ced0e068f71f5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:39.493848 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2881/5748] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:12:39 2022] DELETE /identity/v3/users/fbb0ff2e9f3b4c08a1ced0e068f71f5f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:39.495670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:39.497320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:39.545188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a60fa-6517-4d1c-8462-241c7d55ea14 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:39.602821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f064054-4cfe-42ec-abb8-3842e56fafc3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:39.606896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:39.618390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd57d4-56b8-4ca7-a0d4-6a8c114a55ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:39.619001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd57d4-56b8-4ca7-a0d4-6a8c114a55ea service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:39.619404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd57d4-56b8-4ca7-a0d4-6a8c114a55ea service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:39.621712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd57d4-56b8-4ca7-a0d4-6a8c114a55ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:39.627409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-97dd57d4-56b8-4ca7-a0d4-6a8c114a55ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:39.653394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f01ce-928f-4e5f-b0ff-112675188487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:39.654630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f01ce-928f-4e5f-b0ff-112675188487 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:39.655691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f01ce-928f-4e5f-b0ff-112675188487 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:39.656133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f01ce-928f-4e5f-b0ff-112675188487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:39.659835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5a2f01ce-928f-4e5f-b0ff-112675188487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:39.731978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd57d4-56b8-4ca7-a0d4-6a8c114a55ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:39.734349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd57d4-56b8-4ca7-a0d4-6a8c114a55ea service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:39.747541 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2882/5749] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:39.748873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261b06-cab9-463c-bc83-824c10d8a644 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:39.749278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261b06-cab9-463c-bc83-824c10d8a644 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:39.749623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261b06-cab9-463c-bc83-824c10d8a644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:39.760745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f01ce-928f-4e5f-b0ff-112675188487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:39.762727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f01ce-928f-4e5f-b0ff-112675188487 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:39.774928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ff261b06-cab9-463c-bc83-824c10d8a644 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:39.776385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ff261b06-cab9-463c-bc83-824c10d8a644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:39.780161 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2868/5750] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:39.796641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:39.859749 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2883/5751] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:39.899203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f01ce-928f-4e5f-b0ff-112675188487 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:39.913285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f01ce-928f-4e5f-b0ff-112675188487 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:39.917493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33396987-1e71-4719-8193-cba507958faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:39.918100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33396987-1e71-4719-8193-cba507958faf service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:39.918426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33396987-1e71-4719-8193-cba507958faf service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:39.918694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33396987-1e71-4719-8193-cba507958faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:39.929981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33396987-1e71-4719-8193-cba507958faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:39.931582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33396987-1e71-4719-8193-cba507958faf service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:39.952683 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2869/5752] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:40.979379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261b06-cab9-463c-bc83-824c10d8a644 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:40.981074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ff261b06-cab9-463c-bc83-824c10d8a644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:40.998339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261b06-cab9-463c-bc83-824c10d8a644 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:41.004799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:41.005988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:41.006449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:41.006857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] PATH_INFO: `/v3/projects/d2fba9ff4830494ab0f4259143c4619a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:41.008998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2fba9ff4830494ab0f4259143c4619a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:41.011579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:41.089779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2fba9ff4830494ab0f4259143c4619a', 'request_id': 'req-4a79af4d-39c4-494e-8175-012da6fce5bc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:41.100111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2fba9ff4830494ab0f4259143c4619a', 'request_id': 'req-4a79af4d-39c4-494e-8175-012da6fce5bc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:41.114931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2fba9ff4830494ab0f4259143c4619a', 'request_id': 'req-4a79af4d-39c4-494e-8175-012da6fce5bc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:41.158856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] The token cache is being invalidate because project d2fba9ff4830494ab0f4259143c4619a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:12:41.159323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:41.160240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a79af4d-39c4-494e-8175-012da6fce5bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:12:41.165702 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2884/5753] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:12:40 2022] DELETE /identity/v3/projects/d2fba9ff4830494ab0f4259143c4619a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:41.167605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b82da-d88d-4846-9dff-c390ff776487 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:41.168410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b82da-d88d-4846-9dff-c390ff776487 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:41.168864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b82da-d88d-4846-9dff-c390ff776487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:41.210530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d13b82da-d88d-4846-9dff-c390ff776487 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:41.212037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d13b82da-d88d-4846-9dff-c390ff776487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:41.305718 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2870/5754] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:41.313416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c9078-5d88-44f2-8332-04638451de7f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:41.313694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c9078-5d88-44f2-8332-04638451de7f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:41.313965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c9078-5d88-44f2-8332-04638451de7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:41.351955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-273c9078-5d88-44f2-8332-04638451de7f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:41.357628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-273c9078-5d88-44f2-8332-04638451de7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:41.375476 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2885/5755] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:41.377230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b82da-d88d-4846-9dff-c390ff776487 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:41.394375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b82da-d88d-4846-9dff-c390ff776487 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:41.399768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a321d9-9c4f-4ae3-85b6-a24bd6f4298a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:41.400701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a321d9-9c4f-4ae3-85b6-a24bd6f4298a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:41.401672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a321d9-9c4f-4ae3-85b6-a24bd6f4298a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:41.402192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a321d9-9c4f-4ae3-85b6-a24bd6f4298a admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:41.403789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a321d9-9c4f-4ae3-85b6-a24bd6f4298a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:12:41.404313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a321d9-9c4f-4ae3-85b6-a24bd6f4298a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:41.406222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a321d9-9c4f-4ae3-85b6-a24bd6f4298a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:41.420788 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2871/5756] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:12:41.426685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c9078-5d88-44f2-8332-04638451de7f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:41.436941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c9078-5d88-44f2-8332-04638451de7f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:41.443697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e8333-c65c-479d-8c6e-8985d12a0ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:41.444242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e8333-c65c-479d-8c6e-8985d12a0ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:41.444567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e8333-c65c-479d-8c6e-8985d12a0ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:41.444838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e8333-c65c-479d-8c6e-8985d12a0ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:41.445502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e8333-c65c-479d-8c6e-8985d12a0ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:41.449277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e8333-c65c-479d-8c6e-8985d12a0ec8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:41.486862 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2886/5757] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:12:41 2022] POST /identity/v3/projects => generated 359 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:41.488799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a321d9-9c4f-4ae3-85b6-a24bd6f4298a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:41.515147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a321d9-9c4f-4ae3-85b6-a24bd6f4298a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:41.624825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:41.635713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61df21e-6ccb-49a1-85bd-a68414124bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:41.636345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61df21e-6ccb-49a1-85bd-a68414124bde admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:41.637263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61df21e-6ccb-49a1-85bd-a68414124bde admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:41.637603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61df21e-6ccb-49a1-85bd-a68414124bde admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:41.638629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61df21e-6ccb-49a1-85bd-a68414124bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:41.640664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61df21e-6ccb-49a1-85bd-a68414124bde admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:41.686991 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2872/5758] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:12:41 2022] POST /identity/v3/users => generated 280 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:12:41.690956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e8333-c65c-479d-8c6e-8985d12a0ec8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:41.707218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e8333-c65c-479d-8c6e-8985d12a0ec8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:41.726840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603f911-b93f-4d55-992e-58ffe6c6e0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:41.727774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603f911-b93f-4d55-992e-58ffe6c6e0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:41.728352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603f911-b93f-4d55-992e-58ffe6c6e0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:41.728692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603f911-b93f-4d55-992e-58ffe6c6e0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:41.729635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603f911-b93f-4d55-992e-58ffe6c6e0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:41.731875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603f911-b93f-4d55-992e-58ffe6c6e0d7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:41.754249 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2887/5759] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:41.755931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61df21e-6ccb-49a1-85bd-a68414124bde admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:41.768478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61df21e-6ccb-49a1-85bd-a68414124bde admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:41.784866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0aa12e-7c7b-49ad-86fa-acfe5d5b7b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:41.786190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0aa12e-7c7b-49ad-86fa-acfe5d5b7b24 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:41.787091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0aa12e-7c7b-49ad-86fa-acfe5d5b7b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:41.787452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0aa12e-7c7b-49ad-86fa-acfe5d5b7b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:41.788253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0aa12e-7c7b-49ad-86fa-acfe5d5b7b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:41.790196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0aa12e-7c7b-49ad-86fa-acfe5d5b7b24 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:41.807342 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2873/5760] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:12:41.808646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603f911-b93f-4d55-992e-58ffe6c6e0d7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:41.825515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603f911-b93f-4d55-992e-58ffe6c6e0d7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:41.832875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7386b-7c30-4346-b22b-7256d8a3108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:41.833950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7386b-7c30-4346-b22b-7256d8a3108d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:41.834384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7386b-7c30-4346-b22b-7256d8a3108d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:41.835014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7386b-7c30-4346-b22b-7256d8a3108d admin admin] PATH_INFO: `/v3/projects/bca1e42b396e43f1af93381ad28d1362/users/701f38823f864ec9ba7d70e15d9664ef/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:41.859703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7386b-7c30-4346-b22b-7256d8a3108d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca1e42b396e43f1af93381ad28d1362, user_id=701f38823f864ec9ba7d70e15d9664ef, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:41.864133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7386b-7c30-4346-b22b-7256d8a3108d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:41.892066 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2888/5761] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:12:41 2022] PUT /identity/v3/projects/bca1e42b396e43f1af93381ad28d1362/users/701f38823f864ec9ba7d70e15d9664ef/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:12:41.895603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d7e80-162e-4546-8d8c-aac44e0f0b29 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:41.896221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d7e80-162e-4546-8d8c-aac44e0f0b29 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:41.896688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d7e80-162e-4546-8d8c-aac44e0f0b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:41.947032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2d9d7e80-162e-4546-8d8c-aac44e0f0b29 None None] MFA Rules not processed for user `701f38823f864ec9ba7d70e15d9664ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:41.948523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2d9d7e80-162e-4546-8d8c-aac44e0f0b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:42.087305 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2874/5762] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:42.098058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7386b-7c30-4346-b22b-7256d8a3108d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:42.101223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f6e7386b-7c30-4346-b22b-7256d8a3108d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:42.175949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:42.196533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7386b-7c30-4346-b22b-7256d8a3108d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:42.211546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66094bbe-14b3-417a-928c-940765958547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:42.212167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094bbe-14b3-417a-928c-940765958547 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:42.212439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094bbe-14b3-417a-928c-940765958547 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:42.212703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094bbe-14b3-417a-928c-940765958547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:42.224499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66094bbe-14b3-417a-928c-940765958547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:42.225991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66094bbe-14b3-417a-928c-940765958547 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:42.237918 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2889/5763] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:42 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:12:52.066017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:52.232186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:54.264341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab177d58-3613-4528-926d-5351f2e77e7b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:54.264819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab177d58-3613-4528-926d-5351f2e77e7b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:54.265102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab177d58-3613-4528-926d-5351f2e77e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:54.361599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ab177d58-3613-4528-926d-5351f2e77e7b None None] MFA Rules not processed for user `ca0dba28facd46128c0015cbdf7edd67`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:54.363976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ab177d58-3613-4528-926d-5351f2e77e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:54.568769 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2875/5764] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:54.582029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7af3f-5a96-42d9-bdfb-22c713de9211 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:54.583415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7af3f-5a96-42d9-bdfb-22c713de9211 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:54.584063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7af3f-5a96-42d9-bdfb-22c713de9211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:54.636392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-09f7af3f-5a96-42d9-bdfb-22c713de9211 None None] MFA Rules not processed for user `ca0dba28facd46128c0015cbdf7edd67`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:12:54.646884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-09f7af3f-5a96-42d9-bdfb-22c713de9211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:54.672180 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2890/5765] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:12:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:12:54.689141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab177d58-3613-4528-926d-5351f2e77e7b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:12:54.690763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ab177d58-3613-4528-926d-5351f2e77e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:12:54.847784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab177d58-3613-4528-926d-5351f2e77e7b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:12:54.859671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91345a4-d68b-4855-8901-d0c05f4c9406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:12:54.860821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91345a4-d68b-4855-8901-d0c05f4c9406 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:12:54.861118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91345a4-d68b-4855-8901-d0c05f4c9406 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:12:54.861380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91345a4-d68b-4855-8901-d0c05f4c9406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:12:54.877916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91345a4-d68b-4855-8901-d0c05f4c9406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:12:54.880864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91345a4-d68b-4855-8901-d0c05f4c9406 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:12:54.908155 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2876/5766] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:12:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:04.618119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:04.895964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=66 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:09.548432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7af3f-5a96-42d9-bdfb-22c713de9211 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:09.551294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-09f7af3f-5a96-42d9-bdfb-22c713de9211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:09.603004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7af3f-5a96-42d9-bdfb-22c713de9211 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:09.608135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28319ea-471d-47d3-a845-ef418c0d3e51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:09.609085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28319ea-471d-47d3-a845-ef418c0d3e51 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:09.610066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28319ea-471d-47d3-a845-ef418c0d3e51 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:09.610651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28319ea-471d-47d3-a845-ef418c0d3e51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:09.613220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f28319ea-471d-47d3-a845-ef418c0d3e51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:09.646406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28319ea-471d-47d3-a845-ef418c0d3e51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:09.648191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28319ea-471d-47d3-a845-ef418c0d3e51 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:09.682859 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2891/5767] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:17.091834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa64f2-6c5a-4424-b3f5-5e7923a50d4d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:17.092419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa64f2-6c5a-4424-b3f5-5e7923a50d4d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:17.092574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa64f2-6c5a-4424-b3f5-5e7923a50d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:17.123051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4baa64f2-6c5a-4424-b3f5-5e7923a50d4d None None] MFA Rules not processed for user `1aadd87b97c247cb95321664ba0c32a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:17.126748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4baa64f2-6c5a-4424-b3f5-5e7923a50d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:17.232228 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2877/5768] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:18.523617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28319ea-471d-47d3-a845-ef418c0d3e51 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:18.529879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f28319ea-471d-47d3-a845-ef418c0d3e51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:18.632937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28319ea-471d-47d3-a845-ef418c0d3e51 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:18.638096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:18.638977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:18.639503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:18.640007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] PATH_INFO: `/v3/users/9fa8c518febe491ab7c4ff883fb33c4c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:18.642144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa8c518febe491ab7c4ff883fb33c4c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:18.645828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:18.725242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa8c518febe491ab7c4ff883fb33c4c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:18.751244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa8c518febe491ab7c4ff883fb33c4c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:18.767888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa8c518febe491ab7c4ff883fb33c4c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:18.779194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa8c518febe491ab7c4ff883fb33c4c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:18.780419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa8c518febe491ab7c4ff883fb33c4c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:18.788755 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2892/5769] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:13:18 2022] DELETE /identity/v3/users/9fa8c518febe491ab7c4ff883fb33c4c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:18.792957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec499427-2dd5-4039-981e-df300170308b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:18.793511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec499427-2dd5-4039-981e-df300170308b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:18.793511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec499427-2dd5-4039-981e-df300170308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:18.832901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ec499427-2dd5-4039-981e-df300170308b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:18.836106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ec499427-2dd5-4039-981e-df300170308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:18.939473 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2878/5770] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:18.962143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:18.964834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:19.059013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79639507-8f56-4f4f-a01d-07b2e7100483 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:19.062916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb043fc-0329-4c4b-a3b7-a53a9401992e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:19.063506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb043fc-0329-4c4b-a3b7-a53a9401992e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:19.063833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb043fc-0329-4c4b-a3b7-a53a9401992e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:19.064378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb043fc-0329-4c4b-a3b7-a53a9401992e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:19.076863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb043fc-0329-4c4b-a3b7-a53a9401992e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:19.078948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb043fc-0329-4c4b-a3b7-a53a9401992e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:19.083267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:19.104986 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2893/5771] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:20.309678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec499427-2dd5-4039-981e-df300170308b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:20.311703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ec499427-2dd5-4039-981e-df300170308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:20.337376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec499427-2dd5-4039-981e-df300170308b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:20.356608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:20.359656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:20.360214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:20.360918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] PATH_INFO: `/v3/projects/73a0248de84b42c6b7a4537942c72954` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:20.364592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73a0248de84b42c6b7a4537942c72954)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:20.371216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:20.494594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73a0248de84b42c6b7a4537942c72954', 'request_id': 'req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:20.517896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73a0248de84b42c6b7a4537942c72954', 'request_id': 'req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:20.541007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73a0248de84b42c6b7a4537942c72954', 'request_id': 'req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:20.566040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] The token cache is being invalidate because project 73a0248de84b42c6b7a4537942c72954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:13:20.566421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:20.567798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ea2de3ea-eb0f-43fb-a22a-bf04e1811175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:20.575208 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2879/5772] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:13:20 2022] DELETE /identity/v3/projects/73a0248de84b42c6b7a4537942c72954 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:20.576571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb49b6-5e69-4085-b0f3-01767fce6874 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:20.577077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb49b6-5e69-4085-b0f3-01767fce6874 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:20.578471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb49b6-5e69-4085-b0f3-01767fce6874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:20.620413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d0fb49b6-5e69-4085-b0f3-01767fce6874 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:20.622172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d0fb49b6-5e69-4085-b0f3-01767fce6874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:20.720449 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2894/5773] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:20.728800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e1fec-fd83-4f7d-855f-99c6ab5a8366 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:20.729884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e1fec-fd83-4f7d-855f-99c6ab5a8366 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:20.730164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e1fec-fd83-4f7d-855f-99c6ab5a8366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:20.759322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bb7e1fec-fd83-4f7d-855f-99c6ab5a8366 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:20.760750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bb7e1fec-fd83-4f7d-855f-99c6ab5a8366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:20.773382 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2880/5774] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:20.778207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb49b6-5e69-4085-b0f3-01767fce6874 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:20.793349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb49b6-5e69-4085-b0f3-01767fce6874 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:20.809425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6644f8b-3946-4468-ad41-5252e9d44d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:20.811325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6644f8b-3946-4468-ad41-5252e9d44d17 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:20.811723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6644f8b-3946-4468-ad41-5252e9d44d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:20.812176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6644f8b-3946-4468-ad41-5252e9d44d17 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:20.812939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6644f8b-3946-4468-ad41-5252e9d44d17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:13:20.813332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6644f8b-3946-4468-ad41-5252e9d44d17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:20.815106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6644f8b-3946-4468-ad41-5252e9d44d17 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:20.833318 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2895/5775] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:13:20.837458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e1fec-fd83-4f7d-855f-99c6ab5a8366 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:20.848718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e1fec-fd83-4f7d-855f-99c6ab5a8366 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:20.871797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138bfb-ee6f-4d86-914c-65a6ea3170a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:20.872505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138bfb-ee6f-4d86-914c-65a6ea3170a8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:20.872966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138bfb-ee6f-4d86-914c-65a6ea3170a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:20.873353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138bfb-ee6f-4d86-914c-65a6ea3170a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:20.874138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce138bfb-ee6f-4d86-914c-65a6ea3170a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:20.876361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce138bfb-ee6f-4d86-914c-65a6ea3170a8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:20.912595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:20.935980 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2881/5776] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:13:20 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:20.940475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6644f8b-3946-4468-ad41-5252e9d44d17 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:20.964991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6644f8b-3946-4468-ad41-5252e9d44d17 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:21.068486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fd2e0-bde0-4162-b633-6a1dbaa26718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:21.068838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fd2e0-bde0-4162-b633-6a1dbaa26718 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:21.069180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fd2e0-bde0-4162-b633-6a1dbaa26718 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:21.069481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fd2e0-bde0-4162-b633-6a1dbaa26718 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:21.070117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fd2e0-bde0-4162-b633-6a1dbaa26718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:21.072300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fd2e0-bde0-4162-b633-6a1dbaa26718 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:21.103322 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2896/5777] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:13:20 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:21.107096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138bfb-ee6f-4d86-914c-65a6ea3170a8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:21.137110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138bfb-ee6f-4d86-914c-65a6ea3170a8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:21.144488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcb93c-38b8-4102-8a4d-9db067583970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:21.145672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcb93c-38b8-4102-8a4d-9db067583970 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:21.146057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcb93c-38b8-4102-8a4d-9db067583970 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:21.146730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcb93c-38b8-4102-8a4d-9db067583970 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:21.147872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcb93c-38b8-4102-8a4d-9db067583970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:21.149554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcb93c-38b8-4102-8a4d-9db067583970 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:21.165703 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2882/5778] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:13:21.168159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fd2e0-bde0-4162-b633-6a1dbaa26718 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:21.182359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fd2e0-bde0-4162-b633-6a1dbaa26718 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:21.187997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857bce0-03ae-48de-bf55-e7f605c5dfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:21.188542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857bce0-03ae-48de-bf55-e7f605c5dfca admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:21.188824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857bce0-03ae-48de-bf55-e7f605c5dfca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:21.189169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857bce0-03ae-48de-bf55-e7f605c5dfca admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:21.189874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857bce0-03ae-48de-bf55-e7f605c5dfca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:21.191736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857bce0-03ae-48de-bf55-e7f605c5dfca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:21.217123 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2897/5779] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:21.225589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcb93c-38b8-4102-8a4d-9db067583970 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:21.256966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcb93c-38b8-4102-8a4d-9db067583970 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:21.280022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27829fd6-8451-4721-bbbb-3e69f09528a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:21.281463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27829fd6-8451-4721-bbbb-3e69f09528a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:21.282156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27829fd6-8451-4721-bbbb-3e69f09528a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:21.282997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27829fd6-8451-4721-bbbb-3e69f09528a4 admin admin] PATH_INFO: `/v3/projects/7b7dbcbca8d6411bb4460a9be184ea3e/users/6e69d9ed47c241acbbbda3a4e7561952/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:21.314732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27829fd6-8451-4721-bbbb-3e69f09528a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b7dbcbca8d6411bb4460a9be184ea3e, user_id=6e69d9ed47c241acbbbda3a4e7561952, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:21.319894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27829fd6-8451-4721-bbbb-3e69f09528a4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:21.369512 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2883/5780] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:21 2022] PUT /identity/v3/projects/7b7dbcbca8d6411bb4460a9be184ea3e/users/6e69d9ed47c241acbbbda3a4e7561952/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:21.384730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af849867-d1b4-446d-96f3-f8c53ce507e9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:21.384730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af849867-d1b4-446d-96f3-f8c53ce507e9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:21.384730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af849867-d1b4-446d-96f3-f8c53ce507e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:21.454307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-af849867-d1b4-446d-96f3-f8c53ce507e9 None None] MFA Rules not processed for user `6e69d9ed47c241acbbbda3a4e7561952`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:21.455477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af849867-d1b4-446d-96f3-f8c53ce507e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:21.566861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:21.618064 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2898/5781] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:21.660117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27829fd6-8451-4721-bbbb-3e69f09528a4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:21.663267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-27829fd6-8451-4721-bbbb-3e69f09528a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:21.764944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27829fd6-8451-4721-bbbb-3e69f09528a4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:21.770117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b65a2d-f49d-4d3a-a5e0-02843f4f9e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:21.771777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b65a2d-f49d-4d3a-a5e0-02843f4f9e96 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:21.773438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b65a2d-f49d-4d3a-a5e0-02843f4f9e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:21.774015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b65a2d-f49d-4d3a-a5e0-02843f4f9e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:21.787166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b65a2d-f49d-4d3a-a5e0-02843f4f9e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:21.789334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b65a2d-f49d-4d3a-a5e0-02843f4f9e96 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:21.806417 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2884/5782] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:24.500647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af849867-d1b4-446d-96f3-f8c53ce507e9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:24.502735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af849867-d1b4-446d-96f3-f8c53ce507e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:24.564994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af849867-d1b4-446d-96f3-f8c53ce507e9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:24.587754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185252c9-872b-4a81-9934-be13eedb782d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:24.588393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185252c9-872b-4a81-9934-be13eedb782d service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:24.588664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185252c9-872b-4a81-9934-be13eedb782d service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:24.588927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185252c9-872b-4a81-9934-be13eedb782d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:24.595689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-185252c9-872b-4a81-9934-be13eedb782d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:24.632182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185252c9-872b-4a81-9934-be13eedb782d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:24.634583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185252c9-872b-4a81-9934-be13eedb782d service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:24.664722 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2899/5783] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:31.796549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:34.643511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=13 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:43.129436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10dad7-05bd-41f7-bb27-ce0ca4f9a916 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:43.129984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10dad7-05bd-41f7-bb27-ce0ca4f9a916 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:43.129984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10dad7-05bd-41f7-bb27-ce0ca4f9a916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:43.168564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fc10dad7-05bd-41f7-bb27-ce0ca4f9a916 None None] MFA Rules not processed for user `1aadd87b97c247cb95321664ba0c32a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:43.172220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fc10dad7-05bd-41f7-bb27-ce0ca4f9a916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:43.297330 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2885/5784] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:45.140400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185252c9-872b-4a81-9934-be13eedb782d service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:45.142021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-185252c9-872b-4a81-9934-be13eedb782d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:45.235556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10dad7-05bd-41f7-bb27-ce0ca4f9a916 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:45.243055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fc10dad7-05bd-41f7-bb27-ce0ca4f9a916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:45.323357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185252c9-872b-4a81-9934-be13eedb782d service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:45.336771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597ce56-c159-443f-a2df-20102f1770a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:45.338009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597ce56-c159-443f-a2df-20102f1770a4 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:45.338289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597ce56-c159-443f-a2df-20102f1770a4 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:45.339798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597ce56-c159-443f-a2df-20102f1770a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:45.345652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9597ce56-c159-443f-a2df-20102f1770a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:45.387180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597ce56-c159-443f-a2df-20102f1770a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:45.393006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597ce56-c159-443f-a2df-20102f1770a4 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:45.423884 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2900/5785] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:45.433641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10dad7-05bd-41f7-bb27-ce0ca4f9a916 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:45.446892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:45.449372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:45.449939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:45.450495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] PATH_INFO: `/v3/users/ca0dba28facd46128c0015cbdf7edd67` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:45.457459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0dba28facd46128c0015cbdf7edd67)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:45.462890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:45.552474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597ce56-c159-443f-a2df-20102f1770a4 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:45.554698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9597ce56-c159-443f-a2df-20102f1770a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:45.585944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597ce56-c159-443f-a2df-20102f1770a4 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:45.591950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:45.592845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:45.593277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:45.593643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] PATH_INFO: `/v3/users/de1f5c0c96d848e195426008c6840936` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:45.596657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1f5c0c96d848e195426008c6840936)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:45.600207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:45.611661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0dba28facd46128c0015cbdf7edd67'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:45.670714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0dba28facd46128c0015cbdf7edd67'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:45.712732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1f5c0c96d848e195426008c6840936'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:45.721891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0dba28facd46128c0015cbdf7edd67'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:45.738261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0dba28facd46128c0015cbdf7edd67'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:45.740008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e021424c-a518-4a6b-b45e-c64ddf4f3a9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0dba28facd46128c0015cbdf7edd67'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:45.751989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1f5c0c96d848e195426008c6840936'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:45.767970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1f5c0c96d848e195426008c6840936'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:45.779687 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2886/5786] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:13:45 2022] DELETE /identity/v3/users/ca0dba28facd46128c0015cbdf7edd67 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:45.800952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1f5c0c96d848e195426008c6840936'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:45.801962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1f5c0c96d848e195426008c6840936'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:45.814577 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2901/5787] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:13:45 2022] DELETE /identity/v3/users/de1f5c0c96d848e195426008c6840936 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:45.819882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b50e7-f485-40c5-98ad-ae97a49f8d85 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:45.820167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b50e7-f485-40c5-98ad-ae97a49f8d85 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:45.820656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b50e7-f485-40c5-98ad-ae97a49f8d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:45.856044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-388b50e7-f485-40c5-98ad-ae97a49f8d85 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:45.857529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-388b50e7-f485-40c5-98ad-ae97a49f8d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:45.890888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:45.995850 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2887/5788] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:46.012730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:46.015076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:46.148723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badac17b-ae7e-42ca-af9a-744918608660 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:46.161993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509f0d8-9034-4498-b08b-9a091303ab03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:46.162653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509f0d8-9034-4498-b08b-9a091303ab03 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:46.163101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509f0d8-9034-4498-b08b-9a091303ab03 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:46.163222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509f0d8-9034-4498-b08b-9a091303ab03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:46.181954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509f0d8-9034-4498-b08b-9a091303ab03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:46.183461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509f0d8-9034-4498-b08b-9a091303ab03 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:46.215519 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2902/5789] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:46.624714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b50e7-f485-40c5-98ad-ae97a49f8d85 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:46.632601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-388b50e7-f485-40c5-98ad-ae97a49f8d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:46.646629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b50e7-f485-40c5-98ad-ae97a49f8d85 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:46.652773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:46.653667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:46.654145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:46.655180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] PATH_INFO: `/v3/projects/0133e7f74bc9474b8c47920563402cf1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:46.656595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0133e7f74bc9474b8c47920563402cf1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:46.658713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:46.717225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509f0d8-9034-4498-b08b-9a091303ab03 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:46.718925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b509f0d8-9034-4498-b08b-9a091303ab03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:46.747707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:46.761480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509f0d8-9034-4498-b08b-9a091303ab03 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:46.769180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0133e7f74bc9474b8c47920563402cf1', 'request_id': 'req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:46.783768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0133e7f74bc9474b8c47920563402cf1', 'request_id': 'req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:46.796000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0133e7f74bc9474b8c47920563402cf1', 'request_id': 'req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:46.810507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] The token cache is being invalidate because project 0133e7f74bc9474b8c47920563402cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:13:46.810992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:46.811958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94bc3bf3-75bd-4ab5-a85d-2fd5a8141004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:46.814154 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2888/5790] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:13:46 2022] DELETE /identity/v3/projects/0133e7f74bc9474b8c47920563402cf1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:46.819166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf93bf-47aa-40d6-80f0-6b64ee9aa8fc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:46.819575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf93bf-47aa-40d6-80f0-6b64ee9aa8fc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:46.819952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf93bf-47aa-40d6-80f0-6b64ee9aa8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:46.846398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4daf93bf-47aa-40d6-80f0-6b64ee9aa8fc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:46.847814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4daf93bf-47aa-40d6-80f0-6b64ee9aa8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:46.868588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:46.869782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:46.870396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:46.870890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] PATH_INFO: `/v3/users/6e69d9ed47c241acbbbda3a4e7561952` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:46.873817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e69d9ed47c241acbbbda3a4e7561952)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:46.876834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:46.930484 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2889/5791] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:13:46.942857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9468050-ae6b-4bb3-a0b8-b7fc00f40125 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:46.943115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9468050-ae6b-4bb3-a0b8-b7fc00f40125 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:46.943378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9468050-ae6b-4bb3-a0b8-b7fc00f40125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:46.949019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e69d9ed47c241acbbbda3a4e7561952'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:46.968903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e9468050-ae6b-4bb3-a0b8-b7fc00f40125 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:46.971367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e9468050-ae6b-4bb3-a0b8-b7fc00f40125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:46.972655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e69d9ed47c241acbbbda3a4e7561952'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:46.981183 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2890/5792] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:46.985215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9468050-ae6b-4bb3-a0b8-b7fc00f40125 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:46.988125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e69d9ed47c241acbbbda3a4e7561952'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.009413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e69d9ed47c241acbbbda3a4e7561952'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.011021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1939dc0-1227-4d2b-8fc9-c785b39a6272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e69d9ed47c241acbbbda3a4e7561952'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.012347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9468050-ae6b-4bb3-a0b8-b7fc00f40125 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.019945 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2903/5793] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:13:46 2022] DELETE /identity/v3/users/6e69d9ed47c241acbbbda3a4e7561952 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:47.024072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1b30d-7b36-47cc-a946-f780f42cf522 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.024569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1b30d-7b36-47cc-a946-f780f42cf522 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.025123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1b30d-7b36-47cc-a946-f780f42cf522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.026771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80f2f5-d1d1-4d92-be8e-c4c6cb0d136d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:47.027394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80f2f5-d1d1-4d92-be8e-c4c6cb0d136d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.027670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80f2f5-d1d1-4d92-be8e-c4c6cb0d136d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.027986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80f2f5-d1d1-4d92-be8e-c4c6cb0d136d admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.028649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80f2f5-d1d1-4d92-be8e-c4c6cb0d136d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:13:47.028964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80f2f5-d1d1-4d92-be8e-c4c6cb0d136d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:47.035732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80f2f5-d1d1-4d92-be8e-c4c6cb0d136d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:47.050023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-75c1b30d-7b36-47cc-a946-f780f42cf522 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:47.051812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-75c1b30d-7b36-47cc-a946-f780f42cf522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:47.072274 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2891/5794] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:13:47.074173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80f2f5-d1d1-4d92-be8e-c4c6cb0d136d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.076155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cf80f2f5-d1d1-4d92-be8e-c4c6cb0d136d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:47.105949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80f2f5-d1d1-4d92-be8e-c4c6cb0d136d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.164421 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2904/5795] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:13:47.172061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:47.183665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1b30d-7b36-47cc-a946-f780f42cf522 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.185074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-75c1b30d-7b36-47cc-a946-f780f42cf522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:47.219286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383665c3-3889-42e3-be8e-daa433e17aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:47.220719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383665c3-3889-42e3-be8e-daa433e17aea admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.221244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383665c3-3889-42e3-be8e-daa433e17aea admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.221851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383665c3-3889-42e3-be8e-daa433e17aea admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.225572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383665c3-3889-42e3-be8e-daa433e17aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:47.228501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383665c3-3889-42e3-be8e-daa433e17aea admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:47.245690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:47.282468 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2892/5796] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:13:47 2022] POST /identity/v3/projects => generated 359 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:47.283626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383665c3-3889-42e3-be8e-daa433e17aea admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.285811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-383665c3-3889-42e3-be8e-daa433e17aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:47.300891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1b30d-7b36-47cc-a946-f780f42cf522 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.361475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db8456-3a63-42cf-a5e9-630c69fc0c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:47.362345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383665c3-3889-42e3-be8e-daa433e17aea admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.362552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db8456-3a63-42cf-a5e9-630c69fc0c53 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.362830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db8456-3a63-42cf-a5e9-630c69fc0c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.363098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db8456-3a63-42cf-a5e9-630c69fc0c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.367087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:47.367808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.368074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.368341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] PATH_INFO: `/v3/projects/0be4ac2274794d4392c4f08bc0d2395e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.369493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0be4ac2274794d4392c4f08bc0d2395e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:47.371365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:47.373155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db8456-3a63-42cf-a5e9-630c69fc0c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:47.375788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db8456-3a63-42cf-a5e9-630c69fc0c53 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:47.388433 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2905/5797] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:47.390016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db8456-3a63-42cf-a5e9-630c69fc0c53 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.402897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db8456-3a63-42cf-a5e9-630c69fc0c53 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.413303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc28cc-04cd-4d39-a8b1-516478f5da3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:47.414057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc28cc-04cd-4d39-a8b1-516478f5da3e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.414393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc28cc-04cd-4d39-a8b1-516478f5da3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.414636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc28cc-04cd-4d39-a8b1-516478f5da3e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.417492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc28cc-04cd-4d39-a8b1-516478f5da3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:47.423496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc28cc-04cd-4d39-a8b1-516478f5da3e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:47.433659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0be4ac2274794d4392c4f08bc0d2395e', 'request_id': 'req-8344f88e-8ffe-48f2-ad71-2cbd814ba716'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.448303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0be4ac2274794d4392c4f08bc0d2395e', 'request_id': 'req-8344f88e-8ffe-48f2-ad71-2cbd814ba716'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.456570 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2906/5798] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:13:47 2022] POST /identity/v3/users => generated 280 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:47.460054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc28cc-04cd-4d39-a8b1-516478f5da3e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.464447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0be4ac2274794d4392c4f08bc0d2395e', 'request_id': 'req-8344f88e-8ffe-48f2-ad71-2cbd814ba716'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.475981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc28cc-04cd-4d39-a8b1-516478f5da3e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.485828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] The token cache is being invalidate because project 0be4ac2274794d4392c4f08bc0d2395e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:13:47.486259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.488587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8344f88e-8ffe-48f2-ad71-2cbd814ba716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.492186 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2893/5799] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:13:47 2022] DELETE /identity/v3/projects/0be4ac2274794d4392c4f08bc0d2395e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:47.500482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d9a60-5619-4a5a-8314-da2b148ec6de None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.501017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d9a60-5619-4a5a-8314-da2b148ec6de None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.501309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d9a60-5619-4a5a-8314-da2b148ec6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.526792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-dd6d9a60-5619-4a5a-8314-da2b148ec6de None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:47.528896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dd6d9a60-5619-4a5a-8314-da2b148ec6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:47.560013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887be82-b428-422a-ad25-64eeb745d60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:47.560761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887be82-b428-422a-ad25-64eeb745d60b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.561286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887be82-b428-422a-ad25-64eeb745d60b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.561795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887be82-b428-422a-ad25-64eeb745d60b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.562816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887be82-b428-422a-ad25-64eeb745d60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:47.565846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887be82-b428-422a-ad25-64eeb745d60b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:47.579100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:47.582033 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2907/5800] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:47.585824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887be82-b428-422a-ad25-64eeb745d60b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.589593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b887be82-b428-422a-ad25-64eeb745d60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:47.608584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887be82-b428-422a-ad25-64eeb745d60b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.615483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169c1f1-33dd-48ff-b50e-7f636a4abd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:47.616283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c1f1-33dd-48ff-b50e-7f636a4abd9e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.616717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c1f1-33dd-48ff-b50e-7f636a4abd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.617120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c1f1-33dd-48ff-b50e-7f636a4abd9e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.618009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169c1f1-33dd-48ff-b50e-7f636a4abd9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:47.619834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169c1f1-33dd-48ff-b50e-7f636a4abd9e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:47.626098 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2894/5801] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:47.627830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa4f89-a267-442d-9d05-a40bc0a8b691 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.628200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa4f89-a267-442d-9d05-a40bc0a8b691 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.628474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa4f89-a267-442d-9d05-a40bc0a8b691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.635340 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2908/5802] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:47.639956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169c1f1-33dd-48ff-b50e-7f636a4abd9e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.657309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169c1f1-33dd-48ff-b50e-7f636a4abd9e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.659817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2faa4f89-a267-442d-9d05-a40bc0a8b691 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:47.661790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2faa4f89-a267-442d-9d05-a40bc0a8b691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:47.664156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69392917-17dd-4768-9a6a-e8266efcc9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:47.664902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69392917-17dd-4768-9a6a-e8266efcc9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.665336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69392917-17dd-4768-9a6a-e8266efcc9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.665487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69392917-17dd-4768-9a6a-e8266efcc9ca admin admin] PATH_INFO: `/v3/projects/76a47d2411ba41fb8c7a1f163a07ec74/users/e8f30198ba394d77a21157598d5f4d7e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.680554 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2895/5803] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:47.687574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa4f89-a267-442d-9d05-a40bc0a8b691 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.700159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69392917-17dd-4768-9a6a-e8266efcc9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=76a47d2411ba41fb8c7a1f163a07ec74, user_id=e8f30198ba394d77a21157598d5f4d7e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:47.703976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69392917-17dd-4768-9a6a-e8266efcc9ca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:47.705006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa4f89-a267-442d-9d05-a40bc0a8b691 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.710988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e467822-1146-427f-bb35-6954718a6c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:47.711687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e467822-1146-427f-bb35-6954718a6c93 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.712074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e467822-1146-427f-bb35-6954718a6c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.712482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e467822-1146-427f-bb35-6954718a6c93 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.713254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e467822-1146-427f-bb35-6954718a6c93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:13:47.713713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e467822-1146-427f-bb35-6954718a6c93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:47.715541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e467822-1146-427f-bb35-6954718a6c93 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:47.723539 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2909/5804] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:47 2022] PUT /identity/v3/projects/76a47d2411ba41fb8c7a1f163a07ec74/users/e8f30198ba394d77a21157598d5f4d7e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:47.724999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69392917-17dd-4768-9a6a-e8266efcc9ca admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.726990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-69392917-17dd-4768-9a6a-e8266efcc9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:47.740304 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2896/5805] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:13:47.745440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542bd4e-a11c-4750-8f9e-b30b3af88a72 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.745845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542bd4e-a11c-4750-8f9e-b30b3af88a72 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.746221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542bd4e-a11c-4750-8f9e-b30b3af88a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.777835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:47.795036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0542bd4e-a11c-4750-8f9e-b30b3af88a72 None None] MFA Rules not processed for user `e8f30198ba394d77a21157598d5f4d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:47.796681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0542bd4e-a11c-4750-8f9e-b30b3af88a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:47.831638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69392917-17dd-4768-9a6a-e8266efcc9ca admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.839507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:47.840152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:47.840431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:47.840706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] PATH_INFO: `/v3/projects/7b7dbcbca8d6411bb4460a9be184ea3e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:47.842659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b7dbcbca8d6411bb4460a9be184ea3e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:47.844976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:47.918351 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2897/5806] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:47.919495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542bd4e-a11c-4750-8f9e-b30b3af88a72 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.933448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b7dbcbca8d6411bb4460a9be184ea3e', 'request_id': 'req-434303cb-a1cf-4944-876a-3e35a5477ff4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.945831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542bd4e-a11c-4750-8f9e-b30b3af88a72 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:47.946281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b7dbcbca8d6411bb4460a9be184ea3e', 'request_id': 'req-434303cb-a1cf-4944-876a-3e35a5477ff4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.956999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b7dbcbca8d6411bb4460a9be184ea3e', 'request_id': 'req-434303cb-a1cf-4944-876a-3e35a5477ff4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.971095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] The token cache is being invalidate because project 7b7dbcbca8d6411bb4460a9be184ea3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:13:47.971471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.972325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:47.974152 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2910/5807] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:13:47 2022] DELETE /identity/v3/projects/7b7dbcbca8d6411bb4460a9be184ea3e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:47.975008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:47.978548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:48.047659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e5e13-80c4-49da-873d-2fe6ecb33953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:48.048410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5e13-80c4-49da-873d-2fe6ecb33953 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.048827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5e13-80c4-49da-873d-2fe6ecb33953 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.049243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5e13-80c4-49da-873d-2fe6ecb33953 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.050138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e5e13-80c4-49da-873d-2fe6ecb33953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:48.052653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e5e13-80c4-49da-873d-2fe6ecb33953 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:48.095797 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2898/5808] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:13:47 2022] POST /identity/v3/projects => generated 401 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:48.096134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434303cb-a1cf-4944-876a-3e35a5477ff4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:48.099165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b3541-9d2c-427e-93ef-32b75f8403d3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.100099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b3541-9d2c-427e-93ef-32b75f8403d3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.101145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b3541-9d2c-427e-93ef-32b75f8403d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.153645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f33b3541-9d2c-427e-93ef-32b75f8403d3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:48.156776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f33b3541-9d2c-427e-93ef-32b75f8403d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:48.213790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:48.218550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d69532-8a22-41a7-964b-712d89f43165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:48.219130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69532-8a22-41a7-964b-712d89f43165 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.219130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69532-8a22-41a7-964b-712d89f43165 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.219783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69532-8a22-41a7-964b-712d89f43165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.223249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-88d69532-8a22-41a7-964b-712d89f43165 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:48.278430 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2899/5809] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:48.281445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b3541-9d2c-427e-93ef-32b75f8403d3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:48.282011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:48.287280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f33b3541-9d2c-427e-93ef-32b75f8403d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:48.320120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b3541-9d2c-427e-93ef-32b75f8403d3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:48.333332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d05d05-52a0-4590-836e-ea1288a16502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:48.334031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d05d05-52a0-4590-836e-ea1288a16502 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.334310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d05d05-52a0-4590-836e-ea1288a16502 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.334723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d05d05-52a0-4590-836e-ea1288a16502 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.335415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d05d05-52a0-4590-836e-ea1288a16502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:48.337712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d05d05-52a0-4590-836e-ea1288a16502 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:48.375092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d69532-8a22-41a7-964b-712d89f43165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:48.376898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d69532-8a22-41a7-964b-712d89f43165 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:48.378680 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2900/5810] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:13:48 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:48.380046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedaa52-cbe2-4ac2-a07a-f812cf6e5126 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.380399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedaa52-cbe2-4ac2-a07a-f812cf6e5126 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.380739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedaa52-cbe2-4ac2-a07a-f812cf6e5126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.390330 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2911/5811] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:48.390958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d69532-8a22-41a7-964b-712d89f43165 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:48.402524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d69532-8a22-41a7-964b-712d89f43165 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:48.406066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cbedaa52-cbe2-4ac2-a07a-f812cf6e5126 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:48.407608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cbedaa52-cbe2-4ac2-a07a-f812cf6e5126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:48.409377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb7b5f-e54c-4715-949e-c2c778ea674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:48.409377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb7b5f-e54c-4715-949e-c2c778ea674c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.409681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb7b5f-e54c-4715-949e-c2c778ea674c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.409931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb7b5f-e54c-4715-949e-c2c778ea674c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.410951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb7b5f-e54c-4715-949e-c2c778ea674c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:48.412915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb7b5f-e54c-4715-949e-c2c778ea674c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:48.416954 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2901/5812] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:48.420789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedaa52-cbe2-4ac2-a07a-f812cf6e5126 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:48.426096 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2912/5813] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:48.430604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb7b5f-e54c-4715-949e-c2c778ea674c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:48.435291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedaa52-cbe2-4ac2-a07a-f812cf6e5126 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:48.441343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2db5d-de98-4915-9785-d5ec1fbb83f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:48.442104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2db5d-de98-4915-9785-d5ec1fbb83f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.442440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2db5d-de98-4915-9785-d5ec1fbb83f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.443219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2db5d-de98-4915-9785-d5ec1fbb83f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.443466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2db5d-de98-4915-9785-d5ec1fbb83f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:13:48.443829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2db5d-de98-4915-9785-d5ec1fbb83f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:48.444869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb7b5f-e54c-4715-949e-c2c778ea674c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:48.445711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2db5d-de98-4915-9785-d5ec1fbb83f4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:48.450306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460dbf3-2db5-48d3-86cf-5445c3222b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:48.450885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460dbf3-2db5-48d3-86cf-5445c3222b23 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.451272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460dbf3-2db5-48d3-86cf-5445c3222b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.451875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460dbf3-2db5-48d3-86cf-5445c3222b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.452980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460dbf3-2db5-48d3-86cf-5445c3222b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:48.454933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460dbf3-2db5-48d3-86cf-5445c3222b23 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:48.473336 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2902/5814] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:13:48.473635 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2913/5815] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:48.475203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2db5d-de98-4915-9785-d5ec1fbb83f4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:48.475371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460dbf3-2db5-48d3-86cf-5445c3222b23 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:48.492712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2db5d-de98-4915-9785-d5ec1fbb83f4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:48.501009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460dbf3-2db5-48d3-86cf-5445c3222b23 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:48.506339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a5404-857b-4149-b616-2f640698a3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:48.506339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a5404-857b-4149-b616-2f640698a3ac admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.506339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a5404-857b-4149-b616-2f640698a3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.506339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a5404-857b-4149-b616-2f640698a3ac admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.506948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a5404-857b-4149-b616-2f640698a3ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:48.513772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecce91-c66d-420c-9f68-e5c2299f6a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:48.514569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecce91-c66d-420c-9f68-e5c2299f6a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.514849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecce91-c66d-420c-9f68-e5c2299f6a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.515117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecce91-c66d-420c-9f68-e5c2299f6a7d admin admin] PATH_INFO: `/v3/projects/8ae59b3b399e40bd8b5e1011ed9d353f/users/937a017c09c146a697b48e8e575306b8/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.516551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a5404-857b-4149-b616-2f640698a3ac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:48.537505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecce91-c66d-420c-9f68-e5c2299f6a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae59b3b399e40bd8b5e1011ed9d353f, user_id=937a017c09c146a697b48e8e575306b8, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:48.541369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecce91-c66d-420c-9f68-e5c2299f6a7d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:48.563503 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2914/5816] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:48 2022] PUT /identity/v3/projects/8ae59b3b399e40bd8b5e1011ed9d353f/users/937a017c09c146a697b48e8e575306b8/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:48.573243 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2903/5817] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:13:48 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:48.573481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecce91-c66d-420c-9f68-e5c2299f6a7d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:48.575025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1c445-725c-4a85-85a2-640e88d0c253 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.575320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1c445-725c-4a85-85a2-640e88d0c253 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.575592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1c445-725c-4a85-85a2-640e88d0c253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.642482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-47f1c445-725c-4a85-85a2-640e88d0c253 None None] MFA Rules not processed for user `937a017c09c146a697b48e8e575306b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:48.644951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-47f1c445-725c-4a85-85a2-640e88d0c253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:48.774301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecce91-c66d-420c-9f68-e5c2299f6a7d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:48.796281 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2904/5818] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:48.809288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1c445-725c-4a85-85a2-640e88d0c253 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:48.972489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3090614-b0f5-4fa5-80a0-73a531fa94d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:48.973733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3090614-b0f5-4fa5-80a0-73a531fa94d7 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:48.976399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3090614-b0f5-4fa5-80a0-73a531fa94d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:48.977447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3090614-b0f5-4fa5-80a0-73a531fa94d7 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:48.979072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3090614-b0f5-4fa5-80a0-73a531fa94d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:48.987744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3090614-b0f5-4fa5-80a0-73a531fa94d7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:49.009977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1c445-725c-4a85-85a2-640e88d0c253 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:49.015155 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2915/5819] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:13:48 2022] POST /identity/v3/users => generated 287 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:49.018407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:49.019389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:49.019617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3090614-b0f5-4fa5-80a0-73a531fa94d7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:49.022477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:49.022477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] PATH_INFO: `/v3/users/937a017c09c146a697b48e8e575306b8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:49.025143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=937a017c09c146a697b48e8e575306b8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:49.032006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:49.043875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3090614-b0f5-4fa5-80a0-73a531fa94d7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:49.060493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d230f8-9986-49a8-a7e0-7ac5c3f20616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:49.061613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d230f8-9986-49a8-a7e0-7ac5c3f20616 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:49.062127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d230f8-9986-49a8-a7e0-7ac5c3f20616 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:49.062493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d230f8-9986-49a8-a7e0-7ac5c3f20616 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:49.063240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d230f8-9986-49a8-a7e0-7ac5c3f20616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:49.065518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d230f8-9986-49a8-a7e0-7ac5c3f20616 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:49.066074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:49.110452 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2916/5820] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:49.117484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d230f8-9986-49a8-a7e0-7ac5c3f20616 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:49.135480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937a017c09c146a697b48e8e575306b8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:49.139600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d230f8-9986-49a8-a7e0-7ac5c3f20616 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:49.147366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937a017c09c146a697b48e8e575306b8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:49.157805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56a6a2-9a57-428a-9905-cf83cbb7a2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:49.158834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56a6a2-9a57-428a-9905-cf83cbb7a2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:49.159380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56a6a2-9a57-428a-9905-cf83cbb7a2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:49.159967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56a6a2-9a57-428a-9905-cf83cbb7a2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:49.160625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56a6a2-9a57-428a-9905-cf83cbb7a2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:49.162965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56a6a2-9a57-428a-9905-cf83cbb7a2d5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:49.165325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937a017c09c146a697b48e8e575306b8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:49.167265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:49.177314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937a017c09c146a697b48e8e575306b8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:49.178324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6f0fb5a5-b1fe-4811-93ef-577009fe4b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937a017c09c146a697b48e8e575306b8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:49.191302 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2905/5821] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:13:48 2022] DELETE /identity/v3/users/937a017c09c146a697b48e8e575306b8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:13:49.193927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1d870-6c52-478a-9dbc-6e0e746d557a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:49.194453 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2917/5822] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:49.197418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1d870-6c52-478a-9dbc-6e0e746d557a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:49.197832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1d870-6c52-478a-9dbc-6e0e746d557a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:49.201980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56a6a2-9a57-428a-9905-cf83cbb7a2d5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:49.204134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5d56a6a2-9a57-428a-9905-cf83cbb7a2d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:49.221541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b9e1d870-6c52-478a-9dbc-6e0e746d557a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:49.222284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56a6a2-9a57-428a-9905-cf83cbb7a2d5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:49.223031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b9e1d870-6c52-478a-9dbc-6e0e746d557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:49.388319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e80a7-20b8-4d67-847f-6e226e4a435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:49.390161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e80a7-20b8-4d67-847f-6e226e4a435d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:49.390615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e80a7-20b8-4d67-847f-6e226e4a435d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:49.391191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e80a7-20b8-4d67-847f-6e226e4a435d admin admin] PATH_INFO: `/v3/projects/19bd3439a5804629aaa47e7322edf039/users/54379c322e0949f19310cdb5b4c97674/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:49.401083 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2906/5823] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:49.444784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1d870-6c52-478a-9dbc-6e0e746d557a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:49.447925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b9e1d870-6c52-478a-9dbc-6e0e746d557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:49.450834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e80a7-20b8-4d67-847f-6e226e4a435d admin admin] RBAC: Authorizing `identity:create_grant(project_id=19bd3439a5804629aaa47e7322edf039, user_id=54379c322e0949f19310cdb5b4c97674, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:49.461549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e80a7-20b8-4d67-847f-6e226e4a435d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:49.495438 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2918/5824] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:49 2022] PUT /identity/v3/projects/19bd3439a5804629aaa47e7322edf039/users/54379c322e0949f19310cdb5b4c97674/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:49.498022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0de73-8273-4ab9-8397-de87778e5495 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:49.499114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0de73-8273-4ab9-8397-de87778e5495 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:49.500980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0de73-8273-4ab9-8397-de87778e5495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:49.545908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-89e0de73-8273-4ab9-8397-de87778e5495 None None] MFA Rules not processed for user `54379c322e0949f19310cdb5b4c97674`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:49.547390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-89e0de73-8273-4ab9-8397-de87778e5495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:49.560449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1d870-6c52-478a-9dbc-6e0e746d557a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:49.668455 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2919/5825] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:49.675351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0de73-8273-4ab9-8397-de87778e5495 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:49.704203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:49.716045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcc263-cb3b-44bc-afe7-416840334fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:49.718410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcc263-cb3b-44bc-afe7-416840334fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:49.718830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcc263-cb3b-44bc-afe7-416840334fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:49.721166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcc263-cb3b-44bc-afe7-416840334fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:49.801569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0de73-8273-4ab9-8397-de87778e5495 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:49.808995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd541-9982-4f0a-963b-52249337e2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:49.809797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd541-9982-4f0a-963b-52249337e2c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:49.810197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd541-9982-4f0a-963b-52249337e2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:49.811422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd541-9982-4f0a-963b-52249337e2c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:49.812329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fd541-9982-4f0a-963b-52249337e2c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:49.816254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fd541-9982-4f0a-963b-52249337e2c9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:49.836377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bcc263-cb3b-44bc-afe7-416840334fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:49.838862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:49.840723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bcc263-cb3b-44bc-afe7-416840334fa2 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:49.863950 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2920/5826] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:13:49 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:13:49.868609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd541-9982-4f0a-963b-52249337e2c9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:49.873650 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2907/5827] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:13:49.995401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd541-9982-4f0a-963b-52249337e2c9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:50.003731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0ded0-0383-4ce9-b9b3-bd1cb0b72fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:50.004650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0ded0-0383-4ce9-b9b3-bd1cb0b72fae admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:50.005023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0ded0-0383-4ce9-b9b3-bd1cb0b72fae admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:50.005480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0ded0-0383-4ce9-b9b3-bd1cb0b72fae admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:50.006214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0ded0-0383-4ce9-b9b3-bd1cb0b72fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:50.008383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0ded0-0383-4ce9-b9b3-bd1cb0b72fae admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:50.067404 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2921/5828] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:13:49 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:50.068275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcc263-cb3b-44bc-afe7-416840334fa2 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:50.083790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcc263-cb3b-44bc-afe7-416840334fa2 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:50.103144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e144b5e-7211-4e3f-b3c1-83f34aa1126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:50.103680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e144b5e-7211-4e3f-b3c1-83f34aa1126f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:50.103960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e144b5e-7211-4e3f-b3c1-83f34aa1126f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:50.104461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e144b5e-7211-4e3f-b3c1-83f34aa1126f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:50.105179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e144b5e-7211-4e3f-b3c1-83f34aa1126f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:50.108540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e144b5e-7211-4e3f-b3c1-83f34aa1126f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:50.131744 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2908/5829] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:50.135421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0ded0-0383-4ce9-b9b3-bd1cb0b72fae admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:50.153189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0ded0-0383-4ce9-b9b3-bd1cb0b72fae admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:50.162852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84f713-5067-471e-93a3-832270d89ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:50.165103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84f713-5067-471e-93a3-832270d89ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:50.165784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84f713-5067-471e-93a3-832270d89ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:50.166251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84f713-5067-471e-93a3-832270d89ac4 admin admin] PATH_INFO: `/v3/projects/e7addc44b5d74215917b9dfd170f56e1/users/df1bae284d874d5e8180bf206c536dbe/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:50.194748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84f713-5067-471e-93a3-832270d89ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7addc44b5d74215917b9dfd170f56e1, user_id=df1bae284d874d5e8180bf206c536dbe, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:50.197744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84f713-5067-471e-93a3-832270d89ac4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:50.231671 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2922/5830] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:50 2022] PUT /identity/v3/projects/e7addc44b5d74215917b9dfd170f56e1/users/df1bae284d874d5e8180bf206c536dbe/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:50.233058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e144b5e-7211-4e3f-b3c1-83f34aa1126f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:50.248042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e144b5e-7211-4e3f-b3c1-83f34aa1126f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:50.372760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aedb4d-60dc-48f6-811b-ff7b06c0eef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:50.374239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aedb4d-60dc-48f6-811b-ff7b06c0eef6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:50.374239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aedb4d-60dc-48f6-811b-ff7b06c0eef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:50.375131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aedb4d-60dc-48f6-811b-ff7b06c0eef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:50.378357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aedb4d-60dc-48f6-811b-ff7b06c0eef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:50.381361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aedb4d-60dc-48f6-811b-ff7b06c0eef6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:50.419048 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2909/5831] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:13:50.423207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84f713-5067-471e-93a3-832270d89ac4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:50.438684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84f713-5067-471e-93a3-832270d89ac4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:50.453649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6ed28-dd6f-4b85-93c5-ac870b6f9656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:50.454314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ed28-dd6f-4b85-93c5-ac870b6f9656 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:50.454585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ed28-dd6f-4b85-93c5-ac870b6f9656 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:50.454851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ed28-dd6f-4b85-93c5-ac870b6f9656 admin admin] PATH_INFO: `/v3/projects/e7addc44b5d74215917b9dfd170f56e1/users/df1bae284d874d5e8180bf206c536dbe/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:50.464344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6ed28-dd6f-4b85-93c5-ac870b6f9656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7addc44b5d74215917b9dfd170f56e1, user_id=df1bae284d874d5e8180bf206c536dbe, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:50.467810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6ed28-dd6f-4b85-93c5-ac870b6f9656 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:50.491115 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2923/5832] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:50 2022] PUT /identity/v3/projects/e7addc44b5d74215917b9dfd170f56e1/users/df1bae284d874d5e8180bf206c536dbe/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:50.492903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af273dd-ea7b-448f-a843-92a4c457a734 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:50.493252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af273dd-ea7b-448f-a843-92a4c457a734 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:50.493950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af273dd-ea7b-448f-a843-92a4c457a734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:50.531290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5af273dd-ea7b-448f-a843-92a4c457a734 None None] MFA Rules not processed for user `df1bae284d874d5e8180bf206c536dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:50.532730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5af273dd-ea7b-448f-a843-92a4c457a734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:50.660953 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2910/5833] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:50.676122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6ed28-dd6f-4b85-93c5-ac870b6f9656 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:50.678634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f4e6ed28-dd6f-4b85-93c5-ac870b6f9656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:50.787263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6ed28-dd6f-4b85-93c5-ac870b6f9656 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:50.792477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c60c44-95d6-447f-8d2e-bcc229ff4757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:50.793432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c60c44-95d6-447f-8d2e-bcc229ff4757 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:50.793867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c60c44-95d6-447f-8d2e-bcc229ff4757 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:50.794312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c60c44-95d6-447f-8d2e-bcc229ff4757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:50.809169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c60c44-95d6-447f-8d2e-bcc229ff4757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:50.811113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c60c44-95d6-447f-8d2e-bcc229ff4757 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:50.828652 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2924/5834] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:51.508882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af273dd-ea7b-448f-a843-92a4c457a734 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:51.527617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:51.640249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af273dd-ea7b-448f-a843-92a4c457a734 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:51.645856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:51.647459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:51.648319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:51.648749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] PATH_INFO: `/v3/users/54379c322e0949f19310cdb5b4c97674` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:51.650242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54379c322e0949f19310cdb5b4c97674)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:51.652474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:51.741985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c60c44-95d6-447f-8d2e-bcc229ff4757 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:51.744098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-95c60c44-95d6-447f-8d2e-bcc229ff4757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:51.745741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54379c322e0949f19310cdb5b4c97674'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:51.763259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54379c322e0949f19310cdb5b4c97674'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:51.782288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54379c322e0949f19310cdb5b4c97674'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:51.797437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54379c322e0949f19310cdb5b4c97674'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:51.799013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7e25d838-5848-4b46-aad8-76a1395c1191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54379c322e0949f19310cdb5b4c97674'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:51.811614 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2911/5835] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:13:51 2022] DELETE /identity/v3/users/54379c322e0949f19310cdb5b4c97674 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:51.819050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d765cc-e6dd-4100-94a4-ccaa4055872b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:51.821123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d765cc-e6dd-4100-94a4-ccaa4055872b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:51.821647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d765cc-e6dd-4100-94a4-ccaa4055872b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:51.883695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-01d765cc-e6dd-4100-94a4-ccaa4055872b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:51.885951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-01d765cc-e6dd-4100-94a4-ccaa4055872b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:51.916308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:51.979797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c60c44-95d6-447f-8d2e-bcc229ff4757 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:52.044735 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2912/5836] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:52.159471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811660d-f869-435d-b1b9-83cbeb01ed8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:52.160760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811660d-f869-435d-b1b9-83cbeb01ed8e service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:52.164324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811660d-f869-435d-b1b9-83cbeb01ed8e service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:52.164944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811660d-f869-435d-b1b9-83cbeb01ed8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:52.171474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a811660d-f869-435d-b1b9-83cbeb01ed8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:52.190269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d765cc-e6dd-4100-94a4-ccaa4055872b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:52.191916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-01d765cc-e6dd-4100-94a4-ccaa4055872b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:52.221285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811660d-f869-435d-b1b9-83cbeb01ed8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:52.239364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811660d-f869-435d-b1b9-83cbeb01ed8e service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:52.260223 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2925/5837] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:13:52.289314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:52.350652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d765cc-e6dd-4100-94a4-ccaa4055872b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:52.355192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c68169-4ff7-424d-91a3-07071c0fed0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:52.355984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c68169-4ff7-424d-91a3-07071c0fed0e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:52.356262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c68169-4ff7-424d-91a3-07071c0fed0e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:52.356533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c68169-4ff7-424d-91a3-07071c0fed0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:52.369531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c68169-4ff7-424d-91a3-07071c0fed0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:52.371066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c68169-4ff7-424d-91a3-07071c0fed0e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:52.384404 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2913/5838] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:52.399111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811660d-f869-435d-b1b9-83cbeb01ed8e service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:52.403770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a811660d-f869-435d-b1b9-83cbeb01ed8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:52.440403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811660d-f869-435d-b1b9-83cbeb01ed8e service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:52.445203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8f60a-2fdb-41f4-ae6d-8b42fdddadb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:52.445817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8f60a-2fdb-41f4-ae6d-8b42fdddadb3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:52.446142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8f60a-2fdb-41f4-ae6d-8b42fdddadb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:52.446458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8f60a-2fdb-41f4-ae6d-8b42fdddadb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:52.448303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-95c8f60a-2fdb-41f4-ae6d-8b42fdddadb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:52.466491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8f60a-2fdb-41f4-ae6d-8b42fdddadb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:52.468141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8f60a-2fdb-41f4-ae6d-8b42fdddadb3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:52.487958 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2926/5839] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:52.503431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c68169-4ff7-424d-91a3-07071c0fed0e service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:52.504919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-29c68169-4ff7-424d-91a3-07071c0fed0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:52.532921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c68169-4ff7-424d-91a3-07071c0fed0e service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:52.548497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:52.549228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:52.549503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:52.549785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] PATH_INFO: `/v3/projects/8ae59b3b399e40bd8b5e1011ed9d353f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:52.552718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae59b3b399e40bd8b5e1011ed9d353f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:52.554776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:52.622375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae59b3b399e40bd8b5e1011ed9d353f', 'request_id': 'req-40f5d08a-a0d8-46ab-a202-2df996d99d8d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:52.632551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae59b3b399e40bd8b5e1011ed9d353f', 'request_id': 'req-40f5d08a-a0d8-46ab-a202-2df996d99d8d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:52.642676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae59b3b399e40bd8b5e1011ed9d353f', 'request_id': 'req-40f5d08a-a0d8-46ab-a202-2df996d99d8d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:52.666810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] The token cache is being invalidate because project 8ae59b3b399e40bd8b5e1011ed9d353f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:13:52.667126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:52.667903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-40f5d08a-a0d8-46ab-a202-2df996d99d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:52.675179 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2914/5840] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:13:52 2022] DELETE /identity/v3/projects/8ae59b3b399e40bd8b5e1011ed9d353f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:52.677394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417ed2b-b4c5-47d9-b4a0-43b52a19e073 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:52.678230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417ed2b-b4c5-47d9-b4a0-43b52a19e073 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:52.678230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417ed2b-b4c5-47d9-b4a0-43b52a19e073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:52.718497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e417ed2b-b4c5-47d9-b4a0-43b52a19e073 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:52.720278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e417ed2b-b4c5-47d9-b4a0-43b52a19e073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:52.853272 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2927/5841] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:52.855854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180187e-9d7e-4f24-983c-6facca130149 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:52.856346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180187e-9d7e-4f24-983c-6facca130149 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:52.856978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180187e-9d7e-4f24-983c-6facca130149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:52.890744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1180187e-9d7e-4f24-983c-6facca130149 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:52.893249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1180187e-9d7e-4f24-983c-6facca130149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:52.908524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417ed2b-b4c5-47d9-b4a0-43b52a19e073 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:52.908784 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2915/5842] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:52.922167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417ed2b-b4c5-47d9-b4a0-43b52a19e073 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:52.931736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380323e0-e7f8-496d-a392-d809df5b56fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:52.932737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380323e0-e7f8-496d-a392-d809df5b56fb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:52.933258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380323e0-e7f8-496d-a392-d809df5b56fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:52.933911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380323e0-e7f8-496d-a392-d809df5b56fb admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:52.939810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380323e0-e7f8-496d-a392-d809df5b56fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:13:52.940225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380323e0-e7f8-496d-a392-d809df5b56fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:52.942741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380323e0-e7f8-496d-a392-d809df5b56fb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:52.964049 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2928/5843] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:13:52.973110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180187e-9d7e-4f24-983c-6facca130149 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:53.022926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180187e-9d7e-4f24-983c-6facca130149 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:53.033988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f8d93-4b52-4fac-be6e-e57debb3dd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:53.038731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f8d93-4b52-4fac-be6e-e57debb3dd53 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:53.039150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f8d93-4b52-4fac-be6e-e57debb3dd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:53.039786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f8d93-4b52-4fac-be6e-e57debb3dd53 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:53.058830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f8d93-4b52-4fac-be6e-e57debb3dd53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:53.065290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f8d93-4b52-4fac-be6e-e57debb3dd53 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:53.149101 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2916/5844] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:13:52 2022] POST /identity/v3/projects => generated 363 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:53.150852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380323e0-e7f8-496d-a392-d809df5b56fb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:53.195597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380323e0-e7f8-496d-a392-d809df5b56fb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:53.246505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:53.336465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02202b42-0d71-4de9-a594-73001e174605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:53.337415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02202b42-0d71-4de9-a594-73001e174605 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:53.339099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02202b42-0d71-4de9-a594-73001e174605 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:53.340395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02202b42-0d71-4de9-a594-73001e174605 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:53.344442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02202b42-0d71-4de9-a594-73001e174605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:53.352657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02202b42-0d71-4de9-a594-73001e174605 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:53.402993 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2929/5845] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:13:53 2022] POST /identity/v3/users => generated 282 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:13:53.404919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f8d93-4b52-4fac-be6e-e57debb3dd53 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:53.443510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f8d93-4b52-4fac-be6e-e57debb3dd53 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:53.456293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55997544-9a2f-4516-bf8f-9ae18340e428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:53.457129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55997544-9a2f-4516-bf8f-9ae18340e428 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:53.457511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55997544-9a2f-4516-bf8f-9ae18340e428 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:53.457885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55997544-9a2f-4516-bf8f-9ae18340e428 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:53.458641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55997544-9a2f-4516-bf8f-9ae18340e428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:53.460524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55997544-9a2f-4516-bf8f-9ae18340e428 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:53.487402 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2917/5846] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:53.491014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02202b42-0d71-4de9-a594-73001e174605 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:53.506904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02202b42-0d71-4de9-a594-73001e174605 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:53.511951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6047b-26ef-4df0-9c4b-91099d651418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:53.513168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6047b-26ef-4df0-9c4b-91099d651418 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:53.513457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6047b-26ef-4df0-9c4b-91099d651418 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:53.513763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6047b-26ef-4df0-9c4b-91099d651418 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:53.514685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6047b-26ef-4df0-9c4b-91099d651418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:53.516780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6047b-26ef-4df0-9c4b-91099d651418 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:53.534120 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2930/5847] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:53.537999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55997544-9a2f-4516-bf8f-9ae18340e428 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:53.560941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55997544-9a2f-4516-bf8f-9ae18340e428 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:53.567193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d6234-71c5-466a-a6e3-98081dc7d5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:53.568299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d6234-71c5-466a-a6e3-98081dc7d5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:53.568689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d6234-71c5-466a-a6e3-98081dc7d5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:53.569116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d6234-71c5-466a-a6e3-98081dc7d5f7 admin admin] PATH_INFO: `/v3/projects/4b7eb1d81f374478b3db39266029033b/users/76fe12490e6847a795b9871e47182d08/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:53.594099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d6234-71c5-466a-a6e3-98081dc7d5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7eb1d81f374478b3db39266029033b, user_id=76fe12490e6847a795b9871e47182d08, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:53.599005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d6234-71c5-466a-a6e3-98081dc7d5f7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:53.630745 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2918/5848] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:53 2022] PUT /identity/v3/projects/4b7eb1d81f374478b3db39266029033b/users/76fe12490e6847a795b9871e47182d08/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:53.632763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ca60d-315b-4144-8caf-8cc14b0ba41b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:53.632998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ca60d-315b-4144-8caf-8cc14b0ba41b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:53.633302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ca60d-315b-4144-8caf-8cc14b0ba41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:53.671345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-213ca60d-315b-4144-8caf-8cc14b0ba41b None None] MFA Rules not processed for user `76fe12490e6847a795b9871e47182d08`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:53.672901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-213ca60d-315b-4144-8caf-8cc14b0ba41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:53.754397 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2931/5849] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:53.769007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d6234-71c5-466a-a6e3-98081dc7d5f7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:53.770815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-763d6234-71c5-466a-a6e3-98081dc7d5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:53.872491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:53.904149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d6234-71c5-466a-a6e3-98081dc7d5f7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:53.912622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636801db-2b00-4d03-9bd3-441a689f9297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:53.913523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636801db-2b00-4d03-9bd3-441a689f9297 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:53.913783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636801db-2b00-4d03-9bd3-441a689f9297 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:53.914127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636801db-2b00-4d03-9bd3-441a689f9297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:53.923714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636801db-2b00-4d03-9bd3-441a689f9297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:53.925518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636801db-2b00-4d03-9bd3-441a689f9297 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:53.936991 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2919/5850] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:53.950119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602108d-8b58-4356-9397-71c10ceddb39 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:53.952452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602108d-8b58-4356-9397-71c10ceddb39 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:53.952797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602108d-8b58-4356-9397-71c10ceddb39 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:53.956388 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2932/5851] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 21:13:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:13:53.962621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636801db-2b00-4d03-9bd3-441a689f9297 service nova] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 21:13:53.978553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636801db-2b00-4d03-9bd3-441a689f9297 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:53.989154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636801db-2b00-4d03-9bd3-441a689f9297 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:53.994157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794a14b-955c-40f4-bdab-632937a44875 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fe12490e6847a795b9871e47182d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7eb1d81f374478b3db39266029033b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:53.995008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794a14b-955c-40f4-bdab-632937a44875 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:53.996773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794a14b-955c-40f4-bdab-632937a44875 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:53.997144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794a14b-955c-40f4-bdab-632937a44875 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] PATH_INFO: `/v3/projects/4b7eb1d81f374478b3db39266029033b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:53.998469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794a14b-955c-40f4-bdab-632937a44875 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: Authorizing `identity:get_project(project_id=4b7eb1d81f374478b3db39266029033b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:54.000839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794a14b-955c-40f4-bdab-632937a44875 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:54.006698 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2920/5852] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 21:13:53 2022] GET /identity/v3/projects/4b7eb1d81f374478b3db39266029033b => generated 363 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:13:54.050311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496f72b-70ca-4d28-929e-5716b9891800 None None] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:54.051194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496f72b-70ca-4d28-929e-5716b9891800 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:54.051666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496f72b-70ca-4d28-929e-5716b9891800 None None] PATH_INFO: `/` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:54.054780 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2933/5853] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 21:13:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:13:54.065632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794a14b-955c-40f4-bdab-632937a44875 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 21:13:54.068257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8794a14b-955c-40f4-bdab-632937a44875 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:54.110777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794a14b-955c-40f4-bdab-632937a44875 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:54.123775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794a14b-955c-40f4-bdab-632937a44875 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:54.129404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603a6ec-4154-4604-b7e3-37f686b0a6e5 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fe12490e6847a795b9871e47182d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7eb1d81f374478b3db39266029033b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:54.131089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603a6ec-4154-4604-b7e3-37f686b0a6e5 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:54.131338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603a6ec-4154-4604-b7e3-37f686b0a6e5 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:54.131685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603a6ec-4154-4604-b7e3-37f686b0a6e5 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] PATH_INFO: `/v3/projects/4b7eb1d81f374478b3db39266029033b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:54.133131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603a6ec-4154-4604-b7e3-37f686b0a6e5 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: Authorizing `identity:get_project(project_id=4b7eb1d81f374478b3db39266029033b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:54.135845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603a6ec-4154-4604-b7e3-37f686b0a6e5 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:54.149096 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2921/5854] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 21:13:54 2022] GET /identity/v3/projects/4b7eb1d81f374478b3db39266029033b => generated 363 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:13:54.158456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496f72b-70ca-4d28-929e-5716b9891800 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:54.162355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1496f72b-70ca-4d28-929e-5716b9891800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:54.212670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f07d9c-479f-4d99-a4ec-1c7a6107b88a None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:54.212944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f07d9c-479f-4d99-a4ec-1c7a6107b88a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:54.213245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f07d9c-479f-4d99-a4ec-1c7a6107b88a None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:54.217395 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2922/5855] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 21:13:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:13:54.221514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f07d9c-479f-4d99-a4ec-1c7a6107b88a None None] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 21:13:54.269689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f07d9c-479f-4d99-a4ec-1c7a6107b88a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:54.297597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f07d9c-479f-4d99-a4ec-1c7a6107b88a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:54.311884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f51bf-6870-43ca-9a3d-8ccdff329139 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fe12490e6847a795b9871e47182d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7eb1d81f374478b3db39266029033b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:54.313078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f51bf-6870-43ca-9a3d-8ccdff329139 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:54.313078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f51bf-6870-43ca-9a3d-8ccdff329139 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:54.313500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f51bf-6870-43ca-9a3d-8ccdff329139 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] PATH_INFO: `/v3/projects/4b7eb1d81f374478b3db39266029033b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:54.314930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f51bf-6870-43ca-9a3d-8ccdff329139 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: Authorizing `identity:get_project(project_id=4b7eb1d81f374478b3db39266029033b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:54.316682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f51bf-6870-43ca-9a3d-8ccdff329139 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:54.321577 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2923/5856] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 21:13:54 2022] GET /identity/v3/projects/4b7eb1d81f374478b3db39266029033b => generated 363 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:13:54.366095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496f72b-70ca-4d28-929e-5716b9891800 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:54.370886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b11182-31cc-4665-9c69-9936d18d6d58 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:54.371166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b11182-31cc-4665-9c69-9936d18d6d58 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:54.371436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b11182-31cc-4665-9c69-9936d18d6d58 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:54.373443 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2924/5857] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 21:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:13:54.375487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:54.376599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:54.377214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:54.377755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] PATH_INFO: `/v3/projects/19bd3439a5804629aaa47e7322edf039` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:54.378773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b11182-31cc-4665-9c69-9936d18d6d58 None None] Authenticating service token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 21:13:54.379573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=19bd3439a5804629aaa47e7322edf039)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:54.382510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:54.415404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b11182-31cc-4665-9c69-9936d18d6d58 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:54.448820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b11182-31cc-4665-9c69-9936d18d6d58 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:54.469675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:54.470638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19bd3439a5804629aaa47e7322edf039', 'request_id': 'req-1a49e702-aa4f-4bb4-a12e-a882df817a9f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:54.493994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19bd3439a5804629aaa47e7322edf039', 'request_id': 'req-1a49e702-aa4f-4bb4-a12e-a882df817a9f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:54.508703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19bd3439a5804629aaa47e7322edf039', 'request_id': 'req-1a49e702-aa4f-4bb4-a12e-a882df817a9f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:54.525470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] The token cache is being invalidate because project 19bd3439a5804629aaa47e7322edf039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:13:54.526175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:54.527166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:54.530825 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2934/5858] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:13:54 2022] DELETE /identity/v3/projects/19bd3439a5804629aaa47e7322edf039 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:54.534754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:54.537616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:54.582064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa60219-8df1-4e1b-bc7b-2caf2f29d8a4 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fe12490e6847a795b9871e47182d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7eb1d81f374478b3db39266029033b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:54.582869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa60219-8df1-4e1b-bc7b-2caf2f29d8a4 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:54.583150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa60219-8df1-4e1b-bc7b-2caf2f29d8a4 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:54.583419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa60219-8df1-4e1b-bc7b-2caf2f29d8a4 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] PATH_INFO: `/v3/projects/4b7eb1d81f374478b3db39266029033b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:54.584346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa60219-8df1-4e1b-bc7b-2caf2f29d8a4 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: Authorizing `identity:get_project(project_id=4b7eb1d81f374478b3db39266029033b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:54.586859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa60219-8df1-4e1b-bc7b-2caf2f29d8a4 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:54.601270 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2925/5859] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 21:13:54 2022] GET /identity/v3/projects/4b7eb1d81f374478b3db39266029033b => generated 363 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:13:54.690972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49e702-aa4f-4bb4-a12e-a882df817a9f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:54.697303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:54.698995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:54.699442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:54.699831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] PATH_INFO: `/v3/users/df1bae284d874d5e8180bf206c536dbe` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:54.702352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df1bae284d874d5e8180bf206c536dbe)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:54.705233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:54.805402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df1bae284d874d5e8180bf206c536dbe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:54.821079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df1bae284d874d5e8180bf206c536dbe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:54.834065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df1bae284d874d5e8180bf206c536dbe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:54.844268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df1bae284d874d5e8180bf206c536dbe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:54.845442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df1bae284d874d5e8180bf206c536dbe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:54.853060 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2935/5860] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:13:54 2022] DELETE /identity/v3/users/df1bae284d874d5e8180bf206c536dbe => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:55.114501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c592dcc-536b-44c8-8822-af75c5b260c5 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:55.114802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c592dcc-536b-44c8-8822-af75c5b260c5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:55.115047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c592dcc-536b-44c8-8822-af75c5b260c5 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:55.116528 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2926/5861] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 21:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:13:55.124149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 21:13:55.126605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:55.216717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:55.267170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:55.279363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:55.610490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29f228-728d-4582-906b-0679620296a8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:55.616684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a5082-3ac5-4f56-9ce1-c75d4471308c tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fe12490e6847a795b9871e47182d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7eb1d81f374478b3db39266029033b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:55.617275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a5082-3ac5-4f56-9ce1-c75d4471308c tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:55.617632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a5082-3ac5-4f56-9ce1-c75d4471308c tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:55.618016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a5082-3ac5-4f56-9ce1-c75d4471308c tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] PATH_INFO: `/v3/projects/4b7eb1d81f374478b3db39266029033b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:55.619730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a5082-3ac5-4f56-9ce1-c75d4471308c tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: Authorizing `identity:get_project(project_id=4b7eb1d81f374478b3db39266029033b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:55.621833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a5082-3ac5-4f56-9ce1-c75d4471308c tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:55.624984 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2936/5862] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 21:13:55 2022] GET /identity/v3/projects/4b7eb1d81f374478b3db39266029033b => generated 363 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:13:55.661335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c592dcc-536b-44c8-8822-af75c5b260c5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:55.662934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5c592dcc-536b-44c8-8822-af75c5b260c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:55.755285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c592dcc-536b-44c8-8822-af75c5b260c5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:55.760833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:55.761586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:55.762155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:55.762283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] PATH_INFO: `/v3/users/76fe12490e6847a795b9871e47182d08` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:55.763437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76fe12490e6847a795b9871e47182d08)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:55.766204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:55.830598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76fe12490e6847a795b9871e47182d08'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:55.836890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:55.842232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76fe12490e6847a795b9871e47182d08'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:55.864921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76fe12490e6847a795b9871e47182d08'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:55.875240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76fe12490e6847a795b9871e47182d08'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:55.876150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76fe12490e6847a795b9871e47182d08'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:55.884095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2927/5863] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:13:55 2022] DELETE /identity/v3/users/76fe12490e6847a795b9871e47182d08 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:55.889275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3862-9731-4a8e-9196-dc8c0de73a28 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:55.889665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3862-9731-4a8e-9196-dc8c0de73a28 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:55.889940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3862-9731-4a8e-9196-dc8c0de73a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:55.925635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-92da3862-9731-4a8e-9196-dc8c0de73a28 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:55.926962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-92da3862-9731-4a8e-9196-dc8c0de73a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:56.008247 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2937/5864] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:56.027992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:56.029918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:56.142720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708a669-5bd4-4c10-a0f5-09c0858b7c07 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:56.151847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ec615-e9a1-4685-ac2a-947cfdd0253e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:56.152735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ec615-e9a1-4685-ac2a-947cfdd0253e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:56.153020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ec615-e9a1-4685-ac2a-947cfdd0253e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:56.154473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ec615-e9a1-4685-ac2a-947cfdd0253e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:56.165989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ec615-e9a1-4685-ac2a-947cfdd0253e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:56.167859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ec615-e9a1-4685-ac2a-947cfdd0253e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:56.181884 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2928/5865] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:13:56.953875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da3862-9731-4a8e-9196-dc8c0de73a28 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:56.955831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-92da3862-9731-4a8e-9196-dc8c0de73a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:56.968153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da3862-9731-4a8e-9196-dc8c0de73a28 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:56.973822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:56.974457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:56.974727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:56.975177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] PATH_INFO: `/v3/projects/e7addc44b5d74215917b9dfd170f56e1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:56.976474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7addc44b5d74215917b9dfd170f56e1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:56.979638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:57.036472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7addc44b5d74215917b9dfd170f56e1', 'request_id': 'req-3845bb1e-ff22-4548-8707-39c2b2267658'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:57.046564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7addc44b5d74215917b9dfd170f56e1', 'request_id': 'req-3845bb1e-ff22-4548-8707-39c2b2267658'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:57.055981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7addc44b5d74215917b9dfd170f56e1', 'request_id': 'req-3845bb1e-ff22-4548-8707-39c2b2267658'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:57.112821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] The token cache is being invalidate because project e7addc44b5d74215917b9dfd170f56e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:13:57.113141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:57.114029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3845bb1e-ff22-4548-8707-39c2b2267658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:57.122323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9ed21-5df1-47fe-8abf-5c2b312ee542 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.123585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9ed21-5df1-47fe-8abf-5c2b312ee542 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.123585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9ed21-5df1-47fe-8abf-5c2b312ee542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.126219 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2938/5866] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:13:56 2022] DELETE /identity/v3/projects/e7addc44b5d74215917b9dfd170f56e1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:57.147544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-faf9ed21-5df1-47fe-8abf-5c2b312ee542 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:57.149548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-faf9ed21-5df1-47fe-8abf-5c2b312ee542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:57.217629 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2929/5867] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:57.226045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69089ca5-5fe4-4dfc-8bf6-461ec52b8e48 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.226179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69089ca5-5fe4-4dfc-8bf6-461ec52b8e48 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.226560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69089ca5-5fe4-4dfc-8bf6-461ec52b8e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.253271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-69089ca5-5fe4-4dfc-8bf6-461ec52b8e48 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:57.254598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-69089ca5-5fe4-4dfc-8bf6-461ec52b8e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:57.265743 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2939/5868] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:57.270599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9ed21-5df1-47fe-8abf-5c2b312ee542 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:57.281949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9ed21-5df1-47fe-8abf-5c2b312ee542 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:57.291439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f8268-47e4-4b07-bfda-70232b38aa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:57.292315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f8268-47e4-4b07-bfda-70232b38aa0c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.293014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f8268-47e4-4b07-bfda-70232b38aa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.293553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f8268-47e4-4b07-bfda-70232b38aa0c admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.295003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f8268-47e4-4b07-bfda-70232b38aa0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:13:57.295793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f8268-47e4-4b07-bfda-70232b38aa0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:57.298318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f8268-47e4-4b07-bfda-70232b38aa0c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:57.318911 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2930/5869] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:13:57.319955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69089ca5-5fe4-4dfc-8bf6-461ec52b8e48 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:57.332152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69089ca5-5fe4-4dfc-8bf6-461ec52b8e48 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:57.338455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcc44d-7027-4551-8d84-af637b8464e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:57.339075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc44d-7027-4551-8d84-af637b8464e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.339350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc44d-7027-4551-8d84-af637b8464e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.339617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc44d-7027-4551-8d84-af637b8464e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.340354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcc44d-7027-4551-8d84-af637b8464e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:57.343035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcc44d-7027-4551-8d84-af637b8464e5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:57.356874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:57.379419 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2940/5870] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:13:57 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:13:57.383974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f8268-47e4-4b07-bfda-70232b38aa0c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:57.396279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f8268-47e4-4b07-bfda-70232b38aa0c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:57.472240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592b41e-c7ca-4080-af21-0f0ff14cf734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:57.472866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592b41e-c7ca-4080-af21-0f0ff14cf734 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.473372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592b41e-c7ca-4080-af21-0f0ff14cf734 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.473697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592b41e-c7ca-4080-af21-0f0ff14cf734 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.474358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592b41e-c7ca-4080-af21-0f0ff14cf734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:57.476578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592b41e-c7ca-4080-af21-0f0ff14cf734 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:57.485776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:57.509488 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2931/5871] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:13:57 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:57.511071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcc44d-7027-4551-8d84-af637b8464e5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:57.525675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcc44d-7027-4551-8d84-af637b8464e5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:57.535405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3439e-d309-4be1-8a84-c75ef8171b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:57.537781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592b41e-c7ca-4080-af21-0f0ff14cf734 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:57.538382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3439e-d309-4be1-8a84-c75ef8171b08 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.540227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3439e-d309-4be1-8a84-c75ef8171b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.540359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1592b41e-c7ca-4080-af21-0f0ff14cf734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:57.540855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3439e-d309-4be1-8a84-c75ef8171b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.541933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3439e-d309-4be1-8a84-c75ef8171b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:57.543875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3439e-d309-4be1-8a84-c75ef8171b08 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:57.558712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592b41e-c7ca-4080-af21-0f0ff14cf734 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:57.565396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:57.566150 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2941/5872] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:57.566282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.566640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.566828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] PATH_INFO: `/v3/projects/4b7eb1d81f374478b3db39266029033b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.568480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7eb1d81f374478b3db39266029033b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:57.569112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3439e-d309-4be1-8a84-c75ef8171b08 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:57.570690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:57.633070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3439e-d309-4be1-8a84-c75ef8171b08 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:57.642043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe709fd-fa9f-435e-84e7-9fc1104e74ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:57.642940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe709fd-fa9f-435e-84e7-9fc1104e74ad admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.643284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe709fd-fa9f-435e-84e7-9fc1104e74ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.646054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe709fd-fa9f-435e-84e7-9fc1104e74ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.646927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe709fd-fa9f-435e-84e7-9fc1104e74ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:57.650344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe709fd-fa9f-435e-84e7-9fc1104e74ad admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:57.670841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7eb1d81f374478b3db39266029033b', 'request_id': 'req-610277e8-b4b5-4cc4-b718-783970a5cb3a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:57.689262 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2942/5873] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:57.690394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7eb1d81f374478b3db39266029033b', 'request_id': 'req-610277e8-b4b5-4cc4-b718-783970a5cb3a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:57.694402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe709fd-fa9f-435e-84e7-9fc1104e74ad admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:57.707352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7eb1d81f374478b3db39266029033b', 'request_id': 'req-610277e8-b4b5-4cc4-b718-783970a5cb3a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:57.715410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe709fd-fa9f-435e-84e7-9fc1104e74ad admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:57.743949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] The token cache is being invalidate because project 4b7eb1d81f374478b3db39266029033b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:13:57.746428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:57.748787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-610277e8-b4b5-4cc4-b718-783970a5cb3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:13:57.752279 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2932/5874] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:13:57 2022] DELETE /identity/v3/projects/4b7eb1d81f374478b3db39266029033b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:57.759623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b384ccd-1ee7-45f4-9775-2dc672d24609 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.759913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b384ccd-1ee7-45f4-9775-2dc672d24609 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.760183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b384ccd-1ee7-45f4-9775-2dc672d24609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.792215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7b384ccd-1ee7-45f4-9775-2dc672d24609 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:57.794473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7b384ccd-1ee7-45f4-9775-2dc672d24609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:57.854842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eea6bb-20ae-413a-89c5-893cac5b6796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:57.855771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eea6bb-20ae-413a-89c5-893cac5b6796 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.856055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eea6bb-20ae-413a-89c5-893cac5b6796 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.856329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eea6bb-20ae-413a-89c5-893cac5b6796 admin admin] PATH_INFO: `/v3/projects/32906f5db21341828c064548ddc93b11/users/83ef2c6c30454b6c8ddfaa42858c91bb/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.877131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eea6bb-20ae-413a-89c5-893cac5b6796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32906f5db21341828c064548ddc93b11, user_id=83ef2c6c30454b6c8ddfaa42858c91bb, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:57.880638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eea6bb-20ae-413a-89c5-893cac5b6796 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:57.913242 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2943/5875] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:57 2022] PUT /identity/v3/projects/32906f5db21341828c064548ddc93b11/users/83ef2c6c30454b6c8ddfaa42858c91bb/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:57.915132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07129718-d995-4452-b963-e45d43082eff None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.915416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07129718-d995-4452-b963-e45d43082eff None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.915682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07129718-d995-4452-b963-e45d43082eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:57.956545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-07129718-d995-4452-b963-e45d43082eff None None] MFA Rules not processed for user `83ef2c6c30454b6c8ddfaa42858c91bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:57.957937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-07129718-d995-4452-b963-e45d43082eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:57.966045 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2933/5876] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:13:57.968241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0df3c2-3ae4-489a-8bbb-536d5e853308 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:57.968510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0df3c2-3ae4-489a-8bbb-536d5e853308 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:57.969402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0df3c2-3ae4-489a-8bbb-536d5e853308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.002914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ab0df3c2-3ae4-489a-8bbb-536d5e853308 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:58.004404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ab0df3c2-3ae4-489a-8bbb-536d5e853308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:58.139269 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2944/5877] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:58.140771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07129718-d995-4452-b963-e45d43082eff None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.144213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-07129718-d995-4452-b963-e45d43082eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:58.149165 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2934/5878] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:13:58.150414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0df3c2-3ae4-489a-8bbb-536d5e853308 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.170599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0df3c2-3ae4-489a-8bbb-536d5e853308 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:58.178575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02a6d6-c442-4e18-b8ba-41afdcda9a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:58.180112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02a6d6-c442-4e18-b8ba-41afdcda9a09 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.181669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02a6d6-c442-4e18-b8ba-41afdcda9a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.181669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02a6d6-c442-4e18-b8ba-41afdcda9a09 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.182332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02a6d6-c442-4e18-b8ba-41afdcda9a09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:13:58.182817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02a6d6-c442-4e18-b8ba-41afdcda9a09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:58.183306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07129718-d995-4452-b963-e45d43082eff None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:58.217677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02a6d6-c442-4e18-b8ba-41afdcda9a09 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:58.223406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fe0db-821d-4efe-9436-c1952831f666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:58.224024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fe0db-821d-4efe-9436-c1952831f666 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.224296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fe0db-821d-4efe-9436-c1952831f666 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.224556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fe0db-821d-4efe-9436-c1952831f666 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.225210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fe0db-821d-4efe-9436-c1952831f666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:58.230253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fe0db-821d-4efe-9436-c1952831f666 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:58.249773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:58.262636 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2935/5879] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:13:58.267467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02a6d6-c442-4e18-b8ba-41afdcda9a09 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.280466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02a6d6-c442-4e18-b8ba-41afdcda9a09 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:58.285894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f3392-466d-4bdc-8052-1d81aba78218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:58.286515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f3392-466d-4bdc-8052-1d81aba78218 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.286791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f3392-466d-4bdc-8052-1d81aba78218 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.287095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f3392-466d-4bdc-8052-1d81aba78218 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.287672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f3392-466d-4bdc-8052-1d81aba78218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:58.294016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f3392-466d-4bdc-8052-1d81aba78218 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:58.308350 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2945/5880] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:13:58 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:58.314540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fe0db-821d-4efe-9436-c1952831f666 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.343741 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2936/5881] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:13:58 2022] POST /identity/v3/projects => generated 401 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:13:58.348593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f3392-466d-4bdc-8052-1d81aba78218 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.348782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:58.427101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fe0db-821d-4efe-9436-c1952831f666 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:58.522534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f3392-466d-4bdc-8052-1d81aba78218 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:58.535623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba6e6-5df1-48b6-8891-6f3fddd9161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:58.536301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba6e6-5df1-48b6-8891-6f3fddd9161e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.536654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba6e6-5df1-48b6-8891-6f3fddd9161e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.536961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba6e6-5df1-48b6-8891-6f3fddd9161e admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.540129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ba6e6-5df1-48b6-8891-6f3fddd9161e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:58.544081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ba6e6-5df1-48b6-8891-6f3fddd9161e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:58.617471 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2937/5882] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:13:58 2022] POST /identity/v3/users => generated 301 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:58.621586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba6e6-5df1-48b6-8891-6f3fddd9161e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.644988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba6e6-5df1-48b6-8891-6f3fddd9161e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:58.656280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde0dcd-fb96-4293-83cb-71961d529979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:58.657286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde0dcd-fb96-4293-83cb-71961d529979 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.657871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde0dcd-fb96-4293-83cb-71961d529979 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.658347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde0dcd-fb96-4293-83cb-71961d529979 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.659193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde0dcd-fb96-4293-83cb-71961d529979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:58.661239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde0dcd-fb96-4293-83cb-71961d529979 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:58.667591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeeae78-672d-4f8f-ac24-9fde2a3f18e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:58.670647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeae78-672d-4f8f-ac24-9fde2a3f18e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.671166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeae78-672d-4f8f-ac24-9fde2a3f18e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.671475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeae78-672d-4f8f-ac24-9fde2a3f18e5 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.677570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeeae78-672d-4f8f-ac24-9fde2a3f18e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:58.679547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeeae78-672d-4f8f-ac24-9fde2a3f18e5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:58.699189 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2938/5883] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:58.700978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde0dcd-fb96-4293-83cb-71961d529979 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.715687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde0dcd-fb96-4293-83cb-71961d529979 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:58.723112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695e220-6fc8-4577-a6fd-170e9258ca44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:58.724076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695e220-6fc8-4577-a6fd-170e9258ca44 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.724254 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2946/5884] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:13:58 2022] POST /identity/v3/users => generated 298 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:58.724531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695e220-6fc8-4577-a6fd-170e9258ca44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.724994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695e220-6fc8-4577-a6fd-170e9258ca44 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.725790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695e220-6fc8-4577-a6fd-170e9258ca44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:58.727666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695e220-6fc8-4577-a6fd-170e9258ca44 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:58.728038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeeae78-672d-4f8f-ac24-9fde2a3f18e5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.743699 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2939/5885] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:58.746131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695e220-6fc8-4577-a6fd-170e9258ca44 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.761608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeeae78-672d-4f8f-ac24-9fde2a3f18e5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:58.776460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b5be4-44a0-4fa8-b632-6cdd66576222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:58.777494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b5be4-44a0-4fa8-b632-6cdd66576222 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.777767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b5be4-44a0-4fa8-b632-6cdd66576222 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.778035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b5be4-44a0-4fa8-b632-6cdd66576222 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.778416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695e220-6fc8-4577-a6fd-170e9258ca44 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:58.778692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b5be4-44a0-4fa8-b632-6cdd66576222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:58.784606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b5be4-44a0-4fa8-b632-6cdd66576222 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:58.797214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c2f8c-e9d1-4d9c-8d87-2fbd9a5a326d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:58.798041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c2f8c-e9d1-4d9c-8d87-2fbd9a5a326d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.798414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c2f8c-e9d1-4d9c-8d87-2fbd9a5a326d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.798782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c2f8c-e9d1-4d9c-8d87-2fbd9a5a326d admin admin] PATH_INFO: `/v3/projects/49500b60c6c040b389e6b76ceaac1a98/users/7e8e9f96c9804c4a8902f21b2947e9da/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.809000 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2947/5886] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:58.811094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b5be4-44a0-4fa8-b632-6cdd66576222 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.835145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b5be4-44a0-4fa8-b632-6cdd66576222 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:58.836244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c2f8c-e9d1-4d9c-8d87-2fbd9a5a326d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49500b60c6c040b389e6b76ceaac1a98, user_id=7e8e9f96c9804c4a8902f21b2947e9da, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:58.855549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c2f8c-e9d1-4d9c-8d87-2fbd9a5a326d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:58.855799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40355fd3-6f9f-4da7-bfae-eb0be58dd2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:58.855799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40355fd3-6f9f-4da7-bfae-eb0be58dd2fc admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.855799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40355fd3-6f9f-4da7-bfae-eb0be58dd2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.855799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40355fd3-6f9f-4da7-bfae-eb0be58dd2fc admin admin] PATH_INFO: `/v3/projects/54e39712c0ef4944a51366c629f7ad3f/users/c949612a037348b5a72b90022d9d20eb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.877424 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2940/5887] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:58 2022] PUT /identity/v3/projects/49500b60c6c040b389e6b76ceaac1a98/users/7e8e9f96c9804c4a8902f21b2947e9da/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:58.884196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40355fd3-6f9f-4da7-bfae-eb0be58dd2fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e39712c0ef4944a51366c629f7ad3f, user_id=c949612a037348b5a72b90022d9d20eb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:58.886984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aded4-2f7c-418a-a22e-568f28e9bb79 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:58.887326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aded4-2f7c-418a-a22e-568f28e9bb79 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:58.887664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aded4-2f7c-418a-a22e-568f28e9bb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:58.893891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40355fd3-6f9f-4da7-bfae-eb0be58dd2fc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:58.922684 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2948/5888] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:58 2022] PUT /identity/v3/projects/54e39712c0ef4944a51366c629f7ad3f/users/c949612a037348b5a72b90022d9d20eb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:58.923836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40355fd3-6f9f-4da7-bfae-eb0be58dd2fc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:58.944259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-060aded4-2f7c-418a-a22e-568f28e9bb79 None None] MFA Rules not processed for user `7e8e9f96c9804c4a8902f21b2947e9da`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:58.945693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-060aded4-2f7c-418a-a22e-568f28e9bb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:59.058756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40355fd3-6f9f-4da7-bfae-eb0be58dd2fc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:59.067996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a507d-86e5-43e4-aaf1-e6e54b361264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:59.068840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a507d-86e5-43e4-aaf1-e6e54b361264 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.069268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a507d-86e5-43e4-aaf1-e6e54b361264 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.069723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a507d-86e5-43e4-aaf1-e6e54b361264 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.070468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a507d-86e5-43e4-aaf1-e6e54b361264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:59.074846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a507d-86e5-43e4-aaf1-e6e54b361264 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:59.096779 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2949/5889] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:59.099446 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2941/5890] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:59.104439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aded4-2f7c-418a-a22e-568f28e9bb79 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:59.104690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a507d-86e5-43e4-aaf1-e6e54b361264 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:59.120030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aded4-2f7c-418a-a22e-568f28e9bb79 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:59.122003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a507d-86e5-43e4-aaf1-e6e54b361264 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:59.125728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894cbb1-2d0a-4529-9698-c781c9e7773f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:59.126342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894cbb1-2d0a-4529-9698-c781c9e7773f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.126613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894cbb1-2d0a-4529-9698-c781c9e7773f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.126875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894cbb1-2d0a-4529-9698-c781c9e7773f admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.127558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894cbb1-2d0a-4529-9698-c781c9e7773f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:59.129612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321da13e-b3b1-405e-835a-10c93851a1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:59.130348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321da13e-b3b1-405e-835a-10c93851a1eb admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.130651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894cbb1-2d0a-4529-9698-c781c9e7773f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:59.130796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321da13e-b3b1-405e-835a-10c93851a1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.131123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321da13e-b3b1-405e-835a-10c93851a1eb admin admin] PATH_INFO: `/v3/projects/54e39712c0ef4944a51366c629f7ad3f/users/c949612a037348b5a72b90022d9d20eb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.133980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321da13e-b3b1-405e-835a-10c93851a1eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e39712c0ef4944a51366c629f7ad3f, user_id=c949612a037348b5a72b90022d9d20eb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:59.139757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321da13e-b3b1-405e-835a-10c93851a1eb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:59.144768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:59.159348 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2950/5891] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:59 2022] PUT /identity/v3/projects/54e39712c0ef4944a51366c629f7ad3f/users/c949612a037348b5a72b90022d9d20eb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:59.173931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab3955-7191-49ff-85f7-5839472fa1f4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.174314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab3955-7191-49ff-85f7-5839472fa1f4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.174599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab3955-7191-49ff-85f7-5839472fa1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.186907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:59.188159 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2942/5892] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:13:59 2022] POST /identity/v3/projects => generated 401 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:59.206695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894cbb1-2d0a-4529-9698-c781c9e7773f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:59.238383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b3ab3955-7191-49ff-85f7-5839472fa1f4 None None] MFA Rules not processed for user `c949612a037348b5a72b90022d9d20eb`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:59.239827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b3ab3955-7191-49ff-85f7-5839472fa1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:59.387318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894cbb1-2d0a-4529-9698-c781c9e7773f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:59.393438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeca95d-976f-4673-990b-ce51338f6a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:59.394278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeca95d-976f-4673-990b-ce51338f6a8e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.394560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeca95d-976f-4673-990b-ce51338f6a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.394866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeca95d-976f-4673-990b-ce51338f6a8e admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.395612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeca95d-976f-4673-990b-ce51338f6a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:59.397952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeca95d-976f-4673-990b-ce51338f6a8e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:59.434423 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2943/5893] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:13:59 2022] POST /identity/v3/users => generated 307 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:13:59.435684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeca95d-976f-4673-990b-ce51338f6a8e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:59.455311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeca95d-976f-4673-990b-ce51338f6a8e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:59.467396 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2951/5894] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:13:59.476790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba545f40-dbb8-4d34-86c2-aa1a7eaf4404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:59.477141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab3955-7191-49ff-85f7-5839472fa1f4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:59.478222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba545f40-dbb8-4d34-86c2-aa1a7eaf4404 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.478774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba545f40-dbb8-4d34-86c2-aa1a7eaf4404 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.479689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b3ab3955-7191-49ff-85f7-5839472fa1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:59.480412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba545f40-dbb8-4d34-86c2-aa1a7eaf4404 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.482294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba545f40-dbb8-4d34-86c2-aa1a7eaf4404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:59.485753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba545f40-dbb8-4d34-86c2-aa1a7eaf4404 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:59.519973 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2944/5895] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:13:59.522954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba545f40-dbb8-4d34-86c2-aa1a7eaf4404 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:59.537825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba545f40-dbb8-4d34-86c2-aa1a7eaf4404 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:59.547205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2feac6-86b8-433e-817f-55d593ba2cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:59.547989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2feac6-86b8-433e-817f-55d593ba2cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.548338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2feac6-86b8-433e-817f-55d593ba2cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.548686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2feac6-86b8-433e-817f-55d593ba2cdb admin admin] PATH_INFO: `/v3/projects/fa3cec97aa804db1aa660530848c25c9/users/1f1d2e6845054e60a3d6c31be333d0d9/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.569714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2feac6-86b8-433e-817f-55d593ba2cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa3cec97aa804db1aa660530848c25c9, user_id=1f1d2e6845054e60a3d6c31be333d0d9, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:59.572556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2feac6-86b8-433e-817f-55d593ba2cdb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:59.600389 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2945/5896] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:59 2022] PUT /identity/v3/projects/fa3cec97aa804db1aa660530848c25c9/users/1f1d2e6845054e60a3d6c31be333d0d9/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:59.602104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2feac6-86b8-433e-817f-55d593ba2cdb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:59.647122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab3955-7191-49ff-85f7-5839472fa1f4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:59.751100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92fda9-6a2b-4f07-906b-12dbc7424112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:59.752023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92fda9-6a2b-4f07-906b-12dbc7424112 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.753334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92fda9-6a2b-4f07-906b-12dbc7424112 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.753952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92fda9-6a2b-4f07-906b-12dbc7424112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.754861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2feac6-86b8-433e-817f-55d593ba2cdb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:59.760146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b21ba1-9bff-4f77-9dec-ba44ae2660b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:59.760766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b21ba1-9bff-4f77-9dec-ba44ae2660b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.761076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b21ba1-9bff-4f77-9dec-ba44ae2660b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.761862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b21ba1-9bff-4f77-9dec-ba44ae2660b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.762522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b21ba1-9bff-4f77-9dec-ba44ae2660b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:59.764735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b21ba1-9bff-4f77-9dec-ba44ae2660b4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:59.786288 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2946/5897] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:13:59.787237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b21ba1-9bff-4f77-9dec-ba44ae2660b4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:13:59.804280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b21ba1-9bff-4f77-9dec-ba44ae2660b4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:13:59.812430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baffba8-549e-4dd9-8ec4-9378d4df851d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:13:59.813368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baffba8-549e-4dd9-8ec4-9378d4df851d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.813653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baffba8-549e-4dd9-8ec4-9378d4df851d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.813926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baffba8-549e-4dd9-8ec4-9378d4df851d admin admin] PATH_INFO: `/v3/projects/fa3cec97aa804db1aa660530848c25c9/users/1f1d2e6845054e60a3d6c31be333d0d9/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.816757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baffba8-549e-4dd9-8ec4-9378d4df851d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa3cec97aa804db1aa660530848c25c9, user_id=1f1d2e6845054e60a3d6c31be333d0d9, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:59.836395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baffba8-549e-4dd9-8ec4-9378d4df851d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:59.862839 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2947/5898] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:13:59 2022] PUT /identity/v3/projects/fa3cec97aa804db1aa660530848c25c9/users/1f1d2e6845054e60a3d6c31be333d0d9/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:13:59.867700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92fda9-6a2b-4f07-906b-12dbc7424112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:13:59.869707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92fda9-6a2b-4f07-906b-12dbc7424112 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:13:59.875888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8b662-5dfe-4cbe-bc8c-e67b5fd69cd5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:13:59.876528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8b662-5dfe-4cbe-bc8c-e67b5fd69cd5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:13:59.877068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8b662-5dfe-4cbe-bc8c-e67b5fd69cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:13:59.895803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:59.914013 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2952/5899] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:13:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:13:59.961511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-41b8b662-5dfe-4cbe-bc8c-e67b5fd69cd5 None None] MFA Rules not processed for user `1f1d2e6845054e60a3d6c31be333d0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:13:59.962964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-41b8b662-5dfe-4cbe-bc8c-e67b5fd69cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:13:59.979048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:00.053336 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2948/5900] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:14:00.058187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92fda9-6a2b-4f07-906b-12dbc7424112 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:00.164965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92fda9-6a2b-4f07-906b-12dbc7424112 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:00.173127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d0609-bb75-453e-b9dd-8e2c3e5c86d5 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1d2e6845054e60a3d6c31be333d0d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3cec97aa804db1aa660530848c25c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:00.174199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d0609-bb75-453e-b9dd-8e2c3e5c86d5 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:00.174371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d0609-bb75-453e-b9dd-8e2c3e5c86d5 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:00.174763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d0609-bb75-453e-b9dd-8e2c3e5c86d5 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:00.175640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d0609-bb75-453e-b9dd-8e2c3e5c86d5 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:00.177417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d0609-bb75-453e-b9dd-8e2c3e5c86d5 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:00.215782 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2953/5901] 10.222.0.16 () {60 vars in 1143 bytes} [Sat Oct 15 21:14:00 2022] POST /identity/v3/roles => generated 240 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:00.220407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8b662-5dfe-4cbe-bc8c-e67b5fd69cd5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:00.233392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8b662-5dfe-4cbe-bc8c-e67b5fd69cd5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:00.244396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6316922-504c-4f12-aca1-9baa66485cd1 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1d2e6845054e60a3d6c31be333d0d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3cec97aa804db1aa660530848c25c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:00.245626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6316922-504c-4f12-aca1-9baa66485cd1 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:00.246399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6316922-504c-4f12-aca1-9baa66485cd1 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:00.247884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6316922-504c-4f12-aca1-9baa66485cd1 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] PATH_INFO: `/v3/projects/49500b60c6c040b389e6b76ceaac1a98/users/7e8e9f96c9804c4a8902f21b2947e9da/roles/0c42224f25444b3697ef51c257e80cc5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:00.251852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6316922-504c-4f12-aca1-9baa66485cd1 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] RBAC: Authorizing `identity:create_grant(project_id=49500b60c6c040b389e6b76ceaac1a98, user_id=7e8e9f96c9804c4a8902f21b2947e9da, role_id=0c42224f25444b3697ef51c257e80cc5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:00.259046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6316922-504c-4f12-aca1-9baa66485cd1 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:00.290802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d0609-bb75-453e-b9dd-8e2c3e5c86d5 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:00.291891 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2949/5902] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:00 2022] PUT /identity/v3/projects/49500b60c6c040b389e6b76ceaac1a98/users/7e8e9f96c9804c4a8902f21b2947e9da/roles/0c42224f25444b3697ef51c257e80cc5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:00.477432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d0609-bb75-453e-b9dd-8e2c3e5c86d5 tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:00.623354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6155d2a-8f5c-4065-8231-f7bdb71e72d8 tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e8e9f96c9804c4a8902f21b2947e9da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49500b60c6c040b389e6b76ceaac1a98', 'project_domain_id': 'default', 'roles': ['reader', 'tempest-test_role-1512783692', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:00.625602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6155d2a-8f5c-4065-8231-f7bdb71e72d8 tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:00.625997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6155d2a-8f5c-4065-8231-f7bdb71e72d8 tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:00.627691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6155d2a-8f5c-4065-8231-f7bdb71e72d8 tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] PATH_INFO: `/v3/users/7e8e9f96c9804c4a8902f21b2947e9da/application_credentials` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:00.633085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6155d2a-8f5c-4065-8231-f7bdb71e72d8 tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] RBAC: Authorizing `identity:create_application_credential(user_id=7e8e9f96c9804c4a8902f21b2947e9da)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:00.636774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6155d2a-8f5c-4065-8231-f7bdb71e72d8 tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:00.737370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:00.775383 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2954/5903] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Oct 15 21:14:00 2022] POST /identity/v3/users/7e8e9f96c9804c4a8902f21b2947e9da/application_credentials => generated 659 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:14:00.779560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78670722-8a3f-4036-ae38-62a6de9a921c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:00.779846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78670722-8a3f-4036-ae38-62a6de9a921c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:00.780222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78670722-8a3f-4036-ae38-62a6de9a921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:00.819182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-78670722-8a3f-4036-ae38-62a6de9a921c None None] MFA Rules not processed for user `7e8e9f96c9804c4a8902f21b2947e9da`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:00.820733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-78670722-8a3f-4036-ae38-62a6de9a921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:01.014822 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2950/5904] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3394 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Oct 15 21:14:01.021927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6155d2a-8f5c-4065-8231-f7bdb71e72d8 tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:01.037945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6155d2a-8f5c-4065-8231-f7bdb71e72d8 tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:01.047594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd520f-9444-4cef-b7c4-416744fc48ab tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e8e9f96c9804c4a8902f21b2947e9da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49500b60c6c040b389e6b76ceaac1a98', 'project_domain_id': 'default', 'roles': ['reader', 'tempest-test_role-1512783692', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:01.048627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd520f-9444-4cef-b7c4-416744fc48ab tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:01.049242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd520f-9444-4cef-b7c4-416744fc48ab tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:01.049382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd520f-9444-4cef-b7c4-416744fc48ab tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] PATH_INFO: `/v3/users/7e8e9f96c9804c4a8902f21b2947e9da/application_credentials/521f9699ce814c39880ede93677c0060` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:01.051579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd520f-9444-4cef-b7c4-416744fc48ab tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] RBAC: Authorizing `identity:delete_application_credential(user_id=7e8e9f96c9804c4a8902f21b2947e9da, application_credential_id=521f9699ce814c39880ede93677c0060)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:01.053565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd520f-9444-4cef-b7c4-416744fc48ab tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:01.095817 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2955/5905] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Oct 15 21:14:01 2022] DELETE /identity/v3/users/7e8e9f96c9804c4a8902f21b2947e9da/application_credentials/521f9699ce814c39880ede93677c0060 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:01.100341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78670722-8a3f-4036-ae38-62a6de9a921c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:01.165646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:01.267640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78670722-8a3f-4036-ae38-62a6de9a921c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:01.275377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c298a41-66a5-4e60-9867-297d5fd7c52a tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1d2e6845054e60a3d6c31be333d0d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3cec97aa804db1aa660530848c25c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:01.276086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c298a41-66a5-4e60-9867-297d5fd7c52a tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:01.276425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c298a41-66a5-4e60-9867-297d5fd7c52a tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:01.276763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c298a41-66a5-4e60-9867-297d5fd7c52a tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] PATH_INFO: `/v3/roles/0c42224f25444b3697ef51c257e80cc5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:01.278273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c298a41-66a5-4e60-9867-297d5fd7c52a tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0c42224f25444b3697ef51c257e80cc5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:01.280504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c298a41-66a5-4e60-9867-297d5fd7c52a tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:01.349330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6c298a41-66a5-4e60-9867-297d5fd7c52a tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] Invalidating the token cache because role 0c42224f25444b3697ef51c257e80cc5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:01.350150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6c298a41-66a5-4e60-9867-297d5fd7c52a tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:01.350381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6c298a41-66a5-4e60-9867-297d5fd7c52a tempest-ApplicationCredentialsV3AdminTest-898962849 tempest-ApplicationCredentialsV3AdminTest-898962849-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:01.357921 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2951/5906] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:01 2022] DELETE /identity/v3/roles/0c42224f25444b3697ef51c257e80cc5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:01.359323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd520f-9444-4cef-b7c4-416744fc48ab tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:01.362171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-72cd520f-9444-4cef-b7c4-416744fc48ab tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:01.458623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd520f-9444-4cef-b7c4-416744fc48ab tempest-ApplicationCredentialsV3AdminTest-234798014 tempest-ApplicationCredentialsV3AdminTest-234798014-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:01.531012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:01.531672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:01.531946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:01.532440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] PATH_INFO: `/v3/users/7e8e9f96c9804c4a8902f21b2947e9da` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:01.533389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8e9f96c9804c4a8902f21b2947e9da)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:01.535325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:01.595843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8e9f96c9804c4a8902f21b2947e9da'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:01.610847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8e9f96c9804c4a8902f21b2947e9da'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:01.614955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:01.621853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8e9f96c9804c4a8902f21b2947e9da'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:01.637192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8e9f96c9804c4a8902f21b2947e9da'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:01.639882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8e9f96c9804c4a8902f21b2947e9da'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:01.647828 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2956/5907] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:01 2022] DELETE /identity/v3/users/7e8e9f96c9804c4a8902f21b2947e9da => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:01.652580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c61bc-c704-43da-bdac-429af49980ab None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:01.653676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c61bc-c704-43da-bdac-429af49980ab None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:01.653676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c61bc-c704-43da-bdac-429af49980ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:01.687436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fa6c61bc-c704-43da-bdac-429af49980ab None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:01.688978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa6c61bc-c704-43da-bdac-429af49980ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:01.818355 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2952/5908] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:01.847156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:01.849316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:01.907162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c61bc-c704-43da-bdac-429af49980ab None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:01.908620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa6c61bc-c704-43da-bdac-429af49980ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:01.977292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f18a24-ff4d-4c0b-841b-365098ead712 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:01.987045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfd14b-cbde-4509-ab83-280326197105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:01.987508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfd14b-cbde-4509-ab83-280326197105 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:01.987780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfd14b-cbde-4509-ab83-280326197105 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:01.988037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfd14b-cbde-4509-ab83-280326197105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:02.006477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfd14b-cbde-4509-ab83-280326197105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:02.008931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfd14b-cbde-4509-ab83-280326197105 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:02.039998 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2957/5909] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:02.076254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c61bc-c704-43da-bdac-429af49980ab None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:02.092077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a97f-ae62-4d88-b0f1-3a31b65b367f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:02.093251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a97f-ae62-4d88-b0f1-3a31b65b367f service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:02.093902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a97f-ae62-4d88-b0f1-3a31b65b367f service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:02.098886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a97f-ae62-4d88-b0f1-3a31b65b367f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:02.105968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2c59a97f-ae62-4d88-b0f1-3a31b65b367f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:02.133562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59a97f-ae62-4d88-b0f1-3a31b65b367f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:02.135281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59a97f-ae62-4d88-b0f1-3a31b65b367f service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:02.159719 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2953/5910] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:14:04.464397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfd14b-cbde-4509-ab83-280326197105 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:04.466054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6edfd14b-cbde-4509-ab83-280326197105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:04.493597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfd14b-cbde-4509-ab83-280326197105 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:04.501129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:04.503045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:04.503478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:04.504055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] PATH_INFO: `/v3/projects/49500b60c6c040b389e6b76ceaac1a98` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:04.506177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=49500b60c6c040b389e6b76ceaac1a98)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:04.510260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:04.565472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49500b60c6c040b389e6b76ceaac1a98', 'request_id': 'req-2af37420-93e0-415b-97d8-bae1697e8fbf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:04.588896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49500b60c6c040b389e6b76ceaac1a98', 'request_id': 'req-2af37420-93e0-415b-97d8-bae1697e8fbf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:04.598249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49500b60c6c040b389e6b76ceaac1a98', 'request_id': 'req-2af37420-93e0-415b-97d8-bae1697e8fbf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:04.616156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] The token cache is being invalidate because project 49500b60c6c040b389e6b76ceaac1a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:04.617241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:04.617986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:04.619844 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2958/5911] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:04 2022] DELETE /identity/v3/projects/49500b60c6c040b389e6b76ceaac1a98 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:04.624211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a97f-ae62-4d88-b0f1-3a31b65b367f service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:04.640026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a97f-ae62-4d88-b0f1-3a31b65b367f service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:04.697117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:04.750632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:04.751375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:04.751905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:04.752450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] PATH_INFO: `/v3/users/1f1d2e6845054e60a3d6c31be333d0d9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:04.753917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f1d2e6845054e60a3d6c31be333d0d9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:04.756292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:04.890220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f1d2e6845054e60a3d6c31be333d0d9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:04.899958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f1d2e6845054e60a3d6c31be333d0d9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:04.931108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f1d2e6845054e60a3d6c31be333d0d9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:04.960636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f1d2e6845054e60a3d6c31be333d0d9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:04.961149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f1d2e6845054e60a3d6c31be333d0d9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:04.970405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:04.972175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:04.975313 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2954/5912] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:04 2022] DELETE /identity/v3/users/1f1d2e6845054e60a3d6c31be333d0d9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:05.063753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af37420-93e0-415b-97d8-bae1697e8fbf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:05.095999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:05.131588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e230601-5a83-43d7-80d7-886ec5ff36d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:05.132440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230601-5a83-43d7-80d7-886ec5ff36d8 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:05.132814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230601-5a83-43d7-80d7-886ec5ff36d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:05.133224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230601-5a83-43d7-80d7-886ec5ff36d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:05.135748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4e230601-5a83-43d7-80d7-886ec5ff36d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:05.228392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e230601-5a83-43d7-80d7-886ec5ff36d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:05.229909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e230601-5a83-43d7-80d7-886ec5ff36d8 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:05.242096 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2959/5913] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:14:06.865599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:06.867859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:07.010669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572221d-0c32-4d4c-a9d1-2c8b6dc06fde admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:07.030357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:07.031007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:07.031284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:07.031555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] PATH_INFO: `/v3/projects/fa3cec97aa804db1aa660530848c25c9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:07.039525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa3cec97aa804db1aa660530848c25c9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:07.041180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:07.164110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:07.174044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa3cec97aa804db1aa660530848c25c9', 'request_id': 'req-12c03a41-eb78-45ba-a4f7-9dcba193b48f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:07.183834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa3cec97aa804db1aa660530848c25c9', 'request_id': 'req-12c03a41-eb78-45ba-a4f7-9dcba193b48f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:07.197428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa3cec97aa804db1aa660530848c25c9', 'request_id': 'req-12c03a41-eb78-45ba-a4f7-9dcba193b48f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:07.227527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] The token cache is being invalidate because project fa3cec97aa804db1aa660530848c25c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:07.227844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:07.230826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-12c03a41-eb78-45ba-a4f7-9dcba193b48f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:07.237651 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2955/5914] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:06 2022] DELETE /identity/v3/projects/fa3cec97aa804db1aa660530848c25c9 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:07.238848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e235f-63e8-4c64-887d-69d1b524cd1b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:07.239326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e235f-63e8-4c64-887d-69d1b524cd1b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:07.239421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e235f-63e8-4c64-887d-69d1b524cd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:07.272583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-826e235f-63e8-4c64-887d-69d1b524cd1b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:07.275079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-826e235f-63e8-4c64-887d-69d1b524cd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:07.480904 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2960/5915] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:14:07.490284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57601975-b4f7-4ecd-8926-9ec671ee1e19 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:07.490952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57601975-b4f7-4ecd-8926-9ec671ee1e19 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:07.491711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57601975-b4f7-4ecd-8926-9ec671ee1e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:07.539135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-57601975-b4f7-4ecd-8926-9ec671ee1e19 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:07.540377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-57601975-b4f7-4ecd-8926-9ec671ee1e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:07.570596 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2956/5916] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:07.571364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e235f-63e8-4c64-887d-69d1b524cd1b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:07.584548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e235f-63e8-4c64-887d-69d1b524cd1b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:07.592407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a944c7-a215-4ef9-8a82-e6ee5ac26b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:07.593233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a944c7-a215-4ef9-8a82-e6ee5ac26b90 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:07.593594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a944c7-a215-4ef9-8a82-e6ee5ac26b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:07.593934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a944c7-a215-4ef9-8a82-e6ee5ac26b90 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:07.594733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a944c7-a215-4ef9-8a82-e6ee5ac26b90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:07.595213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a944c7-a215-4ef9-8a82-e6ee5ac26b90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:07.597375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a944c7-a215-4ef9-8a82-e6ee5ac26b90 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:07.614163 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2961/5917] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:07.618943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57601975-b4f7-4ecd-8926-9ec671ee1e19 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:07.634939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57601975-b4f7-4ecd-8926-9ec671ee1e19 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:07.643257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c89594-0b5c-4c3d-a68f-2a0d9d4f0792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:07.643624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c89594-0b5c-4c3d-a68f-2a0d9d4f0792 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:07.644262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c89594-0b5c-4c3d-a68f-2a0d9d4f0792 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:07.645523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c89594-0b5c-4c3d-a68f-2a0d9d4f0792 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:07.646461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c89594-0b5c-4c3d-a68f-2a0d9d4f0792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:07.650530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c89594-0b5c-4c3d-a68f-2a0d9d4f0792 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:07.711704 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2957/5918] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:07 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:07.713258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a944c7-a215-4ef9-8a82-e6ee5ac26b90 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:07.730385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a944c7-a215-4ef9-8a82-e6ee5ac26b90 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:07.849888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12e3c4-c5f0-4a45-8839-c504f03e1492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:07.850784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12e3c4-c5f0-4a45-8839-c504f03e1492 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:07.851154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12e3c4-c5f0-4a45-8839-c504f03e1492 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:07.851547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12e3c4-c5f0-4a45-8839-c504f03e1492 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:07.852274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12e3c4-c5f0-4a45-8839-c504f03e1492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:07.854480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12e3c4-c5f0-4a45-8839-c504f03e1492 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:07.873546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:07.895534 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2962/5919] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:07 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:07.896354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c89594-0b5c-4c3d-a68f-2a0d9d4f0792 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:07.911691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c89594-0b5c-4c3d-a68f-2a0d9d4f0792 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:07.919242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d806e8-3d6c-4b52-a80e-29c4caa51a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:07.920059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d806e8-3d6c-4b52-a80e-29c4caa51a50 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:07.920587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d806e8-3d6c-4b52-a80e-29c4caa51a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:07.921491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d806e8-3d6c-4b52-a80e-29c4caa51a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:07.922611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d806e8-3d6c-4b52-a80e-29c4caa51a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:07.925445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d806e8-3d6c-4b52-a80e-29c4caa51a50 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:07.946530 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2958/5920] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:14:07.952487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12e3c4-c5f0-4a45-8839-c504f03e1492 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:07.966597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12e3c4-c5f0-4a45-8839-c504f03e1492 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:07.972636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc94660-1626-42c4-918b-1b3e8938ff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:07.973426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc94660-1626-42c4-918b-1b3e8938ff7a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:07.973772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc94660-1626-42c4-918b-1b3e8938ff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:07.974193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc94660-1626-42c4-918b-1b3e8938ff7a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:07.974999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc94660-1626-42c4-918b-1b3e8938ff7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:07.977013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc94660-1626-42c4-918b-1b3e8938ff7a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:08.005989 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2963/5921] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:08.007112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d806e8-3d6c-4b52-a80e-29c4caa51a50 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:08.022186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d806e8-3d6c-4b52-a80e-29c4caa51a50 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:08.036813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3fbb0-3118-458b-8cef-1807b4f8d729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:08.038610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3fbb0-3118-458b-8cef-1807b4f8d729 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:08.039194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3fbb0-3118-458b-8cef-1807b4f8d729 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:08.039957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3fbb0-3118-458b-8cef-1807b4f8d729 admin admin] PATH_INFO: `/v3/projects/0229134d004a4a41a7f7f4bf628a7858/users/074aa9aee2de4aa1a60a429921789b3f/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:08.063076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3fbb0-3118-458b-8cef-1807b4f8d729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0229134d004a4a41a7f7f4bf628a7858, user_id=074aa9aee2de4aa1a60a429921789b3f, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:08.066792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3fbb0-3118-458b-8cef-1807b4f8d729 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:08.092206 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2959/5922] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:08 2022] PUT /identity/v3/projects/0229134d004a4a41a7f7f4bf628a7858/users/074aa9aee2de4aa1a60a429921789b3f/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:08.104059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5a26c-98f1-496a-b32c-527b22fbcc9e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:08.104341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5a26c-98f1-496a-b32c-527b22fbcc9e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:08.104613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5a26c-98f1-496a-b32c-527b22fbcc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:08.147516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fbe5a26c-98f1-496a-b32c-527b22fbcc9e None None] MFA Rules not processed for user `074aa9aee2de4aa1a60a429921789b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:08.149070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fbe5a26c-98f1-496a-b32c-527b22fbcc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:08.407288 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2964/5923] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:08.408795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3fbb0-3118-458b-8cef-1807b4f8d729 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:08.582131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3fbb0-3118-458b-8cef-1807b4f8d729 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:08.598246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf251c0f-3483-4ccf-93e3-e030e6bc5854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:08.600814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf251c0f-3483-4ccf-93e3-e030e6bc5854 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:08.601571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf251c0f-3483-4ccf-93e3-e030e6bc5854 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:08.601878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf251c0f-3483-4ccf-93e3-e030e6bc5854 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:08.602964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf251c0f-3483-4ccf-93e3-e030e6bc5854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:08.610587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf251c0f-3483-4ccf-93e3-e030e6bc5854 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:08.689267 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2960/5924] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:08 2022] POST /identity/v3/projects => generated 371 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:14:08.693955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5a26c-98f1-496a-b32c-527b22fbcc9e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:08.710845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5a26c-98f1-496a-b32c-527b22fbcc9e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:08.849670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf251c0f-3483-4ccf-93e3-e030e6bc5854 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:08.851584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cf251c0f-3483-4ccf-93e3-e030e6bc5854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:08.858864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c014d3a-ca8d-4db4-a2a9-01fca04c79fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:08.859849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c014d3a-ca8d-4db4-a2a9-01fca04c79fd admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:08.860505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c014d3a-ca8d-4db4-a2a9-01fca04c79fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:08.860972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c014d3a-ca8d-4db4-a2a9-01fca04c79fd admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:08.862131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c014d3a-ca8d-4db4-a2a9-01fca04c79fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:08.865378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c014d3a-ca8d-4db4-a2a9-01fca04c79fd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:08.943003 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2965/5925] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:08 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:14:08.948825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c014d3a-ca8d-4db4-a2a9-01fca04c79fd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:08.980731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c014d3a-ca8d-4db4-a2a9-01fca04c79fd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:08.999154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646113c-746d-4ec8-9f79-757610027432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:09.002038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646113c-746d-4ec8-9f79-757610027432 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:09.002323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646113c-746d-4ec8-9f79-757610027432 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:09.002598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646113c-746d-4ec8-9f79-757610027432 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:09.004901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646113c-746d-4ec8-9f79-757610027432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:09.009644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646113c-746d-4ec8-9f79-757610027432 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:09.026873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf251c0f-3483-4ccf-93e3-e030e6bc5854 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:09.041105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a0f71-acd6-4fb5-a7b2-953cab839a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:09.042830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a0f71-acd6-4fb5-a7b2-953cab839a9d service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:09.043790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a0f71-acd6-4fb5-a7b2-953cab839a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:09.044853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a0f71-acd6-4fb5-a7b2-953cab839a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:09.048964 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2966/5926] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:09.052600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646113c-746d-4ec8-9f79-757610027432 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:09.056020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f70a0f71-acd6-4fb5-a7b2-953cab839a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:09.084785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646113c-746d-4ec8-9f79-757610027432 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:09.098269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a0f71-acd6-4fb5-a7b2-953cab839a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:09.100014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a0f71-acd6-4fb5-a7b2-953cab839a9d service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:09.101404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cca3e-7fe8-4afb-87e9-57533bc2b2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:09.101605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cca3e-7fe8-4afb-87e9-57533bc2b2fe admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:09.102885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cca3e-7fe8-4afb-87e9-57533bc2b2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:09.103189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cca3e-7fe8-4afb-87e9-57533bc2b2fe admin admin] PATH_INFO: `/v3/projects/8f78a07fdd98414e91f6a46aff3b1111/users/98e25a6bea0c4eb69a845dba63c4de4e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:09.121020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:09.135892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2961/5927] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:14:09.147630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cca3e-7fe8-4afb-87e9-57533bc2b2fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f78a07fdd98414e91f6a46aff3b1111, user_id=98e25a6bea0c4eb69a845dba63c4de4e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:09.152832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cca3e-7fe8-4afb-87e9-57533bc2b2fe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:09.178199 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2967/5928] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:09 2022] PUT /identity/v3/projects/8f78a07fdd98414e91f6a46aff3b1111/users/98e25a6bea0c4eb69a845dba63c4de4e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:09.180267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a0f71-acd6-4fb5-a7b2-953cab839a9d service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:09.200166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a0f71-acd6-4fb5-a7b2-953cab839a9d service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:09.316831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885c028-5df3-4b17-a369-38c21cba4400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:09.317918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885c028-5df3-4b17-a369-38c21cba4400 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:09.318273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885c028-5df3-4b17-a369-38c21cba4400 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:09.318618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885c028-5df3-4b17-a369-38c21cba4400 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:09.319403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885c028-5df3-4b17-a369-38c21cba4400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:09.321191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885c028-5df3-4b17-a369-38c21cba4400 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:09.338265 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2962/5929] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:09.339861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cca3e-7fe8-4afb-87e9-57533bc2b2fe admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:09.357520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cca3e-7fe8-4afb-87e9-57533bc2b2fe admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:09.366078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4b0b6-4824-48e8-8936-a5220ecef164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:09.367155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4b0b6-4824-48e8-8936-a5220ecef164 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:09.367640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4b0b6-4824-48e8-8936-a5220ecef164 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:09.368140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4b0b6-4824-48e8-8936-a5220ecef164 admin admin] PATH_INFO: `/v3/projects/8f78a07fdd98414e91f6a46aff3b1111/users/98e25a6bea0c4eb69a845dba63c4de4e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:09.370742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4b0b6-4824-48e8-8936-a5220ecef164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f78a07fdd98414e91f6a46aff3b1111, user_id=98e25a6bea0c4eb69a845dba63c4de4e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:09.375950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4b0b6-4824-48e8-8936-a5220ecef164 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:09.405266 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2968/5930] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:09 2022] PUT /identity/v3/projects/8f78a07fdd98414e91f6a46aff3b1111/users/98e25a6bea0c4eb69a845dba63c4de4e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:09.407618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef794d-159c-48b4-a98e-0e544f8992e9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:09.408264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef794d-159c-48b4-a98e-0e544f8992e9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:09.408539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef794d-159c-48b4-a98e-0e544f8992e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:09.499791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-efef794d-159c-48b4-a98e-0e544f8992e9 None None] MFA Rules not processed for user `98e25a6bea0c4eb69a845dba63c4de4e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:09.502858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-efef794d-159c-48b4-a98e-0e544f8992e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:09.639990 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2963/5931] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:09.645181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4b0b6-4824-48e8-8936-a5220ecef164 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:09.678275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4b0b6-4824-48e8-8936-a5220ecef164 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:09.684538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c377e3a-c5f6-487a-8024-27099ae6b0f9 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:09.685954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c377e3a-c5f6-487a-8024-27099ae6b0f9 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:09.686518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c377e3a-c5f6-487a-8024-27099ae6b0f9 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:09.687019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c377e3a-c5f6-487a-8024-27099ae6b0f9 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:09.688049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c377e3a-c5f6-487a-8024-27099ae6b0f9 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:09.690290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c377e3a-c5f6-487a-8024-27099ae6b0f9 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:09.711442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:09.724546 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2969/5932] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:09 2022] POST /identity/v3/services => generated 285 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:09.730348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef794d-159c-48b4-a98e-0e544f8992e9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:09.750781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef794d-159c-48b4-a98e-0e544f8992e9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:09.784383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231487f2-7ec8-4634-af28-a027951e6fc0 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:09.789924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231487f2-7ec8-4634-af28-a027951e6fc0 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:09.791705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231487f2-7ec8-4634-af28-a027951e6fc0 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:09.792325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231487f2-7ec8-4634-af28-a027951e6fc0 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:09.793221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231487f2-7ec8-4634-af28-a027951e6fc0 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:09.795019 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-231487f2-7ec8-4634-af28-a027951e6fc0 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:14:09.795019 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:14:09.795019 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:14:09.796173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231487f2-7ec8-4634-af28-a027951e6fc0 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:09.825117 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2964/5933] 10.222.0.16 () {60 vars in 1213 bytes} [Sat Oct 15 21:14:09 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:09.827435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c377e3a-c5f6-487a-8024-27099ae6b0f9 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:09.843514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c377e3a-c5f6-487a-8024-27099ae6b0f9 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:09.850488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa279e-f3fc-4a56-be7f-0ce81188f2c8 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:09.853712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa279e-f3fc-4a56-be7f-0ce81188f2c8 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:09.854504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa279e-f3fc-4a56-be7f-0ce81188f2c8 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:09.855632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa279e-f3fc-4a56-be7f-0ce81188f2c8 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:09.864427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa279e-f3fc-4a56-be7f-0ce81188f2c8 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:09.864427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa279e-f3fc-4a56-be7f-0ce81188f2c8 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:09.905424 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2970/5934] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:09 2022] POST /identity/v3/services => generated 289 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:09.909609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231487f2-7ec8-4634-af28-a027951e6fc0 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:09.924351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231487f2-7ec8-4634-af28-a027951e6fc0 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:09.950262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec02f2d-0ee3-42b5-85ac-1a9f17c091ed tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:09.951461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec02f2d-0ee3-42b5-85ac-1a9f17c091ed tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:09.951987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec02f2d-0ee3-42b5-85ac-1a9f17c091ed tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:09.966862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec02f2d-0ee3-42b5-85ac-1a9f17c091ed tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:09.966862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec02f2d-0ee3-42b5-85ac-1a9f17c091ed tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:09.966862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec02f2d-0ee3-42b5-85ac-1a9f17c091ed tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:09.975899 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2965/5935] 10.222.0.16 () {60 vars in 1213 bytes} [Sat Oct 15 21:14:09 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 332 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:09.980140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa279e-f3fc-4a56-be7f-0ce81188f2c8 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:09.995152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa279e-f3fc-4a56-be7f-0ce81188f2c8 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.003494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c63be2-4b22-490a-8def-fb978451b763 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.005234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63be2-4b22-490a-8def-fb978451b763 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.005234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63be2-4b22-490a-8def-fb978451b763 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.005960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63be2-4b22-490a-8def-fb978451b763 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.006683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c63be2-4b22-490a-8def-fb978451b763 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.007830 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-44c63be2-4b22-490a-8def-fb978451b763 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:14:10.007830 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:14:10.007830 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:14:10.009138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c63be2-4b22-490a-8def-fb978451b763 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.018718 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2971/5936] 10.222.0.16 () {58 vars in 1191 bytes} [Sat Oct 15 21:14:09 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 763 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:14:10.022759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec02f2d-0ee3-42b5-85ac-1a9f17c091ed tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:10.042311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec02f2d-0ee3-42b5-85ac-1a9f17c091ed tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.057085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c41a20-4c07-4ccc-99f9-f381d82e7b44 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.057920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c41a20-4c07-4ccc-99f9-f381d82e7b44 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.059288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c41a20-4c07-4ccc-99f9-f381d82e7b44 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.059288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c41a20-4c07-4ccc-99f9-f381d82e7b44 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/3e210158a323493195e0306c7c9ed2c5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.061230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c41a20-4c07-4ccc-99f9-f381d82e7b44 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=3e210158a323493195e0306c7c9ed2c5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.062781 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-55c41a20-4c07-4ccc-99f9-f381d82e7b44 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:14:10.062781 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:14:10.062781 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:14:10.066453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c41a20-4c07-4ccc-99f9-f381d82e7b44 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.083934 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2966/5937] 10.222.0.16 () {58 vars in 1290 bytes} [Sat Oct 15 21:14:10 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups/3e210158a323493195e0306c7c9ed2c5 => generated 332 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:10.085129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c63be2-4b22-490a-8def-fb978451b763 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:10.106994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c63be2-4b22-490a-8def-fb978451b763 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.120814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1fd94-2c21-4237-848e-01f212a7e186 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.121116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1fd94-2c21-4237-848e-01f212a7e186 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.121388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1fd94-2c21-4237-848e-01f212a7e186 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.121565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1fd94-2c21-4237-848e-01f212a7e186 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/3e210158a323493195e0306c7c9ed2c5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.122687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1fd94-2c21-4237-848e-01f212a7e186 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=3e210158a323493195e0306c7c9ed2c5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.123740 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-d4e1fd94-2c21-4237-848e-01f212a7e186 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:14:10.123740 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:14:10.123740 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:14:10.124892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1fd94-2c21-4237-848e-01f212a7e186 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.143848 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2972/5938] 10.222.0.16 () {58 vars in 1291 bytes} [Sat Oct 15 21:14:10 2022] HEAD /identity/v3/OS-EP-FILTER/endpoint_groups/3e210158a323493195e0306c7c9ed2c5 => generated 0 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Oct 15 21:14:10.149606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c41a20-4c07-4ccc-99f9-f381d82e7b44 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:10.171074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c41a20-4c07-4ccc-99f9-f381d82e7b44 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.194871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e17de-3c93-451b-9792-49983a58e4f2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.201409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e17de-3c93-451b-9792-49983a58e4f2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.205416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e17de-3c93-451b-9792-49983a58e4f2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.205949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e17de-3c93-451b-9792-49983a58e4f2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/3e210158a323493195e0306c7c9ed2c5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.206803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e17de-3c93-451b-9792-49983a58e4f2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=3e210158a323493195e0306c7c9ed2c5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.208050 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-d69e17de-3c93-451b-9792-49983a58e4f2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:14:10.208050 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:14:10.208050 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:14:10.209674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e17de-3c93-451b-9792-49983a58e4f2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.228661 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2967/5939] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 21:14:10 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/3e210158a323493195e0306c7c9ed2c5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:10.229167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1fd94-2c21-4237-848e-01f212a7e186 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:10.241072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1fd94-2c21-4237-848e-01f212a7e186 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.254152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d99dc-3849-4f66-8bac-d7005f637275 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.254848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d99dc-3849-4f66-8bac-d7005f637275 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.255319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d99dc-3849-4f66-8bac-d7005f637275 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.255676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d99dc-3849-4f66-8bac-d7005f637275 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.256683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d99dc-3849-4f66-8bac-d7005f637275 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.266890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d99dc-3849-4f66-8bac-d7005f637275 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.281699 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2973/5940] 10.222.0.16 () {58 vars in 1191 bytes} [Sat Oct 15 21:14:10 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 450 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:10.282653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e17de-3c93-451b-9792-49983a58e4f2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:10.304819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e17de-3c93-451b-9792-49983a58e4f2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.317723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c40755-1b30-4397-8d73-5725e48182e2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.319250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c40755-1b30-4397-8d73-5725e48182e2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.320460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c40755-1b30-4397-8d73-5725e48182e2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.322697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c40755-1b30-4397-8d73-5725e48182e2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/3e210158a323493195e0306c7c9ed2c5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.323611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c40755-1b30-4397-8d73-5725e48182e2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=3e210158a323493195e0306c7c9ed2c5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.330905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c40755-1b30-4397-8d73-5725e48182e2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-70c40755-1b30-4397-8d73-5725e48182e2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Could not find Endpoint Group: 3e210158a323493195e0306c7c9ed2c5.: keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 3e210158a323493195e0306c7c9ed2c5. Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_ep_filter.py", line 107, in delete Oct 15 21:14:10.354217 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return (PROVIDERS.catalog_api.delete_endpoint_group(endpoint_group_id), Oct 15 21:14:10.355461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 519, in delete_endpoint_group Oct 15 21:14:10.355461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application endpoint_group_ref = self._get_endpoint_group(session, Oct 15 21:14:10.355461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 495, in _get_endpoint_group Oct 15 21:14:10.355461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.EndpointGroupNotFound( Oct 15 21:14:10.355461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 3e210158a323493195e0306c7c9ed2c5. Oct 15 21:14:10.355461 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:14:10.359067 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2968/5941] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 21:14:10 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/3e210158a323493195e0306c7c9ed2c5 => generated 120 bytes in 78 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:14:10.363844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d99dc-3849-4f66-8bac-d7005f637275 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:10.382679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d99dc-3849-4f66-8bac-d7005f637275 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.389752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1eefe-dc02-41eb-8e4c-7ab00e54ff82 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.390612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1eefe-dc02-41eb-8e4c-7ab00e54ff82 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.392501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1eefe-dc02-41eb-8e4c-7ab00e54ff82 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.392790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1eefe-dc02-41eb-8e4c-7ab00e54ff82 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/services/db2d3b44f194454aa839411606fba0c8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.393589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1eefe-dc02-41eb-8e4c-7ab00e54ff82 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:delete_service(service_id=db2d3b44f194454aa839411606fba0c8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.396328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1eefe-dc02-41eb-8e4c-7ab00e54ff82 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.473031 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2974/5942] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:10 2022] DELETE /identity/v3/services/db2d3b44f194454aa839411606fba0c8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:10.484154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c40755-1b30-4397-8d73-5725e48182e2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:10.523826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c40755-1b30-4397-8d73-5725e48182e2 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.612679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc354df1-4a8c-419a-b26a-de5c55147422 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.614114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc354df1-4a8c-419a-b26a-de5c55147422 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.614690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc354df1-4a8c-419a-b26a-de5c55147422 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.615140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc354df1-4a8c-419a-b26a-de5c55147422 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.616381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc354df1-4a8c-419a-b26a-de5c55147422 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.624637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc354df1-4a8c-419a-b26a-de5c55147422 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.633705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:10.694686 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2969/5943] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:10 2022] POST /identity/v3/services => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:10.700317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1eefe-dc02-41eb-8e4c-7ab00e54ff82 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:10.738968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1eefe-dc02-41eb-8e4c-7ab00e54ff82 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.780293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f10fa-8b98-484c-a90d-d51ebba4e993 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.780974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f10fa-8b98-484c-a90d-d51ebba4e993 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.781371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f10fa-8b98-484c-a90d-d51ebba4e993 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.781823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f10fa-8b98-484c-a90d-d51ebba4e993 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.782495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f10fa-8b98-484c-a90d-d51ebba4e993 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.784046 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-a57f10fa-8b98-484c-a90d-d51ebba4e993 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:14:10.784046 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:14:10.784046 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:14:10.786098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f10fa-8b98-484c-a90d-d51ebba4e993 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.812898 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2975/5944] 10.222.0.16 () {60 vars in 1213 bytes} [Sat Oct 15 21:14:10 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:10.817759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc354df1-4a8c-419a-b26a-de5c55147422 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:10.839103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc354df1-4a8c-419a-b26a-de5c55147422 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.847716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9fc0-5a30-4286-a76c-a6d487d1c9a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.848122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9fc0-5a30-4286-a76c-a6d487d1c9a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.848492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9fc0-5a30-4286-a76c-a6d487d1c9a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.848875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9fc0-5a30-4286-a76c-a6d487d1c9a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.849694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd9fc0-5a30-4286-a76c-a6d487d1c9a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.851479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd9fc0-5a30-4286-a76c-a6d487d1c9a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.871915 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2970/5945] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:10 2022] POST /identity/v3/services => generated 290 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:10.877308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f10fa-8b98-484c-a90d-d51ebba4e993 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:10.891213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f10fa-8b98-484c-a90d-d51ebba4e993 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:10.934003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd9da6-af01-487b-87b3-b070b1be7038 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:10.935620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd9da6-af01-487b-87b3-b070b1be7038 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:10.935964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd9da6-af01-487b-87b3-b070b1be7038 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:10.936301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd9da6-af01-487b-87b3-b070b1be7038 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/4931e4e25c6a4f20b1a35d1ba8cfa136` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:10.937098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd9da6-af01-487b-87b3-b070b1be7038 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:update_endpoint_group(endpoint_group_id=4931e4e25c6a4f20b1a35d1ba8cfa136)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:10.938376 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-25cd9da6-af01-487b-87b3-b070b1be7038 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:14:10.938376 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:14:10.938376 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:14:10.939450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd9da6-af01-487b-87b3-b070b1be7038 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:10.988184 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2976/5946] 10.222.0.16 () {60 vars in 1313 bytes} [Sat Oct 15 21:14:10 2022] PATCH /identity/v3/OS-EP-FILTER/endpoint_groups/4931e4e25c6a4f20b1a35d1ba8cfa136 => generated 336 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:10.990212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9fc0-5a30-4286-a76c-a6d487d1c9a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:11.004753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9fc0-5a30-4286-a76c-a6d487d1c9a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:11.010692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd9da6-af01-487b-87b3-b070b1be7038 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:11.012433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-25cd9da6-af01-487b-87b3-b070b1be7038 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:11.016700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d940351-db1a-4583-8cf0-eaeec68f5bf1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:11.017660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940351-db1a-4583-8cf0-eaeec68f5bf1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:11.018265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940351-db1a-4583-8cf0-eaeec68f5bf1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:11.019657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940351-db1a-4583-8cf0-eaeec68f5bf1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/services/7cb94928e542459fa57d54ab6f17bb2a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:11.020502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d940351-db1a-4583-8cf0-eaeec68f5bf1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:delete_service(service_id=7cb94928e542459fa57d54ab6f17bb2a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:11.022556 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-0d940351-db1a-4583-8cf0-eaeec68f5bf1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:14:11.022556 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:14:11.022556 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:14:11.024054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d940351-db1a-4583-8cf0-eaeec68f5bf1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:11.060394 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2971/5947] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:10 2022] DELETE /identity/v3/services/7cb94928e542459fa57d54ab6f17bb2a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:11.064587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d940351-db1a-4583-8cf0-eaeec68f5bf1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:11.094187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d940351-db1a-4583-8cf0-eaeec68f5bf1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:11.107062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01abb86-c88f-409e-8062-fa35a76b79a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:11.109205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01abb86-c88f-409e-8062-fa35a76b79a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:11.109917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01abb86-c88f-409e-8062-fa35a76b79a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:11.111014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01abb86-c88f-409e-8062-fa35a76b79a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/4931e4e25c6a4f20b1a35d1ba8cfa136` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:11.112675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01abb86-c88f-409e-8062-fa35a76b79a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=4931e4e25c6a4f20b1a35d1ba8cfa136)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:11.114875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01abb86-c88f-409e-8062-fa35a76b79a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:11.138016 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2972/5948] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 21:14:11 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/4931e4e25c6a4f20b1a35d1ba8cfa136 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:11.143068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01abb86-c88f-409e-8062-fa35a76b79a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:11.146512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd9da6-af01-487b-87b3-b070b1be7038 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:11.167068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01abb86-c88f-409e-8062-fa35a76b79a1 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:11.176760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:11.178052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:11.178336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:11.178610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] PATH_INFO: `/v3/users/83ef2c6c30454b6c8ddfaa42858c91bb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:11.180064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11d165-c7df-4d60-bc33-9080799c5970 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:11.180287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ef2c6c30454b6c8ddfaa42858c91bb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:11.181348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11d165-c7df-4d60-bc33-9080799c5970 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:11.182808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:11.182959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11d165-c7df-4d60-bc33-9080799c5970 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:11.183318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11d165-c7df-4d60-bc33-9080799c5970 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/services/e7b377398c5c4e5a886921aed24f58dd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:11.184912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11d165-c7df-4d60-bc33-9080799c5970 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:delete_service(service_id=e7b377398c5c4e5a886921aed24f58dd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:11.187738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11d165-c7df-4d60-bc33-9080799c5970 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:11.240122 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2973/5949] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:11 2022] DELETE /identity/v3/services/e7b377398c5c4e5a886921aed24f58dd => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:11.247218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11d165-c7df-4d60-bc33-9080799c5970 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:11.271958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=8,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:11.277552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11d165-c7df-4d60-bc33-9080799c5970 tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:11.279066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ef2c6c30454b6c8ddfaa42858c91bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:11.285006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4184b1-2645-4f93-b82f-854bba86a2be tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:11.285710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4184b1-2645-4f93-b82f-854bba86a2be tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:11.285995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4184b1-2645-4f93-b82f-854bba86a2be tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:11.286291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4184b1-2645-4f93-b82f-854bba86a2be tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/fe9154f6d1f140bb91483686202eec6a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:11.287008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4184b1-2645-4f93-b82f-854bba86a2be tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=fe9154f6d1f140bb91483686202eec6a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:11.288869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4184b1-2645-4f93-b82f-854bba86a2be tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:11.309582 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2974/5950] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 21:14:11 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/fe9154f6d1f140bb91483686202eec6a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:11.309801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ef2c6c30454b6c8ddfaa42858c91bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:11.310964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4184b1-2645-4f93-b82f-854bba86a2be tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:11.328915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4184b1-2645-4f93-b82f-854bba86a2be tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:11.339290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ef2c6c30454b6c8ddfaa42858c91bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:11.363901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ef2c6c30454b6c8ddfaa42858c91bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:11.365168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78d75b-7681-4e4c-bbf2-6d0e487009ea tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e25a6bea0c4eb69a845dba63c4de4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f78a07fdd98414e91f6a46aff3b1111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:11.365727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7e9841a-ad71-410f-a2b0-ff5526554ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ef2c6c30454b6c8ddfaa42858c91bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:11.368279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78d75b-7681-4e4c-bbf2-6d0e487009ea tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:11.368571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78d75b-7681-4e4c-bbf2-6d0e487009ea tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:11.368839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78d75b-7681-4e4c-bbf2-6d0e487009ea tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] PATH_INFO: `/v3/services/f8b00dabdfaa483baf79e43142c2a2fb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:11.370319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78d75b-7681-4e4c-bbf2-6d0e487009ea tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorizing `identity:delete_service(service_id=f8b00dabdfaa483baf79e43142c2a2fb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:11.371975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78d75b-7681-4e4c-bbf2-6d0e487009ea tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:11.384219 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2977/5951] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:11 2022] DELETE /identity/v3/users/83ef2c6c30454b6c8ddfaa42858c91bb => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:11.386033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78590-8c2a-434b-bc95-450bc639c14f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:11.386320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78590-8c2a-434b-bc95-450bc639c14f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:11.386591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78590-8c2a-434b-bc95-450bc639c14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:11.439666 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2975/5952] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:11 2022] DELETE /identity/v3/services/f8b00dabdfaa483baf79e43142c2a2fb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:11.446188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78d75b-7681-4e4c-bbf2-6d0e487009ea tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:11.448088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fe78d75b-7681-4e4c-bbf2-6d0e487009ea tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:11.453158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5de78590-8c2a-434b-bc95-450bc639c14f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:11.455934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5de78590-8c2a-434b-bc95-450bc639c14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:11.525637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78d75b-7681-4e4c-bbf2-6d0e487009ea tempest-EndPointGroupsTest-174670987 tempest-EndPointGroupsTest-174670987-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:11.595085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2978/5953] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:11.618113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78590-8c2a-434b-bc95-450bc639c14f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:11.626259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5de78590-8c2a-434b-bc95-450bc639c14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:11.642506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:11.643174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:11.643454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:11.643730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] PATH_INFO: `/v3/users/074aa9aee2de4aa1a60a429921789b3f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:11.645615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=074aa9aee2de4aa1a60a429921789b3f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:11.649192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:11.706300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:11.780471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '074aa9aee2de4aa1a60a429921789b3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:11.792222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78590-8c2a-434b-bc95-450bc639c14f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:11.795064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '074aa9aee2de4aa1a60a429921789b3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:11.798544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececf543-89f6-4e30-bad7-aae03c7242c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:11.799587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececf543-89f6-4e30-bad7-aae03c7242c3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:11.800345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececf543-89f6-4e30-bad7-aae03c7242c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:11.800864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececf543-89f6-4e30-bad7-aae03c7242c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:11.826376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '074aa9aee2de4aa1a60a429921789b3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:11.827786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececf543-89f6-4e30-bad7-aae03c7242c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:11.829866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececf543-89f6-4e30-bad7-aae03c7242c3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:11.844094 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2979/5954] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:11.846585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '074aa9aee2de4aa1a60a429921789b3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:11.847515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '074aa9aee2de4aa1a60a429921789b3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:11.867724 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2976/5955] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:11 2022] DELETE /identity/v3/users/074aa9aee2de4aa1a60a429921789b3f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:11.869759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a5ab8-6ed5-494b-9801-ea311c44e937 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:11.870135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a5ab8-6ed5-494b-9801-ea311c44e937 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:11.870525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a5ab8-6ed5-494b-9801-ea311c44e937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:11.894628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ba9a5ab8-6ed5-494b-9801-ea311c44e937 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:11.896211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ba9a5ab8-6ed5-494b-9801-ea311c44e937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:11.964729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:11.985995 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2980/5956] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:14:12.021704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:12.026398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:12.175646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dbabf-df39-4293-adc0-b3ff0c7e7fd1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:12.181753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17814bb-f02a-459a-ad9e-7348a62df85e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:12.182493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17814bb-f02a-459a-ad9e-7348a62df85e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:12.182799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17814bb-f02a-459a-ad9e-7348a62df85e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:12.183146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17814bb-f02a-459a-ad9e-7348a62df85e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:12.206994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17814bb-f02a-459a-ad9e-7348a62df85e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:12.210063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17814bb-f02a-459a-ad9e-7348a62df85e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:12.244249 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2977/5957] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:13.217248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a5ab8-6ed5-494b-9801-ea311c44e937 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:13.219615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ba9a5ab8-6ed5-494b-9801-ea311c44e937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:13.241695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a5ab8-6ed5-494b-9801-ea311c44e937 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:13.249648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:13.250545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:13.251002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:13.251481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] PATH_INFO: `/v3/projects/32906f5db21341828c064548ddc93b11` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:13.252807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32906f5db21341828c064548ddc93b11)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:13.255017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:13.355224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32906f5db21341828c064548ddc93b11', 'request_id': 'req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.369665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32906f5db21341828c064548ddc93b11', 'request_id': 'req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.379102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32906f5db21341828c064548ddc93b11', 'request_id': 'req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.395649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] The token cache is being invalidate because project 32906f5db21341828c064548ddc93b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:13.395948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.396774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.399460 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2981/5958] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:13 2022] DELETE /identity/v3/projects/32906f5db21341828c064548ddc93b11 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:13.404358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17814bb-f02a-459a-ad9e-7348a62df85e service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:13.422644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17814bb-f02a-459a-ad9e-7348a62df85e service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:13.546564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:13.547243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:13.547598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:13.548112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] PATH_INFO: `/v3/users/c949612a037348b5a72b90022d9d20eb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:13.549292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c949612a037348b5a72b90022d9d20eb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:13.551315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:13.622813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:13.634256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c949612a037348b5a72b90022d9d20eb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.646285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c949612a037348b5a72b90022d9d20eb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.676697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c949612a037348b5a72b90022d9d20eb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.707238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c949612a037348b5a72b90022d9d20eb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.708046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c949612a037348b5a72b90022d9d20eb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.724015 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2978/5959] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:13 2022] DELETE /identity/v3/users/c949612a037348b5a72b90022d9d20eb => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:13.725087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:13.726755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:13.756832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97cc70-e99f-4aec-bd8b-2f7d4c133db3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:13.882854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:13.883578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:13.883860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:13.884194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] PATH_INFO: `/v3/projects/0229134d004a4a41a7f7f4bf628a7858` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:13.887647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0229134d004a4a41a7f7f4bf628a7858)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:13.888026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:13.971492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0229134d004a4a41a7f7f4bf628a7858', 'request_id': 'req-eb65897d-6a0a-40ef-8213-48e9f6f9161c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.980002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0229134d004a4a41a7f7f4bf628a7858', 'request_id': 'req-eb65897d-6a0a-40ef-8213-48e9f6f9161c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:13.987581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0229134d004a4a41a7f7f4bf628a7858', 'request_id': 'req-eb65897d-6a0a-40ef-8213-48e9f6f9161c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:14.001388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] The token cache is being invalidate because project 0229134d004a4a41a7f7f4bf628a7858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:14.001655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:14.002386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:14.004172 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2982/5960] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:13 2022] DELETE /identity/v3/projects/0229134d004a4a41a7f7f4bf628a7858 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:14.007987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:14.009950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:14.023669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa681d-937a-4293-a7ee-96a69e357d9d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:14.096160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:14.096892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:14.097276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:14.097666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] PATH_INFO: `/v3/users/98e25a6bea0c4eb69a845dba63c4de4e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:14.098963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e25a6bea0c4eb69a845dba63c4de4e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:14.100989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:14.168024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e25a6bea0c4eb69a845dba63c4de4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:14.178069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e25a6bea0c4eb69a845dba63c4de4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:14.196558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e25a6bea0c4eb69a845dba63c4de4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:14.211227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e25a6bea0c4eb69a845dba63c4de4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:14.212990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bd16c04a-211c-42cd-8d56-f7fc8bab86dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e25a6bea0c4eb69a845dba63c4de4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:14.224275 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2979/5961] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:14 2022] DELETE /identity/v3/users/98e25a6bea0c4eb69a845dba63c4de4e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:14.768748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:14.770935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:14.793555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65897d-6a0a-40ef-8213-48e9f6f9161c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:14.806082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:14.906917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:14.907554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:14.907822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:14.908084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] PATH_INFO: `/v3/projects/8f78a07fdd98414e91f6a46aff3b1111` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:14.909373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f78a07fdd98414e91f6a46aff3b1111)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:14.911733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:15.000051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f78a07fdd98414e91f6a46aff3b1111', 'request_id': 'req-bff780ce-7c70-4449-8a88-dec059aee822'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:15.010608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f78a07fdd98414e91f6a46aff3b1111', 'request_id': 'req-bff780ce-7c70-4449-8a88-dec059aee822'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:15.020711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f78a07fdd98414e91f6a46aff3b1111', 'request_id': 'req-bff780ce-7c70-4449-8a88-dec059aee822'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:15.034755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] The token cache is being invalidate because project 8f78a07fdd98414e91f6a46aff3b1111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:15.035077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:15.035762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bff780ce-7c70-4449-8a88-dec059aee822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:15.037633 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2983/5962] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:14 2022] DELETE /identity/v3/projects/8f78a07fdd98414e91f6a46aff3b1111 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:15.044449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24ac4f-064a-46f6-a080-854d3c6a98e7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:15.045143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24ac4f-064a-46f6-a080-854d3c6a98e7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:15.045143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24ac4f-064a-46f6-a080-854d3c6a98e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:15.093437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5d24ac4f-064a-46f6-a080-854d3c6a98e7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:15.095094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5d24ac4f-064a-46f6-a080-854d3c6a98e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:15.142807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:15.179401 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2980/5963] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:15.188571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23764ad9-5569-432f-9f87-c3afb529e9c8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:15.188863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23764ad9-5569-432f-9f87-c3afb529e9c8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:15.189163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23764ad9-5569-432f-9f87-c3afb529e9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:15.214912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-23764ad9-5569-432f-9f87-c3afb529e9c8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:15.216832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-23764ad9-5569-432f-9f87-c3afb529e9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:15.229746 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2984/5964] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:15.230979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24ac4f-064a-46f6-a080-854d3c6a98e7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:15.242964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24ac4f-064a-46f6-a080-854d3c6a98e7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:15.250686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bab27e-bb6c-49e6-92b5-9b01a0b7c680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:15.251407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bab27e-bb6c-49e6-92b5-9b01a0b7c680 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:15.251862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bab27e-bb6c-49e6-92b5-9b01a0b7c680 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:15.252222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bab27e-bb6c-49e6-92b5-9b01a0b7c680 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:15.252965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bab27e-bb6c-49e6-92b5-9b01a0b7c680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:15.253389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bab27e-bb6c-49e6-92b5-9b01a0b7c680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:15.255071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bab27e-bb6c-49e6-92b5-9b01a0b7c680 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:15.275415 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2981/5965] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:15.277112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23764ad9-5569-432f-9f87-c3afb529e9c8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:15.288912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23764ad9-5569-432f-9f87-c3afb529e9c8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:15.297263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6eb2f-d205-4ee8-9bb4-fd3de22943b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:15.297263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6eb2f-d205-4ee8-9bb4-fd3de22943b4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:15.297263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6eb2f-d205-4ee8-9bb4-fd3de22943b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:15.297707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6eb2f-d205-4ee8-9bb4-fd3de22943b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:15.298179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6eb2f-d205-4ee8-9bb4-fd3de22943b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:15.300470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6eb2f-d205-4ee8-9bb4-fd3de22943b4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:15.365340 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2985/5966] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:15 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:15.367410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bab27e-bb6c-49e6-92b5-9b01a0b7c680 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:15.381929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bab27e-bb6c-49e6-92b5-9b01a0b7c680 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:15.456036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaaf584-fe02-458f-a01d-9736989bbd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:15.456934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaaf584-fe02-458f-a01d-9736989bbd35 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:15.458136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaaf584-fe02-458f-a01d-9736989bbd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:15.458136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaaf584-fe02-458f-a01d-9736989bbd35 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:15.458347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaaf584-fe02-458f-a01d-9736989bbd35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:15.460172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaaf584-fe02-458f-a01d-9736989bbd35 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:15.499204 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2982/5967] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:15 2022] POST /identity/v3/users => generated 286 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:15.499720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6eb2f-d205-4ee8-9bb4-fd3de22943b4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:15.520184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6eb2f-d205-4ee8-9bb4-fd3de22943b4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:15.535779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce8dd3-5153-46b8-b41b-f46404f9f54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:15.535779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce8dd3-5153-46b8-b41b-f46404f9f54b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:15.535779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce8dd3-5153-46b8-b41b-f46404f9f54b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:15.535779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce8dd3-5153-46b8-b41b-f46404f9f54b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:15.536291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce8dd3-5153-46b8-b41b-f46404f9f54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:15.536291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce8dd3-5153-46b8-b41b-f46404f9f54b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:15.562645 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2986/5968] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:15.565186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaaf584-fe02-458f-a01d-9736989bbd35 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:15.586564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaaf584-fe02-458f-a01d-9736989bbd35 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:15.593963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478f674-e413-4e64-8fd0-4c72e94f664f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:15.594880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478f674-e413-4e64-8fd0-4c72e94f664f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:15.595154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478f674-e413-4e64-8fd0-4c72e94f664f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:15.595544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478f674-e413-4e64-8fd0-4c72e94f664f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:15.596403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478f674-e413-4e64-8fd0-4c72e94f664f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:15.598613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478f674-e413-4e64-8fd0-4c72e94f664f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:15.625099 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2983/5969] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:15.626968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce8dd3-5153-46b8-b41b-f46404f9f54b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:15.650038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce8dd3-5153-46b8-b41b-f46404f9f54b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:15.658054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae6b0a-6789-43f2-aef2-03129c05d1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:15.659671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae6b0a-6789-43f2-aef2-03129c05d1f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:15.660095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae6b0a-6789-43f2-aef2-03129c05d1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:15.660393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae6b0a-6789-43f2-aef2-03129c05d1f1 admin admin] PATH_INFO: `/v3/projects/06c072c77fa840bf96dec81883b4ffab/users/9950cb2a38db441c83b4797b883cedc5/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:15.697716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae6b0a-6789-43f2-aef2-03129c05d1f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c072c77fa840bf96dec81883b4ffab, user_id=9950cb2a38db441c83b4797b883cedc5, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:15.714010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae6b0a-6789-43f2-aef2-03129c05d1f1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:15.719966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478f674-e413-4e64-8fd0-4c72e94f664f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:15.721928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8478f674-e413-4e64-8fd0-4c72e94f664f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:15.756714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478f674-e413-4e64-8fd0-4c72e94f664f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:15.765264 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2987/5970] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:15 2022] PUT /identity/v3/projects/06c072c77fa840bf96dec81883b4ffab/users/9950cb2a38db441c83b4797b883cedc5/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:15.777551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a82a0-aa90-4f37-b512-f6f34294f81e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:15.778197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a82a0-aa90-4f37-b512-f6f34294f81e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:15.779043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a82a0-aa90-4f37-b512-f6f34294f81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:15.832372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fa9a82a0-aa90-4f37-b512-f6f34294f81e None None] MFA Rules not processed for user `9950cb2a38db441c83b4797b883cedc5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:15.834326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fa9a82a0-aa90-4f37-b512-f6f34294f81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:15.849875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:15.850725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:15.851147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:15.851517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] PATH_INFO: `/v3/projects/54e39712c0ef4944a51366c629f7ad3f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:15.852975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e39712c0ef4944a51366c629f7ad3f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:15.855091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:15.893726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:15.942372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:15.956852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e39712c0ef4944a51366c629f7ad3f', 'request_id': 'req-d6c6d89f-6021-4c57-85d5-6c92c5268dda'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:15.976650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e39712c0ef4944a51366c629f7ad3f', 'request_id': 'req-d6c6d89f-6021-4c57-85d5-6c92c5268dda'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:15.985002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e39712c0ef4944a51366c629f7ad3f', 'request_id': 'req-d6c6d89f-6021-4c57-85d5-6c92c5268dda'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:15.999927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] The token cache is being invalidate because project 54e39712c0ef4944a51366c629f7ad3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:16.000124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:16.000869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:16.006330 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2984/5972] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:15 2022] DELETE /identity/v3/projects/54e39712c0ef4944a51366c629f7ad3f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:16.006509 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2988/5972] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:16.008636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238c9fd-0f9d-4c01-acb9-b83b930c3f63 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.009118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238c9fd-0f9d-4c01-acb9-b83b930c3f63 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.009394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238c9fd-0f9d-4c01-acb9-b83b930c3f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.011989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.015840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:16.053158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8238c9fd-0f9d-4c01-acb9-b83b930c3f63 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:16.054446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8238c9fd-0f9d-4c01-acb9-b83b930c3f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:16.164333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6d89f-6021-4c57-85d5-6c92c5268dda admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.175029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5a731-e872-44bd-b13a-18aa973981be tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9950cb2a38db441c83b4797b883cedc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06c072c77fa840bf96dec81883b4ffab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.175687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a731-e872-44bd-b13a-18aa973981be tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.175959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a731-e872-44bd-b13a-18aa973981be tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.176773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a731-e872-44bd-b13a-18aa973981be tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5/application_credentials` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.177478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5a731-e872-44bd-b13a-18aa973981be tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorizing `identity:create_application_credential(user_id=9950cb2a38db441c83b4797b883cedc5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.180797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5a731-e872-44bd-b13a-18aa973981be tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.210678 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2989/5973] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:16.214141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1454b-c77b-4514-b625-1ffa4b4b2c26 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.214419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1454b-c77b-4514-b625-1ffa4b4b2c26 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.215602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1454b-c77b-4514-b625-1ffa4b4b2c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.260899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-09a1454b-c77b-4514-b625-1ffa4b4b2c26 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:16.265585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-09a1454b-c77b-4514-b625-1ffa4b4b2c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:16.288214 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2990/5974] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:16.296528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1454b-c77b-4514-b625-1ffa4b4b2c26 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.319583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1454b-c77b-4514-b625-1ffa4b4b2c26 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.335797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c1413-fbc7-414d-becd-c8ad30b90dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.336935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c1413-fbc7-414d-becd-c8ad30b90dfd admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.337502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c1413-fbc7-414d-becd-c8ad30b90dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.338163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c1413-fbc7-414d-becd-c8ad30b90dfd admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.339573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c1413-fbc7-414d-becd-c8ad30b90dfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:16.348543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c1413-fbc7-414d-becd-c8ad30b90dfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.355220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c1413-fbc7-414d-becd-c8ad30b90dfd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.375391 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2991/5975] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:16.380319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c1413-fbc7-414d-becd-c8ad30b90dfd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.387307 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2985/5976] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Oct 15 21:14:16 2022] POST /identity/v3/users/9950cb2a38db441c83b4797b883cedc5/application_credentials => generated 921 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:16.389686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5a731-e872-44bd-b13a-18aa973981be tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.406837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c1413-fbc7-414d-becd-c8ad30b90dfd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.411309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5a731-e872-44bd-b13a-18aa973981be tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.421722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7696ac-00c9-40bf-a8a1-9bc13fc7238a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.422804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7696ac-00c9-40bf-a8a1-9bc13fc7238a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.423562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b721182-8937-4064-83b3-7a3e67dacf90 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9950cb2a38db441c83b4797b883cedc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06c072c77fa840bf96dec81883b4ffab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.424423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7696ac-00c9-40bf-a8a1-9bc13fc7238a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.424881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b721182-8937-4064-83b3-7a3e67dacf90 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.425269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7696ac-00c9-40bf-a8a1-9bc13fc7238a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.425598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b721182-8937-4064-83b3-7a3e67dacf90 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.426322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7696ac-00c9-40bf-a8a1-9bc13fc7238a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.426701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b721182-8937-4064-83b3-7a3e67dacf90 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules/78083598b85d42d288814c7b4e26d6e9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.429991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b721182-8937-4064-83b3-7a3e67dacf90 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorizing `identity:delete_access_rule(user_id=9950cb2a38db441c83b4797b883cedc5, access_rule_id=78083598b85d42d288814c7b4e26d6e9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.432210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7696ac-00c9-40bf-a8a1-9bc13fc7238a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.433295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b721182-8937-4064-83b3-7a3e67dacf90 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-1b721182-8937-4064-83b3-7a3e67dacf90 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] May not delete access rule in use: keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 256, in delete_access_rule Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application session.delete(ref) Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application next(self.gen) Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application yield resource Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application next(self.gen) Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 21:14:16.490015 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise self.value Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application session.commit() Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.session.flush() Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 21:14:16.495329 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application compat.raise_( Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 577, in execute Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.dependency_processor.process_deletes(uow, states) Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 552, in process_deletes Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._synchronize( Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 610, in _synchronize Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application sync.clear(dest, self.mapper, self.prop.synchronize_pairs) Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/sync.py", line 86, in clear Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise AssertionError( Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application AssertionError: Dependency rule tried to blank-out primary key column 'application_credential_access_rule.access_rule_id' on instance '' Oct 15 21:14:16.496490 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 763, in delete Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application PROVIDERS.application_credential_api.delete_access_rule( Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:14:16.498351 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:14:16.499550 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/core.py", line 252, in delete_access_rule Oct 15 21:14:16.499550 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self.driver.delete_access_rule(access_rule_id) Oct 15 21:14:16.499550 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 258, in delete_access_rule Oct 15 21:14:16.499550 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Oct 15 21:14:16.499550 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Oct 15 21:14:16.499550 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:14:16.499550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2986/5977] 10.222.0.16 () {58 vars in 1362 bytes} [Sat Oct 15 21:14:16 2022] DELETE /identity/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules/78083598b85d42d288814c7b4e26d6e9 => generated 89 bytes in 103 msecs (HTTP/1.1 403) 5 headers in 183 bytes (1 switches on core 0) Oct 15 21:14:16.500834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b721182-8937-4064-83b3-7a3e67dacf90 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.505401 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2992/5978] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:16 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:16.506996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7696ac-00c9-40bf-a8a1-9bc13fc7238a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.514740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b721182-8937-4064-83b3-7a3e67dacf90 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.631552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accd696-e7d2-4752-a730-99f785e836f0 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9950cb2a38db441c83b4797b883cedc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06c072c77fa840bf96dec81883b4ffab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.633464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accd696-e7d2-4752-a730-99f785e836f0 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.635052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accd696-e7d2-4752-a730-99f785e836f0 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.635623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accd696-e7d2-4752-a730-99f785e836f0 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5/application_credentials/d6462f38f75f436eb0e3519a27237237` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.645568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:16.673919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7696ac-00c9-40bf-a8a1-9bc13fc7238a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.675079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accd696-e7d2-4752-a730-99f785e836f0 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorizing `identity:delete_application_credential(user_id=9950cb2a38db441c83b4797b883cedc5, application_credential_id=d6462f38f75f436eb0e3519a27237237)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.678263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accd696-e7d2-4752-a730-99f785e836f0 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.686218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9413b9-e826-462e-ae6c-efed2dfff3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.688740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9413b9-e826-462e-ae6c-efed2dfff3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.689635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9413b9-e826-462e-ae6c-efed2dfff3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.690159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9413b9-e826-462e-ae6c-efed2dfff3e4 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.692266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9413b9-e826-462e-ae6c-efed2dfff3e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.697789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9413b9-e826-462e-ae6c-efed2dfff3e4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.735116 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2987/5979] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Oct 15 21:14:16 2022] DELETE /identity/v3/users/9950cb2a38db441c83b4797b883cedc5/application_credentials/d6462f38f75f436eb0e3519a27237237 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:16.739405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accd696-e7d2-4752-a730-99f785e836f0 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.765913 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2993/5980] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:16 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:16.767162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9413b9-e826-462e-ae6c-efed2dfff3e4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.767361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accd696-e7d2-4752-a730-99f785e836f0 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.779071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fbe9e-e992-415e-ab89-be85da95ec0d tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9950cb2a38db441c83b4797b883cedc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06c072c77fa840bf96dec81883b4ffab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.779886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fbe9e-e992-415e-ab89-be85da95ec0d tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.780169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fbe9e-e992-415e-ab89-be85da95ec0d tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.780441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fbe9e-e992-415e-ab89-be85da95ec0d tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.781452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9413b9-e826-462e-ae6c-efed2dfff3e4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.781948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fbe9e-e992-415e-ab89-be85da95ec0d tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorizing `identity:list_access_rules(user_id=9950cb2a38db441c83b4797b883cedc5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.786418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fbe9e-e992-415e-ab89-be85da95ec0d tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.787387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b5b44-907d-4917-bfe2-0d76aeace299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.788272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b5b44-907d-4917-bfe2-0d76aeace299 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.788557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b5b44-907d-4917-bfe2-0d76aeace299 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.789129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b5b44-907d-4917-bfe2-0d76aeace299 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.789818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b5b44-907d-4917-bfe2-0d76aeace299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.792674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b5b44-907d-4917-bfe2-0d76aeace299 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.801681 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2988/5981] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 21:14:16 2022] GET /identity/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules => generated 369 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:16.805871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fbe9e-e992-415e-ab89-be85da95ec0d tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.825541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fbe9e-e992-415e-ab89-be85da95ec0d tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.827606 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2994/5982] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:16.828882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b5b44-907d-4917-bfe2-0d76aeace299 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.832322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558411ad-26f0-48a6-8a51-2b98c02bb98f tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9950cb2a38db441c83b4797b883cedc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06c072c77fa840bf96dec81883b4ffab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.832994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558411ad-26f0-48a6-8a51-2b98c02bb98f tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.833299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558411ad-26f0-48a6-8a51-2b98c02bb98f tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.833569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558411ad-26f0-48a6-8a51-2b98c02bb98f tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules/78083598b85d42d288814c7b4e26d6e9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.842056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:16.846038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558411ad-26f0-48a6-8a51-2b98c02bb98f tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorizing `identity:delete_access_rule(user_id=9950cb2a38db441c83b4797b883cedc5, access_rule_id=78083598b85d42d288814c7b4e26d6e9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.847375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558411ad-26f0-48a6-8a51-2b98c02bb98f tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.852764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b5b44-907d-4917-bfe2-0d76aeace299 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.862759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b625c-6246-431d-b8fe-00bd293d91a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.863976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b625c-6246-431d-b8fe-00bd293d91a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.866063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b625c-6246-431d-b8fe-00bd293d91a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.866762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b625c-6246-431d-b8fe-00bd293d91a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.868708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b625c-6246-431d-b8fe-00bd293d91a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.873162 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2989/5983] 10.222.0.16 () {58 vars in 1362 bytes} [Sat Oct 15 21:14:16 2022] DELETE /identity/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules/78083598b85d42d288814c7b4e26d6e9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:16.902452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558411ad-26f0-48a6-8a51-2b98c02bb98f tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.902801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b625c-6246-431d-b8fe-00bd293d91a4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.903715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558411ad-26f0-48a6-8a51-2b98c02bb98f tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.913735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c4b06-59c3-43fd-ae64-b345613a3ffd tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9950cb2a38db441c83b4797b883cedc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06c072c77fa840bf96dec81883b4ffab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.914391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c4b06-59c3-43fd-ae64-b345613a3ffd tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.914668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c4b06-59c3-43fd-ae64-b345613a3ffd tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.914940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c4b06-59c3-43fd-ae64-b345613a3ffd tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.920347 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2995/5984] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:16.921676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c4b06-59c3-43fd-ae64-b345613a3ffd tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorizing `identity:list_access_rules(user_id=9950cb2a38db441c83b4797b883cedc5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.923503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c4b06-59c3-43fd-ae64-b345613a3ffd tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.924882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b625c-6246-431d-b8fe-00bd293d91a4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.936235 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2990/5985] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 21:14:16 2022] GET /identity/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules => generated 158 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:14:16.937685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b625c-6246-431d-b8fe-00bd293d91a4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.941943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c4b06-59c3-43fd-ae64-b345613a3ffd tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:16.943697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb83b3-e238-424f-b315-d9d29025cade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.944684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb83b3-e238-424f-b315-d9d29025cade admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.945340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb83b3-e238-424f-b315-d9d29025cade admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.945843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb83b3-e238-424f-b315-d9d29025cade admin admin] PATH_INFO: `/v3/projects/38d8f34110424a279a2847692017065e/users/13da28bb5f0f42938671ed8270f35513/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.962223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c4b06-59c3-43fd-ae64-b345613a3ffd tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:16.976060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b448c38-e652-4e03-8bac-f860239f4c21 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9950cb2a38db441c83b4797b883cedc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06c072c77fa840bf96dec81883b4ffab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:16.976289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b448c38-e652-4e03-8bac-f860239f4c21 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:16.976934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b448c38-e652-4e03-8bac-f860239f4c21 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:16.976934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b448c38-e652-4e03-8bac-f860239f4c21 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5/application_credentials` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:16.977367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b448c38-e652-4e03-8bac-f860239f4c21 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorizing `identity:create_application_credential(user_id=9950cb2a38db441c83b4797b883cedc5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.977963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb83b3-e238-424f-b315-d9d29025cade admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d8f34110424a279a2847692017065e, user_id=13da28bb5f0f42938671ed8270f35513, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:16.978522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b448c38-e652-4e03-8bac-f860239f4c21 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:16.981764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb83b3-e238-424f-b315-d9d29025cade admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:17.004158 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2996/5986] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:16 2022] PUT /identity/v3/projects/38d8f34110424a279a2847692017065e/users/13da28bb5f0f42938671ed8270f35513/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:17.015550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df184c-f86c-4160-9811-cc0e9e03cdfe None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:17.016008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df184c-f86c-4160-9811-cc0e9e03cdfe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:17.016284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df184c-f86c-4160-9811-cc0e9e03cdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:17.052708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-02df184c-f86c-4160-9811-cc0e9e03cdfe None None] MFA Rules not processed for user `13da28bb5f0f42938671ed8270f35513`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:17.054088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-02df184c-f86c-4160-9811-cc0e9e03cdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:17.117170 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2991/5987] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Oct 15 21:14:16 2022] POST /identity/v3/users/9950cb2a38db441c83b4797b883cedc5/application_credentials => generated 921 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:17.118047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b448c38-e652-4e03-8bac-f860239f4c21 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:17.120859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4b448c38-e652-4e03-8bac-f860239f4c21 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:17.143640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b448c38-e652-4e03-8bac-f860239f4c21 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:17.195182 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2997/5988] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:17.197906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df184c-f86c-4160-9811-cc0e9e03cdfe None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:17.236999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:17.238797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:17.239472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:17.239990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] PATH_INFO: `/v3/users/2ce795062cda4070a6d774298799b99e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:17.241609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce795062cda4070a6d774298799b99e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:17.244763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:17.293152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:17.329869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce795062cda4070a6d774298799b99e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:17.334199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df184c-f86c-4160-9811-cc0e9e03cdfe None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:17.339791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a787ddc-6a90-4bd4-b8c8-6018f3d26362 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9950cb2a38db441c83b4797b883cedc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06c072c77fa840bf96dec81883b4ffab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:17.341123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a787ddc-6a90-4bd4-b8c8-6018f3d26362 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:17.341590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce795062cda4070a6d774298799b99e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:17.341976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a787ddc-6a90-4bd4-b8c8-6018f3d26362 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:17.342673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a787ddc-6a90-4bd4-b8c8-6018f3d26362 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:17.344230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a787ddc-6a90-4bd4-b8c8-6018f3d26362 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorizing `identity:list_access_rules(user_id=9950cb2a38db441c83b4797b883cedc5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:17.346110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a787ddc-6a90-4bd4-b8c8-6018f3d26362 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:17.353487 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2998/5989] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 21:14:17 2022] GET /identity/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules => generated 369 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:17.354659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a787ddc-6a90-4bd4-b8c8-6018f3d26362 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:17.359659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5a787ddc-6a90-4bd4-b8c8-6018f3d26362 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:17.362795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce795062cda4070a6d774298799b99e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:17.373965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce795062cda4070a6d774298799b99e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:17.374846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce795062cda4070a6d774298799b99e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:17.387115 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2992/5990] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:17 2022] DELETE /identity/v3/users/2ce795062cda4070a6d774298799b99e => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:17.387115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:17.387115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:17.524245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a787ddc-6a90-4bd4-b8c8-6018f3d26362 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:17.530108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79797dd3-9142-453c-b10e-119fc1d8acd8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:17.532843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:17.542774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31c870-899b-436c-b504-c4b8a820aa77 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9950cb2a38db441c83b4797b883cedc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06c072c77fa840bf96dec81883b4ffab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:17.543512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c870-899b-436c-b504-c4b8a820aa77 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:17.544393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c870-899b-436c-b504-c4b8a820aa77 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:17.544674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c870-899b-436c-b504-c4b8a820aa77 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5/application_credentials` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:17.545631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31c870-899b-436c-b504-c4b8a820aa77 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorizing `identity:create_application_credential(user_id=9950cb2a38db441c83b4797b883cedc5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:17.548418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31c870-899b-436c-b504-c4b8a820aa77 tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:17.645555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a540b3-c3d9-4ef2-bde7-1648f1cd9054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:17.646400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a540b3-c3d9-4ef2-bde7-1648f1cd9054 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:17.646679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a540b3-c3d9-4ef2-bde7-1648f1cd9054 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:17.647251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a540b3-c3d9-4ef2-bde7-1648f1cd9054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:17.656449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-51a540b3-c3d9-4ef2-bde7-1648f1cd9054 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:17.716451 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2993/5991] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Oct 15 21:14:17 2022] POST /identity/v3/users/9950cb2a38db441c83b4797b883cedc5/application_credentials => generated 920 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:17.722077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6eabb-3abc-489a-82f8-448e8af171c7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:17.722363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6eabb-3abc-489a-82f8-448e8af171c7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:17.722635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6eabb-3abc-489a-82f8-448e8af171c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:17.769262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bcc6eabb-3abc-489a-82f8-448e8af171c7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:17.771283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bcc6eabb-3abc-489a-82f8-448e8af171c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:17.808950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a540b3-c3d9-4ef2-bde7-1648f1cd9054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:17.813912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a540b3-c3d9-4ef2-bde7-1648f1cd9054 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:17.832352 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 2999/5992] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:17.834189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a540b3-c3d9-4ef2-bde7-1648f1cd9054 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:17.835807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-51a540b3-c3d9-4ef2-bde7-1648f1cd9054 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:17.888087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:17.948954 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2994/5993] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:17.952349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6eabb-3abc-489a-82f8-448e8af171c7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:17.972074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a540b3-c3d9-4ef2-bde7-1648f1cd9054 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:17.975946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6eabb-3abc-489a-82f8-448e8af171c7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:17.983355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a24d5c-e8d3-4759-8ea7-8fde6efbee5e tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9950cb2a38db441c83b4797b883cedc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06c072c77fa840bf96dec81883b4ffab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:17.984020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24d5c-e8d3-4759-8ea7-8fde6efbee5e tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:17.984294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24d5c-e8d3-4759-8ea7-8fde6efbee5e tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:17.984560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24d5c-e8d3-4759-8ea7-8fde6efbee5e tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules/4075c351d5594613972ad18ae55d2ef6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:17.989023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a24d5c-e8d3-4759-8ea7-8fde6efbee5e tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorizing `identity:get_access_rule(user_id=9950cb2a38db441c83b4797b883cedc5, access_rule_id=4075c351d5594613972ad18ae55d2ef6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:17.990974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a24d5c-e8d3-4759-8ea7-8fde6efbee5e tempest-AccessRulesV3Test-1152862104 tempest-AccessRulesV3Test-1152862104-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:17.992732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:17.994050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:17.994721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:17.995124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] PATH_INFO: `/v3/users/c48346e56c47481cab7f248b67369a27` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:17.999646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c48346e56c47481cab7f248b67369a27)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:18.003152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:18.009126 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2995/5994] 10.222.0.16 () {58 vars in 1359 bytes} [Sat Oct 15 21:14:17 2022] GET /identity/v3/users/9950cb2a38db441c83b4797b883cedc5/access_rules/4075c351d5594613972ad18ae55d2ef6 => generated 229 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:18.009126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1535d-1c97-46d4-a990-51a4479a46d5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:18.009126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1535d-1c97-46d4-a990-51a4479a46d5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:18.009126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1535d-1c97-46d4-a990-51a4479a46d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:18.099322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-12b1535d-1c97-46d4-a990-51a4479a46d5 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:18.100651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c48346e56c47481cab7f248b67369a27'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.100799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-12b1535d-1c97-46d4-a990-51a4479a46d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:18.139324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:18.141776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c48346e56c47481cab7f248b67369a27'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.166417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c48346e56c47481cab7f248b67369a27'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.177994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c48346e56c47481cab7f248b67369a27'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.178870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c48346e56c47481cab7f248b67369a27'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.188016 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3000/5995] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:17 2022] DELETE /identity/v3/users/c48346e56c47481cab7f248b67369a27 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:18.191114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:18.196240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:18.225068 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2996/5996] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:18.225851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1535d-1c97-46d4-a990-51a4479a46d5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:18.227341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-12b1535d-1c97-46d4-a990-51a4479a46d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:18.277151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1535d-1c97-46d4-a990-51a4479a46d5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:18.285980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:18.286821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:18.287210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:18.287598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] PATH_INFO: `/v3/users/e8f30198ba394d77a21157598d5f4d7e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:18.293179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ad4ee-1d85-4275-acca-49ff8db20762 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:18.295402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f30198ba394d77a21157598d5f4d7e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:18.298251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:18.301569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:18.302739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:18.303261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:18.303724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] PATH_INFO: `/v3/users/13da28bb5f0f42938671ed8270f35513` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:18.305094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13da28bb5f0f42938671ed8270f35513)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:18.307311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:18.383372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13da28bb5f0f42938671ed8270f35513'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.400270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f30198ba394d77a21157598d5f4d7e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.402873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13da28bb5f0f42938671ed8270f35513'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.417863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f30198ba394d77a21157598d5f4d7e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.426034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13da28bb5f0f42938671ed8270f35513'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.441351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13da28bb5f0f42938671ed8270f35513'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.442166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13da28bb5f0f42938671ed8270f35513'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.444206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f30198ba394d77a21157598d5f4d7e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.449775 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3001/5997] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:18 2022] DELETE /identity/v3/users/13da28bb5f0f42938671ed8270f35513 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:18.457654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f30198ba394d77a21157598d5f4d7e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.458917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f30198ba394d77a21157598d5f4d7e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.459229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:18.460684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:18.477893 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2997/5998] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:18 2022] DELETE /identity/v3/users/e8f30198ba394d77a21157598d5f4d7e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:18.490027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:18.500331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:18.569670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde339b-8ba3-49be-854a-6a9d4d49a432 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:18.578282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:18.589377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:18.656485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6025687-ab1d-4f34-9ab0-51e9531ad697 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:18.663409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80559502-eedb-43dc-91da-dba78b30c3ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:18.663669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:18.664206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:18.664557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:18.664713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80559502-eedb-43dc-91da-dba78b30c3ba service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:18.664876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] PATH_INFO: `/v3/users/9950cb2a38db441c83b4797b883cedc5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:18.665002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80559502-eedb-43dc-91da-dba78b30c3ba service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:18.665317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80559502-eedb-43dc-91da-dba78b30c3ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:18.666318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9950cb2a38db441c83b4797b883cedc5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:18.667318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-80559502-eedb-43dc-91da-dba78b30c3ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:18.669555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:18.704379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80559502-eedb-43dc-91da-dba78b30c3ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:18.705859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80559502-eedb-43dc-91da-dba78b30c3ba service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:18.721541 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3002/5999] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:18.722434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80559502-eedb-43dc-91da-dba78b30c3ba service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:18.724146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-80559502-eedb-43dc-91da-dba78b30c3ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:18.771905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9950cb2a38db441c83b4797b883cedc5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.782727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9950cb2a38db441c83b4797b883cedc5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.838593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80559502-eedb-43dc-91da-dba78b30c3ba service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:18.846051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05d0b3-ba77-4b16-aaee-42286aaf628b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:18.846798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05d0b3-ba77-4b16-aaee-42286aaf628b service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:18.847194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05d0b3-ba77-4b16-aaee-42286aaf628b service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:18.847499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05d0b3-ba77-4b16-aaee-42286aaf628b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:18.849374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9950cb2a38db441c83b4797b883cedc5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.849615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1c05d0b3-ba77-4b16-aaee-42286aaf628b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:18.861285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9950cb2a38db441c83b4797b883cedc5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.862775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bd7e11a-3dd3-4c12-b903-6510e41ba30c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9950cb2a38db441c83b4797b883cedc5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:18.873116 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2998/6000] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:18 2022] DELETE /identity/v3/users/9950cb2a38db441c83b4797b883cedc5 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:18.881257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e62b5-9501-4224-8e71-6f5f0371393f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:18.881620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e62b5-9501-4224-8e71-6f5f0371393f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:18.882371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e62b5-9501-4224-8e71-6f5f0371393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:18.913213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4a0e62b5-9501-4224-8e71-6f5f0371393f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:18.915609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4a0e62b5-9501-4224-8e71-6f5f0371393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:18.934850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05d0b3-ba77-4b16-aaee-42286aaf628b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:18.936784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05d0b3-ba77-4b16-aaee-42286aaf628b service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:18.950496 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3003/6001] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:14:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:14:18.952571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c7a16-f46d-484e-ba18-c5801fd3b04c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:18.953028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c7a16-f46d-484e-ba18-c5801fd3b04c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:18.953932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c7a16-f46d-484e-ba18-c5801fd3b04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:18.984288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-195c7a16-f46d-484e-ba18-c5801fd3b04c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:18.985918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-195c7a16-f46d-484e-ba18-c5801fd3b04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:19.003744 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3004/6002] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:19.003744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c7a16-f46d-484e-ba18-c5801fd3b04c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:19.004462 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 2999/6003] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:19.004462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e62b5-9501-4224-8e71-6f5f0371393f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:19.007882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-195c7a16-f46d-484e-ba18-c5801fd3b04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:19.009962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4a0e62b5-9501-4224-8e71-6f5f0371393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:19.022431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c7a16-f46d-484e-ba18-c5801fd3b04c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:19.028725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:19.031492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.031492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.031492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] PATH_INFO: `/v3/users/2e580a9f5307454d86f6430ade4910ef` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.032258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e580a9f5307454d86f6430ade4910ef)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:19.034686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:19.037175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e62b5-9501-4224-8e71-6f5f0371393f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:19.044663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:19.045631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.046066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.046490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] PATH_INFO: `/v3/projects/e2b1936403344a4994393f2b098327a0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.047837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b1936403344a4994393f2b098327a0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:19.050174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:19.092246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:19.110615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:19.123929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e580a9f5307454d86f6430ade4910ef'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:19.132885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b1936403344a4994393f2b098327a0', 'request_id': 'req-4e22293d-1526-41c6-a542-339bf647619b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:19.149855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b1936403344a4994393f2b098327a0', 'request_id': 'req-4e22293d-1526-41c6-a542-339bf647619b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:19.151675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e580a9f5307454d86f6430ade4910ef'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:19.167353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b1936403344a4994393f2b098327a0', 'request_id': 'req-4e22293d-1526-41c6-a542-339bf647619b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:19.172209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e580a9f5307454d86f6430ade4910ef'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:19.181490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] The token cache is being invalidate because project e2b1936403344a4994393f2b098327a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:19.181778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:19.182483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4e22293d-1526-41c6-a542-339bf647619b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:19.184366 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3000/6004] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:19 2022] DELETE /identity/v3/projects/e2b1936403344a4994393f2b098327a0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:19.185912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64257e7-0d8f-4bc2-a4cc-5f8d007af60c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.186189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64257e7-0d8f-4bc2-a4cc-5f8d007af60c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.186456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64257e7-0d8f-4bc2-a4cc-5f8d007af60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.198188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e580a9f5307454d86f6430ade4910ef'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:19.199885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e580a9f5307454d86f6430ade4910ef'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:19.216072 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3005/6005] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:19 2022] DELETE /identity/v3/users/2e580a9f5307454d86f6430ade4910ef => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:19.217448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:19.222962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:19.240584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d64257e7-0d8f-4bc2-a4cc-5f8d007af60c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:19.242676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d64257e7-0d8f-4bc2-a4cc-5f8d007af60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:19.383627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc60f6-2aea-4552-9ad3-fffece74a0b7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:19.400043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a58e9-6754-4360-b6b4-dab41cc37253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:19.401024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a58e9-6754-4360-b6b4-dab41cc37253 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.401605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a58e9-6754-4360-b6b4-dab41cc37253 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.402082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a58e9-6754-4360-b6b4-dab41cc37253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.407172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-435a58e9-6754-4360-b6b4-dab41cc37253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:19.414197 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3001/6006] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:19.416039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64257e7-0d8f-4bc2-a4cc-5f8d007af60c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:19.438357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64257e7-0d8f-4bc2-a4cc-5f8d007af60c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:19.439510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a58e9-6754-4360-b6b4-dab41cc37253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:19.441646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a58e9-6754-4360-b6b4-dab41cc37253 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:19.443955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec28ba1-d191-42c1-8efb-72eba2d9d971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:19.444886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec28ba1-d191-42c1-8efb-72eba2d9d971 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.445292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec28ba1-d191-42c1-8efb-72eba2d9d971 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.446033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec28ba1-d191-42c1-8efb-72eba2d9d971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.448998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1ec28ba1-d191-42c1-8efb-72eba2d9d971 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:19.470546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec28ba1-d191-42c1-8efb-72eba2d9d971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:19.472453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec28ba1-d191-42c1-8efb-72eba2d9d971 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:19.482989 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3006/6007] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:19.483480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb7501-3460-4b67-8c20-07286cfe667d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.510588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb7501-3460-4b67-8c20-07286cfe667d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.510588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb7501-3460-4b67-8c20-07286cfe667d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.511125 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3002/6008] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:19.511125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5d950-345f-473b-b1d7-7897a7776ffe None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.511125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5d950-345f-473b-b1d7-7897a7776ffe None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.511125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5d950-345f-473b-b1d7-7897a7776ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.553251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-00c5d950-345f-473b-b1d7-7897a7776ffe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:19.556937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-00c5d950-345f-473b-b1d7-7897a7776ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:19.577816 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3003/6009] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:19.578941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5d950-345f-473b-b1d7-7897a7776ffe None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:19.581987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e8eb7501-3460-4b67-8c20-07286cfe667d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:19.583443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e8eb7501-3460-4b67-8c20-07286cfe667d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:19.607506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5d950-345f-473b-b1d7-7897a7776ffe None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:19.609684 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3007/6010] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:19.610632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb7501-3460-4b67-8c20-07286cfe667d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:19.621216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a6399-f49d-4d2d-9112-b8d79456f352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:19.623179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a6399-f49d-4d2d-9112-b8d79456f352 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.623714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a6399-f49d-4d2d-9112-b8d79456f352 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.624247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a6399-f49d-4d2d-9112-b8d79456f352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.625726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb7501-3460-4b67-8c20-07286cfe667d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:19.630855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70966f1c-1847-4332-847a-cc53ca0789cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:19.631459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70966f1c-1847-4332-847a-cc53ca0789cf service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.631730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70966f1c-1847-4332-847a-cc53ca0789cf service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.631992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70966f1c-1847-4332-847a-cc53ca0789cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.644996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70966f1c-1847-4332-847a-cc53ca0789cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:19.647737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70966f1c-1847-4332-847a-cc53ca0789cf service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:19.650628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a6399-f49d-4d2d-9112-b8d79456f352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:19.655554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a6399-f49d-4d2d-9112-b8d79456f352 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:19.664938 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3008/6011] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:19.673672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b560c-825f-4b83-8412-6e858e2a0f57 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.673944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b560c-825f-4b83-8412-6e858e2a0f57 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.674222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b560c-825f-4b83-8412-6e858e2a0f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.682287 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3004/6012] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:19.734639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-125b560c-825f-4b83-8412-6e858e2a0f57 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:19.736717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-125b560c-825f-4b83-8412-6e858e2a0f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:19.758729 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3009/6013] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:19.762852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a6399-f49d-4d2d-9112-b8d79456f352 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:19.776525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a6399-f49d-4d2d-9112-b8d79456f352 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:19.791924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56622051-ade5-4513-8117-3c909075eab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:19.792949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56622051-ade5-4513-8117-3c909075eab8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.793303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56622051-ade5-4513-8117-3c909075eab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.794170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56622051-ade5-4513-8117-3c909075eab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.795417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56622051-ade5-4513-8117-3c909075eab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:19.795894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56622051-ade5-4513-8117-3c909075eab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:19.798963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56622051-ade5-4513-8117-3c909075eab8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:19.839188 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3005/6014] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:19.842406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b560c-825f-4b83-8412-6e858e2a0f57 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:19.887674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b560c-825f-4b83-8412-6e858e2a0f57 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:19.907128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f9aa7-efaf-4d20-ab67-e6c55e1b4bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:19.910112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f9aa7-efaf-4d20-ab67-e6c55e1b4bab admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:19.910624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f9aa7-efaf-4d20-ab67-e6c55e1b4bab admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:19.912183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f9aa7-efaf-4d20-ab67-e6c55e1b4bab admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:19.913897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f9aa7-efaf-4d20-ab67-e6c55e1b4bab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:19.920737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f9aa7-efaf-4d20-ab67-e6c55e1b4bab admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:20.041489 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3010/6015] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:19 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:20.042314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56622051-ade5-4513-8117-3c909075eab8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:20.057685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56622051-ade5-4513-8117-3c909075eab8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:20.182453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0234-5450-416f-bd1c-326ec88313fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:20.183075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0234-5450-416f-bd1c-326ec88313fe admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:20.183351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0234-5450-416f-bd1c-326ec88313fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:20.183626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0234-5450-416f-bd1c-326ec88313fe admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:20.184969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc0234-5450-416f-bd1c-326ec88313fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:20.187180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc0234-5450-416f-bd1c-326ec88313fe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:20.200698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:20.243375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f9aa7-efaf-4d20-ab67-e6c55e1b4bab admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:20.254331 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3006/6016] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:20 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:14:20.261721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f9aa7-efaf-4d20-ab67-e6c55e1b4bab admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:20.276021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020c882-f99e-4fd0-ac32-924ddee68f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:20.278492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020c882-f99e-4fd0-ac32-924ddee68f58 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:20.278790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020c882-f99e-4fd0-ac32-924ddee68f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:20.282844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020c882-f99e-4fd0-ac32-924ddee68f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:20.284149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020c882-f99e-4fd0-ac32-924ddee68f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:20.286676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020c882-f99e-4fd0-ac32-924ddee68f58 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:20.310398 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3011/6017] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:20.319973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0234-5450-416f-bd1c-326ec88313fe admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:20.340379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0234-5450-416f-bd1c-326ec88313fe admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:20.374438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdae0b4-217d-4ee6-9e3e-9c963d376ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:20.375367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdae0b4-217d-4ee6-9e3e-9c963d376ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:20.375819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdae0b4-217d-4ee6-9e3e-9c963d376ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:20.376265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdae0b4-217d-4ee6-9e3e-9c963d376ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:20.377191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdae0b4-217d-4ee6-9e3e-9c963d376ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:20.382830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdae0b4-217d-4ee6-9e3e-9c963d376ce2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:20.424632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020c882-f99e-4fd0-ac32-924ddee68f58 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:20.437075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020c882-f99e-4fd0-ac32-924ddee68f58 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:20.437890 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3007/6018] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:20.444657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a01e5-42ec-413d-b06e-14f863d4f70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:20.445958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a01e5-42ec-413d-b06e-14f863d4f70d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:20.446524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a01e5-42ec-413d-b06e-14f863d4f70d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:20.446972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a01e5-42ec-413d-b06e-14f863d4f70d admin admin] PATH_INFO: `/v3/projects/87333497286e48ef95cb933c074efa70/users/379c929d93b54b9a9751defeef161f57/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:20.470629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a01e5-42ec-413d-b06e-14f863d4f70d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87333497286e48ef95cb933c074efa70, user_id=379c929d93b54b9a9751defeef161f57, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:20.476873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a01e5-42ec-413d-b06e-14f863d4f70d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:20.509866 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3012/6019] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:20 2022] PUT /identity/v3/projects/87333497286e48ef95cb933c074efa70/users/379c929d93b54b9a9751defeef161f57/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:20.512026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e9374-1ff1-4ae3-83aa-722d22baddf2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:20.513277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e9374-1ff1-4ae3-83aa-722d22baddf2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:20.513750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e9374-1ff1-4ae3-83aa-722d22baddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:20.577608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-768e9374-1ff1-4ae3-83aa-722d22baddf2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:20.579293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-768e9374-1ff1-4ae3-83aa-722d22baddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:20.678649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a01e5-42ec-413d-b06e-14f863d4f70d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:20.682926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1c5a01e5-42ec-413d-b06e-14f863d4f70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:20.764446 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3008/6020] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:20.769772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:20.794865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e9374-1ff1-4ae3-83aa-722d22baddf2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:20.797458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-768e9374-1ff1-4ae3-83aa-722d22baddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:20.809302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a01e5-42ec-413d-b06e-14f863d4f70d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:20.816324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:20.817458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:20.817952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:20.818431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] PATH_INFO: `/v3/projects/bbff1010e0954b2187b8c333fe0dfe1c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:20.819842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbff1010e0954b2187b8c333fe0dfe1c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:20.822323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:20.915689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbff1010e0954b2187b8c333fe0dfe1c', 'request_id': 'req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:20.935073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbff1010e0954b2187b8c333fe0dfe1c', 'request_id': 'req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:20.938875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e9374-1ff1-4ae3-83aa-722d22baddf2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:20.943987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbff1010e0954b2187b8c333fe0dfe1c', 'request_id': 'req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:20.976028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] The token cache is being invalidate because project bbff1010e0954b2187b8c333fe0dfe1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:20.976426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:20.978173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fc8fc4f8-f10f-4207-9457-abcf5eef84a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:20.984696 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3013/6021] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:20 2022] DELETE /identity/v3/projects/bbff1010e0954b2187b8c333fe0dfe1c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:20.990876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594d54-212d-4b6a-bb1f-7f75b6f712f2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:20.991427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594d54-212d-4b6a-bb1f-7f75b6f712f2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:20.991939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594d54-212d-4b6a-bb1f-7f75b6f712f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:21.035771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-74594d54-212d-4b6a-bb1f-7f75b6f712f2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:21.041019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-74594d54-212d-4b6a-bb1f-7f75b6f712f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:21.077382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75613c9-9245-4d4a-bfff-9cd9cd5ae0fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:21.078183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75613c9-9245-4d4a-bfff-9cd9cd5ae0fb service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:21.078572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75613c9-9245-4d4a-bfff-9cd9cd5ae0fb service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:21.078951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75613c9-9245-4d4a-bfff-9cd9cd5ae0fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:21.082991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c75613c9-9245-4d4a-bfff-9cd9cd5ae0fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:21.181602 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3014/6022] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:21.186348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87e84-3e5d-4a51-8323-2e9c3a8d603d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:21.190488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87e84-3e5d-4a51-8323-2e9c3a8d603d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:21.192552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87e84-3e5d-4a51-8323-2e9c3a8d603d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:21.226978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:21.258339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75613c9-9245-4d4a-bfff-9cd9cd5ae0fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:21.260423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75613c9-9245-4d4a-bfff-9cd9cd5ae0fb service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:21.283748 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3009/6023] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:21.284314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f1e87e84-3e5d-4a51-8323-2e9c3a8d603d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:21.285130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75613c9-9245-4d4a-bfff-9cd9cd5ae0fb service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:21.288335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f1e87e84-3e5d-4a51-8323-2e9c3a8d603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:21.289284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c75613c9-9245-4d4a-bfff-9cd9cd5ae0fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:21.313340 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3015/6024] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:21.314450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e87e84-3e5d-4a51-8323-2e9c3a8d603d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:21.316747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75613c9-9245-4d4a-bfff-9cd9cd5ae0fb service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:21.327535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e87e84-3e5d-4a51-8323-2e9c3a8d603d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:21.334570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209174d4-6eef-4138-8571-92b28d883d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:21.335185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209174d4-6eef-4138-8571-92b28d883d1c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:21.335561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209174d4-6eef-4138-8571-92b28d883d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:21.335827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209174d4-6eef-4138-8571-92b28d883d1c admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:21.336618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:21.336917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209174d4-6eef-4138-8571-92b28d883d1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:21.337303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209174d4-6eef-4138-8571-92b28d883d1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:21.339106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209174d4-6eef-4138-8571-92b28d883d1c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:21.340964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:21.342103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:21.342438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] PATH_INFO: `/v3/projects/76a47d2411ba41fb8c7a1f163a07ec74` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:21.349886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=76a47d2411ba41fb8c7a1f163a07ec74)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:21.355636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:21.361786 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3016/6025] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:21.362748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209174d4-6eef-4138-8571-92b28d883d1c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:21.385803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209174d4-6eef-4138-8571-92b28d883d1c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:21.396254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a9b9d-c969-4a1c-8c1d-a41bc6d42bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:21.397219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9b9d-c969-4a1c-8c1d-a41bc6d42bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:21.397498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9b9d-c969-4a1c-8c1d-a41bc6d42bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:21.397765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9b9d-c969-4a1c-8c1d-a41bc6d42bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:21.399988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a9b9d-c969-4a1c-8c1d-a41bc6d42bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:21.401028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a9b9d-c969-4a1c-8c1d-a41bc6d42bb1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:21.477547 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3017/6026] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:21 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:21.482933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a9b9d-c969-4a1c-8c1d-a41bc6d42bb1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:21.485595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76a47d2411ba41fb8c7a1f163a07ec74', 'request_id': 'req-724fed49-6d58-41f7-aa72-ccf072b128ff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:21.494857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9d7a9b9d-c969-4a1c-8c1d-a41bc6d42bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:21.506920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76a47d2411ba41fb8c7a1f163a07ec74', 'request_id': 'req-724fed49-6d58-41f7-aa72-ccf072b128ff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:21.524385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76a47d2411ba41fb8c7a1f163a07ec74', 'request_id': 'req-724fed49-6d58-41f7-aa72-ccf072b128ff'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:21.565587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] The token cache is being invalidate because project 76a47d2411ba41fb8c7a1f163a07ec74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:21.566681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:21.568082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:21.573905 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3010/6027] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:21 2022] DELETE /identity/v3/projects/76a47d2411ba41fb8c7a1f163a07ec74 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:21.575260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:21.591162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:21.607476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:21.688290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a9b9d-c969-4a1c-8c1d-a41bc6d42bb1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:21.705855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:21.708345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:21.710110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:21.710644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] PATH_INFO: `/v3/projects/0205358c6de24847bff60df4cc5a57be` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:21.713505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0205358c6de24847bff60df4cc5a57be)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:21.734654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:21.873565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fed49-6d58-41f7-aa72-ccf072b128ff admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:21.880808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0205358c6de24847bff60df4cc5a57be', 'request_id': 'req-f19d377f-3666-4cf4-812c-15dbed02bca4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:21.893294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0205358c6de24847bff60df4cc5a57be', 'request_id': 'req-f19d377f-3666-4cf4-812c-15dbed02bca4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:21.928596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0205358c6de24847bff60df4cc5a57be', 'request_id': 'req-f19d377f-3666-4cf4-812c-15dbed02bca4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:21.960828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] The token cache is being invalidate because project 0205358c6de24847bff60df4cc5a57be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:21.961917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:21.966149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:21.972613 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3018/6028] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:21 2022] DELETE /identity/v3/projects/0205358c6de24847bff60df4cc5a57be => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:21.973672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:21.975168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:22.053772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:22.055113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:22.055562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:22.056003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] PATH_INFO: `/v3/projects/38d8f34110424a279a2847692017065e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:22.058552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d8f34110424a279a2847692017065e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:22.063571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:22.116938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:22.123576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d377f-3666-4cf4-812c-15dbed02bca4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:22.143879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94d3ab-c0d4-49f1-99e1-7862bff7c774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:22.144499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94d3ab-c0d4-49f1-99e1-7862bff7c774 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:22.144769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94d3ab-c0d4-49f1-99e1-7862bff7c774 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:22.145030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94d3ab-c0d4-49f1-99e1-7862bff7c774 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:22.148496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94d3ab-c0d4-49f1-99e1-7862bff7c774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:22.151791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94d3ab-c0d4-49f1-99e1-7862bff7c774 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:22.166807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d8f34110424a279a2847692017065e', 'request_id': 'req-ffd51381-2c24-4835-8240-07e977e00ec9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.186069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d8f34110424a279a2847692017065e', 'request_id': 'req-ffd51381-2c24-4835-8240-07e977e00ec9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.199748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d8f34110424a279a2847692017065e', 'request_id': 'req-ffd51381-2c24-4835-8240-07e977e00ec9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.219575 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3019/6029] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:21 2022] POST /identity/v3/users => generated 292 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:22.220376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc39475-95d4-48b0-b26e-13229138b80a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:22.223162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc39475-95d4-48b0-b26e-13229138b80a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:22.223689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc39475-95d4-48b0-b26e-13229138b80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:22.236039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] The token cache is being invalidate because project 38d8f34110424a279a2847692017065e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:22.236308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.238679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.240893 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3011/6030] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:21 2022] DELETE /identity/v3/projects/38d8f34110424a279a2847692017065e => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:22.243019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:22.246164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:22.276248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1cc39475-95d4-48b0-b26e-13229138b80a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:22.281150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1cc39475-95d4-48b0-b26e-13229138b80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:22.422560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:22.438236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd51381-2c24-4835-8240-07e977e00ec9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:22.452943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:22.454500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:22.455089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:22.455624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] PATH_INFO: `/v3/projects/06c072c77fa840bf96dec81883b4ffab` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:22.457098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c072c77fa840bf96dec81883b4ffab)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:22.459386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:22.488902 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3020/6031] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:22.490251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc39475-95d4-48b0-b26e-13229138b80a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:22.492075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1cc39475-95d4-48b0-b26e-13229138b80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:22.509209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc39475-95d4-48b0-b26e-13229138b80a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:22.526589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:22.527684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:22.529015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:22.529639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] PATH_INFO: `/v3/users/038805bf16844e2da7cb92c9eedd4d1c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:22.537251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038805bf16844e2da7cb92c9eedd4d1c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:22.548194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:22.585863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c072c77fa840bf96dec81883b4ffab', 'request_id': 'req-659c7454-6ae4-481e-9233-6156985863e4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.618004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c072c77fa840bf96dec81883b4ffab', 'request_id': 'req-659c7454-6ae4-481e-9233-6156985863e4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.628825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c072c77fa840bf96dec81883b4ffab', 'request_id': 'req-659c7454-6ae4-481e-9233-6156985863e4'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.645990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] The token cache is being invalidate because project 06c072c77fa840bf96dec81883b4ffab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:22.649951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.651111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.668447 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3012/6032] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:22 2022] DELETE /identity/v3/projects/06c072c77fa840bf96dec81883b4ffab => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:22.668447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:22.671541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:22.731659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038805bf16844e2da7cb92c9eedd4d1c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.787586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038805bf16844e2da7cb92c9eedd4d1c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.816758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038805bf16844e2da7cb92c9eedd4d1c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.832192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:22.844884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038805bf16844e2da7cb92c9eedd4d1c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.846510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d73c948a-46c6-4e63-92e1-241ca56076e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038805bf16844e2da7cb92c9eedd4d1c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:22.875702 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3021/6033] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:22 2022] DELETE /identity/v3/users/038805bf16844e2da7cb92c9eedd4d1c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:22.875702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45daac-5778-41aa-ae89-a521768a00aa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:22.875702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45daac-5778-41aa-ae89-a521768a00aa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:22.875702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45daac-5778-41aa-ae89-a521768a00aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:22.897464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c7454-6ae4-481e-9233-6156985863e4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:22.931021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cb45daac-5778-41aa-ae89-a521768a00aa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:22.934497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cb45daac-5778-41aa-ae89-a521768a00aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:23.026462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0768d-71b7-4ff2-8d9b-ab741962c19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:23.027313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0768d-71b7-4ff2-8d9b-ab741962c19d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.028332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0768d-71b7-4ff2-8d9b-ab741962c19d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.029950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0768d-71b7-4ff2-8d9b-ab741962c19d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.032831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0768d-71b7-4ff2-8d9b-ab741962c19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:23.036703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0768d-71b7-4ff2-8d9b-ab741962c19d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:23.053347 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3013/6034] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:23.055160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2ce61-f4ec-4677-8304-453924a8f247 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.056013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2ce61-f4ec-4677-8304-453924a8f247 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.056721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2ce61-f4ec-4677-8304-453924a8f247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.085573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-58c2ce61-f4ec-4677-8304-453924a8f247 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:23.087074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-58c2ce61-f4ec-4677-8304-453924a8f247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:23.105794 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3014/6035] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:23.108793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04787-4d33-4333-b238-2174442752aa None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.109326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04787-4d33-4333-b238-2174442752aa None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.111241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04787-4d33-4333-b238-2174442752aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.140144 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3022/6036] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:23.141339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45daac-5778-41aa-ae89-a521768a00aa None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:23.146220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cb45daac-5778-41aa-ae89-a521768a00aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:23.168920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c7d04787-4d33-4333-b238-2174442752aa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:23.171068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c7d04787-4d33-4333-b238-2174442752aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:23.183465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45daac-5778-41aa-ae89-a521768a00aa None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:23.207908 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3015/6037] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:23.209279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04787-4d33-4333-b238-2174442752aa None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:23.219434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759eeca5-6042-4a7e-82d5-13cedc063960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:23.220044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eeca5-6042-4a7e-82d5-13cedc063960 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.220375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eeca5-6042-4a7e-82d5-13cedc063960 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.220645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eeca5-6042-4a7e-82d5-13cedc063960 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.221677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759eeca5-6042-4a7e-82d5-13cedc063960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:23.223290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759eeca5-6042-4a7e-82d5-13cedc063960 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:23.234001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04787-4d33-4333-b238-2174442752aa None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:23.241910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d585574-9239-4062-b0d1-deed7487de46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:23.242752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d585574-9239-4062-b0d1-deed7487de46 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.243194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d585574-9239-4062-b0d1-deed7487de46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.243820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d585574-9239-4062-b0d1-deed7487de46 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.244808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d585574-9239-4062-b0d1-deed7487de46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:23.245312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d585574-9239-4062-b0d1-deed7487de46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:23.248322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d585574-9239-4062-b0d1-deed7487de46 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:23.268854 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3023/6038] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:23.276448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250a61-487c-4469-8621-dad4c7804f78 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.277543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250a61-487c-4469-8621-dad4c7804f78 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.279119 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3016/6039] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:23.282544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250a61-487c-4469-8621-dad4c7804f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.283331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e289e9-5673-4330-90a4-244d167332c5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.283331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e289e9-5673-4330-90a4-244d167332c5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.283331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e289e9-5673-4330-90a4-244d167332c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.323269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6d250a61-487c-4469-8621-dad4c7804f78 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:23.324360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e6e289e9-5673-4330-90a4-244d167332c5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:23.324602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6d250a61-487c-4469-8621-dad4c7804f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:23.331268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e6e289e9-5673-4330-90a4-244d167332c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:23.337461 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3024/6040] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:23.338168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250a61-487c-4469-8621-dad4c7804f78 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:23.360166 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3017/6041] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:23.362500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e289e9-5673-4330-90a4-244d167332c5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:23.362666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250a61-487c-4469-8621-dad4c7804f78 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:23.370049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047ec35-e92f-44e3-99c9-49d78eb6e4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:23.370787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047ec35-e92f-44e3-99c9-49d78eb6e4ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.371064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047ec35-e92f-44e3-99c9-49d78eb6e4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.371335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047ec35-e92f-44e3-99c9-49d78eb6e4ff admin admin] PATH_INFO: `/v3/projects/ea52f7e0604f43f98145459c650dd716/users/53c28def95dc4c779ceeda061a48e544/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.386560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e289e9-5673-4330-90a4-244d167332c5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:23.403372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abc248-2ae3-4050-9be2-db8d55fcd2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:23.404333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abc248-2ae3-4050-9be2-db8d55fcd2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.404857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abc248-2ae3-4050-9be2-db8d55fcd2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.405385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abc248-2ae3-4050-9be2-db8d55fcd2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.406359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abc248-2ae3-4050-9be2-db8d55fcd2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:23.406732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7047ec35-e92f-44e3-99c9-49d78eb6e4ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea52f7e0604f43f98145459c650dd716, user_id=53c28def95dc4c779ceeda061a48e544, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:23.410061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7047ec35-e92f-44e3-99c9-49d78eb6e4ff admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:23.419100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abc248-2ae3-4050-9be2-db8d55fcd2b8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:23.455428 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3025/6042] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:23 2022] PUT /identity/v3/projects/ea52f7e0604f43f98145459c650dd716/users/53c28def95dc4c779ceeda061a48e544/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:23.468481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047ec35-e92f-44e3-99c9-49d78eb6e4ff admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:23.524035 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3018/6043] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:23 2022] POST /identity/v3/projects => generated 405 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:23.524035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abc248-2ae3-4050-9be2-db8d55fcd2b8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:23.527448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:23.648611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047ec35-e92f-44e3-99c9-49d78eb6e4ff admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:23.696035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abc248-2ae3-4050-9be2-db8d55fcd2b8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:23.707077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccadea-259a-407a-abfe-72d6e0619ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:23.707699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccadea-259a-407a-abfe-72d6e0619ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.707979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccadea-259a-407a-abfe-72d6e0619ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.708251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccadea-259a-407a-abfe-72d6e0619ae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.708904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccadea-259a-407a-abfe-72d6e0619ae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:23.709233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccadea-259a-407a-abfe-72d6e0619ae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:23.714849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccadea-259a-407a-abfe-72d6e0619ae8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:23.745839 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3019/6044] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:23.746746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccadea-259a-407a-abfe-72d6e0619ae8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:23.779711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccadea-259a-407a-abfe-72d6e0619ae8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:23.797920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b9d4c-1069-4b41-8736-e82e1778a3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:23.798872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b9d4c-1069-4b41-8736-e82e1778a3cc admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.799111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b9d4c-1069-4b41-8736-e82e1778a3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.799921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b9d4c-1069-4b41-8736-e82e1778a3cc admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.801768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b9d4c-1069-4b41-8736-e82e1778a3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:23.809712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b9d4c-1069-4b41-8736-e82e1778a3cc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:23.811251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df63f66-5727-4c30-a259-fa2770e45552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:23.811884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df63f66-5727-4c30-a259-fa2770e45552 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.812862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df63f66-5727-4c30-a259-fa2770e45552 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.814716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df63f66-5727-4c30-a259-fa2770e45552 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.815481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df63f66-5727-4c30-a259-fa2770e45552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:23.815855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df63f66-5727-4c30-a259-fa2770e45552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:23.819621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df63f66-5727-4c30-a259-fa2770e45552 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:23.867789 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3026/6045] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:23.867789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529f62a-4fab-4b50-b9de-3c64d3aef027 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.867789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529f62a-4fab-4b50-b9de-3c64d3aef027 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.867789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529f62a-4fab-4b50-b9de-3c64d3aef027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.885424 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3020/6046] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:23 2022] POST /identity/v3/users => generated 303 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:23.886855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b9d4c-1069-4b41-8736-e82e1778a3cc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:23.919648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b9d4c-1069-4b41-8736-e82e1778a3cc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:23.932254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2a1de-e57b-4214-9631-6386b7b044e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:23.932878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2a1de-e57b-4214-9631-6386b7b044e2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:23.933283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2a1de-e57b-4214-9631-6386b7b044e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:23.933474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2a1de-e57b-4214-9631-6386b7b044e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:23.935546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2a1de-e57b-4214-9631-6386b7b044e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:23.938128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9529f62a-4fab-4b50-b9de-3c64d3aef027 None None] MFA Rules not processed for user `53c28def95dc4c779ceeda061a48e544`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:23.939788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9529f62a-4fab-4b50-b9de-3c64d3aef027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:23.951184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2a1de-e57b-4214-9631-6386b7b044e2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:23.969931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:24.059155 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3021/6047] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:23 2022] POST /identity/v3/projects => generated 367 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:24.062904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2a1de-e57b-4214-9631-6386b7b044e2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:24.119880 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3027/6048] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:24.124840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529f62a-4fab-4b50-b9de-3c64d3aef027 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:24.167631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529f62a-4fab-4b50-b9de-3c64d3aef027 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:24.323714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2a1de-e57b-4214-9631-6386b7b044e2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:24.344862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4d4f7-9f44-4b4c-bcf1-f84637eb6310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:24.345545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d4f7-9f44-4b4c-bcf1-f84637eb6310 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:24.345826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d4f7-9f44-4b4c-bcf1-f84637eb6310 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:24.347646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:24.349309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d4f7-9f44-4b4c-bcf1-f84637eb6310 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:24.349979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4d4f7-9f44-4b4c-bcf1-f84637eb6310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:24.358849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4d4f7-9f44-4b4c-bcf1-f84637eb6310 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:24.395231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a459b9d-9f1d-49ce-a89c-4c5d1ef3f4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:24.395954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a459b9d-9f1d-49ce-a89c-4c5d1ef3f4cb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:24.396238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a459b9d-9f1d-49ce-a89c-4c5d1ef3f4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:24.396510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a459b9d-9f1d-49ce-a89c-4c5d1ef3f4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:24.398120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a459b9d-9f1d-49ce-a89c-4c5d1ef3f4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:24.400386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a459b9d-9f1d-49ce-a89c-4c5d1ef3f4cb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:24.446463 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3028/6049] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:24.447361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a459b9d-9f1d-49ce-a89c-4c5d1ef3f4cb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:24.455950 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3022/6050] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:24 2022] POST /identity/v3/projects => generated 375 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:24.459453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4d4f7-9f44-4b4c-bcf1-f84637eb6310 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:24.464281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1a459b9d-9f1d-49ce-a89c-4c5d1ef3f4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:24.517193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a459b9d-9f1d-49ce-a89c-4c5d1ef3f4cb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:24.703411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:24.704064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:24.704340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:24.704752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] PATH_INFO: `/v3/projects/f17710ae718b477ea1d24487561ebfc8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:24.710215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f17710ae718b477ea1d24487561ebfc8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:24.713785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:24.727978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4d4f7-9f44-4b4c-bcf1-f84637eb6310 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:24.752412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f62e69-9499-453d-8cab-c3839254110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:24.753480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62e69-9499-453d-8cab-c3839254110d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:24.753954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62e69-9499-453d-8cab-c3839254110d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:24.754434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62e69-9499-453d-8cab-c3839254110d admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:24.755269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f62e69-9499-453d-8cab-c3839254110d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:24.757722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f62e69-9499-453d-8cab-c3839254110d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:24.802770 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3023/6051] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:24 2022] POST /identity/v3/users => generated 284 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:24.805267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f62e69-9499-453d-8cab-c3839254110d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:24.811880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-05f62e69-9499-453d-8cab-c3839254110d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:24.819246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f17710ae718b477ea1d24487561ebfc8', 'request_id': 'req-236b6eb1-285f-4387-b183-6f6c627ae6fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:24.837677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f17710ae718b477ea1d24487561ebfc8', 'request_id': 'req-236b6eb1-285f-4387-b183-6f6c627ae6fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:24.848206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f17710ae718b477ea1d24487561ebfc8', 'request_id': 'req-236b6eb1-285f-4387-b183-6f6c627ae6fb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:24.880068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] The token cache is being invalidate because project f17710ae718b477ea1d24487561ebfc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:24.880474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:24.881389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:24.885397 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3029/6052] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:24 2022] DELETE /identity/v3/projects/f17710ae718b477ea1d24487561ebfc8 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:24.886256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:24.892890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:25.006289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f62e69-9499-453d-8cab-c3839254110d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:25.038450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:25.117257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d393f-702c-4c36-b7d2-3826dc0328d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:25.117774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d393f-702c-4c36-b7d2-3826dc0328d2 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.118045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d393f-702c-4c36-b7d2-3826dc0328d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.118307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d393f-702c-4c36-b7d2-3826dc0328d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.122667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e17d393f-702c-4c36-b7d2-3826dc0328d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:25.133191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b6eb1-285f-4387-b183-6f6c627ae6fb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:25.159903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a94c4-6a0c-4cff-a742-d2973fa9b286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:25.160577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a94c4-6a0c-4cff-a742-d2973fa9b286 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.160853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a94c4-6a0c-4cff-a742-d2973fa9b286 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.162738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a94c4-6a0c-4cff-a742-d2973fa9b286 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.166610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a94c4-6a0c-4cff-a742-d2973fa9b286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:25.169027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a94c4-6a0c-4cff-a742-d2973fa9b286 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:25.193139 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3030/6053] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:25.194018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a94c4-6a0c-4cff-a742-d2973fa9b286 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:25.220535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-401a94c4-6a0c-4cff-a742-d2973fa9b286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:25.272789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a94c4-6a0c-4cff-a742-d2973fa9b286 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:25.291830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3943d-f639-4988-8e36-f95bed2116e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:25.292456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3943d-f639-4988-8e36-f95bed2116e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.292716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3943d-f639-4988-8e36-f95bed2116e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.292982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3943d-f639-4988-8e36-f95bed2116e5 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.294245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3943d-f639-4988-8e36-f95bed2116e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:25.296177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3943d-f639-4988-8e36-f95bed2116e5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:25.311877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d393f-702c-4c36-b7d2-3826dc0328d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:25.313371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d393f-702c-4c36-b7d2-3826dc0328d2 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:25.343786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:25.361654 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3024/6054] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:25.365060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d393f-702c-4c36-b7d2-3826dc0328d2 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:25.365060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e17d393f-702c-4c36-b7d2-3826dc0328d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:25.385462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d393f-702c-4c36-b7d2-3826dc0328d2 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:25.393068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f960b-f7da-4a65-b454-d20ef865da1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:25.393730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f960b-f7da-4a65-b454-d20ef865da1a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.394008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f960b-f7da-4a65-b454-d20ef865da1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.394325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f960b-f7da-4a65-b454-d20ef865da1a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.395246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f960b-f7da-4a65-b454-d20ef865da1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:25.396897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f960b-f7da-4a65-b454-d20ef865da1a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:25.407318 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3031/6055] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:25 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:25.408859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0ff2-c3a7-4437-9606-83a40eb35058 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.409312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0ff2-c3a7-4437-9606-83a40eb35058 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.409589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0ff2-c3a7-4437-9606-83a40eb35058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.415814 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3025/6056] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:25.416793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f960b-f7da-4a65-b454-d20ef865da1a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:25.432308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f960b-f7da-4a65-b454-d20ef865da1a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:25.459451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641b349-eb4b-449a-a636-9e4aaa354c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:25.460127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641b349-eb4b-449a-a636-9e4aaa354c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.460324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641b349-eb4b-449a-a636-9e4aaa354c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.460623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641b349-eb4b-449a-a636-9e4aaa354c1a admin admin] PATH_INFO: `/v3/projects/a3230a9bbf8a4c36b0ce4cf9bb3e743f/users/d5c689e427a04bfc96717e2ae8d78b58/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.511885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641b349-eb4b-449a-a636-9e4aaa354c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3230a9bbf8a4c36b0ce4cf9bb3e743f, user_id=d5c689e427a04bfc96717e2ae8d78b58, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:25.511885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641b349-eb4b-449a-a636-9e4aaa354c1a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:25.534140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-193d0ff2-c3a7-4437-9606-83a40eb35058 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:25.540692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-193d0ff2-c3a7-4437-9606-83a40eb35058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:25.570837 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3026/6057] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:25 2022] PUT /identity/v3/projects/a3230a9bbf8a4c36b0ce4cf9bb3e743f/users/d5c689e427a04bfc96717e2ae8d78b58/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:25.572216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641b349-eb4b-449a-a636-9e4aaa354c1a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:25.738646 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3032/6058] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:25.740220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d0ff2-c3a7-4437-9606-83a40eb35058 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:25.755846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d0ff2-c3a7-4437-9606-83a40eb35058 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:25.760907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641b349-eb4b-449a-a636-9e4aaa354c1a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:25.769681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8899424-f22e-409d-83f0-ae75d095e697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:25.771401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899424-f22e-409d-83f0-ae75d095e697 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.771740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899424-f22e-409d-83f0-ae75d095e697 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.772018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899424-f22e-409d-83f0-ae75d095e697 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.772972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb364f2e-c9e0-44f6-bb86-9167718dbde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:25.773987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8899424-f22e-409d-83f0-ae75d095e697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:25.774526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb364f2e-c9e0-44f6-bb86-9167718dbde3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.775065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb364f2e-c9e0-44f6-bb86-9167718dbde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.775340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb364f2e-c9e0-44f6-bb86-9167718dbde3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.779411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb364f2e-c9e0-44f6-bb86-9167718dbde3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:25.781748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8899424-f22e-409d-83f0-ae75d095e697 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:25.783424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb364f2e-c9e0-44f6-bb86-9167718dbde3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:25.807777 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3033/6059] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:25.810268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e261e-04a3-45b9-86b3-fb41ab265564 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.810600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e261e-04a3-45b9-86b3-fb41ab265564 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.811066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e261e-04a3-45b9-86b3-fb41ab265564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.813981 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3027/6060] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:25.814369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54905f4-18d1-4f48-b970-962942ffe7c4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.814649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54905f4-18d1-4f48-b970-962942ffe7c4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.814915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54905f4-18d1-4f48-b970-962942ffe7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.834251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-013e261e-04a3-45b9-86b3-fb41ab265564 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:25.835460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-013e261e-04a3-45b9-86b3-fb41ab265564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:25.867701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a54905f4-18d1-4f48-b970-962942ffe7c4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:25.869295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a54905f4-18d1-4f48-b970-962942ffe7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:25.874940 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3034/6061] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:25.874940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e261e-04a3-45b9-86b3-fb41ab265564 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:25.894347 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3028/6062] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:25.894881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54905f4-18d1-4f48-b970-962942ffe7c4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:25.896287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e261e-04a3-45b9-86b3-fb41ab265564 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:25.909812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f33dfa-bc39-4b97-9e13-3fc7c2579767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:25.910850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f33dfa-bc39-4b97-9e13-3fc7c2579767 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.911073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f33dfa-bc39-4b97-9e13-3fc7c2579767 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.911319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f33dfa-bc39-4b97-9e13-3fc7c2579767 admin admin] PATH_INFO: `/v3/projects/cf0f13e49c894adeacebaa41f8683a71/users/fbff18e4f1bb4b6da04f508cf7abc6f2/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.924918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54905f4-18d1-4f48-b970-962942ffe7c4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:25.937272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcd5b3-2f74-4109-94a6-2f5449862bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:25.939171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcd5b3-2f74-4109-94a6-2f5449862bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:25.939392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcd5b3-2f74-4109-94a6-2f5449862bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:25.939671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcd5b3-2f74-4109-94a6-2f5449862bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:25.940324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcd5b3-2f74-4109-94a6-2f5449862bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:25.941964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcd5b3-2f74-4109-94a6-2f5449862bc1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:25.950402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f33dfa-bc39-4b97-9e13-3fc7c2579767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0f13e49c894adeacebaa41f8683a71, user_id=fbff18e4f1bb4b6da04f508cf7abc6f2, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:25.953588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f33dfa-bc39-4b97-9e13-3fc7c2579767 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:25.975534 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3035/6063] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:25 2022] PUT /identity/v3/projects/cf0f13e49c894adeacebaa41f8683a71/users/fbff18e4f1bb4b6da04f508cf7abc6f2/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:25.976277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f33dfa-bc39-4b97-9e13-3fc7c2579767 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:26.018268 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3029/6064] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:26.019641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcd5b3-2f74-4109-94a6-2f5449862bc1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:26.021515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f8fcd5b3-2f74-4109-94a6-2f5449862bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:26.052835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:26.161125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f33dfa-bc39-4b97-9e13-3fc7c2579767 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:26.173942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62e068-28c6-4f67-8d7f-9908c5dfac2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:26.176689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62e068-28c6-4f67-8d7f-9908c5dfac2b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:26.176976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62e068-28c6-4f67-8d7f-9908c5dfac2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:26.177278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62e068-28c6-4f67-8d7f-9908c5dfac2b admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:26.178192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62e068-28c6-4f67-8d7f-9908c5dfac2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:26.178369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62e068-28c6-4f67-8d7f-9908c5dfac2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:26.182101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62e068-28c6-4f67-8d7f-9908c5dfac2b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:26.220178 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3036/6065] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:26.221202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca0b1c-0755-4674-b2a8-271e27add911 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:26.221202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca0b1c-0755-4674-b2a8-271e27add911 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:26.221402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca0b1c-0755-4674-b2a8-271e27add911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:26.222926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcd5b3-2f74-4109-94a6-2f5449862bc1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:26.317952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f5c6-2dd7-407a-8992-6836c0f6de05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:26.318784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f5c6-2dd7-407a-8992-6836c0f6de05 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:26.319933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f5c6-2dd7-407a-8992-6836c0f6de05 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:26.320306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f5c6-2dd7-407a-8992-6836c0f6de05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:26.331860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-beca0b1c-0755-4674-b2a8-271e27add911 None None] MFA Rules not processed for user `fbff18e4f1bb4b6da04f508cf7abc6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:26.336598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-beca0b1c-0755-4674-b2a8-271e27add911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:26.343572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f5c6-2dd7-407a-8992-6836c0f6de05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:26.345672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f5c6-2dd7-407a-8992-6836c0f6de05 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:26.368005 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3030/6066] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:26.369483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f5c6-2dd7-407a-8992-6836c0f6de05 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:26.385475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f5c6-2dd7-407a-8992-6836c0f6de05 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:26.391433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d489f9-7746-45e3-ac90-3d156120f112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:26.392221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d489f9-7746-45e3-ac90-3d156120f112 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:26.392689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d489f9-7746-45e3-ac90-3d156120f112 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:26.393020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d489f9-7746-45e3-ac90-3d156120f112 admin admin] PATH_INFO: `/v3/projects/dc81f60338274538872024f0ada0f52f/users/33f21d81aa1d46d0aacb4f36e8a5fb79/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:26.412325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d489f9-7746-45e3-ac90-3d156120f112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc81f60338274538872024f0ada0f52f, user_id=33f21d81aa1d46d0aacb4f36e8a5fb79, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:26.415155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d489f9-7746-45e3-ac90-3d156120f112 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:26.416801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:26.447513 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3031/6067] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:26 2022] PUT /identity/v3/projects/dc81f60338274538872024f0ada0f52f/users/33f21d81aa1d46d0aacb4f36e8a5fb79/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:26.459366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d489f9-7746-45e3-ac90-3d156120f112 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:26.498207 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3037/6068] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:26.498207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca0b1c-0755-4674-b2a8-271e27add911 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:26.503338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-beca0b1c-0755-4674-b2a8-271e27add911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:26.536402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca0b1c-0755-4674-b2a8-271e27add911 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:26.609952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d489f9-7746-45e3-ac90-3d156120f112 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:26.625763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dd427-ed9b-4328-82fd-413606f8a771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:26.626391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dd427-ed9b-4328-82fd-413606f8a771 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:26.626666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dd427-ed9b-4328-82fd-413606f8a771 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:26.626934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dd427-ed9b-4328-82fd-413606f8a771 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:26.627735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dd427-ed9b-4328-82fd-413606f8a771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:26.630779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dd427-ed9b-4328-82fd-413606f8a771 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:26.681540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:26.682349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:26.682540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:26.682896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] PATH_INFO: `/v3/users/53c28def95dc4c779ceeda061a48e544` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:26.692670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c28def95dc4c779ceeda061a48e544)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:26.697491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:26.704433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3032/6069] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:26 2022] POST /identity/v3/projects => generated 383 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:26.705881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ec538-1695-4a34-9ca6-5c602cde4fa3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:26.706347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ec538-1695-4a34-9ca6-5c602cde4fa3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:26.708456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ec538-1695-4a34-9ca6-5c602cde4fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:26.845361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d31ec538-1695-4a34-9ca6-5c602cde4fa3 None None] MFA Rules not processed for user `33f21d81aa1d46d0aacb4f36e8a5fb79`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:26.846593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d31ec538-1695-4a34-9ca6-5c602cde4fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:26.873186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c28def95dc4c779ceeda061a48e544'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:26.922128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c28def95dc4c779ceeda061a48e544'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:26.927496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:26.950950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c28def95dc4c779ceeda061a48e544'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:26.973058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c28def95dc4c779ceeda061a48e544'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:26.974418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c28def95dc4c779ceeda061a48e544'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:26.993660 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3038/6070] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:26 2022] DELETE /identity/v3/users/53c28def95dc4c779ceeda061a48e544 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:26.994793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:26.999526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:27.064573 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3033/6071] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:27.065484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ec538-1695-4a34-9ca6-5c602cde4fa3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:27.069336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d31ec538-1695-4a34-9ca6-5c602cde4fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:27.174636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:27.184925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bf670-2110-47ac-8e24-d2ac1342ec94 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:27.203762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ec538-1695-4a34-9ca6-5c602cde4fa3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:27.214502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3f401-6f56-4a3e-8640-17e1d32b4af8 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbff18e4f1bb4b6da04f508cf7abc6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf0f13e49c894adeacebaa41f8683a71', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:27.216315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3f401-6f56-4a3e-8640-17e1d32b4af8 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:27.216871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3f401-6f56-4a3e-8640-17e1d32b4af8 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:27.218120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3f401-6f56-4a3e-8640-17e1d32b4af8 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] PATH_INFO: `/v3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:27.223622 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3039/6072] 10.222.0.16 () {58 vars in 1104 bytes} [Sat Oct 15 21:14:26 2022] GET /identity/v3 => generated 255 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:27.227586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d988a-60fc-48e1-ae63-976995359138 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:27.228063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d988a-60fc-48e1-ae63-976995359138 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:27.229602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d988a-60fc-48e1-ae63-976995359138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:27.259882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-351d988a-60fc-48e1-ae63-976995359138 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:27.261619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-351d988a-60fc-48e1-ae63-976995359138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:27.359705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2ddb9-db31-4cb4-826d-0bd8d9870654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:27.361650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2ddb9-db31-4cb4-826d-0bd8d9870654 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:27.361930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2ddb9-db31-4cb4-826d-0bd8d9870654 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:27.362195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2ddb9-db31-4cb4-826d-0bd8d9870654 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:27.362826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2ddb9-db31-4cb4-826d-0bd8d9870654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:27.364755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2ddb9-db31-4cb4-826d-0bd8d9870654 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:27.386560 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3040/6073] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:27.387770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d988a-60fc-48e1-ae63-976995359138 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:27.391218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-351d988a-60fc-48e1-ae63-976995359138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:27.465729 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3034/6074] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:27 2022] POST /identity/v3/users => generated 292 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:27.468436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2ddb9-db31-4cb4-826d-0bd8d9870654 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:27.503131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2ddb9-db31-4cb4-826d-0bd8d9870654 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:27.530064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33006058-5a26-49d4-86dd-1c1c1a6254d5 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbff18e4f1bb4b6da04f508cf7abc6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf0f13e49c894adeacebaa41f8683a71', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:27.530685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33006058-5a26-49d4-86dd-1c1c1a6254d5 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:27.530965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33006058-5a26-49d4-86dd-1c1c1a6254d5 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:27.531233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33006058-5a26-49d4-86dd-1c1c1a6254d5 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] PATH_INFO: `/v3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:27.532585 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3035/6075] 10.222.0.16 () {58 vars in 1104 bytes} [Sat Oct 15 21:14:27 2022] GET /identity/v3 => generated 255 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:27.538292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33006058-5a26-49d4-86dd-1c1c1a6254d5 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:27.551666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-33006058-5a26-49d4-86dd-1c1c1a6254d5 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:27.633976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d988a-60fc-48e1-ae63-976995359138 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:27.644595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718e976-6d53-4f70-a8ae-2ea538e5448c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:27.645251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718e976-6d53-4f70-a8ae-2ea538e5448c service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:27.645527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718e976-6d53-4f70-a8ae-2ea538e5448c service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:27.645796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718e976-6d53-4f70-a8ae-2ea538e5448c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:27.730384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33006058-5a26-49d4-86dd-1c1c1a6254d5 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:27.735755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267275a2-b766-4662-b4e5-70442257e8c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:27.736632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267275a2-b766-4662-b4e5-70442257e8c4 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:27.736903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267275a2-b766-4662-b4e5-70442257e8c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:27.737195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267275a2-b766-4662-b4e5-70442257e8c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:27.789700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267275a2-b766-4662-b4e5-70442257e8c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:27.791289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267275a2-b766-4662-b4e5-70442257e8c4 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:27.798308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:27.821280 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3036/6076] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:27.823709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267275a2-b766-4662-b4e5-70442257e8c4 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:27.845427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267275a2-b766-4662-b4e5-70442257e8c4 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:27.857972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f426671-f4f9-49c0-92cf-018cd98fde9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:27.858687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f426671-f4f9-49c0-92cf-018cd98fde9d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:27.859401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f426671-f4f9-49c0-92cf-018cd98fde9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:27.859548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f426671-f4f9-49c0-92cf-018cd98fde9d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:27.861903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f426671-f4f9-49c0-92cf-018cd98fde9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:27.864753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f426671-f4f9-49c0-92cf-018cd98fde9d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:27.909821 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3037/6077] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:27.913549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718e976-6d53-4f70-a8ae-2ea538e5448c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:27.913825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f426671-f4f9-49c0-92cf-018cd98fde9d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:27.915231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718e976-6d53-4f70-a8ae-2ea538e5448c service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:27.942795 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3041/6078] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:27.943590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718e976-6d53-4f70-a8ae-2ea538e5448c service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:27.957722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f426671-f4f9-49c0-92cf-018cd98fde9d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:27.975742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f132ee4-3498-4926-a9f1-67d5a2f69ac1 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbff18e4f1bb4b6da04f508cf7abc6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf0f13e49c894adeacebaa41f8683a71', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:27.976987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718e976-6d53-4f70-a8ae-2ea538e5448c service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:27.979166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f132ee4-3498-4926-a9f1-67d5a2f69ac1 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:27.979455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f132ee4-3498-4926-a9f1-67d5a2f69ac1 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:27.983649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f132ee4-3498-4926-a9f1-67d5a2f69ac1 tempest-TestApiDiscovery-126731201 tempest-TestApiDiscovery-126731201-project] PATH_INFO: `/v3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:27.991496 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3038/6079] 10.222.0.16 () {58 vars in 1104 bytes} [Sat Oct 15 21:14:27 2022] GET /identity/v3 => generated 255 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:27.994709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cabfa-03a2-4067-89d8-0dcc9999b846 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:27.994709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cabfa-03a2-4067-89d8-0dcc9999b846 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:27.994709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cabfa-03a2-4067-89d8-0dcc9999b846 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:28.005196 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3039/6080] 10.222.0.16 () {56 vars in 895 bytes} [Sat Oct 15 21:14:27 2022] GET /identity/ => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:14:28.010371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370154d-3c04-458b-9a5d-3f986ce24600 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:28.010722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370154d-3c04-458b-9a5d-3f986ce24600 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:28.010953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370154d-3c04-458b-9a5d-3f986ce24600 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:28.021282 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3040/6081] 10.222.0.16 () {56 vars in 895 bytes} [Sat Oct 15 21:14:28 2022] GET /identity/ => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:14:28.022166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370154d-3c04-458b-9a5d-3f986ce24600 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:28.022308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09538f-dc8c-4d73-b30e-5ea6297be7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:28.024110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09538f-dc8c-4d73-b30e-5ea6297be7db admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:28.024446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09538f-dc8c-4d73-b30e-5ea6297be7db admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:28.024631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09538f-dc8c-4d73-b30e-5ea6297be7db admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:28.026087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09538f-dc8c-4d73-b30e-5ea6297be7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:28.031181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09538f-dc8c-4d73-b30e-5ea6297be7db admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:28.039947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f370154d-3c04-458b-9a5d-3f986ce24600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:28.059408 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3042/6082] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:28.063254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09538f-dc8c-4d73-b30e-5ea6297be7db admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:28.095799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09538f-dc8c-4d73-b30e-5ea6297be7db admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:28.115683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370154d-3c04-458b-9a5d-3f986ce24600 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:28.122421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13510d-69f1-4246-a660-c5bbf19894bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:28.123097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13510d-69f1-4246-a660-c5bbf19894bb admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:28.132384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13510d-69f1-4246-a660-c5bbf19894bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:28.132727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13510d-69f1-4246-a660-c5bbf19894bb admin admin] PATH_INFO: `/v3/projects/8bd54f2c1f2048ef9ca26761f3484ba4/users/4aa826125c6b4bbea7dec3d2dc5200a1/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:28.136194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:28.136710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:28.137800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:28.138430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] PATH_INFO: `/v3/users/fbff18e4f1bb4b6da04f508cf7abc6f2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:28.149405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbff18e4f1bb4b6da04f508cf7abc6f2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:28.151381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:28.183358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13510d-69f1-4246-a660-c5bbf19894bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd54f2c1f2048ef9ca26761f3484ba4, user_id=4aa826125c6b4bbea7dec3d2dc5200a1, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:28.194576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13510d-69f1-4246-a660-c5bbf19894bb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:28.218975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:28.231689 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3043/6083] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:28 2022] PUT /identity/v3/projects/8bd54f2c1f2048ef9ca26761f3484ba4/users/4aa826125c6b4bbea7dec3d2dc5200a1/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:28.243117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d3e9f-4987-4bd6-96b4-0fde26dcea96 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:28.243438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d3e9f-4987-4bd6-96b4-0fde26dcea96 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:28.243705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d3e9f-4987-4bd6-96b4-0fde26dcea96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:28.354545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbff18e4f1bb4b6da04f508cf7abc6f2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:28.356438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-970d3e9f-4987-4bd6-96b4-0fde26dcea96 None None] MFA Rules not processed for user `4aa826125c6b4bbea7dec3d2dc5200a1`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:28.358514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-970d3e9f-4987-4bd6-96b4-0fde26dcea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:28.385656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbff18e4f1bb4b6da04f508cf7abc6f2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:28.456557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbff18e4f1bb4b6da04f508cf7abc6f2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:28.492597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbff18e4f1bb4b6da04f508cf7abc6f2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:28.497207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f84e3070-22ed-433b-8dd5-9ba14d011af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbff18e4f1bb4b6da04f508cf7abc6f2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:28.521892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3041/6084] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:28 2022] DELETE /identity/v3/users/fbff18e4f1bb4b6da04f508cf7abc6f2 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:28.523696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c1514-6ea3-4461-92b7-1b53def4f880 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:28.523983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c1514-6ea3-4461-92b7-1b53def4f880 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:28.524254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c1514-6ea3-4461-92b7-1b53def4f880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:28.614918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0a2c1514-6ea3-4461-92b7-1b53def4f880 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:28.616157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0a2c1514-6ea3-4461-92b7-1b53def4f880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:28.619473 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3044/6085] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:14:28.625336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d3e9f-4987-4bd6-96b4-0fde26dcea96 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:28.628543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-970d3e9f-4987-4bd6-96b4-0fde26dcea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:28.794977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d3e9f-4987-4bd6-96b4-0fde26dcea96 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:28.883164 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3042/6086] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:28.947337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71700990-267c-4359-a29e-87420f96fc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:28.947951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71700990-267c-4359-a29e-87420f96fc2a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:28.948227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71700990-267c-4359-a29e-87420f96fc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:28.948678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71700990-267c-4359-a29e-87420f96fc2a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:28.949503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71700990-267c-4359-a29e-87420f96fc2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:28.951531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71700990-267c-4359-a29e-87420f96fc2a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:29.032074 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3045/6087] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:28 2022] POST /identity/v3/projects => generated 385 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:29.036039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c1514-6ea3-4461-92b7-1b53def4f880 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:29.049723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c1514-6ea3-4461-92b7-1b53def4f880 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:29.247575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:29.285339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535974a-882c-4c04-ab99-97589e0dc1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:29.285976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535974a-882c-4c04-ab99-97589e0dc1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:29.286252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535974a-882c-4c04-ab99-97589e0dc1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:29.286554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535974a-882c-4c04-ab99-97589e0dc1e0 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:29.287168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535974a-882c-4c04-ab99-97589e0dc1e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:29.289120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535974a-882c-4c04-ab99-97589e0dc1e0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:29.383965 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3043/6088] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:29 2022] POST /identity/v3/users => generated 299 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:29.385451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71700990-267c-4359-a29e-87420f96fc2a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:29.407698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71700990-267c-4359-a29e-87420f96fc2a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:29.430531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a536-2393-4974-92cf-fdd427df48a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:29.432237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a536-2393-4974-92cf-fdd427df48a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:29.435371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a536-2393-4974-92cf-fdd427df48a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:29.437658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a536-2393-4974-92cf-fdd427df48a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:29.439128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0a536-2393-4974-92cf-fdd427df48a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:29.449064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0a536-2393-4974-92cf-fdd427df48a4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:29.497836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535974a-882c-4c04-ab99-97589e0dc1e0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:29.510970 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3046/6089] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:29.514649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535974a-882c-4c04-ab99-97589e0dc1e0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:29.533571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06580de5-d122-4bc6-baf9-e85891d6d382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:29.534453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06580de5-d122-4bc6-baf9-e85891d6d382 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:29.538374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06580de5-d122-4bc6-baf9-e85891d6d382 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:29.538863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06580de5-d122-4bc6-baf9-e85891d6d382 admin admin] PATH_INFO: `/v3/projects/28edd9a0624c499e940d1cccc3fbd0ce/users/70982e52ac1241b2896721316c6e5379/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:29.580785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06580de5-d122-4bc6-baf9-e85891d6d382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28edd9a0624c499e940d1cccc3fbd0ce, user_id=70982e52ac1241b2896721316c6e5379, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:29.583922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06580de5-d122-4bc6-baf9-e85891d6d382 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:29.627875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a536-2393-4974-92cf-fdd427df48a4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:29.628322 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3044/6090] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:29 2022] PUT /identity/v3/projects/28edd9a0624c499e940d1cccc3fbd0ce/users/70982e52ac1241b2896721316c6e5379/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:29.660331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a536-2393-4974-92cf-fdd427df48a4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:29.671710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:29.799123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16f88c-dfd2-4dd3-986c-cc287964a6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:29.799911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16f88c-dfd2-4dd3-986c-cc287964a6ca admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:29.800297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16f88c-dfd2-4dd3-986c-cc287964a6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:29.801161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16f88c-dfd2-4dd3-986c-cc287964a6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:29.802633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16f88c-dfd2-4dd3-986c-cc287964a6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:29.807025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16f88c-dfd2-4dd3-986c-cc287964a6ca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:29.815612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06580de5-d122-4bc6-baf9-e85891d6d382 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:29.817804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-06580de5-d122-4bc6-baf9-e85891d6d382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:29.852028 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3047/6091] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:29.859075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16f88c-dfd2-4dd3-986c-cc287964a6ca admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:29.878701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16f88c-dfd2-4dd3-986c-cc287964a6ca admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:29.892223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa43de0-e542-4467-b7b2-368225041dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:29.893107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa43de0-e542-4467-b7b2-368225041dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:29.893390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa43de0-e542-4467-b7b2-368225041dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:29.893661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa43de0-e542-4467-b7b2-368225041dbe admin admin] PATH_INFO: `/v3/projects/28edd9a0624c499e940d1cccc3fbd0ce/users/70982e52ac1241b2896721316c6e5379/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:29.900241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa43de0-e542-4467-b7b2-368225041dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=28edd9a0624c499e940d1cccc3fbd0ce, user_id=70982e52ac1241b2896721316c6e5379, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:29.902381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa43de0-e542-4467-b7b2-368225041dbe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:29.927172 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3048/6092] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:29 2022] PUT /identity/v3/projects/28edd9a0624c499e940d1cccc3fbd0ce/users/70982e52ac1241b2896721316c6e5379/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:29.940082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb192c7-a017-4482-92ce-d6042cdab706 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:29.940356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb192c7-a017-4482-92ce-d6042cdab706 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:29.940601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb192c7-a017-4482-92ce-d6042cdab706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:30.013573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06580de5-d122-4bc6-baf9-e85891d6d382 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:30.040327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1fb192c7-a017-4482-92ce-d6042cdab706 None None] MFA Rules not processed for user `70982e52ac1241b2896721316c6e5379`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:30.041697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1fb192c7-a017-4482-92ce-d6042cdab706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:30.176752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41891ead-8bf7-4293-b39c-836ae6035433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:30.177746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41891ead-8bf7-4293-b39c-836ae6035433 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:30.183371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41891ead-8bf7-4293-b39c-836ae6035433 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:30.184598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41891ead-8bf7-4293-b39c-836ae6035433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:30.228408 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3049/6093] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:30.267221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb192c7-a017-4482-92ce-d6042cdab706 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:30.268827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1fb192c7-a017-4482-92ce-d6042cdab706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:30.294397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb192c7-a017-4482-92ce-d6042cdab706 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:30.304119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaed0e-91a1-4e65-9404-bc55cfdf38c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:30.304986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaed0e-91a1-4e65-9404-bc55cfdf38c1 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:30.306559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaed0e-91a1-4e65-9404-bc55cfdf38c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:30.307038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaed0e-91a1-4e65-9404-bc55cfdf38c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:30.334594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaed0e-91a1-4e65-9404-bc55cfdf38c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:30.336372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaed0e-91a1-4e65-9404-bc55cfdf38c1 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:30.372565 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3050/6094] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:14:30.388743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41891ead-8bf7-4293-b39c-836ae6035433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:30.393219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41891ead-8bf7-4293-b39c-836ae6035433 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:30.422685 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3045/6095] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:14:30.834994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaed0e-91a1-4e65-9404-bc55cfdf38c1 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:30.836499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-62eaed0e-91a1-4e65-9404-bc55cfdf38c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:30.886619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaed0e-91a1-4e65-9404-bc55cfdf38c1 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:30.898077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:30.899026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:30.913400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:30.914064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] PATH_INFO: `/v3/projects/ea52f7e0604f43f98145459c650dd716` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:30.917321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea52f7e0604f43f98145459c650dd716)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:30.923297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:31.097250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea52f7e0604f43f98145459c650dd716', 'request_id': 'req-6f17e9d7-de90-488c-b821-b5603be20b6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:31.121813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea52f7e0604f43f98145459c650dd716', 'request_id': 'req-6f17e9d7-de90-488c-b821-b5603be20b6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:31.123112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fd269-7788-40dc-9ab4-c149ffc15207 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:31.123936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fd269-7788-40dc-9ab4-c149ffc15207 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:31.124393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fd269-7788-40dc-9ab4-c149ffc15207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:31.135389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea52f7e0604f43f98145459c650dd716', 'request_id': 'req-6f17e9d7-de90-488c-b821-b5603be20b6b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:31.141201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:31.174753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] The token cache is being invalidate because project ea52f7e0604f43f98145459c650dd716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:31.175286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:31.176402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f17e9d7-de90-488c-b821-b5603be20b6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:31.178292 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3051/6096] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:30 2022] DELETE /identity/v3/projects/ea52f7e0604f43f98145459c650dd716 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:31.183340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d106d3-b918-474d-a957-41e985f04438 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:31.184385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d106d3-b918-474d-a957-41e985f04438 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:31.184840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d106d3-b918-474d-a957-41e985f04438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:31.232979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e4d106d3-b918-474d-a957-41e985f04438 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:31.235509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e4d106d3-b918-474d-a957-41e985f04438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:31.251943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ed3fd269-7788-40dc-9ab4-c149ffc15207 None None] MFA Rules not processed for user `d5c689e427a04bfc96717e2ae8d78b58`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:31.253300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ed3fd269-7788-40dc-9ab4-c149ffc15207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:31.343758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:31.417782 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3052/6097] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:31.426899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29be2-d7db-4c49-af16-8c2d650f8a19 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:31.427379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29be2-d7db-4c49-af16-8c2d650f8a19 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:31.427836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29be2-d7db-4c49-af16-8c2d650f8a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:31.490013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ffa29be2-d7db-4c49-af16-8c2d650f8a19 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:31.493557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ffa29be2-d7db-4c49-af16-8c2d650f8a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:31.521203 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3053/6098] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:31.538389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa29be2-d7db-4c49-af16-8c2d650f8a19 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:31.569441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa29be2-d7db-4c49-af16-8c2d650f8a19 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:31.606901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757c86c-1aa7-46cf-8c00-28a6bc606daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:31.607568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757c86c-1aa7-46cf-8c00-28a6bc606daa admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:31.608012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757c86c-1aa7-46cf-8c00-28a6bc606daa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:31.608137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757c86c-1aa7-46cf-8c00-28a6bc606daa admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:31.608781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757c86c-1aa7-46cf-8c00-28a6bc606daa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:31.611630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757c86c-1aa7-46cf-8c00-28a6bc606daa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:31.613467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757c86c-1aa7-46cf-8c00-28a6bc606daa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:31.645865 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3046/6099] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:31 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 522 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:31.646876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fd269-7788-40dc-9ab4-c149ffc15207 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:31.656321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ed3fd269-7788-40dc-9ab4-c149ffc15207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:31.676431 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3054/6100] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:31.681379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757c86c-1aa7-46cf-8c00-28a6bc606daa admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:31.701931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757c86c-1aa7-46cf-8c00-28a6bc606daa admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:31.717568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fd269-7788-40dc-9ab4-c149ffc15207 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:31.731731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51f6ae-0840-47b4-8dfb-e438fa3b757b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:31.732928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51f6ae-0840-47b4-8dfb-e438fa3b757b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:31.733360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51f6ae-0840-47b4-8dfb-e438fa3b757b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:31.733817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51f6ae-0840-47b4-8dfb-e438fa3b757b admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:31.734831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51f6ae-0840-47b4-8dfb-e438fa3b757b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:31.737220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51f6ae-0840-47b4-8dfb-e438fa3b757b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:31.742919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8892ca4-de3a-4d46-979c-9c2339f73850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:31.743547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8892ca4-de3a-4d46-979c-9c2339f73850 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:31.743827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8892ca4-de3a-4d46-979c-9c2339f73850 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:31.744094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8892ca4-de3a-4d46-979c-9c2339f73850 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:31.751933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8892ca4-de3a-4d46-979c-9c2339f73850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:31.753189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8892ca4-de3a-4d46-979c-9c2339f73850 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:31.824936 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3047/6101] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:31 2022] POST /identity/v3/projects => generated 403 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:31.828930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8892ca4-de3a-4d46-979c-9c2339f73850 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:31.846206 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3055/6102] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:31 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:14:31.854200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51f6ae-0840-47b4-8dfb-e438fa3b757b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:32.066719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8892ca4-de3a-4d46-979c-9c2339f73850 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:32.094287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51f6ae-0840-47b4-8dfb-e438fa3b757b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:32.100942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f2ea1-287a-42eb-9ba7-606a67521339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:32.101590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f2ea1-287a-42eb-9ba7-606a67521339 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:32.101863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f2ea1-287a-42eb-9ba7-606a67521339 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:32.102135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f2ea1-287a-42eb-9ba7-606a67521339 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:32.102764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f2ea1-287a-42eb-9ba7-606a67521339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:32.104688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f2ea1-287a-42eb-9ba7-606a67521339 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:32.123283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a43d6-b569-4fa3-91b4-107172957543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:32.124287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a43d6-b569-4fa3-91b4-107172957543 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:32.124768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a43d6-b569-4fa3-91b4-107172957543 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:32.125273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a43d6-b569-4fa3-91b4-107172957543 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:32.126101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a43d6-b569-4fa3-91b4-107172957543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:32.128281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a43d6-b569-4fa3-91b4-107172957543 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:32.196008 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3056/6103] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:31 2022] POST /identity/v3/users => generated 283 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:32.201864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a43d6-b569-4fa3-91b4-107172957543 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:32.229775 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3048/6104] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:31 2022] POST /identity/v3/users => generated 308 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:32.231826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a43d6-b569-4fa3-91b4-107172957543 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:32.235351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f2ea1-287a-42eb-9ba7-606a67521339 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:32.258603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd1d38-1a75-4392-a1cd-776d3ac82f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:32.259349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd1d38-1a75-4392-a1cd-776d3ac82f32 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:32.259797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd1d38-1a75-4392-a1cd-776d3ac82f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:32.260179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd1d38-1a75-4392-a1cd-776d3ac82f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:32.260962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd1d38-1a75-4392-a1cd-776d3ac82f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:32.263339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd1d38-1a75-4392-a1cd-776d3ac82f32 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:32.271690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f2ea1-287a-42eb-9ba7-606a67521339 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:32.287135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f3c64-02d8-4b60-b380-2d63379be4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:32.287821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f3c64-02d8-4b60-b380-2d63379be4cf admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:32.288107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f3c64-02d8-4b60-b380-2d63379be4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:32.288427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f3c64-02d8-4b60-b380-2d63379be4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:32.289565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f3c64-02d8-4b60-b380-2d63379be4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:32.292584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f3c64-02d8-4b60-b380-2d63379be4cf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:32.309963 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3057/6105] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:32.311097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd1d38-1a75-4392-a1cd-776d3ac82f32 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:32.339702 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3049/6106] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:32.349404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f3c64-02d8-4b60-b380-2d63379be4cf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:32.353112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ac6f3c64-02d8-4b60-b380-2d63379be4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:32.356058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd1d38-1a75-4392-a1cd-776d3ac82f32 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:32.382799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f3c64-02d8-4b60-b380-2d63379be4cf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:32.383732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8573eba-c858-4393-b45a-edad467d9751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:32.387141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573eba-c858-4393-b45a-edad467d9751 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:32.394068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573eba-c858-4393-b45a-edad467d9751 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:32.397486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:32.397907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573eba-c858-4393-b45a-edad467d9751 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:32.398293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:32.398757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8573eba-c858-4393-b45a-edad467d9751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:32.398887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:32.399179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] PATH_INFO: `/v3/projects/cf0f13e49c894adeacebaa41f8683a71` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:32.400607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8573eba-c858-4393-b45a-edad467d9751 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:32.401452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0f13e49c894adeacebaa41f8683a71)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:32.407614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:32.440894 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3058/6107] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:32.441841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8573eba-c858-4393-b45a-edad467d9751 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:32.482256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8573eba-c858-4393-b45a-edad467d9751 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:32.510599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12d4b6-b4b1-496b-8d37-35e67ba06e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:32.511564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12d4b6-b4b1-496b-8d37-35e67ba06e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:32.512768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12d4b6-b4b1-496b-8d37-35e67ba06e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:32.513360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12d4b6-b4b1-496b-8d37-35e67ba06e16 admin admin] PATH_INFO: `/v3/projects/3d7bb9dfa9e6468587b04f9f44d0d4cb/users/81ac63fbc3774606a45c26ec5fc131d3/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:32.534037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:32.545744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0f13e49c894adeacebaa41f8683a71', 'request_id': 'req-f815ecdf-5e7c-424f-986e-d83a6c0c806a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:32.550466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12d4b6-b4b1-496b-8d37-35e67ba06e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7bb9dfa9e6468587b04f9f44d0d4cb, user_id=81ac63fbc3774606a45c26ec5fc131d3, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:32.555234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12d4b6-b4b1-496b-8d37-35e67ba06e16 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:32.567481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0f13e49c894adeacebaa41f8683a71', 'request_id': 'req-f815ecdf-5e7c-424f-986e-d83a6c0c806a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:32.590672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0f13e49c894adeacebaa41f8683a71', 'request_id': 'req-f815ecdf-5e7c-424f-986e-d83a6c0c806a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:32.610773 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3059/6108] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:32 2022] PUT /identity/v3/projects/3d7bb9dfa9e6468587b04f9f44d0d4cb/users/81ac63fbc3774606a45c26ec5fc131d3/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:32.612639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12d4b6-b4b1-496b-8d37-35e67ba06e16 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:32.621978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:32.643713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] The token cache is being invalidate because project cf0f13e49c894adeacebaa41f8683a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:32.644011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:32.648920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:32.653080 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3050/6109] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:32 2022] DELETE /identity/v3/projects/cf0f13e49c894adeacebaa41f8683a71 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:32.660782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:32.666620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:32.770176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12d4b6-b4b1-496b-8d37-35e67ba06e16 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:32.777846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee157c-3543-46d0-9032-2e7b9c1f0f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:32.778922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee157c-3543-46d0-9032-2e7b9c1f0f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:32.779230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee157c-3543-46d0-9032-2e7b9c1f0f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:32.779829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee157c-3543-46d0-9032-2e7b9c1f0f45 admin admin] PATH_INFO: `/v3/projects/25e7b1e90c0b40d7960b578605ea987c/users/84218be29d4445c5ad36cb4dfda1ed5e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:32.805805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee157c-3543-46d0-9032-2e7b9c1f0f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e7b1e90c0b40d7960b578605ea987c, user_id=84218be29d4445c5ad36cb4dfda1ed5e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:32.811405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee157c-3543-46d0-9032-2e7b9c1f0f45 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:32.833609 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3060/6110] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:32 2022] PUT /identity/v3/projects/25e7b1e90c0b40d7960b578605ea987c/users/84218be29d4445c5ad36cb4dfda1ed5e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:32.834976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e6f49-cf85-4202-894a-84fbc6621207 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:32.835335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e6f49-cf85-4202-894a-84fbc6621207 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:32.835663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e6f49-cf85-4202-894a-84fbc6621207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:32.870575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2c2e6f49-cf85-4202-894a-84fbc6621207 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:32.874191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2c2e6f49-cf85-4202-894a-84fbc6621207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:32.893614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ecdf-5e7c-424f-986e-d83a6c0c806a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:33.042049 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3061/6111] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:33.043802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e55ac-c12a-4aad-b5dc-352422310410 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:33.044421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e55ac-c12a-4aad-b5dc-352422310410 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:33.045501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e55ac-c12a-4aad-b5dc-352422310410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:33.078257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56167e72-db87-47ec-8de3-8e9d5f4c8fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:33.079606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56167e72-db87-47ec-8de3-8e9d5f4c8fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:33.080170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56167e72-db87-47ec-8de3-8e9d5f4c8fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:33.083045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56167e72-db87-47ec-8de3-8e9d5f4c8fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:33.083835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56167e72-db87-47ec-8de3-8e9d5f4c8fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:33.086316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56167e72-db87-47ec-8de3-8e9d5f4c8fe4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:33.109674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-eb0e55ac-c12a-4aad-b5dc-352422310410 None None] MFA Rules not processed for user `84218be29d4445c5ad36cb4dfda1ed5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:33.111028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eb0e55ac-c12a-4aad-b5dc-352422310410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:33.111771 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3051/6112] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:33.113234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45563bf-b460-4be5-be34-f801f66e4ee9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:33.115163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45563bf-b460-4be5-be34-f801f66e4ee9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:33.115573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45563bf-b460-4be5-be34-f801f66e4ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:33.156919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d45563bf-b460-4be5-be34-f801f66e4ee9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:33.160231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d45563bf-b460-4be5-be34-f801f66e4ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:33.192734 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3052/6113] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:33.194002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45563bf-b460-4be5-be34-f801f66e4ee9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:33.231178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45563bf-b460-4be5-be34-f801f66e4ee9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:33.276658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cd4a7-12bc-4c28-bc90-cfc160bb6f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:33.282354 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3062/6114] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:33.283598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e55ac-c12a-4aad-b5dc-352422310410 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:33.283951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cd4a7-12bc-4c28-bc90-cfc160bb6f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:33.284225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cd4a7-12bc-4c28-bc90-cfc160bb6f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:33.284495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cd4a7-12bc-4c28-bc90-cfc160bb6f45 admin admin] PATH_INFO: `/v3/projects/3d7bb9dfa9e6468587b04f9f44d0d4cb/users/81ac63fbc3774606a45c26ec5fc131d3/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:33.298444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cd4a7-12bc-4c28-bc90-cfc160bb6f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7bb9dfa9e6468587b04f9f44d0d4cb, user_id=81ac63fbc3774606a45c26ec5fc131d3, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:33.300474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:33.308692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e55ac-c12a-4aad-b5dc-352422310410 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:33.312870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cd4a7-12bc-4c28-bc90-cfc160bb6f45 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:33.322589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88517796-8363-408f-aa96-61db6f7428fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:33.323629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88517796-8363-408f-aa96-61db6f7428fe admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:33.324182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88517796-8363-408f-aa96-61db6f7428fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:33.324934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88517796-8363-408f-aa96-61db6f7428fe admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:33.325901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88517796-8363-408f-aa96-61db6f7428fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:33.326850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88517796-8363-408f-aa96-61db6f7428fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:33.331331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88517796-8363-408f-aa96-61db6f7428fe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:33.365751 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3053/6115] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:33 2022] PUT /identity/v3/projects/3d7bb9dfa9e6468587b04f9f44d0d4cb/users/81ac63fbc3774606a45c26ec5fc131d3/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:33.368559 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3063/6116] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:33.373248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cd4a7-12bc-4c28-bc90-cfc160bb6f45 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:33.433868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88517796-8363-408f-aa96-61db6f7428fe admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:33.436168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-88517796-8363-408f-aa96-61db6f7428fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:33.563221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cd4a7-12bc-4c28-bc90-cfc160bb6f45 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:33.583235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d57f1d-1aba-48af-80d1-0ade4fb328bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:33.584021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d57f1d-1aba-48af-80d1-0ade4fb328bb admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:33.584407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d57f1d-1aba-48af-80d1-0ade4fb328bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:33.584797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d57f1d-1aba-48af-80d1-0ade4fb328bb admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:33.585581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d57f1d-1aba-48af-80d1-0ade4fb328bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:33.588448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d57f1d-1aba-48af-80d1-0ade4fb328bb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:33.628066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88517796-8363-408f-aa96-61db6f7428fe admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:33.683126 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3054/6117] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:33 2022] POST /identity/v3/projects => generated 375 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:33.687531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d57f1d-1aba-48af-80d1-0ade4fb328bb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:33.708582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:33.822202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff8d60-e0b1-4329-a6a2-04c0d423312e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:33.823033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff8d60-e0b1-4329-a6a2-04c0d423312e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:33.823487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff8d60-e0b1-4329-a6a2-04c0d423312e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:33.823879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff8d60-e0b1-4329-a6a2-04c0d423312e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:33.935186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d57f1d-1aba-48af-80d1-0ade4fb328bb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:33.942147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c61d8-64e2-4b4f-8308-3b393dde9de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:33.942993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c61d8-64e2-4b4f-8308-3b393dde9de3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:33.943266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c61d8-64e2-4b4f-8308-3b393dde9de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:33.943596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c61d8-64e2-4b4f-8308-3b393dde9de3 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:33.944159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c61d8-64e2-4b4f-8308-3b393dde9de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:33.946126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c61d8-64e2-4b4f-8308-3b393dde9de3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:33.971627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff8d60-e0b1-4329-a6a2-04c0d423312e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:33.980241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff8d60-e0b1-4329-a6a2-04c0d423312e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:34.001828 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3055/6118] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:33 2022] POST /identity/v3/users => generated 288 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:14:34.003508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c61d8-64e2-4b4f-8308-3b393dde9de3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:34.005894 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3064/6119] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:14:34.023616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c61d8-64e2-4b4f-8308-3b393dde9de3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:34.048706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b934e4-2e81-4c69-854b-e523bf324e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:34.049378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b934e4-2e81-4c69-854b-e523bf324e05 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:34.049658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b934e4-2e81-4c69-854b-e523bf324e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:34.049929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b934e4-2e81-4c69-854b-e523bf324e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:34.051328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b934e4-2e81-4c69-854b-e523bf324e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:34.052204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b934e4-2e81-4c69-854b-e523bf324e05 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:34.079716 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3056/6120] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:34.083484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff8d60-e0b1-4329-a6a2-04c0d423312e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:34.103603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff8d60-e0b1-4329-a6a2-04c0d423312e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:34.119817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53ff60-3876-48a0-b460-36a52cd8fe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:34.120828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53ff60-3876-48a0-b460-36a52cd8fe6a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:34.122075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53ff60-3876-48a0-b460-36a52cd8fe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:34.122664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53ff60-3876-48a0-b460-36a52cd8fe6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:34.124066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53ff60-3876-48a0-b460-36a52cd8fe6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:34.131529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53ff60-3876-48a0-b460-36a52cd8fe6a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:34.184355 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3065/6121] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:34.185388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b934e4-2e81-4c69-854b-e523bf324e05 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:34.204217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b934e4-2e81-4c69-854b-e523bf324e05 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:34.218766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737d52b-25a2-41c0-9830-d3dca3f11101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:34.219378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737d52b-25a2-41c0-9830-d3dca3f11101 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:34.220246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737d52b-25a2-41c0-9830-d3dca3f11101 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:34.220524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737d52b-25a2-41c0-9830-d3dca3f11101 admin admin] PATH_INFO: `/v3/projects/3e292697043f4f15a033d35bc9381f7e/users/104b746e21444ff8a555fe6cbae1e7f6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:34.255248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737d52b-25a2-41c0-9830-d3dca3f11101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e292697043f4f15a033d35bc9381f7e, user_id=104b746e21444ff8a555fe6cbae1e7f6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:34.273979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737d52b-25a2-41c0-9830-d3dca3f11101 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:34.317680 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3057/6122] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:34 2022] PUT /identity/v3/projects/3e292697043f4f15a033d35bc9381f7e/users/104b746e21444ff8a555fe6cbae1e7f6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:34.319640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347d72c-9961-443e-bca4-60d24764161a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:34.320692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347d72c-9961-443e-bca4-60d24764161a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:34.324618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347d72c-9961-443e-bca4-60d24764161a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:34.406953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e347d72c-9961-443e-bca4-60d24764161a None None] MFA Rules not processed for user `104b746e21444ff8a555fe6cbae1e7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:34.408569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e347d72c-9961-443e-bca4-60d24764161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:34.527710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:34.595598 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3066/6123] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:34.600644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737d52b-25a2-41c0-9830-d3dca3f11101 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:34.633893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737d52b-25a2-41c0-9830-d3dca3f11101 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:34.641582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50154f-d746-4065-85d9-09a2b4167f04 tempest-IdentityCatalogTest-1030849829 tempest-IdentityCatalogTest-1030849829-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '104b746e21444ff8a555fe6cbae1e7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e292697043f4f15a033d35bc9381f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:34.642236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50154f-d746-4065-85d9-09a2b4167f04 tempest-IdentityCatalogTest-1030849829 tempest-IdentityCatalogTest-1030849829-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:34.642514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50154f-d746-4065-85d9-09a2b4167f04 tempest-IdentityCatalogTest-1030849829 tempest-IdentityCatalogTest-1030849829-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:34.642787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50154f-d746-4065-85d9-09a2b4167f04 tempest-IdentityCatalogTest-1030849829 tempest-IdentityCatalogTest-1030849829-project] PATH_INFO: `/v3/auth/catalog` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:34.643361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50154f-d746-4065-85d9-09a2b4167f04 tempest-IdentityCatalogTest-1030849829 tempest-IdentityCatalogTest-1030849829-project] RBAC: Authorizing `identity:get_auth_catalog()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:34.645019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50154f-d746-4065-85d9-09a2b4167f04 tempest-IdentityCatalogTest-1030849829 tempest-IdentityCatalogTest-1030849829-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:34.651225 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3058/6124] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 21:14:34 2022] GET /identity/v3/auth/catalog => generated 2658 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:14:34.652997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347d72c-9961-443e-bca4-60d24764161a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:34.837741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347d72c-9961-443e-bca4-60d24764161a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:34.855195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:34.857017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:34.858096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:34.858593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] PATH_INFO: `/v3/users/104b746e21444ff8a555fe6cbae1e7f6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:34.860451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=104b746e21444ff8a555fe6cbae1e7f6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:34.867232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:34.904924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50154f-d746-4065-85d9-09a2b4167f04 tempest-IdentityCatalogTest-1030849829 tempest-IdentityCatalogTest-1030849829-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:34.907814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f50154f-d746-4065-85d9-09a2b4167f04 tempest-IdentityCatalogTest-1030849829 tempest-IdentityCatalogTest-1030849829-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:34.954899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50154f-d746-4065-85d9-09a2b4167f04 tempest-IdentityCatalogTest-1030849829 tempest-IdentityCatalogTest-1030849829-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:34.973301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:34.976125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:34.976125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:34.976125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] PATH_INFO: `/v3/users/33f21d81aa1d46d0aacb4f36e8a5fb79` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:34.977530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f21d81aa1d46d0aacb4f36e8a5fb79)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:34.983217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:34.998574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '104b746e21444ff8a555fe6cbae1e7f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:35.021450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '104b746e21444ff8a555fe6cbae1e7f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:35.040546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '104b746e21444ff8a555fe6cbae1e7f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:35.053320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '104b746e21444ff8a555fe6cbae1e7f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:35.054468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9eba99b1-4fe7-46d9-b372-79fd62f01dd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '104b746e21444ff8a555fe6cbae1e7f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:35.069283 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3067/6125] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:34 2022] DELETE /identity/v3/users/104b746e21444ff8a555fe6cbae1e7f6 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:35.071711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c507818-3eaa-4550-9030-65ad27ed6404 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:35.071711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c507818-3eaa-4550-9030-65ad27ed6404 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:35.071711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c507818-3eaa-4550-9030-65ad27ed6404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:35.129484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5c507818-3eaa-4550-9030-65ad27ed6404 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:35.132350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5c507818-3eaa-4550-9030-65ad27ed6404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:35.172155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f21d81aa1d46d0aacb4f36e8a5fb79'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:35.191143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f21d81aa1d46d0aacb4f36e8a5fb79'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:35.244470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f21d81aa1d46d0aacb4f36e8a5fb79'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:35.264909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f21d81aa1d46d0aacb4f36e8a5fb79'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:35.270823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-42cb8275-5bf1-4c00-9f0a-c98fddbef88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f21d81aa1d46d0aacb4f36e8a5fb79'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:35.272185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:35.288154 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3059/6126] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:34 2022] DELETE /identity/v3/users/33f21d81aa1d46d0aacb4f36e8a5fb79 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:35.296772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce729bc-dddb-458a-8e18-b24a5444f2de None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:35.297938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce729bc-dddb-458a-8e18-b24a5444f2de None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:35.299712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce729bc-dddb-458a-8e18-b24a5444f2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:35.305191 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3068/6127] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:35.328109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c507818-3eaa-4550-9030-65ad27ed6404 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:35.329713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5c507818-3eaa-4550-9030-65ad27ed6404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:35.337610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5ce729bc-dddb-458a-8e18-b24a5444f2de None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:35.339720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5ce729bc-dddb-458a-8e18-b24a5444f2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:35.391163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:35.456748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c507818-3eaa-4550-9030-65ad27ed6404 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:35.504288 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3060/6128] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:35.520906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce729bc-dddb-458a-8e18-b24a5444f2de None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:35.651788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbcc52-900b-4646-aadb-c34d1c9dc330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:35.654242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbcc52-900b-4646-aadb-c34d1c9dc330 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:35.656178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbcc52-900b-4646-aadb-c34d1c9dc330 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:35.656178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbcc52-900b-4646-aadb-c34d1c9dc330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:35.683102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbcc52-900b-4646-aadb-c34d1c9dc330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:35.686405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbcc52-900b-4646-aadb-c34d1c9dc330 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:35.694980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce729bc-dddb-458a-8e18-b24a5444f2de None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:35.705382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228105d3-31a5-4f36-a1bb-cc46e6abfbc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:35.706504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228105d3-31a5-4f36-a1bb-cc46e6abfbc4 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:35.706999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228105d3-31a5-4f36-a1bb-cc46e6abfbc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:35.707565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228105d3-31a5-4f36-a1bb-cc46e6abfbc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:35.717600 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3069/6129] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:14:35.729682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228105d3-31a5-4f36-a1bb-cc46e6abfbc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:35.732422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228105d3-31a5-4f36-a1bb-cc46e6abfbc4 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:35.750182 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3061/6130] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:14:37.137663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbcc52-900b-4646-aadb-c34d1c9dc330 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:37.148916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2adbcc52-900b-4646-aadb-c34d1c9dc330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:37.182411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228105d3-31a5-4f36-a1bb-cc46e6abfbc4 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:37.185497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-228105d3-31a5-4f36-a1bb-cc46e6abfbc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:37.217387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbcc52-900b-4646-aadb-c34d1c9dc330 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:37.222655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228105d3-31a5-4f36-a1bb-cc46e6abfbc4 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:37.223215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:37.224130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:37.224825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:37.225541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] PATH_INFO: `/v3/projects/3e292697043f4f15a033d35bc9381f7e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:37.226977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e292697043f4f15a033d35bc9381f7e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:37.229384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:37.252877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:37.255464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:37.255996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:37.258533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] PATH_INFO: `/v3/projects/dc81f60338274538872024f0ada0f52f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:37.265497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc81f60338274538872024f0ada0f52f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:37.268667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:37.372788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e292697043f4f15a033d35bc9381f7e', 'request_id': 'req-497aab4c-a69e-4194-86c9-49e67dd0e8fd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:37.391401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc81f60338274538872024f0ada0f52f', 'request_id': 'req-8ff113ad-fff4-4423-9e0e-2b72720a7228'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:37.395412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e292697043f4f15a033d35bc9381f7e', 'request_id': 'req-497aab4c-a69e-4194-86c9-49e67dd0e8fd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:37.406931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e292697043f4f15a033d35bc9381f7e', 'request_id': 'req-497aab4c-a69e-4194-86c9-49e67dd0e8fd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:37.420470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc81f60338274538872024f0ada0f52f', 'request_id': 'req-8ff113ad-fff4-4423-9e0e-2b72720a7228'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:37.446227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc81f60338274538872024f0ada0f52f', 'request_id': 'req-8ff113ad-fff4-4423-9e0e-2b72720a7228'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:37.447728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] The token cache is being invalidate because project 3e292697043f4f15a033d35bc9381f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:37.449623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:37.454021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-497aab4c-a69e-4194-86c9-49e67dd0e8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:37.457304 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3070/6131] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:37 2022] DELETE /identity/v3/projects/3e292697043f4f15a033d35bc9381f7e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:37.462058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b691e-efb0-45d6-beb7-d8b9374304a4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:37.462693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b691e-efb0-45d6-beb7-d8b9374304a4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:37.463203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b691e-efb0-45d6-beb7-d8b9374304a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:37.467039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] The token cache is being invalidate because project dc81f60338274538872024f0ada0f52f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:37.467328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:37.468235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8ff113ad-fff4-4423-9e0e-2b72720a7228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:37.470285 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3062/6132] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:37 2022] DELETE /identity/v3/projects/dc81f60338274538872024f0ada0f52f => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:37.474814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f2dc-d31d-4568-9150-8aac46b8c414 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:37.475339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f2dc-d31d-4568-9150-8aac46b8c414 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:37.475767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f2dc-d31d-4568-9150-8aac46b8c414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:37.516694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-642b691e-efb0-45d6-beb7-d8b9374304a4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:37.517975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-53e8f2dc-d31d-4568-9150-8aac46b8c414 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:37.519465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-53e8f2dc-d31d-4568-9150-8aac46b8c414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:37.521196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-642b691e-efb0-45d6-beb7-d8b9374304a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:37.625083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:37.627238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:37.656903 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3063/6133] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:14:37.664378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee59e93-6379-4908-a073-73de5c7007e4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:37.664657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee59e93-6379-4908-a073-73de5c7007e4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:37.664926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee59e93-6379-4908-a073-73de5c7007e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:37.706888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-dee59e93-6379-4908-a073-73de5c7007e4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:37.707137 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3071/6134] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:14:37.708095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dee59e93-6379-4908-a073-73de5c7007e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:37.712672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb8a50-741c-4e64-a08f-ac74409e6292 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:37.713103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb8a50-741c-4e64-a08f-ac74409e6292 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:37.713658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb8a50-741c-4e64-a08f-ac74409e6292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:37.728545 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3064/6135] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:14:37.732846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee59e93-6379-4908-a073-73de5c7007e4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:37.755443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee59e93-6379-4908-a073-73de5c7007e4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:37.766489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57855e0-0c3b-4f0e-a561-c005b0e4b82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:37.767123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57855e0-0c3b-4f0e-a561-c005b0e4b82a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:37.767400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57855e0-0c3b-4f0e-a561-c005b0e4b82a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:37.767662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57855e0-0c3b-4f0e-a561-c005b0e4b82a admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:37.768319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57855e0-0c3b-4f0e-a561-c005b0e4b82a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:37.768617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57855e0-0c3b-4f0e-a561-c005b0e4b82a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:37.770259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57855e0-0c3b-4f0e-a561-c005b0e4b82a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:37.770392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4edb8a50-741c-4e64-a08f-ac74409e6292 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:37.773922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4edb8a50-741c-4e64-a08f-ac74409e6292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:37.795536 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3065/6136] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:14:37.797529 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3072/6137] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:37.804294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57855e0-0c3b-4f0e-a561-c005b0e4b82a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:37.806081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb8a50-741c-4e64-a08f-ac74409e6292 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:37.826262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57855e0-0c3b-4f0e-a561-c005b0e4b82a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:37.831887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb8a50-741c-4e64-a08f-ac74409e6292 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:37.843212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a292fae-0899-4616-8de0-a3075441fa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:37.843831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a292fae-0899-4616-8de0-a3075441fa6b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:37.844106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a292fae-0899-4616-8de0-a3075441fa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:37.844371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a292fae-0899-4616-8de0-a3075441fa6b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:37.844993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a292fae-0899-4616-8de0-a3075441fa6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:37.846909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a292fae-0899-4616-8de0-a3075441fa6b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:37.859051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52b76c-65df-45f8-801c-75afba928202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:37.860122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52b76c-65df-45f8-801c-75afba928202 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:37.860392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52b76c-65df-45f8-801c-75afba928202 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:37.860654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52b76c-65df-45f8-801c-75afba928202 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:37.861610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52b76c-65df-45f8-801c-75afba928202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:37.861893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52b76c-65df-45f8-801c-75afba928202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:37.863497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52b76c-65df-45f8-801c-75afba928202 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:37.902540 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3073/6138] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:37.906696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52b76c-65df-45f8-801c-75afba928202 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:37.929608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52b76c-65df-45f8-801c-75afba928202 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:37.936602 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3066/6139] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:37 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:14:37.940722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a292fae-0899-4616-8de0-a3075441fa6b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:37.945406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce562920-6360-4bc2-bbf2-25b956d1c193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:37.946016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce562920-6360-4bc2-bbf2-25b956d1c193 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:37.946286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce562920-6360-4bc2-bbf2-25b956d1c193 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:37.946553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce562920-6360-4bc2-bbf2-25b956d1c193 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:37.947190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce562920-6360-4bc2-bbf2-25b956d1c193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:37.949701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce562920-6360-4bc2-bbf2-25b956d1c193 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:38.032042 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3074/6140] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:37 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:38.036242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce562920-6360-4bc2-bbf2-25b956d1c193 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:38.112128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a292fae-0899-4616-8de0-a3075441fa6b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:38.245745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce562920-6360-4bc2-bbf2-25b956d1c193 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:38.260604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9db728-7e34-412d-867c-229493672ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:38.261990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9db728-7e34-412d-867c-229493672ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.264298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9db728-7e34-412d-867c-229493672ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.264982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9db728-7e34-412d-867c-229493672ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.265927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9db728-7e34-412d-867c-229493672ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:38.270938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9db728-7e34-412d-867c-229493672ed4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:38.290604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81fc10-805d-4150-b72f-6520a4eaf969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:38.291403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81fc10-805d-4150-b72f-6520a4eaf969 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.291804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81fc10-805d-4150-b72f-6520a4eaf969 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.292177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81fc10-805d-4150-b72f-6520a4eaf969 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.295155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81fc10-805d-4150-b72f-6520a4eaf969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:38.297537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81fc10-805d-4150-b72f-6520a4eaf969 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:38.322770 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3075/6141] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:38 2022] POST /identity/v3/users => generated 287 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:38.327147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9db728-7e34-412d-867c-229493672ed4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:38.350918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9db728-7e34-412d-867c-229493672ed4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:38.352281 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3067/6142] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:37 2022] POST /identity/v3/users => generated 288 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:38.357057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81fc10-805d-4150-b72f-6520a4eaf969 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:38.365553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff865a95-9ef9-404e-9b93-299275165296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:38.366456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff865a95-9ef9-404e-9b93-299275165296 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.367888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff865a95-9ef9-404e-9b93-299275165296 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.368807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff865a95-9ef9-404e-9b93-299275165296 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.369607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff865a95-9ef9-404e-9b93-299275165296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:38.375917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff865a95-9ef9-404e-9b93-299275165296 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:38.386361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81fc10-805d-4150-b72f-6520a4eaf969 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:38.399529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c69e8-ccae-4189-8da5-4b8fd21b32a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:38.400328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c69e8-ccae-4189-8da5-4b8fd21b32a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.400949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c69e8-ccae-4189-8da5-4b8fd21b32a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.403289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c69e8-ccae-4189-8da5-4b8fd21b32a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.404085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c69e8-ccae-4189-8da5-4b8fd21b32a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:38.406150 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3076/6143] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:38.407783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c69e8-ccae-4189-8da5-4b8fd21b32a1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:38.409881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff865a95-9ef9-404e-9b93-299275165296 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:38.432074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff865a95-9ef9-404e-9b93-299275165296 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:38.437004 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3068/6144] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:38.439551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c69e8-ccae-4189-8da5-4b8fd21b32a1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:38.455530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c69e8-ccae-4189-8da5-4b8fd21b32a1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:38.456412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440ffc2-c8ac-4c74-a96d-6be10244f376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:38.457302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440ffc2-c8ac-4c74-a96d-6be10244f376 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.459747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440ffc2-c8ac-4c74-a96d-6be10244f376 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.459747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440ffc2-c8ac-4c74-a96d-6be10244f376 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.459747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440ffc2-c8ac-4c74-a96d-6be10244f376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:38.460857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440ffc2-c8ac-4c74-a96d-6be10244f376 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:38.471932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c27ac-9549-4cf0-a9e1-949f9f5139ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:38.472707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c27ac-9549-4cf0-a9e1-949f9f5139ff admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.472989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c27ac-9549-4cf0-a9e1-949f9f5139ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.474353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c27ac-9549-4cf0-a9e1-949f9f5139ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.475097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c27ac-9549-4cf0-a9e1-949f9f5139ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:38.482342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c27ac-9549-4cf0-a9e1-949f9f5139ff admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:38.492391 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3077/6145] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:38.495915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440ffc2-c8ac-4c74-a96d-6be10244f376 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:38.515903 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3069/6146] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:38.516865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c27ac-9549-4cf0-a9e1-949f9f5139ff admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:38.525298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440ffc2-c8ac-4c74-a96d-6be10244f376 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:38.538477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad21ec0-540a-4bad-9765-1b5cb1d5be50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:38.539517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad21ec0-540a-4bad-9765-1b5cb1d5be50 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.540005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad21ec0-540a-4bad-9765-1b5cb1d5be50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.540448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad21ec0-540a-4bad-9765-1b5cb1d5be50 admin admin] PATH_INFO: `/v3/projects/7f3dc26b6a3e4b1dba49b9e6dc395834/users/b914f687682d4a90856bdf4ac7cda45f/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.541396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c27ac-9549-4cf0-a9e1-949f9f5139ff admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:38.553271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160d6a5-94d5-4468-98e7-3ab6186b80a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:38.554852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d6a5-94d5-4468-98e7-3ab6186b80a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.555192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d6a5-94d5-4468-98e7-3ab6186b80a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.555475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d6a5-94d5-4468-98e7-3ab6186b80a8 admin admin] PATH_INFO: `/v3/projects/7235801c632e485daf077a72510e7d38/users/efed6a0917264b5f8db15533c450a9fe/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.588966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad21ec0-540a-4bad-9765-1b5cb1d5be50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3dc26b6a3e4b1dba49b9e6dc395834, user_id=b914f687682d4a90856bdf4ac7cda45f, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:38.597981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160d6a5-94d5-4468-98e7-3ab6186b80a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7235801c632e485daf077a72510e7d38, user_id=efed6a0917264b5f8db15533c450a9fe, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:38.604551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160d6a5-94d5-4468-98e7-3ab6186b80a8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:38.607866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad21ec0-540a-4bad-9765-1b5cb1d5be50 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:38.648658 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3078/6148] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:38 2022] PUT /identity/v3/projects/7f3dc26b6a3e4b1dba49b9e6dc395834/users/b914f687682d4a90856bdf4ac7cda45f/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:38.649135 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3070/6148] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:38 2022] PUT /identity/v3/projects/7235801c632e485daf077a72510e7d38/users/efed6a0917264b5f8db15533c450a9fe/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:38.653847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259144b7-3027-4c6e-ab3b-8a83b5253670 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.654130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259144b7-3027-4c6e-ab3b-8a83b5253670 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.654401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259144b7-3027-4c6e-ab3b-8a83b5253670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.660712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511a27-8c9a-4c8f-8d17-69dc0a3174b4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.661331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511a27-8c9a-4c8f-8d17-69dc0a3174b4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.661826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511a27-8c9a-4c8f-8d17-69dc0a3174b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.679187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:38.731829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-71511a27-8c9a-4c8f-8d17-69dc0a3174b4 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:38.733252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-71511a27-8c9a-4c8f-8d17-69dc0a3174b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:38.742910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-259144b7-3027-4c6e-ab3b-8a83b5253670 None None] MFA Rules not processed for user `efed6a0917264b5f8db15533c450a9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:38.744314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-259144b7-3027-4c6e-ab3b-8a83b5253670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:38.833400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:38.892456 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3071/6149] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:14:38.895415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511a27-8c9a-4c8f-8d17-69dc0a3174b4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:38.916749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511a27-8c9a-4c8f-8d17-69dc0a3174b4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:38.926491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3079/6150] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:38.930034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363860d-04e9-4e8a-a727-259bc69e2822 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:38.930034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363860d-04e9-4e8a-a727-259bc69e2822 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:38.930034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363860d-04e9-4e8a-a727-259bc69e2822 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:38.930348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363860d-04e9-4e8a-a727-259bc69e2822 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:38.938556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259144b7-3027-4c6e-ab3b-8a83b5253670 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:38.939872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-259144b7-3027-4c6e-ab3b-8a83b5253670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:39.036707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b363860d-04e9-4e8a-a727-259bc69e2822 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:39.055234 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3072/6151] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:14:38 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:39.071772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259144b7-3027-4c6e-ab3b-8a83b5253670 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:39.076725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ab7cb-17b5-425c-9759-d43b06135f2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:39.077861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab7cb-17b5-425c-9759-d43b06135f2a service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:39.088906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab7cb-17b5-425c-9759-d43b06135f2a service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:39.089382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab7cb-17b5-425c-9759-d43b06135f2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:39.119070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ab7cb-17b5-425c-9759-d43b06135f2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:39.122374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ab7cb-17b5-425c-9759-d43b06135f2a service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:39.162942 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3080/6152] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:14:40.030476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b977b44-f74f-45ae-a14d-b917b972ea4a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:40.031840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b977b44-f74f-45ae-a14d-b917b972ea4a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:40.032990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b977b44-f74f-45ae-a14d-b917b972ea4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:40.059513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1e515-cea0-435b-9f7e-97f9f6633dda None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:40.060005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1e515-cea0-435b-9f7e-97f9f6633dda None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:40.060102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1e515-cea0-435b-9f7e-97f9f6633dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:40.117636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f4f1e515-cea0-435b-9f7e-97f9f6633dda None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:40.118873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f4f1e515-cea0-435b-9f7e-97f9f6633dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:40.135367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1b977b44-f74f-45ae-a14d-b917b972ea4a None None] MFA Rules not processed for user `81ac63fbc3774606a45c26ec5fc131d3`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:40.140465 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3081/6153] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:40 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:14:40.142875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1b977b44-f74f-45ae-a14d-b917b972ea4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:40.146148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa9bba-8eeb-4fd9-95cf-53ab925ae9f0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:40.146508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa9bba-8eeb-4fd9-95cf-53ab925ae9f0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:40.154576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa9bba-8eeb-4fd9-95cf-53ab925ae9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:40.247032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-18fa9bba-8eeb-4fd9-95cf-53ab925ae9f0 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:40.255803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-18fa9bba-8eeb-4fd9-95cf-53ab925ae9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:40.290619 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3082/6154] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:40.297171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa9bba-8eeb-4fd9-95cf-53ab925ae9f0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:40.325500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa9bba-8eeb-4fd9-95cf-53ab925ae9f0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:40.347691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3b41-244b-4943-996e-da0d31c59ecc tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:40.348364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3b41-244b-4943-996e-da0d31c59ecc tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:40.348644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3b41-244b-4943-996e-da0d31c59ecc tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:40.348916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3b41-244b-4943-996e-da0d31c59ecc tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:40.496771 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3073/6155] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:40 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-8f8b3b41-244b-4943-996e-da0d31c59ecc tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:14:40.503289 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Oct 15 21:14:40.504481 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:14:40.509078 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3083/6156] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:14:40 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 188 bytes in 208 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 15 21:14:40.510611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b977b44-f74f-45ae-a14d-b917b972ea4a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:40.557204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b977b44-f74f-45ae-a14d-b917b972ea4a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:40.575012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c5d77-ebf0-490c-8d8f-52f316a4de71 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:40.575673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c5d77-ebf0-490c-8d8f-52f316a4de71 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:40.575944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c5d77-ebf0-490c-8d8f-52f316a4de71 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:40.576208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c5d77-ebf0-490c-8d8f-52f316a4de71 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-8c4c5d77-ebf0-490c-8d8f-52f316a4de71 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:14:40.771435 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Oct 15 21:14:40.772783 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:14:40.778954 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3074/6157] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:14:40 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 188 bytes in 266 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 15 21:14:40.781014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3b41-244b-4943-996e-da0d31c59ecc tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:40.797225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3b41-244b-4943-996e-da0d31c59ecc tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:40.809228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c831d-5bd9-4d04-a40d-3aa8fbac3c3e tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:40.809876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c831d-5bd9-4d04-a40d-3aa8fbac3c3e tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:40.810157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c831d-5bd9-4d04-a40d-3aa8fbac3c3e tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:40.810424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c831d-5bd9-4d04-a40d-3aa8fbac3c3e tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:40.943501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa8c831d-5bd9-4d04-a40d-3aa8fbac3c3e tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:40.964121 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3084/6158] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:14:40 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:41.835971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c5d77-ebf0-490c-8d8f-52f316a4de71 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:41.838542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8c4c5d77-ebf0-490c-8d8f-52f316a4de71 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:41.970220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833b043-c371-4300-b0a7-928887b9940b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:41.971223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833b043-c371-4300-b0a7-928887b9940b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:41.971720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833b043-c371-4300-b0a7-928887b9940b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:41.976660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:42.058473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6833b043-c371-4300-b0a7-928887b9940b None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:42.059847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6833b043-c371-4300-b0a7-928887b9940b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:42.085662 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3085/6159] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:41 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 21:14:42.087659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af422cfd-e316-40d9-9882-28ecd457abf3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:42.088046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af422cfd-e316-40d9-9882-28ecd457abf3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:42.088414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af422cfd-e316-40d9-9882-28ecd457abf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:42.157162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c5d77-ebf0-490c-8d8f-52f316a4de71 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:42.172854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:42.173710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:42.173980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:42.174260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] PATH_INFO: `/v3/users/4aa826125c6b4bbea7dec3d2dc5200a1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:42.175580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa826125c6b4bbea7dec3d2dc5200a1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:42.182797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:42.190382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-af422cfd-e316-40d9-9882-28ecd457abf3 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:42.194286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af422cfd-e316-40d9-9882-28ecd457abf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:42.251261 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3086/6160] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:42.255500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af422cfd-e316-40d9-9882-28ecd457abf3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:42.283130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af422cfd-e316-40d9-9882-28ecd457abf3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:42.306591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86563351-3ab6-4f12-b496-150eabbfc72b tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:42.307573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86563351-3ab6-4f12-b496-150eabbfc72b tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:42.307924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86563351-3ab6-4f12-b496-150eabbfc72b tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:42.308638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86563351-3ab6-4f12-b496-150eabbfc72b tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:42.380185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa826125c6b4bbea7dec3d2dc5200a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:42.411841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa826125c6b4bbea7dec3d2dc5200a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:42.469158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-86563351-3ab6-4f12-b496-150eabbfc72b tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:42.471771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa826125c6b4bbea7dec3d2dc5200a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:42.489256 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3087/6161] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:14:42 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:42.500032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa826125c6b4bbea7dec3d2dc5200a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:42.500920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa826125c6b4bbea7dec3d2dc5200a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:42.537729 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3075/6162] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:41 2022] DELETE /identity/v3/users/4aa826125c6b4bbea7dec3d2dc5200a1 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:42.538538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9c43-2c4d-4847-a112-dfdf581d6fdb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:42.538652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9c43-2c4d-4847-a112-dfdf581d6fdb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:42.538928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9c43-2c4d-4847-a112-dfdf581d6fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:42.598893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-083a9c43-2c4d-4847-a112-dfdf581d6fdb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:42.601003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-083a9c43-2c4d-4847-a112-dfdf581d6fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:42.651184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:UPDATE=2,keystone:INSERT=4,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:42.814630 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3088/6163] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:14:42.911626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:42.913867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:43.138860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe964b-26ca-4670-86c0-f2321f707032 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:43.150295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9fc9b-bd54-4e50-a4b1-c9f7e66e4eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:43.151277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9fc9b-bd54-4e50-a4b1-c9f7e66e4eec service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:43.152512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9fc9b-bd54-4e50-a4b1-c9f7e66e4eec service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:43.153299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9fc9b-bd54-4e50-a4b1-c9f7e66e4eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:43.207436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9fc9b-bd54-4e50-a4b1-c9f7e66e4eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:43.211629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9fc9b-bd54-4e50-a4b1-c9f7e66e4eec service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:43.265619 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3076/6164] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:43.496825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260d9f1-25af-438b-950c-e71d96a893a6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:43.498442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260d9f1-25af-438b-950c-e71d96a893a6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:43.498893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260d9f1-25af-438b-950c-e71d96a893a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:43.539759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0260d9f1-25af-438b-950c-e71d96a893a6 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:43.541280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0260d9f1-25af-438b-950c-e71d96a893a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:43.546399 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3089/6165] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:43 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:14:43.551159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75f050-bf3a-41f7-b1f8-bd694b86a0b3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:43.551462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75f050-bf3a-41f7-b1f8-bd694b86a0b3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:43.551702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75f050-bf3a-41f7-b1f8-bd694b86a0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:43.621814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5a75f050-bf3a-41f7-b1f8-bd694b86a0b3 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:43.625387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5a75f050-bf3a-41f7-b1f8-bd694b86a0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:43.799843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:43.834334 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3077/6166] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:43.838205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260d9f1-25af-438b-950c-e71d96a893a6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:43.862071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260d9f1-25af-438b-950c-e71d96a893a6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:43.868207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b252b-0306-443c-aaae-d528651cf44d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:43.868878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b252b-0306-443c-aaae-d528651cf44d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:43.869180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b252b-0306-443c-aaae-d528651cf44d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:43.869632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b252b-0306-443c-aaae-d528651cf44d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:43.922599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc38664-dd12-489e-b06b-13d0936f815c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:43.923743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc38664-dd12-489e-b06b-13d0936f815c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:43.924175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc38664-dd12-489e-b06b-13d0936f815c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:43.995683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7dc38664-dd12-489e-b06b-13d0936f815c None None] MFA Rules not processed for user `379c929d93b54b9a9751defeef161f57`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:43.996929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7dc38664-dd12-489e-b06b-13d0936f815c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:44.011677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b54b252b-0306-443c-aaae-d528651cf44d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:44.045745 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3090/6167] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:14:43 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:44.237665 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3078/6168] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:44.254331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b252b-0306-443c-aaae-d528651cf44d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:44.256404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b54b252b-0306-443c-aaae-d528651cf44d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:44.434581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b252b-0306-443c-aaae-d528651cf44d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:44.454079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbfa319-a213-4327-a864-576f1dd39b07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:44.455209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfa319-a213-4327-a864-576f1dd39b07 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:44.455652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfa319-a213-4327-a864-576f1dd39b07 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:44.456095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfa319-a213-4327-a864-576f1dd39b07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:44.491235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbfa319-a213-4327-a864-576f1dd39b07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:44.493964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbfa319-a213-4327-a864-576f1dd39b07 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:44.510221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3091/6169] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:45.053555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b8a43-fc71-4ded-897b-46117e93a16c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:45.054570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b8a43-fc71-4ded-897b-46117e93a16c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:45.055144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b8a43-fc71-4ded-897b-46117e93a16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:45.116767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3b9b8a43-fc71-4ded-897b-46117e93a16c None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:45.118878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3b9b8a43-fc71-4ded-897b-46117e93a16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:45.138455 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3079/6170] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:45 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:14:45.138807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84ace-a746-45f6-8206-0a23a5ff0ec7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:45.139396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84ace-a746-45f6-8206-0a23a5ff0ec7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:45.139568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84ace-a746-45f6-8206-0a23a5ff0ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:45.212277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d7e84ace-a746-45f6-8206-0a23a5ff0ec7 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:45.213844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d7e84ace-a746-45f6-8206-0a23a5ff0ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:45.232690 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3092/6171] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:45.237359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b8a43-fc71-4ded-897b-46117e93a16c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:45.266417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b8a43-fc71-4ded-897b-46117e93a16c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:45.279687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dc184-ca66-4e60-aff2-e20e6b8deb60 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:45.280390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dc184-ca66-4e60-aff2-e20e6b8deb60 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:45.286002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dc184-ca66-4e60-aff2-e20e6b8deb60 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:45.287733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dc184-ca66-4e60-aff2-e20e6b8deb60 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:45.437487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd2dc184-ca66-4e60-aff2-e20e6b8deb60 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:45.463084 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3080/6172] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:14:45 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:46.279007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e84ace-a746-45f6-8206-0a23a5ff0ec7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:46.293952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d7e84ace-a746-45f6-8206-0a23a5ff0ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:46.336491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e84ace-a746-45f6-8206-0a23a5ff0ec7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:46.365005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:46.365965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:46.369894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:46.381487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] PATH_INFO: `/v3/projects/8bd54f2c1f2048ef9ca26761f3484ba4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:46.387254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd54f2c1f2048ef9ca26761f3484ba4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:46.411356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:46.466868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4c68b-3001-441a-90b6-e51974dabab5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:46.467419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4c68b-3001-441a-90b6-e51974dabab5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:46.467664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4c68b-3001-441a-90b6-e51974dabab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:46.520220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0ac4c68b-3001-441a-90b6-e51974dabab5 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:46.523718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0ac4c68b-3001-441a-90b6-e51974dabab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:46.529300 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3081/6173] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:46 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:14:46.588247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:46.599780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd54f2c1f2048ef9ca26761f3484ba4', 'request_id': 'req-5637a108-9e3f-4343-9aff-4437811bb247'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:46.621140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd54f2c1f2048ef9ca26761f3484ba4', 'request_id': 'req-5637a108-9e3f-4343-9aff-4437811bb247'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:46.644607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd54f2c1f2048ef9ca26761f3484ba4', 'request_id': 'req-5637a108-9e3f-4343-9aff-4437811bb247'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:46.677833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] The token cache is being invalidate because project 8bd54f2c1f2048ef9ca26761f3484ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:46.678812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:46.679931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:46.681934 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3093/6174] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:46 2022] DELETE /identity/v3/projects/8bd54f2c1f2048ef9ca26761f3484ba4 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:46.689533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4c68b-3001-441a-90b6-e51974dabab5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:46.713302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:46.715623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:46.862558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4c68b-3001-441a-90b6-e51974dabab5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:46.883473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:46.884130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:46.884418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:46.884728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] PATH_INFO: `/v3/users/70982e52ac1241b2896721316c6e5379` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:46.888184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=70982e52ac1241b2896721316c6e5379)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:46.890161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:46.925527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a108-9e3f-4343-9aff-4437811bb247 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:46.951132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:46.951801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:46.952080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:46.952348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] PATH_INFO: `/v3/users/d5c689e427a04bfc96717e2ae8d78b58` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:46.953996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c689e427a04bfc96717e2ae8d78b58)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:46.955964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:47.027363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:47.056755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70982e52ac1241b2896721316c6e5379'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:47.059505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c689e427a04bfc96717e2ae8d78b58'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:47.071786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70982e52ac1241b2896721316c6e5379'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:47.084077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c689e427a04bfc96717e2ae8d78b58'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:47.101672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c689e427a04bfc96717e2ae8d78b58'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:47.113852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70982e52ac1241b2896721316c6e5379'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:47.129130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c689e427a04bfc96717e2ae8d78b58'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:47.134242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c689e427a04bfc96717e2ae8d78b58'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:47.134448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70982e52ac1241b2896721316c6e5379'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:47.135410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-918e52f4-b9be-4eed-8b87-0510bc0c6f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70982e52ac1241b2896721316c6e5379'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:47.150488 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3082/6175] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:46 2022] DELETE /identity/v3/users/70982e52ac1241b2896721316c6e5379 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:47.174237 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3094/6176] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:46 2022] DELETE /identity/v3/users/d5c689e427a04bfc96717e2ae8d78b58 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:47.535471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589b1b3-3e90-43eb-8d61-2b17edded321 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:47.536372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589b1b3-3e90-43eb-8d61-2b17edded321 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:47.536809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589b1b3-3e90-43eb-8d61-2b17edded321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:47.581681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2589b1b3-3e90-43eb-8d61-2b17edded321 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:47.583663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2589b1b3-3e90-43eb-8d61-2b17edded321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:47.743968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:47.746792 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3083/6177] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:47.750145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:47.756707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c690856-9e8a-43a0-9b9c-061c5d7789c1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:47.757155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c690856-9e8a-43a0-9b9c-061c5d7789c1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:47.757739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c690856-9e8a-43a0-9b9c-061c5d7789c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:47.797764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1c690856-9e8a-43a0-9b9c-061c5d7789c1 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:47.800703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1c690856-9e8a-43a0-9b9c-061c5d7789c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:47.811317 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3084/6178] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:47 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:14:47.812895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55374fa-d939-434a-a207-8e2fbb02ecde None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:47.813633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55374fa-d939-434a-a207-8e2fbb02ecde None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:47.813941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55374fa-d939-434a-a207-8e2fbb02ecde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:47.915444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:47.919567 ncdv-master-1 devstack@keystone.service[99099]: WARNING keystone.server.flask.application [None req-b55374fa-d939-434a-a207-8e2fbb02ecde None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 15 21:14:47.923617 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3085/6179] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:47 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 110 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 15 21:14:47.943398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670de19-1e89-4471-92cf-240f90026b2b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:47.943967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670de19-1e89-4471-92cf-240f90026b2b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:47.944482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670de19-1e89-4471-92cf-240f90026b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:47.967465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2596194-6f9a-44bf-9aa7-cc57ff071081 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:47.980585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:47.981619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:47.982075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:47.982504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] PATH_INFO: `/v3/projects/a3230a9bbf8a4c36b0ce4cf9bb3e743f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:47.985783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3230a9bbf8a4c36b0ce4cf9bb3e743f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:47.988176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:48.035896 ncdv-master-1 devstack@keystone.service[99099]: WARNING keystone.server.flask.application [None req-6670de19-1e89-4471-92cf-240f90026b2b None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 15 21:14:48.048633 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3086/6180] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:47 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 106 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 15 21:14:48.056203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e43bf-63f9-4d1f-a555-66cfe67e13ff None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:48.056887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e43bf-63f9-4d1f-a555-66cfe67e13ff None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:48.057367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e43bf-63f9-4d1f-a555-66cfe67e13ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:48.154654 ncdv-master-1 devstack@keystone.service[99099]: WARNING keystone.server.flask.application [None req-889e43bf-63f9-4d1f-a555-66cfe67e13ff None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 15 21:14:48.161462 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3087/6181] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:48 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 107 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 15 21:14:48.164600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3230a9bbf8a4c36b0ce4cf9bb3e743f', 'request_id': 'req-8fd7b134-8a81-4cb2-9664-eb278328dc2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.185131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3230a9bbf8a4c36b0ce4cf9bb3e743f', 'request_id': 'req-8fd7b134-8a81-4cb2-9664-eb278328dc2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.211261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3230a9bbf8a4c36b0ce4cf9bb3e743f', 'request_id': 'req-8fd7b134-8a81-4cb2-9664-eb278328dc2a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.259803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] The token cache is being invalidate because project a3230a9bbf8a4c36b0ce4cf9bb3e743f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:48.260210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.263414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.264202 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3095/6182] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:47 2022] DELETE /identity/v3/projects/a3230a9bbf8a4c36b0ce4cf9bb3e743f => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:48.272518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e43bf-63f9-4d1f-a555-66cfe67e13ff None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:48.274969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-889e43bf-63f9-4d1f-a555-66cfe67e13ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:48.383368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:48.384925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:48.491813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=1,keystone:UPDATE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:48.498441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e43bf-63f9-4d1f-a555-66cfe67e13ff None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:48.506116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:48.509348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:48.509610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:48.509914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] PATH_INFO: `/v3/users/81ac63fbc3774606a45c26ec5fc131d3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:48.511431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ac63fbc3774606a45c26ec5fc131d3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:48.515759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:48.577766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7b134-8a81-4cb2-9664-eb278328dc2a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:48.599855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:48.600896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:48.601299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:48.601870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] PATH_INFO: `/v3/projects/28edd9a0624c499e940d1cccc3fbd0ce` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:48.606791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28edd9a0624c499e940d1cccc3fbd0ce)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:48.616173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:48.620214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ac63fbc3774606a45c26ec5fc131d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.631787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ac63fbc3774606a45c26ec5fc131d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.652564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ac63fbc3774606a45c26ec5fc131d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.669929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ac63fbc3774606a45c26ec5fc131d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.671113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0d8969dd-0e12-4d2c-960f-36bc99b0dedb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ac63fbc3774606a45c26ec5fc131d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.686907 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3088/6183] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:48 2022] DELETE /identity/v3/users/81ac63fbc3774606a45c26ec5fc131d3 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:48.773651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28edd9a0624c499e940d1cccc3fbd0ce', 'request_id': 'req-8baaa042-0879-4219-9e68-07b96e70b991'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.795113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28edd9a0624c499e940d1cccc3fbd0ce', 'request_id': 'req-8baaa042-0879-4219-9e68-07b96e70b991'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.807542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28edd9a0624c499e940d1cccc3fbd0ce', 'request_id': 'req-8baaa042-0879-4219-9e68-07b96e70b991'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.836742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] The token cache is being invalidate because project 28edd9a0624c499e940d1cccc3fbd0ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:48.838887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.841432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8baaa042-0879-4219-9e68-07b96e70b991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:48.847415 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3096/6184] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:48 2022] DELETE /identity/v3/projects/28edd9a0624c499e940d1cccc3fbd0ce => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:48.852158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465145af-82d6-4f4b-955e-e24e9904e033 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:48.852451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465145af-82d6-4f4b-955e-e24e9904e033 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:48.852705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465145af-82d6-4f4b-955e-e24e9904e033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:48.895817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-465145af-82d6-4f4b-955e-e24e9904e033 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:48.899652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-465145af-82d6-4f4b-955e-e24e9904e033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:49.042936 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3089/6185] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:49.050366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eeebf-a259-4d64-916e-fb4282ffecc2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:49.051118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eeebf-a259-4d64-916e-fb4282ffecc2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:49.051607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eeebf-a259-4d64-916e-fb4282ffecc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:49.098794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e10eeebf-a259-4d64-916e-fb4282ffecc2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:49.102753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e10eeebf-a259-4d64-916e-fb4282ffecc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:49.132314 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3097/6186] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:49.135993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465145af-82d6-4f4b-955e-e24e9904e033 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:49.147981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465145af-82d6-4f4b-955e-e24e9904e033 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:49.156140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8c29f-64a2-43ba-896f-364061dc5de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:49.156890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8c29f-64a2-43ba-896f-364061dc5de2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:49.157271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8c29f-64a2-43ba-896f-364061dc5de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:49.158084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8c29f-64a2-43ba-896f-364061dc5de2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:49.158339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8c29f-64a2-43ba-896f-364061dc5de2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:49.158814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8c29f-64a2-43ba-896f-364061dc5de2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:49.160676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8c29f-64a2-43ba-896f-364061dc5de2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:49.190700 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3090/6187] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:49.201982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eeebf-a259-4d64-916e-fb4282ffecc2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:49.223560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:49.237578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eeebf-a259-4d64-916e-fb4282ffecc2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:49.269913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d15bbe-8ac0-4268-97b5-53b453a86691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:49.272178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d15bbe-8ac0-4268-97b5-53b453a86691 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:49.273197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d15bbe-8ac0-4268-97b5-53b453a86691 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:49.274044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d15bbe-8ac0-4268-97b5-53b453a86691 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:49.278716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d15bbe-8ac0-4268-97b5-53b453a86691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:49.362589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d15bbe-8ac0-4268-97b5-53b453a86691 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:49.382907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3098/6188] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:49 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:49.387901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8c29f-64a2-43ba-896f-364061dc5de2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:49.410626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8c29f-64a2-43ba-896f-364061dc5de2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:49.670337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54e81a-7adb-4df0-bf7b-c1dc0df7dd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:49.671023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54e81a-7adb-4df0-bf7b-c1dc0df7dd74 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:49.671301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54e81a-7adb-4df0-bf7b-c1dc0df7dd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:49.671617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54e81a-7adb-4df0-bf7b-c1dc0df7dd74 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:49.672273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54e81a-7adb-4df0-bf7b-c1dc0df7dd74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:49.689373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54e81a-7adb-4df0-bf7b-c1dc0df7dd74 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:49.711487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:49.740514 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3091/6189] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:49 2022] POST /identity/v3/users => generated 292 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:49.744140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d15bbe-8ac0-4268-97b5-53b453a86691 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:49.769916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d15bbe-8ac0-4268-97b5-53b453a86691 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:49.781370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2d3ca-39a1-41aa-b51b-27be666dd37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:49.782643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2d3ca-39a1-41aa-b51b-27be666dd37a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:49.783059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2d3ca-39a1-41aa-b51b-27be666dd37a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:49.786388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2d3ca-39a1-41aa-b51b-27be666dd37a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:49.787370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2d3ca-39a1-41aa-b51b-27be666dd37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:49.791224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2d3ca-39a1-41aa-b51b-27be666dd37a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:49.820815 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3099/6190] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:49.825387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54e81a-7adb-4df0-bf7b-c1dc0df7dd74 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:49.842646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54e81a-7adb-4df0-bf7b-c1dc0df7dd74 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:49.870607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8d1be-4f45-43e6-ab8a-86ca5cf48a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:49.871517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8d1be-4f45-43e6-ab8a-86ca5cf48a8f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:49.871958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8d1be-4f45-43e6-ab8a-86ca5cf48a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:49.872518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8d1be-4f45-43e6-ab8a-86ca5cf48a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:49.874898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8d1be-4f45-43e6-ab8a-86ca5cf48a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:49.880049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8d1be-4f45-43e6-ab8a-86ca5cf48a8f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:49.908501 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3092/6191] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:49.910061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2d3ca-39a1-41aa-b51b-27be666dd37a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:49.924375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2d3ca-39a1-41aa-b51b-27be666dd37a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:49.935239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a6b6c-3006-40ff-841a-4da00a86cd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:49.937406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a6b6c-3006-40ff-841a-4da00a86cd4c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:49.938397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a6b6c-3006-40ff-841a-4da00a86cd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:49.940036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a6b6c-3006-40ff-841a-4da00a86cd4c admin admin] PATH_INFO: `/v3/projects/44e7b272dfde47d7a9a4ef56e93088d3/users/0a453fc9ecb24af2b526bcfc90d3f584/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:49.952263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f059fbb-2464-4c90-a6ca-6506cfcac723 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:49.952690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f059fbb-2464-4c90-a6ca-6506cfcac723 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:49.953111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f059fbb-2464-4c90-a6ca-6506cfcac723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:49.995022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3f059fbb-2464-4c90-a6ca-6506cfcac723 None None] MFA Rules not processed for user `379c929d93b54b9a9751defeef161f57`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:50.001058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3f059fbb-2464-4c90-a6ca-6506cfcac723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:50.006857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a6b6c-3006-40ff-841a-4da00a86cd4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44e7b272dfde47d7a9a4ef56e93088d3, user_id=0a453fc9ecb24af2b526bcfc90d3f584, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:50.009628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a6b6c-3006-40ff-841a-4da00a86cd4c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:50.056986 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3100/6192] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:49 2022] PUT /identity/v3/projects/44e7b272dfde47d7a9a4ef56e93088d3/users/0a453fc9ecb24af2b526bcfc90d3f584/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:50.067878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd49166-f6da-4d09-b49d-a0887373f539 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:50.068665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd49166-f6da-4d09-b49d-a0887373f539 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:50.071202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd49166-f6da-4d09-b49d-a0887373f539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:50.220508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4dd49166-f6da-4d09-b49d-a0887373f539 None None] MFA Rules not processed for user `0a453fc9ecb24af2b526bcfc90d3f584`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:50.221781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4dd49166-f6da-4d09-b49d-a0887373f539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:50.228252 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3093/6193] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:50.377190 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3101/6194] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:50.397683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f059fbb-2464-4c90-a6ca-6506cfcac723 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:50.399409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3f059fbb-2464-4c90-a6ca-6506cfcac723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:50.508479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd49166-f6da-4d09-b49d-a0887373f539 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:50.509581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f059fbb-2464-4c90-a6ca-6506cfcac723 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:50.509953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4dd49166-f6da-4d09-b49d-a0887373f539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:50.684099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877cd51-65da-45f3-ad9a-3e2ed797f199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:50.684766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877cd51-65da-45f3-ad9a-3e2ed797f199 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:50.685124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877cd51-65da-45f3-ad9a-3e2ed797f199 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:50.685366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877cd51-65da-45f3-ad9a-3e2ed797f199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:50.706791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877cd51-65da-45f3-ad9a-3e2ed797f199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:50.708419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877cd51-65da-45f3-ad9a-3e2ed797f199 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:50.728121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd49166-f6da-4d09-b49d-a0887373f539 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:50.734059 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3094/6195] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:50.750392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:50.751220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:50.751677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:50.752288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] PATH_INFO: `/v3/projects/3d7bb9dfa9e6468587b04f9f44d0d4cb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:50.753710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d7bb9dfa9e6468587b04f9f44d0d4cb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:50.756160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:50.833702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:50.849346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d7bb9dfa9e6468587b04f9f44d0d4cb', 'request_id': 'req-c165b457-b6cd-44fb-9cf3-1739606daa0b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:50.860131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d7bb9dfa9e6468587b04f9f44d0d4cb', 'request_id': 'req-c165b457-b6cd-44fb-9cf3-1739606daa0b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:50.872992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d7bb9dfa9e6468587b04f9f44d0d4cb', 'request_id': 'req-c165b457-b6cd-44fb-9cf3-1739606daa0b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:50.898622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] The token cache is being invalidate because project 3d7bb9dfa9e6468587b04f9f44d0d4cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:50.901611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:50.902524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c165b457-b6cd-44fb-9cf3-1739606daa0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:50.904344 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3102/6196] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:50 2022] DELETE /identity/v3/projects/3d7bb9dfa9e6468587b04f9f44d0d4cb => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:50.909822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b7ac9-1aa6-4d17-b09e-b897af1b64f2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:50.911126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b7ac9-1aa6-4d17-b09e-b897af1b64f2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:50.911505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b7ac9-1aa6-4d17-b09e-b897af1b64f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:50.938283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-695b7ac9-1aa6-4d17-b09e-b897af1b64f2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:50.939815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-695b7ac9-1aa6-4d17-b09e-b897af1b64f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:51.020740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:51.074658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc1362-b5f0-4589-808c-f821ad21f56e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:51.074812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc1362-b5f0-4589-808c-f821ad21f56e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:51.075056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc1362-b5f0-4589-808c-f821ad21f56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:51.081910 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3095/6197] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:51.132576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bbcc1362-b5f0-4589-808c-f821ad21f56e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:51.133911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bbcc1362-b5f0-4589-808c-f821ad21f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:51.159034 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3103/6198] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:51.163112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b7ac9-1aa6-4d17-b09e-b897af1b64f2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:51.175139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b7ac9-1aa6-4d17-b09e-b897af1b64f2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:51.187589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f675-28c1-446d-b9c4-245d6ab16f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:51.188372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f675-28c1-446d-b9c4-245d6ab16f69 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:51.188788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f675-28c1-446d-b9c4-245d6ab16f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:51.189096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f675-28c1-446d-b9c4-245d6ab16f69 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:51.196760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112f675-28c1-446d-b9c4-245d6ab16f69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:51.204668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112f675-28c1-446d-b9c4-245d6ab16f69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:51.208306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112f675-28c1-446d-b9c4-245d6ab16f69 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:51.244518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc1362-b5f0-4589-808c-f821ad21f56e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:51.245757 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3096/6199] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:51.264743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc1362-b5f0-4589-808c-f821ad21f56e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:51.274184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd071-780a-4289-bb83-64906ed98df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:51.275127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd071-780a-4289-bb83-64906ed98df8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:51.275652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd071-780a-4289-bb83-64906ed98df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:51.277269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd071-780a-4289-bb83-64906ed98df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:51.278630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cd071-780a-4289-bb83-64906ed98df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:51.287274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cd071-780a-4289-bb83-64906ed98df8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:51.372106 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3104/6200] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:51 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:51.373801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f675-28c1-446d-b9c4-245d6ab16f69 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:51.390895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f675-28c1-446d-b9c4-245d6ab16f69 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:51.596034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e59c4-8d2b-4e72-a0d8-59111a38a0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:51.597074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e59c4-8d2b-4e72-a0d8-59111a38a0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:51.597591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e59c4-8d2b-4e72-a0d8-59111a38a0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:51.598051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e59c4-8d2b-4e72-a0d8-59111a38a0a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:51.599371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e59c4-8d2b-4e72-a0d8-59111a38a0a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:51.602458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e59c4-8d2b-4e72-a0d8-59111a38a0a0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:51.664407 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3097/6201] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:51 2022] POST /identity/v3/users => generated 288 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:51.667935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd071-780a-4289-bb83-64906ed98df8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:51.691502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd071-780a-4289-bb83-64906ed98df8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:51.705526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186fadf-ee85-43a7-96b5-e8a120ec23cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:51.706156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186fadf-ee85-43a7-96b5-e8a120ec23cf admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:51.707348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186fadf-ee85-43a7-96b5-e8a120ec23cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:51.707617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186fadf-ee85-43a7-96b5-e8a120ec23cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:51.709173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186fadf-ee85-43a7-96b5-e8a120ec23cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:51.710935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186fadf-ee85-43a7-96b5-e8a120ec23cf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:51.749002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e59c4-8d2b-4e72-a0d8-59111a38a0a0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:51.749948 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3105/6202] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:51.780447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e59c4-8d2b-4e72-a0d8-59111a38a0a0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:51.821846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8617349-fd28-4516-8b7d-ecb03b33bcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:51.822691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8617349-fd28-4516-8b7d-ecb03b33bcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:51.823848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8617349-fd28-4516-8b7d-ecb03b33bcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:51.824265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8617349-fd28-4516-8b7d-ecb03b33bcf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:51.825312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8617349-fd28-4516-8b7d-ecb03b33bcf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:51.828970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8617349-fd28-4516-8b7d-ecb03b33bcf8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:51.854325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186fadf-ee85-43a7-96b5-e8a120ec23cf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:51.871823 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3098/6203] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:51.880337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186fadf-ee85-43a7-96b5-e8a120ec23cf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:51.889596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc979f9-8cdf-443c-bbbb-0843a0859861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:51.890265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc979f9-8cdf-443c-bbbb-0843a0859861 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:51.890734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc979f9-8cdf-443c-bbbb-0843a0859861 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:51.891137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc979f9-8cdf-443c-bbbb-0843a0859861 admin admin] PATH_INFO: `/v3/projects/1cbf839ca453402b8694a15b2e5edf97/users/b70d44ed499143b6af2e45ac03605109/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:51.914347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc979f9-8cdf-443c-bbbb-0843a0859861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cbf839ca453402b8694a15b2e5edf97, user_id=b70d44ed499143b6af2e45ac03605109, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:51.918200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc979f9-8cdf-443c-bbbb-0843a0859861 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:51.947411 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3106/6204] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:51 2022] PUT /identity/v3/projects/1cbf839ca453402b8694a15b2e5edf97/users/b70d44ed499143b6af2e45ac03605109/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:51.957149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeec2b-de6b-4871-8c24-71352dc66b5b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:51.957424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeec2b-de6b-4871-8c24-71352dc66b5b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:51.957696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeec2b-de6b-4871-8c24-71352dc66b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:52.028823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8ffeec2b-de6b-4871-8c24-71352dc66b5b None None] MFA Rules not processed for user `b70d44ed499143b6af2e45ac03605109`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:52.030219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8ffeec2b-de6b-4871-8c24-71352dc66b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:52.219259 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3099/6205] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:52.222328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc979f9-8cdf-443c-bbbb-0843a0859861 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:52.322868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeec2b-de6b-4871-8c24-71352dc66b5b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:52.325433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8ffeec2b-de6b-4871-8c24-71352dc66b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:52.368471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc979f9-8cdf-443c-bbbb-0843a0859861 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:52.375858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b21739-d0a0-4f27-8183-63ab72a5ec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:52.376737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b21739-d0a0-4f27-8183-63ab72a5ec73 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:52.377169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b21739-d0a0-4f27-8183-63ab72a5ec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:52.377952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b21739-d0a0-4f27-8183-63ab72a5ec73 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:52.379742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b21739-d0a0-4f27-8183-63ab72a5ec73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:52.382231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b21739-d0a0-4f27-8183-63ab72a5ec73 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:52.432282 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3107/6206] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:52 2022] POST /identity/v3/projects => generated 375 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:52.433521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b21739-d0a0-4f27-8183-63ab72a5ec73 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:52.466166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:52.487134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeec2b-de6b-4871-8c24-71352dc66b5b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:52.519654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:52.565000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b21739-d0a0-4f27-8183-63ab72a5ec73 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:52.592847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef42e74-8d80-4930-95b9-e6fd1f84dd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:52.593566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef42e74-8d80-4930-95b9-e6fd1f84dd4e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:52.593851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef42e74-8d80-4930-95b9-e6fd1f84dd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:52.596763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef42e74-8d80-4930-95b9-e6fd1f84dd4e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:52.598108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef42e74-8d80-4930-95b9-e6fd1f84dd4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:52.602390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef42e74-8d80-4930-95b9-e6fd1f84dd4e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:52.641363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fafd0-87dc-45d5-8295-6a7780592e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:52.643669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fafd0-87dc-45d5-8295-6a7780592e48 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:52.644204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fafd0-87dc-45d5-8295-6a7780592e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:52.644709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fafd0-87dc-45d5-8295-6a7780592e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:52.648562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f5fafd0-87dc-45d5-8295-6a7780592e48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:52.671404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fafd0-87dc-45d5-8295-6a7780592e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:52.673986 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3108/6207] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:52 2022] POST /identity/v3/users => generated 294 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:52.674449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fafd0-87dc-45d5-8295-6a7780592e48 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:52.681247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef42e74-8d80-4930-95b9-e6fd1f84dd4e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:52.689871 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3100/6208] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:52.709393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef42e74-8d80-4930-95b9-e6fd1f84dd4e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:52.723527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394901a-a14c-4010-9c25-baa9cc701510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:52.724944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394901a-a14c-4010-9c25-baa9cc701510 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:52.725268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394901a-a14c-4010-9c25-baa9cc701510 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:52.726761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394901a-a14c-4010-9c25-baa9cc701510 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:52.727870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394901a-a14c-4010-9c25-baa9cc701510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:52.733442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394901a-a14c-4010-9c25-baa9cc701510 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:52.759001 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3109/6209] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:52.760224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fafd0-87dc-45d5-8295-6a7780592e48 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:52.794476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fafd0-87dc-45d5-8295-6a7780592e48 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:52.803205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f1cd7-8954-46e1-b794-e642afc4ae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:52.803903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f1cd7-8954-46e1-b794-e642afc4ae7b admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:52.804246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f1cd7-8954-46e1-b794-e642afc4ae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:52.804464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f1cd7-8954-46e1-b794-e642afc4ae7b admin admin] PATH_INFO: `/v3/projects/162755a515a74ea9866470e068351434/users/d069b91e777444d5b278419a8333a3bb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:52.841709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f1cd7-8954-46e1-b794-e642afc4ae7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=162755a515a74ea9866470e068351434, user_id=d069b91e777444d5b278419a8333a3bb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:52.848858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f1cd7-8954-46e1-b794-e642afc4ae7b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:52.880113 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3101/6210] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:52 2022] PUT /identity/v3/projects/162755a515a74ea9866470e068351434/users/d069b91e777444d5b278419a8333a3bb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:52.881481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394901a-a14c-4010-9c25-baa9cc701510 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:52.898447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394901a-a14c-4010-9c25-baa9cc701510 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:52.997956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ed33a-b603-45fb-abcb-b23c1a7f0bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:52.998940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ed33a-b603-45fb-abcb-b23c1a7f0bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:52.999546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ed33a-b603-45fb-abcb-b23c1a7f0bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:53.000020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ed33a-b603-45fb-abcb-b23c1a7f0bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:53.000993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ed33a-b603-45fb-abcb-b23c1a7f0bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:53.003018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ed33a-b603-45fb-abcb-b23c1a7f0bc9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:53.026897 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3110/6211] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:53.032247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f1cd7-8954-46e1-b794-e642afc4ae7b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:53.048955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f1cd7-8954-46e1-b794-e642afc4ae7b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:53.058081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c4eea-12db-496c-809c-224f9f883569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:53.059020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c4eea-12db-496c-809c-224f9f883569 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:53.059770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c4eea-12db-496c-809c-224f9f883569 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:53.060171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c4eea-12db-496c-809c-224f9f883569 admin admin] PATH_INFO: `/v3/projects/162755a515a74ea9866470e068351434/users/d069b91e777444d5b278419a8333a3bb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:53.063096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c4eea-12db-496c-809c-224f9f883569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162755a515a74ea9866470e068351434, user_id=d069b91e777444d5b278419a8333a3bb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:53.068117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c4eea-12db-496c-809c-224f9f883569 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:53.100191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63313f-2c83-479d-bc1b-5ec07139ddd0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:53.100497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63313f-2c83-479d-bc1b-5ec07139ddd0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:53.100743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63313f-2c83-479d-bc1b-5ec07139ddd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:53.101936 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3102/6212] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:53 2022] PUT /identity/v3/projects/162755a515a74ea9866470e068351434/users/d069b91e777444d5b278419a8333a3bb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:14:53.151488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ba63313f-2c83-479d-bc1b-5ec07139ddd0 None None] MFA Rules not processed for user `d069b91e777444d5b278419a8333a3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:53.161544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ba63313f-2c83-479d-bc1b-5ec07139ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:53.323654 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3111/6213] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:53.339200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c4eea-12db-496c-809c-224f9f883569 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:53.340979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-833c4eea-12db-496c-809c-224f9f883569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:53.438454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c4eea-12db-496c-809c-224f9f883569 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:53.443748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7658fd-bf33-4c6e-b914-6bfcc6e21099 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:53.444379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7658fd-bf33-4c6e-b914-6bfcc6e21099 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:53.444649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7658fd-bf33-4c6e-b914-6bfcc6e21099 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:53.444914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7658fd-bf33-4c6e-b914-6bfcc6e21099 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:53.617788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7658fd-bf33-4c6e-b914-6bfcc6e21099 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:53.619731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7658fd-bf33-4c6e-b914-6bfcc6e21099 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:53.648489 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3103/6214] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:55.237415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63313f-2c83-479d-bc1b-5ec07139ddd0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:55.238958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ba63313f-2c83-479d-bc1b-5ec07139ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:55.334290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:55.381751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63313f-2c83-479d-bc1b-5ec07139ddd0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:55.402850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:55.403677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:55.404179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:55.404514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] PATH_INFO: `/v3/users/efed6a0917264b5f8db15533c450a9fe` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:55.406043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efed6a0917264b5f8db15533c450a9fe)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:55.408292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:55.518240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efed6a0917264b5f8db15533c450a9fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:55.565086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efed6a0917264b5f8db15533c450a9fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:55.593328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efed6a0917264b5f8db15533c450a9fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:55.628627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efed6a0917264b5f8db15533c450a9fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:55.629882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efed6a0917264b5f8db15533c450a9fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:55.674390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9765c0-24c7-43e6-a29f-4acdbc7268d7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:55.674993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9765c0-24c7-43e6-a29f-4acdbc7268d7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:55.675157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9765c0-24c7-43e6-a29f-4acdbc7268d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:55.687997 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3112/6215] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:14:55 2022] DELETE /identity/v3/users/efed6a0917264b5f8db15533c450a9fe => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:55.712281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ed9765c0-24c7-43e6-a29f-4acdbc7268d7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:55.715712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ed9765c0-24c7-43e6-a29f-4acdbc7268d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:55.739855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:55.994608 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3104/6216] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:56.013331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:56.014940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:56.174681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d0175-8d48-4eb4-b7c0-68c22d30b156 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:56.194944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8e901-8370-4b51-a102-c16f5cdb8fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:56.196131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8e901-8370-4b51-a102-c16f5cdb8fef service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:56.197458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8e901-8370-4b51-a102-c16f5cdb8fef service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:56.198782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8e901-8370-4b51-a102-c16f5cdb8fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:56.232247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8e901-8370-4b51-a102-c16f5cdb8fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:56.234412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8e901-8370-4b51-a102-c16f5cdb8fef service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:56.275490 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3113/6217] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:57.313826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9765c0-24c7-43e6-a29f-4acdbc7268d7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:57.315847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ed9765c0-24c7-43e6-a29f-4acdbc7268d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:57.421686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9765c0-24c7-43e6-a29f-4acdbc7268d7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:57.426432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897df14f-9b79-451c-900d-83b88a829e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:57.427023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897df14f-9b79-451c-900d-83b88a829e75 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:57.427301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897df14f-9b79-451c-900d-83b88a829e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:57.427566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897df14f-9b79-451c-900d-83b88a829e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:57.430945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-897df14f-9b79-451c-900d-83b88a829e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:57.619390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897df14f-9b79-451c-900d-83b88a829e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:57.632047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897df14f-9b79-451c-900d-83b88a829e75 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:57.664960 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3105/6218] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:57.752671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8e901-8370-4b51-a102-c16f5cdb8fef service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:57.756635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a1d8e901-8370-4b51-a102-c16f5cdb8fef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:57.785462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8e901-8370-4b51-a102-c16f5cdb8fef service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:57.792725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:57.793661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:57.794205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:57.794649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] PATH_INFO: `/v3/projects/7235801c632e485daf077a72510e7d38` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:57.798476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7235801c632e485daf077a72510e7d38)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:57.803329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:57.908725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7235801c632e485daf077a72510e7d38', 'request_id': 'req-888d5341-ad19-4c06-9c91-389be5097888'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:57.926851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7235801c632e485daf077a72510e7d38', 'request_id': 'req-888d5341-ad19-4c06-9c91-389be5097888'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:57.939292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7235801c632e485daf077a72510e7d38', 'request_id': 'req-888d5341-ad19-4c06-9c91-389be5097888'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:57.948856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:57.969718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] The token cache is being invalidate because project 7235801c632e485daf077a72510e7d38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:14:57.970997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:57.977204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-888d5341-ad19-4c06-9c91-389be5097888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:57.989692 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3114/6219] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:14:57 2022] DELETE /identity/v3/projects/7235801c632e485daf077a72510e7d38 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:57.994611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fec432-3ef4-4ed9-8a02-7f86da642952 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:57.995451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fec432-3ef4-4ed9-8a02-7f86da642952 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:57.995724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fec432-3ef4-4ed9-8a02-7f86da642952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:58.029934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b0fec432-3ef4-4ed9-8a02-7f86da642952 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:58.031768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b0fec432-3ef4-4ed9-8a02-7f86da642952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:58.148357 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3106/6220] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:58.156550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc87b1d-14e9-4944-be2b-37358c4cd0a9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:58.156854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc87b1d-14e9-4944-be2b-37358c4cd0a9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:58.157137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc87b1d-14e9-4944-be2b-37358c4cd0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:58.272875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1fc87b1d-14e9-4944-be2b-37358c4cd0a9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:58.278918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1fc87b1d-14e9-4944-be2b-37358c4cd0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:58.331615 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3115/6221] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:58.335369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fec432-3ef4-4ed9-8a02-7f86da642952 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:58.351680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fec432-3ef4-4ed9-8a02-7f86da642952 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:58.381381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d4114-2e48-4c95-81b2-d6e449676afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:58.382906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d4114-2e48-4c95-81b2-d6e449676afa admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:58.383193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d4114-2e48-4c95-81b2-d6e449676afa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:58.385891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d4114-2e48-4c95-81b2-d6e449676afa admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:58.388297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d4114-2e48-4c95-81b2-d6e449676afa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:14:58.388767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d4114-2e48-4c95-81b2-d6e449676afa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:58.390757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d4114-2e48-4c95-81b2-d6e449676afa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:58.401475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:58.434683 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3107/6222] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:14:58.436869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc87b1d-14e9-4944-be2b-37358c4cd0a9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:58.471249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc87b1d-14e9-4944-be2b-37358c4cd0a9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:58.471249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2b793-783f-4224-8908-22ce1263cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:58.471249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2b793-783f-4224-8908-22ce1263cd2f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:58.471249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2b793-783f-4224-8908-22ce1263cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:58.471590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2b793-783f-4224-8908-22ce1263cd2f admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:58.472259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2b793-783f-4224-8908-22ce1263cd2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:58.480321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2b793-783f-4224-8908-22ce1263cd2f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:58.563838 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3116/6223] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:14:58 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:58.564557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d4114-2e48-4c95-81b2-d6e449676afa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:58.591564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d4114-2e48-4c95-81b2-d6e449676afa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:58.749210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d597340-65b6-406d-99f3-b60ce89de146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:58.750282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d597340-65b6-406d-99f3-b60ce89de146 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:58.750758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d597340-65b6-406d-99f3-b60ce89de146 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:58.751415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d597340-65b6-406d-99f3-b60ce89de146 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:58.752247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d597340-65b6-406d-99f3-b60ce89de146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:58.756282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d597340-65b6-406d-99f3-b60ce89de146 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:58.798659 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3108/6224] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:14:58 2022] POST /identity/v3/users => generated 287 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:14:58.803633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2b793-783f-4224-8908-22ce1263cd2f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:58.829149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2b793-783f-4224-8908-22ce1263cd2f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:58.840585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b368-11cd-45c0-9bf6-057553c327c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:58.841642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b368-11cd-45c0-9bf6-057553c327c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:58.842142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b368-11cd-45c0-9bf6-057553c327c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:58.845658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b368-11cd-45c0-9bf6-057553c327c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:58.846544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47b368-11cd-45c0-9bf6-057553c327c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:58.848411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47b368-11cd-45c0-9bf6-057553c327c4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:58.872310 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3117/6225] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:58.873826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d597340-65b6-406d-99f3-b60ce89de146 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:58.887882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d597340-65b6-406d-99f3-b60ce89de146 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:58.920487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f4047-74fb-4b51-8035-acc8dc487ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:58.921151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f4047-74fb-4b51-8035-acc8dc487ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:58.921433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f4047-74fb-4b51-8035-acc8dc487ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:58.921707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f4047-74fb-4b51-8035-acc8dc487ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:58.925635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f4047-74fb-4b51-8035-acc8dc487ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:58.933744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f4047-74fb-4b51-8035-acc8dc487ed6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:58.971838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b368-11cd-45c0-9bf6-057553c327c4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:58.984398 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3109/6226] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:14:58.989395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b368-11cd-45c0-9bf6-057553c327c4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:58.999399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b1110-f9c5-4232-a6dc-17382b57ae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:59.000083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b1110-f9c5-4232-a6dc-17382b57ae98 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:59.000363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b1110-f9c5-4232-a6dc-17382b57ae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:59.000636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b1110-f9c5-4232-a6dc-17382b57ae98 admin admin] PATH_INFO: `/v3/projects/b0007ee63f7340f68f40f6caf5054654/users/1d86728459fe4fefabd2555de8608ddc/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:59.035623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b1110-f9c5-4232-a6dc-17382b57ae98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0007ee63f7340f68f40f6caf5054654, user_id=1d86728459fe4fefabd2555de8608ddc, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:59.040325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b1110-f9c5-4232-a6dc-17382b57ae98 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:59.070000 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3118/6227] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:14:58 2022] PUT /identity/v3/projects/b0007ee63f7340f68f40f6caf5054654/users/1d86728459fe4fefabd2555de8608ddc/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:14:59.076614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6e2f5-0f60-4fe5-8a59-6a03b5c94f94 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:59.076942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6e2f5-0f60-4fe5-8a59-6a03b5c94f94 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:59.077278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6e2f5-0f60-4fe5-8a59-6a03b5c94f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:59.117996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2ef6e2f5-0f60-4fe5-8a59-6a03b5c94f94 None None] MFA Rules not processed for user `1d86728459fe4fefabd2555de8608ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:59.119243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2ef6e2f5-0f60-4fe5-8a59-6a03b5c94f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:59.176137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa97ad-c1a9-482e-a576-0d95ad94038f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:59.176324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa97ad-c1a9-482e-a576-0d95ad94038f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:59.176591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa97ad-c1a9-482e-a576-0d95ad94038f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:59.286660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-43fa97ad-c1a9-482e-a576-0d95ad94038f None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:14:59.288588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-43fa97ad-c1a9-482e-a576-0d95ad94038f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:59.318981 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3110/6228] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:14:59.326415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6e2f5-0f60-4fe5-8a59-6a03b5c94f94 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:59.330562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2ef6e2f5-0f60-4fe5-8a59-6a03b5c94f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:59.336463 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3119/6229] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:14:59 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 21:14:59.340766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa97ad-c1a9-482e-a576-0d95ad94038f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:14:59.348819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-43fa97ad-c1a9-482e-a576-0d95ad94038f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:14:59.520763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6e2f5-0f60-4fe5-8a59-6a03b5c94f94 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:59.529872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa97ad-c1a9-482e-a576-0d95ad94038f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:14:59.543060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854fe1b-5fed-4cab-8e00-6be8565bf0bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:59.544918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854fe1b-5fed-4cab-8e00-6be8565bf0bc service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:59.546466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854fe1b-5fed-4cab-8e00-6be8565bf0bc service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:59.546754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854fe1b-5fed-4cab-8e00-6be8565bf0bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:59.553332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc0a43-9d8d-4033-bd4a-0e24747ecf1d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:14:59.554583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc0a43-9d8d-4033-bd4a-0e24747ecf1d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:14:59.555077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc0a43-9d8d-4033-bd4a-0e24747ecf1d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:14:59.558270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc0a43-9d8d-4033-bd4a-0e24747ecf1d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:14:59.574714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854fe1b-5fed-4cab-8e00-6be8565bf0bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:14:59.579261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854fe1b-5fed-4cab-8e00-6be8565bf0bc service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:14:59.608908 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3111/6230] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:14:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:14:59.797932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a6fc0a43-9d8d-4033-bd4a-0e24747ecf1d tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:14:59.825111 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3120/6231] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:14:59 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:00.828762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cd3f5-e636-4152-919f-390584845497 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:00.829223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cd3f5-e636-4152-919f-390584845497 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:00.829347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cd3f5-e636-4152-919f-390584845497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:00.849419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:00.868229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-159cd3f5-e636-4152-919f-390584845497 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:00.869513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-159cd3f5-e636-4152-919f-390584845497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:00.886901 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3112/6232] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:00 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:15:00.888441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea719195-680f-4d92-ab30-891081f65341 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:00.888869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea719195-680f-4d92-ab30-891081f65341 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:00.888977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea719195-680f-4d92-ab30-891081f65341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-ea719195-680f-4d92-ab30-891081f65341 None None] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 15 21:15:00.941109 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 185, in authenticate_for_token Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application authenticate(auth_info, auth_context) Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 152, in authenticate Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = method.authenticate(auth_info.get_method_data(method_name)) Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 42, in authenticate Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application token = self._get_token_ref(auth_payload) Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 36, in _get_token_ref Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return PROVIDERS.token_provider_api.validate_token(token_id) Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application self.check_revocation(token) Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:15:00.941932 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:15:00.946022 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3121/6233] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:00 2022] POST /identity/v3/auth/tokens => generated 80 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 21:15:00.947045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40207e-3d96-404d-9a39-172c4cd35050 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:00.947366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40207e-3d96-404d-9a39-172c4cd35050 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:00.947691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40207e-3d96-404d-9a39-172c4cd35050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:01.042146 ncdv-master-1 devstack@keystone.service[99099]: WARNING keystone.server.flask.application [None req-7e40207e-3d96-404d-9a39-172c4cd35050 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 15 21:15:01.050002 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3113/6234] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:00 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 104 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 15 21:15:01.051944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b668c5b-5e60-4145-9a23-5082cdd268cb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:01.052414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b668c5b-5e60-4145-9a23-5082cdd268cb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:01.052947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b668c5b-5e60-4145-9a23-5082cdd268cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:01.117537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0b668c5b-5e60-4145-9a23-5082cdd268cb None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:01.118819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0b668c5b-5e60-4145-9a23-5082cdd268cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:01.131865 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3122/6235] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:01.136117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40207e-3d96-404d-9a39-172c4cd35050 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:01.153165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40207e-3d96-404d-9a39-172c4cd35050 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:01.165778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ddfe7-ff97-4a9c-a605-545e37ce73cf tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:01.166395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ddfe7-ff97-4a9c-a605-545e37ce73cf tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:01.167327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ddfe7-ff97-4a9c-a605-545e37ce73cf tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:01.167897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ddfe7-ff97-4a9c-a605-545e37ce73cf tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:01.274736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a84ddfe7-ff97-4a9c-a605-545e37ce73cf tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:01.295254 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3114/6236] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:15:01 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:01.470357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccd8a8-9bf8-4b70-890c-2e89e2417e36 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:01.470947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccd8a8-9bf8-4b70-890c-2e89e2417e36 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:01.472145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccd8a8-9bf8-4b70-890c-2e89e2417e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:01.497351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-25ccd8a8-9bf8-4b70-890c-2e89e2417e36 None None] MFA Rules not processed for user `1d86728459fe4fefabd2555de8608ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:01.499477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-25ccd8a8-9bf8-4b70-890c-2e89e2417e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:01.512971 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3123/6237] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:02.296425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d76fb1-3527-4fb8-a72b-8fe05e9578a4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:02.297018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d76fb1-3527-4fb8-a72b-8fe05e9578a4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:02.297486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d76fb1-3527-4fb8-a72b-8fe05e9578a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:02.348046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-43d76fb1-3527-4fb8-a72b-8fe05e9578a4 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:02.353523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-43d76fb1-3527-4fb8-a72b-8fe05e9578a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:02.372851 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3115/6238] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:02 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:15:02.377356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c61b9-77b2-4771-9026-d85cfaf4be54 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:02.377824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c61b9-77b2-4771-9026-d85cfaf4be54 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:02.377915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c61b9-77b2-4771-9026-d85cfaf4be54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:02.414370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-af0c61b9-77b2-4771-9026-d85cfaf4be54 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:02.419089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af0c61b9-77b2-4771-9026-d85cfaf4be54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:02.436801 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3124/6239] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:02.441123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d76fb1-3527-4fb8-a72b-8fe05e9578a4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:02.461406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d76fb1-3527-4fb8-a72b-8fe05e9578a4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:02.478160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37f8b5-b5c1-4bae-9331-71da14358552 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:02.479444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37f8b5-b5c1-4bae-9331-71da14358552 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:02.480132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37f8b5-b5c1-4bae-9331-71da14358552 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:02.480567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37f8b5-b5c1-4bae-9331-71da14358552 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:02.569924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ac37f8b5-b5c1-4bae-9331-71da14358552 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:02.601180 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3116/6240] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:15:02 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:03.607401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecfad2-8012-4dc9-9042-d211fe104b9d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:03.607949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecfad2-8012-4dc9-9042-d211fe104b9d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:03.607949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecfad2-8012-4dc9-9042-d211fe104b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:03.669113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b4ecfad2-8012-4dc9-9042-d211fe104b9d None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:03.676196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b4ecfad2-8012-4dc9-9042-d211fe104b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:03.683926 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3125/6241] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:03 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:15:03.688823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aed4e4-eda6-401c-b704-3ffc8c0030c7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:03.690531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aed4e4-eda6-401c-b704-3ffc8c0030c7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:03.691453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aed4e4-eda6-401c-b704-3ffc8c0030c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:03.779756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-60aed4e4-eda6-401c-b704-3ffc8c0030c7 None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:03.788269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-60aed4e4-eda6-401c-b704-3ffc8c0030c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:03.809216 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3117/6242] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:03.811630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecfad2-8012-4dc9-9042-d211fe104b9d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:03.819776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=3,keystone:SELECT=90,keystone:INSERT=4,keystone:UPDATE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:03.832680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecfad2-8012-4dc9-9042-d211fe104b9d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:03.847721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b1455-ef38-4df1-85c1-b0bbcfbc1d52 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b914f687682d4a90856bdf4ac7cda45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:03.848699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b1455-ef38-4df1-85c1-b0bbcfbc1d52 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:03.848978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b1455-ef38-4df1-85c1-b0bbcfbc1d52 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:03.849252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b1455-ef38-4df1-85c1-b0bbcfbc1d52 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f/password` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:03.971021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-be6b1455-ef38-4df1-85c1-b0bbcfbc1d52 tempest-IdentityV3UsersTest-670297865 tempest-IdentityV3UsersTest-670297865-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:03.985593 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3126/6243] 10.222.0.16 () {60 vars in 1269 bytes} [Sat Oct 15 21:15:03 2022] POST /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f/password => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:05.013462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e06a29-dc58-4321-bed3-3024d22a52df None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:05.016460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e06a29-dc58-4321-bed3-3024d22a52df None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:05.017317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e06a29-dc58-4321-bed3-3024d22a52df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:05.047236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-66e06a29-dc58-4321-bed3-3024d22a52df None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:05.048507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-66e06a29-dc58-4321-bed3-3024d22a52df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:05.053949 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3118/6244] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:05 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 21:15:06.056818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10062e6e-525c-4d6f-b85b-f1f00e856a0a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:06.057453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10062e6e-525c-4d6f-b85b-f1f00e856a0a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:06.057580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10062e6e-525c-4d6f-b85b-f1f00e856a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:06.108592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-10062e6e-525c-4d6f-b85b-f1f00e856a0a None None] MFA Rules not processed for user `b914f687682d4a90856bdf4ac7cda45f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:06.110761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-10062e6e-525c-4d6f-b85b-f1f00e856a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:06.131031 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3127/6245] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:06.131448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e06a29-dc58-4321-bed3-3024d22a52df None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:06.132940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-66e06a29-dc58-4321-bed3-3024d22a52df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:06.314307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e06a29-dc58-4321-bed3-3024d22a52df None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:06.328616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:06.329324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:06.329607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:06.329877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] PATH_INFO: `/v3/users/b914f687682d4a90856bdf4ac7cda45f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:06.337926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b914f687682d4a90856bdf4ac7cda45f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:06.340740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:06.447998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:06.456067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=82,keystone:UPDATE=3,keystone:INSERT=4,keystone:DELETE=11 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:06.460507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:06.493635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:06.525268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:06.528550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b914f687682d4a90856bdf4ac7cda45f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:06.554044 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3119/6246] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:06 2022] DELETE /identity/v3/users/b914f687682d4a90856bdf4ac7cda45f => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:06.559387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6043822-ef9a-4130-84d9-31575b6353ef None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:06.559766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6043822-ef9a-4130-84d9-31575b6353ef None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:06.560042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6043822-ef9a-4130-84d9-31575b6353ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:06.600009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f6043822-ef9a-4130-84d9-31575b6353ef None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:06.601340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f6043822-ef9a-4130-84d9-31575b6353ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:06.687845 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3128/6247] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:06.751478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:06.753305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:06.870258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab49d0d-0c10-4b84-9465-a736b7fa2858 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:06.879880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a118a-4d06-43ba-a3b6-eaf42e2489bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:06.881864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a118a-4d06-43ba-a3b6-eaf42e2489bf service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:06.882709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a118a-4d06-43ba-a3b6-eaf42e2489bf service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:06.884249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a118a-4d06-43ba-a3b6-eaf42e2489bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:06.918574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a118a-4d06-43ba-a3b6-eaf42e2489bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:06.921438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a118a-4d06-43ba-a3b6-eaf42e2489bf service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:06.939974 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3120/6248] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:08.807506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6043822-ef9a-4130-84d9-31575b6353ef None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:08.809566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f6043822-ef9a-4130-84d9-31575b6353ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:08.829452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6043822-ef9a-4130-84d9-31575b6353ef None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:08.843219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:08.844159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:08.844652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:08.845740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] PATH_INFO: `/v3/projects/7f3dc26b6a3e4b1dba49b9e6dc395834` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:08.847898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f3dc26b6a3e4b1dba49b9e6dc395834)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:08.850131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:09.004574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'request_id': 'req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:09.032700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'request_id': 'req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:09.058304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f3dc26b6a3e4b1dba49b9e6dc395834', 'request_id': 'req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:09.105554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] The token cache is being invalidate because project 7f3dc26b6a3e4b1dba49b9e6dc395834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:09.106308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:09.107436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:09.111181 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3129/6249] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:08 2022] DELETE /identity/v3/projects/7f3dc26b6a3e4b1dba49b9e6dc395834 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:09.117400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc43c6-829b-481a-b766-291c8ab418a6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:09.118205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc43c6-829b-481a-b766-291c8ab418a6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:09.118616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc43c6-829b-481a-b766-291c8ab418a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:09.159044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f5cc43c6-829b-481a-b766-291c8ab418a6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:09.160266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f5cc43c6-829b-481a-b766-291c8ab418a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:09.233681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:09.236530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:09.319639 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3121/6250] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:09.328122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2e40d-9438-4306-8e6c-32c2dad83111 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:09.328397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2e40d-9438-4306-8e6c-32c2dad83111 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:09.328660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2e40d-9438-4306-8e6c-32c2dad83111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:09.350655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-40a2e40d-9438-4306-8e6c-32c2dad83111 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:09.352112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-40a2e40d-9438-4306-8e6c-32c2dad83111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:09.361452 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3122/6251] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:09.365234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:INSERT=3,keystone:DELETE=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:09.367115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2e40d-9438-4306-8e6c-32c2dad83111 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:09.381393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2e40d-9438-4306-8e6c-32c2dad83111 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:09.394529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54deab6-1125-4202-a907-5bc4aa6a26c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:09.395326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54deab6-1125-4202-a907-5bc4aa6a26c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:09.395777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54deab6-1125-4202-a907-5bc4aa6a26c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:09.396453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54deab6-1125-4202-a907-5bc4aa6a26c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:09.397306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54deab6-1125-4202-a907-5bc4aa6a26c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:09.397930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54deab6-1125-4202-a907-5bc4aa6a26c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:09.401169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54deab6-1125-4202-a907-5bc4aa6a26c7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:09.403867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64659c-f121-44a2-9f8c-ed7839d7ee1e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:09.420066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:09.420066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:09.420066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:09.420432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] PATH_INFO: `/v3/users/84218be29d4445c5ad36cb4dfda1ed5e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:09.421180 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3123/6252] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:09.421918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] RBAC: Authorizing `identity:delete_user(user_id=84218be29d4445c5ad36cb4dfda1ed5e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:09.422472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54deab6-1125-4202-a907-5bc4aa6a26c7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:09.425963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:09.442365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54deab6-1125-4202-a907-5bc4aa6a26c7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:09.455412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cee7db-5d1e-454e-a79e-806e12a71bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:09.457618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cee7db-5d1e-454e-a79e-806e12a71bbc admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:09.458896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cee7db-5d1e-454e-a79e-806e12a71bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:09.459178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cee7db-5d1e-454e-a79e-806e12a71bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:09.459828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cee7db-5d1e-454e-a79e-806e12a71bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:09.463976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cee7db-5d1e-454e-a79e-806e12a71bbc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:09.529864 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3124/6253] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:09 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:09.531007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cee7db-5d1e-454e-a79e-806e12a71bbc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:09.572770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84218be29d4445c5ad36cb4dfda1ed5e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:09.600088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:09.613057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84218be29d4445c5ad36cb4dfda1ed5e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:09.626582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84218be29d4445c5ad36cb4dfda1ed5e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:09.646791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84218be29d4445c5ad36cb4dfda1ed5e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:09.650441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2d213b4-44da-443c-8b9f-3c84caf7598d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84218be29d4445c5ad36cb4dfda1ed5e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:09.656697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cee7db-5d1e-454e-a79e-806e12a71bbc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:09.668737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad68d4f-6ff2-47fd-a25e-bcb6e1f67018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:09.669590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad68d4f-6ff2-47fd-a25e-bcb6e1f67018 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:09.669974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad68d4f-6ff2-47fd-a25e-bcb6e1f67018 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:09.670409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad68d4f-6ff2-47fd-a25e-bcb6e1f67018 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:09.671165 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3130/6254] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:09 2022] DELETE /identity/v3/users/84218be29d4445c5ad36cb4dfda1ed5e => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:09.674993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad68d4f-6ff2-47fd-a25e-bcb6e1f67018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:09.678665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6563a7a-2315-48d5-99d2-30f2a8c0591b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:09.678923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad68d4f-6ff2-47fd-a25e-bcb6e1f67018 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:09.679089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6563a7a-2315-48d5-99d2-30f2a8c0591b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:09.679245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6563a7a-2315-48d5-99d2-30f2a8c0591b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:09.722726 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3125/6255] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:09 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:09.727255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad68d4f-6ff2-47fd-a25e-bcb6e1f67018 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:09.740678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5ad68d4f-6ff2-47fd-a25e-bcb6e1f67018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:09.762776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad68d4f-6ff2-47fd-a25e-bcb6e1f67018 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:09.766335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c6563a7a-2315-48d5-99d2-30f2a8c0591b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:09.767559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c6563a7a-2315-48d5-99d2-30f2a8c0591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:09.888901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a021e62-08a6-42c1-810b-b584be7ada95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:09.890163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a021e62-08a6-42c1-810b-b584be7ada95 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:09.890625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a021e62-08a6-42c1-810b-b584be7ada95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:09.891004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a021e62-08a6-42c1-810b-b584be7ada95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:09.891755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a021e62-08a6-42c1-810b-b584be7ada95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:09.893949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a021e62-08a6-42c1-810b-b584be7ada95 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:09.940054 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3131/6256] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:09.950877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6563a7a-2315-48d5-99d2-30f2a8c0591b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:09.952293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c6563a7a-2315-48d5-99d2-30f2a8c0591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:10.004960 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3126/6257] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:10.006818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a021e62-08a6-42c1-810b-b584be7ada95 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:10.040594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a021e62-08a6-42c1-810b-b584be7ada95 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:10.057592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44655401-246d-4343-a962-afdf353cdc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:10.058701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44655401-246d-4343-a962-afdf353cdc7e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:10.060292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44655401-246d-4343-a962-afdf353cdc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:10.061375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44655401-246d-4343-a962-afdf353cdc7e admin admin] PATH_INFO: `/v3/projects/5ae7ff3c71ce49c4a8378a177cc1e09f/users/30837e55802f4e3ea7393505c169eb7f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:10.109377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44655401-246d-4343-a962-afdf353cdc7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae7ff3c71ce49c4a8378a177cc1e09f, user_id=30837e55802f4e3ea7393505c169eb7f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:10.113703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44655401-246d-4343-a962-afdf353cdc7e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:10.137282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6563a7a-2315-48d5-99d2-30f2a8c0591b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:10.148754 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3127/6258] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:10 2022] PUT /identity/v3/projects/5ae7ff3c71ce49c4a8378a177cc1e09f/users/30837e55802f4e3ea7393505c169eb7f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:10.152823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44655401-246d-4343-a962-afdf353cdc7e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:10.179483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:10.291224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169289-eb77-43ca-8833-82cce1446993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:10.292826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169289-eb77-43ca-8833-82cce1446993 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:10.293279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169289-eb77-43ca-8833-82cce1446993 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:10.294545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169289-eb77-43ca-8833-82cce1446993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:10.328190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44655401-246d-4343-a962-afdf353cdc7e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:10.333637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71be187-2b64-43cf-ace4-f3ef74da243f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:10.334799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71be187-2b64-43cf-ace4-f3ef74da243f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:10.335074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71be187-2b64-43cf-ace4-f3ef74da243f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:10.335340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71be187-2b64-43cf-ace4-f3ef74da243f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:10.336484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71be187-2b64-43cf-ace4-f3ef74da243f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:10.339302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71be187-2b64-43cf-ace4-f3ef74da243f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:10.358260 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3128/6259] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:15:10.365496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71be187-2b64-43cf-ace4-f3ef74da243f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:10.379290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71be187-2b64-43cf-ace4-f3ef74da243f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:10.386093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b78672-0bbd-4650-aec0-55d0e50b5ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:10.386900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b78672-0bbd-4650-aec0-55d0e50b5ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:10.387177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b78672-0bbd-4650-aec0-55d0e50b5ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:10.387446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b78672-0bbd-4650-aec0-55d0e50b5ac1 admin admin] PATH_INFO: `/v3/projects/5ae7ff3c71ce49c4a8378a177cc1e09f/users/30837e55802f4e3ea7393505c169eb7f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:10.389600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b78672-0bbd-4650-aec0-55d0e50b5ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae7ff3c71ce49c4a8378a177cc1e09f, user_id=30837e55802f4e3ea7393505c169eb7f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:10.393505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b78672-0bbd-4650-aec0-55d0e50b5ac1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:10.393836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc169289-eb77-43ca-8833-82cce1446993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:10.396699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc169289-eb77-43ca-8833-82cce1446993 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:10.409072 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3129/6260] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:10 2022] PUT /identity/v3/projects/5ae7ff3c71ce49c4a8378a177cc1e09f/users/30837e55802f4e3ea7393505c169eb7f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:10.411814 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3132/6261] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:15:10.427174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f53d4-21d2-491a-be10-dcdd8143f416 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:10.427673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f53d4-21d2-491a-be10-dcdd8143f416 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:10.428733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f53d4-21d2-491a-be10-dcdd8143f416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:10.474833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-598f53d4-21d2-491a-be10-dcdd8143f416 None None] MFA Rules not processed for user `30837e55802f4e3ea7393505c169eb7f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:10.481390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-598f53d4-21d2-491a-be10-dcdd8143f416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:10.489939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:10.647110 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3130/6262] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:10.651600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169289-eb77-43ca-8833-82cce1446993 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:10.680291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169289-eb77-43ca-8833-82cce1446993 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:10.804190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f53d4-21d2-491a-be10-dcdd8143f416 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:10.806341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-598f53d4-21d2-491a-be10-dcdd8143f416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:10.863261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d03c1f-d63e-49ba-a12a-313a41468ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:10.863762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d03c1f-d63e-49ba-a12a-313a41468ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:10.864046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d03c1f-d63e-49ba-a12a-313a41468ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:10.864318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d03c1f-d63e-49ba-a12a-313a41468ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:10.865200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d03c1f-d63e-49ba-a12a-313a41468ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:10.867571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d03c1f-d63e-49ba-a12a-313a41468ff5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:10.938777 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3133/6263] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:10 2022] POST /identity/v3/projects => generated 383 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:10.943289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d03c1f-d63e-49ba-a12a-313a41468ff5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:10.988159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f53d4-21d2-491a-be10-dcdd8143f416 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.083042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:11.088815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d03c1f-d63e-49ba-a12a-313a41468ff5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.088966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.089901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.090197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.090458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] PATH_INFO: `/v3/projects/25e7b1e90c0b40d7960b578605ea987c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.091630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e7b1e90c0b40d7960b578605ea987c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.094386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.098820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff006b-6068-41e6-bd15-b4e340e3412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.099710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff006b-6068-41e6-bd15-b4e340e3412f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.100272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff006b-6068-41e6-bd15-b4e340e3412f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.100880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff006b-6068-41e6-bd15-b4e340e3412f admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.102148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff006b-6068-41e6-bd15-b4e340e3412f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.105122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff006b-6068-41e6-bd15-b4e340e3412f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.135533 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3134/6264] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:10 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:11.139172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff006b-6068-41e6-bd15-b4e340e3412f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:11.157066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff006b-6068-41e6-bd15-b4e340e3412f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.167573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e7b1e90c0b40d7960b578605ea987c', 'request_id': 'req-02026050-0927-4620-99aa-0ba8ff28367a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:11.169510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:11.182525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e7b1e90c0b40d7960b578605ea987c', 'request_id': 'req-02026050-0927-4620-99aa-0ba8ff28367a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:11.192674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e7b1e90c0b40d7960b578605ea987c', 'request_id': 'req-02026050-0927-4620-99aa-0ba8ff28367a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:11.206676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] The token cache is being invalidate because project 25e7b1e90c0b40d7960b578605ea987c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:11.207010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:11.207819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02026050-0927-4620-99aa-0ba8ff28367a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:11.209774 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3131/6265] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:10 2022] DELETE /identity/v3/projects/25e7b1e90c0b40d7960b578605ea987c => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:11.215162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18fd9d-a621-4238-a0dc-53a7e9b9b705 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.215162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18fd9d-a621-4238-a0dc-53a7e9b9b705 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.215435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18fd9d-a621-4238-a0dc-53a7e9b9b705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.241600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3f18fd9d-a621-4238-a0dc-53a7e9b9b705 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:11.243350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3f18fd9d-a621-4238-a0dc-53a7e9b9b705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:11.251889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e81f9-f174-40e4-8799-58ef63d94f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.252602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e81f9-f174-40e4-8799-58ef63d94f7c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.252950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e81f9-f174-40e4-8799-58ef63d94f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.254196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e81f9-f174-40e4-8799-58ef63d94f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.254919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e81f9-f174-40e4-8799-58ef63d94f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.257152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e81f9-f174-40e4-8799-58ef63d94f7c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.270130 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3135/6266] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:15:11.273810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e81f9-f174-40e4-8799-58ef63d94f7c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:11.275660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b82e81f9-f174-40e4-8799-58ef63d94f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:11.293332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e81f9-f174-40e4-8799-58ef63d94f7c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.298646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e8439-6489-4c93-8b25-8805fa7fd95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.299260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e8439-6489-4c93-8b25-8805fa7fd95a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.299534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e8439-6489-4c93-8b25-8805fa7fd95a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.300049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e8439-6489-4c93-8b25-8805fa7fd95a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.300429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e8439-6489-4c93-8b25-8805fa7fd95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.302418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e8439-6489-4c93-8b25-8805fa7fd95a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.315843 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3136/6267] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:15:11.319656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e8439-6489-4c93-8b25-8805fa7fd95a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:11.324124 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3132/6268] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:11.333220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd760bd5-f956-4014-b004-a79a3d0644b6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.333509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd760bd5-f956-4014-b004-a79a3d0644b6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.333768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd760bd5-f956-4014-b004-a79a3d0644b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.334161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e8439-6489-4c93-8b25-8805fa7fd95a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.340310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e077a4-644d-44fe-93f7-95579d06e8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.341109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e077a4-644d-44fe-93f7-95579d06e8e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.341462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e077a4-644d-44fe-93f7-95579d06e8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.341809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e077a4-644d-44fe-93f7-95579d06e8e1 admin admin] PATH_INFO: `/v3/projects/9e87a8e1c985498697dc2b129d97b67a/users/0c73bc0dc4a74ced86290054b688bb87/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.355281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cd760bd5-f956-4014-b004-a79a3d0644b6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:11.356538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cd760bd5-f956-4014-b004-a79a3d0644b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:11.362514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e077a4-644d-44fe-93f7-95579d06e8e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e87a8e1c985498697dc2b129d97b67a, user_id=0c73bc0dc4a74ced86290054b688bb87, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.365430 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3133/6269] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:11.365572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e077a4-644d-44fe-93f7-95579d06e8e1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.369710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd760bd5-f956-4014-b004-a79a3d0644b6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:11.381664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd760bd5-f956-4014-b004-a79a3d0644b6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.385341 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3137/6270] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:11 2022] PUT /identity/v3/projects/9e87a8e1c985498697dc2b129d97b67a/users/0c73bc0dc4a74ced86290054b688bb87/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:11.398276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53018-ba34-4ac6-813f-fd5c0da90bb7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.398593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53018-ba34-4ac6-813f-fd5c0da90bb7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.398823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53018-ba34-4ac6-813f-fd5c0da90bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.433560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b0c53018-ba34-4ac6-813f-fd5c0da90bb7 None None] MFA Rules not processed for user `0c73bc0dc4a74ced86290054b688bb87`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:11.434885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b0c53018-ba34-4ac6-813f-fd5c0da90bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:11.454724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf23262-b2c9-45a3-be83-80ccc5171ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.455550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf23262-b2c9-45a3-be83-80ccc5171ace admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.455876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf23262-b2c9-45a3-be83-80ccc5171ace admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.456127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf23262-b2c9-45a3-be83-80ccc5171ace admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.456783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf23262-b2c9-45a3-be83-80ccc5171ace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:11.457128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf23262-b2c9-45a3-be83-80ccc5171ace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.459479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf23262-b2c9-45a3-be83-80ccc5171ace admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.480046 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3134/6271] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:15:11.484286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf23262-b2c9-45a3-be83-80ccc5171ace admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:11.496676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf23262-b2c9-45a3-be83-80ccc5171ace admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.501752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb8e65-b567-4c8d-91be-354ae969b919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.502370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb8e65-b567-4c8d-91be-354ae969b919 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.502660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb8e65-b567-4c8d-91be-354ae969b919 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.502913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb8e65-b567-4c8d-91be-354ae969b919 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.503544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb8e65-b567-4c8d-91be-354ae969b919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.505518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb8e65-b567-4c8d-91be-354ae969b919 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.514558 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3138/6272] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:11.526688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c53018-ba34-4ac6-813f-fd5c0da90bb7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:11.528838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b0c53018-ba34-4ac6-813f-fd5c0da90bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:11.541318 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3135/6273] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:11 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:15:11.542681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb8e65-b567-4c8d-91be-354ae969b919 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:11.582964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:11.600125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:11.607494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c53018-ba34-4ac6-813f-fd5c0da90bb7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.669916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb8e65-b567-4c8d-91be-354ae969b919 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.687834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d6a9a-e1b1-4ef7-997d-6361a3b3d6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.690309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d6a9a-e1b1-4ef7-997d-6361a3b3d6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.691213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d6a9a-e1b1-4ef7-997d-6361a3b3d6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.692302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d6a9a-e1b1-4ef7-997d-6361a3b3d6e8 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.693714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d6a9a-e1b1-4ef7-997d-6361a3b3d6e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.694715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806fa38-1d40-4c8d-9f3e-6801a9716115 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.696865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806fa38-1d40-4c8d-9f3e-6801a9716115 service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.696865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806fa38-1d40-4c8d-9f3e-6801a9716115 service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.696865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806fa38-1d40-4c8d-9f3e-6801a9716115 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.699240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d6a9a-e1b1-4ef7-997d-6361a3b3d6e8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.702948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d806fa38-1d40-4c8d-9f3e-6801a9716115 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:11.757091 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3136/6274] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:11 2022] POST /identity/v3/users => generated 291 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:11.761474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d6a9a-e1b1-4ef7-997d-6361a3b3d6e8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:11.784552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d6a9a-e1b1-4ef7-997d-6361a3b3d6e8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.794805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763d771-8da7-4fd8-bd6f-be6af6cd79e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.795456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763d771-8da7-4fd8-bd6f-be6af6cd79e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.795832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763d771-8da7-4fd8-bd6f-be6af6cd79e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.796215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763d771-8da7-4fd8-bd6f-be6af6cd79e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.797593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763d771-8da7-4fd8-bd6f-be6af6cd79e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.803142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763d771-8da7-4fd8-bd6f-be6af6cd79e8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.812117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806fa38-1d40-4c8d-9f3e-6801a9716115 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.813878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806fa38-1d40-4c8d-9f3e-6801a9716115 service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.837238 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3139/6275] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:11.873158 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3137/6276] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:11.874418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806fa38-1d40-4c8d-9f3e-6801a9716115 service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:11.891048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806fa38-1d40-4c8d-9f3e-6801a9716115 service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.908053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43b7e0-6188-4b52-8f2a-f16478aa934c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.910979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43b7e0-6188-4b52-8f2a-f16478aa934c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.911704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43b7e0-6188-4b52-8f2a-f16478aa934c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.912460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43b7e0-6188-4b52-8f2a-f16478aa934c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:11.917095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43b7e0-6188-4b52-8f2a-f16478aa934c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:11.922813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43b7e0-6188-4b52-8f2a-f16478aa934c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:11.954699 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3140/6277] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:11.957198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763d771-8da7-4fd8-bd6f-be6af6cd79e8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:11.973131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763d771-8da7-4fd8-bd6f-be6af6cd79e8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:11.982934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0d533-9c70-41c6-a7df-7e3dabb12e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:11.983779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0d533-9c70-41c6-a7df-7e3dabb12e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:11.984080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0d533-9c70-41c6-a7df-7e3dabb12e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:11.984338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0d533-9c70-41c6-a7df-7e3dabb12e81 admin admin] PATH_INFO: `/v3/projects/baadbf9da41041719cc9f50d8b5ef52b/users/fa6c55b6a8f14b2fb6f1df38f9f8cd39/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:12.005611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0d533-9c70-41c6-a7df-7e3dabb12e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baadbf9da41041719cc9f50d8b5ef52b, user_id=fa6c55b6a8f14b2fb6f1df38f9f8cd39, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:12.009275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0d533-9c70-41c6-a7df-7e3dabb12e81 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:12.041424 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3138/6278] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:11 2022] PUT /identity/v3/projects/baadbf9da41041719cc9f50d8b5ef52b/users/fa6c55b6a8f14b2fb6f1df38f9f8cd39/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:12.050169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be62f8-572e-495d-a95e-3bf90f4dfca7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:12.050585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be62f8-572e-495d-a95e-3bf90f4dfca7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:12.050941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be62f8-572e-495d-a95e-3bf90f4dfca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:12.086982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-41be62f8-572e-495d-a95e-3bf90f4dfca7 None None] MFA Rules not processed for user `fa6c55b6a8f14b2fb6f1df38f9f8cd39`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:12.088327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-41be62f8-572e-495d-a95e-3bf90f4dfca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:12.197606 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3141/6279] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:12.201223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0d533-9c70-41c6-a7df-7e3dabb12e81 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:12.301816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0d533-9c70-41c6-a7df-7e3dabb12e81 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:12.310145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3c8b0-c56c-42b8-860b-81695dec69d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:12.310760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3c8b0-c56c-42b8-860b-81695dec69d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:12.311038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3c8b0-c56c-42b8-860b-81695dec69d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:12.311308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3c8b0-c56c-42b8-860b-81695dec69d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:12.311951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3c8b0-c56c-42b8-860b-81695dec69d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:12.313881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3c8b0-c56c-42b8-860b-81695dec69d1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:12.357092 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3139/6280] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:12 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:12.359506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be62f8-572e-495d-a95e-3bf90f4dfca7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:12.375000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be62f8-572e-495d-a95e-3bf90f4dfca7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:12.400341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:12.505776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19409369-308d-468c-a599-ef4da9ef27d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:12.506366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19409369-308d-468c-a599-ef4da9ef27d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:12.506638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19409369-308d-468c-a599-ef4da9ef27d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:12.506968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19409369-308d-468c-a599-ef4da9ef27d6 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:12.507528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19409369-308d-468c-a599-ef4da9ef27d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:12.510135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19409369-308d-468c-a599-ef4da9ef27d6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:12.527539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3c8b0-c56c-42b8-860b-81695dec69d1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:12.529794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7ac3c8b0-c56c-42b8-860b-81695dec69d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:12.542410 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3142/6281] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:12 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:12.545205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19409369-308d-468c-a599-ef4da9ef27d6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:12.562444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19409369-308d-468c-a599-ef4da9ef27d6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:12.570917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f10e4-4d10-4d20-bf17-e6c9c6797827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:12.571790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f10e4-4d10-4d20-bf17-e6c9c6797827 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:12.572189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f10e4-4d10-4d20-bf17-e6c9c6797827 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:12.572994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f10e4-4d10-4d20-bf17-e6c9c6797827 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:12.574097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f10e4-4d10-4d20-bf17-e6c9c6797827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:12.576090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f10e4-4d10-4d20-bf17-e6c9c6797827 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:12.590950 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3143/6282] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:12.594796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f10e4-4d10-4d20-bf17-e6c9c6797827 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:12.612713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f10e4-4d10-4d20-bf17-e6c9c6797827 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:12.620560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6298d8c-4817-489f-b0ea-4648f22e50a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:12.623168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6298d8c-4817-489f-b0ea-4648f22e50a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:12.623168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6298d8c-4817-489f-b0ea-4648f22e50a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:12.623168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6298d8c-4817-489f-b0ea-4648f22e50a9 admin admin] PATH_INFO: `/v3/projects/5ed1de1718684e39987868560cb90dd2/users/6fd0d88ede8d4b25ac0b4607dbc16ae0/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:12.656570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6298d8c-4817-489f-b0ea-4648f22e50a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed1de1718684e39987868560cb90dd2, user_id=6fd0d88ede8d4b25ac0b4607dbc16ae0, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:12.661005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6298d8c-4817-489f-b0ea-4648f22e50a9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:12.685565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3c8b0-c56c-42b8-860b-81695dec69d1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:12.701788 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3144/6283] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:12 2022] PUT /identity/v3/projects/5ed1de1718684e39987868560cb90dd2/users/6fd0d88ede8d4b25ac0b4607dbc16ae0/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:12.705216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6298d8c-4817-489f-b0ea-4648f22e50a9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:12.726432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:12.844380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6298d8c-4817-489f-b0ea-4648f22e50a9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:12.846943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90812da0-6764-4970-8bcc-21dad59fd574 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:12.847598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90812da0-6764-4970-8bcc-21dad59fd574 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:12.847827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90812da0-6764-4970-8bcc-21dad59fd574 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:12.848095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90812da0-6764-4970-8bcc-21dad59fd574 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:12.852147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-90812da0-6764-4970-8bcc-21dad59fd574 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:12.854708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb613a-d48d-427c-9663-d7ef2ec5643a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:12.855626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb613a-d48d-427c-9663-d7ef2ec5643a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:12.855906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb613a-d48d-427c-9663-d7ef2ec5643a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:12.856176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb613a-d48d-427c-9663-d7ef2ec5643a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:12.857329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb613a-d48d-427c-9663-d7ef2ec5643a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:12.859788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb613a-d48d-427c-9663-d7ef2ec5643a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:12.866563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90812da0-6764-4970-8bcc-21dad59fd574 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:12.868609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90812da0-6764-4970-8bcc-21dad59fd574 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:12.875094 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3145/6284] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:12.875094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb613a-d48d-427c-9663-d7ef2ec5643a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:12.884013 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3140/6285] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:15:12.885438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90812da0-6764-4970-8bcc-21dad59fd574 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:12.889094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-90812da0-6764-4970-8bcc-21dad59fd574 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:12.893803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb613a-d48d-427c-9663-d7ef2ec5643a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:12.899995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:12.900733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:12.901008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:12.901776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] PATH_INFO: `/v3/users/30837e55802f4e3ea7393505c169eb7f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:12.903139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30837e55802f4e3ea7393505c169eb7f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:12.906459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90812da0-6764-4970-8bcc-21dad59fd574 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:12.906582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:12.912501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:12.913157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:12.913432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:12.913704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] PATH_INFO: `/v3/users/0a453fc9ecb24af2b526bcfc90d3f584` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:12.914858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a453fc9ecb24af2b526bcfc90d3f584)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:12.917355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:12.969634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30837e55802f4e3ea7393505c169eb7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:12.989693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30837e55802f4e3ea7393505c169eb7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:12.992837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:13.005220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30837e55802f4e3ea7393505c169eb7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:13.006989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a453fc9ecb24af2b526bcfc90d3f584'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:13.015192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a453fc9ecb24af2b526bcfc90d3f584'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:13.015425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30837e55802f4e3ea7393505c169eb7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:13.016501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30837e55802f4e3ea7393505c169eb7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:13.025882 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3146/6286] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:12 2022] DELETE /identity/v3/users/30837e55802f4e3ea7393505c169eb7f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:13.027463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:13.030296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a453fc9ecb24af2b526bcfc90d3f584'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:13.030615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:13.038000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a453fc9ecb24af2b526bcfc90d3f584'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:13.038900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a2a1fdd-5861-43b9-921c-b78bb0b72408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a453fc9ecb24af2b526bcfc90d3f584'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:13.064588 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3141/6287] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:12 2022] DELETE /identity/v3/users/0a453fc9ecb24af2b526bcfc90d3f584 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:13.068340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18b9b2-b16b-4880-a57e-b46ac05ef998 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:13.068643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18b9b2-b16b-4880-a57e-b46ac05ef998 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:13.068910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18b9b2-b16b-4880-a57e-b46ac05ef998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:13.117096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bc18b9b2-b16b-4880-a57e-b46ac05ef998 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:13.120912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bc18b9b2-b16b-4880-a57e-b46ac05ef998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:13.131983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04453654-2ebe-485a-873e-71b4cc035898 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:13.263993 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3142/6288] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:13.265396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5314d-df59-4e08-95ce-812e30ed4c2e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:13.265633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5314d-df59-4e08-95ce-812e30ed4c2e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:13.265901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5314d-df59-4e08-95ce-812e30ed4c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:13.269013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619a442-db99-4ae2-a8b1-d3d35946a326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:13.269772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619a442-db99-4ae2-a8b1-d3d35946a326 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:13.270045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619a442-db99-4ae2-a8b1-d3d35946a326 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:13.270312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619a442-db99-4ae2-a8b1-d3d35946a326 admin admin] PATH_INFO: `/v3/projects/5ed1de1718684e39987868560cb90dd2/users/6fd0d88ede8d4b25ac0b4607dbc16ae0/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:13.275657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619a442-db99-4ae2-a8b1-d3d35946a326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed1de1718684e39987868560cb90dd2, user_id=6fd0d88ede8d4b25ac0b4607dbc16ae0, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:13.285864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619a442-db99-4ae2-a8b1-d3d35946a326 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:13.332519 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3147/6289] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:13 2022] PUT /identity/v3/projects/5ed1de1718684e39987868560cb90dd2/users/6fd0d88ede8d4b25ac0b4607dbc16ae0/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:13.333922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619a442-db99-4ae2-a8b1-d3d35946a326 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:13.336207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8619a442-db99-4ae2-a8b1-d3d35946a326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:13.341579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-53c5314d-df59-4e08-95ce-812e30ed4c2e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:13.342866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-53c5314d-df59-4e08-95ce-812e30ed4c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:13.371288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:13.466022 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3143/6290] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:13.467699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813a8ea-11fd-4a66-a828-e3100b2bb0e2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:13.469630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813a8ea-11fd-4a66-a828-e3100b2bb0e2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:13.475685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619a442-db99-4ae2-a8b1-d3d35946a326 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:13.477470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813a8ea-11fd-4a66-a828-e3100b2bb0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:13.486706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d24172-0e69-460d-8a34-040c1d6625a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:13.486706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d24172-0e69-460d-8a34-040c1d6625a4 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:13.486706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d24172-0e69-460d-8a34-040c1d6625a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:13.487793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d24172-0e69-460d-8a34-040c1d6625a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:13.515634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d24172-0e69-460d-8a34-040c1d6625a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:13.517653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d24172-0e69-460d-8a34-040c1d6625a4 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:13.526927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7813a8ea-11fd-4a66-a828-e3100b2bb0e2 None None] MFA Rules not processed for user `6fd0d88ede8d4b25ac0b4607dbc16ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:13.529866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7813a8ea-11fd-4a66-a828-e3100b2bb0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:13.550928 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3148/6291] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:13.567834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d24172-0e69-460d-8a34-040c1d6625a4 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:13.602112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d24172-0e69-460d-8a34-040c1d6625a4 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:13.607490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515504dd-bae5-45d1-9ec8-e08b764e5928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:13.608222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515504dd-bae5-45d1-9ec8-e08b764e5928 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:13.608864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515504dd-bae5-45d1-9ec8-e08b764e5928 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:13.609271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515504dd-bae5-45d1-9ec8-e08b764e5928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:13.623092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515504dd-bae5-45d1-9ec8-e08b764e5928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:13.625666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515504dd-bae5-45d1-9ec8-e08b764e5928 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:13.640385 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3144/6292] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:13.652298 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3149/6293] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:13.654471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813a8ea-11fd-4a66-a828-e3100b2bb0e2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:13.655978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7813a8ea-11fd-4a66-a828-e3100b2bb0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:13.764943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813a8ea-11fd-4a66-a828-e3100b2bb0e2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:13.769741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eec339-8e55-4405-b000-f268914aa4c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:13.772050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eec339-8e55-4405-b000-f268914aa4c9 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:13.772663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eec339-8e55-4405-b000-f268914aa4c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:13.773173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eec339-8e55-4405-b000-f268914aa4c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:13.787560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eec339-8e55-4405-b000-f268914aa4c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:13.789590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eec339-8e55-4405-b000-f268914aa4c9 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:13.826030 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3145/6294] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:13.927685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515504dd-bae5-45d1-9ec8-e08b764e5928 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:13.943145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515504dd-bae5-45d1-9ec8-e08b764e5928 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:13.947621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408dc3e-f229-4b50-baa8-41d8af2e9af9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:13.948352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408dc3e-f229-4b50-baa8-41d8af2e9af9 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:13.948776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408dc3e-f229-4b50-baa8-41d8af2e9af9 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:13.949068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408dc3e-f229-4b50-baa8-41d8af2e9af9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:13.950916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0408dc3e-f229-4b50-baa8-41d8af2e9af9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:13.959579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eec339-8e55-4405-b000-f268914aa4c9 service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:13.962494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76eec339-8e55-4405-b000-f268914aa4c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:13.972132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:14.008684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eec339-8e55-4405-b000-f268914aa4c9 service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:14.015242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:14.015881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:14.016159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:14.016423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] PATH_INFO: `/v3/projects/5ae7ff3c71ce49c4a8378a177cc1e09f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:14.018796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ae7ff3c71ce49c4a8378a177cc1e09f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:14.020704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:14.033359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408dc3e-f229-4b50-baa8-41d8af2e9af9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:14.035126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408dc3e-f229-4b50-baa8-41d8af2e9af9 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:14.044880 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3150/6295] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:14.080438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:14.085792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ae7ff3c71ce49c4a8378a177cc1e09f', 'request_id': 'req-bac8075a-1522-467f-b638-e4464d71721c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.095175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ae7ff3c71ce49c4a8378a177cc1e09f', 'request_id': 'req-bac8075a-1522-467f-b638-e4464d71721c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.105376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ae7ff3c71ce49c4a8378a177cc1e09f', 'request_id': 'req-bac8075a-1522-467f-b638-e4464d71721c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.131645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] The token cache is being invalidate because project 5ae7ff3c71ce49c4a8378a177cc1e09f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:14.132234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.132972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.137965 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3146/6296] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:13 2022] DELETE /identity/v3/projects/5ae7ff3c71ce49c4a8378a177cc1e09f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:14.140654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408dc3e-f229-4b50-baa8-41d8af2e9af9 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:14.157753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408dc3e-f229-4b50-baa8-41d8af2e9af9 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:14.277226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:14.278107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:14.278828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:14.279177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] PATH_INFO: `/v3/users/0c73bc0dc4a74ced86290054b688bb87` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:14.280492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c73bc0dc4a74ced86290054b688bb87)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:14.283187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:14.359193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c73bc0dc4a74ced86290054b688bb87'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.379081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c73bc0dc4a74ced86290054b688bb87'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.404642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c73bc0dc4a74ced86290054b688bb87'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.424571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c73bc0dc4a74ced86290054b688bb87'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.425279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-551f6a4b-703b-4cad-a916-f703bb81f24c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c73bc0dc4a74ced86290054b688bb87'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.446644 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3151/6297] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:14 2022] DELETE /identity/v3/users/0c73bc0dc4a74ced86290054b688bb87 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:14.467989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:14.469699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:14.495851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8075a-1522-467f-b638-e4464d71721c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:14.626411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:14.626886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:14.627312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:14.627743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] PATH_INFO: `/v3/projects/44e7b272dfde47d7a9a4ef56e93088d3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:14.630302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] RBAC: Authorizing `identity:delete_project(project_id=44e7b272dfde47d7a9a4ef56e93088d3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:14.632677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:14.718622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44e7b272dfde47d7a9a4ef56e93088d3', 'request_id': 'req-a0519d9b-57a8-41fc-81b5-85717757250e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.736919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44e7b272dfde47d7a9a4ef56e93088d3', 'request_id': 'req-a0519d9b-57a8-41fc-81b5-85717757250e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.750082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44e7b272dfde47d7a9a4ef56e93088d3', 'request_id': 'req-a0519d9b-57a8-41fc-81b5-85717757250e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.766867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] The token cache is being invalidate because project 44e7b272dfde47d7a9a4ef56e93088d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:14.767275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.767997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a0519d9b-57a8-41fc-81b5-85717757250e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:14.770418 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3147/6298] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:14 2022] DELETE /identity/v3/projects/44e7b272dfde47d7a9a4ef56e93088d3 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:14.776672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604cf2b-74c1-4d3d-b572-db24b44424bf None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:14.777157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604cf2b-74c1-4d3d-b572-db24b44424bf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:14.777527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604cf2b-74c1-4d3d-b572-db24b44424bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:14.802353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5604cf2b-74c1-4d3d-b572-db24b44424bf None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:14.803692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5604cf2b-74c1-4d3d-b572-db24b44424bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:14.837549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:14.873431 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3152/6299] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:14.881506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56347d91-8753-4f5c-b7ef-4e1a1346cd0d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:14.881779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56347d91-8753-4f5c-b7ef-4e1a1346cd0d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:14.882076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56347d91-8753-4f5c-b7ef-4e1a1346cd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:14.902670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-56347d91-8753-4f5c-b7ef-4e1a1346cd0d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:14.903874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-56347d91-8753-4f5c-b7ef-4e1a1346cd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:14.911889 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3148/6300] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:14.916914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604cf2b-74c1-4d3d-b572-db24b44424bf None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:14.927014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604cf2b-74c1-4d3d-b572-db24b44424bf None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:14.933188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf330f-d11c-4cd0-a030-ab41cb747c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:14.933979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf330f-d11c-4cd0-a030-ab41cb747c17 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:14.934249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf330f-d11c-4cd0-a030-ab41cb747c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:14.934506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf330f-d11c-4cd0-a030-ab41cb747c17 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:14.935153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf330f-d11c-4cd0-a030-ab41cb747c17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:14.935446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf330f-d11c-4cd0-a030-ab41cb747c17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:14.937143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf330f-d11c-4cd0-a030-ab41cb747c17 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:14.953583 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3153/6301] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:14.954954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56347d91-8753-4f5c-b7ef-4e1a1346cd0d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:14.965608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56347d91-8753-4f5c-b7ef-4e1a1346cd0d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:14.970992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f9e9-5799-4b42-a6d1-3e0c153e80d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:14.971614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f9e9-5799-4b42-a6d1-3e0c153e80d9 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:14.971888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f9e9-5799-4b42-a6d1-3e0c153e80d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:14.972157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f9e9-5799-4b42-a6d1-3e0c153e80d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:14.972820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77f9e9-5799-4b42-a6d1-3e0c153e80d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:14.974872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77f9e9-5799-4b42-a6d1-3e0c153e80d9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:15.008422 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3149/6302] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:14 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:15.009719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf330f-d11c-4cd0-a030-ab41cb747c17 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:15.020675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf330f-d11c-4cd0-a030-ab41cb747c17 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:15.084741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa897f-fc10-45ea-b1d5-c19124d4dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:15.085870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa897f-fc10-45ea-b1d5-c19124d4dfbd admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:15.086223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa897f-fc10-45ea-b1d5-c19124d4dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:15.086572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa897f-fc10-45ea-b1d5-c19124d4dfbd admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:15.089573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa897f-fc10-45ea-b1d5-c19124d4dfbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:15.090976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa897f-fc10-45ea-b1d5-c19124d4dfbd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:15.127066 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3154/6303] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:15 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:15.132749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f9e9-5799-4b42-a6d1-3e0c153e80d9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:15.146600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f9e9-5799-4b42-a6d1-3e0c153e80d9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:15.151765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08321f-542d-4eb2-b484-723671cabdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:15.152733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08321f-542d-4eb2-b484-723671cabdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:15.153105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08321f-542d-4eb2-b484-723671cabdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:15.153490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08321f-542d-4eb2-b484-723671cabdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:15.154596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08321f-542d-4eb2-b484-723671cabdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:15.156226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08321f-542d-4eb2-b484-723671cabdc1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:15.168205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3150/6304] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:15.172155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa897f-fc10-45ea-b1d5-c19124d4dfbd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:15.186024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa897f-fc10-45ea-b1d5-c19124d4dfbd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:15.197615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d5ac8-68f2-41c5-ae21-5e6422527ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:15.198666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d5ac8-68f2-41c5-ae21-5e6422527ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:15.199090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d5ac8-68f2-41c5-ae21-5e6422527ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:15.199505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d5ac8-68f2-41c5-ae21-5e6422527ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:15.200298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d5ac8-68f2-41c5-ae21-5e6422527ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:15.202099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d5ac8-68f2-41c5-ae21-5e6422527ba5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:15.217289 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3155/6305] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:15.221744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08321f-542d-4eb2-b484-723671cabdc1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:15.232931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08321f-542d-4eb2-b484-723671cabdc1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:15.238237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940041c-1a00-4d89-a40d-ad4f2bedf9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:15.239351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940041c-1a00-4d89-a40d-ad4f2bedf9cc admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:15.239351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940041c-1a00-4d89-a40d-ad4f2bedf9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:15.239504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940041c-1a00-4d89-a40d-ad4f2bedf9cc admin admin] PATH_INFO: `/v3/projects/870d73d996244e4199f4ba8033056277/users/5b78c768bf14494f89baa259e135ecc9/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:15.263641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940041c-1a00-4d89-a40d-ad4f2bedf9cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=870d73d996244e4199f4ba8033056277, user_id=5b78c768bf14494f89baa259e135ecc9, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:15.263641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940041c-1a00-4d89-a40d-ad4f2bedf9cc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:15.271620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:15.314398 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3151/6306] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:15 2022] PUT /identity/v3/projects/870d73d996244e4199f4ba8033056277/users/5b78c768bf14494f89baa259e135ecc9/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:15.318926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21254940-6eec-479d-ae05-a81a823593a3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:15.319409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21254940-6eec-479d-ae05-a81a823593a3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:15.319844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21254940-6eec-479d-ae05-a81a823593a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:15.363577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-21254940-6eec-479d-ae05-a81a823593a3 None None] MFA Rules not processed for user `5b78c768bf14494f89baa259e135ecc9`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:15.365595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-21254940-6eec-479d-ae05-a81a823593a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:15.500989 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3156/6307] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:15.508177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940041c-1a00-4d89-a40d-ad4f2bedf9cc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:15.510836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6940041c-1a00-4d89-a40d-ad4f2bedf9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:15.638166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940041c-1a00-4d89-a40d-ad4f2bedf9cc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:15.644105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326ac05-1342-46a7-8b2d-998669bcbde5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:15.645457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326ac05-1342-46a7-8b2d-998669bcbde5 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:15.645997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326ac05-1342-46a7-8b2d-998669bcbde5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:15.646564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326ac05-1342-46a7-8b2d-998669bcbde5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:15.663942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326ac05-1342-46a7-8b2d-998669bcbde5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:15.668818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326ac05-1342-46a7-8b2d-998669bcbde5 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:15.693113 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3152/6308] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:15.894868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21254940-6eec-479d-ae05-a81a823593a3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:15.905845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-21254940-6eec-479d-ae05-a81a823593a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:16.072630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:16.173135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21254940-6eec-479d-ae05-a81a823593a3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:16.193415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:16.193581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:16.193764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:16.194034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] PATH_INFO: `/v3/projects/9e87a8e1c985498697dc2b129d97b67a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:16.197702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e87a8e1c985498697dc2b129d97b67a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:16.202303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:16.447697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e87a8e1c985498697dc2b129d97b67a', 'request_id': 'req-ac3292b6-36ef-4a0a-b93e-39f9997669e5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:16.463715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e87a8e1c985498697dc2b129d97b67a', 'request_id': 'req-ac3292b6-36ef-4a0a-b93e-39f9997669e5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:16.471295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e87a8e1c985498697dc2b129d97b67a', 'request_id': 'req-ac3292b6-36ef-4a0a-b93e-39f9997669e5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:16.491967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] The token cache is being invalidate because project 9e87a8e1c985498697dc2b129d97b67a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:16.492273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:16.493050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac3292b6-36ef-4a0a-b93e-39f9997669e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:16.499363 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3157/6309] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:15 2022] DELETE /identity/v3/projects/9e87a8e1c985498697dc2b129d97b67a => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:16.500487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8223649-3c36-4d47-918e-0d2c208d5eff None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:16.500863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8223649-3c36-4d47-918e-0d2c208d5eff None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:16.502252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8223649-3c36-4d47-918e-0d2c208d5eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:16.574510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c8223649-3c36-4d47-918e-0d2c208d5eff None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:16.576605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c8223649-3c36-4d47-918e-0d2c208d5eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:16.711427 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3153/6310] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:16.720388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbd761-2475-4ef5-92e7-7c338ab5f1ae None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:16.720663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbd761-2475-4ef5-92e7-7c338ab5f1ae None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:16.720936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbd761-2475-4ef5-92e7-7c338ab5f1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:16.778657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3acbd761-2475-4ef5-92e7-7c338ab5f1ae None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:16.780244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3acbd761-2475-4ef5-92e7-7c338ab5f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:16.803258 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3158/6311] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:16.808775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8223649-3c36-4d47-918e-0d2c208d5eff None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:16.840978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8223649-3c36-4d47-918e-0d2c208d5eff None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:16.851242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2b4d2-334a-473d-8aba-3a34d37be559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:16.852247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2b4d2-334a-473d-8aba-3a34d37be559 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:16.852749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2b4d2-334a-473d-8aba-3a34d37be559 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:16.853457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2b4d2-334a-473d-8aba-3a34d37be559 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:16.854409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2b4d2-334a-473d-8aba-3a34d37be559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:16.855143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2b4d2-334a-473d-8aba-3a34d37be559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:16.859299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2b4d2-334a-473d-8aba-3a34d37be559 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:16.889594 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3154/6312] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:15:16.895021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbd761-2475-4ef5-92e7-7c338ab5f1ae None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:16.923826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbd761-2475-4ef5-92e7-7c338ab5f1ae None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:16.930604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282252b-0d2f-4bb2-9f1b-75e8a4c91e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:16.932244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282252b-0d2f-4bb2-9f1b-75e8a4c91e0c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:16.932603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282252b-0d2f-4bb2-9f1b-75e8a4c91e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:16.933121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282252b-0d2f-4bb2-9f1b-75e8a4c91e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:16.933889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282252b-0d2f-4bb2-9f1b-75e8a4c91e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:16.937594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282252b-0d2f-4bb2-9f1b-75e8a4c91e0c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:16.976743 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3159/6313] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:16 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:16.982452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2b4d2-334a-473d-8aba-3a34d37be559 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:17.004328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2b4d2-334a-473d-8aba-3a34d37be559 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:17.186177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee81bb-db15-44d0-a109-a7c1761e5c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:17.188217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee81bb-db15-44d0-a109-a7c1761e5c7d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:17.196433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee81bb-db15-44d0-a109-a7c1761e5c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:17.196433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee81bb-db15-44d0-a109-a7c1761e5c7d admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:17.196433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee81bb-db15-44d0-a109-a7c1761e5c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:17.198065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee81bb-db15-44d0-a109-a7c1761e5c7d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:17.239281 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3155/6314] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:16 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:17.243363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282252b-0d2f-4bb2-9f1b-75e8a4c91e0c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:17.261438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282252b-0d2f-4bb2-9f1b-75e8a4c91e0c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:17.273289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd284242-7b05-4692-8061-64b3ff07144c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:17.276591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284242-7b05-4692-8061-64b3ff07144c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:17.276591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284242-7b05-4692-8061-64b3ff07144c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:17.276591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284242-7b05-4692-8061-64b3ff07144c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:17.279642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd284242-7b05-4692-8061-64b3ff07144c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:17.284695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd284242-7b05-4692-8061-64b3ff07144c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:17.313363 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3160/6315] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:15:17.317348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee81bb-db15-44d0-a109-a7c1761e5c7d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:17.339720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee81bb-db15-44d0-a109-a7c1761e5c7d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:17.352316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcef68-2681-4a0f-92ff-1db1d00deebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:17.353635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcef68-2681-4a0f-92ff-1db1d00deebd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:17.354095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcef68-2681-4a0f-92ff-1db1d00deebd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:17.354367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcef68-2681-4a0f-92ff-1db1d00deebd admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:17.355164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcef68-2681-4a0f-92ff-1db1d00deebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:17.358628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcef68-2681-4a0f-92ff-1db1d00deebd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:17.396266 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3156/6316] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:17.397474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd284242-7b05-4692-8061-64b3ff07144c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:17.424147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd284242-7b05-4692-8061-64b3ff07144c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:17.429739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ebe4f-e6ad-450c-bc91-eab9a75a1890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:17.430399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ebe4f-e6ad-450c-bc91-eab9a75a1890 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:17.430673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ebe4f-e6ad-450c-bc91-eab9a75a1890 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:17.430934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ebe4f-e6ad-450c-bc91-eab9a75a1890 admin admin] PATH_INFO: `/v3/projects/0ee7826b3c01475db29de402244cd4aa/users/8ef16f7ada164345896bcc34d3babe66/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:17.457149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ebe4f-e6ad-450c-bc91-eab9a75a1890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee7826b3c01475db29de402244cd4aa, user_id=8ef16f7ada164345896bcc34d3babe66, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:17.472040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ebe4f-e6ad-450c-bc91-eab9a75a1890 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:17.515971 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3161/6317] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:17 2022] PUT /identity/v3/projects/0ee7826b3c01475db29de402244cd4aa/users/8ef16f7ada164345896bcc34d3babe66/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:17.518264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e3bfb-ea9c-4088-8982-20c1e5c1e7b5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:17.518545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e3bfb-ea9c-4088-8982-20c1e5c1e7b5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:17.518830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e3bfb-ea9c-4088-8982-20c1e5c1e7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:17.539165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:17.578433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fd7e3bfb-ea9c-4088-8982-20c1e5c1e7b5 None None] MFA Rules not processed for user `8ef16f7ada164345896bcc34d3babe66`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:17.583560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd7e3bfb-ea9c-4088-8982-20c1e5c1e7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:17.782268 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3157/6318] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:17.805027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ebe4f-e6ad-450c-bc91-eab9a75a1890 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:17.808498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-331ebe4f-e6ad-450c-bc91-eab9a75a1890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:17.989266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ebe4f-e6ad-450c-bc91-eab9a75a1890 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:17.999576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352490e8-3100-4076-81bd-a19d7b87abb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:18.001568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352490e8-3100-4076-81bd-a19d7b87abb1 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:18.003489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352490e8-3100-4076-81bd-a19d7b87abb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:18.004140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352490e8-3100-4076-81bd-a19d7b87abb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:18.024535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352490e8-3100-4076-81bd-a19d7b87abb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:18.031962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352490e8-3100-4076-81bd-a19d7b87abb1 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:18.037393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:18.051820 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3162/6319] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:19.856094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e3bfb-ea9c-4088-8982-20c1e5c1e7b5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:19.861659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd7e3bfb-ea9c-4088-8982-20c1e5c1e7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:20.026318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e3bfb-ea9c-4088-8982-20c1e5c1e7b5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:20.032427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:20.033171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:20.033452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:20.033724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] PATH_INFO: `/v3/users/b70d44ed499143b6af2e45ac03605109` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:20.034921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b70d44ed499143b6af2e45ac03605109)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:20.036876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:20.106587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b70d44ed499143b6af2e45ac03605109'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:20.118087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b70d44ed499143b6af2e45ac03605109'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:20.139080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b70d44ed499143b6af2e45ac03605109'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:20.152378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b70d44ed499143b6af2e45ac03605109'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:20.153297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b70d44ed499143b6af2e45ac03605109'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:20.162888 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3158/6320] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:19 2022] DELETE /identity/v3/users/b70d44ed499143b6af2e45ac03605109 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:20.170316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f260-e9b1-4727-a118-4e364967160e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:20.171228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f260-e9b1-4727-a118-4e364967160e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:20.171730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f260-e9b1-4727-a118-4e364967160e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:20.199155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7f58f260-e9b1-4727-a118-4e364967160e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:20.201298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f58f260-e9b1-4727-a118-4e364967160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:20.286737 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3163/6321] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:20.305669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:20.308826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:20.341684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:20.432900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b09be3-e9ca-4412-81ff-01a9b3cdd428 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:20.438001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09faa895-2f6b-4d1b-8f5b-d2aa590da603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:20.438617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09faa895-2f6b-4d1b-8f5b-d2aa590da603 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:20.438888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09faa895-2f6b-4d1b-8f5b-d2aa590da603 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:20.439151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09faa895-2f6b-4d1b-8f5b-d2aa590da603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:20.453177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09faa895-2f6b-4d1b-8f5b-d2aa590da603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:20.454647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09faa895-2f6b-4d1b-8f5b-d2aa590da603 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:20.468876 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3159/6322] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:22.474599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58f260-e9b1-4727-a118-4e364967160e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:22.481745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f58f260-e9b1-4727-a118-4e364967160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:22.533235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58f260-e9b1-4727-a118-4e364967160e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:22.555660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:22.558719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:22.560620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:22.563628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] PATH_INFO: `/v3/projects/1cbf839ca453402b8694a15b2e5edf97` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:22.565832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cbf839ca453402b8694a15b2e5edf97)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:22.571226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:22.719113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cbf839ca453402b8694a15b2e5edf97', 'request_id': 'req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:22.740989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cbf839ca453402b8694a15b2e5edf97', 'request_id': 'req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:22.760712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cbf839ca453402b8694a15b2e5edf97', 'request_id': 'req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:22.808303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] The token cache is being invalidate because project 1cbf839ca453402b8694a15b2e5edf97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:22.809123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:22.812879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:22.819849 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3164/6323] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:22 2022] DELETE /identity/v3/projects/1cbf839ca453402b8694a15b2e5edf97 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:22.824258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09faa895-2f6b-4d1b-8f5b-d2aa590da603 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:22.844283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09faa895-2f6b-4d1b-8f5b-d2aa590da603 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:22.966877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:22.968570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:22.969109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:22.969745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] PATH_INFO: `/v3/users/d069b91e777444d5b278419a8333a3bb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:22.971164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d069b91e777444d5b278419a8333a3bb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:22.973439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:23.123312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd069b91e777444d5b278419a8333a3bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:23.138705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd069b91e777444d5b278419a8333a3bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:23.172232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd069b91e777444d5b278419a8333a3bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:23.191188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd069b91e777444d5b278419a8333a3bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:23.191188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d0a026f6-6548-4fd9-bd99-e3e11e802ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd069b91e777444d5b278419a8333a3bb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:23.203811 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3160/6324] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:22 2022] DELETE /identity/v3/users/d069b91e777444d5b278419a8333a3bb => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:24.098960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:24.101118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:24.121455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6f5c-5e72-41aa-a388-00f64b1c2fd2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:24.206366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:24.207633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:24.207917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:24.208238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] PATH_INFO: `/v3/users/fa6c55b6a8f14b2fb6f1df38f9f8cd39` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:24.210942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6c55b6a8f14b2fb6f1df38f9f8cd39)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:24.214531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:24.249538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:24.310334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6c55b6a8f14b2fb6f1df38f9f8cd39'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:24.340106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6c55b6a8f14b2fb6f1df38f9f8cd39'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:24.355900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6c55b6a8f14b2fb6f1df38f9f8cd39'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:24.371057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6c55b6a8f14b2fb6f1df38f9f8cd39'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:24.372134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6c55b6a8f14b2fb6f1df38f9f8cd39'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:24.380949 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3165/6325] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:24 2022] DELETE /identity/v3/users/fa6c55b6a8f14b2fb6f1df38f9f8cd39 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:24.387904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29169396-8617-42e2-97bb-347b3dddeb17 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:24.389153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29169396-8617-42e2-97bb-347b3dddeb17 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:24.389603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29169396-8617-42e2-97bb-347b3dddeb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:24.433145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-29169396-8617-42e2-97bb-347b3dddeb17 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:24.436412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-29169396-8617-42e2-97bb-347b3dddeb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:24.492621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:24.531280 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3161/6326] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:24.542201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:24.543975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:24.618027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8be70-0120-45d6-96a8-dd5fd39fa511 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:24.622656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca656814-d877-480d-8353-9a5e0de15117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:24.623308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca656814-d877-480d-8353-9a5e0de15117 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:24.623583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca656814-d877-480d-8353-9a5e0de15117 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:24.624000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca656814-d877-480d-8353-9a5e0de15117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:24.635684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca656814-d877-480d-8353-9a5e0de15117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:24.637233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca656814-d877-480d-8353-9a5e0de15117 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:24.650956 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3166/6327] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:25.322840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29169396-8617-42e2-97bb-347b3dddeb17 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:25.324536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-29169396-8617-42e2-97bb-347b3dddeb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:25.338946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29169396-8617-42e2-97bb-347b3dddeb17 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:25.344837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:25.345498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:25.345882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:25.346182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] PATH_INFO: `/v3/projects/162755a515a74ea9866470e068351434` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:25.347410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=162755a515a74ea9866470e068351434)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:25.349529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:25.445281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '162755a515a74ea9866470e068351434', 'request_id': 'req-30d52e80-bb96-418a-ac40-c45b3ed4e47f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:25.457987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '162755a515a74ea9866470e068351434', 'request_id': 'req-30d52e80-bb96-418a-ac40-c45b3ed4e47f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:25.474301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '162755a515a74ea9866470e068351434', 'request_id': 'req-30d52e80-bb96-418a-ac40-c45b3ed4e47f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:25.527498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] The token cache is being invalidate because project 162755a515a74ea9866470e068351434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:25.527953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:25.528841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:25.535262 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3162/6328] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:25 2022] DELETE /identity/v3/projects/162755a515a74ea9866470e068351434 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:25.536637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967e441-4f99-453a-a1ce-8cd27f73dec7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:25.537523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967e441-4f99-453a-a1ce-8cd27f73dec7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:25.537920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967e441-4f99-453a-a1ce-8cd27f73dec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:25.563946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4967e441-4f99-453a-a1ce-8cd27f73dec7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:25.565390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4967e441-4f99-453a-a1ce-8cd27f73dec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:25.744651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:25.754820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:25.757076 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3167/6329] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:25.764007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1973d0-9056-4e8c-b531-71adf5b3695f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:25.764351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1973d0-9056-4e8c-b531-71adf5b3695f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:25.764729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1973d0-9056-4e8c-b531-71adf5b3695f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:25.784192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d52e80-bb96-418a-ac40-c45b3ed4e47f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:25.799361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:25.800023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:25.800294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:25.800558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] PATH_INFO: `/v3/projects/baadbf9da41041719cc9f50d8b5ef52b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:25.803559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=baadbf9da41041719cc9f50d8b5ef52b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:25.807240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:25.863936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3b1973d0-9056-4e8c-b531-71adf5b3695f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:25.883822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3b1973d0-9056-4e8c-b531-71adf5b3695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:25.958052 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3168/6330] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:25.963368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1973d0-9056-4e8c-b531-71adf5b3695f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:25.976431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1973d0-9056-4e8c-b531-71adf5b3695f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:26.025867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baadbf9da41041719cc9f50d8b5ef52b', 'request_id': 'req-3ebefbac-9c34-4212-8f32-155374162a8b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:26.027546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:26.059579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baadbf9da41041719cc9f50d8b5ef52b', 'request_id': 'req-3ebefbac-9c34-4212-8f32-155374162a8b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:26.073447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baadbf9da41041719cc9f50d8b5ef52b', 'request_id': 'req-3ebefbac-9c34-4212-8f32-155374162a8b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:26.090847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] The token cache is being invalidate because project baadbf9da41041719cc9f50d8b5ef52b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:26.091290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:26.095466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:26.098158 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3163/6331] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:25 2022] DELETE /identity/v3/projects/baadbf9da41041719cc9f50d8b5ef52b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:26.102082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:26.106390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:26.106529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2168d3-888c-469f-93d8-b51e54734058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:26.107125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2168d3-888c-469f-93d8-b51e54734058 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:26.110342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2168d3-888c-469f-93d8-b51e54734058 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:26.110889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2168d3-888c-469f-93d8-b51e54734058 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:26.111737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2168d3-888c-469f-93d8-b51e54734058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:26.112048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2168d3-888c-469f-93d8-b51e54734058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:26.117062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2168d3-888c-469f-93d8-b51e54734058 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:26.135494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebefbac-9c34-4212-8f32-155374162a8b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:26.141761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:26.142903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:26.143358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:26.143950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] PATH_INFO: `/v3/users/6fd0d88ede8d4b25ac0b4607dbc16ae0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:26.145615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd0d88ede8d4b25ac0b4607dbc16ae0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:26.148198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:26.153996 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3169/6332] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:26.164874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2168d3-888c-469f-93d8-b51e54734058 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:26.168411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dc2168d3-888c-469f-93d8-b51e54734058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:26.223811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2168d3-888c-469f-93d8-b51e54734058 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:26.232805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48eaa7d-547a-45a9-999f-a8c4c963811c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:26.233947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eaa7d-547a-45a9-999f-a8c4c963811c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:26.234601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eaa7d-547a-45a9-999f-a8c4c963811c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:26.235118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eaa7d-547a-45a9-999f-a8c4c963811c admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:26.236631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48eaa7d-547a-45a9-999f-a8c4c963811c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:26.241177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48eaa7d-547a-45a9-999f-a8c4c963811c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:26.285872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:26.301399 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3170/6333] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:26 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:26.301651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd0d88ede8d4b25ac0b4607dbc16ae0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:26.324725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48eaa7d-547a-45a9-999f-a8c4c963811c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:26.324864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd0d88ede8d4b25ac0b4607dbc16ae0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:26.350227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd0d88ede8d4b25ac0b4607dbc16ae0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:26.367034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd0d88ede8d4b25ac0b4607dbc16ae0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:26.367972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd0d88ede8d4b25ac0b4607dbc16ae0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:26.377085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3164/6334] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:26 2022] DELETE /identity/v3/users/6fd0d88ede8d4b25ac0b4607dbc16ae0 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:26.475596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48eaa7d-547a-45a9-999f-a8c4c963811c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:26.612546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9a0fc-21a8-4f5e-835e-b8f551cdea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:26.614068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9a0fc-21a8-4f5e-835e-b8f551cdea55 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:26.614347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9a0fc-21a8-4f5e-835e-b8f551cdea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:26.614617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9a0fc-21a8-4f5e-835e-b8f551cdea55 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:26.615396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9a0fc-21a8-4f5e-835e-b8f551cdea55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:26.617637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9a0fc-21a8-4f5e-835e-b8f551cdea55 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:26.654833 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3171/6335] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:26 2022] POST /identity/v3/users => generated 291 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:26.661923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:26.664507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:26.686844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13670c20-0d06-4c30-b5d9-3416bb92bcfc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:26.693810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f697821-d2e1-4721-85d1-1350792ac8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:26.695480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f697821-d2e1-4721-85d1-1350792ac8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:26.696904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f697821-d2e1-4721-85d1-1350792ac8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:26.697571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f697821-d2e1-4721-85d1-1350792ac8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:26.698346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f697821-d2e1-4721-85d1-1350792ac8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:26.702635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f697821-d2e1-4721-85d1-1350792ac8d3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:26.739347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9a0fc-21a8-4f5e-835e-b8f551cdea55 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:26.748749 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3165/6336] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:26.753848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9a0fc-21a8-4f5e-835e-b8f551cdea55 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:26.760792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174e999-f7f4-4cca-a801-772e09f7ff8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:26.761808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174e999-f7f4-4cca-a801-772e09f7ff8c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:26.762087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174e999-f7f4-4cca-a801-772e09f7ff8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:26.762367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174e999-f7f4-4cca-a801-772e09f7ff8c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:26.763739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174e999-f7f4-4cca-a801-772e09f7ff8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:26.764653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174e999-f7f4-4cca-a801-772e09f7ff8c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:26.780176 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3172/6337] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:26.781194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f697821-d2e1-4721-85d1-1350792ac8d3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:26.800063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f697821-d2e1-4721-85d1-1350792ac8d3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:26.808703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85b14f-bf85-403e-a903-2614eaf7142d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:26.809621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85b14f-bf85-403e-a903-2614eaf7142d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:26.810239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85b14f-bf85-403e-a903-2614eaf7142d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:26.810676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85b14f-bf85-403e-a903-2614eaf7142d admin admin] PATH_INFO: `/v3/projects/2ed54e51ee9641acbe3f8f242497b1dd/users/ad524efa33e64cba9ff7f8f56424f83b/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:26.837242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85b14f-bf85-403e-a903-2614eaf7142d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed54e51ee9641acbe3f8f242497b1dd, user_id=ad524efa33e64cba9ff7f8f56424f83b, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:26.840452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85b14f-bf85-403e-a903-2614eaf7142d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:26.861783 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3166/6338] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:26 2022] PUT /identity/v3/projects/2ed54e51ee9641acbe3f8f242497b1dd/users/ad524efa33e64cba9ff7f8f56424f83b/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:26.873927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20e87-12cb-4277-bffd-75ab15db34fe None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:26.874429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20e87-12cb-4277-bffd-75ab15db34fe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:26.874857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20e87-12cb-4277-bffd-75ab15db34fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:26.911138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dac20e87-12cb-4277-bffd-75ab15db34fe None None] MFA Rules not processed for user `ad524efa33e64cba9ff7f8f56424f83b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:26.912424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dac20e87-12cb-4277-bffd-75ab15db34fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:26.980271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:27.103365 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3173/6339] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:27.107662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85b14f-bf85-403e-a903-2614eaf7142d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:27.228253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85b14f-bf85-403e-a903-2614eaf7142d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:27.234623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ee71-572f-4159-9902-04ff2e95da33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:27.235431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ee71-572f-4159-9902-04ff2e95da33 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:27.236039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ee71-572f-4159-9902-04ff2e95da33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:27.237012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ee71-572f-4159-9902-04ff2e95da33 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:27.237813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5ee71-572f-4159-9902-04ff2e95da33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:27.239990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5ee71-572f-4159-9902-04ff2e95da33 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:27.288759 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3167/6340] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:27 2022] POST /identity/v3/projects => generated 381 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:15:27.295712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20e87-12cb-4277-bffd-75ab15db34fe None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:27.309066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20e87-12cb-4277-bffd-75ab15db34fe None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:27.395952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729f965-93cf-4904-a152-b23c4e5b4ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:27.396584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729f965-93cf-4904-a152-b23c4e5b4ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:27.396861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729f965-93cf-4904-a152-b23c4e5b4ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:27.398091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729f965-93cf-4904-a152-b23c4e5b4ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:27.399106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729f965-93cf-4904-a152-b23c4e5b4ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:27.403321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729f965-93cf-4904-a152-b23c4e5b4ed5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:27.454485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ee71-572f-4159-9902-04ff2e95da33 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:27.455194 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3174/6341] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:27 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:27.470651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ee71-572f-4159-9902-04ff2e95da33 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:27.479633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638129ba-4c41-4158-83fc-a731ef04757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:27.479633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638129ba-4c41-4158-83fc-a731ef04757f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:27.479633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638129ba-4c41-4158-83fc-a731ef04757f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:27.479633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638129ba-4c41-4158-83fc-a731ef04757f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:27.480160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638129ba-4c41-4158-83fc-a731ef04757f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:27.481586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638129ba-4c41-4158-83fc-a731ef04757f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:27.496991 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3168/6342] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:27.501530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729f965-93cf-4904-a152-b23c4e5b4ed5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:27.515858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729f965-93cf-4904-a152-b23c4e5b4ed5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:27.523260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27557a-f89f-48bc-89f8-a0c53f9c10a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:27.524074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27557a-f89f-48bc-89f8-a0c53f9c10a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:27.524356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27557a-f89f-48bc-89f8-a0c53f9c10a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:27.524665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27557a-f89f-48bc-89f8-a0c53f9c10a9 admin admin] PATH_INFO: `/v3/projects/ca9fec694ac44940b718f7f7bcf80864/users/10e3d73a197346a88716a7342c9c4716/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:27.550220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27557a-f89f-48bc-89f8-a0c53f9c10a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9fec694ac44940b718f7f7bcf80864, user_id=10e3d73a197346a88716a7342c9c4716, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:27.556814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27557a-f89f-48bc-89f8-a0c53f9c10a9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:27.590638 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3175/6343] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:27 2022] PUT /identity/v3/projects/ca9fec694ac44940b718f7f7bcf80864/users/10e3d73a197346a88716a7342c9c4716/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:27.590824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638129ba-4c41-4158-83fc-a731ef04757f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:27.602412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638129ba-4c41-4158-83fc-a731ef04757f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:27.697394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:27.708837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b43d13-1582-4d18-af42-aab31fe805ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:27.709706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b43d13-1582-4d18-af42-aab31fe805ee admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:27.709992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b43d13-1582-4d18-af42-aab31fe805ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:27.710259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b43d13-1582-4d18-af42-aab31fe805ee admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:27.711150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b43d13-1582-4d18-af42-aab31fe805ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:27.713685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b43d13-1582-4d18-af42-aab31fe805ee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:27.733303 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3169/6344] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:15:27.736554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27557a-f89f-48bc-89f8-a0c53f9c10a9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:27.752945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27557a-f89f-48bc-89f8-a0c53f9c10a9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:27.758441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afedd9-9bb1-40ab-836d-73afce740da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:27.760756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afedd9-9bb1-40ab-836d-73afce740da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:27.761145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afedd9-9bb1-40ab-836d-73afce740da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:27.761318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b43d13-1582-4d18-af42-aab31fe805ee admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:27.761597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afedd9-9bb1-40ab-836d-73afce740da3 admin admin] PATH_INFO: `/v3/projects/ca9fec694ac44940b718f7f7bcf80864/users/10e3d73a197346a88716a7342c9c4716/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:27.763611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-28b43d13-1582-4d18-af42-aab31fe805ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:27.764062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afedd9-9bb1-40ab-836d-73afce740da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9fec694ac44940b718f7f7bcf80864, user_id=10e3d73a197346a88716a7342c9c4716, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:27.768671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afedd9-9bb1-40ab-836d-73afce740da3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:27.786076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b43d13-1582-4d18-af42-aab31fe805ee admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:27.798686 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3176/6345] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:27 2022] PUT /identity/v3/projects/ca9fec694ac44940b718f7f7bcf80864/users/10e3d73a197346a88716a7342c9c4716/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:27.807175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96082aa5-0804-4fb6-bc44-bdf7a720a159 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:27.808202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96082aa5-0804-4fb6-bc44-bdf7a720a159 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:27.808202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96082aa5-0804-4fb6-bc44-bdf7a720a159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:27.857454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-96082aa5-0804-4fb6-bc44-bdf7a720a159 None None] MFA Rules not processed for user `10e3d73a197346a88716a7342c9c4716`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:27.858731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-96082aa5-0804-4fb6-bc44-bdf7a720a159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:28.001685 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3177/6346] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:28.023490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96082aa5-0804-4fb6-bc44-bdf7a720a159 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:28.026049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-96082aa5-0804-4fb6-bc44-bdf7a720a159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:28.030411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:28.031087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:28.031363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:28.031829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] PATH_INFO: `/v3/projects/5ed1de1718684e39987868560cb90dd2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:28.031935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:28.032971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ed1de1718684e39987868560cb90dd2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:28.035889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:28.107059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ed1de1718684e39987868560cb90dd2', 'request_id': 'req-9455c198-4ab5-4d98-ab57-a2d95b7e6232'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:28.121063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ed1de1718684e39987868560cb90dd2', 'request_id': 'req-9455c198-4ab5-4d98-ab57-a2d95b7e6232'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:28.130879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ed1de1718684e39987868560cb90dd2', 'request_id': 'req-9455c198-4ab5-4d98-ab57-a2d95b7e6232'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:28.146207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] The token cache is being invalidate because project 5ed1de1718684e39987868560cb90dd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:28.146513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:28.147333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9455c198-4ab5-4d98-ab57-a2d95b7e6232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:28.153371 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3170/6347] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:27 2022] DELETE /identity/v3/projects/5ed1de1718684e39987868560cb90dd2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:28.157390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d231b-49d1-4e15-80d7-6384b67144e8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:28.157525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d231b-49d1-4e15-80d7-6384b67144e8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:28.157713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d231b-49d1-4e15-80d7-6384b67144e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:28.184623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96082aa5-0804-4fb6-bc44-bdf7a720a159 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:28.219674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0c8d231b-49d1-4e15-80d7-6384b67144e8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:28.224168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0c8d231b-49d1-4e15-80d7-6384b67144e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:28.327827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cf1ad-bfb7-4074-963e-7a87a46b167f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:28.328650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cf1ad-bfb7-4074-963e-7a87a46b167f service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:28.328933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cf1ad-bfb7-4074-963e-7a87a46b167f service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:28.329236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cf1ad-bfb7-4074-963e-7a87a46b167f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:28.331001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-842cf1ad-bfb7-4074-963e-7a87a46b167f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:28.361922 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3171/6348] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:28.370545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cd140-5fd3-4b27-af52-55e1450f7057 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:28.371489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cd140-5fd3-4b27-af52-55e1450f7057 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:28.372640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cd140-5fd3-4b27-af52-55e1450f7057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:28.397421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7f1cd140-5fd3-4b27-af52-55e1450f7057 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:28.398633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7f1cd140-5fd3-4b27-af52-55e1450f7057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:28.409338 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3172/6349] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:28.413905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cd140-5fd3-4b27-af52-55e1450f7057 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:28.429067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cd140-5fd3-4b27-af52-55e1450f7057 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:28.437532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f82da-81d0-4fc2-8310-572f5f41ea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:28.438277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f82da-81d0-4fc2-8310-572f5f41ea45 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:28.438644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f82da-81d0-4fc2-8310-572f5f41ea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:28.439013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f82da-81d0-4fc2-8310-572f5f41ea45 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:28.452687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cf1ad-bfb7-4074-963e-7a87a46b167f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:28.453370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f82da-81d0-4fc2-8310-572f5f41ea45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:28.453922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f82da-81d0-4fc2-8310-572f5f41ea45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:28.456143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f82da-81d0-4fc2-8310-572f5f41ea45 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:28.462202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cf1ad-bfb7-4074-963e-7a87a46b167f service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:28.481780 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3178/6350] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:15:28.487459 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3173/6351] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:28.491705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cf1ad-bfb7-4074-963e-7a87a46b167f service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:28.506302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cf1ad-bfb7-4074-963e-7a87a46b167f service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:28.514289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00b57-0a23-4cb5-8ced-324a3190c657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:28.515009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00b57-0a23-4cb5-8ced-324a3190c657 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:28.516623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00b57-0a23-4cb5-8ced-324a3190c657 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:28.517149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00b57-0a23-4cb5-8ced-324a3190c657 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:28.518432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd00b57-0a23-4cb5-8ced-324a3190c657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:28.520546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd00b57-0a23-4cb5-8ced-324a3190c657 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:28.562149 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3179/6352] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:28 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:28.566470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f82da-81d0-4fc2-8310-572f5f41ea45 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:28.573561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:28.604743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f82da-81d0-4fc2-8310-572f5f41ea45 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:28.732918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9649579-b8f8-407b-9834-b1eb43f0c662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:28.733583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9649579-b8f8-407b-9834-b1eb43f0c662 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:28.733820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9649579-b8f8-407b-9834-b1eb43f0c662 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:28.737413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9649579-b8f8-407b-9834-b1eb43f0c662 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:28.738325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9649579-b8f8-407b-9834-b1eb43f0c662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:28.740797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9649579-b8f8-407b-9834-b1eb43f0c662 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:28.792222 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3174/6353] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:28 2022] POST /identity/v3/users => generated 282 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:28.796052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00b57-0a23-4cb5-8ced-324a3190c657 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:28.812815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00b57-0a23-4cb5-8ced-324a3190c657 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:28.827172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfd63d-39d8-4a43-b2a0-2421388c1af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:28.827974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfd63d-39d8-4a43-b2a0-2421388c1af1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:28.828359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfd63d-39d8-4a43-b2a0-2421388c1af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:28.828719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfd63d-39d8-4a43-b2a0-2421388c1af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:28.829619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfd63d-39d8-4a43-b2a0-2421388c1af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:28.839057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfd63d-39d8-4a43-b2a0-2421388c1af1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:28.870863 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3180/6354] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:15:28.874825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9649579-b8f8-407b-9834-b1eb43f0c662 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:28.891803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9649579-b8f8-407b-9834-b1eb43f0c662 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:28.908659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ac2a9-afce-4555-a929-414af1b38c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:28.909693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac2a9-afce-4555-a929-414af1b38c5d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:28.910182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac2a9-afce-4555-a929-414af1b38c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:28.910462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac2a9-afce-4555-a929-414af1b38c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:28.911177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ac2a9-afce-4555-a929-414af1b38c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:28.913373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ac2a9-afce-4555-a929-414af1b38c5d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:28.964307 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3175/6355] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:28.969088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfd63d-39d8-4a43-b2a0-2421388c1af1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:28.991127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfd63d-39d8-4a43-b2a0-2421388c1af1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:29.006140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c647e23-1232-4f3a-983f-c1153ab3d824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:29.007664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c647e23-1232-4f3a-983f-c1153ab3d824 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:29.008053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c647e23-1232-4f3a-983f-c1153ab3d824 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:29.009732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c647e23-1232-4f3a-983f-c1153ab3d824 admin admin] PATH_INFO: `/v3/projects/5139e0e162734067965cfe7c20ee9f88/users/95ceb5467635488ca6c249ca155942a0/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:29.053030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c647e23-1232-4f3a-983f-c1153ab3d824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5139e0e162734067965cfe7c20ee9f88, user_id=95ceb5467635488ca6c249ca155942a0, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:29.059830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c647e23-1232-4f3a-983f-c1153ab3d824 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:29.093432 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3181/6356] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:28 2022] PUT /identity/v3/projects/5139e0e162734067965cfe7c20ee9f88/users/95ceb5467635488ca6c249ca155942a0/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:29.094616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ac2a9-afce-4555-a929-414af1b38c5d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:29.105416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a19fdc-2ea8-42e8-a03e-cdaef5e8383a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:29.107958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a19fdc-2ea8-42e8-a03e-cdaef5e8383a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:29.108598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a19fdc-2ea8-42e8-a03e-cdaef5e8383a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:29.152892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:29.173549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-91a19fdc-2ea8-42e8-a03e-cdaef5e8383a None None] MFA Rules not processed for user `95ceb5467635488ca6c249ca155942a0`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:29.175023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-91a19fdc-2ea8-42e8-a03e-cdaef5e8383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:29.260339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ac2a9-afce-4555-a929-414af1b38c5d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:29.327348 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3182/6357] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:29.332177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a19fdc-2ea8-42e8-a03e-cdaef5e8383a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:29.389853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029dd92-f7c0-45a2-9132-e9e8d345c03a tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10e3d73a197346a88716a7342c9c4716', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca9fec694ac44940b718f7f7bcf80864', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:29.390850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029dd92-f7c0-45a2-9132-e9e8d345c03a tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:29.391191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029dd92-f7c0-45a2-9132-e9e8d345c03a tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:29.391683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029dd92-f7c0-45a2-9132-e9e8d345c03a tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:29.392352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029dd92-f7c0-45a2-9132-e9e8d345c03a tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:29.394195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029dd92-f7c0-45a2-9132-e9e8d345c03a tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:29.396919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029dd92-f7c0-45a2-9132-e9e8d345c03a tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:29.428167 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3176/6358] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:29.432445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029dd92-f7c0-45a2-9132-e9e8d345c03a tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:29.444312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029dd92-f7c0-45a2-9132-e9e8d345c03a tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:29.455687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ee0a8-87d8-48d6-89f1-44bf913269d2 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10e3d73a197346a88716a7342c9c4716', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca9fec694ac44940b718f7f7bcf80864', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:29.456316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ee0a8-87d8-48d6-89f1-44bf913269d2 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:29.456596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ee0a8-87d8-48d6-89f1-44bf913269d2 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:29.456870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ee0a8-87d8-48d6-89f1-44bf913269d2 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:29.457585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ee0a8-87d8-48d6-89f1-44bf913269d2 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:29.459503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ee0a8-87d8-48d6-89f1-44bf913269d2 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:29.470563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a19fdc-2ea8-42e8-a03e-cdaef5e8383a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:29.485155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00854303-d082-45c7-8771-444ee24ae6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:29.486128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00854303-d082-45c7-8771-444ee24ae6bb admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:29.486306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00854303-d082-45c7-8771-444ee24ae6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:29.486571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00854303-d082-45c7-8771-444ee24ae6bb admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:29.487208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00854303-d082-45c7-8771-444ee24ae6bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:29.489188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00854303-d082-45c7-8771-444ee24ae6bb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:29.492453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:29.506438 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3177/6359] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:29 2022] POST /identity/v3/projects => generated 354 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:15:29.507765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ee0a8-87d8-48d6-89f1-44bf913269d2 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:29.557158 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3183/6360] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:29 2022] POST /identity/v3/projects => generated 365 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:29.561797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00854303-d082-45c7-8771-444ee24ae6bb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:29.627799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ee0a8-87d8-48d6-89f1-44bf913269d2 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:29.752689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00854303-d082-45c7-8771-444ee24ae6bb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:29.753260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067f025-78b5-4d64-9b12-0605a8bdbc72 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10e3d73a197346a88716a7342c9c4716', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca9fec694ac44940b718f7f7bcf80864', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:29.754082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f025-78b5-4d64-9b12-0605a8bdbc72 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:29.755481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f025-78b5-4d64-9b12-0605a8bdbc72 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:29.755854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f025-78b5-4d64-9b12-0605a8bdbc72 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:29.757785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067f025-78b5-4d64-9b12-0605a8bdbc72 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:29.758174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067f025-78b5-4d64-9b12-0605a8bdbc72 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:29.758595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19116a-2abf-4b8d-b6e0-6b2fd3d5e449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:29.760451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19116a-2abf-4b8d-b6e0-6b2fd3d5e449 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:29.760734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19116a-2abf-4b8d-b6e0-6b2fd3d5e449 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:29.761005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19116a-2abf-4b8d-b6e0-6b2fd3d5e449 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:29.763217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19116a-2abf-4b8d-b6e0-6b2fd3d5e449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:29.763505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067f025-78b5-4d64-9b12-0605a8bdbc72 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:29.764207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19116a-2abf-4b8d-b6e0-6b2fd3d5e449 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:29.793729 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3178/6361] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:29.816198 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3184/6362] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:29 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:29.821012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067f025-78b5-4d64-9b12-0605a8bdbc72 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:29.837027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067f025-78b5-4d64-9b12-0605a8bdbc72 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:29.844695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc361a16-97fd-4999-b36d-0bf02b8e5d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:29.845812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc361a16-97fd-4999-b36d-0bf02b8e5d56 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:29.846267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc361a16-97fd-4999-b36d-0bf02b8e5d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:29.846811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc361a16-97fd-4999-b36d-0bf02b8e5d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:29.847658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc361a16-97fd-4999-b36d-0bf02b8e5d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:29.850212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc361a16-97fd-4999-b36d-0bf02b8e5d56 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:29.862777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19116a-2abf-4b8d-b6e0-6b2fd3d5e449 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:29.874640 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3179/6363] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:29.875554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc361a16-97fd-4999-b36d-0bf02b8e5d56 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:29.882457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19116a-2abf-4b8d-b6e0-6b2fd3d5e449 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:29.888234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc361a16-97fd-4999-b36d-0bf02b8e5d56 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:29.896425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10e3d73a197346a88716a7342c9c4716', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca9fec694ac44940b718f7f7bcf80864', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:29.897318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:29.897598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:29.897882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] PATH_INFO: `/v3/projects/e4fe952102dd46de8d514821bdd0cad5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:29.899156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8419e-4667-4578-89aa-6ed8c6978ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:29.901746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8419e-4667-4578-89aa-6ed8c6978ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:29.902874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8419e-4667-4578-89aa-6ed8c6978ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:29.902874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8419e-4667-4578-89aa-6ed8c6978ec4 admin admin] PATH_INFO: `/v3/projects/3b3d1b2dc95b4f60b780f35f028d8f5b/users/df699280404e4d0bb0240a7054480fc2/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:29.903046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e4fe952102dd46de8d514821bdd0cad5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:29.906397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:29.937052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8419e-4667-4578-89aa-6ed8c6978ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3d1b2dc95b4f60b780f35f028d8f5b, user_id=df699280404e4d0bb0240a7054480fc2, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:29.940523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8419e-4667-4578-89aa-6ed8c6978ec4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:29.965315 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3180/6364] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:29 2022] PUT /identity/v3/projects/3b3d1b2dc95b4f60b780f35f028d8f5b/users/df699280404e4d0bb0240a7054480fc2/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:29.968597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8419e-4667-4578-89aa-6ed8c6978ec4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:29.975906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:30.114829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fe952102dd46de8d514821bdd0cad5', 'request_id': 'req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:30.130986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fe952102dd46de8d514821bdd0cad5', 'request_id': 'req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:30.139982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8419e-4667-4578-89aa-6ed8c6978ec4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:30.150567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fe952102dd46de8d514821bdd0cad5', 'request_id': 'req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:30.152898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:30.191980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] The token cache is being invalidate because project e4fe952102dd46de8d514821bdd0cad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:30.192273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:30.192977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:30.194917 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3185/6365] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:29 2022] DELETE /identity/v3/projects/e4fe952102dd46de8d514821bdd0cad5 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:30.251963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdde2a9-5d43-40b5-966a-61c615b515dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:30.252584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdde2a9-5d43-40b5-966a-61c615b515dd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:30.252859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdde2a9-5d43-40b5-966a-61c615b515dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:30.253159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdde2a9-5d43-40b5-966a-61c615b515dd admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:30.253809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdde2a9-5d43-40b5-966a-61c615b515dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:30.256148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdde2a9-5d43-40b5-966a-61c615b515dd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:30.294797 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3181/6366] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:30.295360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:30.297669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:30.316978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89bef-1ab5-45e0-a9c5-ec20e2463e82 tempest-VolumeQuotaClassesTest-1610336666 tempest-VolumeQuotaClassesTest-1610336666-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:30.323047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177790c4-d3c0-4838-b713-deae1f65c4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:30.323722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177790c4-d3c0-4838-b713-deae1f65c4de admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:30.324057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177790c4-d3c0-4838-b713-deae1f65c4de admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:30.324333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177790c4-d3c0-4838-b713-deae1f65c4de admin admin] PATH_INFO: `/v3/projects/3b3d1b2dc95b4f60b780f35f028d8f5b/users/df699280404e4d0bb0240a7054480fc2/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:30.327448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177790c4-d3c0-4838-b713-deae1f65c4de admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3d1b2dc95b4f60b780f35f028d8f5b, user_id=df699280404e4d0bb0240a7054480fc2, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:30.330446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177790c4-d3c0-4838-b713-deae1f65c4de admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:30.403105 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3186/6367] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:30 2022] PUT /identity/v3/projects/3b3d1b2dc95b4f60b780f35f028d8f5b/users/df699280404e4d0bb0240a7054480fc2/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:30.405722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4eccd-0587-42a1-8cd5-74e793eda905 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:30.406112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4eccd-0587-42a1-8cd5-74e793eda905 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:30.407415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4eccd-0587-42a1-8cd5-74e793eda905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:30.471371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-80d4eccd-0587-42a1-8cd5-74e793eda905 None None] MFA Rules not processed for user `df699280404e4d0bb0240a7054480fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:30.474696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80d4eccd-0587-42a1-8cd5-74e793eda905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:30.546620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e238011-905f-4fb6-8e98-b94c023efd4d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:30.546900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e238011-905f-4fb6-8e98-b94c023efd4d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:30.547317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e238011-905f-4fb6-8e98-b94c023efd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:30.615859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1e238011-905f-4fb6-8e98-b94c023efd4d None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:30.617221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e238011-905f-4fb6-8e98-b94c023efd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:30.724659 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3182/6368] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:30.725026 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3187/6369] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:15:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:30.740110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4eccd-0587-42a1-8cd5-74e793eda905 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:30.742983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80d4eccd-0587-42a1-8cd5-74e793eda905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:30.768472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e238011-905f-4fb6-8e98-b94c023efd4d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:30.773252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e238011-905f-4fb6-8e98-b94c023efd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:30.866483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4eccd-0587-42a1-8cd5-74e793eda905 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:30.874093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c03b0-1c21-4dec-9198-e611d9f20ce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:30.874919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c03b0-1c21-4dec-9198-e611d9f20ce9 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:30.875531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c03b0-1c21-4dec-9198-e611d9f20ce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:30.877208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c03b0-1c21-4dec-9198-e611d9f20ce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:30.900158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c03b0-1c21-4dec-9198-e611d9f20ce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:30.902134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c03b0-1c21-4dec-9198-e611d9f20ce9 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:30.911631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e238011-905f-4fb6-8e98-b94c023efd4d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:30.917383 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3183/6370] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:30.918473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c03b0-1c21-4dec-9198-e611d9f20ce9 service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:30.919992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2f1c03b0-1c21-4dec-9198-e611d9f20ce9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:30.922320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:30.923001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:30.923246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:30.923520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] PATH_INFO: `/v3/users/ad524efa33e64cba9ff7f8f56424f83b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:30.938762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad524efa33e64cba9ff7f8f56424f83b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:30.941503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:31.027693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:31.036781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:31.077237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad524efa33e64cba9ff7f8f56424f83b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:31.120297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad524efa33e64cba9ff7f8f56424f83b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:31.138360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c03b0-1c21-4dec-9198-e611d9f20ce9 service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:31.141405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad524efa33e64cba9ff7f8f56424f83b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:31.154799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099aa9f-04e6-419d-a433-e65792024562 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:31.155597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099aa9f-04e6-419d-a433-e65792024562 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:31.157775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad524efa33e64cba9ff7f8f56424f83b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:31.158225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099aa9f-04e6-419d-a433-e65792024562 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:31.158562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad524efa33e64cba9ff7f8f56424f83b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:31.160383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099aa9f-04e6-419d-a433-e65792024562 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:31.166380 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3188/6371] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:30 2022] DELETE /identity/v3/users/ad524efa33e64cba9ff7f8f56424f83b => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:31.168109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f099aa9f-04e6-419d-a433-e65792024562 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:31.170125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:31.178663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:31.227541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099aa9f-04e6-419d-a433-e65792024562 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:31.234376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099aa9f-04e6-419d-a433-e65792024562 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:31.277432 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3184/6372] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:15:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:15:31.281877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ffa5a-ef94-4c31-996f-f9263f6a0b79 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:31.283339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ffa5a-ef94-4c31-996f-f9263f6a0b79 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:31.289783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ffa5a-ef94-4c31-996f-f9263f6a0b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:31.343332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-203ffa5a-ef94-4c31-996f-f9263f6a0b79 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:31.345304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-203ffa5a-ef94-4c31-996f-f9263f6a0b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:31.400470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff1d9f-7911-465f-8b88-c26e70815028 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:31.411939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b500b2-d1dd-4a11-b188-95ea52ee56e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:31.414547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b500b2-d1dd-4a11-b188-95ea52ee56e3 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:31.415188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b500b2-d1dd-4a11-b188-95ea52ee56e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:31.415768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b500b2-d1dd-4a11-b188-95ea52ee56e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:31.439212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f0b500b2-d1dd-4a11-b188-95ea52ee56e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:31.459980 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3185/6373] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:31.497455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ffa5a-ef94-4c31-996f-f9263f6a0b79 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:31.499267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-203ffa5a-ef94-4c31-996f-f9263f6a0b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:31.603771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b500b2-d1dd-4a11-b188-95ea52ee56e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:31.607907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b500b2-d1dd-4a11-b188-95ea52ee56e3 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:31.637128 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3189/6374] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:31.705092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ffa5a-ef94-4c31-996f-f9263f6a0b79 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:31.720497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489b22d-1f14-439d-a8f8-86cbc935ab16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:31.721162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489b22d-1f14-439d-a8f8-86cbc935ab16 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:31.721444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489b22d-1f14-439d-a8f8-86cbc935ab16 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:31.721721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489b22d-1f14-439d-a8f8-86cbc935ab16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:31.752068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489b22d-1f14-439d-a8f8-86cbc935ab16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:31.753965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489b22d-1f14-439d-a8f8-86cbc935ab16 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:31.773007 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3186/6375] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:32.026356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b500b2-d1dd-4a11-b188-95ea52ee56e3 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:32.028958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f0b500b2-d1dd-4a11-b188-95ea52ee56e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:32.075368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b500b2-d1dd-4a11-b188-95ea52ee56e3 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:32.088912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc5906-8743-4007-bccf-013aec0a6d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:32.090324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc5906-8743-4007-bccf-013aec0a6d58 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:32.090850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc5906-8743-4007-bccf-013aec0a6d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:32.092352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc5906-8743-4007-bccf-013aec0a6d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:32.095341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-efcc5906-8743-4007-bccf-013aec0a6d58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:32.246591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9914e2-f902-40e3-8f9c-a5213c414ffa None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:32.247037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9914e2-f902-40e3-8f9c-a5213c414ffa None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:32.247212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9914e2-f902-40e3-8f9c-a5213c414ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:32.292104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ca9914e2-f902-40e3-8f9c-a5213c414ffa None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:32.293288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ca9914e2-f902-40e3-8f9c-a5213c414ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:32.293447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:32.322869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc5906-8743-4007-bccf-013aec0a6d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:32.324521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc5906-8743-4007-bccf-013aec0a6d58 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:32.355097 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3190/6376] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:32.428656 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3187/6377] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:15:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:32.449930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc5906-8743-4007-bccf-013aec0a6d58 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:32.451728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-efcc5906-8743-4007-bccf-013aec0a6d58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:32.577320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc5906-8743-4007-bccf-013aec0a6d58 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:32.585173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53823b2-2816-43a0-992f-8668c3937e48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:32.585994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53823b2-2816-43a0-992f-8668c3937e48 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:32.586452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53823b2-2816-43a0-992f-8668c3937e48 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:32.588397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53823b2-2816-43a0-992f-8668c3937e48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:32.599062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53823b2-2816-43a0-992f-8668c3937e48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:32.602006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53823b2-2816-43a0-992f-8668c3937e48 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:32.617365 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3191/6378] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:15:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:15:33.317341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9914e2-f902-40e3-8f9c-a5213c414ffa None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:33.318782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ca9914e2-f902-40e3-8f9c-a5213c414ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:33.345489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9914e2-f902-40e3-8f9c-a5213c414ffa None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:33.358314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:33.359290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:33.361941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:33.362326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] PATH_INFO: `/v3/projects/2ed54e51ee9641acbe3f8f242497b1dd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:33.364821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed54e51ee9641acbe3f8f242497b1dd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:33.369958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:33.405200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:33.494718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed54e51ee9641acbe3f8f242497b1dd', 'request_id': 'req-520bacc1-91af-42b0-a41f-1dc3aeb388a2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:33.528957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed54e51ee9641acbe3f8f242497b1dd', 'request_id': 'req-520bacc1-91af-42b0-a41f-1dc3aeb388a2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:33.546432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed54e51ee9641acbe3f8f242497b1dd', 'request_id': 'req-520bacc1-91af-42b0-a41f-1dc3aeb388a2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:33.582408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] The token cache is being invalidate because project 2ed54e51ee9641acbe3f8f242497b1dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:33.583611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:33.584594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-520bacc1-91af-42b0-a41f-1dc3aeb388a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:33.591851 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3188/6379] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:33 2022] DELETE /identity/v3/projects/2ed54e51ee9641acbe3f8f242497b1dd => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:33.592550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53823b2-2816-43a0-992f-8668c3937e48 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:33.609289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53823b2-2816-43a0-992f-8668c3937e48 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:33.702810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:33.704526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:33.705198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:33.706019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] PATH_INFO: `/v3/users/10e3d73a197346a88716a7342c9c4716` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:33.708536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e3d73a197346a88716a7342c9c4716)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:33.710955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:33.823843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cd630-c861-477d-95b6-943f5f04389b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:33.824101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cd630-c861-477d-95b6-943f5f04389b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:33.824579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cd630-c861-477d-95b6-943f5f04389b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:33.875847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e3d73a197346a88716a7342c9c4716'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:33.877115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-454cd630-c861-477d-95b6-943f5f04389b None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:33.878354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-454cd630-c861-477d-95b6-943f5f04389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:33.912789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e3d73a197346a88716a7342c9c4716'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:33.932766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e3d73a197346a88716a7342c9c4716'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:33.942669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e3d73a197346a88716a7342c9c4716'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:33.944060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e3d73a197346a88716a7342c9c4716'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:33.946812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:33.958273 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3192/6380] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:33 2022] DELETE /identity/v3/users/10e3d73a197346a88716a7342c9c4716 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:33.959337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:33.968102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:34.050278 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3189/6381] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:34.075628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cd630-c861-477d-95b6-943f5f04389b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:34.076802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-454cd630-c861-477d-95b6-943f5f04389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:34.146497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfa9f6-5127-48d0-8a3f-fad31965a3ba admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:34.161327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e330a-6398-4f2c-b8ab-e8614feb32ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:34.162299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e330a-6398-4f2c-b8ab-e8614feb32ee service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:34.162830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e330a-6398-4f2c-b8ab-e8614feb32ee service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:34.162830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e330a-6398-4f2c-b8ab-e8614feb32ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:34.164649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-523e330a-6398-4f2c-b8ab-e8614feb32ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:34.214496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e330a-6398-4f2c-b8ab-e8614feb32ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:34.216024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e330a-6398-4f2c-b8ab-e8614feb32ee service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:34.240045 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3193/6382] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:34.255075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cd630-c861-477d-95b6-943f5f04389b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:34.365804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:34.415965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354dc2b-a6ac-4b7b-9a4b-d55ed7289c49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:34.416888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354dc2b-a6ac-4b7b-9a4b-d55ed7289c49 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:34.417216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354dc2b-a6ac-4b7b-9a4b-d55ed7289c49 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:34.417994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354dc2b-a6ac-4b7b-9a4b-d55ed7289c49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:34.551435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354dc2b-a6ac-4b7b-9a4b-d55ed7289c49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:34.553260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354dc2b-a6ac-4b7b-9a4b-d55ed7289c49 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:34.573843 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3190/6383] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:15:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:15:35.025642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facda69-e2f8-4c3c-84e9-8580962e5fc7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:35.026576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facda69-e2f8-4c3c-84e9-8580962e5fc7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:35.027166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facda69-e2f8-4c3c-84e9-8580962e5fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:35.053089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3facda69-e2f8-4c3c-84e9-8580962e5fc7 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:35.054583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3facda69-e2f8-4c3c-84e9-8580962e5fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:35.064533 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3194/6384] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:35.080956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354dc2b-a6ac-4b7b-9a4b-d55ed7289c49 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:35.097489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354dc2b-a6ac-4b7b-9a4b-d55ed7289c49 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:35.105613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f312a51-c8b2-4215-9231-b1cc539e8461 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:35.107442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f312a51-c8b2-4215-9231-b1cc539e8461 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:35.108014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f312a51-c8b2-4215-9231-b1cc539e8461 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:35.108510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f312a51-c8b2-4215-9231-b1cc539e8461 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:35.126741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f312a51-c8b2-4215-9231-b1cc539e8461 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:35.130165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f312a51-c8b2-4215-9231-b1cc539e8461 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:35.157865 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3191/6385] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:15:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:15:35.415957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facda69-e2f8-4c3c-84e9-8580962e5fc7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:35.423950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3facda69-e2f8-4c3c-84e9-8580962e5fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:35.620182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facda69-e2f8-4c3c-84e9-8580962e5fc7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:35.627247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:35.628166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:35.628913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:35.629396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] PATH_INFO: `/v3/projects/ca9fec694ac44940b718f7f7bcf80864` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:35.631432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9fec694ac44940b718f7f7bcf80864)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:35.634548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:35.761548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9fec694ac44940b718f7f7bcf80864', 'request_id': 'req-2d8b3ffa-7b07-4716-858a-f64fa12152b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:35.794535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9fec694ac44940b718f7f7bcf80864', 'request_id': 'req-2d8b3ffa-7b07-4716-858a-f64fa12152b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:35.809364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9fec694ac44940b718f7f7bcf80864', 'request_id': 'req-2d8b3ffa-7b07-4716-858a-f64fa12152b3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:35.825593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] The token cache is being invalidate because project ca9fec694ac44940b718f7f7bcf80864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:35.826400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:35.827546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d8b3ffa-7b07-4716-858a-f64fa12152b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:35.836066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c988d-427e-40ce-9e8e-8b8f3ecc304a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:35.837446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c988d-427e-40ce-9e8e-8b8f3ecc304a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:35.837919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c988d-427e-40ce-9e8e-8b8f3ecc304a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:35.842514 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3195/6386] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:35 2022] DELETE /identity/v3/projects/ca9fec694ac44940b718f7f7bcf80864 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:35.874612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-500c988d-427e-40ce-9e8e-8b8f3ecc304a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:35.875839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-500c988d-427e-40ce-9e8e-8b8f3ecc304a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:35.896162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77cdda-406b-4a90-b777-7b4b8aea3f1f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:35.896451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77cdda-406b-4a90-b777-7b4b8aea3f1f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:35.896728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77cdda-406b-4a90-b777-7b4b8aea3f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:35.956554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8a77cdda-406b-4a90-b777-7b4b8aea3f1f None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:35.962812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a77cdda-406b-4a90-b777-7b4b8aea3f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:35.975757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:36.017216 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3192/6387] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:36.025455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a1a30-b748-4e9a-ad5f-5a05d5a447ae None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:36.026413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a1a30-b748-4e9a-ad5f-5a05d5a447ae None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:36.026835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a1a30-b748-4e9a-ad5f-5a05d5a447ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:36.059585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1d0a1a30-b748-4e9a-ad5f-5a05d5a447ae None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:36.062266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1d0a1a30-b748-4e9a-ad5f-5a05d5a447ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:36.076690 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3193/6388] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:36.080006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a1a30-b748-4e9a-ad5f-5a05d5a447ae None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:36.094960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a1a30-b748-4e9a-ad5f-5a05d5a447ae None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:36.101483 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3196/6389] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:36.103431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a3587-bbcf-4d2b-93c5-c91e8ac9b8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:36.104439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a3587-bbcf-4d2b-93c5-c91e8ac9b8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:36.108804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a3587-bbcf-4d2b-93c5-c91e8ac9b8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:36.109502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a3587-bbcf-4d2b-93c5-c91e8ac9b8d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:36.110380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a3587-bbcf-4d2b-93c5-c91e8ac9b8d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:36.116584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a3587-bbcf-4d2b-93c5-c91e8ac9b8d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:36.117517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77cdda-406b-4a90-b777-7b4b8aea3f1f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:36.119320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a77cdda-406b-4a90-b777-7b4b8aea3f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:36.120679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a3587-bbcf-4d2b-93c5-c91e8ac9b8d7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:36.184257 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3194/6390] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:36.187127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a3587-bbcf-4d2b-93c5-c91e8ac9b8d7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:36.232282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a3587-bbcf-4d2b-93c5-c91e8ac9b8d7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:36.244517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a3049-9166-44dc-92de-8e7db51b71bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:36.246722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a3049-9166-44dc-92de-8e7db51b71bc admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:36.247147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a3049-9166-44dc-92de-8e7db51b71bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:36.247801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a3049-9166-44dc-92de-8e7db51b71bc admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:36.250706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a3049-9166-44dc-92de-8e7db51b71bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:36.255999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a3049-9166-44dc-92de-8e7db51b71bc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:36.297447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77cdda-406b-4a90-b777-7b4b8aea3f1f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:36.312203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6aadb-8aa9-4d27-934d-5980bf2ee318 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:36.312830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6aadb-8aa9-4d27-934d-5980bf2ee318 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:36.313165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6aadb-8aa9-4d27-934d-5980bf2ee318 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:36.313440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6aadb-8aa9-4d27-934d-5980bf2ee318 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:36.323225 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3195/6391] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:36 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:36.328403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a3049-9166-44dc-92de-8e7db51b71bc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:36.333382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6aadb-8aa9-4d27-934d-5980bf2ee318 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:36.334998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6aadb-8aa9-4d27-934d-5980bf2ee318 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:36.367176 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3197/6392] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:15:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:15:36.470641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:36.571076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a3049-9166-44dc-92de-8e7db51b71bc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:36.587294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf949a4c-98fe-43e8-a875-2f3bee0a4496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:36.587789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf949a4c-98fe-43e8-a875-2f3bee0a4496 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:36.588062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf949a4c-98fe-43e8-a875-2f3bee0a4496 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:36.588327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf949a4c-98fe-43e8-a875-2f3bee0a4496 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:36.589999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf949a4c-98fe-43e8-a875-2f3bee0a4496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:36.596221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf949a4c-98fe-43e8-a875-2f3bee0a4496 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:36.655318 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3196/6393] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:36 2022] POST /identity/v3/users => generated 290 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:36.660007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6aadb-8aa9-4d27-934d-5980bf2ee318 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:36.692224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6aadb-8aa9-4d27-934d-5980bf2ee318 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:36.707140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bfb5c-f1be-432b-ad98-265a975e62f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:36.709639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bfb5c-f1be-432b-ad98-265a975e62f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:36.712162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bfb5c-f1be-432b-ad98-265a975e62f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:36.712662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bfb5c-f1be-432b-ad98-265a975e62f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:36.713471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bfb5c-f1be-432b-ad98-265a975e62f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:36.722822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bfb5c-f1be-432b-ad98-265a975e62f0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:36.769142 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3198/6394] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:36.772948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf949a4c-98fe-43e8-a875-2f3bee0a4496 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:36.787902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf949a4c-98fe-43e8-a875-2f3bee0a4496 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:36.811907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c895265-498b-4841-a9a0-13e8b4d39d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:36.813272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c895265-498b-4841-a9a0-13e8b4d39d92 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:36.813593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c895265-498b-4841-a9a0-13e8b4d39d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:36.813767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c895265-498b-4841-a9a0-13e8b4d39d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:36.815020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c895265-498b-4841-a9a0-13e8b4d39d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:36.816893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c895265-498b-4841-a9a0-13e8b4d39d92 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:36.830973 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3197/6395] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:36.834130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bfb5c-f1be-432b-ad98-265a975e62f0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:36.851400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bfb5c-f1be-432b-ad98-265a975e62f0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:36.867474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70172b-c94a-4d6c-ad6f-009bcca87e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:36.868729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70172b-c94a-4d6c-ad6f-009bcca87e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:36.869193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70172b-c94a-4d6c-ad6f-009bcca87e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:36.869613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70172b-c94a-4d6c-ad6f-009bcca87e9e admin admin] PATH_INFO: `/v3/projects/a872801e32fb48c388b7a696b1b6a485/users/47f790a3672b449d81e96842483481ee/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:36.896930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70172b-c94a-4d6c-ad6f-009bcca87e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a872801e32fb48c388b7a696b1b6a485, user_id=47f790a3672b449d81e96842483481ee, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:36.902306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70172b-c94a-4d6c-ad6f-009bcca87e9e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:36.954148 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3199/6396] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:36 2022] PUT /identity/v3/projects/a872801e32fb48c388b7a696b1b6a485/users/47f790a3672b449d81e96842483481ee/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:36.971976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653df2f-cb87-4dd7-8e0e-0963a2e070ba None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:36.972540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653df2f-cb87-4dd7-8e0e-0963a2e070ba None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:36.973010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653df2f-cb87-4dd7-8e0e-0963a2e070ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:37.037988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6653df2f-cb87-4dd7-8e0e-0963a2e070ba None None] MFA Rules not processed for user `47f790a3672b449d81e96842483481ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:37.042966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6653df2f-cb87-4dd7-8e0e-0963a2e070ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:37.206221 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3198/6397] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:37.218832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70172b-c94a-4d6c-ad6f-009bcca87e9e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:37.221317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7a70172b-c94a-4d6c-ad6f-009bcca87e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:37.342578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70172b-c94a-4d6c-ad6f-009bcca87e9e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:37.347334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413851c-358f-4c19-8209-ef20bc2a9197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:37.348110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413851c-358f-4c19-8209-ef20bc2a9197 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:37.348532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413851c-358f-4c19-8209-ef20bc2a9197 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:37.348944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413851c-358f-4c19-8209-ef20bc2a9197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:37.362011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413851c-358f-4c19-8209-ef20bc2a9197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:37.363627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413851c-358f-4c19-8209-ef20bc2a9197 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:37.382379 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3200/6398] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:41.063681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653df2f-cb87-4dd7-8e0e-0963a2e070ba None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:41.065162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6653df2f-cb87-4dd7-8e0e-0963a2e070ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:41.191517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653df2f-cb87-4dd7-8e0e-0963a2e070ba None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:41.203948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:41.207128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:41.207410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:41.207677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] PATH_INFO: `/v3/users/1d86728459fe4fefabd2555de8608ddc` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:41.209309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d86728459fe4fefabd2555de8608ddc)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:41.211528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:41.300375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:41.304576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d86728459fe4fefabd2555de8608ddc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:41.323623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d86728459fe4fefabd2555de8608ddc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:41.345379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d86728459fe4fefabd2555de8608ddc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:41.375621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d86728459fe4fefabd2555de8608ddc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:41.382789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d86728459fe4fefabd2555de8608ddc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:41.392906 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3199/6399] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:41 2022] DELETE /identity/v3/users/1d86728459fe4fefabd2555de8608ddc => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:41.397280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29ba2c-6004-482a-b125-9c6d99cfe0e6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:41.397754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29ba2c-6004-482a-b125-9c6d99cfe0e6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:41.397853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29ba2c-6004-482a-b125-9c6d99cfe0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:41.436982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bf29ba2c-6004-482a-b125-9c6d99cfe0e6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:41.438577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bf29ba2c-6004-482a-b125-9c6d99cfe0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:41.451701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:41.529853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:41.531482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:41.596931 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3201/6400] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:41.641421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29ba2c-6004-482a-b125-9c6d99cfe0e6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:41.643364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bf29ba2c-6004-482a-b125-9c6d99cfe0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:41.766625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537efc35-52dd-4b90-a33b-482e4eb6fcfe admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:41.795906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29ba2c-6004-482a-b125-9c6d99cfe0e6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:41.805737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbea38c-48d2-4e54-b707-3edfe056c914 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:41.805737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbea38c-48d2-4e54-b707-3edfe056c914 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:41.805737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbea38c-48d2-4e54-b707-3edfe056c914 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:41.805737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbea38c-48d2-4e54-b707-3edfe056c914 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:41.808968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7dbea38c-48d2-4e54-b707-3edfe056c914 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:41.809705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33b73f-d00f-49f2-80d5-f3675fbaa2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:41.810555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33b73f-d00f-49f2-80d5-f3675fbaa2eb service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:41.811018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33b73f-d00f-49f2-80d5-f3675fbaa2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:41.811488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33b73f-d00f-49f2-80d5-f3675fbaa2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:41.822846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33b73f-d00f-49f2-80d5-f3675fbaa2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:41.824839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33b73f-d00f-49f2-80d5-f3675fbaa2eb service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:41.843193 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3202/6401] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:42.092666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbea38c-48d2-4e54-b707-3edfe056c914 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:42.094716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbea38c-48d2-4e54-b707-3edfe056c914 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:42.173697 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3200/6402] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:15:42.884963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33b73f-d00f-49f2-80d5-f3675fbaa2eb service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:42.888815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3c33b73f-d00f-49f2-80d5-f3675fbaa2eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:42.914971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33b73f-d00f-49f2-80d5-f3675fbaa2eb service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:42.923937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:42.924817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:42.925273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:42.925683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] PATH_INFO: `/v3/projects/b0007ee63f7340f68f40f6caf5054654` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:42.930277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0007ee63f7340f68f40f6caf5054654)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:42.933152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:43.044569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0007ee63f7340f68f40f6caf5054654', 'request_id': 'req-3bcc18f8-4665-4417-92e9-dd5fe56a6507'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:43.060828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0007ee63f7340f68f40f6caf5054654', 'request_id': 'req-3bcc18f8-4665-4417-92e9-dd5fe56a6507'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:43.072645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0007ee63f7340f68f40f6caf5054654', 'request_id': 'req-3bcc18f8-4665-4417-92e9-dd5fe56a6507'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:43.122798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] The token cache is being invalidate because project b0007ee63f7340f68f40f6caf5054654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:43.123949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:43.128660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3bcc18f8-4665-4417-92e9-dd5fe56a6507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:43.138264 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3203/6403] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:42 2022] DELETE /identity/v3/projects/b0007ee63f7340f68f40f6caf5054654 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:43.141478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cc650-254f-48dd-92a3-1733853e4918 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:43.142517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cc650-254f-48dd-92a3-1733853e4918 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:43.142997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cc650-254f-48dd-92a3-1733853e4918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:43.182518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d44cc650-254f-48dd-92a3-1733853e4918 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:43.184524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d44cc650-254f-48dd-92a3-1733853e4918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:43.246379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:43.318313 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3201/6404] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:43.327541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0c33d-1889-4ac5-b452-6211a9718219 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:43.327967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0c33d-1889-4ac5-b452-6211a9718219 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:43.328100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0c33d-1889-4ac5-b452-6211a9718219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:43.372150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-08a0c33d-1889-4ac5-b452-6211a9718219 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:43.373339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-08a0c33d-1889-4ac5-b452-6211a9718219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:43.395986 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3204/6405] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:43.400514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cc650-254f-48dd-92a3-1733853e4918 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:43.418267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cc650-254f-48dd-92a3-1733853e4918 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:43.423268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2ce69-1565-4512-9695-9bce790203f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:43.423885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2ce69-1565-4512-9695-9bce790203f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:43.424162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2ce69-1565-4512-9695-9bce790203f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:43.424437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2ce69-1565-4512-9695-9bce790203f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:43.425114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2ce69-1565-4512-9695-9bce790203f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:43.425420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2ce69-1565-4512-9695-9bce790203f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:43.427434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2ce69-1565-4512-9695-9bce790203f2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:43.446479 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3202/6406] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:43.450962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0c33d-1889-4ac5-b452-6211a9718219 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:43.463948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0c33d-1889-4ac5-b452-6211a9718219 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:43.465339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2ce69-1565-4512-9695-9bce790203f2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:43.467490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-87e2ce69-1565-4512-9695-9bce790203f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:43.484933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2ce69-1565-4512-9695-9bce790203f2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:43.490040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed23ea6-60c9-4923-b2e4-5945d9b31efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:43.490805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed23ea6-60c9-4923-b2e4-5945d9b31efd admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:43.491155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed23ea6-60c9-4923-b2e4-5945d9b31efd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:43.491603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed23ea6-60c9-4923-b2e4-5945d9b31efd admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:43.493875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed23ea6-60c9-4923-b2e4-5945d9b31efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:43.494670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:43.495554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:43.496003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:43.496468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] PATH_INFO: `/v3/users/701f38823f864ec9ba7d70e15d9664ef` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:43.496582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed23ea6-60c9-4923-b2e4-5945d9b31efd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:43.498809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] RBAC: Authorizing `identity:delete_user(user_id=701f38823f864ec9ba7d70e15d9664ef)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:43.501356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:43.577528 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3205/6407] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:43 2022] POST /identity/v3/projects => generated 391 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:43.581513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed23ea6-60c9-4923-b2e4-5945d9b31efd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:43.609188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:43.650864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '701f38823f864ec9ba7d70e15d9664ef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:43.708011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '701f38823f864ec9ba7d70e15d9664ef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:43.742879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '701f38823f864ec9ba7d70e15d9664ef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:43.755436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '701f38823f864ec9ba7d70e15d9664ef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:43.756535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b3e49822-569a-4547-927a-75b2a4a2573e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '701f38823f864ec9ba7d70e15d9664ef'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:43.770311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed23ea6-60c9-4923-b2e4-5945d9b31efd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:43.779566 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3203/6408] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:43 2022] DELETE /identity/v3/users/701f38823f864ec9ba7d70e15d9664ef => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:43.781621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c4ab3-e84b-4fda-9585-648b23b5bb17 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:43.782114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c4ab3-e84b-4fda-9585-648b23b5bb17 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:43.782613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c4ab3-e84b-4fda-9585-648b23b5bb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:43.814414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-dc5c4ab3-e84b-4fda-9585-648b23b5bb17 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:43.816008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dc5c4ab3-e84b-4fda-9585-648b23b5bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:43.943357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978acbd-829f-46f4-9edc-03e48672a2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:43.944673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978acbd-829f-46f4-9edc-03e48672a2ea admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:43.945173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978acbd-829f-46f4-9edc-03e48672a2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:43.945712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978acbd-829f-46f4-9edc-03e48672a2ea admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:43.946697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978acbd-829f-46f4-9edc-03e48672a2ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:43.951574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978acbd-829f-46f4-9edc-03e48672a2ea admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:43.998414 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3206/6409] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:43 2022] POST /identity/v3/users => generated 296 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:43.999826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978acbd-829f-46f4-9edc-03e48672a2ea admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:44.004727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7978acbd-829f-46f4-9edc-03e48672a2ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:44.028214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978acbd-829f-46f4-9edc-03e48672a2ea admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:44.036199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:44.037784 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3204/6410] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:44.038597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c4ab3-e84b-4fda-9585-648b23b5bb17 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:44.040117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:44.041139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:44.041557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] PATH_INFO: `/v3/users/95ceb5467635488ca6c249ca155942a0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:44.041725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dc5c4ab3-e84b-4fda-9585-648b23b5bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:44.049810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ceb5467635488ca6c249ca155942a0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:44.052087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:44.059964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c4ab3-e84b-4fda-9585-648b23b5bb17 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:44.067618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46d26b-79ee-495e-88e1-2de573359b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:44.068391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46d26b-79ee-495e-88e1-2de573359b5f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:44.068860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46d26b-79ee-495e-88e1-2de573359b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:44.069227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46d26b-79ee-495e-88e1-2de573359b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:44.069999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46d26b-79ee-495e-88e1-2de573359b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:44.079331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46d26b-79ee-495e-88e1-2de573359b5f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:44.093988 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3205/6411] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:44.094920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46d26b-79ee-495e-88e1-2de573359b5f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:44.098273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9b46d26b-79ee-495e-88e1-2de573359b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:44.132388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:44.159973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ceb5467635488ca6c249ca155942a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:44.183070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ceb5467635488ca6c249ca155942a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:44.195113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ceb5467635488ca6c249ca155942a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:44.209361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46d26b-79ee-495e-88e1-2de573359b5f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:44.212900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ceb5467635488ca6c249ca155942a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:44.213893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ceb5467635488ca6c249ca155942a0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:44.219897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e35c8e-1a8e-4cf5-9bbf-6a7317ea8a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:44.220639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e35c8e-1a8e-4cf5-9bbf-6a7317ea8a3d service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:44.221209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e35c8e-1a8e-4cf5-9bbf-6a7317ea8a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:44.221703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e35c8e-1a8e-4cf5-9bbf-6a7317ea8a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:44.224486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b1e35c8e-1a8e-4cf5-9bbf-6a7317ea8a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:44.231020 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3207/6412] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:43 2022] DELETE /identity/v3/users/95ceb5467635488ca6c249ca155942a0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:44.233649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:44.240244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:44.256342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e35c8e-1a8e-4cf5-9bbf-6a7317ea8a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:44.259153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e35c8e-1a8e-4cf5-9bbf-6a7317ea8a3d service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:44.271615 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3206/6413] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:44.272762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e35c8e-1a8e-4cf5-9bbf-6a7317ea8a3d service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:44.275537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b1e35c8e-1a8e-4cf5-9bbf-6a7317ea8a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:44.291669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e35c8e-1a8e-4cf5-9bbf-6a7317ea8a3d service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:44.292469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:44.397370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fc396-b0c0-4708-99eb-a1736b998446 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:44.409442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b6951-75b5-467f-823d-d3643f0ed161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:44.410103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b6951-75b5-467f-823d-d3643f0ed161 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:44.410379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b6951-75b5-467f-823d-d3643f0ed161 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:44.410769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b6951-75b5-467f-823d-d3643f0ed161 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:44.411294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b6951-75b5-467f-823d-d3643f0ed161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:44.424415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b6951-75b5-467f-823d-d3643f0ed161 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:44.425252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:44.426202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:44.427216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:44.428336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] PATH_INFO: `/v3/users/5b78c768bf14494f89baa259e135ecc9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:44.432233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b78c768bf14494f89baa259e135ecc9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:44.438270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:44.454924 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3208/6414] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:44.459833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9797f5-a7fb-4c2c-98c2-309a8af7d6a8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:44.460251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9797f5-a7fb-4c2c-98c2-309a8af7d6a8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:44.460535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9797f5-a7fb-4c2c-98c2-309a8af7d6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:44.515655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1e9797f5-a7fb-4c2c-98c2-309a8af7d6a8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:44.517016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e9797f5-a7fb-4c2c-98c2-309a8af7d6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:44.535378 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3209/6415] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:44.542767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9797f5-a7fb-4c2c-98c2-309a8af7d6a8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:44.562239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b78c768bf14494f89baa259e135ecc9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:44.580697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9797f5-a7fb-4c2c-98c2-309a8af7d6a8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:44.591801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b78c768bf14494f89baa259e135ecc9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:44.595784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb2bd2-6a39-4d60-b02c-7141c2df8185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:44.596520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb2bd2-6a39-4d60-b02c-7141c2df8185 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:44.596836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb2bd2-6a39-4d60-b02c-7141c2df8185 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:44.598314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb2bd2-6a39-4d60-b02c-7141c2df8185 admin admin] PATH_INFO: `/v3/projects/662bfb99c7c6456eb782cdc2d74c95a4/users/d549b5c857bb4563a7c192bb75e678c4/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:44.635722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b78c768bf14494f89baa259e135ecc9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:44.645818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb2bd2-6a39-4d60-b02c-7141c2df8185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=662bfb99c7c6456eb782cdc2d74c95a4, user_id=d549b5c857bb4563a7c192bb75e678c4, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:44.651197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b78c768bf14494f89baa259e135ecc9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:44.651741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb2bd2-6a39-4d60-b02c-7141c2df8185 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:44.652582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b78c768bf14494f89baa259e135ecc9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:44.664821 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3207/6416] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:44 2022] DELETE /identity/v3/users/5b78c768bf14494f89baa259e135ecc9 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:44.667895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:44.672102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:44.702919 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3210/6417] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:44 2022] PUT /identity/v3/projects/662bfb99c7c6456eb782cdc2d74c95a4/users/d549b5c857bb4563a7c192bb75e678c4/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:44.704491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dce108-f404-4103-9409-f46b1724eb88 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:44.704776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dce108-f404-4103-9409-f46b1724eb88 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:44.705063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dce108-f404-4103-9409-f46b1724eb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:44.766167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a2dce108-f404-4103-9409-f46b1724eb88 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:44.770136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a2dce108-f404-4103-9409-f46b1724eb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:44.811879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:44.868662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd19d52-8968-469e-93c3-92bee061ba42 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:44.935590 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3211/6418] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:44.947984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd5cd0-85ab-4512-b014-7f98444051a5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:44.948928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd5cd0-85ab-4512-b014-7f98444051a5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:44.949820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd5cd0-85ab-4512-b014-7f98444051a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:45.007746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb492a-7469-4ea6-ab41-8094284dd569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:45.009899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb492a-7469-4ea6-ab41-8094284dd569 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:45.010361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb492a-7469-4ea6-ab41-8094284dd569 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:45.010795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9cfd5cd0-85ab-4512-b014-7f98444051a5 None None] MFA Rules not processed for user `d549b5c857bb4563a7c192bb75e678c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:45.012299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9cfd5cd0-85ab-4512-b014-7f98444051a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:45.014092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb492a-7469-4ea6-ab41-8094284dd569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:45.015998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4ceb492a-7469-4ea6-ab41-8094284dd569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:45.040899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb492a-7469-4ea6-ab41-8094284dd569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:45.042399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb492a-7469-4ea6-ab41-8094284dd569 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:45.069841 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3208/6419] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:45.071290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb492a-7469-4ea6-ab41-8094284dd569 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:45.091274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb492a-7469-4ea6-ab41-8094284dd569 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:45.103154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f017-1f54-408b-acbd-6a91a429b5aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:45.103836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f017-1f54-408b-acbd-6a91a429b5aa service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:45.104111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f017-1f54-408b-acbd-6a91a429b5aa service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:45.104389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f017-1f54-408b-acbd-6a91a429b5aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:45.129588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55f017-1f54-408b-acbd-6a91a429b5aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:45.134391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55f017-1f54-408b-acbd-6a91a429b5aa service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:45.161089 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3209/6420] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:45.165440 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3212/6421] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:45.177343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f017-1f54-408b-acbd-6a91a429b5aa service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:45.178910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4c55f017-1f54-408b-acbd-6a91a429b5aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:45.332936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f017-1f54-408b-acbd-6a91a429b5aa service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:45.341894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4067a6d-f928-4b66-b453-76b30a779576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:45.342633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4067a6d-f928-4b66-b453-76b30a779576 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:45.343482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4067a6d-f928-4b66-b453-76b30a779576 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:45.343808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4067a6d-f928-4b66-b453-76b30a779576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:45.365082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4067a6d-f928-4b66-b453-76b30a779576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:45.369489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4067a6d-f928-4b66-b453-76b30a779576 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:45.396390 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3210/6422] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:45.636375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd5cd0-85ab-4512-b014-7f98444051a5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:45.638928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9cfd5cd0-85ab-4512-b014-7f98444051a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:45.667768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd5cd0-85ab-4512-b014-7f98444051a5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:45.681767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:45.682422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:45.682696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:45.682965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] PATH_INFO: `/v3/projects/bca1e42b396e43f1af93381ad28d1362` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:45.687371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca1e42b396e43f1af93381ad28d1362)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:45.689354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:45.721667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:45.796544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca1e42b396e43f1af93381ad28d1362', 'request_id': 'req-aa7b09c6-355a-480f-a221-30f9d804d553'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:45.807092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca1e42b396e43f1af93381ad28d1362', 'request_id': 'req-aa7b09c6-355a-480f-a221-30f9d804d553'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:45.831033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca1e42b396e43f1af93381ad28d1362', 'request_id': 'req-aa7b09c6-355a-480f-a221-30f9d804d553'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:45.849629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] The token cache is being invalidate because project bca1e42b396e43f1af93381ad28d1362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:45.858384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:45.858384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa7b09c6-355a-480f-a221-30f9d804d553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:45.858384 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3213/6423] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:45 2022] DELETE /identity/v3/projects/bca1e42b396e43f1af93381ad28d1362 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:45.859850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6623-17e0-4092-b528-246d733ab6a8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:45.860133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6623-17e0-4092-b528-246d733ab6a8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:45.860664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6623-17e0-4092-b528-246d733ab6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:45.899357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-29bc6623-17e0-4092-b528-246d733ab6a8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:45.900607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-29bc6623-17e0-4092-b528-246d733ab6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:46.016678 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3211/6424] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:46.025925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ad98-0c3f-435c-9554-f0c5503d0b4f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:46.026424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ad98-0c3f-435c-9554-f0c5503d0b4f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:46.028553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ad98-0c3f-435c-9554-f0c5503d0b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:46.084878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2f53ad98-0c3f-435c-9554-f0c5503d0b4f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:46.088845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2f53ad98-0c3f-435c-9554-f0c5503d0b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:46.110654 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3214/6425] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:46.114556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6623-17e0-4092-b528-246d733ab6a8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:46.139340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6623-17e0-4092-b528-246d733ab6a8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:46.178052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adfc5d-7457-4222-9b7b-1106523f9c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:46.178742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adfc5d-7457-4222-9b7b-1106523f9c95 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:46.179216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adfc5d-7457-4222-9b7b-1106523f9c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:46.179630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adfc5d-7457-4222-9b7b-1106523f9c95 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:46.180920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adfc5d-7457-4222-9b7b-1106523f9c95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:46.181310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adfc5d-7457-4222-9b7b-1106523f9c95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:46.183070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adfc5d-7457-4222-9b7b-1106523f9c95 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:46.216422 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3212/6426] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:46.218665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53ad98-0c3f-435c-9554-f0c5503d0b4f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:46.233593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53ad98-0c3f-435c-9554-f0c5503d0b4f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:46.243890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841805d-3686-4d31-83b9-76d3e34820c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:46.244833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841805d-3686-4d31-83b9-76d3e34820c6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:46.245521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841805d-3686-4d31-83b9-76d3e34820c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:46.246076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841805d-3686-4d31-83b9-76d3e34820c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:46.247147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841805d-3686-4d31-83b9-76d3e34820c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:46.251034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841805d-3686-4d31-83b9-76d3e34820c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:46.332980 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3215/6427] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:46 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:46.338699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adfc5d-7457-4222-9b7b-1106523f9c95 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:46.352009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adfc5d-7457-4222-9b7b-1106523f9c95 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:46.368199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:46.417271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841805d-3686-4d31-83b9-76d3e34820c6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:46.419318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2841805d-3686-4d31-83b9-76d3e34820c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:46.473809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca279d-6e87-4f3f-a517-a5cdc8e69e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:46.474548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca279d-6e87-4f3f-a517-a5cdc8e69e17 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:46.474822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca279d-6e87-4f3f-a517-a5cdc8e69e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:46.475089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca279d-6e87-4f3f-a517-a5cdc8e69e17 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:46.475809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca279d-6e87-4f3f-a517-a5cdc8e69e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:46.479140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca279d-6e87-4f3f-a517-a5cdc8e69e17 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:46.507762 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3213/6428] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:46 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:46.512295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca279d-6e87-4f3f-a517-a5cdc8e69e17 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:46.518182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841805d-3686-4d31-83b9-76d3e34820c6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:46.524366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:46.524982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:46.525954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:46.525954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] PATH_INFO: `/v3/projects/870d73d996244e4199f4ba8033056277` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:46.527168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=870d73d996244e4199f4ba8033056277)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:46.529226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:46.533656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca279d-6e87-4f3f-a517-a5cdc8e69e17 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:46.540254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b60ca4-76d6-4c04-9073-444440ac6c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:46.540866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b60ca4-76d6-4c04-9073-444440ac6c4b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:46.541186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b60ca4-76d6-4c04-9073-444440ac6c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:46.541435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b60ca4-76d6-4c04-9073-444440ac6c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:46.542678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b60ca4-76d6-4c04-9073-444440ac6c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:46.544298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b60ca4-76d6-4c04-9073-444440ac6c4b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:46.561552 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3214/6429] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:46.563239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b60ca4-76d6-4c04-9073-444440ac6c4b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:46.577536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b60ca4-76d6-4c04-9073-444440ac6c4b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:46.618675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '870d73d996244e4199f4ba8033056277', 'request_id': 'req-4fedd176-b786-4188-9046-e3ac31b1ae2e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:46.630914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '870d73d996244e4199f4ba8033056277', 'request_id': 'req-4fedd176-b786-4188-9046-e3ac31b1ae2e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:46.639211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '870d73d996244e4199f4ba8033056277', 'request_id': 'req-4fedd176-b786-4188-9046-e3ac31b1ae2e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:46.665251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:46.674423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] The token cache is being invalidate because project 870d73d996244e4199f4ba8033056277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:46.674729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:46.677675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4fedd176-b786-4188-9046-e3ac31b1ae2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:46.683548 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3216/6430] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:46 2022] DELETE /identity/v3/projects/870d73d996244e4199f4ba8033056277 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:46.685636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245addf3-286b-4dce-bd56-0790c819012a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:46.686325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245addf3-286b-4dce-bd56-0790c819012a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:46.692091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245addf3-286b-4dce-bd56-0790c819012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:46.731389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a909a1-484f-453f-8dbb-710c0fb0fc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:46.732257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a909a1-484f-453f-8dbb-710c0fb0fc6c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:46.732682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a909a1-484f-453f-8dbb-710c0fb0fc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:46.734693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-245addf3-286b-4dce-bd56-0790c819012a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:46.736624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-245addf3-286b-4dce-bd56-0790c819012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:46.744438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a909a1-484f-453f-8dbb-710c0fb0fc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:46.745984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a909a1-484f-453f-8dbb-710c0fb0fc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:46.751733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a909a1-484f-453f-8dbb-710c0fb0fc6c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:46.755894 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3217/6431] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:46.763141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245addf3-286b-4dce-bd56-0790c819012a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:46.766576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-245addf3-286b-4dce-bd56-0790c819012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:46.780035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245addf3-286b-4dce-bd56-0790c819012a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:46.787204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:46.787935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:46.788321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:46.788657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] PATH_INFO: `/v3/projects/5139e0e162734067965cfe7c20ee9f88` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:46.790676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5139e0e162734067965cfe7c20ee9f88)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:46.794624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:46.823137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3215/6432] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:46.831771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee73c5-8fd5-4bb6-932e-c223b6270989 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:46.833220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee73c5-8fd5-4bb6-932e-c223b6270989 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:46.833888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee73c5-8fd5-4bb6-932e-c223b6270989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:46.875794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-33ee73c5-8fd5-4bb6-932e-c223b6270989 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:46.877084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-33ee73c5-8fd5-4bb6-932e-c223b6270989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:46.911293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5139e0e162734067965cfe7c20ee9f88', 'request_id': 'req-16010380-71ff-48ad-b925-852ac65c8f58'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:46.924385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5139e0e162734067965cfe7c20ee9f88', 'request_id': 'req-16010380-71ff-48ad-b925-852ac65c8f58'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:46.941848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5139e0e162734067965cfe7c20ee9f88', 'request_id': 'req-16010380-71ff-48ad-b925-852ac65c8f58'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:46.968150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] The token cache is being invalidate because project 5139e0e162734067965cfe7c20ee9f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:46.968446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:46.972397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:46.975900 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3216/6433] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:46.975900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee73c5-8fd5-4bb6-932e-c223b6270989 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:46.977615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-33ee73c5-8fd5-4bb6-932e-c223b6270989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:46.978518 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3218/6434] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:46 2022] DELETE /identity/v3/projects/5139e0e162734067965cfe7c20ee9f88 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:46.979401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:46.983472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:47.018974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16010380-71ff-48ad-b925-852ac65c8f58 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:47.025659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35987d7-3473-4715-8ece-6d6d6478d7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:47.027011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35987d7-3473-4715-8ece-6d6d6478d7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:47.028382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35987d7-3473-4715-8ece-6d6d6478d7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:47.028797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35987d7-3473-4715-8ece-6d6d6478d7e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:47.029620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35987d7-3473-4715-8ece-6d6d6478d7e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:47.030062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35987d7-3473-4715-8ece-6d6d6478d7e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:47.031872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35987d7-3473-4715-8ece-6d6d6478d7e7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:47.035905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee73c5-8fd5-4bb6-932e-c223b6270989 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:47.055990 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3219/6435] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:47.057820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35987d7-3473-4715-8ece-6d6d6478d7e7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:47.057964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae59c4-5a20-4bc1-badf-6664482f43f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:47.059051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae59c4-5a20-4bc1-badf-6664482f43f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:47.059398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae59c4-5a20-4bc1-badf-6664482f43f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:47.059652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae59c4-5a20-4bc1-badf-6664482f43f6 admin admin] PATH_INFO: `/v3/projects/c9895fbb4dd64581b6f8a706068ca366/users/e93f849687174bc89d2d4e6483dbc7e6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:47.061156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e35987d7-3473-4715-8ece-6d6d6478d7e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:47.084563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35987d7-3473-4715-8ece-6d6d6478d7e7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:47.091778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae59c4-5a20-4bc1-badf-6664482f43f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9895fbb4dd64581b6f8a706068ca366, user_id=e93f849687174bc89d2d4e6483dbc7e6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:47.105282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:47.107005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:47.107274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae59c4-5a20-4bc1-badf-6664482f43f6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:47.108999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:47.109488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] PATH_INFO: `/v3/users/df699280404e4d0bb0240a7054480fc2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:47.112645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df699280404e4d0bb0240a7054480fc2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:47.120312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:47.167098 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3217/6436] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:46 2022] PUT /identity/v3/projects/c9895fbb4dd64581b6f8a706068ca366/users/e93f849687174bc89d2d4e6483dbc7e6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:47.177222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae59c4-5a20-4bc1-badf-6664482f43f6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:47.198726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:47.256471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df699280404e4d0bb0240a7054480fc2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:47.277091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df699280404e4d0bb0240a7054480fc2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:47.294942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df699280404e4d0bb0240a7054480fc2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:47.325286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df699280404e4d0bb0240a7054480fc2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:47.326408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-33aeab5f-afc7-42a6-ae62-118bb6867d2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df699280404e4d0bb0240a7054480fc2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:47.344282 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3220/6437] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:47 2022] DELETE /identity/v3/users/df699280404e4d0bb0240a7054480fc2 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:47.345660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a47e2-5ed6-4edf-a073-1c7df6f8f6c0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:47.345939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a47e2-5ed6-4edf-a073-1c7df6f8f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:47.346207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a47e2-5ed6-4edf-a073-1c7df6f8f6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:47.365354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae59c4-5a20-4bc1-badf-6664482f43f6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:47.411215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c64a47e2-5ed6-4edf-a073-1c7df6f8f6c0 None None] MFA Rules not processed for user `e93f849687174bc89d2d4e6483dbc7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:47.413375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c64a47e2-5ed6-4edf-a073-1c7df6f8f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:47.458605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:47.513785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91caff-369f-4214-81aa-681c298ac5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:47.514867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91caff-369f-4214-81aa-681c298ac5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:47.515167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91caff-369f-4214-81aa-681c298ac5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:47.516137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91caff-369f-4214-81aa-681c298ac5f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:47.518092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91caff-369f-4214-81aa-681c298ac5f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:47.521942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91caff-369f-4214-81aa-681c298ac5f6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:47.573891 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3221/6438] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:47.576712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a47e2-5ed6-4edf-a073-1c7df6f8f6c0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:47.578369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c64a47e2-5ed6-4edf-a073-1c7df6f8f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:47.595438 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3218/6439] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:47 2022] POST /identity/v3/projects => generated 371 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:47.599814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91caff-369f-4214-81aa-681c298ac5f6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:47.604231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bf91caff-369f-4214-81aa-681c298ac5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:47.608367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a47e2-5ed6-4edf-a073-1c7df6f8f6c0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:47.751802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6ec16-e070-44b1-a6ca-966aba84b336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:47.752288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6ec16-e070-44b1-a6ca-966aba84b336 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:47.752563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6ec16-e070-44b1-a6ca-966aba84b336 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:47.752831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6ec16-e070-44b1-a6ca-966aba84b336 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:47.753541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6ec16-e070-44b1-a6ca-966aba84b336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:47.755582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6ec16-e070-44b1-a6ca-966aba84b336 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:47.780021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91caff-369f-4214-81aa-681c298ac5f6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:47.796548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748eea90-e5f7-4f86-b72a-f2938fb8e3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:47.797958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748eea90-e5f7-4f86-b72a-f2938fb8e3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:47.798502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748eea90-e5f7-4f86-b72a-f2938fb8e3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:47.798909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748eea90-e5f7-4f86-b72a-f2938fb8e3f1 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:47.799927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748eea90-e5f7-4f86-b72a-f2938fb8e3f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:47.804226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748eea90-e5f7-4f86-b72a-f2938fb8e3f1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:47.840361 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3222/6440] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:47 2022] POST /identity/v3/projects => generated 367 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:15:47.844333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6ec16-e070-44b1-a6ca-966aba84b336 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:47.861563 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3219/6441] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:47 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:47.866145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748eea90-e5f7-4f86-b72a-f2938fb8e3f1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:47.890459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748eea90-e5f7-4f86-b72a-f2938fb8e3f1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:47.967161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:47.993673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6ec16-e070-44b1-a6ca-966aba84b336 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:48.010629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b699da-988a-4271-91ae-9de67771d8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:48.011240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b699da-988a-4271-91ae-9de67771d8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.011520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b699da-988a-4271-91ae-9de67771d8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.011936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b699da-988a-4271-91ae-9de67771d8c8 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.012570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b699da-988a-4271-91ae-9de67771d8c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:48.015537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b699da-988a-4271-91ae-9de67771d8c8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:48.046937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728eadaf-9807-4502-bac3-e4d5462e5a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:48.047801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728eadaf-9807-4502-bac3-e4d5462e5a3f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.048084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728eadaf-9807-4502-bac3-e4d5462e5a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.048482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728eadaf-9807-4502-bac3-e4d5462e5a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.049183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728eadaf-9807-4502-bac3-e4d5462e5a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:48.051001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728eadaf-9807-4502-bac3-e4d5462e5a3f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:48.067594 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3223/6442] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:47 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:48.070015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b699da-988a-4271-91ae-9de67771d8c8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:48.094521 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3220/6443] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:48.100515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728eadaf-9807-4502-bac3-e4d5462e5a3f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:48.100686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b699da-988a-4271-91ae-9de67771d8c8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:48.114945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728eadaf-9807-4502-bac3-e4d5462e5a3f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:48.121971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e68387-8927-4f02-8b36-2fbd540b5c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:48.122597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e68387-8927-4f02-8b36-2fbd540b5c49 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.122873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e68387-8927-4f02-8b36-2fbd540b5c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.123153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e68387-8927-4f02-8b36-2fbd540b5c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.123249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8237e6f-0f46-46e7-9c0b-dd62cd7f16d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:48.123795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e68387-8927-4f02-8b36-2fbd540b5c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:48.123900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8237e6f-0f46-46e7-9c0b-dd62cd7f16d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.124131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8237e6f-0f46-46e7-9c0b-dd62cd7f16d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.124402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8237e6f-0f46-46e7-9c0b-dd62cd7f16d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.125082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8237e6f-0f46-46e7-9c0b-dd62cd7f16d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:48.125446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e68387-8927-4f02-8b36-2fbd540b5c49 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:48.126648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8237e6f-0f46-46e7-9c0b-dd62cd7f16d6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:48.144979 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3224/6444] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:48.146123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e68387-8927-4f02-8b36-2fbd540b5c49 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:48.147227 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3221/6445] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:48.149587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e6e68387-8927-4f02-8b36-2fbd540b5c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:48.149800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8237e6f-0f46-46e7-9c0b-dd62cd7f16d6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:48.169291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8237e6f-0f46-46e7-9c0b-dd62cd7f16d6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:48.175480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12dcf3-db57-44d5-b1cb-9abb79c80525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:48.176333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12dcf3-db57-44d5-b1cb-9abb79c80525 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.177351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12dcf3-db57-44d5-b1cb-9abb79c80525 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.177351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12dcf3-db57-44d5-b1cb-9abb79c80525 admin admin] PATH_INFO: `/v3/projects/f4628d64883e4f0e88b015e71aef1018/users/e75e282cd33042fcacf2b93e6c41af90/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.196682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e68387-8927-4f02-8b36-2fbd540b5c49 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:48.198500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12dcf3-db57-44d5-b1cb-9abb79c80525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4628d64883e4f0e88b015e71aef1018, user_id=e75e282cd33042fcacf2b93e6c41af90, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:48.201338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12dcf3-db57-44d5-b1cb-9abb79c80525 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:48.204198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:48.205435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.206044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.206460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] PATH_INFO: `/v3/projects/3b3d1b2dc95b4f60b780f35f028d8f5b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.207576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3d1b2dc95b4f60b780f35f028d8f5b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:48.212035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:48.221600 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3222/6446] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:48 2022] PUT /identity/v3/projects/f4628d64883e4f0e88b015e71aef1018/users/e75e282cd33042fcacf2b93e6c41af90/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:48.222647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12dcf3-db57-44d5-b1cb-9abb79c80525 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:48.266729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:48.306099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3d1b2dc95b4f60b780f35f028d8f5b', 'request_id': 'req-0db11800-c32f-472a-a03d-07958cefdec6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:48.324564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3d1b2dc95b4f60b780f35f028d8f5b', 'request_id': 'req-0db11800-c32f-472a-a03d-07958cefdec6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:48.343710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3d1b2dc95b4f60b780f35f028d8f5b', 'request_id': 'req-0db11800-c32f-472a-a03d-07958cefdec6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:48.345831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12dcf3-db57-44d5-b1cb-9abb79c80525 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:48.365399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] The token cache is being invalidate because project 3b3d1b2dc95b4f60b780f35f028d8f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:48.366100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:48.367056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:48.373383 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3225/6447] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:48 2022] DELETE /identity/v3/projects/3b3d1b2dc95b4f60b780f35f028d8f5b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:48.375264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:48.376972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:48.430257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d188d-27d2-4425-9203-901c92c886fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:48.431646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d188d-27d2-4425-9203-901c92c886fc admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.432100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d188d-27d2-4425-9203-901c92c886fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.432388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d188d-27d2-4425-9203-901c92c886fc admin admin] PATH_INFO: `/v3/projects/9504a2c03cb44ff099f07d078cc25de3/users/064d2c9b31bd412996bb666dd8a90b9e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.452957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d188d-27d2-4425-9203-901c92c886fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9504a2c03cb44ff099f07d078cc25de3, user_id=064d2c9b31bd412996bb666dd8a90b9e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:48.456102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d188d-27d2-4425-9203-901c92c886fc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:48.476838 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3223/6448] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:48 2022] PUT /identity/v3/projects/9504a2c03cb44ff099f07d078cc25de3/users/064d2c9b31bd412996bb666dd8a90b9e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:48.478570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb55b6b-b139-4e71-814d-318670d70574 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.479464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb55b6b-b139-4e71-814d-318670d70574 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.479877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb55b6b-b139-4e71-814d-318670d70574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.485160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db11800-c32f-472a-a03d-07958cefdec6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:48.502928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9bb55b6b-b139-4e71-814d-318670d70574 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:48.507074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9bb55b6b-b139-4e71-814d-318670d70574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:48.531722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:48.609858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edd151-9f6e-4c3a-80b4-dd5d0f101b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:48.611761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd151-9f6e-4c3a-80b4-dd5d0f101b68 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.612081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd151-9f6e-4c3a-80b4-dd5d0f101b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.613747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd151-9f6e-4c3a-80b4-dd5d0f101b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.615278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edd151-9f6e-4c3a-80b4-dd5d0f101b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:48.619844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edd151-9f6e-4c3a-80b4-dd5d0f101b68 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:48.647218 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3226/6449] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:48.650928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38517ff-fa05-4543-96bb-1f37e0ffa243 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.651999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38517ff-fa05-4543-96bb-1f37e0ffa243 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.652505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38517ff-fa05-4543-96bb-1f37e0ffa243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.686459 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3224/6450] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:48.690874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb55b6b-b139-4e71-814d-318670d70574 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:48.712466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb55b6b-b139-4e71-814d-318670d70574 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:48.725865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5642b-dcf9-4f1b-b868-0fe16c385177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:48.727238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5642b-dcf9-4f1b-b868-0fe16c385177 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.729613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5642b-dcf9-4f1b-b868-0fe16c385177 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.730842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5642b-dcf9-4f1b-b868-0fe16c385177 admin admin] PATH_INFO: `/v3/projects/f4628d64883e4f0e88b015e71aef1018/users/e75e282cd33042fcacf2b93e6c41af90/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.734258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c38517ff-fa05-4543-96bb-1f37e0ffa243 None None] MFA Rules not processed for user `064d2c9b31bd412996bb666dd8a90b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:48.735222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5642b-dcf9-4f1b-b868-0fe16c385177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4628d64883e4f0e88b015e71aef1018, user_id=e75e282cd33042fcacf2b93e6c41af90, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:48.739292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5642b-dcf9-4f1b-b868-0fe16c385177 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:48.740094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c38517ff-fa05-4543-96bb-1f37e0ffa243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:48.773110 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3225/6451] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:48 2022] PUT /identity/v3/projects/f4628d64883e4f0e88b015e71aef1018/users/e75e282cd33042fcacf2b93e6c41af90/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:48.773110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d9232-6e24-4e83-a6cc-5ca1bd92234d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.773110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d9232-6e24-4e83-a6cc-5ca1bd92234d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.773110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d9232-6e24-4e83-a6cc-5ca1bd92234d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.818407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d35d9232-6e24-4e83-a6cc-5ca1bd92234d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:48.819644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d35d9232-6e24-4e83-a6cc-5ca1bd92234d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:48.848405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:48.895503 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3227/6452] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:48.900208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0a790-0256-4697-86b7-5e0f8b661d06 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:48.900493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0a790-0256-4697-86b7-5e0f8b661d06 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:48.900765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0a790-0256-4697-86b7-5e0f8b661d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:48.973693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-edc0a790-0256-4697-86b7-5e0f8b661d06 None None] MFA Rules not processed for user `e75e282cd33042fcacf2b93e6c41af90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:48.974949 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3226/6453] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:48.975818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d9232-6e24-4e83-a6cc-5ca1bd92234d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:48.976517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-edc0a790-0256-4697-86b7-5e0f8b661d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:48.979656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d35d9232-6e24-4e83-a6cc-5ca1bd92234d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:49.113257 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3228/6454] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:49.114788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0a790-0256-4697-86b7-5e0f8b661d06 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:49.127819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d9232-6e24-4e83-a6cc-5ca1bd92234d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:49.137597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd438-b4a7-44e7-84cc-5cc75031a56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:49.138836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd438-b4a7-44e7-84cc-5cc75031a56e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:49.139117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd438-b4a7-44e7-84cc-5cc75031a56e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:49.139384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd438-b4a7-44e7-84cc-5cc75031a56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:49.151809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0a790-0256-4697-86b7-5e0f8b661d06 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:49.160220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1b302-160f-4425-88e6-5a8a6a2945ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:49.162235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b302-160f-4425-88e6-5a8a6a2945ba admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:49.164019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b302-160f-4425-88e6-5a8a6a2945ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:49.165445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b302-160f-4425-88e6-5a8a6a2945ba admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:49.166714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1b302-160f-4425-88e6-5a8a6a2945ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:49.171459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1b302-160f-4425-88e6-5a8a6a2945ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:49.175730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1b302-160f-4425-88e6-5a8a6a2945ba admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:49.233682 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3229/6455] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:49.235896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1b302-160f-4425-88e6-5a8a6a2945ba admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:49.238974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-56f1b302-160f-4425-88e6-5a8a6a2945ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:49.240505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cd438-b4a7-44e7-84cc-5cc75031a56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:49.243498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cd438-b4a7-44e7-84cc-5cc75031a56e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:49.259031 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3227/6456] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:49.260122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd438-b4a7-44e7-84cc-5cc75031a56e service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:49.272467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:49.284449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd438-b4a7-44e7-84cc-5cc75031a56e service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:49.293852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470bc82-e393-440d-9c0a-1bedc51975ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:49.313429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470bc82-e393-440d-9c0a-1bedc51975ec admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:49.313745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470bc82-e393-440d-9c0a-1bedc51975ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:49.318909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470bc82-e393-440d-9c0a-1bedc51975ec admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:49.319745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470bc82-e393-440d-9c0a-1bedc51975ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:49.322720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470bc82-e393-440d-9c0a-1bedc51975ec admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:49.387662 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3228/6457] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:49 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:49.390116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1b302-160f-4425-88e6-5a8a6a2945ba admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:49.391712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470bc82-e393-440d-9c0a-1bedc51975ec admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:49.458235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f987a9c-2bdb-469b-b8ac-07e878341b66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:49.459158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f987a9c-2bdb-469b-b8ac-07e878341b66 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:49.459566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f987a9c-2bdb-469b-b8ac-07e878341b66 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:49.459935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f987a9c-2bdb-469b-b8ac-07e878341b66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:49.493730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:49.515801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470bc82-e393-440d-9c0a-1bedc51975ec admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:49.522062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bfd5e-6ebd-4a34-8e96-2d3ca1d9551c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:49.522983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bfd5e-6ebd-4a34-8e96-2d3ca1d9551c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:49.523833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bfd5e-6ebd-4a34-8e96-2d3ca1d9551c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:49.524540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bfd5e-6ebd-4a34-8e96-2d3ca1d9551c admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:49.525579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bfd5e-6ebd-4a34-8e96-2d3ca1d9551c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:49.527802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bfd5e-6ebd-4a34-8e96-2d3ca1d9551c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:49.557914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f987a9c-2bdb-469b-b8ac-07e878341b66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:49.559582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f987a9c-2bdb-469b-b8ac-07e878341b66 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:49.561813 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3229/6458] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:49 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:49.567248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bfd5e-6ebd-4a34-8e96-2d3ca1d9551c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:49.588210 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3230/6459] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:49.588409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bfd5e-6ebd-4a34-8e96-2d3ca1d9551c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:49.603389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1924b5-b195-4dd3-9a54-b51a847b3c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:49.604365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1924b5-b195-4dd3-9a54-b51a847b3c12 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:49.605152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1924b5-b195-4dd3-9a54-b51a847b3c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:49.606239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1924b5-b195-4dd3-9a54-b51a847b3c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:49.607024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1924b5-b195-4dd3-9a54-b51a847b3c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:49.609327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1924b5-b195-4dd3-9a54-b51a847b3c12 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:49.626037 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3230/6460] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:15:49.631612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f987a9c-2bdb-469b-b8ac-07e878341b66 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:49.661866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f987a9c-2bdb-469b-b8ac-07e878341b66 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:49.668565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd5e84-80c8-4566-8fe9-4c534d1b40de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:49.671687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5e84-80c8-4566-8fe9-4c534d1b40de admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:49.672233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5e84-80c8-4566-8fe9-4c534d1b40de admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:49.675365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5e84-80c8-4566-8fe9-4c534d1b40de admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:49.677170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd5e84-80c8-4566-8fe9-4c534d1b40de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:49.681967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd5e84-80c8-4566-8fe9-4c534d1b40de admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:49.711904 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3231/6461] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:49.716080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1924b5-b195-4dd3-9a54-b51a847b3c12 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:49.746804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1924b5-b195-4dd3-9a54-b51a847b3c12 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:49.756346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc60a4-ea28-43ff-802b-ac87cdf15099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:49.757015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc60a4-ea28-43ff-802b-ac87cdf15099 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:49.757333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc60a4-ea28-43ff-802b-ac87cdf15099 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:49.759928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc60a4-ea28-43ff-802b-ac87cdf15099 admin admin] PATH_INFO: `/v3/projects/92649601ccb2448fa0a61be61a2b0b5d/users/141536d08ba64ef5812e871743bb3c1d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:49.813717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc60a4-ea28-43ff-802b-ac87cdf15099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92649601ccb2448fa0a61be61a2b0b5d, user_id=141536d08ba64ef5812e871743bb3c1d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:49.817437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc60a4-ea28-43ff-802b-ac87cdf15099 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:49.843174 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3231/6462] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:49 2022] PUT /identity/v3/projects/92649601ccb2448fa0a61be61a2b0b5d/users/141536d08ba64ef5812e871743bb3c1d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:49.859068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bc3a-b4bc-4e05-ab80-a952ca750f2a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:49.859538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bc3a-b4bc-4e05-ab80-a952ca750f2a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:49.860666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bc3a-b4bc-4e05-ab80-a952ca750f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:49.944124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3813bc3a-b4bc-4e05-ab80-a952ca750f2a None None] MFA Rules not processed for user `141536d08ba64ef5812e871743bb3c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:49.946329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3813bc3a-b4bc-4e05-ab80-a952ca750f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:50.117253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc60a4-ea28-43ff-802b-ac87cdf15099 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:50.117811 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3232/6463] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:50.245695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc60a4-ea28-43ff-802b-ac87cdf15099 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:50.251895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf41d6-8839-4a85-b18e-9bc18d702db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:50.252625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf41d6-8839-4a85-b18e-9bc18d702db0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:50.253205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf41d6-8839-4a85-b18e-9bc18d702db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:50.253584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf41d6-8839-4a85-b18e-9bc18d702db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:50.254477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf41d6-8839-4a85-b18e-9bc18d702db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:50.256655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf41d6-8839-4a85-b18e-9bc18d702db0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:50.302049 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3232/6464] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:50 2022] POST /identity/v3/projects => generated 369 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:50.306392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813bc3a-b4bc-4e05-ab80-a952ca750f2a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:50.322455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813bc3a-b4bc-4e05-ab80-a952ca750f2a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:50.466867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bfc3d-2977-4880-9707-041961cacc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:50.468486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bfc3d-2977-4880-9707-041961cacc66 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:50.469225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bfc3d-2977-4880-9707-041961cacc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:50.470112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bfc3d-2977-4880-9707-041961cacc66 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:50.472071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bfc3d-2977-4880-9707-041961cacc66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:50.476294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bfc3d-2977-4880-9707-041961cacc66 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:50.531394 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3233/6465] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:50 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:50.535569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf41d6-8839-4a85-b18e-9bc18d702db0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:50.567342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf41d6-8839-4a85-b18e-9bc18d702db0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:50.593720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06715a0a-00a1-454d-8ef3-33b2d863af42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:50.615846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715a0a-00a1-454d-8ef3-33b2d863af42 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:50.618421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715a0a-00a1-454d-8ef3-33b2d863af42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:50.619918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715a0a-00a1-454d-8ef3-33b2d863af42 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:50.620625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06715a0a-00a1-454d-8ef3-33b2d863af42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:50.630619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06715a0a-00a1-454d-8ef3-33b2d863af42 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:50.670579 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3233/6466] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:50.674799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bfc3d-2977-4880-9707-041961cacc66 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:50.713494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bfc3d-2977-4880-9707-041961cacc66 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:50.732067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdd121-af81-4448-b02e-5360ebbf92df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:50.732851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdd121-af81-4448-b02e-5360ebbf92df admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:50.733930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdd121-af81-4448-b02e-5360ebbf92df admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:50.734312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdd121-af81-4448-b02e-5360ebbf92df admin admin] PATH_INFO: `/v3/projects/dcfe574e0a314efe9ce779ca66bde1ee/users/2c3b0fd333b3497881999fccd6f95fd0/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:50.762579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fdd121-af81-4448-b02e-5360ebbf92df admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcfe574e0a314efe9ce779ca66bde1ee, user_id=2c3b0fd333b3497881999fccd6f95fd0, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:50.780176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fdd121-af81-4448-b02e-5360ebbf92df admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:50.808894 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3234/6467] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:50 2022] PUT /identity/v3/projects/dcfe574e0a314efe9ce779ca66bde1ee/users/2c3b0fd333b3497881999fccd6f95fd0/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:50.813488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06715a0a-00a1-454d-8ef3-33b2d863af42 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:50.854321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06715a0a-00a1-454d-8ef3-33b2d863af42 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:50.875387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:51.071036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3460d2-0ac8-4e68-9c4b-64d95a44d845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:51.075145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3460d2-0ac8-4e68-9c4b-64d95a44d845 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:51.075581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3460d2-0ac8-4e68-9c4b-64d95a44d845 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:51.076156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3460d2-0ac8-4e68-9c4b-64d95a44d845 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:51.076944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3460d2-0ac8-4e68-9c4b-64d95a44d845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:51.083020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3460d2-0ac8-4e68-9c4b-64d95a44d845 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:51.112268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdd121-af81-4448-b02e-5360ebbf92df admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:51.114312 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3234/6468] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:51.132151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdd121-af81-4448-b02e-5360ebbf92df admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:51.154219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e432e-f5e5-4ab0-88e2-d3caa384338d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:51.155260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e432e-f5e5-4ab0-88e2-d3caa384338d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:51.155830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e432e-f5e5-4ab0-88e2-d3caa384338d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:51.156116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e432e-f5e5-4ab0-88e2-d3caa384338d admin admin] PATH_INFO: `/v3/projects/dcfe574e0a314efe9ce779ca66bde1ee/users/2c3b0fd333b3497881999fccd6f95fd0/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:51.158473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e432e-f5e5-4ab0-88e2-d3caa384338d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcfe574e0a314efe9ce779ca66bde1ee, user_id=2c3b0fd333b3497881999fccd6f95fd0, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:51.169171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e432e-f5e5-4ab0-88e2-d3caa384338d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:51.177340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:51.220787 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3235/6469] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:51 2022] PUT /identity/v3/projects/dcfe574e0a314efe9ce779ca66bde1ee/users/2c3b0fd333b3497881999fccd6f95fd0/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:51.235011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f9b2-306c-40eb-906b-94a860c76a76 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:51.235544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f9b2-306c-40eb-906b-94a860c76a76 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:51.235816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f9b2-306c-40eb-906b-94a860c76a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:51.281463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1635f9b2-306c-40eb-906b-94a860c76a76 None None] MFA Rules not processed for user `2c3b0fd333b3497881999fccd6f95fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:51.282702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1635f9b2-306c-40eb-906b-94a860c76a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:51.436462 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3235/6470] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:51.457135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e432e-f5e5-4ab0-88e2-d3caa384338d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:51.605498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e432e-f5e5-4ab0-88e2-d3caa384338d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:51.616204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39806cfd-402d-4cfb-88a6-cf49f48b6fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:51.619518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39806cfd-402d-4cfb-88a6-cf49f48b6fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:51.620808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39806cfd-402d-4cfb-88a6-cf49f48b6fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:51.622203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39806cfd-402d-4cfb-88a6-cf49f48b6fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:51.637079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39806cfd-402d-4cfb-88a6-cf49f48b6fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:51.639048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39806cfd-402d-4cfb-88a6-cf49f48b6fe5 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:51.678271 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3236/6471] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:51.788480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635f9b2-306c-40eb-906b-94a860c76a76 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:51.813932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635f9b2-306c-40eb-906b-94a860c76a76 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:51.821841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7f92b-9302-4635-b986-3f533b6007a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:51.823811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7f92b-9302-4635-b986-3f533b6007a8 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:51.824173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7f92b-9302-4635-b986-3f533b6007a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:51.824958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7f92b-9302-4635-b986-3f533b6007a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:52.035079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7f92b-9302-4635-b986-3f533b6007a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:52.040593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7f92b-9302-4635-b986-3f533b6007a8 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:52.078242 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3236/6472] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:52.774796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39806cfd-402d-4cfb-88a6-cf49f48b6fe5 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:52.924160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39806cfd-402d-4cfb-88a6-cf49f48b6fe5 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:52.934954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:52.934954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:52.942081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:52.943289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] PATH_INFO: `/v3/users/e93f849687174bc89d2d4e6483dbc7e6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:52.951833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93f849687174bc89d2d4e6483dbc7e6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:52.954272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:53.058514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93f849687174bc89d2d4e6483dbc7e6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:53.111186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93f849687174bc89d2d4e6483dbc7e6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:53.134809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93f849687174bc89d2d4e6483dbc7e6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:53.155479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93f849687174bc89d2d4e6483dbc7e6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:53.156765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93f849687174bc89d2d4e6483dbc7e6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:53.183873 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3237/6473] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:52 2022] DELETE /identity/v3/users/e93f849687174bc89d2d4e6483dbc7e6 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:15:53.185544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b56ce2-01f9-420a-a0dd-41f193581f8d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:53.187681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b56ce2-01f9-420a-a0dd-41f193581f8d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:53.188900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b56ce2-01f9-420a-a0dd-41f193581f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:53.251640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-42b56ce2-01f9-420a-a0dd-41f193581f8d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:53.253634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-42b56ce2-01f9-420a-a0dd-41f193581f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:53.313131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:53.393245 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3237/6474] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:53.436080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:53.437586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:53.484833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:53.619811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ee645-30ca-410f-b8c0-68fb90fb39b5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:53.631394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b4c58-13dd-4d0c-a0bb-7cb7caa3dc04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:53.631394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b4c58-13dd-4d0c-a0bb-7cb7caa3dc04 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:53.632021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b4c58-13dd-4d0c-a0bb-7cb7caa3dc04 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:53.632366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b4c58-13dd-4d0c-a0bb-7cb7caa3dc04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:53.666487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b4c58-13dd-4d0c-a0bb-7cb7caa3dc04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:53.668110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b4c58-13dd-4d0c-a0bb-7cb7caa3dc04 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:53.693227 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3238/6475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:53.792854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b56ce2-01f9-420a-a0dd-41f193581f8d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:53.831180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b56ce2-01f9-420a-a0dd-41f193581f8d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:53.867450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad726a5-e54d-4110-bdb8-785cb2bfba0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:53.872603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad726a5-e54d-4110-bdb8-785cb2bfba0f service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:53.875457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad726a5-e54d-4110-bdb8-785cb2bfba0f service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:53.875809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad726a5-e54d-4110-bdb8-785cb2bfba0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:53.888962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5ad726a5-e54d-4110-bdb8-785cb2bfba0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:53.955951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad726a5-e54d-4110-bdb8-785cb2bfba0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:53.961910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad726a5-e54d-4110-bdb8-785cb2bfba0f service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:54.012900 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3238/6476] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:54.815184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b4c58-13dd-4d0c-a0bb-7cb7caa3dc04 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:54.816723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5e6b4c58-13dd-4d0c-a0bb-7cb7caa3dc04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:54.861276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b4c58-13dd-4d0c-a0bb-7cb7caa3dc04 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:54.889063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:54.889721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:54.890001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:54.890269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] PATH_INFO: `/v3/projects/c9895fbb4dd64581b6f8a706068ca366` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:54.891596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9895fbb4dd64581b6f8a706068ca366)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:54.893682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:55.045284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9895fbb4dd64581b6f8a706068ca366', 'request_id': 'req-e6edf6ab-11fd-466d-b120-352f3684544b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:55.047653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9895fbb4dd64581b6f8a706068ca366', 'request_id': 'req-e6edf6ab-11fd-466d-b120-352f3684544b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:55.058255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9895fbb4dd64581b6f8a706068ca366', 'request_id': 'req-e6edf6ab-11fd-466d-b120-352f3684544b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:55.106446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] The token cache is being invalidate because project c9895fbb4dd64581b6f8a706068ca366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:55.108015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:55.109725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:55.114434 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3239/6477] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:54 2022] DELETE /identity/v3/projects/c9895fbb4dd64581b6f8a706068ca366 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:55.119198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad726a5-e54d-4110-bdb8-785cb2bfba0f service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:55.135560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad726a5-e54d-4110-bdb8-785cb2bfba0f service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:55.296405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:55.297225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:55.297594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:55.297994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] PATH_INFO: `/v3/users/e75e282cd33042fcacf2b93e6c41af90` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:55.300572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e75e282cd33042fcacf2b93e6c41af90)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:55.304976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:55.425013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e75e282cd33042fcacf2b93e6c41af90'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:55.441575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e75e282cd33042fcacf2b93e6c41af90'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:55.482661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e75e282cd33042fcacf2b93e6c41af90'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:55.508439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e75e282cd33042fcacf2b93e6c41af90'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:55.509721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dd635286-dbf6-4404-8886-1a5ed1fed78e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e75e282cd33042fcacf2b93e6c41af90'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:55.529422 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3239/6478] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:55 2022] DELETE /identity/v3/users/e75e282cd33042fcacf2b93e6c41af90 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:56.302931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:56.306394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:56.350963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf6ab-11fd-466d-b120-352f3684544b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:56.535492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:56.536437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:56.536804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:56.537207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] PATH_INFO: `/v3/projects/f4628d64883e4f0e88b015e71aef1018` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:56.539279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4628d64883e4f0e88b015e71aef1018)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:56.542967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:56.713766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:56.722032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4628d64883e4f0e88b015e71aef1018', 'request_id': 'req-3b92335c-fd20-40ec-97fa-4169e74e0423'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:56.742399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4628d64883e4f0e88b015e71aef1018', 'request_id': 'req-3b92335c-fd20-40ec-97fa-4169e74e0423'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:56.764296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4628d64883e4f0e88b015e71aef1018', 'request_id': 'req-3b92335c-fd20-40ec-97fa-4169e74e0423'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:56.801322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] The token cache is being invalidate because project f4628d64883e4f0e88b015e71aef1018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:15:56.802271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:56.803780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3b92335c-fd20-40ec-97fa-4169e74e0423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:56.809289 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3240/6479] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:15:56 2022] DELETE /identity/v3/projects/f4628d64883e4f0e88b015e71aef1018 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:56.814651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422b786-7942-4b33-9f5f-d7d92ac9b841 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:56.815603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422b786-7942-4b33-9f5f-d7d92ac9b841 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:56.816122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422b786-7942-4b33-9f5f-d7d92ac9b841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:56.856749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e422b786-7942-4b33-9f5f-d7d92ac9b841 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:56.865887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e422b786-7942-4b33-9f5f-d7d92ac9b841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:56.963936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:57.042074 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3240/6480] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:57.051027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981849c-f4cc-4c52-a5cf-fffb63219e3d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:57.051320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981849c-f4cc-4c52-a5cf-fffb63219e3d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:57.051590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981849c-f4cc-4c52-a5cf-fffb63219e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:57.118231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b981849c-f4cc-4c52-a5cf-fffb63219e3d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:57.119451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b981849c-f4cc-4c52-a5cf-fffb63219e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:57.136393 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3241/6481] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:57.140412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422b786-7942-4b33-9f5f-d7d92ac9b841 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:57.152469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422b786-7942-4b33-9f5f-d7d92ac9b841 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:57.170669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e0e9-45e4-4101-bf7e-5ffe0e9d8234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:57.171598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e0e9-45e4-4101-bf7e-5ffe0e9d8234 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:57.171698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e0e9-45e4-4101-bf7e-5ffe0e9d8234 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:57.172015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e0e9-45e4-4101-bf7e-5ffe0e9d8234 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:57.172852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e0e9-45e4-4101-bf7e-5ffe0e9d8234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:15:57.173778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e0e9-45e4-4101-bf7e-5ffe0e9d8234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:57.180341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e0e9-45e4-4101-bf7e-5ffe0e9d8234 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:57.216125 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3241/6482] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:15:57.224585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981849c-f4cc-4c52-a5cf-fffb63219e3d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:57.253410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981849c-f4cc-4c52-a5cf-fffb63219e3d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:57.280809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c39fd-3287-44e1-a7e5-fadd454cb0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:57.281998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c39fd-3287-44e1-a7e5-fadd454cb0e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:57.285243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c39fd-3287-44e1-a7e5-fadd454cb0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:57.285243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c39fd-3287-44e1-a7e5-fadd454cb0e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:57.285693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c39fd-3287-44e1-a7e5-fadd454cb0e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:57.294494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c39fd-3287-44e1-a7e5-fadd454cb0e7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:57.395436 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3242/6483] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:57 2022] POST /identity/v3/projects => generated 389 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:57.400061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e0e9-45e4-4101-bf7e-5ffe0e9d8234 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:57.411994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e0e9-45e4-4101-bf7e-5ffe0e9d8234 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:57.615127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71842433-202a-47f9-a244-8863f83dd19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:57.615127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71842433-202a-47f9-a244-8863f83dd19b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:57.615127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71842433-202a-47f9-a244-8863f83dd19b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:57.618899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71842433-202a-47f9-a244-8863f83dd19b admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:57.623075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71842433-202a-47f9-a244-8863f83dd19b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:57.630003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71842433-202a-47f9-a244-8863f83dd19b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:57.694159 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3242/6484] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:57 2022] POST /identity/v3/users => generated 295 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:15:57.696909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c39fd-3287-44e1-a7e5-fadd454cb0e7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:57.710526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c39fd-3287-44e1-a7e5-fadd454cb0e7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:57.728339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333292b-1ec4-4806-b42c-40983bab9406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:57.728339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333292b-1ec4-4806-b42c-40983bab9406 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:57.728922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333292b-1ec4-4806-b42c-40983bab9406 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:57.729628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333292b-1ec4-4806-b42c-40983bab9406 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:57.730736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333292b-1ec4-4806-b42c-40983bab9406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:57.732936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333292b-1ec4-4806-b42c-40983bab9406 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:57.773314 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3243/6485] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:57.774996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71842433-202a-47f9-a244-8863f83dd19b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:57.794482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71842433-202a-47f9-a244-8863f83dd19b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:57.808757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43f507-b640-4d8c-ae57-b6decd5623a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:57.810896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f507-b640-4d8c-ae57-b6decd5623a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:57.811668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f507-b640-4d8c-ae57-b6decd5623a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:57.813999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f507-b640-4d8c-ae57-b6decd5623a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:57.814800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43f507-b640-4d8c-ae57-b6decd5623a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:57.820532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43f507-b640-4d8c-ae57-b6decd5623a3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:57.843384 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3243/6486] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:57.846388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333292b-1ec4-4806-b42c-40983bab9406 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:57.865227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333292b-1ec4-4806-b42c-40983bab9406 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:57.876859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f607685-bfa0-4dcf-99fa-4673d381b7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:57.877769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f607685-bfa0-4dcf-99fa-4673d381b7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:57.878191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f607685-bfa0-4dcf-99fa-4673d381b7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:57.878819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f607685-bfa0-4dcf-99fa-4673d381b7ac admin admin] PATH_INFO: `/v3/projects/2605ac87a00a438da8cf5da292c905ce/users/1837db4b189e49f1825e7fd675eb713a/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:57.973846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f607685-bfa0-4dcf-99fa-4673d381b7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2605ac87a00a438da8cf5da292c905ce, user_id=1837db4b189e49f1825e7fd675eb713a, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:57.976898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f607685-bfa0-4dcf-99fa-4673d381b7ac admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:58.031560 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3244/6487] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:57 2022] PUT /identity/v3/projects/2605ac87a00a438da8cf5da292c905ce/users/1837db4b189e49f1825e7fd675eb713a/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:58.048692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee29dce-dc44-4964-a8e1-1da7a8741925 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:58.049213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee29dce-dc44-4964-a8e1-1da7a8741925 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:58.049663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee29dce-dc44-4964-a8e1-1da7a8741925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:58.111906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9ee29dce-dc44-4964-a8e1-1da7a8741925 None None] MFA Rules not processed for user `1837db4b189e49f1825e7fd675eb713a`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:58.116344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9ee29dce-dc44-4964-a8e1-1da7a8741925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:58.193347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f607685-bfa0-4dcf-99fa-4673d381b7ac admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:58.196006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f607685-bfa0-4dcf-99fa-4673d381b7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:58.233519 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3244/6488] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:58.237568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee29dce-dc44-4964-a8e1-1da7a8741925 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:58.310095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:58.354692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f607685-bfa0-4dcf-99fa-4673d381b7ac admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:58.368824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:58.371334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:58.371895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:58.372272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] PATH_INFO: `/v3/users/141536d08ba64ef5812e871743bb3c1d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:58.373730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=141536d08ba64ef5812e871743bb3c1d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:58.378254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:58.387955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee29dce-dc44-4964-a8e1-1da7a8741925 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:58.405820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706150ec-f5ec-42da-84bd-c286e1e0b16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:58.406793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706150ec-f5ec-42da-84bd-c286e1e0b16a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:58.407249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706150ec-f5ec-42da-84bd-c286e1e0b16a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:58.407659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706150ec-f5ec-42da-84bd-c286e1e0b16a admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:58.408438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706150ec-f5ec-42da-84bd-c286e1e0b16a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:58.410875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706150ec-f5ec-42da-84bd-c286e1e0b16a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:58.474585 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3245/6489] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:15:58 2022] POST /identity/v3/projects => generated 387 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:15:58.479434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706150ec-f5ec-42da-84bd-c286e1e0b16a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:58.486996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '141536d08ba64ef5812e871743bb3c1d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:58.521465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '141536d08ba64ef5812e871743bb3c1d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:58.534982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:58.543713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '141536d08ba64ef5812e871743bb3c1d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:58.563017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '141536d08ba64ef5812e871743bb3c1d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:58.565236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ddaaaab0-7178-46c2-83d7-0c25e9ab8d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '141536d08ba64ef5812e871743bb3c1d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:15:58.582247 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3245/6490] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:15:58 2022] DELETE /identity/v3/users/141536d08ba64ef5812e871743bb3c1d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:58.586755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02d5c2-477a-45be-acff-a945ef1b2438 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:58.587728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02d5c2-477a-45be-acff-a945ef1b2438 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:58.588102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02d5c2-477a-45be-acff-a945ef1b2438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:58.634545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6c02d5c2-477a-45be-acff-a945ef1b2438 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:58.636290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6c02d5c2-477a-45be-acff-a945ef1b2438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:58.679095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706150ec-f5ec-42da-84bd-c286e1e0b16a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:58.811619 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3246/6491] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:15:58.883863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02d5c2-477a-45be-acff-a945ef1b2438 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:58.884664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaffc88-d969-4456-bba7-17f2b8a65e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:58.886279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6c02d5c2-477a-45be-acff-a945ef1b2438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:58.891874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffc88-d969-4456-bba7-17f2b8a65e42 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:58.893771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffc88-d969-4456-bba7-17f2b8a65e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:58.893771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffc88-d969-4456-bba7-17f2b8a65e42 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:58.894346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaffc88-d969-4456-bba7-17f2b8a65e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:58.897746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaffc88-d969-4456-bba7-17f2b8a65e42 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:58.975604 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3246/6492] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:15:58 2022] POST /identity/v3/users => generated 300 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:15:58.978501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaffc88-d969-4456-bba7-17f2b8a65e42 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:58.981905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fdaffc88-d969-4456-bba7-17f2b8a65e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:59.006051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaffc88-d969-4456-bba7-17f2b8a65e42 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:59.016534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9b7f0-5df5-4a21-bae0-de709201cb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:59.017969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9b7f0-5df5-4a21-bae0-de709201cb95 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:59.018512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9b7f0-5df5-4a21-bae0-de709201cb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:59.019325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9b7f0-5df5-4a21-bae0-de709201cb95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:59.020350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9b7f0-5df5-4a21-bae0-de709201cb95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:59.022362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9b7f0-5df5-4a21-bae0-de709201cb95 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:59.053563 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3247/6493] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:15:59.055291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9b7f0-5df5-4a21-bae0-de709201cb95 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:59.069453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02d5c2-477a-45be-acff-a945ef1b2438 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:59.075734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9b7f0-5df5-4a21-bae0-de709201cb95 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:59.088161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760797eb-aaad-430e-a5a1-705cb67ea323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:59.090742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68600af-f8b6-4c76-859f-4d6ee9f73464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:59.091790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760797eb-aaad-430e-a5a1-705cb67ea323 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:59.092611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760797eb-aaad-430e-a5a1-705cb67ea323 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:59.095689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68600af-f8b6-4c76-859f-4d6ee9f73464 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:59.096826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760797eb-aaad-430e-a5a1-705cb67ea323 admin admin] PATH_INFO: `/v3/projects/3e4c7836a8b3415bab71fd09bf534ace/users/4c76eb2493624b3dbaf130d14c9c3153/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:59.096952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68600af-f8b6-4c76-859f-4d6ee9f73464 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:59.098232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68600af-f8b6-4c76-859f-4d6ee9f73464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:59.124818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760797eb-aaad-430e-a5a1-705cb67ea323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4c7836a8b3415bab71fd09bf534ace, user_id=4c76eb2493624b3dbaf130d14c9c3153, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:59.129743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760797eb-aaad-430e-a5a1-705cb67ea323 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:59.130634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68600af-f8b6-4c76-859f-4d6ee9f73464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:59.133850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68600af-f8b6-4c76-859f-4d6ee9f73464 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:59.172647 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3248/6494] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:59 2022] PUT /identity/v3/projects/3e4c7836a8b3415bab71fd09bf534ace/users/4c76eb2493624b3dbaf130d14c9c3153/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:59.176253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760797eb-aaad-430e-a5a1-705cb67ea323 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:59.182388 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3247/6495] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:15:59.222979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:59.321705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760797eb-aaad-430e-a5a1-705cb67ea323 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:59.335917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650bed32-cb05-4ca7-834d-6eb47a329150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:59.336667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650bed32-cb05-4ca7-834d-6eb47a329150 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:59.337018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650bed32-cb05-4ca7-834d-6eb47a329150 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:59.337365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650bed32-cb05-4ca7-834d-6eb47a329150 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:59.338240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650bed32-cb05-4ca7-834d-6eb47a329150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:59.339765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650bed32-cb05-4ca7-834d-6eb47a329150 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:59.368537 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3249/6496] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:15:59.372917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68600af-f8b6-4c76-859f-4d6ee9f73464 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:59.386204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68600af-f8b6-4c76-859f-4d6ee9f73464 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:59.400358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187e256-bc58-4292-8f66-ccccfb15b33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:59.401046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187e256-bc58-4292-8f66-ccccfb15b33f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:59.401332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187e256-bc58-4292-8f66-ccccfb15b33f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:59.401610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187e256-bc58-4292-8f66-ccccfb15b33f admin admin] PATH_INFO: `/v3/projects/3e4c7836a8b3415bab71fd09bf534ace/users/4c76eb2493624b3dbaf130d14c9c3153/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:59.408050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187e256-bc58-4292-8f66-ccccfb15b33f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4c7836a8b3415bab71fd09bf534ace, user_id=4c76eb2493624b3dbaf130d14c9c3153, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:59.410981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187e256-bc58-4292-8f66-ccccfb15b33f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:15:59.437261 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3248/6497] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:15:59 2022] PUT /identity/v3/projects/3e4c7836a8b3415bab71fd09bf534ace/users/4c76eb2493624b3dbaf130d14c9c3153/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:15:59.448281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e9a26-f863-4289-b089-494520b45ff1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:59.448740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e9a26-f863-4289-b089-494520b45ff1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:59.449204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e9a26-f863-4289-b089-494520b45ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:59.534331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fd9e9a26-f863-4289-b089-494520b45ff1 None None] MFA Rules not processed for user `4c76eb2493624b3dbaf130d14c9c3153`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:15:59.536168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd9e9a26-f863-4289-b089-494520b45ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:59.683870 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3250/6498] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:15:59.700210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187e256-bc58-4292-8f66-ccccfb15b33f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:15:59.702885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0187e256-bc58-4292-8f66-ccccfb15b33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:15:59.955657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187e256-bc58-4292-8f66-ccccfb15b33f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:15:59.960587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed9e0-efca-44b3-af74-f98ffdfe84e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:15:59.964499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed9e0-efca-44b3-af74-f98ffdfe84e4 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:15:59.965088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed9e0-efca-44b3-af74-f98ffdfe84e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:15:59.966269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed9e0-efca-44b3-af74-f98ffdfe84e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:15:59.988678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ed9e0-efca-44b3-af74-f98ffdfe84e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:15:59.990807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ed9e0-efca-44b3-af74-f98ffdfe84e4 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:00.014430 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3249/6499] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:15:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:00.089060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e9a26-f863-4289-b089-494520b45ff1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:00.092037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd9e9a26-f863-4289-b089-494520b45ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:00.270324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e9a26-f863-4289-b089-494520b45ff1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:00.277528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:00.278166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:00.278436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:00.278699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] PATH_INFO: `/v3/projects/92649601ccb2448fa0a61be61a2b0b5d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:00.286708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=92649601ccb2448fa0a61be61a2b0b5d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:00.294094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:00.410843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:00.428703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92649601ccb2448fa0a61be61a2b0b5d', 'request_id': 'req-38772382-32e1-4c01-a36c-3b76f61987dd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:00.435490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed9e0-efca-44b3-af74-f98ffdfe84e4 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:00.437410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0b7ed9e0-efca-44b3-af74-f98ffdfe84e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:00.457896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92649601ccb2448fa0a61be61a2b0b5d', 'request_id': 'req-38772382-32e1-4c01-a36c-3b76f61987dd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:00.472661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92649601ccb2448fa0a61be61a2b0b5d', 'request_id': 'req-38772382-32e1-4c01-a36c-3b76f61987dd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:00.500175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed9e0-efca-44b3-af74-f98ffdfe84e4 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:00.517211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] The token cache is being invalidate because project 92649601ccb2448fa0a61be61a2b0b5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:00.517611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:00.519304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:00.523256 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3251/6500] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:00 2022] DELETE /identity/v3/projects/92649601ccb2448fa0a61be61a2b0b5d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:00.524348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:00.527082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:00.597244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:00.751764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:00.753974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:00.754414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:00.754833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] PATH_INFO: `/v3/users/47f790a3672b449d81e96842483481ee` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:00.758474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f790a3672b449d81e96842483481ee)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:00.760764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:00.764352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38772382-32e1-4c01-a36c-3b76f61987dd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:00.769794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:00.771133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:00.771520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:00.771921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] PATH_INFO: `/v3/users/1837db4b189e49f1825e7fd675eb713a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:00.774508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1837db4b189e49f1825e7fd675eb713a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:00.804656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:00.900410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f790a3672b449d81e96842483481ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:00.943696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f790a3672b449d81e96842483481ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:00.947237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1837db4b189e49f1825e7fd675eb713a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:00.964025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1837db4b189e49f1825e7fd675eb713a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:00.966497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f790a3672b449d81e96842483481ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.013502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f790a3672b449d81e96842483481ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.022205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f790a3672b449d81e96842483481ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.052530 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3250/6501] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:00 2022] DELETE /identity/v3/users/47f790a3672b449d81e96842483481ee => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:01.053823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:01.059828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1837db4b189e49f1825e7fd675eb713a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.061023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:01.095193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1837db4b189e49f1825e7fd675eb713a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.095839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1837db4b189e49f1825e7fd675eb713a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.105526 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3252/6502] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:00 2022] DELETE /identity/v3/users/1837db4b189e49f1825e7fd675eb713a => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:01.108439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:01.114741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:01.241215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6921cf5-71a4-473e-9fce-adfd8b12bbb7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:01.254213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:01.302195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:01.336625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:01.337265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:01.337539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:01.337808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] PATH_INFO: `/v3/users/2c3b0fd333b3497881999fccd6f95fd0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:01.340333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c3b0fd333b3497881999fccd6f95fd0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:01.342382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:01.354810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2aa40-6714-4a6e-9d1b-68b535faffd8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:01.370549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a9254-6a1d-4ad9-b3a3-ff0d3046d2b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:01.371378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a9254-6a1d-4ad9-b3a3-ff0d3046d2b0 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:01.371726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a9254-6a1d-4ad9-b3a3-ff0d3046d2b0 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:01.372141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a9254-6a1d-4ad9-b3a3-ff0d3046d2b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:01.379404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-075a9254-6a1d-4ad9-b3a3-ff0d3046d2b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:01.415163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a9254-6a1d-4ad9-b3a3-ff0d3046d2b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:01.417125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a9254-6a1d-4ad9-b3a3-ff0d3046d2b0 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:01.433790 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3253/6503] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:01.448114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b89df-c971-44e5-a1ae-e0143b533117 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:01.448452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b89df-c971-44e5-a1ae-e0143b533117 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:01.450531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b89df-c971-44e5-a1ae-e0143b533117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:01.459582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c3b0fd333b3497881999fccd6f95fd0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.510677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c3b0fd333b3497881999fccd6f95fd0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.532439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5a0b89df-c971-44e5-a1ae-e0143b533117 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:01.533700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5a0b89df-c971-44e5-a1ae-e0143b533117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:01.539016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c3b0fd333b3497881999fccd6f95fd0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.570547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c3b0fd333b3497881999fccd6f95fd0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.575368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c3b0fd333b3497881999fccd6f95fd0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:01.578400 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3254/6504] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:01.587614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564ef0-0901-4338-bc12-2f154352cb7a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:01.587614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564ef0-0901-4338-bc12-2f154352cb7a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:01.589364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564ef0-0901-4338-bc12-2f154352cb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:01.599471 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3251/6505] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:01 2022] DELETE /identity/v3/users/2c3b0fd333b3497881999fccd6f95fd0 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:01.636223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8f564ef0-0901-4338-bc12-2f154352cb7a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:01.637657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f564ef0-0901-4338-bc12-2f154352cb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:01.659569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:01.662196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:01.784073 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3255/6506] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:01.809619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cbbfd-b9cb-4558-a745-9b187a3a4b35 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:01.813918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d162c5-8e02-4867-b39d-1cb3a605259c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:01.814801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d162c5-8e02-4867-b39d-1cb3a605259c service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:01.815165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d162c5-8e02-4867-b39d-1cb3a605259c service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:01.815526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d162c5-8e02-4867-b39d-1cb3a605259c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:01.817912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-11d162c5-8e02-4867-b39d-1cb3a605259c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:01.862077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d162c5-8e02-4867-b39d-1cb3a605259c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:01.866922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d162c5-8e02-4867-b39d-1cb3a605259c service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:01.884625 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3252/6507] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:01.927247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f564ef0-0901-4338-bc12-2f154352cb7a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:01.930224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f564ef0-0901-4338-bc12-2f154352cb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:01.974692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f564ef0-0901-4338-bc12-2f154352cb7a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:01.996665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f223b5-0e13-4bab-b24a-0be8914822ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:01.997849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f223b5-0e13-4bab-b24a-0be8914822ea service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:01.998222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f223b5-0e13-4bab-b24a-0be8914822ea service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:01.998584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f223b5-0e13-4bab-b24a-0be8914822ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:02.018693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f223b5-0e13-4bab-b24a-0be8914822ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:02.021512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f223b5-0e13-4bab-b24a-0be8914822ea service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:02.040330 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3256/6508] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:02.555222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d162c5-8e02-4867-b39d-1cb3a605259c service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:02.556805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-11d162c5-8e02-4867-b39d-1cb3a605259c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:02.596012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d162c5-8e02-4867-b39d-1cb3a605259c service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:02.602300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:02.603677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:02.604019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:02.608067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] PATH_INFO: `/v3/projects/a872801e32fb48c388b7a696b1b6a485` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:02.612381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a872801e32fb48c388b7a696b1b6a485)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:02.620408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:02.732831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a872801e32fb48c388b7a696b1b6a485', 'request_id': 'req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:02.745835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a872801e32fb48c388b7a696b1b6a485', 'request_id': 'req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:02.753575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a872801e32fb48c388b7a696b1b6a485', 'request_id': 'req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:02.757966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:02.769832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] The token cache is being invalidate because project a872801e32fb48c388b7a696b1b6a485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:02.770617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:02.773445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:02.776997 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3253/6509] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:02 2022] DELETE /identity/v3/projects/a872801e32fb48c388b7a696b1b6a485 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:02.782215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e899ece-f6f8-4175-ba55-214899acebb0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:02.783122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e899ece-f6f8-4175-ba55-214899acebb0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:02.783568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e899ece-f6f8-4175-ba55-214899acebb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:02.808785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6e899ece-f6f8-4175-ba55-214899acebb0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:02.810284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e899ece-f6f8-4175-ba55-214899acebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:02.881693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:02.884966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:02.963704 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3257/6510] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:02.965331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9230a2-2b9c-4136-b1e9-cf80268d8ddc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:02.965623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9230a2-2b9c-4136-b1e9-cf80268d8ddc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:02.965900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9230a2-2b9c-4136-b1e9-cf80268d8ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:02.990133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8e9230a2-2b9c-4136-b1e9-cf80268d8ddc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:02.991750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8e9230a2-2b9c-4136-b1e9-cf80268d8ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:03.002517 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3258/6511] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:03.008574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9230a2-2b9c-4136-b1e9-cf80268d8ddc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:03.020713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b3c36-f6b1-4cbd-b2e7-704ddab1e082 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:03.023297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9230a2-2b9c-4136-b1e9-cf80268d8ddc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:03.029161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb329cd-a441-4fa0-9aee-1e06bec3e0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:03.029831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb329cd-a441-4fa0-9aee-1e06bec3e0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.030118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb329cd-a441-4fa0-9aee-1e06bec3e0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.030389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb329cd-a441-4fa0-9aee-1e06bec3e0c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.030666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:03.031241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb329cd-a441-4fa0-9aee-1e06bec3e0c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:03.031549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb329cd-a441-4fa0-9aee-1e06bec3e0c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:03.032021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.033386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.033505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb329cd-a441-4fa0-9aee-1e06bec3e0c5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:03.033983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] PATH_INFO: `/v3/users/d549b5c857bb4563a7c192bb75e678c4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.037563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d549b5c857bb4563a7c192bb75e678c4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:03.044117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:03.054743 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3259/6512] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:16:03.055890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb329cd-a441-4fa0-9aee-1e06bec3e0c5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:03.072801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb329cd-a441-4fa0-9aee-1e06bec3e0c5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:03.081933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a2ca5-83c5-4b8a-aa87-9c47d92d86cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:03.082565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a2ca5-83c5-4b8a-aa87-9c47d92d86cd admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.082891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a2ca5-83c5-4b8a-aa87-9c47d92d86cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.083217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a2ca5-83c5-4b8a-aa87-9c47d92d86cd admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.083914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a2ca5-83c5-4b8a-aa87-9c47d92d86cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:03.088285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a2ca5-83c5-4b8a-aa87-9c47d92d86cd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:03.148543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd549b5c857bb4563a7c192bb75e678c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:03.153316 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3260/6513] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:03 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:03.154459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a2ca5-83c5-4b8a-aa87-9c47d92d86cd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:03.168346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:03.203221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd549b5c857bb4563a7c192bb75e678c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:03.219203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd549b5c857bb4563a7c192bb75e678c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:03.231222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd549b5c857bb4563a7c192bb75e678c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:03.231941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d75f9b2-a0b6-4cbc-8cf9-fcb468558f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd549b5c857bb4563a7c192bb75e678c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:03.239844 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3254/6514] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:02 2022] DELETE /identity/v3/users/d549b5c857bb4563a7c192bb75e678c4 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:03.245121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faab23-beb9-42d6-80c5-1293c2278f6e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.245788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faab23-beb9-42d6-80c5-1293c2278f6e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.246274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faab23-beb9-42d6-80c5-1293c2278f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.274857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-37faab23-beb9-42d6-80c5-1293c2278f6e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:03.276564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-37faab23-beb9-42d6-80c5-1293c2278f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:03.276964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a2ca5-83c5-4b8a-aa87-9c47d92d86cd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:03.385012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851456e-6afa-4671-bee5-c6fe47868f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:03.385847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851456e-6afa-4671-bee5-c6fe47868f6e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.386131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851456e-6afa-4671-bee5-c6fe47868f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.387021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851456e-6afa-4671-bee5-c6fe47868f6e admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.387998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851456e-6afa-4671-bee5-c6fe47868f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:03.390352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851456e-6afa-4671-bee5-c6fe47868f6e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:03.421553 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3255/6515] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:16:03.428351 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3261/6516] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:03 2022] POST /identity/v3/users => generated 290 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:03.433855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37faab23-beb9-42d6-80c5-1293c2278f6e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:03.436048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-37faab23-beb9-42d6-80c5-1293c2278f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:03.440035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851456e-6afa-4671-bee5-c6fe47868f6e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:03.445127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8851456e-6afa-4671-bee5-c6fe47868f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:03.471742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37faab23-beb9-42d6-80c5-1293c2278f6e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:03.483392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1edec-f2ef-4cd6-9a2d-b591ac9cd41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:03.484969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1edec-f2ef-4cd6-9a2d-b591ac9cd41f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.485765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1edec-f2ef-4cd6-9a2d-b591ac9cd41f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.486252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1edec-f2ef-4cd6-9a2d-b591ac9cd41f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.488079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1edec-f2ef-4cd6-9a2d-b591ac9cd41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:03.490699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1edec-f2ef-4cd6-9a2d-b591ac9cd41f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:03.501487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:03.514863 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3256/6517] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:16:03.516105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1edec-f2ef-4cd6-9a2d-b591ac9cd41f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:03.537879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1edec-f2ef-4cd6-9a2d-b591ac9cd41f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:03.546184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8f58b-b331-413f-b540-333ff4f75588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:03.546961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8f58b-b331-413f-b540-333ff4f75588 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.548628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8f58b-b331-413f-b540-333ff4f75588 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.549008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8f58b-b331-413f-b540-333ff4f75588 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.549786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8f58b-b331-413f-b540-333ff4f75588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:03.551888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8f58b-b331-413f-b540-333ff4f75588 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:03.573310 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3257/6518] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:03.575494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8f58b-b331-413f-b540-333ff4f75588 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:03.579249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851456e-6afa-4671-bee5-c6fe47868f6e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:03.590565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc9621-1f29-44d7-9b4c-7951a7cdade2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:03.592564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc9621-1f29-44d7-9b4c-7951a7cdade2 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.592964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc9621-1f29-44d7-9b4c-7951a7cdade2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.594690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc9621-1f29-44d7-9b4c-7951a7cdade2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.600466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8f58b-b331-413f-b540-333ff4f75588 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:03.615188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e6ec6-d4ec-4a94-8441-7c872700c16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:03.623305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e6ec6-d4ec-4a94-8441-7c872700c16e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.623305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e6ec6-d4ec-4a94-8441-7c872700c16e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.623305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e6ec6-d4ec-4a94-8441-7c872700c16e admin admin] PATH_INFO: `/v3/projects/5cfaff69dc084157a616e7d16c0e8569/users/0f9276faf05349aeb1469250b2e199d5/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.640066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc9621-1f29-44d7-9b4c-7951a7cdade2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:03.651543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc9621-1f29-44d7-9b4c-7951a7cdade2 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:03.652510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e6ec6-d4ec-4a94-8441-7c872700c16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cfaff69dc084157a616e7d16c0e8569, user_id=0f9276faf05349aeb1469250b2e199d5, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:03.656875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e6ec6-d4ec-4a94-8441-7c872700c16e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:03.692820 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3262/6519] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:03.697355 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3258/6520] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:03 2022] PUT /identity/v3/projects/5cfaff69dc084157a616e7d16c0e8569/users/0f9276faf05349aeb1469250b2e199d5/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:16:03.700890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932736b-6adb-46c0-9dae-024b7d7ac1b4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.701398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932736b-6adb-46c0-9dae-024b7d7ac1b4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.701784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932736b-6adb-46c0-9dae-024b7d7ac1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.756100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9932736b-6adb-46c0-9dae-024b7d7ac1b4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:03.758728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9932736b-6adb-46c0-9dae-024b7d7ac1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:03.844659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:03.888939 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3263/6521] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:03.912533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e6ec6-d4ec-4a94-8441-7c872700c16e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:03.919653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932736b-6adb-46c0-9dae-024b7d7ac1b4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:03.924850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9932736b-6adb-46c0-9dae-024b7d7ac1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:03.954043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932736b-6adb-46c0-9dae-024b7d7ac1b4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:03.967457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:03.970422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:03.970561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:03.970802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] PATH_INFO: `/v3/projects/dcfe574e0a314efe9ce779ca66bde1ee` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:03.974382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcfe574e0a314efe9ce779ca66bde1ee)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:03.978230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:04.086707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e6ec6-d4ec-4a94-8441-7c872700c16e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:04.118922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcfe574e0a314efe9ce779ca66bde1ee', 'request_id': 'req-59bdd130-eddc-406f-abdc-606882ec644a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.130321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcfe574e0a314efe9ce779ca66bde1ee', 'request_id': 'req-59bdd130-eddc-406f-abdc-606882ec644a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.142822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcfe574e0a314efe9ce779ca66bde1ee', 'request_id': 'req-59bdd130-eddc-406f-abdc-606882ec644a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.165887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] The token cache is being invalidate because project dcfe574e0a314efe9ce779ca66bde1ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:04.166289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.169070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59bdd130-eddc-406f-abdc-606882ec644a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.178437 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3264/6522] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:03 2022] DELETE /identity/v3/projects/dcfe574e0a314efe9ce779ca66bde1ee => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:04.180772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc900e3-bc64-4b1b-9008-1f10cbe8de5b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:04.182268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc900e3-bc64-4b1b-9008-1f10cbe8de5b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:04.182647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc900e3-bc64-4b1b-9008-1f10cbe8de5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:04.227582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5fc900e3-bc64-4b1b-9008-1f10cbe8de5b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:04.229978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5fc900e3-bc64-4b1b-9008-1f10cbe8de5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:04.283910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b426c8d-d37c-4d2a-82fa-6aef475bcb87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:04.284792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b426c8d-d37c-4d2a-82fa-6aef475bcb87 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:04.285141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b426c8d-d37c-4d2a-82fa-6aef475bcb87 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:04.285527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b426c8d-d37c-4d2a-82fa-6aef475bcb87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:04.287414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6b426c8d-d37c-4d2a-82fa-6aef475bcb87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:04.391144 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3265/6523] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:04.393872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc900e3-bc64-4b1b-9008-1f10cbe8de5b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:04.397148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5fc900e3-bc64-4b1b-9008-1f10cbe8de5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:04.424729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc900e3-bc64-4b1b-9008-1f10cbe8de5b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:04.438666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:04.439321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:04.439598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:04.439876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] PATH_INFO: `/v3/projects/2605ac87a00a438da8cf5da292c905ce` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:04.444215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2605ac87a00a438da8cf5da292c905ce)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:04.447057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:04.470122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b426c8d-d37c-4d2a-82fa-6aef475bcb87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:04.471831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b426c8d-d37c-4d2a-82fa-6aef475bcb87 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:04.484265 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3259/6524] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:04.485401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b426c8d-d37c-4d2a-82fa-6aef475bcb87 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:04.487667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6b426c8d-d37c-4d2a-82fa-6aef475bcb87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:04.508004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b426c8d-d37c-4d2a-82fa-6aef475bcb87 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:04.516298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:04.517115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:04.517382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:04.517654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] PATH_INFO: `/v3/projects/662bfb99c7c6456eb782cdc2d74c95a4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:04.518925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=662bfb99c7c6456eb782cdc2d74c95a4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:04.521497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:04.598769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:04.614648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2605ac87a00a438da8cf5da292c905ce', 'request_id': 'req-431af4b0-3530-435f-9cf8-7b1491792e59'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.633405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2605ac87a00a438da8cf5da292c905ce', 'request_id': 'req-431af4b0-3530-435f-9cf8-7b1491792e59'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.634911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '662bfb99c7c6456eb782cdc2d74c95a4', 'request_id': 'req-332f3f95-31a1-4d00-8b15-a8ae9d243d40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.645806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2605ac87a00a438da8cf5da292c905ce', 'request_id': 'req-431af4b0-3530-435f-9cf8-7b1491792e59'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.651198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '662bfb99c7c6456eb782cdc2d74c95a4', 'request_id': 'req-332f3f95-31a1-4d00-8b15-a8ae9d243d40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.654585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:04.663793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '662bfb99c7c6456eb782cdc2d74c95a4', 'request_id': 'req-332f3f95-31a1-4d00-8b15-a8ae9d243d40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.669127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] The token cache is being invalidate because project 2605ac87a00a438da8cf5da292c905ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:04.669577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.671271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-431af4b0-3530-435f-9cf8-7b1491792e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.678492 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3266/6525] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:04 2022] DELETE /identity/v3/projects/2605ac87a00a438da8cf5da292c905ce => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:04.680893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8bff-76de-4b2a-ba1f-8a582af8a011 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:04.681400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8bff-76de-4b2a-ba1f-8a582af8a011 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:04.681809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8bff-76de-4b2a-ba1f-8a582af8a011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:04.686394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] The token cache is being invalidate because project 662bfb99c7c6456eb782cdc2d74c95a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:04.686767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.688516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.692265 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3260/6526] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:04 2022] DELETE /identity/v3/projects/662bfb99c7c6456eb782cdc2d74c95a4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:04.692933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:04.696024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:04.708410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-081e8bff-76de-4b2a-ba1f-8a582af8a011 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:04.713075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-081e8bff-76de-4b2a-ba1f-8a582af8a011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:04.801006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f3f95-31a1-4d00-8b15-a8ae9d243d40 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:04.805595 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3267/6527] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:04.807248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e8bff-76de-4b2a-ba1f-8a582af8a011 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:04.809607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-081e8bff-76de-4b2a-ba1f-8a582af8a011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:04.811356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:04.812412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:04.812771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:04.813931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] PATH_INFO: `/v3/users/1aadd87b97c247cb95321664ba0c32a9` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:04.815205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aadd87b97c247cb95321664ba0c32a9)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:04.818233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:04.832414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e8bff-76de-4b2a-ba1f-8a582af8a011 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:04.839801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:04.840766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:04.841191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:04.841568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] PATH_INFO: `/v3/users/4c76eb2493624b3dbaf130d14c9c3153` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:04.842925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c76eb2493624b3dbaf130d14c9c3153)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:04.845411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:04.895676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aadd87b97c247cb95321664ba0c32a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.916619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aadd87b97c247cb95321664ba0c32a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.921343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c76eb2493624b3dbaf130d14c9c3153'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.931316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aadd87b97c247cb95321664ba0c32a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.932297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c76eb2493624b3dbaf130d14c9c3153'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.944152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aadd87b97c247cb95321664ba0c32a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.946233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-960bb457-9632-4214-b52b-e9aa9eae0c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aadd87b97c247cb95321664ba0c32a9'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.954028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c76eb2493624b3dbaf130d14c9c3153'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.957428 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3261/6528] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:04 2022] DELETE /identity/v3/users/1aadd87b97c247cb95321664ba0c32a9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:04.958098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86f05d-634d-4aad-9a3f-f024e488ab2f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:04.958380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86f05d-634d-4aad-9a3f-f024e488ab2f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:04.961256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86f05d-634d-4aad-9a3f-f024e488ab2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:04.967858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c76eb2493624b3dbaf130d14c9c3153'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.968910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c76eb2493624b3dbaf130d14c9c3153'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:04.979596 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3268/6529] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:04 2022] DELETE /identity/v3/users/4c76eb2493624b3dbaf130d14c9c3153 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:04.980214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:04.984907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:05.010027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3a86f05d-634d-4aad-9a3f-f024e488ab2f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:05.010404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3a86f05d-634d-4aad-9a3f-f024e488ab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:05.102710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:05.152657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc4137-7246-430a-958c-5e71decb52ae admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:05.158395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e12a7-7585-4221-abfb-6b6835708148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:05.159064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e12a7-7585-4221-abfb-6b6835708148 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.159340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e12a7-7585-4221-abfb-6b6835708148 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.159713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e12a7-7585-4221-abfb-6b6835708148 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.160363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e12a7-7585-4221-abfb-6b6835708148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:05.160665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e12a7-7585-4221-abfb-6b6835708148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:05.162482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e12a7-7585-4221-abfb-6b6835708148 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:05.174393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:05.181526 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3269/6530] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:16:05.184388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e12a7-7585-4221-abfb-6b6835708148 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:05.191822 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3262/6531] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:05.194900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f494fd9-c360-4bd0-9c54-d923589796b1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.195287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f494fd9-c360-4bd0-9c54-d923589796b1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.196473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f494fd9-c360-4bd0-9c54-d923589796b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.215393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e12a7-7585-4221-abfb-6b6835708148 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:05.224405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ff9fe-7582-4bd0-af77-d2e27d5d9882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:05.225275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ff9fe-7582-4bd0-af77-d2e27d5d9882 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.226220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ff9fe-7582-4bd0-af77-d2e27d5d9882 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.226648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ff9fe-7582-4bd0-af77-d2e27d5d9882 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.227418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ff9fe-7582-4bd0-af77-d2e27d5d9882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:05.231119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ff9fe-7582-4bd0-af77-d2e27d5d9882 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:05.241697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7f494fd9-c360-4bd0-9c54-d923589796b1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:05.246798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f494fd9-c360-4bd0-9c54-d923589796b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:05.268910 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3263/6532] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:16:05.275674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f494fd9-c360-4bd0-9c54-d923589796b1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:05.283486 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3270/6533] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:05 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:05.286559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ff9fe-7582-4bd0-af77-d2e27d5d9882 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:05.295257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f494fd9-c360-4bd0-9c54-d923589796b1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:05.393741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28152f-d0e6-40ee-936f-30f5aac21b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:05.397070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28152f-d0e6-40ee-936f-30f5aac21b06 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.397722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28152f-d0e6-40ee-936f-30f5aac21b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.398184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28152f-d0e6-40ee-936f-30f5aac21b06 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.399914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28152f-d0e6-40ee-936f-30f5aac21b06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:05.400396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28152f-d0e6-40ee-936f-30f5aac21b06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:05.403323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28152f-d0e6-40ee-936f-30f5aac21b06 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:05.411037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ff9fe-7582-4bd0-af77-d2e27d5d9882 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:05.419901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c2d8a-39ad-4b07-b630-f4c6bd50d6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:05.420346 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3264/6534] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:16:05.420530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c2d8a-39ad-4b07-b630-f4c6bd50d6bd admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.420804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c2d8a-39ad-4b07-b630-f4c6bd50d6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.421120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c2d8a-39ad-4b07-b630-f4c6bd50d6bd admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.421743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c2d8a-39ad-4b07-b630-f4c6bd50d6bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:05.424197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c2d8a-39ad-4b07-b630-f4c6bd50d6bd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:05.424544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28152f-d0e6-40ee-936f-30f5aac21b06 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:05.450054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28152f-d0e6-40ee-936f-30f5aac21b06 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:05.454882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0e001-473a-4665-abdc-23f9b0487a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:05.455599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0e001-473a-4665-abdc-23f9b0487a36 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.455888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0e001-473a-4665-abdc-23f9b0487a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.456160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0e001-473a-4665-abdc-23f9b0487a36 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.456796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0e001-473a-4665-abdc-23f9b0487a36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:05.458937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0e001-473a-4665-abdc-23f9b0487a36 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:05.483577 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3271/6535] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:05 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:05.485770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c2d8a-39ad-4b07-b630-f4c6bd50d6bd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:05.488254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ca3c2d8a-39ad-4b07-b630-f4c6bd50d6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:05.507634 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3265/6536] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:05 2022] POST /identity/v3/projects => generated 395 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:16:05.507634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0e001-473a-4665-abdc-23f9b0487a36 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:05.519873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c2d8a-39ad-4b07-b630-f4c6bd50d6bd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:05.630199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:05.631110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.631554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.632871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] PATH_INFO: `/v3/projects/38b9244bbab846b7baf42985cebc72c2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.635055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b9244bbab846b7baf42985cebc72c2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:05.637385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:05.642798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0e001-473a-4665-abdc-23f9b0487a36 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:05.650671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9fe32-fa53-43aa-9aa9-82316e88bd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:05.651515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9fe32-fa53-43aa-9aa9-82316e88bd6e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.651802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9fe32-fa53-43aa-9aa9-82316e88bd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.652070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9fe32-fa53-43aa-9aa9-82316e88bd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.653238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9fe32-fa53-43aa-9aa9-82316e88bd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:05.656285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9fe32-fa53-43aa-9aa9-82316e88bd6e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:05.674403 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3266/6537] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:05.680097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9fe32-fa53-43aa-9aa9-82316e88bd6e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:05.708231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9fe32-fa53-43aa-9aa9-82316e88bd6e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:05.722608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc403c-6d05-440a-b8b1-379ff9f5e836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:05.725292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc403c-6d05-440a-b8b1-379ff9f5e836 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.726786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc403c-6d05-440a-b8b1-379ff9f5e836 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.741745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc403c-6d05-440a-b8b1-379ff9f5e836 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.743891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc403c-6d05-440a-b8b1-379ff9f5e836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:05.745914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b9244bbab846b7baf42985cebc72c2', 'request_id': 'req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:05.758474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc403c-6d05-440a-b8b1-379ff9f5e836 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:05.759227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b9244bbab846b7baf42985cebc72c2', 'request_id': 'req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:05.773680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b9244bbab846b7baf42985cebc72c2', 'request_id': 'req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:05.804256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:05.814313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] The token cache is being invalidate because project 38b9244bbab846b7baf42985cebc72c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:05.814313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:05.814524 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3267/6538] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:05 2022] POST /identity/v3/users => generated 298 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:05.815089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8adf19-bf9d-4be0-a428-6545ef53a768 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.815089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8adf19-bf9d-4be0-a428-6545ef53a768 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.815320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8adf19-bf9d-4be0-a428-6545ef53a768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.815893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:05.821095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3272/6539] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:05 2022] DELETE /identity/v3/projects/38b9244bbab846b7baf42985cebc72c2 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:05.825242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:05.837342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:05.859007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ae8adf19-bf9d-4be0-a428-6545ef53a768 None None] MFA Rules not processed for user `379c929d93b54b9a9751defeef161f57`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:05.860841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ae8adf19-bf9d-4be0-a428-6545ef53a768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:05.886448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:05.940049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdc174-ba74-4ac5-9886-8c2d8b81ac0c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:05.948065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8ee6a-9a35-4f2a-b4b1-90abcb66ed0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:05.949086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ee6a-9a35-4f2a-b4b1-90abcb66ed0f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:05.949335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ee6a-9a35-4f2a-b4b1-90abcb66ed0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:05.949712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ee6a-9a35-4f2a-b4b1-90abcb66ed0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:05.950492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8ee6a-9a35-4f2a-b4b1-90abcb66ed0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:05.952299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8ee6a-9a35-4f2a-b4b1-90abcb66ed0f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:05.968892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3273/6540] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:05.968892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8ee6a-9a35-4f2a-b4b1-90abcb66ed0f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:05.969651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6ab8ee6a-9a35-4f2a-b4b1-90abcb66ed0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:05.982871 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3268/6541] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:05.984351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8adf19-bf9d-4be0-a428-6545ef53a768 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:05.986316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ae8adf19-bf9d-4be0-a428-6545ef53a768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:06.009323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8ee6a-9a35-4f2a-b4b1-90abcb66ed0f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:06.024456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:06.025314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.025604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.025876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] PATH_INFO: `/v3/projects/3e4c7836a8b3415bab71fd09bf534ace` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.028607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4c7836a8b3415bab71fd09bf534ace)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:06.032778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:06.112608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8adf19-bf9d-4be0-a428-6545ef53a768 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:06.132517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e791b-09e8-4548-95ec-dfa9a4987c21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:06.133223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e791b-09e8-4548-95ec-dfa9a4987c21 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.133563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e791b-09e8-4548-95ec-dfa9a4987c21 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.133903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e791b-09e8-4548-95ec-dfa9a4987c21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.136984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e7e791b-09e8-4548-95ec-dfa9a4987c21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:06.158097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e791b-09e8-4548-95ec-dfa9a4987c21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:06.161181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e791b-09e8-4548-95ec-dfa9a4987c21 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:06.179855 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3269/6542] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:06.181236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e791b-09e8-4548-95ec-dfa9a4987c21 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:06.183012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e7e791b-09e8-4548-95ec-dfa9a4987c21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:06.200739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e791b-09e8-4548-95ec-dfa9a4987c21 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:06.213708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4c7836a8b3415bab71fd09bf534ace', 'request_id': 'req-8fe20986-e284-405b-9429-9e4bfcb11bda'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:06.232020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4c7836a8b3415bab71fd09bf534ace', 'request_id': 'req-8fe20986-e284-405b-9429-9e4bfcb11bda'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:06.247440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4c7836a8b3415bab71fd09bf534ace', 'request_id': 'req-8fe20986-e284-405b-9429-9e4bfcb11bda'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:06.289335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] The token cache is being invalidate because project 3e4c7836a8b3415bab71fd09bf534ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:06.291238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:06.292225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8fe20986-e284-405b-9429-9e4bfcb11bda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:06.295984 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3274/6543] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:05 2022] DELETE /identity/v3/projects/3e4c7836a8b3415bab71fd09bf534ace => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:06.297628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730b013-c194-4c7e-be11-79024ec50216 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.297971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730b013-c194-4c7e-be11-79024ec50216 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.298397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730b013-c194-4c7e-be11-79024ec50216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.354439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f730b013-c194-4c7e-be11-79024ec50216 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:06.365956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f730b013-c194-4c7e-be11-79024ec50216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:06.427569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b697b-44da-40ef-a725-8fbda4235f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:06.428308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b697b-44da-40ef-a725-8fbda4235f93 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.429472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b697b-44da-40ef-a725-8fbda4235f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.429757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b697b-44da-40ef-a725-8fbda4235f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.430626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b697b-44da-40ef-a725-8fbda4235f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:06.432980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b697b-44da-40ef-a725-8fbda4235f93 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:06.468854 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3270/6544] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:06.483601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b697b-44da-40ef-a725-8fbda4235f93 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:06.493846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-242b697b-44da-40ef-a725-8fbda4235f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:06.526673 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3275/6545] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:06.528944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d58a-1aba-4425-9023-77f35f46a06f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.529960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d58a-1aba-4425-9023-77f35f46a06f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.530237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d58a-1aba-4425-9023-77f35f46a06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.540032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b697b-44da-40ef-a725-8fbda4235f93 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:06.562699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb36ee2-ba4e-4635-8143-7f6b98c27cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:06.564602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb36ee2-ba4e-4635-8143-7f6b98c27cac admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.564896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb36ee2-ba4e-4635-8143-7f6b98c27cac admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.565201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb36ee2-ba4e-4635-8143-7f6b98c27cac admin admin] PATH_INFO: `/v3/projects/29df4ff7640541869b91114e44a57507/users/37792ba657664eabac44fde87370c178/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.575352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-62a3d58a-1aba-4425-9023-77f35f46a06f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:06.576586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-62a3d58a-1aba-4425-9023-77f35f46a06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:06.594897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb36ee2-ba4e-4635-8143-7f6b98c27cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=29df4ff7640541869b91114e44a57507, user_id=37792ba657664eabac44fde87370c178, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:06.598455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb36ee2-ba4e-4635-8143-7f6b98c27cac admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:06.606518 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3276/6546] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:06.607236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d58a-1aba-4425-9023-77f35f46a06f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:06.608735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-62a3d58a-1aba-4425-9023-77f35f46a06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:06.621724 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3271/6547] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:06 2022] PUT /identity/v3/projects/29df4ff7640541869b91114e44a57507/users/37792ba657664eabac44fde87370c178/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:06.623530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520a423-1058-4ba9-a5de-ef7e93674cfb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.624233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520a423-1058-4ba9-a5de-ef7e93674cfb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.624728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520a423-1058-4ba9-a5de-ef7e93674cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.638863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d58a-1aba-4425-9023-77f35f46a06f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:06.660557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:06.672725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2520a423-1058-4ba9-a5de-ef7e93674cfb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:06.675408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2520a423-1058-4ba9-a5de-ef7e93674cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:06.719579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:06.764260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e49d9-938f-41bd-8208-d6b355b08ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:06.764894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e49d9-938f-41bd-8208-d6b355b08ebc admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.765194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e49d9-938f-41bd-8208-d6b355b08ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.765459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e49d9-938f-41bd-8208-d6b355b08ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.768296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e49d9-938f-41bd-8208-d6b355b08ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:06.771759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e49d9-938f-41bd-8208-d6b355b08ebc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:06.790091 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3277/6548] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:06.790091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865b29-8886-47d2-84f4-bd9be202b0b9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.790091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865b29-8886-47d2-84f4-bd9be202b0b9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.790091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865b29-8886-47d2-84f4-bd9be202b0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.825509 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3272/6549] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:06.827150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeaf34-df5d-4013-9d88-629214243c4b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.828665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeaf34-df5d-4013-9d88-629214243c4b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.832991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeaf34-df5d-4013-9d88-629214243c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.850068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cd865b29-8886-47d2-84f4-bd9be202b0b9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:06.852148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cd865b29-8886-47d2-84f4-bd9be202b0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:06.862042 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3278/6550] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:06.865969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd865b29-8886-47d2-84f4-bd9be202b0b9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:06.881084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd865b29-8886-47d2-84f4-bd9be202b0b9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:06.889153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0aba5-a02e-466b-871c-a43a5afba197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:06.890379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0aba5-a02e-466b-871c-a43a5afba197 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:06.890594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0aba5-a02e-466b-871c-a43a5afba197 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:06.890873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0aba5-a02e-466b-871c-a43a5afba197 admin admin] PATH_INFO: `/v3/projects/eabfc9e340d64291b5c94181d0d5bd65/users/9e9264a86ff940fab5b64eedca06fff1/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:06.912870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0aba5-a02e-466b-871c-a43a5afba197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eabfc9e340d64291b5c94181d0d5bd65, user_id=9e9264a86ff940fab5b64eedca06fff1, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:06.914208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-acdeaf34-df5d-4013-9d88-629214243c4b None None] MFA Rules not processed for user `37792ba657664eabac44fde87370c178`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:06.915754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-acdeaf34-df5d-4013-9d88-629214243c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:06.917152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0aba5-a02e-466b-871c-a43a5afba197 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:06.967719 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3279/6551] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:06 2022] PUT /identity/v3/projects/eabfc9e340d64291b5c94181d0d5bd65/users/9e9264a86ff940fab5b64eedca06fff1/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:06.967719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0aba5-a02e-466b-871c-a43a5afba197 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.092610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0aba5-a02e-466b-871c-a43a5afba197 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.098634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7305789-07c7-40e3-800f-9988ea13c8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.099434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7305789-07c7-40e3-800f-9988ea13c8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.099944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7305789-07c7-40e3-800f-9988ea13c8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.100405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7305789-07c7-40e3-800f-9988ea13c8e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.102540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7305789-07c7-40e3-800f-9988ea13c8e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:07.103068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7305789-07c7-40e3-800f-9988ea13c8e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.105292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7305789-07c7-40e3-800f-9988ea13c8e5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.108051 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3273/6552] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:07.110806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeaf34-df5d-4013-9d88-629214243c4b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.129197 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3280/6553] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:16:07.130878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e6f6f-6099-4140-a567-eafed0e79259 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.131176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e6f6f-6099-4140-a567-eafed0e79259 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.131627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e6f6f-6099-4140-a567-eafed0e79259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.138237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeaf34-df5d-4013-9d88-629214243c4b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.163094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8904a90-c8f0-408e-8760-a3f3e3d4f4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.164002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8904a90-c8f0-408e-8760-a3f3e3d4f4ce admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.164313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8904a90-c8f0-408e-8760-a3f3e3d4f4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.164581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8904a90-c8f0-408e-8760-a3f3e3d4f4ce admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.165415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8904a90-c8f0-408e-8760-a3f3e3d4f4ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:07.165722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8904a90-c8f0-408e-8760-a3f3e3d4f4ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.169397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8904a90-c8f0-408e-8760-a3f3e3d4f4ce admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.187920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-318e6f6f-6099-4140-a567-eafed0e79259 None None] MFA Rules not processed for user `9e9264a86ff940fab5b64eedca06fff1`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:07.208823 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3274/6554] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:16:07.208986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-318e6f6f-6099-4140-a567-eafed0e79259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:07.210482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8904a90-c8f0-408e-8760-a3f3e3d4f4ce admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.238940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8904a90-c8f0-408e-8760-a3f3e3d4f4ce admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.252199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b647f0-57e9-4bdb-b036-f959577f6cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.252820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b647f0-57e9-4bdb-b036-f959577f6cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.253118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b647f0-57e9-4bdb-b036-f959577f6cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.253389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b647f0-57e9-4bdb-b036-f959577f6cd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.254033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b647f0-57e9-4bdb-b036-f959577f6cd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.255994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b647f0-57e9-4bdb-b036-f959577f6cd2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.324227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:07.325624 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3275/6555] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:07 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:07.326509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b647f0-57e9-4bdb-b036-f959577f6cd2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.335211 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3281/6556] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:07.335211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e6f6f-6099-4140-a567-eafed0e79259 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.353263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e6f6f-6099-4140-a567-eafed0e79259 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.448492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3169feb-eb67-4def-b86c-ca6200c4c50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.449500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3169feb-eb67-4def-b86c-ca6200c4c50e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.449781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3169feb-eb67-4def-b86c-ca6200c4c50e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.450055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3169feb-eb67-4def-b86c-ca6200c4c50e admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.450743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3169feb-eb67-4def-b86c-ca6200c4c50e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.453053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3169feb-eb67-4def-b86c-ca6200c4c50e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.461386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b647f0-57e9-4bdb-b036-f959577f6cd2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.467910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc994e7-df95-496f-9604-08dc9d19ec96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.468754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc994e7-df95-496f-9604-08dc9d19ec96 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.469710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc994e7-df95-496f-9604-08dc9d19ec96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.470395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc994e7-df95-496f-9604-08dc9d19ec96 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.471458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc994e7-df95-496f-9604-08dc9d19ec96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.474301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc994e7-df95-496f-9604-08dc9d19ec96 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.490756 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3282/6557] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:07 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:07.492801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3169feb-eb67-4def-b86c-ca6200c4c50e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.522742 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3276/6558] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:07 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:07.522742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc994e7-df95-496f-9604-08dc9d19ec96 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.525265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dfc994e7-df95-496f-9604-08dc9d19ec96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:07.540112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:07.590976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3169feb-eb67-4def-b86c-ca6200c4c50e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.624437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc994e7-df95-496f-9604-08dc9d19ec96 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.630226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144589f-b6b8-4a71-a3b5-9fd3ec832a34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.631378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144589f-b6b8-4a71-a3b5-9fd3ec832a34 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.632326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144589f-b6b8-4a71-a3b5-9fd3ec832a34 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.632915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144589f-b6b8-4a71-a3b5-9fd3ec832a34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.680117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858dcee-c460-4223-94cd-c4c59b3c3d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.680678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858dcee-c460-4223-94cd-c4c59b3c3d12 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.680953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858dcee-c460-4223-94cd-c4c59b3c3d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.681614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858dcee-c460-4223-94cd-c4c59b3c3d12 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.682407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858dcee-c460-4223-94cd-c4c59b3c3d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.685459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858dcee-c460-4223-94cd-c4c59b3c3d12 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.716441 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3283/6559] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:07 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:07.717750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858dcee-c460-4223-94cd-c4c59b3c3d12 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.718392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144589f-b6b8-4a71-a3b5-9fd3ec832a34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.720020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144589f-b6b8-4a71-a3b5-9fd3ec832a34 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.731779 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3277/6560] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:07 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:07.732833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858dcee-c460-4223-94cd-c4c59b3c3d12 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.733173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144589f-b6b8-4a71-a3b5-9fd3ec832a34 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.739889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e1cf4-bfff-4b04-8c94-ff59a0258bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.740771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e1cf4-bfff-4b04-8c94-ff59a0258bea admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.742283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e1cf4-bfff-4b04-8c94-ff59a0258bea admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.742705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e1cf4-bfff-4b04-8c94-ff59a0258bea admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.743488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e1cf4-bfff-4b04-8c94-ff59a0258bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.746194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e1cf4-bfff-4b04-8c94-ff59a0258bea admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.746576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144589f-b6b8-4a71-a3b5-9fd3ec832a34 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.753086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28804c6-5d98-45d9-b7a9-0ec52b3f2758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.753993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28804c6-5d98-45d9-b7a9-0ec52b3f2758 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.754237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28804c6-5d98-45d9-b7a9-0ec52b3f2758 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.754514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28804c6-5d98-45d9-b7a9-0ec52b3f2758 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.755149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28804c6-5d98-45d9-b7a9-0ec52b3f2758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.758166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28804c6-5d98-45d9-b7a9-0ec52b3f2758 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.787218 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3284/6561] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:07 2022] POST /identity/v3/users => generated 292 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:07.788163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e1cf4-bfff-4b04-8c94-ff59a0258bea admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.796252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:07.800361 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3278/6562] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:07 2022] POST /identity/v3/users => generated 291 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:07.802445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28804c6-5d98-45d9-b7a9-0ec52b3f2758 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.805711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e1cf4-bfff-4b04-8c94-ff59a0258bea admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.813829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a15c-5704-41ac-976e-ee99594aa0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.815531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a15c-5704-41ac-976e-ee99594aa0df admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.816172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a15c-5704-41ac-976e-ee99594aa0df admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.816519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a15c-5704-41ac-976e-ee99594aa0df admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.817906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4a15c-5704-41ac-976e-ee99594aa0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.820562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4a15c-5704-41ac-976e-ee99594aa0df admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.827257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28804c6-5d98-45d9-b7a9-0ec52b3f2758 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.838073 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3285/6563] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:07.841078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a15c-5704-41ac-976e-ee99594aa0df admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.842182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca413f-7197-4c09-ac4f-cd4435d90c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.844579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca413f-7197-4c09-ac4f-cd4435d90c15 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.845307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca413f-7197-4c09-ac4f-cd4435d90c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.845812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca413f-7197-4c09-ac4f-cd4435d90c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.847195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca413f-7197-4c09-ac4f-cd4435d90c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.852583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca413f-7197-4c09-ac4f-cd4435d90c15 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.864679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a15c-5704-41ac-976e-ee99594aa0df admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.882436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436089d4-8b79-4a50-a003-c3a140d4c8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.885831 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3279/6564] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:07.885964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436089d4-8b79-4a50-a003-c3a140d4c8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.885964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436089d4-8b79-4a50-a003-c3a140d4c8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.885964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436089d4-8b79-4a50-a003-c3a140d4c8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.885964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436089d4-8b79-4a50-a003-c3a140d4c8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.886921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca413f-7197-4c09-ac4f-cd4435d90c15 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.891563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436089d4-8b79-4a50-a003-c3a140d4c8f3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.907829 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3286/6565] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:07.910074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca413f-7197-4c09-ac4f-cd4435d90c15 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.912218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436089d4-8b79-4a50-a003-c3a140d4c8f3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:07.926706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696bf24-1433-4204-9025-2ce1ffb60e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.927597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696bf24-1433-4204-9025-2ce1ffb60e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.929847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696bf24-1433-4204-9025-2ce1ffb60e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.930294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696bf24-1433-4204-9025-2ce1ffb60e07 admin admin] PATH_INFO: `/v3/projects/ec2340bec88d491f9fc7aae34be13fa8/users/a47a4ab203474a1fb284a8da302bb205/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.943447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436089d4-8b79-4a50-a003-c3a140d4c8f3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:07.957071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43442e1a-5dc7-4efc-8c36-bef9c996952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:07.958601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442e1a-5dc7-4efc-8c36-bef9c996952d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:07.959406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442e1a-5dc7-4efc-8c36-bef9c996952d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:07.960874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442e1a-5dc7-4efc-8c36-bef9c996952d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:07.961701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43442e1a-5dc7-4efc-8c36-bef9c996952d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.967509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43442e1a-5dc7-4efc-8c36-bef9c996952d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:07.978905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696bf24-1433-4204-9025-2ce1ffb60e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2340bec88d491f9fc7aae34be13fa8, user_id=a47a4ab203474a1fb284a8da302bb205, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:07.988111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696bf24-1433-4204-9025-2ce1ffb60e07 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:08.003054 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3287/6566] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:08.004169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43442e1a-5dc7-4efc-8c36-bef9c996952d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:08.024974 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3280/6567] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:07 2022] PUT /identity/v3/projects/ec2340bec88d491f9fc7aae34be13fa8/users/a47a4ab203474a1fb284a8da302bb205/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:08.026084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696bf24-1433-4204-9025-2ce1ffb60e07 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:08.031729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43442e1a-5dc7-4efc-8c36-bef9c996952d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:08.144440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696bf24-1433-4204-9025-2ce1ffb60e07 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:08.151467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ad5c-9895-4e68-b547-1c0a5e6bb206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:08.152321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ad5c-9895-4e68-b547-1c0a5e6bb206 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:08.152721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ad5c-9895-4e68-b547-1c0a5e6bb206 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:08.153361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ad5c-9895-4e68-b547-1c0a5e6bb206 admin admin] PATH_INFO: `/v3/projects/fa69c761ce3444c4b8e286710a87d978/users/2b1288b8dbd9473084d02d9ccd9bdc4e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:08.175770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878f51f-fce8-4db7-a83c-62296d2a2417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:08.176482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878f51f-fce8-4db7-a83c-62296d2a2417 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:08.176763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878f51f-fce8-4db7-a83c-62296d2a2417 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:08.179296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878f51f-fce8-4db7-a83c-62296d2a2417 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:08.180491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878f51f-fce8-4db7-a83c-62296d2a2417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:08.183305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878f51f-fce8-4db7-a83c-62296d2a2417 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:08.225308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:08.242618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657ad5c-9895-4e68-b547-1c0a5e6bb206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa69c761ce3444c4b8e286710a87d978, user_id=2b1288b8dbd9473084d02d9ccd9bdc4e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:08.250794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657ad5c-9895-4e68-b547-1c0a5e6bb206 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:08.254147 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3288/6568] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:08.262872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878f51f-fce8-4db7-a83c-62296d2a2417 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:08.298142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878f51f-fce8-4db7-a83c-62296d2a2417 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:08.302515 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3281/6569] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:08 2022] PUT /identity/v3/projects/fa69c761ce3444c4b8e286710a87d978/users/2b1288b8dbd9473084d02d9ccd9bdc4e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:08.306464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ad5c-9895-4e68-b547-1c0a5e6bb206 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:08.466123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9b07e-a04e-4437-a10a-d24a13c92858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:08.481494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9b07e-a04e-4437-a10a-d24a13c92858 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:08.481494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9b07e-a04e-4437-a10a-d24a13c92858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:08.481494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9b07e-a04e-4437-a10a-d24a13c92858 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:08.481494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9b07e-a04e-4437-a10a-d24a13c92858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:08.481494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9b07e-a04e-4437-a10a-d24a13c92858 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:08.501858 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3289/6570] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:08.502478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a658-4e45-4fcf-aff1-688ebda3c174 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:08.504181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a658-4e45-4fcf-aff1-688ebda3c174 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:08.504447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a658-4e45-4fcf-aff1-688ebda3c174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:08.534958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ad5c-9895-4e68-b547-1c0a5e6bb206 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:08.549493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fecdc7-1554-485e-b814-8fff666197c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:08.550645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fecdc7-1554-485e-b814-8fff666197c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:08.552494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fecdc7-1554-485e-b814-8fff666197c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:08.553023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fecdc7-1554-485e-b814-8fff666197c6 admin admin] PATH_INFO: `/v3/projects/8d0dcf5fc5e0456a9e14fc7edbda53cc/users/655b3dac7ef04774a94c1294ad38b292/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:08.603544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fd51a658-4e45-4fcf-aff1-688ebda3c174 None None] MFA Rules not processed for user `2b1288b8dbd9473084d02d9ccd9bdc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:08.607430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd51a658-4e45-4fcf-aff1-688ebda3c174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:08.617534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fecdc7-1554-485e-b814-8fff666197c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0dcf5fc5e0456a9e14fc7edbda53cc, user_id=655b3dac7ef04774a94c1294ad38b292, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:08.628606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fecdc7-1554-485e-b814-8fff666197c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:08.668698 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3282/6571] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:08 2022] PUT /identity/v3/projects/8d0dcf5fc5e0456a9e14fc7edbda53cc/users/655b3dac7ef04774a94c1294ad38b292/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:08.670970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fecdc7-1554-485e-b814-8fff666197c6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:08.757277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:08.770197 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3290/6572] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:08.771715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6b91-c6e0-4e4d-85c2-83fcc13a3226 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:08.771993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6b91-c6e0-4e4d-85c2-83fcc13a3226 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:08.772264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6b91-c6e0-4e4d-85c2-83fcc13a3226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:08.796128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e29c6b91-c6e0-4e4d-85c2-83fcc13a3226 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:08.797962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e29c6b91-c6e0-4e4d-85c2-83fcc13a3226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:08.852855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fecdc7-1554-485e-b814-8fff666197c6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:08.865098 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3291/6573] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:08.868779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271dca1-75a0-41c9-8ab0-351d6f442fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:08.869022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c6b91-c6e0-4e4d-85c2-83fcc13a3226 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:08.869132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271dca1-75a0-41c9-8ab0-351d6f442fab admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:08.869132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271dca1-75a0-41c9-8ab0-351d6f442fab admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:08.869132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271dca1-75a0-41c9-8ab0-351d6f442fab admin admin] PATH_INFO: `/v3/projects/ec2340bec88d491f9fc7aae34be13fa8/users/a47a4ab203474a1fb284a8da302bb205/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:08.874039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271dca1-75a0-41c9-8ab0-351d6f442fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2340bec88d491f9fc7aae34be13fa8, user_id=a47a4ab203474a1fb284a8da302bb205, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:08.876571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c6b91-c6e0-4e4d-85c2-83fcc13a3226 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:08.877998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271dca1-75a0-41c9-8ab0-351d6f442fab admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:08.882361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b4d7e-75fc-4d1b-9073-309c519d4a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:08.882981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b4d7e-75fc-4d1b-9073-309c519d4a8e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:08.883262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b4d7e-75fc-4d1b-9073-309c519d4a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:08.883516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b4d7e-75fc-4d1b-9073-309c519d4a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:08.884151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b4d7e-75fc-4d1b-9073-309c519d4a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:08.886288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b4d7e-75fc-4d1b-9073-309c519d4a8e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:08.916736 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3283/6574] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:08 2022] PUT /identity/v3/projects/ec2340bec88d491f9fc7aae34be13fa8/users/a47a4ab203474a1fb284a8da302bb205/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:08.922610 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3292/6575] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:08 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:08.925635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271dca1-75a0-41c9-8ab0-351d6f442fab admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:08.926397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424308-8807-4f4a-8a44-2b61afbb18a4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:08.926789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424308-8807-4f4a-8a44-2b61afbb18a4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:08.927161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424308-8807-4f4a-8a44-2b61afbb18a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:08.964903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bb424308-8807-4f4a-8a44-2b61afbb18a4 None None] MFA Rules not processed for user `a47a4ab203474a1fb284a8da302bb205`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:08.966707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bb424308-8807-4f4a-8a44-2b61afbb18a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:09.089793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271dca1-75a0-41c9-8ab0-351d6f442fab admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:09.101673 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3293/6576] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:09.104862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb424308-8807-4f4a-8a44-2b61afbb18a4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:09.107952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bb424308-8807-4f4a-8a44-2b61afbb18a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:09.129820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:09.204730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb424308-8807-4f4a-8a44-2b61afbb18a4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:09.212770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400350d0-0315-49ff-b299-ddceddc2274e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:09.212770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400350d0-0315-49ff-b299-ddceddc2274e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:09.213008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400350d0-0315-49ff-b299-ddceddc2274e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:09.213971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400350d0-0315-49ff-b299-ddceddc2274e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:09.226245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:09.297261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a538f6c-4102-44dd-a17d-ecbe3483197b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:09.299214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a538f6c-4102-44dd-a17d-ecbe3483197b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:09.301845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a538f6c-4102-44dd-a17d-ecbe3483197b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:09.303339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a538f6c-4102-44dd-a17d-ecbe3483197b admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:09.305420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400350d0-0315-49ff-b299-ddceddc2274e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:09.307625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a538f6c-4102-44dd-a17d-ecbe3483197b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:09.308288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400350d0-0315-49ff-b299-ddceddc2274e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:09.317651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a538f6c-4102-44dd-a17d-ecbe3483197b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:09.325029 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3294/6577] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:09.327028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400350d0-0315-49ff-b299-ddceddc2274e service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:09.328577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-400350d0-0315-49ff-b299-ddceddc2274e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:09.402945 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3284/6578] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:08 2022] POST /identity/v3/users => generated 298 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:09.407029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a538f6c-4102-44dd-a17d-ecbe3483197b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:09.445974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a538f6c-4102-44dd-a17d-ecbe3483197b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:09.454161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400350d0-0315-49ff-b299-ddceddc2274e service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:09.460287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b933e-0f63-4149-82b8-73778c8cae91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:09.460837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b933e-0f63-4149-82b8-73778c8cae91 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:09.461142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b933e-0f63-4149-82b8-73778c8cae91 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:09.461420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b933e-0f63-4149-82b8-73778c8cae91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:09.461709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48033db2-deb0-4038-b352-92dd27ecaa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:09.462337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48033db2-deb0-4038-b352-92dd27ecaa55 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:09.462614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48033db2-deb0-4038-b352-92dd27ecaa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:09.462877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48033db2-deb0-4038-b352-92dd27ecaa55 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:09.464101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48033db2-deb0-4038-b352-92dd27ecaa55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:09.470196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48033db2-deb0-4038-b352-92dd27ecaa55 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:09.485648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b933e-0f63-4149-82b8-73778c8cae91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:09.487119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b933e-0f63-4149-82b8-73778c8cae91 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:09.513163 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3285/6579] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:09.514201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48033db2-deb0-4038-b352-92dd27ecaa55 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:09.537392 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3295/6580] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:09.589099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48033db2-deb0-4038-b352-92dd27ecaa55 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:09.651324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ccc03-5374-4a07-a6de-d1e774ca3aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:09.652749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ccc03-5374-4a07-a6de-d1e774ca3aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:09.653063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ccc03-5374-4a07-a6de-d1e774ca3aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:09.654123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ccc03-5374-4a07-a6de-d1e774ca3aa3 admin admin] PATH_INFO: `/v3/projects/3b890c57ad0d4fd9852974d0aaf33a15/users/96b5a434378c427caaac9c33f6eb0e43/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:09.687920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b933e-0f63-4149-82b8-73778c8cae91 service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:09.715448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b933e-0f63-4149-82b8-73778c8cae91 service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:09.733566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:09.734221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:09.734500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:09.734772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] PATH_INFO: `/v3/users/37792ba657664eabac44fde87370c178` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:09.738498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37792ba657664eabac44fde87370c178)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:09.740024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ccc03-5374-4a07-a6de-d1e774ca3aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b890c57ad0d4fd9852974d0aaf33a15, user_id=96b5a434378c427caaac9c33f6eb0e43, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:09.745947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:09.748265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ccc03-5374-4a07-a6de-d1e774ca3aa3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:09.805446 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3286/6581] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:09 2022] PUT /identity/v3/projects/3b890c57ad0d4fd9852974d0aaf33a15/users/96b5a434378c427caaac9c33f6eb0e43/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:09.810980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ccc03-5374-4a07-a6de-d1e774ca3aa3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:09.985575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37792ba657664eabac44fde87370c178'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:10.016438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37792ba657664eabac44fde87370c178'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:10.053567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37792ba657664eabac44fde87370c178'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:10.061028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ccc03-5374-4a07-a6de-d1e774ca3aa3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:10.072827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db252d-ae5b-4d3c-b0b7-e93bc5ed7e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:10.073482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db252d-ae5b-4d3c-b0b7-e93bc5ed7e2b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:10.073776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db252d-ae5b-4d3c-b0b7-e93bc5ed7e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:10.074008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db252d-ae5b-4d3c-b0b7-e93bc5ed7e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:10.074278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37792ba657664eabac44fde87370c178'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:10.074711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db252d-ae5b-4d3c-b0b7-e93bc5ed7e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:10.075055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fb152430-b422-485a-bed0-46863a569321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37792ba657664eabac44fde87370c178'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:10.076338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db252d-ae5b-4d3c-b0b7-e93bc5ed7e2b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:10.088664 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3296/6582] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:09 2022] DELETE /identity/v3/users/37792ba657664eabac44fde87370c178 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:10.093807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f022cad-6dd3-4f78-b310-c32915e3db44 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:10.094289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f022cad-6dd3-4f78-b310-c32915e3db44 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:10.094570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f022cad-6dd3-4f78-b310-c32915e3db44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:10.127025 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3287/6583] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:10.128498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db252d-ae5b-4d3c-b0b7-e93bc5ed7e2b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:10.134738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-04db252d-ae5b-4d3c-b0b7-e93bc5ed7e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:10.164755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db252d-ae5b-4d3c-b0b7-e93bc5ed7e2b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:10.212583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5f022cad-6dd3-4f78-b310-c32915e3db44 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:10.219824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5f022cad-6dd3-4f78-b310-c32915e3db44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:10.334372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:10.446104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82000e-5a3f-4a59-95af-dae660d480b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:10.447506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82000e-5a3f-4a59-95af-dae660d480b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:10.449209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82000e-5a3f-4a59-95af-dae660d480b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:10.449583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82000e-5a3f-4a59-95af-dae660d480b9 admin admin] PATH_INFO: `/v3/projects/3b890c57ad0d4fd9852974d0aaf33a15/users/96b5a434378c427caaac9c33f6eb0e43/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:10.455655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82000e-5a3f-4a59-95af-dae660d480b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b890c57ad0d4fd9852974d0aaf33a15, user_id=96b5a434378c427caaac9c33f6eb0e43, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:10.458218 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3297/6584] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:10.470071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82000e-5a3f-4a59-95af-dae660d480b9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:10.490670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:10.495774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f022cad-6dd3-4f78-b310-c32915e3db44 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:10.498256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5f022cad-6dd3-4f78-b310-c32915e3db44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:10.500249 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3288/6585] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:10 2022] PUT /identity/v3/projects/3b890c57ad0d4fd9852974d0aaf33a15/users/96b5a434378c427caaac9c33f6eb0e43/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:10.521898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a57c94-6151-453e-8120-f2cea9e4a0f8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:10.523321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a57c94-6151-453e-8120-f2cea9e4a0f8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:10.525689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a57c94-6151-453e-8120-f2cea9e4a0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:10.592848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a2a57c94-6151-453e-8120-f2cea9e4a0f8 None None] MFA Rules not processed for user `96b5a434378c427caaac9c33f6eb0e43`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:10.594191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a2a57c94-6151-453e-8120-f2cea9e4a0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:10.636684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f022cad-6dd3-4f78-b310-c32915e3db44 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:10.731637 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3289/6586] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:10.751257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a57c94-6151-453e-8120-f2cea9e4a0f8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:10.753470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a2a57c94-6151-453e-8120-f2cea9e4a0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:10.799090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7723e-7f3e-4037-aaee-c64fa5d7e7af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:10.799709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7723e-7f3e-4037-aaee-c64fa5d7e7af service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:10.799993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7723e-7f3e-4037-aaee-c64fa5d7e7af service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:10.800260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7723e-7f3e-4037-aaee-c64fa5d7e7af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:10.896465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a57c94-6151-453e-8120-f2cea9e4a0f8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:10.905283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978da56-4bdd-48ef-a83c-30fc202969a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:10.906592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978da56-4bdd-48ef-a83c-30fc202969a8 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:10.906958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978da56-4bdd-48ef-a83c-30fc202969a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:10.907322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978da56-4bdd-48ef-a83c-30fc202969a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:10.922143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978da56-4bdd-48ef-a83c-30fc202969a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:10.924704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978da56-4bdd-48ef-a83c-30fc202969a8 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:10.942939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7723e-7f3e-4037-aaee-c64fa5d7e7af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:10.945414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7723e-7f3e-4037-aaee-c64fa5d7e7af service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:10.946439 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3290/6587] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:16:10.972267 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3298/6588] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:16:11.115412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978da56-4bdd-48ef-a83c-30fc202969a8 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:11.135340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978da56-4bdd-48ef-a83c-30fc202969a8 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:11.139410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a36db-c345-4ce4-92cf-e7aa42dddb28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:11.140019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a36db-c345-4ce4-92cf-e7aa42dddb28 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:11.140301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a36db-c345-4ce4-92cf-e7aa42dddb28 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:11.140571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a36db-c345-4ce4-92cf-e7aa42dddb28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:11.142890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b48a36db-c345-4ce4-92cf-e7aa42dddb28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:11.288912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a36db-c345-4ce4-92cf-e7aa42dddb28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:11.294540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a36db-c345-4ce4-92cf-e7aa42dddb28 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:11.332922 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3291/6589] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:11.900457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7723e-7f3e-4037-aaee-c64fa5d7e7af service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:11.910519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7723e-7f3e-4037-aaee-c64fa5d7e7af service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:11.926677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:11.927522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:11.927903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:11.928280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] PATH_INFO: `/v3/users/2b1288b8dbd9473084d02d9ccd9bdc4e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:11.938151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1288b8dbd9473084d02d9ccd9bdc4e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:11.940231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:11.981948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:11.990129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a36db-c345-4ce4-92cf-e7aa42dddb28 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:11.992178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b48a36db-c345-4ce4-92cf-e7aa42dddb28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:12.022561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a36db-c345-4ce4-92cf-e7aa42dddb28 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:12.030994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1288b8dbd9473084d02d9ccd9bdc4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.034019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:12.034966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:12.035250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:12.035512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] PATH_INFO: `/v3/projects/29df4ff7640541869b91114e44a57507` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:12.038992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=29df4ff7640541869b91114e44a57507)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:12.041512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1288b8dbd9473084d02d9ccd9bdc4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.043058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:12.055862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1288b8dbd9473084d02d9ccd9bdc4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.067206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1288b8dbd9473084d02d9ccd9bdc4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.068146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c66dafb9-c481-4374-b4cd-0b4673c878d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1288b8dbd9473084d02d9ccd9bdc4e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.077505 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3299/6590] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:11 2022] DELETE /identity/v3/users/2b1288b8dbd9473084d02d9ccd9bdc4e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:12.081721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e7cf9-bad9-4781-a670-e0beefbd9982 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:12.082264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e7cf9-bad9-4781-a670-e0beefbd9982 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:12.082768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e7cf9-bad9-4781-a670-e0beefbd9982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:12.099141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:12.112355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ce2e7cf9-bad9-4781-a670-e0beefbd9982 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:12.114451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ce2e7cf9-bad9-4781-a670-e0beefbd9982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:12.223998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29df4ff7640541869b91114e44a57507', 'request_id': 'req-0d0cc225-e157-46ca-aad7-f65d57c06c2c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.247842 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3300/6591] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:16:12.249453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29df4ff7640541869b91114e44a57507', 'request_id': 'req-0d0cc225-e157-46ca-aad7-f65d57c06c2c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.259460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29df4ff7640541869b91114e44a57507', 'request_id': 'req-0d0cc225-e157-46ca-aad7-f65d57c06c2c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.273692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] The token cache is being invalidate because project 29df4ff7640541869b91114e44a57507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:12.274263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.275029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.279216 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3292/6592] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:11 2022] DELETE /identity/v3/projects/29df4ff7640541869b91114e44a57507 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:12.283173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e7cf9-bad9-4781-a670-e0beefbd9982 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:12.284682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ce2e7cf9-bad9-4781-a670-e0beefbd9982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:12.307422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:12.310074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e7cf9-bad9-4781-a670-e0beefbd9982 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:12.310210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:12.466200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:12.466900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:12.467178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:12.467449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] PATH_INFO: `/v3/users/a47a4ab203474a1fb284a8da302bb205` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:12.469380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47a4ab203474a1fb284a8da302bb205)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:12.469858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc225-e157-46ca-aad7-f65d57c06c2c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:12.472002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:12.479713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a985b56-dc76-41c2-9a01-befd9fcec6ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:12.480341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a985b56-dc76-41c2-9a01-befd9fcec6ca service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:12.480613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a985b56-dc76-41c2-9a01-befd9fcec6ca service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:12.480879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a985b56-dc76-41c2-9a01-befd9fcec6ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:12.486977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3a985b56-dc76-41c2-9a01-befd9fcec6ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:12.543249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a985b56-dc76-41c2-9a01-befd9fcec6ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:12.549056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a985b56-dc76-41c2-9a01-befd9fcec6ca service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:12.569360 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3293/6593] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:12.622507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:12.628916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47a4ab203474a1fb284a8da302bb205'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.641612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47a4ab203474a1fb284a8da302bb205'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.661288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47a4ab203474a1fb284a8da302bb205'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.677515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47a4ab203474a1fb284a8da302bb205'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.678614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47a4ab203474a1fb284a8da302bb205'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:12.689459 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3301/6594] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:12 2022] DELETE /identity/v3/users/a47a4ab203474a1fb284a8da302bb205 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:14.293304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a985b56-dc76-41c2-9a01-befd9fcec6ca service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:14.295279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3a985b56-dc76-41c2-9a01-befd9fcec6ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:14.327242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a985b56-dc76-41c2-9a01-befd9fcec6ca service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:14.509971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:14.510641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:14.510913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:14.511182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] PATH_INFO: `/v3/projects/fa69c761ce3444c4b8e286710a87d978` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:14.514344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa69c761ce3444c4b8e286710a87d978)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:14.516130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:14.634724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa69c761ce3444c4b8e286710a87d978', 'request_id': 'req-f512b3aa-555e-4343-9f5d-8bd7360ac34b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:14.639250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:14.650896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa69c761ce3444c4b8e286710a87d978', 'request_id': 'req-f512b3aa-555e-4343-9f5d-8bd7360ac34b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:14.665554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa69c761ce3444c4b8e286710a87d978', 'request_id': 'req-f512b3aa-555e-4343-9f5d-8bd7360ac34b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:14.695191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] The token cache is being invalidate because project fa69c761ce3444c4b8e286710a87d978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:14.695496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:14.697524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:14.702250 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3294/6595] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:14 2022] DELETE /identity/v3/projects/fa69c761ce3444c4b8e286710a87d978 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:14.703970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:14.706053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:14.724370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5f9eb-1dd4-40e2-a866-ad98c88a20ba admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:14.862977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:14.864121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:14.864587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:14.865132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] PATH_INFO: `/v3/users/96b5a434378c427caaac9c33f6eb0e43` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:14.867267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96b5a434378c427caaac9c33f6eb0e43)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:14.869603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:14.963874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:14.966225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:14.980621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96b5a434378c427caaac9c33f6eb0e43'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:14.982464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512b3aa-555e-4343-9f5d-8bd7360ac34b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:14.991780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:14.992482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:14.992761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:14.993051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] PATH_INFO: `/v3/users/9e9264a86ff940fab5b64eedca06fff1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:14.997629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9264a86ff940fab5b64eedca06fff1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:15.004342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:15.006912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96b5a434378c427caaac9c33f6eb0e43'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:15.037016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96b5a434378c427caaac9c33f6eb0e43'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:15.056636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96b5a434378c427caaac9c33f6eb0e43'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:15.057633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ed7a0519-b5f9-49ab-b7cf-47efcaa52220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96b5a434378c427caaac9c33f6eb0e43'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:15.080628 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3302/6596] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:14 2022] DELETE /identity/v3/users/96b5a434378c427caaac9c33f6eb0e43 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:15.127358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9264a86ff940fab5b64eedca06fff1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:15.160742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9264a86ff940fab5b64eedca06fff1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:15.174157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9264a86ff940fab5b64eedca06fff1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:15.189995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9264a86ff940fab5b64eedca06fff1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:15.199798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9264a86ff940fab5b64eedca06fff1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:15.219437 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3295/6597] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:14 2022] DELETE /identity/v3/users/9e9264a86ff940fab5b64eedca06fff1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:15.225159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca910-7d3b-4505-80f9-2ca026f6697f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:15.225851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca910-7d3b-4505-80f9-2ca026f6697f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:15.226514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca910-7d3b-4505-80f9-2ca026f6697f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:15.268776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-63aca910-7d3b-4505-80f9-2ca026f6697f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:15.273021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-63aca910-7d3b-4505-80f9-2ca026f6697f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:15.361928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:15.408504 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3303/6598] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:15.831992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:15.834501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:15.874218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aca910-7d3b-4505-80f9-2ca026f6697f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:15.875858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-63aca910-7d3b-4505-80f9-2ca026f6697f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:15.900046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aca910-7d3b-4505-80f9-2ca026f6697f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:15.908830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:15.909567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:15.909850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:15.910246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] PATH_INFO: `/v3/projects/ec2340bec88d491f9fc7aae34be13fa8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:15.911692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2340bec88d491f9fc7aae34be13fa8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:15.917222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:16.026206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e065-8c09-4d47-804c-b3b1690c11d7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:16.058660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2340bec88d491f9fc7aae34be13fa8', 'request_id': 'req-1190a41d-321d-4327-b5f2-062cb11a89dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:16.070862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2340bec88d491f9fc7aae34be13fa8', 'request_id': 'req-1190a41d-321d-4327-b5f2-062cb11a89dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:16.087239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2340bec88d491f9fc7aae34be13fa8', 'request_id': 'req-1190a41d-321d-4327-b5f2-062cb11a89dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:16.106534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] The token cache is being invalidate because project ec2340bec88d491f9fc7aae34be13fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:16.107203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:16.107981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1190a41d-321d-4327-b5f2-062cb11a89dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:16.110443 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3304/6599] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:15 2022] DELETE /identity/v3/projects/ec2340bec88d491f9fc7aae34be13fa8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:16.118192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8354c4-d0c8-429b-9226-3359832ab231 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:16.118589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8354c4-d0c8-429b-9226-3359832ab231 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:16.118934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8354c4-d0c8-429b-9226-3359832ab231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:16.174182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c6b59-f907-4a3e-a235-7a35c07a70aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:16.175451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c6b59-f907-4a3e-a235-7a35c07a70aa service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:16.175451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c6b59-f907-4a3e-a235-7a35c07a70aa service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:16.175451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c6b59-f907-4a3e-a235-7a35c07a70aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:16.181541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d46c6b59-f907-4a3e-a235-7a35c07a70aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:16.221438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9d8354c4-d0c8-429b-9226-3359832ab231 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:16.224091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9d8354c4-d0c8-429b-9226-3359832ab231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:16.250014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:16.337701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c6b59-f907-4a3e-a235-7a35c07a70aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:16.340809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c6b59-f907-4a3e-a235-7a35c07a70aa service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:16.355164 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3296/6600] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:16.375185 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3305/6601] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:16.384416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bd127-f67f-4b31-88d1-7327f6afa829 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:16.385014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bd127-f67f-4b31-88d1-7327f6afa829 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:16.385285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bd127-f67f-4b31-88d1-7327f6afa829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:16.419252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-910bd127-f67f-4b31-88d1-7327f6afa829 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:16.420640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-910bd127-f67f-4b31-88d1-7327f6afa829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:16.432886 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3297/6602] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:16.437024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8354c4-d0c8-429b-9226-3359832ab231 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:16.452699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8354c4-d0c8-429b-9226-3359832ab231 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:16.459341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22b5fc-7fe6-465b-9190-b8d8248dcbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:16.461186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22b5fc-7fe6-465b-9190-b8d8248dcbab admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:16.461646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22b5fc-7fe6-465b-9190-b8d8248dcbab admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:16.462021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22b5fc-7fe6-465b-9190-b8d8248dcbab admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:16.462784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22b5fc-7fe6-465b-9190-b8d8248dcbab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:16.464273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22b5fc-7fe6-465b-9190-b8d8248dcbab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:16.477069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22b5fc-7fe6-465b-9190-b8d8248dcbab admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:16.505451 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3306/6603] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:16:16.510268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bd127-f67f-4b31-88d1-7327f6afa829 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:16.524434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bd127-f67f-4b31-88d1-7327f6afa829 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:16.538225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9293329-c722-445e-897b-9b1d7295f3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:16.539273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9293329-c722-445e-897b-9b1d7295f3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:16.539807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9293329-c722-445e-897b-9b1d7295f3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:16.545340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9293329-c722-445e-897b-9b1d7295f3c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:16.546431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9293329-c722-445e-897b-9b1d7295f3c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:16.548899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9293329-c722-445e-897b-9b1d7295f3c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:16.604836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22b5fc-7fe6-465b-9190-b8d8248dcbab admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:16.606012 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3298/6604] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:16 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:16.611759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6e22b5fc-7fe6-465b-9190-b8d8248dcbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:16.612665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9293329-c722-445e-897b-9b1d7295f3c6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:16.642015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22b5fc-7fe6-465b-9190-b8d8248dcbab admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:16.770049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:16.787813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9293329-c722-445e-897b-9b1d7295f3c6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:16.804028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792651bc-08e5-42e9-944f-135f3d2659f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:16.804850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792651bc-08e5-42e9-944f-135f3d2659f2 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:16.804964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:16.805224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792651bc-08e5-42e9-944f-135f3d2659f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:16.805422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792651bc-08e5-42e9-944f-135f3d2659f2 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:16.806236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792651bc-08e5-42e9-944f-135f3d2659f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:16.806600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:16.808278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:16.808536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] PATH_INFO: `/v3/projects/3b890c57ad0d4fd9852974d0aaf33a15` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:16.811051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792651bc-08e5-42e9-944f-135f3d2659f2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:16.821384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b890c57ad0d4fd9852974d0aaf33a15)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:16.823675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:16.867795 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3299/6605] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:16 2022] POST /identity/v3/users => generated 294 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:16.869929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792651bc-08e5-42e9-944f-135f3d2659f2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:16.892371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792651bc-08e5-42e9-944f-135f3d2659f2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:16.904570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e33b9-a424-4d97-bd5f-40944339776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:16.905832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e33b9-a424-4d97-bd5f-40944339776e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:16.907092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e33b9-a424-4d97-bd5f-40944339776e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:16.907929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e33b9-a424-4d97-bd5f-40944339776e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:16.909901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e33b9-a424-4d97-bd5f-40944339776e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:16.912849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e33b9-a424-4d97-bd5f-40944339776e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:16.939423 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3300/6606] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:16.940730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e33b9-a424-4d97-bd5f-40944339776e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:16.955900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e33b9-a424-4d97-bd5f-40944339776e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:16.983124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b890c57ad0d4fd9852974d0aaf33a15', 'request_id': 'req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:17.006505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b890c57ad0d4fd9852974d0aaf33a15', 'request_id': 'req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:17.020218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b890c57ad0d4fd9852974d0aaf33a15', 'request_id': 'req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:17.045651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] The token cache is being invalidate because project 3b890c57ad0d4fd9852974d0aaf33a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:17.046389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:17.047667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60cd7fb6-960c-4f33-b7d7-9b603071bcfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:17.058468 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3307/6607] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:16 2022] DELETE /identity/v3/projects/3b890c57ad0d4fd9852974d0aaf33a15 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:17.062143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd615ac-c216-45b1-a7b3-0034c94f6be8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.064157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd615ac-c216-45b1-a7b3-0034c94f6be8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.064462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd615ac-c216-45b1-a7b3-0034c94f6be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.102566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9fd615ac-c216-45b1-a7b3-0034c94f6be8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:17.104504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9fd615ac-c216-45b1-a7b3-0034c94f6be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:17.105505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa0dfa-4ac6-4b29-9567-dfccde1a11df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:17.106136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa0dfa-4ac6-4b29-9567-dfccde1a11df admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.106408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa0dfa-4ac6-4b29-9567-dfccde1a11df admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.106674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa0dfa-4ac6-4b29-9567-dfccde1a11df admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.107617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa0dfa-4ac6-4b29-9567-dfccde1a11df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:17.110268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa0dfa-4ac6-4b29-9567-dfccde1a11df admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:17.129703 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3308/6608] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:17.134032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a31813-541b-4b9b-8a75-8d0fdf992b37 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.134315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a31813-541b-4b9b-8a75-8d0fdf992b37 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.134588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a31813-541b-4b9b-8a75-8d0fdf992b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.141756 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3301/6609] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:17.146618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa0dfa-4ac6-4b29-9567-dfccde1a11df admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:17.152274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-14fa0dfa-4ac6-4b29-9567-dfccde1a11df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:17.171974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:17.180574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-35a31813-541b-4b9b-8a75-8d0fdf992b37 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:17.182070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-35a31813-541b-4b9b-8a75-8d0fdf992b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:17.182753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa0dfa-4ac6-4b29-9567-dfccde1a11df admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:17.202033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13365806-05be-41b9-9af6-a80bac5b034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:17.202339 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3309/6610] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:17.202708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13365806-05be-41b9-9af6-a80bac5b034a admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.202989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13365806-05be-41b9-9af6-a80bac5b034a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.203260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13365806-05be-41b9-9af6-a80bac5b034a admin admin] PATH_INFO: `/v3/projects/ca3eff2089c64b8dbafd63f4c80f2949/users/495cc21e67204ad6bd46e4be37f39578/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.206813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a31813-541b-4b9b-8a75-8d0fdf992b37 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:17.230242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a31813-541b-4b9b-8a75-8d0fdf992b37 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:17.238193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13365806-05be-41b9-9af6-a80bac5b034a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca3eff2089c64b8dbafd63f4c80f2949, user_id=495cc21e67204ad6bd46e4be37f39578, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:17.242784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13365806-05be-41b9-9af6-a80bac5b034a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:17.245212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404245c-3146-4067-9935-ccad00c1adbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:17.246966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404245c-3146-4067-9935-ccad00c1adbd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.247216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404245c-3146-4067-9935-ccad00c1adbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.250360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404245c-3146-4067-9935-ccad00c1adbd admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.251119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404245c-3146-4067-9935-ccad00c1adbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:17.251347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404245c-3146-4067-9935-ccad00c1adbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:17.257405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404245c-3146-4067-9935-ccad00c1adbd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:17.282518 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3302/6611] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:17 2022] PUT /identity/v3/projects/ca3eff2089c64b8dbafd63f4c80f2949/users/495cc21e67204ad6bd46e4be37f39578/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:17.289128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871f516-350d-4e22-90cf-bf1d0142485c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.289381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871f516-350d-4e22-90cf-bf1d0142485c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.289687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871f516-350d-4e22-90cf-bf1d0142485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.293393 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3310/6612] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:16:17.298221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404245c-3146-4067-9935-ccad00c1adbd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:17.327219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404245c-3146-4067-9935-ccad00c1adbd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:17.348251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3871f516-350d-4e22-90cf-bf1d0142485c None None] MFA Rules not processed for user `495cc21e67204ad6bd46e4be37f39578`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:17.349514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3871f516-350d-4e22-90cf-bf1d0142485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:17.432059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb520c24-5077-4714-8f00-c2f5f51e87f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:17.436618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb520c24-5077-4714-8f00-c2f5f51e87f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.437154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb520c24-5077-4714-8f00-c2f5f51e87f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.439509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb520c24-5077-4714-8f00-c2f5f51e87f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.440419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb520c24-5077-4714-8f00-c2f5f51e87f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:17.447723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb520c24-5077-4714-8f00-c2f5f51e87f0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:17.492368 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3303/6613] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:17.497151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871f516-350d-4e22-90cf-bf1d0142485c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:17.508551 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3311/6614] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:17 2022] POST /identity/v3/projects => generated 359 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:17.523999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb520c24-5077-4714-8f00-c2f5f51e87f0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:17.524944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871f516-350d-4e22-90cf-bf1d0142485c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:17.619944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:17.653550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb520c24-5077-4714-8f00-c2f5f51e87f0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:17.661056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a59d8-3fe3-4b11-ad02-3f547fb3f158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:17.662086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a59d8-3fe3-4b11-ad02-3f547fb3f158 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.662358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a59d8-3fe3-4b11-ad02-3f547fb3f158 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.662986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a59d8-3fe3-4b11-ad02-3f547fb3f158 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.663832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a59d8-3fe3-4b11-ad02-3f547fb3f158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:17.665899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a59d8-3fe3-4b11-ad02-3f547fb3f158 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:17.666398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381c511-877b-47fe-b372-81a937aad0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:17.667047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381c511-877b-47fe-b372-81a937aad0bf admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.667316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381c511-877b-47fe-b372-81a937aad0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.667625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381c511-877b-47fe-b372-81a937aad0bf admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.668283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381c511-877b-47fe-b372-81a937aad0bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:17.670238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381c511-877b-47fe-b372-81a937aad0bf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:17.720581 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3304/6615] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:17 2022] POST /identity/v3/projects => generated 387 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:17.721278 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3312/6616] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:17 2022] POST /identity/v3/users => generated 280 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:17.724933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a59d8-3fe3-4b11-ad02-3f547fb3f158 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:17.726203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381c511-877b-47fe-b372-81a937aad0bf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:17.755776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a59d8-3fe3-4b11-ad02-3f547fb3f158 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:17.825794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:17.836664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381c511-877b-47fe-b372-81a937aad0bf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:17.847402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0ed6b-3f0f-4b50-8a08-fc849b485eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:17.848256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0ed6b-3f0f-4b50-8a08-fc849b485eab admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.848680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0ed6b-3f0f-4b50-8a08-fc849b485eab admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.849147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0ed6b-3f0f-4b50-8a08-fc849b485eab admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.850574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0ed6b-3f0f-4b50-8a08-fc849b485eab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:17.854492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0ed6b-3f0f-4b50-8a08-fc849b485eab admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:17.893022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a83cd9-b26b-498c-9f39-a884759ecb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:17.894550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a83cd9-b26b-498c-9f39-a884759ecb11 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.895798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a83cd9-b26b-498c-9f39-a884759ecb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.896072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a83cd9-b26b-498c-9f39-a884759ecb11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.901984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a83cd9-b26b-498c-9f39-a884759ecb11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:17.905986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a83cd9-b26b-498c-9f39-a884759ecb11 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:17.906638 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3305/6617] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:17 2022] POST /identity/v3/users => generated 294 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:16:17.910833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0ed6b-3f0f-4b50-8a08-fc849b485eab admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:17.936772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0ed6b-3f0f-4b50-8a08-fc849b485eab admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:17.941363 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3313/6618] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:17.943283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a83cd9-b26b-498c-9f39-a884759ecb11 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:17.955481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673aaad-9ab6-4dbc-8231-17113d805599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:17.955807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673aaad-9ab6-4dbc-8231-17113d805599 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.956045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673aaad-9ab6-4dbc-8231-17113d805599 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.956318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673aaad-9ab6-4dbc-8231-17113d805599 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.956962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673aaad-9ab6-4dbc-8231-17113d805599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:17.960862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673aaad-9ab6-4dbc-8231-17113d805599 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:17.963988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a83cd9-b26b-498c-9f39-a884759ecb11 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:17.982439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b04f85-095b-4aa7-a467-153057715906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:17.983059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b04f85-095b-4aa7-a467-153057715906 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:17.983394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b04f85-095b-4aa7-a467-153057715906 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:17.983936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b04f85-095b-4aa7-a467-153057715906 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:17.984294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b04f85-095b-4aa7-a467-153057715906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:17.988590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b04f85-095b-4aa7-a467-153057715906 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:17.991512 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3306/6619] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:17.994332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673aaad-9ab6-4dbc-8231-17113d805599 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:18.020828 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3314/6620] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:18.022032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b04f85-095b-4aa7-a467-153057715906 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:18.022338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673aaad-9ab6-4dbc-8231-17113d805599 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:18.035178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b40f7-8804-430f-860c-0de4ad00e52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:18.035794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b40f7-8804-430f-860c-0de4ad00e52c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:18.036563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b40f7-8804-430f-860c-0de4ad00e52c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:18.038422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b40f7-8804-430f-860c-0de4ad00e52c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:18.039121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b40f7-8804-430f-860c-0de4ad00e52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:18.040941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b40f7-8804-430f-860c-0de4ad00e52c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:18.045020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b04f85-095b-4aa7-a467-153057715906 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:18.059307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a6123-8223-4639-9607-22017b57ad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:18.059973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a6123-8223-4639-9607-22017b57ad5e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:18.060290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a6123-8223-4639-9607-22017b57ad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:18.060496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a6123-8223-4639-9607-22017b57ad5e admin admin] PATH_INFO: `/v3/projects/117151032f0a47bc90e36532ee041644/users/5733a79d1d0c4b9ca54bb67b34a803f4/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:18.065526 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3307/6621] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:16:18.067878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b40f7-8804-430f-860c-0de4ad00e52c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:18.092192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b40f7-8804-430f-860c-0de4ad00e52c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:18.094365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a6123-8223-4639-9607-22017b57ad5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=117151032f0a47bc90e36532ee041644, user_id=5733a79d1d0c4b9ca54bb67b34a803f4, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:18.098512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a6123-8223-4639-9607-22017b57ad5e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:18.104945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17099c-d253-409e-9c1e-cc93b110f051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:18.106482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17099c-d253-409e-9c1e-cc93b110f051 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:18.106482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17099c-d253-409e-9c1e-cc93b110f051 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:18.106482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17099c-d253-409e-9c1e-cc93b110f051 admin admin] PATH_INFO: `/v3/projects/4cd7d5f2b59a4db891257613d436f046/users/b8a49ec85b274f7a9efdec93e3f85297/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:18.142267 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3315/6622] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:18 2022] PUT /identity/v3/projects/117151032f0a47bc90e36532ee041644/users/5733a79d1d0c4b9ca54bb67b34a803f4/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:18.146002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5b2e5-f0f2-48d2-b619-c66298783391 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:18.148775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5b2e5-f0f2-48d2-b619-c66298783391 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:18.149178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5b2e5-f0f2-48d2-b619-c66298783391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:18.158237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17099c-d253-409e-9c1e-cc93b110f051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd7d5f2b59a4db891257613d436f046, user_id=b8a49ec85b274f7a9efdec93e3f85297, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:18.170156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17099c-d253-409e-9c1e-cc93b110f051 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:18.187378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4af5b2e5-f0f2-48d2-b619-c66298783391 None None] MFA Rules not processed for user `5733a79d1d0c4b9ca54bb67b34a803f4`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:18.188818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4af5b2e5-f0f2-48d2-b619-c66298783391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:18.232614 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3308/6623] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:18 2022] PUT /identity/v3/projects/4cd7d5f2b59a4db891257613d436f046/users/b8a49ec85b274f7a9efdec93e3f85297/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:18.234225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73bffd-04d4-45ff-b76c-d61a0dd5a2f6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:18.234505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73bffd-04d4-45ff-b76c-d61a0dd5a2f6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:18.234778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73bffd-04d4-45ff-b76c-d61a0dd5a2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:18.312759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1c73bffd-04d4-45ff-b76c-d61a0dd5a2f6 None None] MFA Rules not processed for user `b8a49ec85b274f7a9efdec93e3f85297`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:18.314186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1c73bffd-04d4-45ff-b76c-d61a0dd5a2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:18.379053 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3316/6624] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:18.415536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5b2e5-f0f2-48d2-b619-c66298783391 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:18.418194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4af5b2e5-f0f2-48d2-b619-c66298783391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:18.489859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:18.511956 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3309/6625] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:18.513889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73bffd-04d4-45ff-b76c-d61a0dd5a2f6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:18.519297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1c73bffd-04d4-45ff-b76c-d61a0dd5a2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:18.606661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5b2e5-f0f2-48d2-b619-c66298783391 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:18.703943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9264e-6506-4fe2-8113-bfa51a3fd2eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:18.704939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9264e-6506-4fe2-8113-bfa51a3fd2eb service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:18.704939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9264e-6506-4fe2-8113-bfa51a3fd2eb service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:18.705382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9264e-6506-4fe2-8113-bfa51a3fd2eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:18.705649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73bffd-04d4-45ff-b76c-d61a0dd5a2f6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:18.721677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:18.722643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:18.722930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:18.723203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] PATH_INFO: `/v3/projects/eabfc9e340d64291b5c94181d0d5bd65` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:18.727515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=eabfc9e340d64291b5c94181d0d5bd65)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:18.729515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:18.809954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9264e-6506-4fe2-8113-bfa51a3fd2eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:18.812194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9264e-6506-4fe2-8113-bfa51a3fd2eb service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:18.825233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:18.852168 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3317/6626] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:16:18.854361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9264e-6506-4fe2-8113-bfa51a3fd2eb service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:18.863631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eabfc9e340d64291b5c94181d0d5bd65', 'request_id': 'req-54f564bf-2904-4470-89a5-436b31dbe7bc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:18.897898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eabfc9e340d64291b5c94181d0d5bd65', 'request_id': 'req-54f564bf-2904-4470-89a5-436b31dbe7bc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:18.910817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9264e-6506-4fe2-8113-bfa51a3fd2eb service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:18.921294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eabfc9e340d64291b5c94181d0d5bd65', 'request_id': 'req-54f564bf-2904-4470-89a5-436b31dbe7bc'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:18.957874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] The token cache is being invalidate because project eabfc9e340d64291b5c94181d0d5bd65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:18.958226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:18.961150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:18.963031 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3310/6627] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:18 2022] DELETE /identity/v3/projects/eabfc9e340d64291b5c94181d0d5bd65 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:18.965059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:18.970491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:19.123644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87179f5c-2986-4244-adad-945e7b59b1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:19.129300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87179f5c-2986-4244-adad-945e7b59b1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:19.130205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87179f5c-2986-4244-adad-945e7b59b1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:19.131152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87179f5c-2986-4244-adad-945e7b59b1d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:19.134255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87179f5c-2986-4244-adad-945e7b59b1d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:19.140545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87179f5c-2986-4244-adad-945e7b59b1d0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:19.258636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f564bf-2904-4470-89a5-436b31dbe7bc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:19.272567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:19.275556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:19.275704 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3318/6628] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:18 2022] POST /identity/v3/projects => generated 387 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:19.275961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:19.277295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32272b57-cab9-4bf4-a508-938fb51eeeed None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:19.277579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32272b57-cab9-4bf4-a508-938fb51eeeed None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:19.277861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32272b57-cab9-4bf4-a508-938fb51eeeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:19.283208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] PATH_INFO: `/v3/users/5733a79d1d0c4b9ca54bb67b34a803f4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:19.287221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5733a79d1d0c4b9ca54bb67b34a803f4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:19.297566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:19.329921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-32272b57-cab9-4bf4-a508-938fb51eeeed None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:19.333208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-32272b57-cab9-4bf4-a508-938fb51eeeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:19.357869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:19.442826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5733a79d1d0c4b9ca54bb67b34a803f4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:19.489459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5733a79d1d0c4b9ca54bb67b34a803f4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:19.509535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5733a79d1d0c4b9ca54bb67b34a803f4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:19.523539 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3319/6629] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:19.540626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5733a79d1d0c4b9ca54bb67b34a803f4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:19.544699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2cea275-73de-4569-a4fe-5995ca1fc186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5733a79d1d0c4b9ca54bb67b34a803f4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:19.545745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32272b57-cab9-4bf4-a508-938fb51eeeed None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:19.564420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-32272b57-cab9-4bf4-a508-938fb51eeeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:19.582640 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3311/6630] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:18 2022] DELETE /identity/v3/users/5733a79d1d0c4b9ca54bb67b34a803f4 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:19.586598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f0dcc-4a25-4436-bb63-7d618b5c96ed None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:19.586882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f0dcc-4a25-4436-bb63-7d618b5c96ed None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:19.587152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f0dcc-4a25-4436-bb63-7d618b5c96ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:19.608333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32272b57-cab9-4bf4-a508-938fb51eeeed None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:19.659602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ac7f0dcc-4a25-4436-bb63-7d618b5c96ed None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:19.665111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac7f0dcc-4a25-4436-bb63-7d618b5c96ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:19.842030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:19.904583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b268072-963c-4218-877e-7c5d46b52f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:19.906738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b268072-963c-4218-877e-7c5d46b52f3a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:19.907015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b268072-963c-4218-877e-7c5d46b52f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:19.907281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b268072-963c-4218-877e-7c5d46b52f3a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:19.911160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b268072-963c-4218-877e-7c5d46b52f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:19.917809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b268072-963c-4218-877e-7c5d46b52f3a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:19.950425 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3312/6631] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:19.951739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449700-95e1-4e47-8ea2-d49a1d4f06d4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:19.952021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449700-95e1-4e47-8ea2-d49a1d4f06d4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:19.952288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449700-95e1-4e47-8ea2-d49a1d4f06d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.017451 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3320/6632] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:19 2022] POST /identity/v3/users => generated 300 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:20.021388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b268072-963c-4218-877e-7c5d46b52f3a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:20.038697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a1449700-95e1-4e47-8ea2-d49a1d4f06d4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:20.046616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a1449700-95e1-4e47-8ea2-d49a1d4f06d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:20.047764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b268072-963c-4218-877e-7c5d46b52f3a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:20.064804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b56207-d10f-45d6-be34-c5f27bcc6427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:20.065596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b56207-d10f-45d6-be34-c5f27bcc6427 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:20.066715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b56207-d10f-45d6-be34-c5f27bcc6427 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:20.069281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b56207-d10f-45d6-be34-c5f27bcc6427 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.070172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b56207-d10f-45d6-be34-c5f27bcc6427 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:20.070564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b56207-d10f-45d6-be34-c5f27bcc6427 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:20.074631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b56207-d10f-45d6-be34-c5f27bcc6427 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:20.085631 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3313/6633] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:20.087424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1449700-95e1-4e47-8ea2-d49a1d4f06d4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:20.092322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a1449700-95e1-4e47-8ea2-d49a1d4f06d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:20.109019 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3321/6634] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:16:20.113662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b56207-d10f-45d6-be34-c5f27bcc6427 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:20.171589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b56207-d10f-45d6-be34-c5f27bcc6427 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:20.172166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1449700-95e1-4e47-8ea2-d49a1d4f06d4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:20.180046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee80521-555c-4c54-836a-a6105a8e03cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:20.181332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d71344-0b5e-43ee-b210-78e35894ee1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:20.181698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee80521-555c-4c54-836a-a6105a8e03cf admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:20.181946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d71344-0b5e-43ee-b210-78e35894ee1b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:20.182108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee80521-555c-4c54-836a-a6105a8e03cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:20.182205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d71344-0b5e-43ee-b210-78e35894ee1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:20.182469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d71344-0b5e-43ee-b210-78e35894ee1b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.182569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee80521-555c-4c54-836a-a6105a8e03cf admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.183230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee80521-555c-4c54-836a-a6105a8e03cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:20.185375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee80521-555c-4c54-836a-a6105a8e03cf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:20.186482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d71344-0b5e-43ee-b210-78e35894ee1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:20.192218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d71344-0b5e-43ee-b210-78e35894ee1b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:20.241375 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3314/6635] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:20.243373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d71344-0b5e-43ee-b210-78e35894ee1b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:20.265134 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3322/6636] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:20 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:20.268738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee80521-555c-4c54-836a-a6105a8e03cf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:20.274993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d71344-0b5e-43ee-b210-78e35894ee1b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:20.276779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dee80521-555c-4c54-836a-a6105a8e03cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:20.407060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd673ab9-f5a5-4032-9147-99826bc9944c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:20.408383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd673ab9-f5a5-4032-9147-99826bc9944c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:20.408864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd673ab9-f5a5-4032-9147-99826bc9944c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:20.410875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd673ab9-f5a5-4032-9147-99826bc9944c admin admin] PATH_INFO: `/v3/projects/2b79c048af684c74a20a60aa75e312e2/users/4574934f8e4b4a0da36a97a63e8850ab/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.487331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee80521-555c-4c54-836a-a6105a8e03cf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:20.491289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd673ab9-f5a5-4032-9147-99826bc9944c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b79c048af684c74a20a60aa75e312e2, user_id=4574934f8e4b4a0da36a97a63e8850ab, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:20.494511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd673ab9-f5a5-4032-9147-99826bc9944c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:20.501497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887122a5-6a7c-456a-8cf9-4cd1575b683c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:20.502132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887122a5-6a7c-456a-8cf9-4cd1575b683c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:20.502895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887122a5-6a7c-456a-8cf9-4cd1575b683c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:20.503168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887122a5-6a7c-456a-8cf9-4cd1575b683c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.521270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887122a5-6a7c-456a-8cf9-4cd1575b683c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:20.523557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887122a5-6a7c-456a-8cf9-4cd1575b683c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:20.540598 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3323/6637] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:20.541277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887122a5-6a7c-456a-8cf9-4cd1575b683c service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:20.543088 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3315/6638] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:20 2022] PUT /identity/v3/projects/2b79c048af684c74a20a60aa75e312e2/users/4574934f8e4b4a0da36a97a63e8850ab/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:20.547688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd673ab9-f5a5-4032-9147-99826bc9944c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:20.554467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887122a5-6a7c-456a-8cf9-4cd1575b683c service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:20.566438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:20.704617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90e12c-a6a3-4622-87b6-2ef8c8e08b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:20.705913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd673ab9-f5a5-4032-9147-99826bc9944c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:20.706400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90e12c-a6a3-4622-87b6-2ef8c8e08b3a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:20.706684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90e12c-a6a3-4622-87b6-2ef8c8e08b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:20.706958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90e12c-a6a3-4622-87b6-2ef8c8e08b3a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.707683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90e12c-a6a3-4622-87b6-2ef8c8e08b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:20.711914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90e12c-a6a3-4622-87b6-2ef8c8e08b3a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:20.713923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb20c1-dd78-4fe3-af2c-41213ebd521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:20.714721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb20c1-dd78-4fe3-af2c-41213ebd521a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:20.715129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb20c1-dd78-4fe3-af2c-41213ebd521a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:20.715502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb20c1-dd78-4fe3-af2c-41213ebd521a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.716672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb20c1-dd78-4fe3-af2c-41213ebd521a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:20.719234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb20c1-dd78-4fe3-af2c-41213ebd521a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:20.736395 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3316/6639] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:20.741160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb20c1-dd78-4fe3-af2c-41213ebd521a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:20.757993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb20c1-dd78-4fe3-af2c-41213ebd521a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:20.762479 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3324/6640] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:20 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:20.764759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e949-a379-4ace-b9b4-d0a57c90afcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:20.765681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e949-a379-4ace-b9b4-d0a57c90afcd admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:20.765902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e949-a379-4ace-b9b4-d0a57c90afcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:20.766172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e949-a379-4ace-b9b4-d0a57c90afcd admin admin] PATH_INFO: `/v3/projects/2b79c048af684c74a20a60aa75e312e2/users/4574934f8e4b4a0da36a97a63e8850ab/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.766617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90e12c-a6a3-4622-87b6-2ef8c8e08b3a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:20.768488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279e949-a379-4ace-b9b4-d0a57c90afcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b79c048af684c74a20a60aa75e312e2, user_id=4574934f8e4b4a0da36a97a63e8850ab, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:20.771514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279e949-a379-4ace-b9b4-d0a57c90afcd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:20.788739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90e12c-a6a3-4622-87b6-2ef8c8e08b3a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:20.801692 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3317/6641] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:20 2022] PUT /identity/v3/projects/2b79c048af684c74a20a60aa75e312e2/users/4574934f8e4b4a0da36a97a63e8850ab/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:20.804439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee86853-acf0-4325-8081-2e4558cba705 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:20.804788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee86853-acf0-4325-8081-2e4558cba705 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:20.805100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee86853-acf0-4325-8081-2e4558cba705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.853560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-aee86853-acf0-4325-8081-2e4558cba705 None None] MFA Rules not processed for user `4574934f8e4b4a0da36a97a63e8850ab`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:20.857826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aee86853-acf0-4325-8081-2e4558cba705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:20.892272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:20.948662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4072cc9-9aab-4ae6-a92f-9e9aba3ba290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:20.949522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4072cc9-9aab-4ae6-a92f-9e9aba3ba290 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:20.950094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4072cc9-9aab-4ae6-a92f-9e9aba3ba290 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:20.950375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4072cc9-9aab-4ae6-a92f-9e9aba3ba290 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:20.951162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4072cc9-9aab-4ae6-a92f-9e9aba3ba290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:20.953316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4072cc9-9aab-4ae6-a92f-9e9aba3ba290 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:20.980060 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3325/6642] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:20.984491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4072cc9-9aab-4ae6-a92f-9e9aba3ba290 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:21.004739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4072cc9-9aab-4ae6-a92f-9e9aba3ba290 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:21.018780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e28e1c-c813-4634-833a-6fbbb4d48238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:21.021147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e28e1c-c813-4634-833a-6fbbb4d48238 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:21.022388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e28e1c-c813-4634-833a-6fbbb4d48238 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:21.024150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e28e1c-c813-4634-833a-6fbbb4d48238 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:21.026485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e28e1c-c813-4634-833a-6fbbb4d48238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:21.030448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e28e1c-c813-4634-833a-6fbbb4d48238 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:21.061646 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3326/6643] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:21.067075 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3318/6644] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:21.073031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e28e1c-c813-4634-833a-6fbbb4d48238 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:21.083757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee86853-acf0-4325-8081-2e4558cba705 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:21.087863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aee86853-acf0-4325-8081-2e4558cba705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:21.105728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e28e1c-c813-4634-833a-6fbbb4d48238 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:21.118792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3dd44-980f-4524-8d7c-317126c0bfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:21.119483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3dd44-980f-4524-8d7c-317126c0bfbe admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:21.119736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3dd44-980f-4524-8d7c-317126c0bfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:21.123769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3dd44-980f-4524-8d7c-317126c0bfbe admin admin] PATH_INFO: `/v3/projects/c78e90827d8c4f0eb91abbd2da06e509/users/e2224a51f6f043a19683b59c9cd0f276/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:21.174340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3dd44-980f-4524-8d7c-317126c0bfbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78e90827d8c4f0eb91abbd2da06e509, user_id=e2224a51f6f043a19683b59c9cd0f276, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:21.177410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3dd44-980f-4524-8d7c-317126c0bfbe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:21.232197 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3327/6645] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:21 2022] PUT /identity/v3/projects/c78e90827d8c4f0eb91abbd2da06e509/users/e2224a51f6f043a19683b59c9cd0f276/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:21.234680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee86853-acf0-4325-8081-2e4558cba705 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:21.242850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0e771-f20a-4679-b33c-f04693c25bce None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:21.243127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0e771-f20a-4679-b33c-f04693c25bce None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:21.243423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0e771-f20a-4679-b33c-f04693c25bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:21.286314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2cf0e771-f20a-4679-b33c-f04693c25bce None None] MFA Rules not processed for user `e2224a51f6f043a19683b59c9cd0f276`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:21.287783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2cf0e771-f20a-4679-b33c-f04693c25bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:21.378318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c318578-2224-4ee5-bbaa-8cf0e0fa5dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:21.380188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c318578-2224-4ee5-bbaa-8cf0e0fa5dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:21.380507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c318578-2224-4ee5-bbaa-8cf0e0fa5dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:21.380723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c318578-2224-4ee5-bbaa-8cf0e0fa5dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:21.450640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:21.472452 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3328/6646] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:21.473598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0e771-f20a-4679-b33c-f04693c25bce None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:21.541357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c318578-2224-4ee5-bbaa-8cf0e0fa5dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:21.543242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c318578-2224-4ee5-bbaa-8cf0e0fa5dc5 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:21.560905 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3319/6647] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:21.624474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c318578-2224-4ee5-bbaa-8cf0e0fa5dc5 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:21.626433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2c318578-2224-4ee5-bbaa-8cf0e0fa5dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:21.628540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0e771-f20a-4679-b33c-f04693c25bce None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:21.644936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3461-bbc1-455c-865f-90d8e1929528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:21.645588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3461-bbc1-455c-865f-90d8e1929528 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:21.645860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3461-bbc1-455c-865f-90d8e1929528 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:21.646209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3461-bbc1-455c-865f-90d8e1929528 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:21.646943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3461-bbc1-455c-865f-90d8e1929528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:21.649784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3461-bbc1-455c-865f-90d8e1929528 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:21.655164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c318578-2224-4ee5-bbaa-8cf0e0fa5dc5 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:21.675208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:21.675921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:21.676204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:21.676547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] PATH_INFO: `/v3/projects/117151032f0a47bc90e36532ee041644` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:21.682352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] RBAC: Authorizing `identity:delete_project(project_id=117151032f0a47bc90e36532ee041644)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:21.688556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:21.723257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:21.731739 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3329/6648] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:21 2022] POST /identity/v3/projects => generated 387 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:21.733301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3461-bbc1-455c-865f-90d8e1929528 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:21.798884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '117151032f0a47bc90e36532ee041644', 'request_id': 'req-619aa429-a670-489c-af01-46aafb6127af'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:21.813999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '117151032f0a47bc90e36532ee041644', 'request_id': 'req-619aa429-a670-489c-af01-46aafb6127af'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:21.828994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '117151032f0a47bc90e36532ee041644', 'request_id': 'req-619aa429-a670-489c-af01-46aafb6127af'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:21.842559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3461-bbc1-455c-865f-90d8e1929528 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:21.855230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] The token cache is being invalidate because project 117151032f0a47bc90e36532ee041644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:21.855541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:21.856173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-619aa429-a670-489c-af01-46aafb6127af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:21.858361 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3320/6649] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:21 2022] DELETE /identity/v3/projects/117151032f0a47bc90e36532ee041644 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:21.863695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf382cbc-82fc-4bf1-aa12-a0f9f65458a0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:21.863969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf382cbc-82fc-4bf1-aa12-a0f9f65458a0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:21.864235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf382cbc-82fc-4bf1-aa12-a0f9f65458a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:21.894785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cf382cbc-82fc-4bf1-aa12-a0f9f65458a0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:21.896150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cf382cbc-82fc-4bf1-aa12-a0f9f65458a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:21.982204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf68b40-7d56-463f-9f56-032dbc69101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:21.982915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf68b40-7d56-463f-9f56-032dbc69101b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:21.986696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf68b40-7d56-463f-9f56-032dbc69101b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:21.986988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf68b40-7d56-463f-9f56-032dbc69101b admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:21.987637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf68b40-7d56-463f-9f56-032dbc69101b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:21.990042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf68b40-7d56-463f-9f56-032dbc69101b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:21.995560 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3321/6650] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:22.003972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154a23e-b27e-465d-8a45-1e6d741271dc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.004480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154a23e-b27e-465d-8a45-1e6d741271dc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.004898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154a23e-b27e-465d-8a45-1e6d741271dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.028961 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3330/6651] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:21 2022] POST /identity/v3/users => generated 301 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:22.030415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf68b40-7d56-463f-9f56-032dbc69101b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.032584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1bf68b40-7d56-463f-9f56-032dbc69101b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:22.032858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a154a23e-b27e-465d-8a45-1e6d741271dc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:22.034245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a154a23e-b27e-465d-8a45-1e6d741271dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:22.045185 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3322/6652] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:16:22.048316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf68b40-7d56-463f-9f56-032dbc69101b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.049235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154a23e-b27e-465d-8a45-1e6d741271dc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.065369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154a23e-b27e-465d-8a45-1e6d741271dc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.067704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f64ff6-9c6e-462b-b4e2-2fcc47433720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:22.069344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f64ff6-9c6e-462b-b4e2-2fcc47433720 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.069624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f64ff6-9c6e-462b-b4e2-2fcc47433720 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.070634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f64ff6-9c6e-462b-b4e2-2fcc47433720 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.071442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f64ff6-9c6e-462b-b4e2-2fcc47433720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:22.073441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f64ff6-9c6e-462b-b4e2-2fcc47433720 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:22.077525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fb167-c0fd-4a20-b980-e60eacd87f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:22.078259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fb167-c0fd-4a20-b980-e60eacd87f5e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.078542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fb167-c0fd-4a20-b980-e60eacd87f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.078806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fb167-c0fd-4a20-b980-e60eacd87f5e admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.079455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fb167-c0fd-4a20-b980-e60eacd87f5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:22.079750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fb167-c0fd-4a20-b980-e60eacd87f5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:22.082054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fb167-c0fd-4a20-b980-e60eacd87f5e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:22.089078 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3331/6653] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:22.090966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f64ff6-9c6e-462b-b4e2-2fcc47433720 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.104174 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3323/6654] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:16:22.112632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f64ff6-9c6e-462b-b4e2-2fcc47433720 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.114091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:22.120228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fb167-c0fd-4a20-b980-e60eacd87f5e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.121767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da38b7-598f-442e-9210-e6143976e852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:22.122650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da38b7-598f-442e-9210-e6143976e852 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.122996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da38b7-598f-442e-9210-e6143976e852 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.123299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da38b7-598f-442e-9210-e6143976e852 admin admin] PATH_INFO: `/v3/projects/bf06b4d5a71543078ce7a04f6044a7fe/users/df8363a970b540afaf1c39b040ad1d5d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.135409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fb167-c0fd-4a20-b980-e60eacd87f5e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.146090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13676706-3185-4c76-b943-f32f697dbe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:22.148497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13676706-3185-4c76-b943-f32f697dbe50 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.148497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13676706-3185-4c76-b943-f32f697dbe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.148497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13676706-3185-4c76-b943-f32f697dbe50 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.149335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13676706-3185-4c76-b943-f32f697dbe50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:22.151255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13676706-3185-4c76-b943-f32f697dbe50 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:22.154595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da38b7-598f-442e-9210-e6143976e852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf06b4d5a71543078ce7a04f6044a7fe, user_id=df8363a970b540afaf1c39b040ad1d5d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:22.167179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da38b7-598f-442e-9210-e6143976e852 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:22.225500 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3332/6655] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:22 2022] PUT /identity/v3/projects/bf06b4d5a71543078ce7a04f6044a7fe/users/df8363a970b540afaf1c39b040ad1d5d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:16:22.227631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7b01a-bf42-435b-82a1-3fe8e0c0155b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.228226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7b01a-bf42-435b-82a1-3fe8e0c0155b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.228489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7b01a-bf42-435b-82a1-3fe8e0c0155b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.253360 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3324/6656] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:22 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:22.257519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13676706-3185-4c76-b943-f32f697dbe50 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.323989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0ee7b01a-bf42-435b-82a1-3fe8e0c0155b None None] MFA Rules not processed for user `df8363a970b540afaf1c39b040ad1d5d`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:22.327893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0ee7b01a-bf42-435b-82a1-3fe8e0c0155b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:22.394424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13676706-3185-4c76-b943-f32f697dbe50 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.404442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cedc67-9f7d-496f-875e-3e6dcfb40a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:22.405213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cedc67-9f7d-496f-875e-3e6dcfb40a11 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.405800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cedc67-9f7d-496f-875e-3e6dcfb40a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.406170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cedc67-9f7d-496f-875e-3e6dcfb40a11 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.407089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cedc67-9f7d-496f-875e-3e6dcfb40a11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:22.409668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cedc67-9f7d-496f-875e-3e6dcfb40a11 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:22.420366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:22.487046 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3325/6657] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:22 2022] POST /identity/v3/users => generated 291 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:22.491073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47b53b-a9c2-4d1e-8d65-5d078af8d062 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.491665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47b53b-a9c2-4d1e-8d65-5d078af8d062 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.492134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47b53b-a9c2-4d1e-8d65-5d078af8d062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.504301 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3333/6658] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:22 2022] POST /identity/v3/auth/tokens => generated 3271 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:16:22.505067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7b01a-bf42-435b-82a1-3fe8e0c0155b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.520142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7b01a-bf42-435b-82a1-3fe8e0c0155b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.528227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80216255-38e2-4436-972b-a76441523ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:22.529192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80216255-38e2-4436-972b-a76441523ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.529538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80216255-38e2-4436-972b-a76441523ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.529898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80216255-38e2-4436-972b-a76441523ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.530619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80216255-38e2-4436-972b-a76441523ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:22.532401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80216255-38e2-4436-972b-a76441523ee4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:22.551089 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3334/6659] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:22.551089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80216255-38e2-4436-972b-a76441523ee4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.559945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80216255-38e2-4436-972b-a76441523ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:22.570106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0f47b53b-a9c2-4d1e-8d65-5d078af8d062 None None] MFA Rules not processed for user `0f9276faf05349aeb1469250b2e199d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:22.571191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0f47b53b-a9c2-4d1e-8d65-5d078af8d062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:22.691605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80216255-38e2-4436-972b-a76441523ee4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.700052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3d32c-711e-45a5-9302-503c3d1ae73c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:22.701432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3d32c-711e-45a5-9302-503c3d1ae73c service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.701947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3d32c-711e-45a5-9302-503c3d1ae73c service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.702420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3d32c-711e-45a5-9302-503c3d1ae73c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.703079 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3326/6660] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:22.704227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47b53b-a9c2-4d1e-8d65-5d078af8d062 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.713606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3d32c-711e-45a5-9302-503c3d1ae73c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:22.716896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3d32c-711e-45a5-9302-503c3d1ae73c service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:22.717520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47b53b-a9c2-4d1e-8d65-5d078af8d062 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.722568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06c039-5c29-4312-b979-11ee17ce7d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:22.723414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06c039-5c29-4312-b979-11ee17ce7d0d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.723699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06c039-5c29-4312-b979-11ee17ce7d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.724029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06c039-5c29-4312-b979-11ee17ce7d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.724681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06c039-5c29-4312-b979-11ee17ce7d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:22.726768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06c039-5c29-4312-b979-11ee17ce7d0d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:22.730109 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3335/6661] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:16:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 680 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:16:22.732241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3d32c-711e-45a5-9302-503c3d1ae73c service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.737421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3cd3d32c-711e-45a5-9302-503c3d1ae73c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:22.739581 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3327/6662] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:22.740943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06c039-5c29-4312-b979-11ee17ce7d0d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.742730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8b06c039-5c29-4312-b979-11ee17ce7d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:22.759171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06c039-5c29-4312-b979-11ee17ce7d0d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.767893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09574c6a-09c5-4a20-be76-478883210116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:22.768547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09574c6a-09c5-4a20-be76-478883210116 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.768824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09574c6a-09c5-4a20-be76-478883210116 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.769295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09574c6a-09c5-4a20-be76-478883210116 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.770310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09574c6a-09c5-4a20-be76-478883210116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:22.772633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09574c6a-09c5-4a20-be76-478883210116 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:22.817310 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3328/6663] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:22 2022] POST /identity/v3/projects => generated 381 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:22.822942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09574c6a-09c5-4a20-be76-478883210116 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:22.877335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:22.890231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3d32c-711e-45a5-9302-503c3d1ae73c service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.930363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09574c6a-09c5-4a20-be76-478883210116 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:22.939032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334f6e3-b7e6-4a18-962f-3500928b63c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:22.940471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334f6e3-b7e6-4a18-962f-3500928b63c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:22.940947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334f6e3-b7e6-4a18-962f-3500928b63c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:22.941417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334f6e3-b7e6-4a18-962f-3500928b63c4 admin admin] PATH_INFO: `/v3/projects/598097829bcb44a083af8e17215e2e43/users/3a946be91d624f038c36ccaffb601651/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:22.965933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334f6e3-b7e6-4a18-962f-3500928b63c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=598097829bcb44a083af8e17215e2e43, user_id=3a946be91d624f038c36ccaffb601651, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:22.970413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334f6e3-b7e6-4a18-962f-3500928b63c4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:22.994670 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3329/6664] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:22 2022] PUT /identity/v3/projects/598097829bcb44a083af8e17215e2e43/users/3a946be91d624f038c36ccaffb601651/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:22.995643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334f6e3-b7e6-4a18-962f-3500928b63c4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:23.013157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7311a-ada7-4c30-b44a-9cfa1893d477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:23.014097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7311a-ada7-4c30-b44a-9cfa1893d477 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:23.019293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7311a-ada7-4c30-b44a-9cfa1893d477 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:23.020005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7311a-ada7-4c30-b44a-9cfa1893d477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:23.028560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-73b7311a-ada7-4c30-b44a-9cfa1893d477 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:23.084581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:23.168906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7311a-ada7-4c30-b44a-9cfa1893d477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:23.170616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7311a-ada7-4c30-b44a-9cfa1893d477 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:23.175268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334f6e3-b7e6-4a18-962f-3500928b63c4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:23.182500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf552132-22d8-438a-a5d0-c920feea8cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:23.184454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf552132-22d8-438a-a5d0-c920feea8cbc admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:23.184888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf552132-22d8-438a-a5d0-c920feea8cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:23.185368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf552132-22d8-438a-a5d0-c920feea8cbc admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:23.186420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf552132-22d8-438a-a5d0-c920feea8cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:23.189226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf552132-22d8-438a-a5d0-c920feea8cbc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:23.199089 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3336/6665] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:23.203560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d6e5a-c37c-4fd0-9aba-1294a4995f38 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:23.204358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d6e5a-c37c-4fd0-9aba-1294a4995f38 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:23.205554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d6e5a-c37c-4fd0-9aba-1294a4995f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:23.279582 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3330/6666] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:22 2022] POST /identity/v3/users => generated 297 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:23.284265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf552132-22d8-438a-a5d0-c920feea8cbc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:23.367375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf552132-22d8-438a-a5d0-c920feea8cbc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:23.369569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a32d6e5a-c37c-4fd0-9aba-1294a4995f38 None None] MFA Rules not processed for user `3a946be91d624f038c36ccaffb601651`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:23.378739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a32d6e5a-c37c-4fd0-9aba-1294a4995f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:23.396534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f2c06-8569-41f5-bff1-52812629c55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:23.398998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f2c06-8569-41f5-bff1-52812629c55a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:23.403581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f2c06-8569-41f5-bff1-52812629c55a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:23.404452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f2c06-8569-41f5-bff1-52812629c55a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:23.409334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f2c06-8569-41f5-bff1-52812629c55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:23.424575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f2c06-8569-41f5-bff1-52812629c55a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:23.459581 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3331/6667] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:16:23.463868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f2c06-8569-41f5-bff1-52812629c55a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:23.500516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f2c06-8569-41f5-bff1-52812629c55a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:23.519181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49562b7e-d71b-43b9-b9d4-781f1e4745eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:23.519854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49562b7e-d71b-43b9-b9d4-781f1e4745eb admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:23.520127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49562b7e-d71b-43b9-b9d4-781f1e4745eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:23.520393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49562b7e-d71b-43b9-b9d4-781f1e4745eb admin admin] PATH_INFO: `/v3/projects/08c053b93b2b4576a373343c328a1353/users/0d15db49ef774494baa77a9ec31b22c2/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:23.549699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49562b7e-d71b-43b9-b9d4-781f1e4745eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c053b93b2b4576a373343c328a1353, user_id=0d15db49ef774494baa77a9ec31b22c2, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:23.552452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49562b7e-d71b-43b9-b9d4-781f1e4745eb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:23.577428 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3332/6668] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:23 2022] PUT /identity/v3/projects/08c053b93b2b4576a373343c328a1353/users/0d15db49ef774494baa77a9ec31b22c2/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:16:23.581202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49562b7e-d71b-43b9-b9d4-781f1e4745eb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:23.657797 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3337/6669] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:23.681740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d6e5a-c37c-4fd0-9aba-1294a4995f38 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:23.726806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d6e5a-c37c-4fd0-9aba-1294a4995f38 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:23.734726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49562b7e-d71b-43b9-b9d4-781f1e4745eb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:23.739917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a627f9c-5bdb-46a8-9387-96f090304630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:23.741993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a627f9c-5bdb-46a8-9387-96f090304630 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:23.743336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a627f9c-5bdb-46a8-9387-96f090304630 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:23.743680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a627f9c-5bdb-46a8-9387-96f090304630 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:23.744898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a627f9c-5bdb-46a8-9387-96f090304630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:23.750004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a8853-4d20-4b7a-a989-41ea4b8e6620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:23.751617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a627f9c-5bdb-46a8-9387-96f090304630 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:23.752494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a8853-4d20-4b7a-a989-41ea4b8e6620 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:23.752975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a8853-4d20-4b7a-a989-41ea4b8e6620 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:23.753393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a8853-4d20-4b7a-a989-41ea4b8e6620 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:23.754178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a8853-4d20-4b7a-a989-41ea4b8e6620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:23.761769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a8853-4d20-4b7a-a989-41ea4b8e6620 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:23.806645 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3333/6670] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:23.810546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a8853-4d20-4b7a-a989-41ea4b8e6620 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:23.830063 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3338/6671] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:23 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:23.841691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a627f9c-5bdb-46a8-9387-96f090304630 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:23.844747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a8853-4d20-4b7a-a989-41ea4b8e6620 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:23.941274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:23.977454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08934f0-b0e3-4c4e-b5ff-d17db8ad3280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:23.978404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08934f0-b0e3-4c4e-b5ff-d17db8ad3280 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:23.979084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08934f0-b0e3-4c4e-b5ff-d17db8ad3280 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:23.979084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08934f0-b0e3-4c4e-b5ff-d17db8ad3280 admin admin] PATH_INFO: `/v3/projects/08c053b93b2b4576a373343c328a1353/users/0d15db49ef774494baa77a9ec31b22c2/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:23.981957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08934f0-b0e3-4c4e-b5ff-d17db8ad3280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c053b93b2b4576a373343c328a1353, user_id=0d15db49ef774494baa77a9ec31b22c2, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:23.985497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08934f0-b0e3-4c4e-b5ff-d17db8ad3280 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:24.002783 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3334/6672] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:23 2022] PUT /identity/v3/projects/08c053b93b2b4576a373343c328a1353/users/0d15db49ef774494baa77a9ec31b22c2/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:16:24.003735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08934f0-b0e3-4c4e-b5ff-d17db8ad3280 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:24.018532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a627f9c-5bdb-46a8-9387-96f090304630 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:24.059593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:24.142016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43bb80-d458-41c5-9402-99166cfb7030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:24.142626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43bb80-d458-41c5-9402-99166cfb7030 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:24.143053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43bb80-d458-41c5-9402-99166cfb7030 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:24.143386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43bb80-d458-41c5-9402-99166cfb7030 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:24.144028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43bb80-d458-41c5-9402-99166cfb7030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:24.147043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43bb80-d458-41c5-9402-99166cfb7030 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:24.200309 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3339/6673] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:23 2022] POST /identity/v3/users => generated 290 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:24.205603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43bb80-d458-41c5-9402-99166cfb7030 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:24.240957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43bb80-d458-41c5-9402-99166cfb7030 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:24.249954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cf85a-f063-4f01-b146-8fee38d6b667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:24.250593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cf85a-f063-4f01-b146-8fee38d6b667 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:24.250871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cf85a-f063-4f01-b146-8fee38d6b667 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:24.251133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cf85a-f063-4f01-b146-8fee38d6b667 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:24.251772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cf85a-f063-4f01-b146-8fee38d6b667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:24.253425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cf85a-f063-4f01-b146-8fee38d6b667 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:24.280797 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3340/6674] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:24.282036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cf85a-f063-4f01-b146-8fee38d6b667 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:24.295909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cf85a-f063-4f01-b146-8fee38d6b667 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:24.303763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2dcaf-2ddb-41a1-b9fd-71f01a4751ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:24.304587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2dcaf-2ddb-41a1-b9fd-71f01a4751ef admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:24.304902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2dcaf-2ddb-41a1-b9fd-71f01a4751ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:24.305234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2dcaf-2ddb-41a1-b9fd-71f01a4751ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:24.305926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2dcaf-2ddb-41a1-b9fd-71f01a4751ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:24.308817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2dcaf-2ddb-41a1-b9fd-71f01a4751ef admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:24.327502 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3341/6675] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:24.328684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2dcaf-2ddb-41a1-b9fd-71f01a4751ef admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:24.342048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08934f0-b0e3-4c4e-b5ff-d17db8ad3280 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:24.346271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2dcaf-2ddb-41a1-b9fd-71f01a4751ef admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:24.364261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53cfa7-bfaf-40df-bd5a-edeea77dec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:24.366695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:24.366880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53cfa7-bfaf-40df-bd5a-edeea77dec9a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:24.369594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:24.371960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:24.372250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] PATH_INFO: `/v3/users/e2224a51f6f043a19683b59c9cd0f276` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:24.379061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2224a51f6f043a19683b59c9cd0f276)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:24.381176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53cfa7-bfaf-40df-bd5a-edeea77dec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:24.382184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53cfa7-bfaf-40df-bd5a-edeea77dec9a admin admin] PATH_INFO: `/v3/projects/6dec478fc4e14bb789174f4dac1a3009/users/22131348360342dabb06a5b7878faa29/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:24.390447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:24.409593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53cfa7-bfaf-40df-bd5a-edeea77dec9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dec478fc4e14bb789174f4dac1a3009, user_id=22131348360342dabb06a5b7878faa29, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:24.416370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53cfa7-bfaf-40df-bd5a-edeea77dec9a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:24.463750 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3342/6676] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:24 2022] PUT /identity/v3/projects/6dec478fc4e14bb789174f4dac1a3009/users/22131348360342dabb06a5b7878faa29/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:24.477014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d19b-e50e-495e-9541-09f78f2194af None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:24.477505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d19b-e50e-495e-9541-09f78f2194af None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:24.477955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d19b-e50e-495e-9541-09f78f2194af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:24.519939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6bb5d19b-e50e-495e-9541-09f78f2194af None None] MFA Rules not processed for user `22131348360342dabb06a5b7878faa29`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:24.522932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6bb5d19b-e50e-495e-9541-09f78f2194af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:24.598455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2224a51f6f043a19683b59c9cd0f276'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:24.627738 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3343/6677] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:24.629137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5d19b-e50e-495e-9541-09f78f2194af None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:24.644057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:24.669158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2224a51f6f043a19683b59c9cd0f276'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:24.696278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2224a51f6f043a19683b59c9cd0f276'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:24.709399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2224a51f6f043a19683b59c9cd0f276'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:24.711513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-40c959f2-094f-43db-8738-e1c55d172f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2224a51f6f043a19683b59c9cd0f276'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:24.739030 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3335/6678] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:24 2022] DELETE /identity/v3/users/e2224a51f6f043a19683b59c9cd0f276 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:24.742501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2fd5-8768-45e0-847e-536955d102ea None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:24.744504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2fd5-8768-45e0-847e-536955d102ea None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:24.744879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2fd5-8768-45e0-847e-536955d102ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:24.753842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5d19b-e50e-495e-9541-09f78f2194af None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:24.799545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-34bb2fd5-8768-45e0-847e-536955d102ea None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:24.802399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-34bb2fd5-8768-45e0-847e-536955d102ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:24.911511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6316be-36a2-47ce-9f32-0e725dd8eeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:24.912671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6316be-36a2-47ce-9f32-0e725dd8eeab admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:24.912944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6316be-36a2-47ce-9f32-0e725dd8eeab admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:24.913391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6316be-36a2-47ce-9f32-0e725dd8eeab admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:24.914238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6316be-36a2-47ce-9f32-0e725dd8eeab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:24.916201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6316be-36a2-47ce-9f32-0e725dd8eeab admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:24.982351 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3344/6679] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:24 2022] POST /identity/v3/projects => generated 385 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:24.986122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6316be-36a2-47ce-9f32-0e725dd8eeab admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:24.991156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fe6316be-36a2-47ce-9f32-0e725dd8eeab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:25.018260 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3336/6680] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:25.064967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb2fd5-8768-45e0-847e-536955d102ea None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:25.066611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-34bb2fd5-8768-45e0-847e-536955d102ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:25.140734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:25.166471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6316be-36a2-47ce-9f32-0e725dd8eeab admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:25.176649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eb50e-612c-4d8d-8570-18ec6cc8aa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:25.178761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb50e-612c-4d8d-8570-18ec6cc8aa43 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:25.179277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb50e-612c-4d8d-8570-18ec6cc8aa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:25.179753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb50e-612c-4d8d-8570-18ec6cc8aa43 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:25.180928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44eb50e-612c-4d8d-8570-18ec6cc8aa43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:25.184434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44eb50e-612c-4d8d-8570-18ec6cc8aa43 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:25.224551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb2fd5-8768-45e0-847e-536955d102ea None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:25.235526 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3345/6681] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:24 2022] POST /identity/v3/users => generated 299 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:25.242278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eb50e-612c-4d8d-8570-18ec6cc8aa43 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:25.260424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eb50e-612c-4d8d-8570-18ec6cc8aa43 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:25.277354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b7b85-3b3e-4cb5-979d-9def763527b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:25.279035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b7b85-3b3e-4cb5-979d-9def763527b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:25.279035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b7b85-3b3e-4cb5-979d-9def763527b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:25.279035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b7b85-3b3e-4cb5-979d-9def763527b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:25.279739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b7b85-3b3e-4cb5-979d-9def763527b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:25.281550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b7b85-3b3e-4cb5-979d-9def763527b1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:25.308750 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3346/6682] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:25.309890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b7b85-3b3e-4cb5-979d-9def763527b1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:25.346477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b7b85-3b3e-4cb5-979d-9def763527b1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:25.365330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b99f48-0501-4bea-b0c7-0a5b7532df57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:25.368045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99f48-0501-4bea-b0c7-0a5b7532df57 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:25.368205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99f48-0501-4bea-b0c7-0a5b7532df57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:25.368486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99f48-0501-4bea-b0c7-0a5b7532df57 admin admin] PATH_INFO: `/v3/projects/80186644015d43c7abd3674ac9dff753/users/fd4c1675a5954f3d874268539397bd42/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:25.395032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cac9b6-9992-4fec-8aa2-9c4a3c44169e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:25.395684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cac9b6-9992-4fec-8aa2-9c4a3c44169e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:25.403153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cac9b6-9992-4fec-8aa2-9c4a3c44169e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:25.403607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cac9b6-9992-4fec-8aa2-9c4a3c44169e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:25.412953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b99f48-0501-4bea-b0c7-0a5b7532df57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80186644015d43c7abd3674ac9dff753, user_id=fd4c1675a5954f3d874268539397bd42, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:25.416668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b99f48-0501-4bea-b0c7-0a5b7532df57 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:25.420689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:25.426530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cac9b6-9992-4fec-8aa2-9c4a3c44169e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:25.430480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cac9b6-9992-4fec-8aa2-9c4a3c44169e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:25.445061 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3347/6683] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:25 2022] PUT /identity/v3/projects/80186644015d43c7abd3674ac9dff753/users/fd4c1675a5954f3d874268539397bd42/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:25.454255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b99f48-0501-4bea-b0c7-0a5b7532df57 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:25.475931 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3337/6684] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:25.601163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b99f48-0501-4bea-b0c7-0a5b7532df57 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:25.611006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34e6ca-4892-420f-b833-febc96ebc3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:25.625864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34e6ca-4892-420f-b833-febc96ebc3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:25.626271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34e6ca-4892-420f-b833-febc96ebc3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:25.626964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34e6ca-4892-420f-b833-febc96ebc3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:25.627434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34e6ca-4892-420f-b833-febc96ebc3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:25.629402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34e6ca-4892-420f-b833-febc96ebc3a8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:25.657348 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3348/6685] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:25.664739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cac9b6-9992-4fec-8aa2-9c4a3c44169e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:25.678828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cac9b6-9992-4fec-8aa2-9c4a3c44169e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:25.686107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8667d6f-4857-46ac-b252-7f758dc68cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:25.686275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8667d6f-4857-46ac-b252-7f758dc68cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:25.686502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8667d6f-4857-46ac-b252-7f758dc68cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:25.686926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8667d6f-4857-46ac-b252-7f758dc68cdb admin admin] PATH_INFO: `/v3/projects/80186644015d43c7abd3674ac9dff753/users/fd4c1675a5954f3d874268539397bd42/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:25.690822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8667d6f-4857-46ac-b252-7f758dc68cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=80186644015d43c7abd3674ac9dff753, user_id=fd4c1675a5954f3d874268539397bd42, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:25.695168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8667d6f-4857-46ac-b252-7f758dc68cdb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:25.716021 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3338/6686] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:25 2022] PUT /identity/v3/projects/80186644015d43c7abd3674ac9dff753/users/fd4c1675a5954f3d874268539397bd42/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:25.727423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f464a8-381a-4c6c-8659-b653bd8ac5d4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:25.728675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f464a8-381a-4c6c-8659-b653bd8ac5d4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:25.729480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f464a8-381a-4c6c-8659-b653bd8ac5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:25.798009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-50f464a8-381a-4c6c-8659-b653bd8ac5d4 None None] MFA Rules not processed for user `fd4c1675a5954f3d874268539397bd42`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:25.801990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-50f464a8-381a-4c6c-8659-b653bd8ac5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:25.999876 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3349/6687] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:26.008954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8667d6f-4857-46ac-b252-7f758dc68cdb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:26.012859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e8667d6f-4857-46ac-b252-7f758dc68cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:26.234239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8667d6f-4857-46ac-b252-7f758dc68cdb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:26.251021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d4a48-1291-4b19-925d-24a416a75396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:26.251689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d4a48-1291-4b19-925d-24a416a75396 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:26.252961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d4a48-1291-4b19-925d-24a416a75396 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:26.253264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d4a48-1291-4b19-925d-24a416a75396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:26.262354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5e9d4a48-1291-4b19-925d-24a416a75396 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:26.446748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d4a48-1291-4b19-925d-24a416a75396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:26.448882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d4a48-1291-4b19-925d-24a416a75396 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:26.475806 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3339/6688] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:28.036951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f464a8-381a-4c6c-8659-b653bd8ac5d4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:28.038739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-50f464a8-381a-4c6c-8659-b653bd8ac5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:28.183959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f464a8-381a-4c6c-8659-b653bd8ac5d4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:28.194696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:28.195495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:28.195761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:28.196153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] PATH_INFO: `/v3/projects/c78e90827d8c4f0eb91abbd2da06e509` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:28.197683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78e90827d8c4f0eb91abbd2da06e509)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:28.201029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:28.289871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:28.398688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78e90827d8c4f0eb91abbd2da06e509', 'request_id': 'req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:28.428919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78e90827d8c4f0eb91abbd2da06e509', 'request_id': 'req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:28.438454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78e90827d8c4f0eb91abbd2da06e509', 'request_id': 'req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:28.458206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] The token cache is being invalidate because project c78e90827d8c4f0eb91abbd2da06e509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:28.460111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:28.460111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:28.465885 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3350/6689] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:28 2022] DELETE /identity/v3/projects/c78e90827d8c4f0eb91abbd2da06e509 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:28.466936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d4a48-1291-4b19-925d-24a416a75396 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:28.492341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d4a48-1291-4b19-925d-24a416a75396 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:28.552343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:28.564240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:28.565873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:28.642576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:28.646488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:28.647413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:28.648758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] PATH_INFO: `/v3/users/df8363a970b540afaf1c39b040ad1d5d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:28.655527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df8363a970b540afaf1c39b040ad1d5d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:28.658784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:28.718109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d8714-ae03-4b6e-81b8-1d531c7dd3a3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:28.729929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a7729-85ac-48bb-9d6a-12c5fcb79d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:28.730387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a7729-85ac-48bb-9d6a-12c5fcb79d3f service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:28.730666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a7729-85ac-48bb-9d6a-12c5fcb79d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:28.730939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a7729-85ac-48bb-9d6a-12c5fcb79d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:28.736647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9f4a7729-85ac-48bb-9d6a-12c5fcb79d3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:28.836985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df8363a970b540afaf1c39b040ad1d5d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:28.867021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df8363a970b540afaf1c39b040ad1d5d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:28.888553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df8363a970b540afaf1c39b040ad1d5d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:28.901371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df8363a970b540afaf1c39b040ad1d5d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:28.902430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df8363a970b540afaf1c39b040ad1d5d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:28.916713 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3340/6690] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:28 2022] DELETE /identity/v3/users/df8363a970b540afaf1c39b040ad1d5d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:16:28.916713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:28.919297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:28.928486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a7729-85ac-48bb-9d6a-12c5fcb79d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:28.938199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a7729-85ac-48bb-9d6a-12c5fcb79d3f service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:28.967131 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3351/6691] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:29.184110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56fff8-d822-434a-8383-1d8cb31aebe2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:29.196830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fbf95-a125-4b94-b5fc-1ed4f3bb22b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:29.197507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fbf95-a125-4b94-b5fc-1ed4f3bb22b4 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:29.197863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fbf95-a125-4b94-b5fc-1ed4f3bb22b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:29.198142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fbf95-a125-4b94-b5fc-1ed4f3bb22b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:29.203198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-456fbf95-a125-4b94-b5fc-1ed4f3bb22b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:29.367468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fbf95-a125-4b94-b5fc-1ed4f3bb22b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:29.371269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fbf95-a125-4b94-b5fc-1ed4f3bb22b4 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:29.383381 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3341/6692] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:30.685488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a7729-85ac-48bb-9d6a-12c5fcb79d3f service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:30.687176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9f4a7729-85ac-48bb-9d6a-12c5fcb79d3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:30.743506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a7729-85ac-48bb-9d6a-12c5fcb79d3f service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:30.767439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245b80a-627e-4dce-b144-f379ecac131b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:30.769054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245b80a-627e-4dce-b144-f379ecac131b service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:30.769384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245b80a-627e-4dce-b144-f379ecac131b service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:30.769750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245b80a-627e-4dce-b144-f379ecac131b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:30.777412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6245b80a-627e-4dce-b144-f379ecac131b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:30.995314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:31.008906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245b80a-627e-4dce-b144-f379ecac131b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:31.010315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245b80a-627e-4dce-b144-f379ecac131b service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:31.053120 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3352/6693] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:31.143705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fbf95-a125-4b94-b5fc-1ed4f3bb22b4 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:31.145177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-456fbf95-a125-4b94-b5fc-1ed4f3bb22b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:31.175600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:31.429283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fbf95-a125-4b94-b5fc-1ed4f3bb22b4 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:31.465428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:31.466169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:31.466452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:31.466839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] PATH_INFO: `/v3/projects/bf06b4d5a71543078ce7a04f6044a7fe` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:31.476011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf06b4d5a71543078ce7a04f6044a7fe)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:31.478035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:31.679341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf06b4d5a71543078ce7a04f6044a7fe', 'request_id': 'req-663d8b90-5f63-4f14-8e20-a9203dc8c860'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:31.712171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf06b4d5a71543078ce7a04f6044a7fe', 'request_id': 'req-663d8b90-5f63-4f14-8e20-a9203dc8c860'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:31.733016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf06b4d5a71543078ce7a04f6044a7fe', 'request_id': 'req-663d8b90-5f63-4f14-8e20-a9203dc8c860'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:31.768106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] The token cache is being invalidate because project bf06b4d5a71543078ce7a04f6044a7fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:31.768448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:31.769159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-663d8b90-5f63-4f14-8e20-a9203dc8c860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:31.813570 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3342/6694] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:31 2022] DELETE /identity/v3/projects/bf06b4d5a71543078ce7a04f6044a7fe => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:32.766392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245b80a-627e-4dce-b144-f379ecac131b service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:32.767847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6245b80a-627e-4dce-b144-f379ecac131b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:32.809833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245b80a-627e-4dce-b144-f379ecac131b service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:33.030976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:33.031873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:33.033136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:33.034176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] PATH_INFO: `/v3/users/495cc21e67204ad6bd46e4be37f39578` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:33.036381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=495cc21e67204ad6bd46e4be37f39578)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:33.039891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:33.132425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '495cc21e67204ad6bd46e4be37f39578'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:33.143754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '495cc21e67204ad6bd46e4be37f39578'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:33.194010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '495cc21e67204ad6bd46e4be37f39578'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:33.205527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '495cc21e67204ad6bd46e4be37f39578'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:33.206391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '495cc21e67204ad6bd46e4be37f39578'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:33.216054 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3353/6695] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:32 2022] DELETE /identity/v3/users/495cc21e67204ad6bd46e4be37f39578 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:33.220348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb70c4-d3fb-481f-8262-ab2759e06cb4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:33.221276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb70c4-d3fb-481f-8262-ab2759e06cb4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:33.221703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb70c4-d3fb-481f-8262-ab2759e06cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:33.273839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-eebb70c4-d3fb-481f-8262-ab2759e06cb4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:33.280259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eebb70c4-d3fb-481f-8262-ab2759e06cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:33.428483 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3343/6696] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:33.464967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:33.467869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:33.679351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a2a29-3b41-4a0a-ac72-f2a81ce562b3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:33.688877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb67a3c-80b6-4ed3-9505-2fe71c3cf2cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:33.689881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb67a3c-80b6-4ed3-9505-2fe71c3cf2cf service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:33.690324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb67a3c-80b6-4ed3-9505-2fe71c3cf2cf service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:33.690731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb67a3c-80b6-4ed3-9505-2fe71c3cf2cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:33.716262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb67a3c-80b6-4ed3-9505-2fe71c3cf2cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:33.717749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb67a3c-80b6-4ed3-9505-2fe71c3cf2cf service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:33.731032 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3354/6697] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:34.633535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb70c4-d3fb-481f-8262-ab2759e06cb4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:34.635006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eebb70c4-d3fb-481f-8262-ab2759e06cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:34.689340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb70c4-d3fb-481f-8262-ab2759e06cb4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:34.714394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:34.714805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:34.715091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:34.717357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] PATH_INFO: `/v3/projects/ca3eff2089c64b8dbafd63f4c80f2949` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:34.723041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca3eff2089c64b8dbafd63f4c80f2949)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:34.733775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:34.842606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:34.895632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca3eff2089c64b8dbafd63f4c80f2949', 'request_id': 'req-b93a54e4-36a9-455a-865d-28fd9a593b40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:34.920434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca3eff2089c64b8dbafd63f4c80f2949', 'request_id': 'req-b93a54e4-36a9-455a-865d-28fd9a593b40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:34.930339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca3eff2089c64b8dbafd63f4c80f2949', 'request_id': 'req-b93a54e4-36a9-455a-865d-28fd9a593b40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:34.947573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] The token cache is being invalidate because project ca3eff2089c64b8dbafd63f4c80f2949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:34.947980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:34.952000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:34.954974 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3344/6698] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:34 2022] DELETE /identity/v3/projects/ca3eff2089c64b8dbafd63f4c80f2949 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:34.959673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb67a3c-80b6-4ed3-9505-2fe71c3cf2cf service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:34.978799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb67a3c-80b6-4ed3-9505-2fe71c3cf2cf service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:35.021865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:35.127457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:35.133271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:35.133784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:35.134911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] PATH_INFO: `/v3/users/b8a49ec85b274f7a9efdec93e3f85297` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:35.136971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8a49ec85b274f7a9efdec93e3f85297)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:35.140090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:35.265751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8a49ec85b274f7a9efdec93e3f85297'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:35.323088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8a49ec85b274f7a9efdec93e3f85297'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:35.389297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8a49ec85b274f7a9efdec93e3f85297'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:35.404943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8a49ec85b274f7a9efdec93e3f85297'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:35.406012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62639e04-969d-4aba-b57f-786e3d808dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8a49ec85b274f7a9efdec93e3f85297'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:35.419083 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3355/6699] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:34 2022] DELETE /identity/v3/users/b8a49ec85b274f7a9efdec93e3f85297 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:36.237417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:36.238746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:36.265173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a54e4-36a9-455a-865d-28fd9a593b40 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:36.312152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b2b06-53c7-452a-961b-82700de4d505 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:36.312690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b2b06-53c7-452a-961b-82700de4d505 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:36.312770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b2b06-53c7-452a-961b-82700de4d505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:36.387369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2b2b2b06-53c7-452a-961b-82700de4d505 None None] MFA Rules not processed for user `655b3dac7ef04774a94c1294ad38b292`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:36.388601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2b2b2b06-53c7-452a-961b-82700de4d505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:36.393403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:36.395225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:36.395635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:36.396685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] PATH_INFO: `/v3/users/3a946be91d624f038c36ccaffb601651` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:36.400222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a946be91d624f038c36ccaffb601651)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:36.402638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:36.509506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a946be91d624f038c36ccaffb601651'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:36.557403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a946be91d624f038c36ccaffb601651'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:36.573901 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3356/6700] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:36.575322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47a302-a0e8-4c37-9c01-99ea1377463d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:36.575619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47a302-a0e8-4c37-9c01-99ea1377463d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:36.575896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47a302-a0e8-4c37-9c01-99ea1377463d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:36.576132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a946be91d624f038c36ccaffb601651'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:36.592978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a946be91d624f038c36ccaffb601651'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:36.596543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-25b8ebb1-a62e-461a-9057-cb2e8bbecb4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a946be91d624f038c36ccaffb601651'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:36.607904 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3345/6701] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:36 2022] DELETE /identity/v3/users/3a946be91d624f038c36ccaffb601651 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:36.612633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eddb9-b2ef-4588-be88-fa863421485f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:36.613214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eddb9-b2ef-4588-be88-fa863421485f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:36.613696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eddb9-b2ef-4588-be88-fa863421485f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:36.637358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7b47a302-a0e8-4c37-9c01-99ea1377463d None None] MFA Rules not processed for user `655b3dac7ef04774a94c1294ad38b292`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:36.639842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-380eddb9-b2ef-4588-be88-fa863421485f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:36.641357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-380eddb9-b2ef-4588-be88-fa863421485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:36.651329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7b47a302-a0e8-4c37-9c01-99ea1377463d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:36.699879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:36.713967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:36.802663 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3357/6702] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:36.808709 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3346/6703] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:16:36.815562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47a302-a0e8-4c37-9c01-99ea1377463d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:36.817523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7b47a302-a0e8-4c37-9c01-99ea1377463d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:36.827921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380eddb9-b2ef-4588-be88-fa863421485f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:36.832394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-380eddb9-b2ef-4588-be88-fa863421485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:36.980509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47a302-a0e8-4c37-9c01-99ea1377463d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:36.992501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3703a-321b-4179-b34a-43831ce115a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:36.994545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3703a-321b-4179-b34a-43831ce115a6 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:36.995029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3703a-321b-4179-b34a-43831ce115a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:36.995444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3703a-321b-4179-b34a-43831ce115a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:37.031370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3703a-321b-4179-b34a-43831ce115a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:37.035233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3703a-321b-4179-b34a-43831ce115a6 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:37.067320 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3358/6704] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:37.086448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380eddb9-b2ef-4588-be88-fa863421485f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:37.096167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c5dc2-666c-4607-8b8a-e325bb0296c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:37.096879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c5dc2-666c-4607-8b8a-e325bb0296c0 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:37.097443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c5dc2-666c-4607-8b8a-e325bb0296c0 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:37.097788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c5dc2-666c-4607-8b8a-e325bb0296c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:37.100000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-067c5dc2-666c-4607-8b8a-e325bb0296c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:37.117471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c5dc2-666c-4607-8b8a-e325bb0296c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:37.119456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c5dc2-666c-4607-8b8a-e325bb0296c0 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:37.138816 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3347/6705] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:37.413600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3703a-321b-4179-b34a-43831ce115a6 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:37.415154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6df3703a-321b-4179-b34a-43831ce115a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:37.449893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3703a-321b-4179-b34a-43831ce115a6 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:37.463262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:37.464320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:37.464699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:37.465259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] PATH_INFO: `/v3/projects/4cd7d5f2b59a4db891257613d436f046` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:37.466970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd7d5f2b59a4db891257613d436f046)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:37.470331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:37.607857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd7d5f2b59a4db891257613d436f046', 'request_id': 'req-9fa8650c-7582-4b2d-9103-56bbd8fdce06'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:37.628415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd7d5f2b59a4db891257613d436f046', 'request_id': 'req-9fa8650c-7582-4b2d-9103-56bbd8fdce06'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:37.652997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd7d5f2b59a4db891257613d436f046', 'request_id': 'req-9fa8650c-7582-4b2d-9103-56bbd8fdce06'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:37.675072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] The token cache is being invalidate because project 4cd7d5f2b59a4db891257613d436f046 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:37.675465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:37.676723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9fa8650c-7582-4b2d-9103-56bbd8fdce06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:37.680386 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3359/6706] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:37 2022] DELETE /identity/v3/projects/4cd7d5f2b59a4db891257613d436f046 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:37.692564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c5dc2-666c-4607-8b8a-e325bb0296c0 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:37.712569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c5dc2-666c-4607-8b8a-e325bb0296c0 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:37.893308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:37.895111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:37.895564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:37.897437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] PATH_INFO: `/v3/users/4574934f8e4b4a0da36a97a63e8850ab` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:37.899022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4574934f8e4b4a0da36a97a63e8850ab)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:37.901672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:38.019123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4574934f8e4b4a0da36a97a63e8850ab'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:38.022629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:38.044455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4574934f8e4b4a0da36a97a63e8850ab'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:38.058049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4574934f8e4b4a0da36a97a63e8850ab'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:38.076021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4574934f8e4b4a0da36a97a63e8850ab'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:38.077349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4574934f8e4b4a0da36a97a63e8850ab'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:38.088589 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3348/6707] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:37 2022] DELETE /identity/v3/users/4574934f8e4b4a0da36a97a63e8850ab => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:38.556032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582825b-7662-4256-949b-beeae1bbad5c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:38.556901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582825b-7662-4256-949b-beeae1bbad5c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:38.557352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582825b-7662-4256-949b-beeae1bbad5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:38.621381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2582825b-7662-4256-949b-beeae1bbad5c None None] MFA Rules not processed for user `0d15db49ef774494baa77a9ec31b22c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:38.622769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2582825b-7662-4256-949b-beeae1bbad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:38.778168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:38.829797 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3360/6708] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:38.844509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:38.847451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:38.988531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac287d-15ba-4e89-a96c-39fe8d1a7b2a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:39.021417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fd9d7-ebcb-46b0-8d0e-09d48bd823f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:39.022689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fd9d7-ebcb-46b0-8d0e-09d48bd823f9 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:39.023153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fd9d7-ebcb-46b0-8d0e-09d48bd823f9 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:39.023943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fd9d7-ebcb-46b0-8d0e-09d48bd823f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:39.029947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582825b-7662-4256-949b-beeae1bbad5c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:39.031537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2582825b-7662-4256-949b-beeae1bbad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:39.042480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fd9d7-ebcb-46b0-8d0e-09d48bd823f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:39.044548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fd9d7-ebcb-46b0-8d0e-09d48bd823f9 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:39.061557 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3349/6709] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:39.183580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582825b-7662-4256-949b-beeae1bbad5c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:39.215236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:39.216013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:39.216379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:39.216803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] PATH_INFO: `/v3/projects/598097829bcb44a083af8e17215e2e43` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:39.218198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=598097829bcb44a083af8e17215e2e43)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:39.220828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:39.346995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '598097829bcb44a083af8e17215e2e43', 'request_id': 'req-135721d9-146f-48b4-b63d-e619fc3126e0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:39.368858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '598097829bcb44a083af8e17215e2e43', 'request_id': 'req-135721d9-146f-48b4-b63d-e619fc3126e0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:39.377379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '598097829bcb44a083af8e17215e2e43', 'request_id': 'req-135721d9-146f-48b4-b63d-e619fc3126e0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:39.392015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] The token cache is being invalidate because project 598097829bcb44a083af8e17215e2e43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:39.392432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:39.401160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:39.404275 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3361/6710] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:39 2022] DELETE /identity/v3/projects/598097829bcb44a083af8e17215e2e43 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:16:39.408193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fd9d7-ebcb-46b0-8d0e-09d48bd823f9 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:39.433493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fd9d7-ebcb-46b0-8d0e-09d48bd823f9 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:39.704607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:39.710815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:39.711133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:39.711485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] PATH_INFO: `/v3/users/22131348360342dabb06a5b7878faa29` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:39.717665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22131348360342dabb06a5b7878faa29)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:39.739454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:39.905938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22131348360342dabb06a5b7878faa29'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:39.924220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:39.949380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22131348360342dabb06a5b7878faa29'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:39.980973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22131348360342dabb06a5b7878faa29'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:39.998950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22131348360342dabb06a5b7878faa29'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:40.002328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22131348360342dabb06a5b7878faa29'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:40.018621 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3350/6711] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:39 2022] DELETE /identity/v3/users/22131348360342dabb06a5b7878faa29 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:40.236344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:40.237837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:40.262537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135721d9-146f-48b4-b63d-e619fc3126e0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:40.433769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:40.434430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:40.434708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:40.434978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] PATH_INFO: `/v3/projects/2b79c048af684c74a20a60aa75e312e2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:40.439970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b79c048af684c74a20a60aa75e312e2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:40.447541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:40.480984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:40.525652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:40.528971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:40.595427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b79c048af684c74a20a60aa75e312e2', 'request_id': 'req-6b56c3da-fe3d-41cc-8266-bd7cbea41078'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:40.624203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b79c048af684c74a20a60aa75e312e2', 'request_id': 'req-6b56c3da-fe3d-41cc-8266-bd7cbea41078'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:40.637713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b79c048af684c74a20a60aa75e312e2', 'request_id': 'req-6b56c3da-fe3d-41cc-8266-bd7cbea41078'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:40.691418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] The token cache is being invalidate because project 2b79c048af684c74a20a60aa75e312e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:40.691642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:40.697810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b56c3da-fe3d-41cc-8266-bd7cbea41078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:40.709653 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3362/6712] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:40 2022] DELETE /identity/v3/projects/2b79c048af684c74a20a60aa75e312e2 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:40.715274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebdf87-11d6-40d9-b215-b8474035cc84 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:40.715559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebdf87-11d6-40d9-b215-b8474035cc84 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:40.715901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebdf87-11d6-40d9-b215-b8474035cc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:40.758701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad898c-c7d1-49e4-9c39-563883191088 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:40.787974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-60ebdf87-11d6-40d9-b215-b8474035cc84 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:40.789839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-60ebdf87-11d6-40d9-b215-b8474035cc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:40.962336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681c237-830e-4695-aca2-cff675af9f15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:40.983011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681c237-830e-4695-aca2-cff675af9f15 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:40.983709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681c237-830e-4695-aca2-cff675af9f15 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:40.985711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681c237-830e-4695-aca2-cff675af9f15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:40.995437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d681c237-830e-4695-aca2-cff675af9f15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:41.008891 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3363/6713] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:16:41.012059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ee346-a873-480d-a923-4f3a8e53725b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:41.016260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ee346-a873-480d-a923-4f3a8e53725b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:41.016680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ee346-a873-480d-a923-4f3a8e53725b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:41.048577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681c237-830e-4695-aca2-cff675af9f15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:41.052142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681c237-830e-4695-aca2-cff675af9f15 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:41.069594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f11ee346-a873-480d-a923-4f3a8e53725b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:41.072500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f11ee346-a873-480d-a923-4f3a8e53725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:41.095140 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3364/6714] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:41.099039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ee346-a873-480d-a923-4f3a8e53725b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:41.107162 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3351/6715] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:16:41.116362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ee346-a873-480d-a923-4f3a8e53725b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:41.132400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b03480-fda3-42b8-b26f-4fd01fca785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:41.133086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b03480-fda3-42b8-b26f-4fd01fca785d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:41.133358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b03480-fda3-42b8-b26f-4fd01fca785d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:41.133612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b03480-fda3-42b8-b26f-4fd01fca785d admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:41.134275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b03480-fda3-42b8-b26f-4fd01fca785d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:41.134576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b03480-fda3-42b8-b26f-4fd01fca785d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:41.142251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b03480-fda3-42b8-b26f-4fd01fca785d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:41.232659 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3365/6716] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:16:41.236000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681c237-830e-4695-aca2-cff675af9f15 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:41.297091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681c237-830e-4695-aca2-cff675af9f15 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:41.323783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4bec-0e95-49ac-bed6-096404516a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:41.324412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4bec-0e95-49ac-bed6-096404516a83 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:41.324688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4bec-0e95-49ac-bed6-096404516a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:41.324954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4bec-0e95-49ac-bed6-096404516a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:41.325612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a4bec-0e95-49ac-bed6-096404516a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:41.327539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a4bec-0e95-49ac-bed6-096404516a83 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:41.431930 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3352/6717] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:41 2022] POST /identity/v3/projects => generated 409 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:41.433717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b03480-fda3-42b8-b26f-4fd01fca785d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:41.449455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b03480-fda3-42b8-b26f-4fd01fca785d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:41.699631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c882045-d2f8-4916-b268-2f5fb1676dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:41.700212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c882045-d2f8-4916-b268-2f5fb1676dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:41.700506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c882045-d2f8-4916-b268-2f5fb1676dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:41.701123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c882045-d2f8-4916-b268-2f5fb1676dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:41.701446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c882045-d2f8-4916-b268-2f5fb1676dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:41.703439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c882045-d2f8-4916-b268-2f5fb1676dc0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:41.747644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4bec-0e95-49ac-bed6-096404516a83 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:41.750372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-025a4bec-0e95-49ac-bed6-096404516a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:41.785609 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3366/6718] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:41 2022] POST /identity/v3/users => generated 305 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:41.788344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c882045-d2f8-4916-b268-2f5fb1676dc0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:41.826124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c882045-d2f8-4916-b268-2f5fb1676dc0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:41.857072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8e3a2-46ab-43b5-963f-10d4f51a2567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:41.857692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8e3a2-46ab-43b5-963f-10d4f51a2567 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:41.866307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8e3a2-46ab-43b5-963f-10d4f51a2567 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:41.867017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8e3a2-46ab-43b5-963f-10d4f51a2567 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:41.867755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8e3a2-46ab-43b5-963f-10d4f51a2567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:41.878485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8e3a2-46ab-43b5-963f-10d4f51a2567 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:41.910503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:41.944595 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3367/6719] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:41.957326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8e3a2-46ab-43b5-963f-10d4f51a2567 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:41.985149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:41.996955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8e3a2-46ab-43b5-963f-10d4f51a2567 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:42.017547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645bb41-296a-4f7b-95d7-2e8501bc73b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:42.018181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645bb41-296a-4f7b-95d7-2e8501bc73b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:42.018459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645bb41-296a-4f7b-95d7-2e8501bc73b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:42.018731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645bb41-296a-4f7b-95d7-2e8501bc73b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:42.019386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645bb41-296a-4f7b-95d7-2e8501bc73b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:42.031643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645bb41-296a-4f7b-95d7-2e8501bc73b3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:42.076124 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3368/6720] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:42.079365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645bb41-296a-4f7b-95d7-2e8501bc73b3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:42.090581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4bec-0e95-49ac-bed6-096404516a83 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:42.118946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8eba2-fdfa-4a72-baed-2b17a66c195e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:42.119585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8eba2-fdfa-4a72-baed-2b17a66c195e service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:42.120654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8eba2-fdfa-4a72-baed-2b17a66c195e service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:42.120934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8eba2-fdfa-4a72-baed-2b17a66c195e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:42.122045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645bb41-296a-4f7b-95d7-2e8501bc73b3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:42.123043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-86a8eba2-fdfa-4a72-baed-2b17a66c195e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:42.136947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2d16f-729b-4b1e-a3c1-f1d7213b5c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:42.138358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2d16f-729b-4b1e-a3c1-f1d7213b5c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:42.140211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2d16f-729b-4b1e-a3c1-f1d7213b5c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:42.140726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2d16f-729b-4b1e-a3c1-f1d7213b5c87 admin admin] PATH_INFO: `/v3/projects/fc8a27fa09a04c3d84bca7b917da5358/users/8a59e6c4d365463183cc011731255c08/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:42.156302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8eba2-fdfa-4a72-baed-2b17a66c195e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:42.157999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8eba2-fdfa-4a72-baed-2b17a66c195e service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:42.193333 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3353/6721] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:42.193887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2d16f-729b-4b1e-a3c1-f1d7213b5c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc8a27fa09a04c3d84bca7b917da5358, user_id=8a59e6c4d365463183cc011731255c08, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:42.206061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8eba2-fdfa-4a72-baed-2b17a66c195e service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:42.215436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2d16f-729b-4b1e-a3c1-f1d7213b5c87 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:42.216290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-86a8eba2-fdfa-4a72-baed-2b17a66c195e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:42.268591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8eba2-fdfa-4a72-baed-2b17a66c195e service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:42.283854 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3369/6722] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:42 2022] PUT /identity/v3/projects/fc8a27fa09a04c3d84bca7b917da5358/users/8a59e6c4d365463183cc011731255c08/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:42.297728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbd6e5-ea82-4f8e-aace-822d682a1edd None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:42.298106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbd6e5-ea82-4f8e-aace-822d682a1edd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:42.298390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbd6e5-ea82-4f8e-aace-822d682a1edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:42.442589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-69bbd6e5-ea82-4f8e-aace-822d682a1edd None None] MFA Rules not processed for user `8a59e6c4d365463183cc011731255c08`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:42.445075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-69bbd6e5-ea82-4f8e-aace-822d682a1edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:42.460803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:42.461656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:42.463817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:42.464215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] PATH_INFO: `/v3/projects/6dec478fc4e14bb789174f4dac1a3009` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:42.468705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dec478fc4e14bb789174f4dac1a3009)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:42.470784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:42.565157 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3370/6723] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:42 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:42.570188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbd6e5-ea82-4f8e-aace-822d682a1edd None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:42.591927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbd6e5-ea82-4f8e-aace-822d682a1edd None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:42.624238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dec478fc4e14bb789174f4dac1a3009', 'request_id': 'req-e1884195-cf7d-4e2b-ac57-2403c2106d85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:42.635616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dec478fc4e14bb789174f4dac1a3009', 'request_id': 'req-e1884195-cf7d-4e2b-ac57-2403c2106d85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:42.648400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dec478fc4e14bb789174f4dac1a3009', 'request_id': 'req-e1884195-cf7d-4e2b-ac57-2403c2106d85'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:42.669937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] The token cache is being invalidate because project 6dec478fc4e14bb789174f4dac1a3009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:42.670321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:42.671069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:42.689167 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3354/6724] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:42 2022] DELETE /identity/v3/projects/6dec478fc4e14bb789174f4dac1a3009 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:42.694359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:42.697626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:42.767773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a36241-f477-4170-bff3-b9a988f1e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:42.769487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a36241-f477-4170-bff3-b9a988f1e95d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:42.770464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a36241-f477-4170-bff3-b9a988f1e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:42.772649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a36241-f477-4170-bff3-b9a988f1e95d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:42.774350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a36241-f477-4170-bff3-b9a988f1e95d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:42.778991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a36241-f477-4170-bff3-b9a988f1e95d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:42.890817 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3371/6725] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:42 2022] POST /identity/v3/projects => generated 407 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:16:42.891942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a36241-f477-4170-bff3-b9a988f1e95d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:42.902933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a8a36241-f477-4170-bff3-b9a988f1e95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:42.951019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:42.978708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1884195-cf7d-4e2b-ac57-2403c2106d85 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:43.015206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:43.075496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a36241-f477-4170-bff3-b9a988f1e95d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:43.093562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aea29c-1147-48de-acd4-62ddb4f65cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:43.094417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aea29c-1147-48de-acd4-62ddb4f65cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:43.095052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aea29c-1147-48de-acd4-62ddb4f65cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:43.095565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aea29c-1147-48de-acd4-62ddb4f65cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:43.096397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aea29c-1147-48de-acd4-62ddb4f65cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:43.112914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aea29c-1147-48de-acd4-62ddb4f65cd4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:43.149110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:43.149912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:43.150331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:43.150811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] PATH_INFO: `/v3/users/fd4c1675a5954f3d874268539397bd42` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:43.153128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4c1675a5954f3d874268539397bd42)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:43.159808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:43.245443 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3372/6726] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:42 2022] POST /identity/v3/users => generated 310 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:43.248373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aea29c-1147-48de-acd4-62ddb4f65cd4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:43.264224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aea29c-1147-48de-acd4-62ddb4f65cd4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:43.272494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d020c31-903d-43d4-858b-952d60c31498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:43.273379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d020c31-903d-43d4-858b-952d60c31498 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:43.274134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d020c31-903d-43d4-858b-952d60c31498 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:43.274299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d020c31-903d-43d4-858b-952d60c31498 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:43.274935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d020c31-903d-43d4-858b-952d60c31498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:43.277287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d020c31-903d-43d4-858b-952d60c31498 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:43.292090 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3373/6727] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:43.295242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4c1675a5954f3d874268539397bd42'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:43.297661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d020c31-903d-43d4-858b-952d60c31498 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:43.312923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d020c31-903d-43d4-858b-952d60c31498 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:43.319917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e011447-aad0-4e92-bbbc-b6e6474323c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:43.320715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e011447-aad0-4e92-bbbc-b6e6474323c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:43.321210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e011447-aad0-4e92-bbbc-b6e6474323c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:43.321549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e011447-aad0-4e92-bbbc-b6e6474323c8 admin admin] PATH_INFO: `/v3/projects/6345d4f208a54109a0a12b4ba5609d0e/users/1464a912add04ac4bfd70198ca9a6297/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:43.337699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4c1675a5954f3d874268539397bd42'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:43.353170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e011447-aad0-4e92-bbbc-b6e6474323c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6345d4f208a54109a0a12b4ba5609d0e, user_id=1464a912add04ac4bfd70198ca9a6297, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:43.356412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e011447-aad0-4e92-bbbc-b6e6474323c8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:43.357378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4c1675a5954f3d874268539397bd42'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:43.376779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4c1675a5954f3d874268539397bd42'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:43.379074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4c1675a5954f3d874268539397bd42'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:43.384324 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3374/6728] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:43 2022] PUT /identity/v3/projects/6345d4f208a54109a0a12b4ba5609d0e/users/1464a912add04ac4bfd70198ca9a6297/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:43.387944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e011447-aad0-4e92-bbbc-b6e6474323c8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:43.389331 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3355/6729] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:42 2022] DELETE /identity/v3/users/fd4c1675a5954f3d874268539397bd42 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:43.390223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e011447-aad0-4e92-bbbc-b6e6474323c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:43.517921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e011447-aad0-4e92-bbbc-b6e6474323c8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:43.696251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c425cc3-89b7-4586-ad38-bbb72bdd680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:43.705522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c425cc3-89b7-4586-ad38-bbb72bdd680d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:43.705806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c425cc3-89b7-4586-ad38-bbb72bdd680d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:43.706075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c425cc3-89b7-4586-ad38-bbb72bdd680d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:43.706741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c425cc3-89b7-4586-ad38-bbb72bdd680d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:43.713562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c425cc3-89b7-4586-ad38-bbb72bdd680d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:43.737921 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3375/6730] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:16:43.741638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:43.755982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0981ec-7bd6-46e0-b433-908546dbdfeb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:43.771565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784ccd-940f-40c7-a907-ccb7adc44f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:43.774956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784ccd-940f-40c7-a907-ccb7adc44f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:43.775280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784ccd-940f-40c7-a907-ccb7adc44f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:43.775555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784ccd-940f-40c7-a907-ccb7adc44f0e admin admin] PATH_INFO: `/v3/projects/6345d4f208a54109a0a12b4ba5609d0e/users/1464a912add04ac4bfd70198ca9a6297/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:43.779756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10784ccd-940f-40c7-a907-ccb7adc44f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6345d4f208a54109a0a12b4ba5609d0e, user_id=1464a912add04ac4bfd70198ca9a6297, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:43.789779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10784ccd-940f-40c7-a907-ccb7adc44f0e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:43.819491 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3356/6731] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:43 2022] PUT /identity/v3/projects/6345d4f208a54109a0a12b4ba5609d0e/users/1464a912add04ac4bfd70198ca9a6297/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:43.830671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfecaf-4388-43bd-a14e-0b5f2d319fa3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:43.831011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfecaf-4388-43bd-a14e-0b5f2d319fa3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:43.831284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfecaf-4388-43bd-a14e-0b5f2d319fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:43.937268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-05bfecaf-4388-43bd-a14e-0b5f2d319fa3 None None] MFA Rules not processed for user `1464a912add04ac4bfd70198ca9a6297`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:43.949109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-05bfecaf-4388-43bd-a14e-0b5f2d319fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:44.036485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:44.128875 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3376/6732] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:44.144826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784ccd-940f-40c7-a907-ccb7adc44f0e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:44.147225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-10784ccd-940f-40c7-a907-ccb7adc44f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:44.294359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784ccd-940f-40c7-a907-ccb7adc44f0e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:44.303774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be650e79-ce7c-40f3-8493-da632d7295e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:44.303774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be650e79-ce7c-40f3-8493-da632d7295e5 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:44.303774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be650e79-ce7c-40f3-8493-da632d7295e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:44.304094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be650e79-ce7c-40f3-8493-da632d7295e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:44.340897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be650e79-ce7c-40f3-8493-da632d7295e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:44.342433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be650e79-ce7c-40f3-8493-da632d7295e5 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:44.407979 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3357/6733] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:44 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:16:44.786659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfecaf-4388-43bd-a14e-0b5f2d319fa3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:44.805548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfecaf-4388-43bd-a14e-0b5f2d319fa3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:44.818094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cf50f-de66-4e65-be24-66227f5eab99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:44.820483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf50f-de66-4e65-be24-66227f5eab99 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:44.820762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf50f-de66-4e65-be24-66227f5eab99 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:44.821028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf50f-de66-4e65-be24-66227f5eab99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:44.829704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-980cf50f-de66-4e65-be24-66227f5eab99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:44.989429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be650e79-ce7c-40f3-8493-da632d7295e5 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:44.992188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-be650e79-ce7c-40f3-8493-da632d7295e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:45.023634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:45.025594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cf50f-de66-4e65-be24-66227f5eab99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:45.029526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cf50f-de66-4e65-be24-66227f5eab99 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:45.057961 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3377/6734] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:44 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:45.173691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be650e79-ce7c-40f3-8493-da632d7295e5 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:45.180443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:45.181760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:45.182258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:45.183003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] PATH_INFO: `/v3/projects/80186644015d43c7abd3674ac9dff753` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:45.184554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80186644015d43c7abd3674ac9dff753)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:45.187118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:45.336449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80186644015d43c7abd3674ac9dff753', 'request_id': 'req-c831ddeb-8f29-4280-8c6c-f3d3754f7935'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:45.356301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80186644015d43c7abd3674ac9dff753', 'request_id': 'req-c831ddeb-8f29-4280-8c6c-f3d3754f7935'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:45.383804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80186644015d43c7abd3674ac9dff753', 'request_id': 'req-c831ddeb-8f29-4280-8c6c-f3d3754f7935'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:45.424821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] The token cache is being invalidate because project 80186644015d43c7abd3674ac9dff753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:45.426416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:45.428390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c831ddeb-8f29-4280-8c6c-f3d3754f7935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:45.434815 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3358/6735] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:44 2022] DELETE /identity/v3/projects/80186644015d43c7abd3674ac9dff753 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:16:45.439702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7c799-4a28-413b-985c-08190c00b16c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:45.441795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7c799-4a28-413b-985c-08190c00b16c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:45.442350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7c799-4a28-413b-985c-08190c00b16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:45.470366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3ba7c799-4a28-413b-985c-08190c00b16c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:45.473894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3ba7c799-4a28-413b-985c-08190c00b16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:45.599370 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3378/6736] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:45.611216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4fb7c-2d46-4110-a33d-37fc2091cc29 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:45.614456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4fb7c-2d46-4110-a33d-37fc2091cc29 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:45.614884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4fb7c-2d46-4110-a33d-37fc2091cc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:45.673457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-97a4fb7c-2d46-4110-a33d-37fc2091cc29 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:45.674678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-97a4fb7c-2d46-4110-a33d-37fc2091cc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:45.717391 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3359/6737] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:45.719753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7c799-4a28-413b-985c-08190c00b16c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:45.735508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7c799-4a28-413b-985c-08190c00b16c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:45.750051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0af39b-49fc-4418-8e96-dd6163efa103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:45.752295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0af39b-49fc-4418-8e96-dd6163efa103 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:45.753619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0af39b-49fc-4418-8e96-dd6163efa103 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:45.753940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0af39b-49fc-4418-8e96-dd6163efa103 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:45.756009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0af39b-49fc-4418-8e96-dd6163efa103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:45.757956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0af39b-49fc-4418-8e96-dd6163efa103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:45.763072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0af39b-49fc-4418-8e96-dd6163efa103 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:45.810191 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3379/6738] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:16:45.811664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4fb7c-2d46-4110-a33d-37fc2091cc29 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:45.831549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4fb7c-2d46-4110-a33d-37fc2091cc29 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:45.856692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e781b18-1033-468a-bb2b-6ad37ba1c835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:45.857346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e781b18-1033-468a-bb2b-6ad37ba1c835 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:45.862065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e781b18-1033-468a-bb2b-6ad37ba1c835 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:45.862421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e781b18-1033-468a-bb2b-6ad37ba1c835 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:45.865534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e781b18-1033-468a-bb2b-6ad37ba1c835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:45.874555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e781b18-1033-468a-bb2b-6ad37ba1c835 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:45.988313 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3360/6739] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:45 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:45.992595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0af39b-49fc-4418-8e96-dd6163efa103 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:46.014154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0af39b-49fc-4418-8e96-dd6163efa103 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:46.144688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:46.162979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec6f51-2e2b-4723-a010-f81012ee2782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:46.163714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec6f51-2e2b-4723-a010-f81012ee2782 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:46.164136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec6f51-2e2b-4723-a010-f81012ee2782 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:46.164505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec6f51-2e2b-4723-a010-f81012ee2782 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:46.165312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec6f51-2e2b-4723-a010-f81012ee2782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:46.167446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec6f51-2e2b-4723-a010-f81012ee2782 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:46.225468 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3380/6740] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:45 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:46.230346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e781b18-1033-468a-bb2b-6ad37ba1c835 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:46.260466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e781b18-1033-468a-bb2b-6ad37ba1c835 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:46.279133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582e16c-ca17-4e3b-8d47-63b3c87c1d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:46.279587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582e16c-ca17-4e3b-8d47-63b3c87c1d0a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:46.279865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582e16c-ca17-4e3b-8d47-63b3c87c1d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:46.280134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582e16c-ca17-4e3b-8d47-63b3c87c1d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:46.281626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582e16c-ca17-4e3b-8d47-63b3c87c1d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:46.291216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582e16c-ca17-4e3b-8d47-63b3c87c1d0a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:46.351431 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3361/6741] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:16:46.357191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec6f51-2e2b-4723-a010-f81012ee2782 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:46.372627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec6f51-2e2b-4723-a010-f81012ee2782 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:46.382752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe6649-a970-42ac-9a13-9d6b4ae19d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:46.383507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe6649-a970-42ac-9a13-9d6b4ae19d02 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:46.383957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe6649-a970-42ac-9a13-9d6b4ae19d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:46.384340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe6649-a970-42ac-9a13-9d6b4ae19d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:46.385110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe6649-a970-42ac-9a13-9d6b4ae19d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:46.387433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe6649-a970-42ac-9a13-9d6b4ae19d02 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:46.416718 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3381/6742] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:46.420856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582e16c-ca17-4e3b-8d47-63b3c87c1d0a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:46.435520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582e16c-ca17-4e3b-8d47-63b3c87c1d0a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:46.442937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be75e7-657c-47f4-9ab6-765eef349fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:46.444212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be75e7-657c-47f4-9ab6-765eef349fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:46.444657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be75e7-657c-47f4-9ab6-765eef349fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:46.445130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be75e7-657c-47f4-9ab6-765eef349fb2 admin admin] PATH_INFO: `/v3/projects/93b76fd929ef4db587b2e4d6a32708d0/users/cce3066485e44c84a0ee52d58c18595c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:46.485333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be75e7-657c-47f4-9ab6-765eef349fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b76fd929ef4db587b2e4d6a32708d0, user_id=cce3066485e44c84a0ee52d58c18595c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:46.490401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be75e7-657c-47f4-9ab6-765eef349fb2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:46.524464 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3362/6743] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:46 2022] PUT /identity/v3/projects/93b76fd929ef4db587b2e4d6a32708d0/users/cce3066485e44c84a0ee52d58c18595c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:46.539293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10515bd-ebab-442b-9c2e-4e68172752d1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:46.540494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10515bd-ebab-442b-9c2e-4e68172752d1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:46.542758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10515bd-ebab-442b-9c2e-4e68172752d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:46.653623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b10515bd-ebab-442b-9c2e-4e68172752d1 None None] MFA Rules not processed for user `cce3066485e44c84a0ee52d58c18595c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:46.654871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b10515bd-ebab-442b-9c2e-4e68172752d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:46.797411 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3382/6744] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:46.803759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be75e7-657c-47f4-9ab6-765eef349fb2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:46.921259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:46.969395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be75e7-657c-47f4-9ab6-765eef349fb2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:46.980881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483903f-bfeb-4c03-9f5c-cecf3c9f8f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:46.981705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483903f-bfeb-4c03-9f5c-cecf3c9f8f96 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:46.983694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483903f-bfeb-4c03-9f5c-cecf3c9f8f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:46.984211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483903f-bfeb-4c03-9f5c-cecf3c9f8f96 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:46.987102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483903f-bfeb-4c03-9f5c-cecf3c9f8f96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:46.993117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483903f-bfeb-4c03-9f5c-cecf3c9f8f96 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:47.091161 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3363/6745] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:46 2022] POST /identity/v3/projects => generated 383 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:47.094927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10515bd-ebab-442b-9c2e-4e68172752d1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:47.117872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10515bd-ebab-442b-9c2e-4e68172752d1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:47.259472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb5bb5-4c26-4f85-8ec4-a82470603cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:47.260907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb5bb5-4c26-4f85-8ec4-a82470603cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:47.261463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb5bb5-4c26-4f85-8ec4-a82470603cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:47.262680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb5bb5-4c26-4f85-8ec4-a82470603cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:47.263564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb5bb5-4c26-4f85-8ec4-a82470603cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:47.266150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb5bb5-4c26-4f85-8ec4-a82470603cb2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:47.321906 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3383/6746] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:47 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:47.325232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483903f-bfeb-4c03-9f5c-cecf3c9f8f96 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:47.398957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483903f-bfeb-4c03-9f5c-cecf3c9f8f96 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:47.434614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf2cab-09f8-4f68-ba87-b24c937117f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:47.452516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf2cab-09f8-4f68-ba87-b24c937117f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:47.453170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf2cab-09f8-4f68-ba87-b24c937117f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:47.453928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf2cab-09f8-4f68-ba87-b24c937117f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:47.463120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf2cab-09f8-4f68-ba87-b24c937117f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:47.476513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf2cab-09f8-4f68-ba87-b24c937117f8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:47.519078 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3364/6747] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:47.523310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb5bb5-4c26-4f85-8ec4-a82470603cb2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:47.544723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb5bb5-4c26-4f85-8ec4-a82470603cb2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:47.555415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3a6e3-1a02-4228-b3ef-8894eb3282ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:47.556047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3a6e3-1a02-4228-b3ef-8894eb3282ad admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:47.556766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3a6e3-1a02-4228-b3ef-8894eb3282ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:47.557075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3a6e3-1a02-4228-b3ef-8894eb3282ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:47.558179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3a6e3-1a02-4228-b3ef-8894eb3282ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:47.561198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3a6e3-1a02-4228-b3ef-8894eb3282ad admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:47.582189 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3384/6748] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:47.586818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf2cab-09f8-4f68-ba87-b24c937117f8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:47.601385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf2cab-09f8-4f68-ba87-b24c937117f8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:47.607893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca8e5f-363a-4426-8c7e-ec0fe0d1b4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:47.608746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca8e5f-363a-4426-8c7e-ec0fe0d1b4c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:47.609050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca8e5f-363a-4426-8c7e-ec0fe0d1b4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:47.609341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca8e5f-363a-4426-8c7e-ec0fe0d1b4c0 admin admin] PATH_INFO: `/v3/projects/793984b4844f49168887326721aaccb8/users/2cadd2ab41ed4485bd5d6cd3a364600c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:47.679463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca8e5f-363a-4426-8c7e-ec0fe0d1b4c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=793984b4844f49168887326721aaccb8, user_id=2cadd2ab41ed4485bd5d6cd3a364600c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:47.686089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca8e5f-363a-4426-8c7e-ec0fe0d1b4c0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:47.716105 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3365/6749] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:47 2022] PUT /identity/v3/projects/793984b4844f49168887326721aaccb8/users/2cadd2ab41ed4485bd5d6cd3a364600c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:47.727480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef277f-5e21-468f-a8c1-9d6637e5b2da None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:47.727974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef277f-5e21-468f-a8c1-9d6637e5b2da None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:47.728419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef277f-5e21-468f-a8c1-9d6637e5b2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:47.785210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3cef277f-5e21-468f-a8c1-9d6637e5b2da None None] MFA Rules not processed for user `2cadd2ab41ed4485bd5d6cd3a364600c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:47.787442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3cef277f-5e21-468f-a8c1-9d6637e5b2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:47.873675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:47.986898 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3385/6750] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:47.992028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca8e5f-363a-4426-8c7e-ec0fe0d1b4c0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:48.163588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca8e5f-363a-4426-8c7e-ec0fe0d1b4c0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:48.170431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddd549-24b7-42ba-869f-065697dbf0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:48.171044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddd549-24b7-42ba-869f-065697dbf0aa admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:48.171314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddd549-24b7-42ba-869f-065697dbf0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:48.171577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddd549-24b7-42ba-869f-065697dbf0aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:48.172203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddd549-24b7-42ba-869f-065697dbf0aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:48.174205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddd549-24b7-42ba-869f-065697dbf0aa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:48.264203 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3366/6751] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:47 2022] POST /identity/v3/projects => generated 383 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:48.265853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef277f-5e21-468f-a8c1-9d6637e5b2da None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:48.287216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef277f-5e21-468f-a8c1-9d6637e5b2da None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:48.446127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d9eca-0f56-4b9c-8023-a7ac803322e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:48.448752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d9eca-0f56-4b9c-8023-a7ac803322e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:48.453978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d9eca-0f56-4b9c-8023-a7ac803322e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:48.454417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d9eca-0f56-4b9c-8023-a7ac803322e5 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:48.455246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d9eca-0f56-4b9c-8023-a7ac803322e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:48.459999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d9eca-0f56-4b9c-8023-a7ac803322e5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:48.514691 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3386/6752] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:48 2022] POST /identity/v3/users => generated 298 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:48.518648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddd549-24b7-42ba-869f-065697dbf0aa admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:48.550746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddd549-24b7-42ba-869f-065697dbf0aa admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:48.569122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19032f0-438a-46a7-8e7f-b0047423c0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:48.569977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19032f0-438a-46a7-8e7f-b0047423c0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:48.570496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19032f0-438a-46a7-8e7f-b0047423c0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:48.575290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19032f0-438a-46a7-8e7f-b0047423c0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:48.576327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19032f0-438a-46a7-8e7f-b0047423c0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:48.578235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19032f0-438a-46a7-8e7f-b0047423c0a3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:48.611785 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3367/6753] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:48.615386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d9eca-0f56-4b9c-8023-a7ac803322e5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:48.633602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d9eca-0f56-4b9c-8023-a7ac803322e5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:48.646098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0509d1e-4020-4047-b977-4b7a33b0c4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:48.646788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0509d1e-4020-4047-b977-4b7a33b0c4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:48.647065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0509d1e-4020-4047-b977-4b7a33b0c4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:48.647336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0509d1e-4020-4047-b977-4b7a33b0c4c4 admin admin] PATH_INFO: `/v3/projects/77e80243ffa747509eab4eb858cc58d8/users/f671c753262640a8be0457ac169d9cfa/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:48.689994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0509d1e-4020-4047-b977-4b7a33b0c4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77e80243ffa747509eab4eb858cc58d8, user_id=f671c753262640a8be0457ac169d9cfa, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:48.694824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0509d1e-4020-4047-b977-4b7a33b0c4c4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:48.730886 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3387/6754] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:48 2022] PUT /identity/v3/projects/77e80243ffa747509eab4eb858cc58d8/users/f671c753262640a8be0457ac169d9cfa/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:48.731959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19032f0-438a-46a7-8e7f-b0047423c0a3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:48.752905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19032f0-438a-46a7-8e7f-b0047423c0a3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:48.963685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b53f85-f6ff-4b23-8b9a-bfce776c27c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:48.964409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53f85-f6ff-4b23-8b9a-bfce776c27c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:48.964762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53f85-f6ff-4b23-8b9a-bfce776c27c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:48.965142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53f85-f6ff-4b23-8b9a-bfce776c27c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:48.965872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b53f85-f6ff-4b23-8b9a-bfce776c27c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:48.967599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b53f85-f6ff-4b23-8b9a-bfce776c27c9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:48.981871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:49.003933 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3368/6755] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:49.006223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0509d1e-4020-4047-b977-4b7a33b0c4c4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:49.025962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0509d1e-4020-4047-b977-4b7a33b0c4c4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:49.039740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bedfff-a653-4fa0-80d9-229c9ebf87c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:49.041791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bedfff-a653-4fa0-80d9-229c9ebf87c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:49.044282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bedfff-a653-4fa0-80d9-229c9ebf87c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:49.045424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bedfff-a653-4fa0-80d9-229c9ebf87c0 admin admin] PATH_INFO: `/v3/projects/77e80243ffa747509eab4eb858cc58d8/users/f671c753262640a8be0457ac169d9cfa/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:49.050828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bedfff-a653-4fa0-80d9-229c9ebf87c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77e80243ffa747509eab4eb858cc58d8, user_id=f671c753262640a8be0457ac169d9cfa, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:49.062791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bedfff-a653-4fa0-80d9-229c9ebf87c0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:49.090749 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3388/6756] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:49 2022] PUT /identity/v3/projects/77e80243ffa747509eab4eb858cc58d8/users/f671c753262640a8be0457ac169d9cfa/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:49.103787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4f056-34b0-420c-8071-84c49fbd85f1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:49.104067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4f056-34b0-420c-8071-84c49fbd85f1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:49.104393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4f056-34b0-420c-8071-84c49fbd85f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:49.173870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-aef4f056-34b0-420c-8071-84c49fbd85f1 None None] MFA Rules not processed for user `f671c753262640a8be0457ac169d9cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:49.175141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aef4f056-34b0-420c-8071-84c49fbd85f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:49.418521 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3369/6757] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:49.434801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bedfff-a653-4fa0-80d9-229c9ebf87c0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:49.436957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-44bedfff-a653-4fa0-80d9-229c9ebf87c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:49.701939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bedfff-a653-4fa0-80d9-229c9ebf87c0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:49.722652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384b89f-9586-43f9-af5d-84a84b0dfa09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:49.723293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b89f-9586-43f9-af5d-84a84b0dfa09 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:49.724251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b89f-9586-43f9-af5d-84a84b0dfa09 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:49.725220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b89f-9586-43f9-af5d-84a84b0dfa09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:49.837761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:49.925452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384b89f-9586-43f9-af5d-84a84b0dfa09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:49.928272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384b89f-9586-43f9-af5d-84a84b0dfa09 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:49.974414 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3389/6758] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:51.981961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4f056-34b0-420c-8071-84c49fbd85f1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:51.983541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aef4f056-34b0-420c-8071-84c49fbd85f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:52.005649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4f056-34b0-420c-8071-84c49fbd85f1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:52.018151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecb476-3aee-4848-a9ea-9f35267b8994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:52.019022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecb476-3aee-4848-a9ea-9f35267b8994 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:52.019476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecb476-3aee-4848-a9ea-9f35267b8994 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:52.019935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecb476-3aee-4848-a9ea-9f35267b8994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:52.090421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecb476-3aee-4848-a9ea-9f35267b8994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:52.091895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecb476-3aee-4848-a9ea-9f35267b8994 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:52.112084 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3370/6759] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:53.299257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384b89f-9586-43f9-af5d-84a84b0dfa09 service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:53.315603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384b89f-9586-43f9-af5d-84a84b0dfa09 service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:53.327925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5ec6b-9516-405b-86a7-7f2a165aa143 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:53.328545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5ec6b-9516-405b-86a7-7f2a165aa143 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:53.328818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5ec6b-9516-405b-86a7-7f2a165aa143 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:53.329100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5ec6b-9516-405b-86a7-7f2a165aa143 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:53.353452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5ec6b-9516-405b-86a7-7f2a165aa143 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:53.355334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5ec6b-9516-405b-86a7-7f2a165aa143 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:53.382414 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3390/6760] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:53.518201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecb476-3aee-4848-a9ea-9f35267b8994 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:53.521669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-11ecb476-3aee-4848-a9ea-9f35267b8994 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:53.664405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecb476-3aee-4848-a9ea-9f35267b8994 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:53.683141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:53.686538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:53.686995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:53.687436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] PATH_INFO: `/v3/users/8a59e6c4d365463183cc011731255c08` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:53.692727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a59e6c4d365463183cc011731255c08)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:53.701304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:53.774784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:53.842982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a59e6c4d365463183cc011731255c08'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:53.884355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a59e6c4d365463183cc011731255c08'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:53.899309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a59e6c4d365463183cc011731255c08'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:53.921861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a59e6c4d365463183cc011731255c08'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:53.923697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a59e6c4d365463183cc011731255c08'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:53.935634 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3371/6761] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:53 2022] DELETE /identity/v3/users/8a59e6c4d365463183cc011731255c08 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:53.940602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dc2ec-6cc2-4526-9187-c7fccd513784 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:53.944379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dc2ec-6cc2-4526-9187-c7fccd513784 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:53.944904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dc2ec-6cc2-4526-9187-c7fccd513784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:53.984312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-403dc2ec-6cc2-4526-9187-c7fccd513784 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:53.988242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-403dc2ec-6cc2-4526-9187-c7fccd513784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:54.079469 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3391/6762] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:54.104425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:54.106103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:54.243400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0d522-0d77-4a10-a300-3b9a09fa4bf7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:54.248735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b585f6e-9763-4200-ab8d-f01a4b92d85f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:54.249777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b585f6e-9763-4200-ab8d-f01a4b92d85f service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:54.249915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b585f6e-9763-4200-ab8d-f01a4b92d85f service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:54.250159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b585f6e-9763-4200-ab8d-f01a4b92d85f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:54.266309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b585f6e-9763-4200-ab8d-f01a4b92d85f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:54.269281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b585f6e-9763-4200-ab8d-f01a4b92d85f service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:54.293973 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3372/6763] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:55.213838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dc2ec-6cc2-4526-9187-c7fccd513784 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:55.215340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-403dc2ec-6cc2-4526-9187-c7fccd513784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:55.230621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dc2ec-6cc2-4526-9187-c7fccd513784 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:55.238136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:55.239753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:55.240633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:55.241157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] PATH_INFO: `/v3/projects/fc8a27fa09a04c3d84bca7b917da5358` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:55.242685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc8a27fa09a04c3d84bca7b917da5358)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:55.245590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:55.359315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc8a27fa09a04c3d84bca7b917da5358', 'request_id': 'req-f00a907b-2215-4e1f-a8a0-22397eaf7219'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:55.377427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc8a27fa09a04c3d84bca7b917da5358', 'request_id': 'req-f00a907b-2215-4e1f-a8a0-22397eaf7219'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:55.398091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc8a27fa09a04c3d84bca7b917da5358', 'request_id': 'req-f00a907b-2215-4e1f-a8a0-22397eaf7219'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:55.424152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] The token cache is being invalidate because project fc8a27fa09a04c3d84bca7b917da5358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:55.424396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:55.425419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:55.430611 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3392/6764] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:55 2022] DELETE /identity/v3/projects/fc8a27fa09a04c3d84bca7b917da5358 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:55.431937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b585f6e-9763-4200-ab8d-f01a4b92d85f service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:55.449193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b585f6e-9763-4200-ab8d-f01a4b92d85f service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:55.577324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:55.577993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:55.578300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:55.578559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] PATH_INFO: `/v3/users/1464a912add04ac4bfd70198ca9a6297` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:55.579770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=1464a912add04ac4bfd70198ca9a6297)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:55.581836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:55.702776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1464a912add04ac4bfd70198ca9a6297'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:55.704595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:55.729282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1464a912add04ac4bfd70198ca9a6297'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:55.748524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1464a912add04ac4bfd70198ca9a6297'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:55.760956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1464a912add04ac4bfd70198ca9a6297'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:55.760956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1464a912add04ac4bfd70198ca9a6297'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:55.791071 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3373/6765] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:55 2022] DELETE /identity/v3/users/1464a912add04ac4bfd70198ca9a6297 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:57.929072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:57.931236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:57.975453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a907b-2215-4e1f-a8a0-22397eaf7219 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:58.029705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:58.031328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:58.254373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:58.298080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47936c71-47dc-4200-929f-e5c746df95ca admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:58.303970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:58.304907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:58.305620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:58.305902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] PATH_INFO: `/v3/users/cce3066485e44c84a0ee52d58c18595c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:58.308401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cce3066485e44c84a0ee52d58c18595c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:58.310840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:58.311701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:58.312609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:58.312985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:58.313770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] PATH_INFO: `/v3/projects/6345d4f208a54109a0a12b4ba5609d0e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:58.317711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6345d4f208a54109a0a12b4ba5609d0e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:58.320964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:58.388778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cce3066485e44c84a0ee52d58c18595c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:58.407358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cce3066485e44c84a0ee52d58c18595c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:58.419004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cce3066485e44c84a0ee52d58c18595c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:58.433284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cce3066485e44c84a0ee52d58c18595c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:58.434166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d71fb97b-f635-4487-9201-d110b7c9849e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cce3066485e44c84a0ee52d58c18595c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:58.454730 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3374/6766] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:16:58 2022] DELETE /identity/v3/users/cce3066485e44c84a0ee52d58c18595c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:58.461584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa19765-196b-49a9-b865-90f713a76cbf None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:58.462260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa19765-196b-49a9-b865-90f713a76cbf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:58.462954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa19765-196b-49a9-b865-90f713a76cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:58.468980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6345d4f208a54109a0a12b4ba5609d0e', 'request_id': 'req-f6485b05-f736-493a-bbb6-a53105c45332'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:58.491128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6345d4f208a54109a0a12b4ba5609d0e', 'request_id': 'req-f6485b05-f736-493a-bbb6-a53105c45332'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:58.501279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6345d4f208a54109a0a12b4ba5609d0e', 'request_id': 'req-f6485b05-f736-493a-bbb6-a53105c45332'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:58.531194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8aa19765-196b-49a9-b865-90f713a76cbf None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:58.532441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8aa19765-196b-49a9-b865-90f713a76cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:58.540602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] The token cache is being invalidate because project 6345d4f208a54109a0a12b4ba5609d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:16:58.540907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:58.541963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f6485b05-f736-493a-bbb6-a53105c45332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:58.549679 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3393/6767] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:57 2022] DELETE /identity/v3/projects/6345d4f208a54109a0a12b4ba5609d0e => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:58.552874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bc23c-a28d-4bdd-bef6-2c3ed0f81eec None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:58.553827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bc23c-a28d-4bdd-bef6-2c3ed0f81eec None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:58.554087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bc23c-a28d-4bdd-bef6-2c3ed0f81eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:58.652448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:58.660256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-621bc23c-a28d-4bdd-bef6-2c3ed0f81eec None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:58.667981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-621bc23c-a28d-4bdd-bef6-2c3ed0f81eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:58.721725 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3375/6768] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:58.757011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa19765-196b-49a9-b865-90f713a76cbf None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:58.758691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8aa19765-196b-49a9-b865-90f713a76cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:58.886951 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3394/6769] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:16:58.896067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b1a22-350f-4fee-a86d-c606003c99fe None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:58.896317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b1a22-350f-4fee-a86d-c606003c99fe None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:58.896689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b1a22-350f-4fee-a86d-c606003c99fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:58.948789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa19765-196b-49a9-b865-90f713a76cbf None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:58.961492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-964b1a22-350f-4fee-a86d-c606003c99fe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:58.962868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-964b1a22-350f-4fee-a86d-c606003c99fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:58.965336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b293e4-1fb1-4eee-b5f9-3340aabe1b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:58.965955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b293e4-1fb1-4eee-b5f9-3340aabe1b4e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:58.966222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b293e4-1fb1-4eee-b5f9-3340aabe1b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:58.966485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b293e4-1fb1-4eee-b5f9-3340aabe1b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:58.980737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b293e4-1fb1-4eee-b5f9-3340aabe1b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:58.982260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b293e4-1fb1-4eee-b5f9-3340aabe1b4e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:58.993236 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3395/6770] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:16:58.997776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b1a22-350f-4fee-a86d-c606003c99fe None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:59.012139 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3376/6771] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:16:59.018366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b1a22-350f-4fee-a86d-c606003c99fe None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:59.034478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca527e51-5cfb-4af6-a3ec-ee72f4c67cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:59.035274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca527e51-5cfb-4af6-a3ec-ee72f4c67cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:59.035748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca527e51-5cfb-4af6-a3ec-ee72f4c67cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:59.036108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca527e51-5cfb-4af6-a3ec-ee72f4c67cf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:59.036931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca527e51-5cfb-4af6-a3ec-ee72f4c67cf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:16:59.037344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca527e51-5cfb-4af6-a3ec-ee72f4c67cf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:59.039031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca527e51-5cfb-4af6-a3ec-ee72f4c67cf9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:59.058150 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3396/6772] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:16:59.062573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b293e4-1fb1-4eee-b5f9-3340aabe1b4e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:59.080622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b293e4-1fb1-4eee-b5f9-3340aabe1b4e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:59.090547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ae3d6-a15d-40d6-a154-1065b6875c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:59.092181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ae3d6-a15d-40d6-a154-1065b6875c0d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:59.092562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ae3d6-a15d-40d6-a154-1065b6875c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:59.092945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ae3d6-a15d-40d6-a154-1065b6875c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:59.094268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ae3d6-a15d-40d6-a154-1065b6875c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:59.096347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ae3d6-a15d-40d6-a154-1065b6875c0d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:59.156264 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3377/6773] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:16:59 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:16:59.160851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca527e51-5cfb-4af6-a3ec-ee72f4c67cf9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:59.184513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca527e51-5cfb-4af6-a3ec-ee72f4c67cf9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:59.313507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:59.371430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdaf58-3d3a-4e1d-a710-8c18687ad8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:59.372509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdaf58-3d3a-4e1d-a710-8c18687ad8c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:59.372926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdaf58-3d3a-4e1d-a710-8c18687ad8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:59.373266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdaf58-3d3a-4e1d-a710-8c18687ad8c2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:59.373941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdaf58-3d3a-4e1d-a710-8c18687ad8c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:59.376461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdaf58-3d3a-4e1d-a710-8c18687ad8c2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:59.477800 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3397/6774] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:16:59 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:16:59.479409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ae3d6-a15d-40d6-a154-1065b6875c0d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:59.496411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ae3d6-a15d-40d6-a154-1065b6875c0d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:59.508766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd74de0-009a-415c-aa0c-6bb0a828d253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:59.509428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd74de0-009a-415c-aa0c-6bb0a828d253 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:59.509703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd74de0-009a-415c-aa0c-6bb0a828d253 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:59.509985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd74de0-009a-415c-aa0c-6bb0a828d253 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:59.510624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd74de0-009a-415c-aa0c-6bb0a828d253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:59.512322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd74de0-009a-415c-aa0c-6bb0a828d253 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:59.537803 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3378/6775] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:59.539925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdaf58-3d3a-4e1d-a710-8c18687ad8c2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:59.571591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdaf58-3d3a-4e1d-a710-8c18687ad8c2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:59.586797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97612f2f-6466-4089-8ca0-ff80bf7f6b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:59.587549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97612f2f-6466-4089-8ca0-ff80bf7f6b57 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:59.587871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97612f2f-6466-4089-8ca0-ff80bf7f6b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:59.588203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97612f2f-6466-4089-8ca0-ff80bf7f6b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:59.589012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97612f2f-6466-4089-8ca0-ff80bf7f6b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:59.592330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97612f2f-6466-4089-8ca0-ff80bf7f6b57 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:59.632254 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3398/6776] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:16:59.636183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd74de0-009a-415c-aa0c-6bb0a828d253 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:59.652493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97612f2f-6466-4089-8ca0-ff80bf7f6b57 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:16:59.652917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd74de0-009a-415c-aa0c-6bb0a828d253 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:59.655463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-97612f2f-6466-4089-8ca0-ff80bf7f6b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:59.680416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711bb92d-2607-4830-8d9f-07117ca5b273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:59.681131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bb92d-2607-4830-8d9f-07117ca5b273 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:59.681411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bb92d-2607-4830-8d9f-07117ca5b273 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:59.681680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bb92d-2607-4830-8d9f-07117ca5b273 admin admin] PATH_INFO: `/v3/projects/b6b01365f0734dfb87ed1d9083a11327/users/d81e7df9c9ca4391be8567cfb6b9605c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:59.694186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97612f2f-6466-4089-8ca0-ff80bf7f6b57 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:16:59.712907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:16:59.713770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:59.714165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:59.714522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] PATH_INFO: `/v3/projects/93b76fd929ef4db587b2e4d6a32708d0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:59.716555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93b76fd929ef4db587b2e4d6a32708d0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:59.721520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:59.758871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711bb92d-2607-4830-8d9f-07117ca5b273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b01365f0734dfb87ed1d9083a11327, user_id=d81e7df9c9ca4391be8567cfb6b9605c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:16:59.761706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711bb92d-2607-4830-8d9f-07117ca5b273 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:16:59.803734 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3379/6777] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:16:59 2022] PUT /identity/v3/projects/b6b01365f0734dfb87ed1d9083a11327/users/d81e7df9c9ca4391be8567cfb6b9605c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:16:59.805509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e91e16-811f-457b-a7cc-3183bf845127 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:16:59.807389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e91e16-811f-457b-a7cc-3183bf845127 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:16:59.807680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e91e16-811f-457b-a7cc-3183bf845127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:16:59.885841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b76fd929ef4db587b2e4d6a32708d0', 'request_id': 'req-e28a02de-89b4-49fa-934c-2c651c6f2283'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:59.888085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-41e91e16-811f-457b-a7cc-3183bf845127 None None] MFA Rules not processed for user `d81e7df9c9ca4391be8567cfb6b9605c`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:16:59.890262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-41e91e16-811f-457b-a7cc-3183bf845127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:16:59.913410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b76fd929ef4db587b2e4d6a32708d0', 'request_id': 'req-e28a02de-89b4-49fa-934c-2c651c6f2283'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:16:59.939303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b76fd929ef4db587b2e4d6a32708d0', 'request_id': 'req-e28a02de-89b4-49fa-934c-2c651c6f2283'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:00.006220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] The token cache is being invalidate because project 93b76fd929ef4db587b2e4d6a32708d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:00.006835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:00.011799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:00.017941 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3399/6778] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:16:59 2022] DELETE /identity/v3/projects/93b76fd929ef4db587b2e4d6a32708d0 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:17:00.022766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:00.023987 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3380/6779] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:00.028260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:00.029787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e91e16-811f-457b-a7cc-3183bf845127 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:00.085464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:00.204015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a02de-89b4-49fa-934c-2c651c6f2283 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:00.216877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:00.218772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e91e16-811f-457b-a7cc-3183bf845127 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:00.219992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:00.220286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:00.221019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] PATH_INFO: `/v3/users/2cadd2ab41ed4485bd5d6cd3a364600c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:00.225116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cadd2ab41ed4485bd5d6cd3a364600c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:00.232570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:00.233908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3e24-faa5-4519-b26a-8af95047b6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:00.236049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3e24-faa5-4519-b26a-8af95047b6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:00.236329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3e24-faa5-4519-b26a-8af95047b6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:00.236594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3e24-faa5-4519-b26a-8af95047b6f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:00.238102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b3e24-faa5-4519-b26a-8af95047b6f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:00.245178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b3e24-faa5-4519-b26a-8af95047b6f9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:00.324735 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3381/6780] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:17:00 2022] POST /identity/v3/projects => generated 385 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:00.338222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=83,keystone:INSERT=4,keystone:DELETE=13 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:00.345672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3e24-faa5-4519-b26a-8af95047b6f9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:00.349725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-378b3e24-faa5-4519-b26a-8af95047b6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:00.356050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cadd2ab41ed4485bd5d6cd3a364600c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:00.376856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cadd2ab41ed4485bd5d6cd3a364600c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:00.412793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cadd2ab41ed4485bd5d6cd3a364600c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:00.435567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cadd2ab41ed4485bd5d6cd3a364600c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:00.436333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cadd2ab41ed4485bd5d6cd3a364600c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:00.450800 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3400/6781] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:17:00 2022] DELETE /identity/v3/users/2cadd2ab41ed4485bd5d6cd3a364600c => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:00.545776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3e24-faa5-4519-b26a-8af95047b6f9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:00.678368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54796357-ee6a-4533-a22c-6ac6ccdc95ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:00.679004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54796357-ee6a-4533-a22c-6ac6ccdc95ea admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:00.680782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54796357-ee6a-4533-a22c-6ac6ccdc95ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:00.681081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54796357-ee6a-4533-a22c-6ac6ccdc95ea admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:00.681736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54796357-ee6a-4533-a22c-6ac6ccdc95ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:00.684675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54796357-ee6a-4533-a22c-6ac6ccdc95ea admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:00.737346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:00.738414 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3382/6782] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:17:00 2022] POST /identity/v3/users => generated 299 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:00.739848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:00.780056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70dc36-dce3-4fbb-968c-1b1831aa16eb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:00.792767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178eb63-0912-4903-a878-c0f0ca6b232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:00.793719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178eb63-0912-4903-a878-c0f0ca6b232a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:00.794186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178eb63-0912-4903-a878-c0f0ca6b232a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:00.794652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178eb63-0912-4903-a878-c0f0ca6b232a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:00.795505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178eb63-0912-4903-a878-c0f0ca6b232a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:00.797370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178eb63-0912-4903-a878-c0f0ca6b232a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:00.815447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54796357-ee6a-4533-a22c-6ac6ccdc95ea admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:00.823212 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3401/6783] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:00.829564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54796357-ee6a-4533-a22c-6ac6ccdc95ea admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:00.850455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e465b1-8c5e-454d-b468-00a648c3561f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:00.852498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e465b1-8c5e-454d-b468-00a648c3561f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:00.853087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e465b1-8c5e-454d-b468-00a648c3561f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:00.854072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e465b1-8c5e-454d-b468-00a648c3561f admin admin] PATH_INFO: `/v3/projects/42ad5118e8e14816823196f67f01acb2/users/9c4f1f3f91624ae792098a3bc96dcf0d/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:00.904182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e465b1-8c5e-454d-b468-00a648c3561f admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ad5118e8e14816823196f67f01acb2, user_id=9c4f1f3f91624ae792098a3bc96dcf0d, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:00.911390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e465b1-8c5e-454d-b468-00a648c3561f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:00.942911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178eb63-0912-4903-a878-c0f0ca6b232a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:00.949090 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3383/6784] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:17:00 2022] PUT /identity/v3/projects/42ad5118e8e14816823196f67f01acb2/users/9c4f1f3f91624ae792098a3bc96dcf0d/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:00.957950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178eb63-0912-4903-a878-c0f0ca6b232a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:01.090689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013c669-7248-4719-a412-34558033b375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:01.093639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013c669-7248-4719-a412-34558033b375 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:01.093930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013c669-7248-4719-a412-34558033b375 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:01.094198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013c669-7248-4719-a412-34558033b375 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:01.097480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013c669-7248-4719-a412-34558033b375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:01.101947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013c669-7248-4719-a412-34558033b375 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:01.132676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e465b1-8c5e-454d-b468-00a648c3561f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:01.133930 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3402/6785] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:01.148029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e465b1-8c5e-454d-b468-00a648c3561f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:01.160185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43540b01-278f-4e67-b720-594a38e87cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:01.161327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43540b01-278f-4e67-b720-594a38e87cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:01.161544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43540b01-278f-4e67-b720-594a38e87cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:01.163022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43540b01-278f-4e67-b720-594a38e87cfd admin admin] PATH_INFO: `/v3/projects/42ad5118e8e14816823196f67f01acb2/users/9c4f1f3f91624ae792098a3bc96dcf0d/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:01.168807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43540b01-278f-4e67-b720-594a38e87cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ad5118e8e14816823196f67f01acb2, user_id=9c4f1f3f91624ae792098a3bc96dcf0d, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:01.171599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43540b01-278f-4e67-b720-594a38e87cfd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:01.211203 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3384/6786] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:17:01 2022] PUT /identity/v3/projects/42ad5118e8e14816823196f67f01acb2/users/9c4f1f3f91624ae792098a3bc96dcf0d/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:17:01.212847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9cabf-0334-4a17-b2ee-e97067df92ad None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:01.213072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9cabf-0334-4a17-b2ee-e97067df92ad None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:01.213448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9cabf-0334-4a17-b2ee-e97067df92ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:01.278160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-37e9cabf-0334-4a17-b2ee-e97067df92ad None None] MFA Rules not processed for user `9c4f1f3f91624ae792098a3bc96dcf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:01.279426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-37e9cabf-0334-4a17-b2ee-e97067df92ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:01.372644 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3403/6787] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:17:01.391699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43540b01-278f-4e67-b720-594a38e87cfd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:01.393616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-43540b01-278f-4e67-b720-594a38e87cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:01.403353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:01.495081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43540b01-278f-4e67-b720-594a38e87cfd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:01.500143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a99ba-5733-4a8e-b33c-4f5cc5fe7f0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:01.501260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a99ba-5733-4a8e-b33c-4f5cc5fe7f0d service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:01.501727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a99ba-5733-4a8e-b33c-4f5cc5fe7f0d service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:01.502455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a99ba-5733-4a8e-b33c-4f5cc5fe7f0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:01.515715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a99ba-5733-4a8e-b33c-4f5cc5fe7f0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:01.517772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a99ba-5733-4a8e-b33c-4f5cc5fe7f0d service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:01.545857 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3385/6788] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:17:02.250655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9cabf-0334-4a17-b2ee-e97067df92ad None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:02.252979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-37e9cabf-0334-4a17-b2ee-e97067df92ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:02.314500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:02.371315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9cabf-0334-4a17-b2ee-e97067df92ad None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:02.377645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:02.378524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:02.379293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:02.379293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] PATH_INFO: `/v3/projects/793984b4844f49168887326721aaccb8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:02.381556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=793984b4844f49168887326721aaccb8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:02.383546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:02.473642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '793984b4844f49168887326721aaccb8', 'request_id': 'req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.507155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '793984b4844f49168887326721aaccb8', 'request_id': 'req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.538969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '793984b4844f49168887326721aaccb8', 'request_id': 'req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.567327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] The token cache is being invalidate because project 793984b4844f49168887326721aaccb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:02.567874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.573095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.576360 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3404/6789] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:17:02 2022] DELETE /identity/v3/projects/793984b4844f49168887326721aaccb8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:02.584561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a99ba-5733-4a8e-b33c-4f5cc5fe7f0d service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:02.596569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a99ba-5733-4a8e-b33c-4f5cc5fe7f0d service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:02.644334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:02.645884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:02.711594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:02.712617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:02.713105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:02.713522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] PATH_INFO: `/v3/users/f671c753262640a8be0457ac169d9cfa` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:02.715966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f671c753262640a8be0457ac169d9cfa)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:02.718596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:02.798069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f671c753262640a8be0457ac169d9cfa'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.806351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233359a8-7eeb-4b04-b0e5-b0f3dfc503b6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:02.821563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:02.822537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:02.824550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:02.824913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] PATH_INFO: `/v3/users/d81e7df9c9ca4391be8567cfb6b9605c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:02.827175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d81e7df9c9ca4391be8567cfb6b9605c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:02.830598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:02.833519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f671c753262640a8be0457ac169d9cfa'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.852653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f671c753262640a8be0457ac169d9cfa'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.864626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f671c753262640a8be0457ac169d9cfa'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.865929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-36260ee6-563d-4488-a100-851bee03d9ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f671c753262640a8be0457ac169d9cfa'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.876868 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3386/6790] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:17:02 2022] DELETE /identity/v3/users/f671c753262640a8be0457ac169d9cfa => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:02.943334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd81e7df9c9ca4391be8567cfb6b9605c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.954230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd81e7df9c9ca4391be8567cfb6b9605c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.964161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:02.975588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd81e7df9c9ca4391be8567cfb6b9605c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.989557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd81e7df9c9ca4391be8567cfb6b9605c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.990377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd81e7df9c9ca4391be8567cfb6b9605c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:02.998702 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3405/6791] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:17:02 2022] DELETE /identity/v3/users/d81e7df9c9ca4391be8567cfb6b9605c => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:03.003319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaab7b-70c4-431f-96b9-d4d9e1aaaf75 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:03.003602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaab7b-70c4-431f-96b9-d4d9e1aaaf75 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:03.004998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaab7b-70c4-431f-96b9-d4d9e1aaaf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:03.029279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dbcaab7b-70c4-431f-96b9-d4d9e1aaaf75 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:03.030938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dbcaab7b-70c4-431f-96b9-d4d9e1aaaf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:03.059799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:03.120158 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3387/6792] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:03.221928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:03.223671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:03.299340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020ebf-5afe-4781-b561-0af46ed1d36c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:03.311599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d0a0-5e83-4328-9f44-907038b234c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:03.313090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d0a0-5e83-4328-9f44-907038b234c8 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:03.314802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d0a0-5e83-4328-9f44-907038b234c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:03.315320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d0a0-5e83-4328-9f44-907038b234c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:03.333447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d0a0-5e83-4328-9f44-907038b234c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:03.335397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d0a0-5e83-4328-9f44-907038b234c8 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:03.349944 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3406/6793] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:17:04.344712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d0a0-5e83-4328-9f44-907038b234c8 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:04.345889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaab7b-70c4-431f-96b9-d4d9e1aaaf75 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:04.349096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dbcaab7b-70c4-431f-96b9-d4d9e1aaaf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:04.350129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3ef7d0a0-5e83-4328-9f44-907038b234c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:04.380340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaab7b-70c4-431f-96b9-d4d9e1aaaf75 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:04.390949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d0a0-5e83-4328-9f44-907038b234c8 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:04.408996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:04.410055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:04.411418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:04.411988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:04.413121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] PATH_INFO: `/v3/projects/77e80243ffa747509eab4eb858cc58d8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:04.414791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:04.414791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:04.418549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] PATH_INFO: `/v3/users/0f9276faf05349aeb1469250b2e199d5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:04.420183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=77e80243ffa747509eab4eb858cc58d8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:04.421155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9276faf05349aeb1469250b2e199d5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:04.425297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:04.427513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:04.542955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77e80243ffa747509eab4eb858cc58d8', 'request_id': 'req-35238083-1f3f-4301-9d8d-362435fdec7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:04.561192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77e80243ffa747509eab4eb858cc58d8', 'request_id': 'req-35238083-1f3f-4301-9d8d-362435fdec7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:04.588652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9276faf05349aeb1469250b2e199d5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:04.590030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77e80243ffa747509eab4eb858cc58d8', 'request_id': 'req-35238083-1f3f-4301-9d8d-362435fdec7f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:04.618957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9276faf05349aeb1469250b2e199d5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:04.627693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] The token cache is being invalidate because project 77e80243ffa747509eab4eb858cc58d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:04.628393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:04.630689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-35238083-1f3f-4301-9d8d-362435fdec7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:04.633125 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3388/6794] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:17:04 2022] DELETE /identity/v3/projects/77e80243ffa747509eab4eb858cc58d8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:04.639005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a74c2-3499-4b73-878b-1513dc5f1b09 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:04.639701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a74c2-3499-4b73-878b-1513dc5f1b09 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:04.640214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a74c2-3499-4b73-878b-1513dc5f1b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:04.653779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9276faf05349aeb1469250b2e199d5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:04.671262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9276faf05349aeb1469250b2e199d5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:04.672235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9276faf05349aeb1469250b2e199d5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:04.682423 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3407/6795] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:17:04 2022] DELETE /identity/v3/users/0f9276faf05349aeb1469250b2e199d5 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:04.683925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1b2a74c2-3499-4b73-878b-1513dc5f1b09 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:04.685411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1b2a74c2-3499-4b73-878b-1513dc5f1b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:04.750606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:04.752054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:04.825802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:04.846279 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3389/6796] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:04.855601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798f93b-92b7-4f97-a034-570b49416dd3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:04.855982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798f93b-92b7-4f97-a034-570b49416dd3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:04.856353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798f93b-92b7-4f97-a034-570b49416dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:04.860566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9637c-fe59-4827-ab8b-e706f60ae07e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:04.868194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:04.869523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:04.870134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:04.870542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] PATH_INFO: `/v3/projects/b6b01365f0734dfb87ed1d9083a11327` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:04.877564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b01365f0734dfb87ed1d9083a11327)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:04.880959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:04.888778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f798f93b-92b7-4f97-a034-570b49416dd3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:04.890761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f798f93b-92b7-4f97-a034-570b49416dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:04.923856 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3390/6797] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:04.929909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798f93b-92b7-4f97-a034-570b49416dd3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:04.961503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798f93b-92b7-4f97-a034-570b49416dd3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:04.993995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b01365f0734dfb87ed1d9083a11327', 'request_id': 'req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:05.006741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b01365f0734dfb87ed1d9083a11327', 'request_id': 'req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:05.017713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b01365f0734dfb87ed1d9083a11327', 'request_id': 'req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:05.047112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] The token cache is being invalidate because project b6b01365f0734dfb87ed1d9083a11327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:05.047450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:05.049472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:05.058080 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3408/6798] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:17:04 2022] DELETE /identity/v3/projects/b6b01365f0734dfb87ed1d9083a11327 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:17:05.060341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:05.063687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:05.104503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:05.175225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6db45-776a-4f53-b00b-93d9d7c9dd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:05.177339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6db45-776a-4f53-b00b-93d9d7c9dd74 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:05.177627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6db45-776a-4f53-b00b-93d9d7c9dd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:05.181551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6db45-776a-4f53-b00b-93d9d7c9dd74 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:05.182314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6db45-776a-4f53-b00b-93d9d7c9dd74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:17:05.182633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6db45-776a-4f53-b00b-93d9d7c9dd74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:05.184317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6db45-776a-4f53-b00b-93d9d7c9dd74 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:05.242999 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3391/6799] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:17:05.248221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe9c2c-6b98-4f9f-8ce7-40e983f0c35b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:05.248979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6db45-776a-4f53-b00b-93d9d7c9dd74 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:05.256586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f4f6db45-776a-4f53-b00b-93d9d7c9dd74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:05.269346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:05.291269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:05.291634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:05.291945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] PATH_INFO: `/v3/users/9c4f1f3f91624ae792098a3bc96dcf0d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:05.300870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6db45-776a-4f53-b00b-93d9d7c9dd74 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:05.301525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c4f1f3f91624ae792098a3bc96dcf0d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:05.314798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a2213-8125-49e9-9567-ebbb1dcdeed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:05.315638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a2213-8125-49e9-9567-ebbb1dcdeed8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:05.315934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a2213-8125-49e9-9567-ebbb1dcdeed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:05.316059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:05.316185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a2213-8125-49e9-9567-ebbb1dcdeed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:05.317540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a2213-8125-49e9-9567-ebbb1dcdeed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:05.340258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a2213-8125-49e9-9567-ebbb1dcdeed8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:05.433872 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3392/6800] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:17:05 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:05.438216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a2213-8125-49e9-9567-ebbb1dcdeed8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:05.497457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c4f1f3f91624ae792098a3bc96dcf0d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:05.528421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c4f1f3f91624ae792098a3bc96dcf0d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:05.590099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c4f1f3f91624ae792098a3bc96dcf0d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:05.604054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a2213-8125-49e9-9567-ebbb1dcdeed8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:05.613114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c4f1f3f91624ae792098a3bc96dcf0d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:05.616768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c4f1f3f91624ae792098a3bc96dcf0d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:05.634294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cb2ba-a1c2-446e-a4e9-642dbf30f129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:05.634892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3409/6801] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:17:05 2022] DELETE /identity/v3/users/9c4f1f3f91624ae792098a3bc96dcf0d => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:17:05.635010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cb2ba-a1c2-446e-a4e9-642dbf30f129 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:05.635207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cb2ba-a1c2-446e-a4e9-642dbf30f129 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:05.635585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cb2ba-a1c2-446e-a4e9-642dbf30f129 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:05.637030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cb2ba-a1c2-446e-a4e9-642dbf30f129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:05.638407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cb2ba-a1c2-446e-a4e9-642dbf30f129 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:05.656638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:05.671240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:05.690331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:05.691304 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3393/6802] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:17:05 2022] POST /identity/v3/users => generated 287 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:05.697377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cb2ba-a1c2-446e-a4e9-642dbf30f129 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:05.702820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dd5cb2ba-a1c2-446e-a4e9-642dbf30f129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:05.747354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cb2ba-a1c2-446e-a4e9-642dbf30f129 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:05.912024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aae83e-e699-46fc-8a36-415b4f2892c3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:05.933431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:05.934383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:05.934777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:05.937270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] PATH_INFO: `/v3/projects/5cfaff69dc084157a616e7d16c0e8569` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:05.939866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cfaff69dc084157a616e7d16c0e8569)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:05.942582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:05.967518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1aa4d2-e18b-499d-b7a7-fc2d95cb2d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:05.968294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aa4d2-e18b-499d-b7a7-fc2d95cb2d43 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:05.968705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aa4d2-e18b-499d-b7a7-fc2d95cb2d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:05.969110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aa4d2-e18b-499d-b7a7-fc2d95cb2d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:05.969883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1aa4d2-e18b-499d-b7a7-fc2d95cb2d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:05.971690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1aa4d2-e18b-499d-b7a7-fc2d95cb2d43 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:06.007608 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3394/6803] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:06.020160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1aa4d2-e18b-499d-b7a7-fc2d95cb2d43 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:06.064788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1aa4d2-e18b-499d-b7a7-fc2d95cb2d43 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:06.093921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dab6e-97d0-4f61-9bd8-f56429cf010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:06.097290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dab6e-97d0-4f61-9bd8-f56429cf010c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:06.097852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dab6e-97d0-4f61-9bd8-f56429cf010c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:06.098314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dab6e-97d0-4f61-9bd8-f56429cf010c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:06.100189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dab6e-97d0-4f61-9bd8-f56429cf010c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:06.103527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dab6e-97d0-4f61-9bd8-f56429cf010c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:06.130085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3395/6804] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:06.132196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dab6e-97d0-4f61-9bd8-f56429cf010c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:06.134151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cfaff69dc084157a616e7d16c0e8569', 'request_id': 'req-3530cf4e-9254-45a5-8442-9bd4831e3ef1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:06.146715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dab6e-97d0-4f61-9bd8-f56429cf010c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:06.183203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cfaff69dc084157a616e7d16c0e8569', 'request_id': 'req-3530cf4e-9254-45a5-8442-9bd4831e3ef1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:06.205075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cfaff69dc084157a616e7d16c0e8569', 'request_id': 'req-3530cf4e-9254-45a5-8442-9bd4831e3ef1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:06.244275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] The token cache is being invalidate because project 5cfaff69dc084157a616e7d16c0e8569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:06.244756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:06.245500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:06.246110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:06.253525 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3410/6805] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:17:05 2022] DELETE /identity/v3/projects/5cfaff69dc084157a616e7d16c0e8569 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:06.261029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:06.266129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:06.342990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b006a-70d7-4331-8610-8310440e3c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:06.343782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b006a-70d7-4331-8610-8310440e3c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:06.344012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b006a-70d7-4331-8610-8310440e3c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:06.344222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b006a-70d7-4331-8610-8310440e3c91 admin admin] PATH_INFO: `/v3/projects/6b0324b37fb14723b8c4071057f61fb9/users/a41804510e0746c781fa0f73bce0ecf2/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:06.401952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b006a-70d7-4331-8610-8310440e3c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0324b37fb14723b8c4071057f61fb9, user_id=a41804510e0746c781fa0f73bce0ecf2, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:06.408645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b006a-70d7-4331-8610-8310440e3c91 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:06.451618 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3396/6806] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:17:06 2022] PUT /identity/v3/projects/6b0324b37fb14723b8c4071057f61fb9/users/a41804510e0746c781fa0f73bce0ecf2/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:06.453298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1d039-2271-489f-b422-6379fc7ebb43 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:06.453757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1d039-2271-489f-b422-6379fc7ebb43 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:06.454206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1d039-2271-489f-b422-6379fc7ebb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:06.523581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530cf4e-9254-45a5-8442-9bd4831e3ef1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:06.528436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-02e1d039-2271-489f-b422-6379fc7ebb43 None None] MFA Rules not processed for user `a41804510e0746c781fa0f73bce0ecf2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:06.530438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-02e1d039-2271-489f-b422-6379fc7ebb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:06.614628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:06.676437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:06.678127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:06.678620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:06.679105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] PATH_INFO: `/v3/users/0d15db49ef774494baa77a9ec31b22c2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:06.681126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d15db49ef774494baa77a9ec31b22c2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:06.684875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:06.727674 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3397/6807] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:17:06.744612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1d039-2271-489f-b422-6379fc7ebb43 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:06.746371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-02e1d039-2271-489f-b422-6379fc7ebb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:06.799758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d15db49ef774494baa77a9ec31b22c2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:06.823915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d15db49ef774494baa77a9ec31b22c2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:06.870120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d15db49ef774494baa77a9ec31b22c2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:06.899479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1d039-2271-489f-b422-6379fc7ebb43 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:06.905850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d15db49ef774494baa77a9ec31b22c2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:06.906759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74577b54-bedc-4e5a-9072-61e63ba08c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:06.907258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d15db49ef774494baa77a9ec31b22c2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:06.907499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74577b54-bedc-4e5a-9072-61e63ba08c90 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:06.908410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74577b54-bedc-4e5a-9072-61e63ba08c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:06.908700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74577b54-bedc-4e5a-9072-61e63ba08c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:06.911891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-74577b54-bedc-4e5a-9072-61e63ba08c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:06.925444 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3411/6808] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:17:06 2022] DELETE /identity/v3/users/0d15db49ef774494baa77a9ec31b22c2 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:06.933810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74577b54-bedc-4e5a-9072-61e63ba08c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:06.939119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74577b54-bedc-4e5a-9072-61e63ba08c90 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:06.959815 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3398/6809] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:17:07.571767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:07.573760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:07.597547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74577b54-bedc-4e5a-9072-61e63ba08c90 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:07.599622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-74577b54-bedc-4e5a-9072-61e63ba08c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:07.642371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74577b54-bedc-4e5a-9072-61e63ba08c90 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:07.789807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cd14e-e409-4ef9-b28a-cff197d706e7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:07.814753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:07.817010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:07.819559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:07.832506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] PATH_INFO: `/v3/projects/08c053b93b2b4576a373343c328a1353` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:07.834669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:07.840439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c053b93b2b4576a373343c328a1353)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:07.854971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:07.884269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:07.885119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:07.885796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:07.888294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] PATH_INFO: `/v3/projects/42ad5118e8e14816823196f67f01acb2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:07.890655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=42ad5118e8e14816823196f67f01acb2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:07.893631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:08.045696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c053b93b2b4576a373343c328a1353', 'request_id': 'req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:08.065853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c053b93b2b4576a373343c328a1353', 'request_id': 'req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:08.079331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c053b93b2b4576a373343c328a1353', 'request_id': 'req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:08.082916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42ad5118e8e14816823196f67f01acb2', 'request_id': 'req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:08.084243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:08.103330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42ad5118e8e14816823196f67f01acb2', 'request_id': 'req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:08.115147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] The token cache is being invalidate because project 08c053b93b2b4576a373343c328a1353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:08.115598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:08.117017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42ad5118e8e14816823196f67f01acb2', 'request_id': 'req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:08.117581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9f6d9a5a-84f0-476d-a547-291a8b42a2ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:08.121088 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3412/6810] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:17:07 2022] DELETE /identity/v3/projects/08c053b93b2b4576a373343c328a1353 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:08.179935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] The token cache is being invalidate because project 42ad5118e8e14816823196f67f01acb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:08.180684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:08.182426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3a7fa3be-be63-431e-b1cf-461b0ce0f10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:08.188286 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3399/6811] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:17:07 2022] DELETE /identity/v3/projects/42ad5118e8e14816823196f67f01acb2 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:17:08.191845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953d01-8140-478d-b472-01f73638df14 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:08.192109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953d01-8140-478d-b472-01f73638df14 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:08.192377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953d01-8140-478d-b472-01f73638df14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:08.308995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-82953d01-8140-478d-b472-01f73638df14 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:08.312104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-82953d01-8140-478d-b472-01f73638df14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:08.529791 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3413/6812] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:08.543283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be270f2-5ebe-4af0-9872-3bc27610527f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:08.544760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be270f2-5ebe-4af0-9872-3bc27610527f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:08.545091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be270f2-5ebe-4af0-9872-3bc27610527f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:08.615078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0be270f2-5ebe-4af0-9872-3bc27610527f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:08.616350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0be270f2-5ebe-4af0-9872-3bc27610527f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:08.639493 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3400/6813] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:08.643418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82953d01-8140-478d-b472-01f73638df14 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:08.657122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82953d01-8140-478d-b472-01f73638df14 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:08.670632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a42e1-61dd-4425-9723-2b9a8a464b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:08.671836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a42e1-61dd-4425-9723-2b9a8a464b6c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:08.672337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a42e1-61dd-4425-9723-2b9a8a464b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:08.672822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a42e1-61dd-4425-9723-2b9a8a464b6c admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:08.673758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a42e1-61dd-4425-9723-2b9a8a464b6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:17:08.674283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a42e1-61dd-4425-9723-2b9a8a464b6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:08.676717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a42e1-61dd-4425-9723-2b9a8a464b6c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:08.704992 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3414/6814] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:17:08.710616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be270f2-5ebe-4af0-9872-3bc27610527f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:08.724480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be270f2-5ebe-4af0-9872-3bc27610527f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:08.730280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda0e3b-ba3c-479f-a5cb-49b27573c25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:08.731295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda0e3b-ba3c-479f-a5cb-49b27573c25a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:08.731955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda0e3b-ba3c-479f-a5cb-49b27573c25a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:08.732773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda0e3b-ba3c-479f-a5cb-49b27573c25a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:08.733775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda0e3b-ba3c-479f-a5cb-49b27573c25a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:08.736291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda0e3b-ba3c-479f-a5cb-49b27573c25a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:08.790836 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3401/6815] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:17:08 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:08.795689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a42e1-61dd-4425-9723-2b9a8a464b6c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:08.816861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a42e1-61dd-4425-9723-2b9a8a464b6c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:08.846644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fc72b-607a-46ed-bdc4-359738160eed None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:08.847046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fc72b-607a-46ed-bdc4-359738160eed None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:08.847450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fc72b-607a-46ed-bdc4-359738160eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:08.882727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-960fc72b-607a-46ed-bdc4-359738160eed None None] MFA Rules not processed for user `a41804510e0746c781fa0f73bce0ecf2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:08.884409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-960fc72b-607a-46ed-bdc4-359738160eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:08.946816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000f8b1-f837-42f7-987e-a70a0ea7d006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:08.947593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000f8b1-f837-42f7-987e-a70a0ea7d006 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:08.948301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000f8b1-f837-42f7-987e-a70a0ea7d006 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:08.948301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000f8b1-f837-42f7-987e-a70a0ea7d006 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:08.948832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000f8b1-f837-42f7-987e-a70a0ea7d006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:08.953819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000f8b1-f837-42f7-987e-a70a0ea7d006 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:08.999908 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3415/6816] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:17:08 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:09.005726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000f8b1-f837-42f7-987e-a70a0ea7d006 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:09.006684 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3402/6817] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:17:09.020991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000f8b1-f837-42f7-987e-a70a0ea7d006 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:09.027100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db490822-f869-40a7-a571-773a5b331b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:09.027884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db490822-f869-40a7-a571-773a5b331b2f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:09.028504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db490822-f869-40a7-a571-773a5b331b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:09.029062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db490822-f869-40a7-a571-773a5b331b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:09.030330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db490822-f869-40a7-a571-773a5b331b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:09.032723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db490822-f869-40a7-a571-773a5b331b2f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:09.076893 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3416/6818] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:09.078264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fc72b-607a-46ed-bdc4-359738160eed None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:09.113860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fc72b-607a-46ed-bdc4-359738160eed None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:09.120470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b073510-caff-46c9-b791-0985471b3cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:09.121644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b073510-caff-46c9-b791-0985471b3cfb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:09.122083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b073510-caff-46c9-b791-0985471b3cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:09.122499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b073510-caff-46c9-b791-0985471b3cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:09.123308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b073510-caff-46c9-b791-0985471b3cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:09.125167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b073510-caff-46c9-b791-0985471b3cfb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:09.143414 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3403/6819] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:09.145575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db490822-f869-40a7-a571-773a5b331b2f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:09.157818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db490822-f869-40a7-a571-773a5b331b2f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:09.174248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb8ff9-15ae-43e4-b0c5-180cc88f2956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:09.175232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb8ff9-15ae-43e4-b0c5-180cc88f2956 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:09.176158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb8ff9-15ae-43e4-b0c5-180cc88f2956 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:09.176502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb8ff9-15ae-43e4-b0c5-180cc88f2956 admin admin] PATH_INFO: `/v3/projects/bf9bc22750a54b9888679eacb2fa7810/users/7031144183f3482d8c8e9a6dc1b1f0c5/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:09.208480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb8ff9-15ae-43e4-b0c5-180cc88f2956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9bc22750a54b9888679eacb2fa7810, user_id=7031144183f3482d8c8e9a6dc1b1f0c5, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:09.211818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb8ff9-15ae-43e4-b0c5-180cc88f2956 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:09.253332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2210-137d-4e40-8fd6-68a9122123b4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:09.253841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2210-137d-4e40-8fd6-68a9122123b4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:09.254223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2210-137d-4e40-8fd6-68a9122123b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:09.260839 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3417/6820] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:17:09 2022] PUT /identity/v3/projects/bf9bc22750a54b9888679eacb2fa7810/users/7031144183f3482d8c8e9a6dc1b1f0c5/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:09.302734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-045a2210-137d-4e40-8fd6-68a9122123b4 None None] MFA Rules not processed for user `7031144183f3482d8c8e9a6dc1b1f0c5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:09.305758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-045a2210-137d-4e40-8fd6-68a9122123b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:09.351732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:09.398459 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3404/6821] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:09.402792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb8ff9-15ae-43e4-b0c5-180cc88f2956 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:09.441519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:09.478792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb8ff9-15ae-43e4-b0c5-180cc88f2956 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:09.484076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd00541-dbd0-422a-b785-efeb0341ebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:09.485155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd00541-dbd0-422a-b785-efeb0341ebd2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:09.485355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd00541-dbd0-422a-b785-efeb0341ebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:09.485741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd00541-dbd0-422a-b785-efeb0341ebd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:09.486451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd00541-dbd0-422a-b785-efeb0341ebd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:09.488985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd00541-dbd0-422a-b785-efeb0341ebd2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:09.532341 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3418/6822] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:17:09 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:09.533931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a2210-137d-4e40-8fd6-68a9122123b4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:09.552677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a2210-137d-4e40-8fd6-68a9122123b4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:09.648551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4f391-1e7f-4a62-9f31-8af3e7c85349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:09.649251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4f391-1e7f-4a62-9f31-8af3e7c85349 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:09.649487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4f391-1e7f-4a62-9f31-8af3e7c85349 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:09.649922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4f391-1e7f-4a62-9f31-8af3e7c85349 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:09.650709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4f391-1e7f-4a62-9f31-8af3e7c85349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:09.652819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4f391-1e7f-4a62-9f31-8af3e7c85349 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:09.702719 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3405/6823] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:17:09 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:09.703046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd00541-dbd0-422a-b785-efeb0341ebd2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:09.719163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd00541-dbd0-422a-b785-efeb0341ebd2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:09.733631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485f980-1271-4851-a262-f45e27609886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:09.734880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485f980-1271-4851-a262-f45e27609886 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:09.735311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485f980-1271-4851-a262-f45e27609886 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:09.735759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485f980-1271-4851-a262-f45e27609886 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:09.736643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485f980-1271-4851-a262-f45e27609886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:09.738846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485f980-1271-4851-a262-f45e27609886 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:09.755759 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3419/6824] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:17:09.760293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4f391-1e7f-4a62-9f31-8af3e7c85349 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:09.772822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4f391-1e7f-4a62-9f31-8af3e7c85349 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:09.778773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228c3f4-2293-45bc-87b2-554a1ece05c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:09.779853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228c3f4-2293-45bc-87b2-554a1ece05c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:09.780415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228c3f4-2293-45bc-87b2-554a1ece05c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:09.780694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228c3f4-2293-45bc-87b2-554a1ece05c5 admin admin] PATH_INFO: `/v3/projects/52448c57158b41b69182626c67a6ddde/users/30db5e37bd2140ed9c664b5a845e75b1/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:09.812188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228c3f4-2293-45bc-87b2-554a1ece05c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52448c57158b41b69182626c67a6ddde, user_id=30db5e37bd2140ed9c664b5a845e75b1, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:09.815006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228c3f4-2293-45bc-87b2-554a1ece05c5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:09.835295 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3406/6825] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:17:09 2022] PUT /identity/v3/projects/52448c57158b41b69182626c67a6ddde/users/30db5e37bd2140ed9c664b5a845e75b1/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:09.839170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485f980-1271-4851-a262-f45e27609886 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:09.852240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485f980-1271-4851-a262-f45e27609886 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:09.937224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114cee5-fc2e-4f31-9014-93743f4836e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:09.938119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114cee5-fc2e-4f31-9014-93743f4836e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:09.938555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114cee5-fc2e-4f31-9014-93743f4836e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:09.938980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114cee5-fc2e-4f31-9014-93743f4836e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:09.939819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114cee5-fc2e-4f31-9014-93743f4836e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:09.942494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114cee5-fc2e-4f31-9014-93743f4836e3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:09.961100 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3420/6826] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:17:09.965170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228c3f4-2293-45bc-87b2-554a1ece05c5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:09.992614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228c3f4-2293-45bc-87b2-554a1ece05c5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:10.014306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22251d9e-a51c-41ba-8f31-2ee702f99b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:10.016792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22251d9e-a51c-41ba-8f31-2ee702f99b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:10.017127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22251d9e-a51c-41ba-8f31-2ee702f99b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:10.017399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22251d9e-a51c-41ba-8f31-2ee702f99b8e admin admin] PATH_INFO: `/v3/projects/52448c57158b41b69182626c67a6ddde/users/30db5e37bd2140ed9c664b5a845e75b1/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:10.020617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22251d9e-a51c-41ba-8f31-2ee702f99b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=52448c57158b41b69182626c67a6ddde, user_id=30db5e37bd2140ed9c664b5a845e75b1, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:10.023460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22251d9e-a51c-41ba-8f31-2ee702f99b8e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:10.037703 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3407/6827] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:17:09 2022] PUT /identity/v3/projects/52448c57158b41b69182626c67a6ddde/users/30db5e37bd2140ed9c664b5a845e75b1/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:10.048415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50e95d-f062-4106-8a7d-f464c2e82dfd None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:10.048714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50e95d-f062-4106-8a7d-f464c2e82dfd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:10.048994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50e95d-f062-4106-8a7d-f464c2e82dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:10.121355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4c50e95d-f062-4106-8a7d-f464c2e82dfd None None] MFA Rules not processed for user `30db5e37bd2140ed9c664b5a845e75b1`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:10.122776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4c50e95d-f062-4106-8a7d-f464c2e82dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:10.257856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22251d9e-a51c-41ba-8f31-2ee702f99b8e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:10.260375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-22251d9e-a51c-41ba-8f31-2ee702f99b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:10.264384 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3421/6828] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:10.350770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22251d9e-a51c-41ba-8f31-2ee702f99b8e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:10.355361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41985f95-6812-4358-8218-884cd6bd0cd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:10.356092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41985f95-6812-4358-8218-884cd6bd0cd3 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:10.356478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41985f95-6812-4358-8218-884cd6bd0cd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:10.357085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41985f95-6812-4358-8218-884cd6bd0cd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:10.452598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:10.507611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41985f95-6812-4358-8218-884cd6bd0cd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:10.509644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41985f95-6812-4358-8218-884cd6bd0cd3 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:10.546053 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3408/6829] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:17:16.926517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50e95d-f062-4106-8a7d-f464c2e82dfd None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:16.962587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50e95d-f062-4106-8a7d-f464c2e82dfd None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:16.972649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d9ea4-56c5-439a-a6fd-40aa28deb4ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:16.973277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d9ea4-56c5-439a-a6fd-40aa28deb4ac service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:16.973545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d9ea4-56c5-439a-a6fd-40aa28deb4ac service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:16.973813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d9ea4-56c5-439a-a6fd-40aa28deb4ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:16.993027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d9ea4-56c5-439a-a6fd-40aa28deb4ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:16.997100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d9ea4-56c5-439a-a6fd-40aa28deb4ac service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:17.017934 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3422/6830] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:17:17.849114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41985f95-6812-4358-8218-884cd6bd0cd3 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:17.853432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-41985f95-6812-4358-8218-884cd6bd0cd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:18.009503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41985f95-6812-4358-8218-884cd6bd0cd3 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:18.016226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:18.017525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:18.018258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:18.019696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] PATH_INFO: `/v3/users/655b3dac7ef04774a94c1294ad38b292` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:18.021402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=655b3dac7ef04774a94c1294ad38b292)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:18.024670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:18.103264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655b3dac7ef04774a94c1294ad38b292'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:18.127152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655b3dac7ef04774a94c1294ad38b292'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:18.147409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655b3dac7ef04774a94c1294ad38b292'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:18.160709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655b3dac7ef04774a94c1294ad38b292'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:18.162164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d1dbd07-9b6b-4943-8a19-916b60512b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655b3dac7ef04774a94c1294ad38b292'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:18.175909 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3409/6831] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:17:17 2022] DELETE /identity/v3/users/655b3dac7ef04774a94c1294ad38b292 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:17:19.183045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d9ea4-56c5-439a-a6fd-40aa28deb4ac service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:19.207536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d9ea4-56c5-439a-a6fd-40aa28deb4ac service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:19.231744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:19.400753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:19.401612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:19.402004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:19.402371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] PATH_INFO: `/v3/projects/8d0dcf5fc5e0456a9e14fc7edbda53cc` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:19.403899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0dcf5fc5e0456a9e14fc7edbda53cc)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:19.413065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:19.546828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'request_id': 'req-e43a3546-179b-45fc-babb-221ba700209d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:19.570348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'request_id': 'req-e43a3546-179b-45fc-babb-221ba700209d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:19.581455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'request_id': 'req-e43a3546-179b-45fc-babb-221ba700209d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:19.611141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] The token cache is being invalidate because project 8d0dcf5fc5e0456a9e14fc7edbda53cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:19.611433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:19.612126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e43a3546-179b-45fc-babb-221ba700209d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:19.617122 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3423/6832] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:17:19 2022] DELETE /identity/v3/projects/8d0dcf5fc5e0456a9e14fc7edbda53cc => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:19.618949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629fbf0-3c89-4717-9e3c-62b0bcdf6a02 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:19.619408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629fbf0-3c89-4717-9e3c-62b0bcdf6a02 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:19.619511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629fbf0-3c89-4717-9e3c-62b0bcdf6a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:19.648206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c629fbf0-3c89-4717-9e3c-62b0bcdf6a02 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:19.650041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c629fbf0-3c89-4717-9e3c-62b0bcdf6a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:19.783831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:19.804029 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3410/6833] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:19.805547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21455218-e2c5-4e02-b4cf-0ad4615ff8b9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:19.805547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21455218-e2c5-4e02-b4cf-0ad4615ff8b9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:19.805803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21455218-e2c5-4e02-b4cf-0ad4615ff8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:19.840879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-21455218-e2c5-4e02-b4cf-0ad4615ff8b9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:19.842506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-21455218-e2c5-4e02-b4cf-0ad4615ff8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:19.857246 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3424/6834] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:17:19.859318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629fbf0-3c89-4717-9e3c-62b0bcdf6a02 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:19.870963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629fbf0-3c89-4717-9e3c-62b0bcdf6a02 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:19.876786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406202-0035-4f0a-919e-b918ad60fc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:19.877721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406202-0035-4f0a-919e-b918ad60fc05 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:19.878234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406202-0035-4f0a-919e-b918ad60fc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:19.878595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406202-0035-4f0a-919e-b918ad60fc05 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:19.879360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406202-0035-4f0a-919e-b918ad60fc05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:17:19.879833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406202-0035-4f0a-919e-b918ad60fc05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:19.881670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406202-0035-4f0a-919e-b918ad60fc05 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:19.908242 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3411/6835] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:17:19.910243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21455218-e2c5-4e02-b4cf-0ad4615ff8b9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:19.931252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21455218-e2c5-4e02-b4cf-0ad4615ff8b9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:19.940279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7e045-3acf-4cee-a8d6-cf18cc871091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:19.941357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7e045-3acf-4cee-a8d6-cf18cc871091 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:19.941983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7e045-3acf-4cee-a8d6-cf18cc871091 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:19.942444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7e045-3acf-4cee-a8d6-cf18cc871091 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:19.944132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7e045-3acf-4cee-a8d6-cf18cc871091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:19.946409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7e045-3acf-4cee-a8d6-cf18cc871091 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:20.004964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406202-0035-4f0a-919e-b918ad60fc05 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:20.005261 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3425/6836] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:17:19 2022] POST /identity/v3/projects => generated 399 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:17:20.022595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406202-0035-4f0a-919e-b918ad60fc05 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:20.131748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db716e-dc78-4a0b-a9a0-7e9ec07232be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:20.132421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db716e-dc78-4a0b-a9a0-7e9ec07232be admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:20.132705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db716e-dc78-4a0b-a9a0-7e9ec07232be admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:20.132979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db716e-dc78-4a0b-a9a0-7e9ec07232be admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:20.135794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db716e-dc78-4a0b-a9a0-7e9ec07232be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:20.137976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db716e-dc78-4a0b-a9a0-7e9ec07232be admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:20.188214 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3412/6837] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:17:20 2022] POST /identity/v3/users => generated 300 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:17:20.190020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7e045-3acf-4cee-a8d6-cf18cc871091 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:20.216049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7e045-3acf-4cee-a8d6-cf18cc871091 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:20.229908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3d4c7-3cd8-4cf6-b490-9a76fa7ce501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:20.230803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3d4c7-3cd8-4cf6-b490-9a76fa7ce501 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:20.232670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3d4c7-3cd8-4cf6-b490-9a76fa7ce501 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:20.233048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3d4c7-3cd8-4cf6-b490-9a76fa7ce501 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:20.234416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3d4c7-3cd8-4cf6-b490-9a76fa7ce501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:20.240338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3d4c7-3cd8-4cf6-b490-9a76fa7ce501 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:20.275243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db716e-dc78-4a0b-a9a0-7e9ec07232be admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:20.287135 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3426/6838] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:20.287374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db716e-dc78-4a0b-a9a0-7e9ec07232be admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:20.303026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea183130-028a-4ae5-98c8-bb2f4fef26ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:20.305407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea183130-028a-4ae5-98c8-bb2f4fef26ea admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:20.305750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea183130-028a-4ae5-98c8-bb2f4fef26ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:20.307947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea183130-028a-4ae5-98c8-bb2f4fef26ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:20.308697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea183130-028a-4ae5-98c8-bb2f4fef26ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:20.313897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea183130-028a-4ae5-98c8-bb2f4fef26ea admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:20.345846 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3413/6839] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:20.352261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3d4c7-3cd8-4cf6-b490-9a76fa7ce501 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:20.387114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3d4c7-3cd8-4cf6-b490-9a76fa7ce501 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:20.394348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ab0fe-0f7a-40cd-a583-d98a0d829851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:20.398856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab0fe-0f7a-40cd-a583-d98a0d829851 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:20.398856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab0fe-0f7a-40cd-a583-d98a0d829851 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:20.398856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab0fe-0f7a-40cd-a583-d98a0d829851 admin admin] PATH_INFO: `/v3/projects/66100212a95b45afb4d16f48f0fca0e3/users/58294fd300b840db9d21d4c8de89a144/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:20.417600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ab0fe-0f7a-40cd-a583-d98a0d829851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66100212a95b45afb4d16f48f0fca0e3, user_id=58294fd300b840db9d21d4c8de89a144, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:20.427892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ab0fe-0f7a-40cd-a583-d98a0d829851 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:20.459003 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3427/6840] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:17:20 2022] PUT /identity/v3/projects/66100212a95b45afb4d16f48f0fca0e3/users/58294fd300b840db9d21d4c8de89a144/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:20.464443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f37d9e-62c7-4087-96a7-e85697ef3afe None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:20.464740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f37d9e-62c7-4087-96a7-e85697ef3afe None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:20.465010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f37d9e-62c7-4087-96a7-e85697ef3afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:20.517685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-45f37d9e-62c7-4087-96a7-e85697ef3afe None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:20.519368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-45f37d9e-62c7-4087-96a7-e85697ef3afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:20.662347 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3414/6841] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:20.682905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ab0fe-0f7a-40cd-a583-d98a0d829851 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:20.684822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3d7ab0fe-0f7a-40cd-a583-d98a0d829851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:20.781989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:20.799671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ab0fe-0f7a-40cd-a583-d98a0d829851 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:20.804949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2f90a-d48b-476b-a026-093d7a37c9af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:20.805600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2f90a-d48b-476b-a026-093d7a37c9af service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:20.806055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2f90a-d48b-476b-a026-093d7a37c9af service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:20.806741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2f90a-d48b-476b-a026-093d7a37c9af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:20.818656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2f90a-d48b-476b-a026-093d7a37c9af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:20.820311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2f90a-d48b-476b-a026-093d7a37c9af service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:20.831180 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3428/6842] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:17:30.107466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9cf08-5893-45bb-890b-75def9e031eb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:30.108199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9cf08-5893-45bb-890b-75def9e031eb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:30.108199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9cf08-5893-45bb-890b-75def9e031eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:30.164865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c5b9cf08-5893-45bb-890b-75def9e031eb None None] MFA Rules not processed for user `58294fd300b840db9d21d4c8de89a144`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:30.166652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c5b9cf08-5893-45bb-890b-75def9e031eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:30.312341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:30.353914 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3415/6843] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:30 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:30.365024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d7603-5aeb-47a8-bcb4-148de8bfce28 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:30.365024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d7603-5aeb-47a8-bcb4-148de8bfce28 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:30.365024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d7603-5aeb-47a8-bcb4-148de8bfce28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:30.415737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c66d7603-5aeb-47a8-bcb4-148de8bfce28 None None] MFA Rules not processed for user `58294fd300b840db9d21d4c8de89a144`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:30.419560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c66d7603-5aeb-47a8-bcb4-148de8bfce28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:30.434438 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3429/6844] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:30 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:17:30.447044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9cf08-5893-45bb-890b-75def9e031eb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:30.449126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c5b9cf08-5893-45bb-890b-75def9e031eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:30.636761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9cf08-5893-45bb-890b-75def9e031eb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:30.642878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29043321-ae94-44de-9f91-264c4c772137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:30.643620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29043321-ae94-44de-9f91-264c4c772137 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:30.643895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29043321-ae94-44de-9f91-264c4c772137 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:30.644161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29043321-ae94-44de-9f91-264c4c772137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:30.654793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29043321-ae94-44de-9f91-264c4c772137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:30.656764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29043321-ae94-44de-9f91-264c4c772137 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:30.675859 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3416/6845] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:17:40.398429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=11 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:40.534776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d7603-5aeb-47a8-bcb4-148de8bfce28 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:40.539988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c66d7603-5aeb-47a8-bcb4-148de8bfce28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:40.565623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d7603-5aeb-47a8-bcb4-148de8bfce28 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:40.573842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:40.575311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:40.576006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:40.576502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] PATH_INFO: `/v3/users/7031144183f3482d8c8e9a6dc1b1f0c5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:40.578006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7031144183f3482d8c8e9a6dc1b1f0c5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:40.580646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:40.664101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=37 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:40.701674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7031144183f3482d8c8e9a6dc1b1f0c5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:40.717007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7031144183f3482d8c8e9a6dc1b1f0c5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:40.734684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7031144183f3482d8c8e9a6dc1b1f0c5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:40.751177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7031144183f3482d8c8e9a6dc1b1f0c5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:40.752998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7031144183f3482d8c8e9a6dc1b1f0c5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:40.775197 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3430/6846] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:17:40 2022] DELETE /identity/v3/users/7031144183f3482d8c8e9a6dc1b1f0c5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:40.777522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd340e13-36d4-4d8d-8e95-b9a7323d39cb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:40.777816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd340e13-36d4-4d8d-8e95-b9a7323d39cb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:40.778744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd340e13-36d4-4d8d-8e95-b9a7323d39cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:40.824051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dd340e13-36d4-4d8d-8e95-b9a7323d39cb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:40.834350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dd340e13-36d4-4d8d-8e95-b9a7323d39cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:40.983293 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3417/6847] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:40.999247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:41.000701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:41.140129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f41d7e-4aa4-4a1f-9200-cb395a5fc134 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:41.156157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a14097-a63e-4ec4-804e-1b1d4f44343b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:41.157103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14097-a63e-4ec4-804e-1b1d4f44343b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:41.157589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14097-a63e-4ec4-804e-1b1d4f44343b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:41.158034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14097-a63e-4ec4-804e-1b1d4f44343b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:41.187014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a14097-a63e-4ec4-804e-1b1d4f44343b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:41.188464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a14097-a63e-4ec4-804e-1b1d4f44343b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:41.230891 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3431/6848] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:17:41.826817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd340e13-36d4-4d8d-8e95-b9a7323d39cb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:41.828849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dd340e13-36d4-4d8d-8e95-b9a7323d39cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:41.850627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd340e13-36d4-4d8d-8e95-b9a7323d39cb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:41.866728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:41.867933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:41.868556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:41.869781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] PATH_INFO: `/v3/projects/bf9bc22750a54b9888679eacb2fa7810` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:41.874070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf9bc22750a54b9888679eacb2fa7810)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:41.876272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:41.974788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf9bc22750a54b9888679eacb2fa7810', 'request_id': 'req-187ef441-b868-415e-9daa-1f2dbe5fbc02'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:42.005385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf9bc22750a54b9888679eacb2fa7810', 'request_id': 'req-187ef441-b868-415e-9daa-1f2dbe5fbc02'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:42.024155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf9bc22750a54b9888679eacb2fa7810', 'request_id': 'req-187ef441-b868-415e-9daa-1f2dbe5fbc02'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:42.058652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] The token cache is being invalidate because project bf9bc22750a54b9888679eacb2fa7810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:42.059153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:42.060016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:42.062750 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3418/6849] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:17:41 2022] DELETE /identity/v3/projects/bf9bc22750a54b9888679eacb2fa7810 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:42.067265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a14097-a63e-4ec4-804e-1b1d4f44343b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:42.087165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a14097-a63e-4ec4-804e-1b1d4f44343b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:42.289933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:42.290658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:42.290847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:42.291120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] PATH_INFO: `/v3/users/30db5e37bd2140ed9c664b5a845e75b1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:42.292441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30db5e37bd2140ed9c664b5a845e75b1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:42.294724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:42.331608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:42.393652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30db5e37bd2140ed9c664b5a845e75b1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:42.406121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30db5e37bd2140ed9c664b5a845e75b1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:42.436678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30db5e37bd2140ed9c664b5a845e75b1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:42.449182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30db5e37bd2140ed9c664b5a845e75b1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:42.451492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fb3888f-1649-42ca-af38-6976ab2356f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30db5e37bd2140ed9c664b5a845e75b1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:42.465799 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3432/6850] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:17:42 2022] DELETE /identity/v3/users/30db5e37bd2140ed9c664b5a845e75b1 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:43.155131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:43.161392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:43.197770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ef441-b868-415e-9daa-1f2dbe5fbc02 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:43.356942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:43.361833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:43.362342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:43.362842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] PATH_INFO: `/v3/projects/52448c57158b41b69182626c67a6ddde` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:43.365455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52448c57158b41b69182626c67a6ddde)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:43.375719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:43.473760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:43.557167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52448c57158b41b69182626c67a6ddde', 'request_id': 'req-52c545b2-4766-4f2a-8f05-4106d7d00b30'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:43.584193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52448c57158b41b69182626c67a6ddde', 'request_id': 'req-52c545b2-4766-4f2a-8f05-4106d7d00b30'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:43.593794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52448c57158b41b69182626c67a6ddde', 'request_id': 'req-52c545b2-4766-4f2a-8f05-4106d7d00b30'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:43.626084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] The token cache is being invalidate because project 52448c57158b41b69182626c67a6ddde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:43.628614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:43.632007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-52c545b2-4766-4f2a-8f05-4106d7d00b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:43.638893 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3419/6851] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:17:43 2022] DELETE /identity/v3/projects/52448c57158b41b69182626c67a6ddde => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:43.643826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e5fda-d924-4f70-9168-52e7d23a84fa None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:43.644693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e5fda-d924-4f70-9168-52e7d23a84fa None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:43.645205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e5fda-d924-4f70-9168-52e7d23a84fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:43.688228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-369e5fda-d924-4f70-9168-52e7d23a84fa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:43.693448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-369e5fda-d924-4f70-9168-52e7d23a84fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:43.844314 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3433/6852] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:43.973265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258f5ec-34c6-473f-9579-84378c845d64 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:43.973664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258f5ec-34c6-473f-9579-84378c845d64 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:43.974280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258f5ec-34c6-473f-9579-84378c845d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:44.013436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5258f5ec-34c6-473f-9579-84378c845d64 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:44.015060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5258f5ec-34c6-473f-9579-84378c845d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:44.029539 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3420/6853] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:44.034438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e5fda-d924-4f70-9168-52e7d23a84fa None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:44.045669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e5fda-d924-4f70-9168-52e7d23a84fa None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:44.052180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e12aaa-ce6a-4fa3-ad7c-afbf157e61dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:44.053030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e12aaa-ce6a-4fa3-ad7c-afbf157e61dc admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:44.053498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e12aaa-ce6a-4fa3-ad7c-afbf157e61dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:44.054052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e12aaa-ce6a-4fa3-ad7c-afbf157e61dc admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:44.054861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e12aaa-ce6a-4fa3-ad7c-afbf157e61dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:17:44.055304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e12aaa-ce6a-4fa3-ad7c-afbf157e61dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:44.057354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e12aaa-ce6a-4fa3-ad7c-afbf157e61dc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:44.073563 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3434/6854] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:17:44.077877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258f5ec-34c6-473f-9579-84378c845d64 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:44.088211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258f5ec-34c6-473f-9579-84378c845d64 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:44.098534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41842932-ea5c-4f84-9fe7-1609b365cc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:44.099455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41842932-ea5c-4f84-9fe7-1609b365cc42 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:44.099944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41842932-ea5c-4f84-9fe7-1609b365cc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:44.100532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41842932-ea5c-4f84-9fe7-1609b365cc42 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:44.101416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41842932-ea5c-4f84-9fe7-1609b365cc42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:44.104567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41842932-ea5c-4f84-9fe7-1609b365cc42 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:44.153588 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3421/6855] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:17:44 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:44.157349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e12aaa-ce6a-4fa3-ad7c-afbf157e61dc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:44.171662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e12aaa-ce6a-4fa3-ad7c-afbf157e61dc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:44.262109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436af42d-a535-4fa4-be74-09e5aac0d43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:44.263139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436af42d-a535-4fa4-be74-09e5aac0d43d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:44.263701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436af42d-a535-4fa4-be74-09e5aac0d43d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:44.264122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436af42d-a535-4fa4-be74-09e5aac0d43d admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:44.264901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436af42d-a535-4fa4-be74-09e5aac0d43d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:44.268876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436af42d-a535-4fa4-be74-09e5aac0d43d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:44.304991 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3435/6856] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:17:44 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:44.306358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41842932-ea5c-4f84-9fe7-1609b365cc42 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:44.326019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41842932-ea5c-4f84-9fe7-1609b365cc42 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:44.331759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a79f4b-154f-4d1a-82dd-9d0ea3d8e61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:44.333666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a79f4b-154f-4d1a-82dd-9d0ea3d8e61c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:44.334173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a79f4b-154f-4d1a-82dd-9d0ea3d8e61c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:44.335454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a79f4b-154f-4d1a-82dd-9d0ea3d8e61c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:44.337000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a79f4b-154f-4d1a-82dd-9d0ea3d8e61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:44.339134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a79f4b-154f-4d1a-82dd-9d0ea3d8e61c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:44.371879 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3422/6857] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:44.373576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436af42d-a535-4fa4-be74-09e5aac0d43d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:44.387755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436af42d-a535-4fa4-be74-09e5aac0d43d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:44.401314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e80db-8509-4e2c-91b7-9c2e8cd1ca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:44.401931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e80db-8509-4e2c-91b7-9c2e8cd1ca98 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:44.402205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e80db-8509-4e2c-91b7-9c2e8cd1ca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:44.402472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e80db-8509-4e2c-91b7-9c2e8cd1ca98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:44.403113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e80db-8509-4e2c-91b7-9c2e8cd1ca98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:44.405066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e80db-8509-4e2c-91b7-9c2e8cd1ca98 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:44.419203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:44.426133 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3436/6858] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:44.430244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a79f4b-154f-4d1a-82dd-9d0ea3d8e61c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:44.446589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a79f4b-154f-4d1a-82dd-9d0ea3d8e61c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:44.462053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d0f60-c8b3-440e-a63b-23ed765b3c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:44.463128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d0f60-c8b3-440e-a63b-23ed765b3c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:44.463410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d0f60-c8b3-440e-a63b-23ed765b3c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:44.463685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d0f60-c8b3-440e-a63b-23ed765b3c42 admin admin] PATH_INFO: `/v3/projects/8a2ab62208d3458a95a70b3d8bf8f512/users/82c0186d681c4a7cb49d48b142ad1b1d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:44.501495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d0f60-c8b3-440e-a63b-23ed765b3c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a2ab62208d3458a95a70b3d8bf8f512, user_id=82c0186d681c4a7cb49d48b142ad1b1d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:44.504822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d0f60-c8b3-440e-a63b-23ed765b3c42 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:44.541086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3423/6859] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:17:44 2022] PUT /identity/v3/projects/8a2ab62208d3458a95a70b3d8bf8f512/users/82c0186d681c4a7cb49d48b142ad1b1d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:17:44.541780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70c84f-3134-4a57-9530-e7b90420f598 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:44.542238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70c84f-3134-4a57-9530-e7b90420f598 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:44.542561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70c84f-3134-4a57-9530-e7b90420f598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:44.584226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cb70c84f-3134-4a57-9530-e7b90420f598 None None] MFA Rules not processed for user `82c0186d681c4a7cb49d48b142ad1b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:44.585640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cb70c84f-3134-4a57-9530-e7b90420f598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:44.739312 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3437/6860] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:17:44.750359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d0f60-c8b3-440e-a63b-23ed765b3c42 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:44.754096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af9d0f60-c8b3-440e-a63b-23ed765b3c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:44.868984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d0f60-c8b3-440e-a63b-23ed765b3c42 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:44.874793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be9987-4024-4c66-8b56-b884bb10cda6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:44.875710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be9987-4024-4c66-8b56-b884bb10cda6 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:44.876145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be9987-4024-4c66-8b56-b884bb10cda6 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:44.876616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be9987-4024-4c66-8b56-b884bb10cda6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:44.889312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be9987-4024-4c66-8b56-b884bb10cda6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:44.891250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be9987-4024-4c66-8b56-b884bb10cda6 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:44.906312 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3424/6861] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:17:53.951608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70c84f-3134-4a57-9530-e7b90420f598 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:54.094583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70c84f-3134-4a57-9530-e7b90420f598 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:54.101087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:54.102367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:54.102967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:54.103463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] PATH_INFO: `/v3/users/82c0186d681c4a7cb49d48b142ad1b1d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:54.104733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c0186d681c4a7cb49d48b142ad1b1d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:54.108402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:54.201529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c0186d681c4a7cb49d48b142ad1b1d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:54.216125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c0186d681c4a7cb49d48b142ad1b1d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:54.245465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c0186d681c4a7cb49d48b142ad1b1d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:54.269571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c0186d681c4a7cb49d48b142ad1b1d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:54.270541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c0186d681c4a7cb49d48b142ad1b1d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:54.280157 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3438/6862] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:17:53 2022] DELETE /identity/v3/users/82c0186d681c4a7cb49d48b142ad1b1d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:54.285170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeec8ff-8385-4c0a-b9db-2394da51fb18 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:54.286035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeec8ff-8385-4c0a-b9db-2394da51fb18 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:54.286563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeec8ff-8385-4c0a-b9db-2394da51fb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:54.326129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5eeec8ff-8385-4c0a-b9db-2394da51fb18 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:54.332242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5eeec8ff-8385-4c0a-b9db-2394da51fb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:54.463238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:54.495173 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3425/6863] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:54.535025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:54.535025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:54.559450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:54.704183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e601e79-727b-4d50-9436-d1ef4ff3eccc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:54.716067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770a26e-9cd0-49a5-917b-251483da7d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:54.716701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770a26e-9cd0-49a5-917b-251483da7d73 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:54.716971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770a26e-9cd0-49a5-917b-251483da7d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:54.717258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770a26e-9cd0-49a5-917b-251483da7d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:54.737540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770a26e-9cd0-49a5-917b-251483da7d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:54.740002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770a26e-9cd0-49a5-917b-251483da7d73 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:54.765750 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3439/6864] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:17:55.399367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeec8ff-8385-4c0a-b9db-2394da51fb18 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:55.400917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5eeec8ff-8385-4c0a-b9db-2394da51fb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:55.424028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeec8ff-8385-4c0a-b9db-2394da51fb18 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:55.445728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:55.446385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:55.446657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:55.447650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] PATH_INFO: `/v3/projects/8a2ab62208d3458a95a70b3d8bf8f512` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:55.454591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a2ab62208d3458a95a70b3d8bf8f512)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:55.457917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:55.560825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a2ab62208d3458a95a70b3d8bf8f512', 'request_id': 'req-924caa0f-ec71-41de-b77a-e6787f4205c2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:55.576886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a2ab62208d3458a95a70b3d8bf8f512', 'request_id': 'req-924caa0f-ec71-41de-b77a-e6787f4205c2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:55.586074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a2ab62208d3458a95a70b3d8bf8f512', 'request_id': 'req-924caa0f-ec71-41de-b77a-e6787f4205c2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:55.612475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] The token cache is being invalidate because project 8a2ab62208d3458a95a70b3d8bf8f512 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:17:55.614083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:55.616078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-924caa0f-ec71-41de-b77a-e6787f4205c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:17:55.623496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d1f07-1e65-4d32-b037-fa91b78b5653 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:55.623496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d1f07-1e65-4d32-b037-fa91b78b5653 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:55.624129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d1f07-1e65-4d32-b037-fa91b78b5653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:55.625145 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3426/6865] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:17:55 2022] DELETE /identity/v3/projects/8a2ab62208d3458a95a70b3d8bf8f512 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:55.676024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-536d1f07-1e65-4d32-b037-fa91b78b5653 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:55.677353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-536d1f07-1e65-4d32-b037-fa91b78b5653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:55.851749 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3440/6866] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:55.852414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39eac7e-265e-4c30-8e67-3f70af191012 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:55.852757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39eac7e-265e-4c30-8e67-3f70af191012 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:55.853087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39eac7e-265e-4c30-8e67-3f70af191012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:55.892780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d39eac7e-265e-4c30-8e67-3f70af191012 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:55.896476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d39eac7e-265e-4c30-8e67-3f70af191012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:55.910305 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3427/6867] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:55.914865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d1f07-1e65-4d32-b037-fa91b78b5653 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:55.929539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d1f07-1e65-4d32-b037-fa91b78b5653 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:55.938354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e698850-18af-401f-8021-b17b4c2ddebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:55.940300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e698850-18af-401f-8021-b17b4c2ddebc admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:55.940975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e698850-18af-401f-8021-b17b4c2ddebc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:55.941684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e698850-18af-401f-8021-b17b4c2ddebc admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:55.942719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e698850-18af-401f-8021-b17b4c2ddebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:17:55.943337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e698850-18af-401f-8021-b17b4c2ddebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:55.946253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e698850-18af-401f-8021-b17b4c2ddebc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:55.968149 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3441/6868] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:17:55.973003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39eac7e-265e-4c30-8e67-3f70af191012 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:55.988095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39eac7e-265e-4c30-8e67-3f70af191012 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:55.999325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f508c-13d9-4508-a989-0ebc4ea32235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:56.000775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f508c-13d9-4508-a989-0ebc4ea32235 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:56.001337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f508c-13d9-4508-a989-0ebc4ea32235 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:56.001650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f508c-13d9-4508-a989-0ebc4ea32235 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:56.002339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f508c-13d9-4508-a989-0ebc4ea32235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:56.023831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f508c-13d9-4508-a989-0ebc4ea32235 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:56.118329 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3428/6869] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:17:55 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:56.121109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e698850-18af-401f-8021-b17b4c2ddebc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:56.136176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e698850-18af-401f-8021-b17b4c2ddebc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:56.333779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415e979-490d-4ffe-9c47-ea88e04fe314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:56.334916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415e979-490d-4ffe-9c47-ea88e04fe314 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:56.335390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415e979-490d-4ffe-9c47-ea88e04fe314 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:56.335816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415e979-490d-4ffe-9c47-ea88e04fe314 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:56.337193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415e979-490d-4ffe-9c47-ea88e04fe314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:56.339944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415e979-490d-4ffe-9c47-ea88e04fe314 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:56.391119 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3442/6870] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:17:56 2022] POST /identity/v3/users => generated 288 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:17:56.395701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f508c-13d9-4508-a989-0ebc4ea32235 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:56.413763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f508c-13d9-4508-a989-0ebc4ea32235 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:56.421814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dfb26-4e03-4119-b6a5-2de15cff830e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:56.422697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dfb26-4e03-4119-b6a5-2de15cff830e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:56.423572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dfb26-4e03-4119-b6a5-2de15cff830e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:56.424174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dfb26-4e03-4119-b6a5-2de15cff830e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:56.426543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5dfb26-4e03-4119-b6a5-2de15cff830e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:56.428527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5dfb26-4e03-4119-b6a5-2de15cff830e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:56.450031 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3429/6871] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:17:56.451264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415e979-490d-4ffe-9c47-ea88e04fe314 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:56.467015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415e979-490d-4ffe-9c47-ea88e04fe314 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:56.481015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39423da5-ecc6-40db-a6f2-376454f3fe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:56.481759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39423da5-ecc6-40db-a6f2-376454f3fe9e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:56.482150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39423da5-ecc6-40db-a6f2-376454f3fe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:56.482361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39423da5-ecc6-40db-a6f2-376454f3fe9e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:56.483016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39423da5-ecc6-40db-a6f2-376454f3fe9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:56.491353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39423da5-ecc6-40db-a6f2-376454f3fe9e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:56.524167 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3443/6872] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:17:56.529273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dfb26-4e03-4119-b6a5-2de15cff830e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:56.546577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dfb26-4e03-4119-b6a5-2de15cff830e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:56.565296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a634f-ca5c-482d-ae8c-beff291ee539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:56.566990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a634f-ca5c-482d-ae8c-beff291ee539 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:56.569135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a634f-ca5c-482d-ae8c-beff291ee539 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:56.569668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a634f-ca5c-482d-ae8c-beff291ee539 admin admin] PATH_INFO: `/v3/projects/e161dad96ac34c7b8cb1406b623a3434/users/dbd32ef88226498a96715bc3f5365b6f/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:56.593923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a634f-ca5c-482d-ae8c-beff291ee539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e161dad96ac34c7b8cb1406b623a3434, user_id=dbd32ef88226498a96715bc3f5365b6f, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:56.596949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a634f-ca5c-482d-ae8c-beff291ee539 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:56.629178 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3430/6873] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:17:56 2022] PUT /identity/v3/projects/e161dad96ac34c7b8cb1406b623a3434/users/dbd32ef88226498a96715bc3f5365b6f/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:17:56.640712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5251b-d15b-4aa2-b37d-8eccbc94ba61 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:56.641313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5251b-d15b-4aa2-b37d-8eccbc94ba61 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:56.645152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5251b-d15b-4aa2-b37d-8eccbc94ba61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:56.702404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-74e5251b-d15b-4aa2-b37d-8eccbc94ba61 None None] MFA Rules not processed for user `dbd32ef88226498a96715bc3f5365b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:17:56.704070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-74e5251b-d15b-4aa2-b37d-8eccbc94ba61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:56.712714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:56.852796 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3444/6874] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:17:56.866269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a634f-ca5c-482d-ae8c-beff291ee539 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:17:56.868496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-823a634f-ca5c-482d-ae8c-beff291ee539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:17:57.009966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a634f-ca5c-482d-ae8c-beff291ee539 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:17:57.023333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c817a8-ce6b-4411-b2d2-f07248a88650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:17:57.023931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c817a8-ce6b-4411-b2d2-f07248a88650 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:17:57.024209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c817a8-ce6b-4411-b2d2-f07248a88650 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:17:57.024521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c817a8-ce6b-4411-b2d2-f07248a88650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:17:57.040817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c817a8-ce6b-4411-b2d2-f07248a88650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:17:57.044532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c817a8-ce6b-4411-b2d2-f07248a88650 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:17:57.096867 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3431/6875] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:17:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:04.097819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5251b-d15b-4aa2-b37d-8eccbc94ba61 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:04.102991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-74e5251b-d15b-4aa2-b37d-8eccbc94ba61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:04.297438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5251b-d15b-4aa2-b37d-8eccbc94ba61 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:04.298784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c817a8-ce6b-4411-b2d2-f07248a88650 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:04.302978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-66c817a8-ce6b-4411-b2d2-f07248a88650 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:04.329639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff0430-49c5-4037-9a44-16b8bcb7b15b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:04.330328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0430-49c5-4037-9a44-16b8bcb7b15b service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:04.330604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0430-49c5-4037-9a44-16b8bcb7b15b service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:04.330873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0430-49c5-4037-9a44-16b8bcb7b15b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:04.333815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b2ff0430-49c5-4037-9a44-16b8bcb7b15b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:04.339665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:04.409384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff0430-49c5-4037-9a44-16b8bcb7b15b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:04.411104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff0430-49c5-4037-9a44-16b8bcb7b15b service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:04.446070 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3445/6876] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:04.497483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c817a8-ce6b-4411-b2d2-f07248a88650 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:04.513450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:04.513450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:04.513450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:04.513450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] PATH_INFO: `/v3/users/58294fd300b840db9d21d4c8de89a144` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:04.513901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=58294fd300b840db9d21d4c8de89a144)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:04.515227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:04.648430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58294fd300b840db9d21d4c8de89a144'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:04.677129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58294fd300b840db9d21d4c8de89a144'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:04.695072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58294fd300b840db9d21d4c8de89a144'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:04.712763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58294fd300b840db9d21d4c8de89a144'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:04.714731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b3639fde-d1ee-4bab-9c59-f26ff8f0b2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58294fd300b840db9d21d4c8de89a144'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:04.729182 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3432/6877] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:04 2022] DELETE /identity/v3/users/58294fd300b840db9d21d4c8de89a144 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:05.254464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff0430-49c5-4037-9a44-16b8bcb7b15b service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:05.289958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff0430-49c5-4037-9a44-16b8bcb7b15b service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:05.452565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:05.452565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:05.452565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:05.452565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] PATH_INFO: `/v3/projects/66100212a95b45afb4d16f48f0fca0e3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:05.453483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66100212a95b45afb4d16f48f0fca0e3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:05.459450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:05.742240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:05.763457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66100212a95b45afb4d16f48f0fca0e3', 'request_id': 'req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:05.803663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66100212a95b45afb4d16f48f0fca0e3', 'request_id': 'req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:05.835557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66100212a95b45afb4d16f48f0fca0e3', 'request_id': 'req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:05.876133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] The token cache is being invalidate because project 66100212a95b45afb4d16f48f0fca0e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:05.876532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:05.877979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1eee725d-9a3a-4d4a-a46a-0f6f8ad28723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:05.882992 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3446/6878] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:05 2022] DELETE /identity/v3/projects/66100212a95b45afb4d16f48f0fca0e3 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:05.893076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b6b2b-bcd4-4328-b709-fa9bf40d3c3d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:05.893829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b6b2b-bcd4-4328-b709-fa9bf40d3c3d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:05.893829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b6b2b-bcd4-4328-b709-fa9bf40d3c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:05.933223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3e7b6b2b-bcd4-4328-b709-fa9bf40d3c3d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:05.935245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3e7b6b2b-bcd4-4328-b709-fa9bf40d3c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:06.246921 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3433/6879] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:06.257401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b82c1-e35f-41ce-9f89-fc56ecf1e889 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:06.258034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b82c1-e35f-41ce-9f89-fc56ecf1e889 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:06.258034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b82c1-e35f-41ce-9f89-fc56ecf1e889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:06.311401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-045b82c1-e35f-41ce-9f89-fc56ecf1e889 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:06.313424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-045b82c1-e35f-41ce-9f89-fc56ecf1e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:06.348358 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3447/6880] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:06.353489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b6b2b-bcd4-4328-b709-fa9bf40d3c3d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:06.397368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b6b2b-bcd4-4328-b709-fa9bf40d3c3d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:06.412167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1514665-4032-4fb9-9a0a-9f614e63ca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:06.413073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1514665-4032-4fb9-9a0a-9f614e63ca32 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:06.413562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1514665-4032-4fb9-9a0a-9f614e63ca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:06.414170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1514665-4032-4fb9-9a0a-9f614e63ca32 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:06.416481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1514665-4032-4fb9-9a0a-9f614e63ca32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:06.417260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1514665-4032-4fb9-9a0a-9f614e63ca32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:06.419113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1514665-4032-4fb9-9a0a-9f614e63ca32 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:06.463515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b82c1-e35f-41ce-9f89-fc56ecf1e889 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:06.467690 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3434/6881] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:06.476888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b82c1-e35f-41ce-9f89-fc56ecf1e889 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:06.491460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d38ec-46e4-4ede-a367-da122561df45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:06.493395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d38ec-46e4-4ede-a367-da122561df45 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:06.493869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d38ec-46e4-4ede-a367-da122561df45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:06.494096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d38ec-46e4-4ede-a367-da122561df45 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:06.494872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d38ec-46e4-4ede-a367-da122561df45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:06.497301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d38ec-46e4-4ede-a367-da122561df45 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:06.587584 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3448/6882] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:06 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:06.593695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1514665-4032-4fb9-9a0a-9f614e63ca32 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:06.612668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1514665-4032-4fb9-9a0a-9f614e63ca32 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:06.686581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:06.874306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49d015-80ae-4f95-9707-f750e0d411c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:06.875344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49d015-80ae-4f95-9707-f750e0d411c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:06.875344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49d015-80ae-4f95-9707-f750e0d411c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:06.876238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49d015-80ae-4f95-9707-f750e0d411c3 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:06.877690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49d015-80ae-4f95-9707-f750e0d411c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:06.881293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49d015-80ae-4f95-9707-f750e0d411c3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:06.972800 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3435/6883] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:06 2022] POST /identity/v3/users => generated 293 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:06.976771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d38ec-46e4-4ede-a367-da122561df45 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:06.998453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d38ec-46e4-4ede-a367-da122561df45 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:07.005142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64de32-dad7-44f0-a59f-5da942f12796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:07.007334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64de32-dad7-44f0-a59f-5da942f12796 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:07.007652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64de32-dad7-44f0-a59f-5da942f12796 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:07.007922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64de32-dad7-44f0-a59f-5da942f12796 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:07.008689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64de32-dad7-44f0-a59f-5da942f12796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:07.015845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64de32-dad7-44f0-a59f-5da942f12796 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:07.053373 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3449/6884] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:07.054768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49d015-80ae-4f95-9707-f750e0d411c3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:07.087278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49d015-80ae-4f95-9707-f750e0d411c3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:07.103244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a978de2-3882-4522-a42f-8fae2d6fdb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:07.103244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a978de2-3882-4522-a42f-8fae2d6fdb0a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:07.103244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a978de2-3882-4522-a42f-8fae2d6fdb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:07.103244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a978de2-3882-4522-a42f-8fae2d6fdb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:07.104066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a978de2-3882-4522-a42f-8fae2d6fdb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:07.107336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a978de2-3882-4522-a42f-8fae2d6fdb0a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:07.148484 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3436/6885] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:07.150758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64de32-dad7-44f0-a59f-5da942f12796 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:07.200266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64de32-dad7-44f0-a59f-5da942f12796 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:07.247253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02814832-2fba-40ba-8f06-525cf1a4a677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:07.250365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02814832-2fba-40ba-8f06-525cf1a4a677 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:07.251468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02814832-2fba-40ba-8f06-525cf1a4a677 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:07.252556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02814832-2fba-40ba-8f06-525cf1a4a677 admin admin] PATH_INFO: `/v3/projects/ac17e75269244ea598e106f7a177b811/users/2b54d1497d584aca997dc6b2d7bc69a1/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:07.324565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02814832-2fba-40ba-8f06-525cf1a4a677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac17e75269244ea598e106f7a177b811, user_id=2b54d1497d584aca997dc6b2d7bc69a1, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:07.333572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02814832-2fba-40ba-8f06-525cf1a4a677 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:07.377676 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3450/6886] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:07 2022] PUT /identity/v3/projects/ac17e75269244ea598e106f7a177b811/users/2b54d1497d584aca997dc6b2d7bc69a1/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:07.383803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd258e58-0616-49f7-837b-d1c5c4d9d1d2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:07.384373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd258e58-0616-49f7-837b-d1c5c4d9d1d2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:07.384817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd258e58-0616-49f7-837b-d1c5c4d9d1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:07.453015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cd258e58-0616-49f7-837b-d1c5c4d9d1d2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:07.457187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cd258e58-0616-49f7-837b-d1c5c4d9d1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:07.670496 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3437/6887] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:07.716898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02814832-2fba-40ba-8f06-525cf1a4a677 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:07.728366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-02814832-2fba-40ba-8f06-525cf1a4a677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:07.925742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02814832-2fba-40ba-8f06-525cf1a4a677 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:07.937859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39e368-cc23-4d36-afc0-fee34ac53384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:07.939028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39e368-cc23-4d36-afc0-fee34ac53384 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:07.939449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39e368-cc23-4d36-afc0-fee34ac53384 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:07.940049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39e368-cc23-4d36-afc0-fee34ac53384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:07.954827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39e368-cc23-4d36-afc0-fee34ac53384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:07.956656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39e368-cc23-4d36-afc0-fee34ac53384 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:07.992722 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3451/6888] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:08.900718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44da954-4ede-401c-bbee-404c5b1bfd4a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:08.901264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44da954-4ede-401c-bbee-404c5b1bfd4a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:08.901348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44da954-4ede-401c-bbee-404c5b1bfd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:09.008996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f44da954-4ede-401c-bbee-404c5b1bfd4a None None] MFA Rules not processed for user `2b54d1497d584aca997dc6b2d7bc69a1`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:09.010267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f44da954-4ede-401c-bbee-404c5b1bfd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:09.120469 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3438/6889] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:09.130955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab113e6-5e77-44cc-9267-d280368c1311 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:09.131531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab113e6-5e77-44cc-9267-d280368c1311 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:09.131531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab113e6-5e77-44cc-9267-d280368c1311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:09.166975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bab113e6-5e77-44cc-9267-d280368c1311 None None] MFA Rules not processed for user `2b54d1497d584aca997dc6b2d7bc69a1`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:09.168559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bab113e6-5e77-44cc-9267-d280368c1311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:09.179450 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3452/6890] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:09.198434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44da954-4ede-401c-bbee-404c5b1bfd4a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:09.199993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f44da954-4ede-401c-bbee-404c5b1bfd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:09.263229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:09.341336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44da954-4ede-401c-bbee-404c5b1bfd4a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:09.350465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d4464-62e6-491c-8111-bc8a28802a68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:09.351164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d4464-62e6-491c-8111-bc8a28802a68 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:09.351439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d4464-62e6-491c-8111-bc8a28802a68 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:09.351707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d4464-62e6-491c-8111-bc8a28802a68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:09.363566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d4464-62e6-491c-8111-bc8a28802a68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:09.365864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d4464-62e6-491c-8111-bc8a28802a68 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:09.382166 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3439/6891] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:09.943830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab113e6-5e77-44cc-9267-d280368c1311 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:09.961474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab113e6-5e77-44cc-9267-d280368c1311 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:09.973231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed11570-1c76-4d22-a879-6f99dd83d1b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:09.974238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed11570-1c76-4d22-a879-6f99dd83d1b5 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:09.974737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed11570-1c76-4d22-a879-6f99dd83d1b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:09.975325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed11570-1c76-4d22-a879-6f99dd83d1b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:09.990141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5ed11570-1c76-4d22-a879-6f99dd83d1b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:10.018760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed11570-1c76-4d22-a879-6f99dd83d1b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:10.020795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed11570-1c76-4d22-a879-6f99dd83d1b5 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:10.051191 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3453/6892] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:10.119263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d4464-62e6-491c-8111-bc8a28802a68 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:10.121889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4d1d4464-62e6-491c-8111-bc8a28802a68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:10.153591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d4464-62e6-491c-8111-bc8a28802a68 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:10.179305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b4849-7122-461a-9580-71b236a14943 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:10.180095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:10.180279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b4849-7122-461a-9580-71b236a14943 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:10.180554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b4849-7122-461a-9580-71b236a14943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:10.181827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:10.183990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:10.184281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] PATH_INFO: `/v3/users/dbd32ef88226498a96715bc3f5365b6f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:10.188938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbd32ef88226498a96715bc3f5365b6f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:10.191396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:10.228151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-872b4849-7122-461a-9580-71b236a14943 None None] MFA Rules not processed for user `379c929d93b54b9a9751defeef161f57`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:10.229958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-872b4849-7122-461a-9580-71b236a14943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:10.315868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd32ef88226498a96715bc3f5365b6f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:10.351958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd32ef88226498a96715bc3f5365b6f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:10.372778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd32ef88226498a96715bc3f5365b6f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:10.389768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd32ef88226498a96715bc3f5365b6f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:10.390913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a0c9d511-a2b2-415c-9a5f-925bec9b40f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd32ef88226498a96715bc3f5365b6f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:10.400770 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3440/6893] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:10 2022] DELETE /identity/v3/users/dbd32ef88226498a96715bc3f5365b6f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:10.404951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13454e10-7af2-4774-b80e-1cd805cb084c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:10.407350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13454e10-7af2-4774-b80e-1cd805cb084c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:10.408214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13454e10-7af2-4774-b80e-1cd805cb084c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:10.440495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-13454e10-7af2-4774-b80e-1cd805cb084c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:10.442362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-13454e10-7af2-4774-b80e-1cd805cb084c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:10.531099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:10.576071 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3454/6894] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:10.577160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b4849-7122-461a-9580-71b236a14943 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:10.581057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-872b4849-7122-461a-9580-71b236a14943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:10.605900 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3441/6895] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:10.608961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b4849-7122-461a-9580-71b236a14943 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:10.614005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:10.614651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:10.615087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:10.615799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] PATH_INFO: `/v3/users/064d2c9b31bd412996bb666dd8a90b9e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:10.616275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13454e10-7af2-4774-b80e-1cd805cb084c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:10.617378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=064d2c9b31bd412996bb666dd8a90b9e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:10.618323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-13454e10-7af2-4774-b80e-1cd805cb084c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:10.619728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:10.699671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:10.745897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '064d2c9b31bd412996bb666dd8a90b9e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:10.757157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '064d2c9b31bd412996bb666dd8a90b9e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:10.769219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13454e10-7af2-4774-b80e-1cd805cb084c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:10.784683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '064d2c9b31bd412996bb666dd8a90b9e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:10.786015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1beca-dd8e-48e3-8b1d-9e3082792910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:10.789327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1beca-dd8e-48e3-8b1d-9e3082792910 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:10.790056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1beca-dd8e-48e3-8b1d-9e3082792910 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:10.790056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1beca-dd8e-48e3-8b1d-9e3082792910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:10.796093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '064d2c9b31bd412996bb666dd8a90b9e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:10.800934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c929ca16-e250-4c4b-b492-06777f0a1b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '064d2c9b31bd412996bb666dd8a90b9e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:10.803443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1beca-dd8e-48e3-8b1d-9e3082792910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:10.805300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1beca-dd8e-48e3-8b1d-9e3082792910 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:10.820041 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3455/6896] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:10 2022] DELETE /identity/v3/users/064d2c9b31bd412996bb666dd8a90b9e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:10.823520 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3442/6897] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:10.824729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1be61-b5a2-4191-8f0e-8dd4ff306155 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:10.825063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1be61-b5a2-4191-8f0e-8dd4ff306155 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:10.825504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1be61-b5a2-4191-8f0e-8dd4ff306155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:10.863022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2cf1be61-b5a2-4191-8f0e-8dd4ff306155 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:10.864614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2cf1be61-b5a2-4191-8f0e-8dd4ff306155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:10.950899 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3456/6898] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:10.976672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1beca-dd8e-48e3-8b1d-9e3082792910 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:11.012006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1beca-dd8e-48e3-8b1d-9e3082792910 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:11.106657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0e53c-d553-45d0-84fe-cdafb84e3b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:11.107372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0e53c-d553-45d0-84fe-cdafb84e3b56 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:11.107656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0e53c-d553-45d0-84fe-cdafb84e3b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:11.107925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0e53c-d553-45d0-84fe-cdafb84e3b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:11.118770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0e53c-d553-45d0-84fe-cdafb84e3b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:11.120236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0e53c-d553-45d0-84fe-cdafb84e3b56 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:11.132514 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3443/6899] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:11.211463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1be61-b5a2-4191-8f0e-8dd4ff306155 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:11.213912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2cf1be61-b5a2-4191-8f0e-8dd4ff306155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:11.234287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1be61-b5a2-4191-8f0e-8dd4ff306155 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:11.242270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:11.243060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:11.243572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:11.243929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] PATH_INFO: `/v3/users/2b54d1497d584aca997dc6b2d7bc69a1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:11.253572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b54d1497d584aca997dc6b2d7bc69a1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:11.255870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:11.353469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b54d1497d584aca997dc6b2d7bc69a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.369399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b54d1497d584aca997dc6b2d7bc69a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.372393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:11.398857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b54d1497d584aca997dc6b2d7bc69a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.410133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b54d1497d584aca997dc6b2d7bc69a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.411790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b54d1497d584aca997dc6b2d7bc69a1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.425211 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3457/6900] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:11 2022] DELETE /identity/v3/users/2b54d1497d584aca997dc6b2d7bc69a1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:11.501384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0e53c-d553-45d0-84fe-cdafb84e3b56 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:11.504725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3df0e53c-d553-45d0-84fe-cdafb84e3b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:11.526248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0e53c-d553-45d0-84fe-cdafb84e3b56 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:11.572877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:11.575789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:11.671917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcaba1-3f87-470d-8802-78a280dc7c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:11.673090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcaba1-3f87-470d-8802-78a280dc7c05 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:11.673454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcaba1-3f87-470d-8802-78a280dc7c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:11.673716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcaba1-3f87-470d-8802-78a280dc7c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:11.680916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1bdcaba1-3f87-470d-8802-78a280dc7c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:11.712075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdcaba1-3f87-470d-8802-78a280dc7c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:11.712075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdcaba1-3f87-470d-8802-78a280dc7c05 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:11.754842 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3444/6901] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:11.754842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcaba1-3f87-470d-8802-78a280dc7c05 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:11.760202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1bdcaba1-3f87-470d-8802-78a280dc7c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:11.789699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d535e73-21cd-4971-b6dd-6966ef1bf817 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:11.797000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcaba1-3f87-470d-8802-78a280dc7c05 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:11.802235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:11.803190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:11.803628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:11.805254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] PATH_INFO: `/v3/projects/9504a2c03cb44ff099f07d078cc25de3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:11.806666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:11.807323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:11.808078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:11.808078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] PATH_INFO: `/v3/projects/ac17e75269244ea598e106f7a177b811` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:11.808381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9504a2c03cb44ff099f07d078cc25de3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:11.810705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac17e75269244ea598e106f7a177b811)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:11.812548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:11.812695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:11.911658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac17e75269244ea598e106f7a177b811', 'request_id': 'req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.913637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9504a2c03cb44ff099f07d078cc25de3', 'request_id': 'req-de79aa9f-9299-4b2a-95d6-db12de062799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.925844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac17e75269244ea598e106f7a177b811', 'request_id': 'req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.932062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9504a2c03cb44ff099f07d078cc25de3', 'request_id': 'req-de79aa9f-9299-4b2a-95d6-db12de062799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.941283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9504a2c03cb44ff099f07d078cc25de3', 'request_id': 'req-de79aa9f-9299-4b2a-95d6-db12de062799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.947880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac17e75269244ea598e106f7a177b811', 'request_id': 'req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.961938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] The token cache is being invalidate because project 9504a2c03cb44ff099f07d078cc25de3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:11.962562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.963561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.965695 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3458/6902] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:11 2022] DELETE /identity/v3/projects/9504a2c03cb44ff099f07d078cc25de3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:11.966742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:11.967187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:11.970673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:11.982463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] The token cache is being invalidate because project ac17e75269244ea598e106f7a177b811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:11.982973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.984214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f6116f95-f44d-4169-81cf-7e4d9c5b7b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:11.990697 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3445/6903] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:11 2022] DELETE /identity/v3/projects/ac17e75269244ea598e106f7a177b811 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:11.990697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a634e-1ae9-430b-84ca-03a6fd005325 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:11.990697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a634e-1ae9-430b-84ca-03a6fd005325 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:11.990697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a634e-1ae9-430b-84ca-03a6fd005325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.030483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e89a634e-1ae9-430b-84ca-03a6fd005325 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:12.032354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e89a634e-1ae9-430b-84ca-03a6fd005325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:12.078504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79aa9f-9299-4b2a-95d6-db12de062799 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.102248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:12.147982 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3446/6904] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:12.150835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946911c1-9d73-4d38-9baf-9a9d281c85c9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.151424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946911c1-9d73-4d38-9baf-9a9d281c85c9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.151941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946911c1-9d73-4d38-9baf-9a9d281c85c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.172637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.174658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.175045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.175424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] PATH_INFO: `/v3/projects/e161dad96ac34c7b8cb1406b623a3434` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.176743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e161dad96ac34c7b8cb1406b623a3434)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.180118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.180830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-946911c1-9d73-4d38-9baf-9a9d281c85c9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:12.182543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-946911c1-9d73-4d38-9baf-9a9d281c85c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:12.209140 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3447/6905] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:12.210606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6780959-25ec-41c0-8c9b-cc0ba5d2dcaa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.211635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6780959-25ec-41c0-8c9b-cc0ba5d2dcaa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.212072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6780959-25ec-41c0-8c9b-cc0ba5d2dcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.244302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b6780959-25ec-41c0-8c9b-cc0ba5d2dcaa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:12.246100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b6780959-25ec-41c0-8c9b-cc0ba5d2dcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:12.266157 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3448/6906] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:12.268023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047cfe5-a1ac-4175-a105-8791a6ed8b5f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.268444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047cfe5-a1ac-4175-a105-8791a6ed8b5f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.268926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047cfe5-a1ac-4175-a105-8791a6ed8b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.282422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e161dad96ac34c7b8cb1406b623a3434', 'request_id': 'req-560e3342-fcad-445d-80ae-ee08f1e21d35'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:12.295239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e161dad96ac34c7b8cb1406b623a3434', 'request_id': 'req-560e3342-fcad-445d-80ae-ee08f1e21d35'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:12.302143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f047cfe5-a1ac-4175-a105-8791a6ed8b5f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:12.303882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f047cfe5-a1ac-4175-a105-8791a6ed8b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:12.309640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e161dad96ac34c7b8cb1406b623a3434', 'request_id': 'req-560e3342-fcad-445d-80ae-ee08f1e21d35'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:12.327474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] The token cache is being invalidate because project e161dad96ac34c7b8cb1406b623a3434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:12.327987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:12.329313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:12.331375 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3459/6907] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:11 2022] DELETE /identity/v3/projects/e161dad96ac34c7b8cb1406b623a3434 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:12.332551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.336481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:12.450480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e3342-fcad-445d-80ae-ee08f1e21d35 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.457634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f618c5d-1ae7-4265-b1b4-62ebeb6e37a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.458820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f618c5d-1ae7-4265-b1b4-62ebeb6e37a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.459206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f618c5d-1ae7-4265-b1b4-62ebeb6e37a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.459574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f618c5d-1ae7-4265-b1b4-62ebeb6e37a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.460473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f618c5d-1ae7-4265-b1b4-62ebeb6e37a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:12.461435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f618c5d-1ae7-4265-b1b4-62ebeb6e37a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.463163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f618c5d-1ae7-4265-b1b4-62ebeb6e37a8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.470523 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3449/6908] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:12.470523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a005554-de4a-4b7f-89c9-db3a5ed6cf69 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.470523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a005554-de4a-4b7f-89c9-db3a5ed6cf69 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.471875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a005554-de4a-4b7f-89c9-db3a5ed6cf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.485423 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3460/6909] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:12.485423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f618c5d-1ae7-4265-b1b4-62ebeb6e37a8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.495601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f618c5d-1ae7-4265-b1b4-62ebeb6e37a8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.501535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f9e60-583d-4fb1-b927-3fdff9d848ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.502323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f9e60-583d-4fb1-b927-3fdff9d848ab admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.502982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f9e60-583d-4fb1-b927-3fdff9d848ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.502982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f9e60-583d-4fb1-b927-3fdff9d848ab admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.503816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f9e60-583d-4fb1-b927-3fdff9d848ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:12.504280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f9e60-583d-4fb1-b927-3fdff9d848ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.507279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f9e60-583d-4fb1-b927-3fdff9d848ab admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.522321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0a005554-de4a-4b7f-89c9-db3a5ed6cf69 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:12.523557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0a005554-de4a-4b7f-89c9-db3a5ed6cf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:12.538134 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3461/6910] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:12.538134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f9e60-583d-4fb1-b927-3fdff9d848ab admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.543794 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3450/6911] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:12.556604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f9e60-583d-4fb1-b927-3fdff9d848ab admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.557634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a005554-de4a-4b7f-89c9-db3a5ed6cf69 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.569069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1f79c-7597-4f16-bb31-270bce36d30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.569835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f79c-7597-4f16-bb31-270bce36d30b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.570107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f79c-7597-4f16-bb31-270bce36d30b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.570466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f79c-7597-4f16-bb31-270bce36d30b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.571221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1f79c-7597-4f16-bb31-270bce36d30b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.575221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1f79c-7597-4f16-bb31-270bce36d30b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.579092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a005554-de4a-4b7f-89c9-db3a5ed6cf69 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.589080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adb065-a555-4b85-a033-21ac8809d564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.589689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adb065-a555-4b85-a033-21ac8809d564 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.589962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adb065-a555-4b85-a033-21ac8809d564 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.590244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adb065-a555-4b85-a033-21ac8809d564 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.592722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adb065-a555-4b85-a033-21ac8809d564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.595253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adb065-a555-4b85-a033-21ac8809d564 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.635007 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3462/6912] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:12 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:12.637993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39197c2b-298c-4ace-9da3-899a5e9997c8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.638334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39197c2b-298c-4ace-9da3-899a5e9997c8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.638649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39197c2b-298c-4ace-9da3-899a5e9997c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.672688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:12.682765 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3451/6913] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:12 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:12.683776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adb065-a555-4b85-a033-21ac8809d564 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.706966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-39197c2b-298c-4ace-9da3-899a5e9997c8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:12.708425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-39197c2b-298c-4ace-9da3-899a5e9997c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:12.780273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adb065-a555-4b85-a033-21ac8809d564 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.794970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42575b-d8d3-4403-b15b-c24686bdf232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.794970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42575b-d8d3-4403-b15b-c24686bdf232 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.794970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42575b-d8d3-4403-b15b-c24686bdf232 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.794970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42575b-d8d3-4403-b15b-c24686bdf232 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.795529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42575b-d8d3-4403-b15b-c24686bdf232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.801794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42575b-d8d3-4403-b15b-c24686bdf232 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.809648 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3463/6914] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:12.810546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39197c2b-298c-4ace-9da3-899a5e9997c8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.813295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:12.822281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39197c2b-298c-4ace-9da3-899a5e9997c8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.830505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a46398-7161-41de-9366-ed411b72d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.831573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a46398-7161-41de-9366-ed411b72d14a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.832082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a46398-7161-41de-9366-ed411b72d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.832564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a46398-7161-41de-9366-ed411b72d14a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.833581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a46398-7161-41de-9366-ed411b72d14a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.834208 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3452/6915] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:12 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:12.835233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42575b-d8d3-4403-b15b-c24686bdf232 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.837207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a46398-7161-41de-9366-ed411b72d14a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.863571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42575b-d8d3-4403-b15b-c24686bdf232 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.869815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb76647-4e1b-4bb5-bc17-45cd303480ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.870434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb76647-4e1b-4bb5-bc17-45cd303480ba admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.870708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb76647-4e1b-4bb5-bc17-45cd303480ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.870974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb76647-4e1b-4bb5-bc17-45cd303480ba admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.871619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb76647-4e1b-4bb5-bc17-45cd303480ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:12.871998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb76647-4e1b-4bb5-bc17-45cd303480ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.873589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb76647-4e1b-4bb5-bc17-45cd303480ba admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.889703 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3464/6916] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:12 2022] POST /identity/v3/users => generated 288 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:12.891349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a46398-7161-41de-9366-ed411b72d14a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.894585 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3453/6917] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:12.896474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb76647-4e1b-4bb5-bc17-45cd303480ba admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.910903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb76647-4e1b-4bb5-bc17-45cd303480ba admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.918102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c13956-3cf8-4d74-92bc-9f132f6293ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.918692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c13956-3cf8-4d74-92bc-9f132f6293ea admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.918950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c13956-3cf8-4d74-92bc-9f132f6293ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.919043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a46398-7161-41de-9366-ed411b72d14a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.919237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c13956-3cf8-4d74-92bc-9f132f6293ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.920855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c13956-3cf8-4d74-92bc-9f132f6293ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.923057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c13956-3cf8-4d74-92bc-9f132f6293ea admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.925884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357d96d-db5a-44ef-bd67-5eec8cb49754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.926113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d96d-db5a-44ef-bd67-5eec8cb49754 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.926373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d96d-db5a-44ef-bd67-5eec8cb49754 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.926860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d96d-db5a-44ef-bd67-5eec8cb49754 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.927575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357d96d-db5a-44ef-bd67-5eec8cb49754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.929767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357d96d-db5a-44ef-bd67-5eec8cb49754 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.937394 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3454/6918] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:12.938442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c13956-3cf8-4d74-92bc-9f132f6293ea admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.951554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c13956-3cf8-4d74-92bc-9f132f6293ea admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.957006 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3465/6919] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:12.957006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357d96d-db5a-44ef-bd67-5eec8cb49754 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:12.957885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9b5fb-4b18-4234-b055-c03124777d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.958637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9b5fb-4b18-4234-b055-c03124777d68 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.959010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9b5fb-4b18-4234-b055-c03124777d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.959457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9b5fb-4b18-4234-b055-c03124777d68 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.960297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9b5fb-4b18-4234-b055-c03124777d68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.962627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9b5fb-4b18-4234-b055-c03124777d68 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.973452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357d96d-db5a-44ef-bd67-5eec8cb49754 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:12.978473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae60d0-0f8a-4f9e-8bbd-30ba1a90c192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:12.979269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae60d0-0f8a-4f9e-8bbd-30ba1a90c192 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:12.979813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae60d0-0f8a-4f9e-8bbd-30ba1a90c192 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:12.980232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae60d0-0f8a-4f9e-8bbd-30ba1a90c192 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:12.981223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae60d0-0f8a-4f9e-8bbd-30ba1a90c192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:12.982819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae60d0-0f8a-4f9e-8bbd-30ba1a90c192 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:12.996760 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3466/6920] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:12.997319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae60d0-0f8a-4f9e-8bbd-30ba1a90c192 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:13.006591 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3455/6921] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:12 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:13.006591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9b5fb-4b18-4234-b055-c03124777d68 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:13.008670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae60d0-0f8a-4f9e-8bbd-30ba1a90c192 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:13.102869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e021e6-f48d-4caa-a01f-9da293a48b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:13.104097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e021e6-f48d-4caa-a01f-9da293a48b3f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.104365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e021e6-f48d-4caa-a01f-9da293a48b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.104620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e021e6-f48d-4caa-a01f-9da293a48b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.111910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e021e6-f48d-4caa-a01f-9da293a48b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:13.115760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e021e6-f48d-4caa-a01f-9da293a48b3f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:13.143541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9b5fb-4b18-4234-b055-c03124777d68 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:13.153647 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3467/6922] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:13.154963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e021e6-f48d-4caa-a01f-9da293a48b3f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:13.155864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42675fb4-1ea4-4d0f-81e8-c119d1985bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:13.156790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42675fb4-1ea4-4d0f-81e8-c119d1985bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.157071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42675fb4-1ea4-4d0f-81e8-c119d1985bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.157448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42675fb4-1ea4-4d0f-81e8-c119d1985bb2 admin admin] PATH_INFO: `/v3/projects/3d93feae88b74581947ed0762b881936/users/36e9976cfdd64ddda17538745faa5a69/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.172598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e021e6-f48d-4caa-a01f-9da293a48b3f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:13.186974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d299860-0c6c-49ea-9f5b-8a3d6d71d592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:13.187242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d299860-0c6c-49ea-9f5b-8a3d6d71d592 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.190047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d299860-0c6c-49ea-9f5b-8a3d6d71d592 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.190399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d299860-0c6c-49ea-9f5b-8a3d6d71d592 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.194131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d299860-0c6c-49ea-9f5b-8a3d6d71d592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:13.195489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42675fb4-1ea4-4d0f-81e8-c119d1985bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d93feae88b74581947ed0762b881936, user_id=36e9976cfdd64ddda17538745faa5a69, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:13.197153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d299860-0c6c-49ea-9f5b-8a3d6d71d592 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:13.203955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42675fb4-1ea4-4d0f-81e8-c119d1985bb2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:13.231927 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3468/6923] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:13 2022] POST /identity/v3/users => generated 285 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:13.232292 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3456/6924] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:13 2022] PUT /identity/v3/projects/3d93feae88b74581947ed0762b881936/users/36e9976cfdd64ddda17538745faa5a69/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:13.233873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d299860-0c6c-49ea-9f5b-8a3d6d71d592 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:13.245570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42675fb4-1ea4-4d0f-81e8-c119d1985bb2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:13.247271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d299860-0c6c-49ea-9f5b-8a3d6d71d592 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:13.327529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:13.353278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8602afa-ae2f-47fd-b2f9-a6214bc3d6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:13.353942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8602afa-ae2f-47fd-b2f9-a6214bc3d6f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.354213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8602afa-ae2f-47fd-b2f9-a6214bc3d6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.354477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8602afa-ae2f-47fd-b2f9-a6214bc3d6f8 admin admin] PATH_INFO: `/v3/projects/4ef7c713556943e683842221879d1d11/users/69f518181583421b83367ccdee90813b/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.366210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42675fb4-1ea4-4d0f-81e8-c119d1985bb2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:13.372308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4250e-ab6f-4d56-8d6c-15ada78fb24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:13.373333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4250e-ab6f-4d56-8d6c-15ada78fb24a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.373597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4250e-ab6f-4d56-8d6c-15ada78fb24a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.373866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4250e-ab6f-4d56-8d6c-15ada78fb24a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.374857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4250e-ab6f-4d56-8d6c-15ada78fb24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:13.376663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4250e-ab6f-4d56-8d6c-15ada78fb24a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:13.382243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8602afa-ae2f-47fd-b2f9-a6214bc3d6f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef7c713556943e683842221879d1d11, user_id=69f518181583421b83367ccdee90813b, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:13.388598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8602afa-ae2f-47fd-b2f9-a6214bc3d6f8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:13.392937 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3457/6925] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:13.392937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8c862-7ffd-4a3f-bbed-4d00c81824c5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.392937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8c862-7ffd-4a3f-bbed-4d00c81824c5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.392937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8c862-7ffd-4a3f-bbed-4d00c81824c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.410010 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3469/6926] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:13 2022] PUT /identity/v3/projects/4ef7c713556943e683842221879d1d11/users/69f518181583421b83367ccdee90813b/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:13.410010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8602afa-ae2f-47fd-b2f9-a6214bc3d6f8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:13.462803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1dc8c862-7ffd-4a3f-bbed-4d00c81824c5 None None] MFA Rules not processed for user `36e9976cfdd64ddda17538745faa5a69`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:13.464055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1dc8c862-7ffd-4a3f-bbed-4d00c81824c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:13.486939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:13.525415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8602afa-ae2f-47fd-b2f9-a6214bc3d6f8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:13.532441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3e3a1-3245-4dd4-8f86-402f10aa744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:13.533196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3e3a1-3245-4dd4-8f86-402f10aa744e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.533785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3e3a1-3245-4dd4-8f86-402f10aa744e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.533890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3e3a1-3245-4dd4-8f86-402f10aa744e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.534667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3e3a1-3245-4dd4-8f86-402f10aa744e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:13.537282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3e3a1-3245-4dd4-8f86-402f10aa744e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:13.552241 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3470/6927] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:13.559962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661cfb2-c5ff-4601-a68c-50de87e245e5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.560241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661cfb2-c5ff-4601-a68c-50de87e245e5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.562775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661cfb2-c5ff-4601-a68c-50de87e245e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.606279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8661cfb2-c5ff-4601-a68c-50de87e245e5 None None] MFA Rules not processed for user `69f518181583421b83367ccdee90813b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:13.607636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8661cfb2-c5ff-4601-a68c-50de87e245e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:13.621573 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3458/6928] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:13.622774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8c862-7ffd-4a3f-bbed-4d00c81824c5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:13.646066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8c862-7ffd-4a3f-bbed-4d00c81824c5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:13.655114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df7ea8-78a5-45ee-bf1e-0706aa25e1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:13.656188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df7ea8-78a5-45ee-bf1e-0706aa25e1c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.657209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df7ea8-78a5-45ee-bf1e-0706aa25e1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.657741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df7ea8-78a5-45ee-bf1e-0706aa25e1c6 admin admin] PATH_INFO: `/v3/projects/3084ad4a90084be6a95ee563a39c9233/users/cfb51555eb104b03a912909cb6915077/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.690320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df7ea8-78a5-45ee-bf1e-0706aa25e1c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3084ad4a90084be6a95ee563a39c9233, user_id=cfb51555eb104b03a912909cb6915077, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:13.695724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df7ea8-78a5-45ee-bf1e-0706aa25e1c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:13.749651 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3459/6929] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:13 2022] PUT /identity/v3/projects/3084ad4a90084be6a95ee563a39c9233/users/cfb51555eb104b03a912909cb6915077/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:13.749651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df7ea8-78a5-45ee-bf1e-0706aa25e1c6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:13.761633 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3471/6930] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:13.763026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f877030-b153-4f3e-bba2-84aaa0452441 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.764391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f877030-b153-4f3e-bba2-84aaa0452441 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.764391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f877030-b153-4f3e-bba2-84aaa0452441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.867166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4f877030-b153-4f3e-bba2-84aaa0452441 None None] MFA Rules not processed for user `cfb51555eb104b03a912909cb6915077`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:13.868564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4f877030-b153-4f3e-bba2-84aaa0452441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:13.911253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df7ea8-78a5-45ee-bf1e-0706aa25e1c6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:13.924238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb9947-e3df-4c79-8804-a08418249e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:13.925047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb9947-e3df-4c79-8804-a08418249e2d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:13.925404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb9947-e3df-4c79-8804-a08418249e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:13.925788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb9947-e3df-4c79-8804-a08418249e2d admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:13.926553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb9947-e3df-4c79-8804-a08418249e2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:13.930885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb9947-e3df-4c79-8804-a08418249e2d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:14.002099 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3460/6931] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:13 2022] POST /identity/v3/projects => generated 375 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:14.003040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb9947-e3df-4c79-8804-a08418249e2d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:14.011248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83eb9947-e3df-4c79-8804-a08418249e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:14.025996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:14.120147 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3472/6932] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:14.121280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f877030-b153-4f3e-bba2-84aaa0452441 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:14.137292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb9947-e3df-4c79-8804-a08418249e2d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:14.150261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b7b96-eaf2-479c-b769-b61d27892b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:14.151522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b7b96-eaf2-479c-b769-b61d27892b8c service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:14.152043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b7b96-eaf2-479c-b769-b61d27892b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:14.152574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b7b96-eaf2-479c-b769-b61d27892b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:14.171816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f877030-b153-4f3e-bba2-84aaa0452441 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:14.274992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:14.427555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b7b96-eaf2-479c-b769-b61d27892b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:14.430525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a446b40-a4e1-4121-afb6-e8dd4c13ff19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:14.431259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a446b40-a4e1-4121-afb6-e8dd4c13ff19 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:14.433485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a446b40-a4e1-4121-afb6-e8dd4c13ff19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:14.433859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a446b40-a4e1-4121-afb6-e8dd4c13ff19 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:14.434510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b7b96-eaf2-479c-b769-b61d27892b8c service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:14.434631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a446b40-a4e1-4121-afb6-e8dd4c13ff19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:14.439293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a446b40-a4e1-4121-afb6-e8dd4c13ff19 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:14.465093 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3461/6933] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:14.466799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b7b96-eaf2-479c-b769-b61d27892b8c service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:14.471872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bc7b7b96-eaf2-479c-b769-b61d27892b8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:14.511081 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3473/6934] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:14 2022] POST /identity/v3/users => generated 294 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:14.512981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a446b40-a4e1-4121-afb6-e8dd4c13ff19 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:14.544193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a446b40-a4e1-4121-afb6-e8dd4c13ff19 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:14.560453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:14.561747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:14.561747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:14.561980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] PATH_INFO: `/v3/users/69f518181583421b83367ccdee90813b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:14.565948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f518181583421b83367ccdee90813b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:14.571118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:14.620855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b7b96-eaf2-479c-b769-b61d27892b8c service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:14.635823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883fac41-a12e-4e82-93c1-5e9d519e9d0d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:14.639210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fac41-a12e-4e82-93c1-5e9d519e9d0d service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:14.639210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fac41-a12e-4e82-93c1-5e9d519e9d0d service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:14.641744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fac41-a12e-4e82-93c1-5e9d519e9d0d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:14.755842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f518181583421b83367ccdee90813b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:14.772578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f518181583421b83367ccdee90813b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:14.810173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f518181583421b83367ccdee90813b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:14.842673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fac41-a12e-4e82-93c1-5e9d519e9d0d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:14.844930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fac41-a12e-4e82-93c1-5e9d519e9d0d service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:14.845745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f518181583421b83367ccdee90813b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:14.846754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f518181583421b83367ccdee90813b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:14.861785 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3474/6935] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:14 2022] DELETE /identity/v3/users/69f518181583421b83367ccdee90813b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:14.863009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:14.873303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:14.893002 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3462/6936] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:14.894530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315fa5a-b958-4983-8227-c31c0dc52719 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:14.894894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315fa5a-b958-4983-8227-c31c0dc52719 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:14.903387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315fa5a-b958-4983-8227-c31c0dc52719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:14.981824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6315fa5a-b958-4983-8227-c31c0dc52719 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:14.984760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6315fa5a-b958-4983-8227-c31c0dc52719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:15.028057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:15.076783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a0197-13e3-491e-92e0-a20d3efb5f9f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:15.107063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce25cb-0673-4c12-9e8c-8c11cb3fb525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:15.111340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce25cb-0673-4c12-9e8c-8c11cb3fb525 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:15.112010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce25cb-0673-4c12-9e8c-8c11cb3fb525 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:15.113157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce25cb-0673-4c12-9e8c-8c11cb3fb525 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:15.127609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce25cb-0673-4c12-9e8c-8c11cb3fb525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:15.129722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce25cb-0673-4c12-9e8c-8c11cb3fb525 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:15.144069 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3475/6937] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:15.160291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce25cb-0673-4c12-9e8c-8c11cb3fb525 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:15.201914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce25cb-0673-4c12-9e8c-8c11cb3fb525 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:15.227143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87090b-69c7-4d2a-a40f-d0ac83578b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:15.227831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87090b-69c7-4d2a-a40f-d0ac83578b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:15.228102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87090b-69c7-4d2a-a40f-d0ac83578b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:15.228373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87090b-69c7-4d2a-a40f-d0ac83578b33 admin admin] PATH_INFO: `/v3/projects/9a943c2901e144a297e4ae5ecd6296f7/users/0c69164569654f90b4700e51537dcc93/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:15.266463 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3463/6938] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:15.279374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315fa5a-b958-4983-8227-c31c0dc52719 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:15.287048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6315fa5a-b958-4983-8227-c31c0dc52719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:15.302955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87090b-69c7-4d2a-a40f-d0ac83578b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a943c2901e144a297e4ae5ecd6296f7, user_id=0c69164569654f90b4700e51537dcc93, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:15.311038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87090b-69c7-4d2a-a40f-d0ac83578b33 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:15.371375 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3476/6939] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:15 2022] PUT /identity/v3/projects/9a943c2901e144a297e4ae5ecd6296f7/users/0c69164569654f90b4700e51537dcc93/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:15.373644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87090b-69c7-4d2a-a40f-d0ac83578b33 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:15.378070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:15.561945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315fa5a-b958-4983-8227-c31c0dc52719 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:15.678897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87090b-69c7-4d2a-a40f-d0ac83578b33 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:15.690042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71662fd6-8084-418d-9885-92c9b44bdf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:15.690887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71662fd6-8084-418d-9885-92c9b44bdf0b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:15.691360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71662fd6-8084-418d-9885-92c9b44bdf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:15.709291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71662fd6-8084-418d-9885-92c9b44bdf0b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:15.710613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71662fd6-8084-418d-9885-92c9b44bdf0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:15.713471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71662fd6-8084-418d-9885-92c9b44bdf0b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:15.751976 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3477/6940] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:15.758103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71662fd6-8084-418d-9885-92c9b44bdf0b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:15.758659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235553cf-6ce3-4c62-8536-6f6a5c8dde0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:15.759251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235553cf-6ce3-4c62-8536-6f6a5c8dde0f service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:15.759541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235553cf-6ce3-4c62-8536-6f6a5c8dde0f service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:15.759809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235553cf-6ce3-4c62-8536-6f6a5c8dde0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:15.780852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71662fd6-8084-418d-9885-92c9b44bdf0b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:15.813663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235553cf-6ce3-4c62-8536-6f6a5c8dde0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:15.815048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235553cf-6ce3-4c62-8536-6f6a5c8dde0f service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:15.817166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b45c-21f0-49c0-8172-84ad5043fb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:15.818522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b45c-21f0-49c0-8172-84ad5043fb98 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:15.818522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b45c-21f0-49c0-8172-84ad5043fb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:15.818727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b45c-21f0-49c0-8172-84ad5043fb98 admin admin] PATH_INFO: `/v3/projects/9a943c2901e144a297e4ae5ecd6296f7/users/0c69164569654f90b4700e51537dcc93/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:15.821090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513b45c-21f0-49c0-8172-84ad5043fb98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a943c2901e144a297e4ae5ecd6296f7, user_id=0c69164569654f90b4700e51537dcc93, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:15.823562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513b45c-21f0-49c0-8172-84ad5043fb98 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:15.840466 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3464/6941] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:15.868398 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3478/6942] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:15 2022] PUT /identity/v3/projects/9a943c2901e144a297e4ae5ecd6296f7/users/0c69164569654f90b4700e51537dcc93/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:15.881452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23cbb7-ffb2-4377-a1ec-37135332d37e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:15.883561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23cbb7-ffb2-4377-a1ec-37135332d37e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:15.883695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23cbb7-ffb2-4377-a1ec-37135332d37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:15.971019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6c23cbb7-ffb2-4377-a1ec-37135332d37e None None] MFA Rules not processed for user `0c69164569654f90b4700e51537dcc93`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:15.982802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6c23cbb7-ffb2-4377-a1ec-37135332d37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:16.226178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b45c-21f0-49c0-8172-84ad5043fb98 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:16.226403 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3465/6943] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:16.248340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b45c-21f0-49c0-8172-84ad5043fb98 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:16.258249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63df20-a577-49a3-b8e9-4b67473119df tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c69164569654f90b4700e51537dcc93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a943c2901e144a297e4ae5ecd6296f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:16.272193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63df20-a577-49a3-b8e9-4b67473119df tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:16.272193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63df20-a577-49a3-b8e9-4b67473119df tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:16.272193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63df20-a577-49a3-b8e9-4b67473119df tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:16.273500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63df20-a577-49a3-b8e9-4b67473119df tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:16.277257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63df20-a577-49a3-b8e9-4b67473119df tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:16.318545 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3479/6944] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:18:16 2022] POST /identity/v3/domains => generated 246 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:16.322134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23cbb7-ffb2-4377-a1ec-37135332d37e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:16.340921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23cbb7-ffb2-4377-a1ec-37135332d37e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:16.526644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f2317-1624-4c68-959d-70c280cd87b8 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c69164569654f90b4700e51537dcc93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a943c2901e144a297e4ae5ecd6296f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:16.532104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f2317-1624-4c68-959d-70c280cd87b8 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:16.532845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f2317-1624-4c68-959d-70c280cd87b8 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:16.533549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f2317-1624-4c68-959d-70c280cd87b8 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:16.534813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f2317-1624-4c68-959d-70c280cd87b8 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:16.541524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f2317-1624-4c68-959d-70c280cd87b8 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:16.603580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:16.714038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63df20-a577-49a3-b8e9-4b67473119df tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:16.717167 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3466/6945] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:16 2022] POST /identity/v3/projects => generated 409 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:16.758349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63df20-a577-49a3-b8e9-4b67473119df tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:16.962307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16628606-2419-4e4d-96cd-fc5bed602866 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c69164569654f90b4700e51537dcc93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a943c2901e144a297e4ae5ecd6296f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:16.963107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628606-2419-4e4d-96cd-fc5bed602866 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:16.963397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628606-2419-4e4d-96cd-fc5bed602866 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:16.963755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628606-2419-4e4d-96cd-fc5bed602866 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:16.964410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16628606-2419-4e4d-96cd-fc5bed602866 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:16.967097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16628606-2419-4e4d-96cd-fc5bed602866 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:17.031549 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3480/6946] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:16 2022] POST /identity/v3/users => generated 348 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:17.036785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f2317-1624-4c68-959d-70c280cd87b8 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:17.060837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f2317-1624-4c68-959d-70c280cd87b8 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:17.077059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e34b68-614d-4bb2-920c-02424cf1ac60 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c69164569654f90b4700e51537dcc93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a943c2901e144a297e4ae5ecd6296f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:17.080431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e34b68-614d-4bb2-920c-02424cf1ac60 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:17.080735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e34b68-614d-4bb2-920c-02424cf1ac60 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:17.082605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e34b68-614d-4bb2-920c-02424cf1ac60 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:17.082800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e34b68-614d-4bb2-920c-02424cf1ac60 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:17.089796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e34b68-614d-4bb2-920c-02424cf1ac60 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:17.136891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16628606-2419-4e4d-96cd-fc5bed602866 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:17.149789 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3467/6947] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:17.160112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16628606-2419-4e4d-96cd-fc5bed602866 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:17.173490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8f560-07a9-4360-8312-46724f6998b6 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c69164569654f90b4700e51537dcc93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a943c2901e144a297e4ae5ecd6296f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:17.175761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8f560-07a9-4360-8312-46724f6998b6 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:17.178339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8f560-07a9-4360-8312-46724f6998b6 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:17.178637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8f560-07a9-4360-8312-46724f6998b6 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] PATH_INFO: `/v3/projects/44b51e0e264147c3b4a578fc7f87bfa1/users/8084bdebb17347308b4fe5bfe0fe4c6b/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:17.214850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:17.220374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8f560-07a9-4360-8312-46724f6998b6 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorizing `identity:create_grant(project_id=44b51e0e264147c3b4a578fc7f87bfa1, user_id=8084bdebb17347308b4fe5bfe0fe4c6b, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:17.229189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8f560-07a9-4360-8312-46724f6998b6 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:17.301130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3998aa0-ecfd-4009-8d12-8055c47acbc4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:17.301546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3998aa0-ecfd-4009-8d12-8055c47acbc4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:17.301992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3998aa0-ecfd-4009-8d12-8055c47acbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:17.305222 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3481/6948] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:17 2022] PUT /identity/v3/projects/44b51e0e264147c3b4a578fc7f87bfa1/users/8084bdebb17347308b4fe5bfe0fe4c6b/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:17.583691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e3998aa0-ecfd-4009-8d12-8055c47acbc4 None None] MFA Rules not processed for user `8084bdebb17347308b4fe5bfe0fe4c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:17.585693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e3998aa0-ecfd-4009-8d12-8055c47acbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:17.636617 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3468/6949] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:17 2022] POST /identity/v3/auth/tokens => generated 3363 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:17.649424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b4033-065f-4db5-9248-e6b6482e4de2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:17.649808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b4033-065f-4db5-9248-e6b6482e4de2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:17.650131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b4033-065f-4db5-9248-e6b6482e4de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:17.683490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-298b4033-065f-4db5-9248-e6b6482e4de2 None None] MFA Rules not processed for user `8084bdebb17347308b4fe5bfe0fe4c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:17.684836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-298b4033-065f-4db5-9248-e6b6482e4de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:17.731281 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3482/6950] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:17 2022] POST /identity/v3/auth/tokens => generated 3363 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:17.734455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3998aa0-ecfd-4009-8d12-8055c47acbc4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:17.747779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3998aa0-ecfd-4009-8d12-8055c47acbc4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:17.764272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773d01c-0a39-47bb-8949-49f4bafc5728 tempest-test_project-1435875974 tempest-user-1997522729] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8084bdebb17347308b4fe5bfe0fe4c6b', 'user_domain_id': '4b3c8ec9174747fe9a870a9715ec4585', 'system_scope': None, 'project_id': '44b51e0e264147c3b4a578fc7f87bfa1', 'project_domain_id': '4b3c8ec9174747fe9a870a9715ec4585', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:17.773345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773d01c-0a39-47bb-8949-49f4bafc5728 tempest-test_project-1435875974 tempest-user-1997522729] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:17.773668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773d01c-0a39-47bb-8949-49f4bafc5728 tempest-test_project-1435875974 tempest-user-1997522729] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:17.775953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773d01c-0a39-47bb-8949-49f4bafc5728 tempest-test_project-1435875974 tempest-user-1997522729] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:17.787198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773d01c-0a39-47bb-8949-49f4bafc5728 tempest-test_project-1435875974 tempest-user-1997522729] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:17.789356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773d01c-0a39-47bb-8949-49f4bafc5728 tempest-test_project-1435875974 tempest-user-1997522729] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:17.800617 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3469/6951] 10.222.0.16 () {56 vars in 1284 bytes} [Sat Oct 15 21:18:17 2022] GET /identity/v3/auth/tokens => generated 3363 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:17.811689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b4033-065f-4db5-9248-e6b6482e4de2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:17.976041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b4033-065f-4db5-9248-e6b6482e4de2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:17.998167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c69164569654f90b4700e51537dcc93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a943c2901e144a297e4ae5ecd6296f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:17.999252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:17.999252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:17.999541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] PATH_INFO: `/v3/users/8084bdebb17347308b4fe5bfe0fe4c6b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:18.001196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorizing `identity:delete_user(user_id=8084bdebb17347308b4fe5bfe0fe4c6b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:18.003912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:18.157303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8084bdebb17347308b4fe5bfe0fe4c6b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:18.170214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8084bdebb17347308b4fe5bfe0fe4c6b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:18.210683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8084bdebb17347308b4fe5bfe0fe4c6b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:18.236868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8084bdebb17347308b4fe5bfe0fe4c6b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:18.237717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8084bdebb17347308b4fe5bfe0fe4c6b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:18.248810 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3483/6952] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:17 2022] DELETE /identity/v3/users/8084bdebb17347308b4fe5bfe0fe4c6b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:18.253347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773d01c-0a39-47bb-8949-49f4bafc5728 tempest-test_project-1435875974 tempest-user-1997522729] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:18.279659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773d01c-0a39-47bb-8949-49f4bafc5728 tempest-test_project-1435875974 tempest-user-1997522729] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:18.395094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:18.400746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:18.541595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c69164569654f90b4700e51537dcc93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a943c2901e144a297e4ae5ecd6296f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:18.542526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:18.543746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:18.544674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] PATH_INFO: `/v3/projects/44b51e0e264147c3b4a578fc7f87bfa1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:18.550670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorizing `identity:delete_project(project_id=44b51e0e264147c3b4a578fc7f87bfa1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:18.554530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:18.673134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4cf56-4f74-40cd-97b2-ed04f4fbb872 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:18.680717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:18.681413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:18.774533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b51e0e264147c3b4a578fc7f87bfa1', 'request_id': 'req-71f948d4-bf06-4d36-a856-6edc7783dc10'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:18.800008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b51e0e264147c3b4a578fc7f87bfa1', 'request_id': 'req-71f948d4-bf06-4d36-a856-6edc7783dc10'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:18.814691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b51e0e264147c3b4a578fc7f87bfa1', 'request_id': 'req-71f948d4-bf06-4d36-a856-6edc7783dc10'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:18.840449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] The token cache is being invalidate because project 44b51e0e264147c3b4a578fc7f87bfa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:18.841381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:18.843458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:18.846688 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3470/6953] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:18 2022] DELETE /identity/v3/projects/44b51e0e264147c3b4a578fc7f87bfa1 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:18.852816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:18.858710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:18.890008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:18.891836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:18.892196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:18.892543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] PATH_INFO: `/v3/projects/4ef7c713556943e683842221879d1d11` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:18.922715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef7c713556943e683842221879d1d11)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:18.927852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:19.117899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef7c713556943e683842221879d1d11', 'request_id': 'req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:19.149142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef7c713556943e683842221879d1d11', 'request_id': 'req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:19.162907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef7c713556943e683842221879d1d11', 'request_id': 'req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:19.197896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] The token cache is being invalidate because project 4ef7c713556943e683842221879d1d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:19.197896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:19.200272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2be377d3-d7ba-40c1-9ab5-7be8fa68dd31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:19.206841 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3484/6954] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:18 2022] DELETE /identity/v3/projects/4ef7c713556943e683842221879d1d11 => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:19.211794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f06067-762f-4206-9938-0c0f96fb1d67 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:19.211990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f06067-762f-4206-9938-0c0f96fb1d67 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:19.212402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f06067-762f-4206-9938-0c0f96fb1d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:19.218397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f948d4-bf06-4d36-a856-6edc7783dc10 tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:19.301302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-53f06067-762f-4206-9938-0c0f96fb1d67 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:19.308619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-53f06067-762f-4206-9938-0c0f96fb1d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:19.459694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c69164569654f90b4700e51537dcc93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a943c2901e144a297e4ae5ecd6296f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:19.460348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:19.460624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:19.460884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] PATH_INFO: `/v3/domains/4b3c8ec9174747fe9a870a9715ec4585` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:19.462551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=4b3c8ec9174747fe9a870a9715ec4585)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:19.464592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:19.530841 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3485/6955] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:19.539154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c1768-ddb9-4123-9213-527b6e1c1807 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:19.540087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c1768-ddb9-4123-9213-527b6e1c1807 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:19.540500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c1768-ddb9-4123-9213-527b6e1c1807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:19.550071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:19.551836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:19.620545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5e4c1768-ddb9-4123-9213-527b6e1c1807 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:19.625641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5e4c1768-ddb9-4123-9213-527b6e1c1807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:19.655145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:19.656880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:19.664751 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3471/6956] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 21:18:18 2022] PATCH /identity/v3/domains/4b3c8ec9174747fe9a870a9715ec4585 => generated 247 bytes in 811 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:19.668422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:19.669968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:19.779219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:19.780346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:19.827484 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3486/6957] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:19.831299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c1768-ddb9-4123-9213-527b6e1c1807 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:19.842360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c1768-ddb9-4123-9213-527b6e1c1807 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:19.848291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062de91-4895-4421-938b-ea681158b674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:19.849127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062de91-4895-4421-938b-ea681158b674 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:19.849282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062de91-4895-4421-938b-ea681158b674 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:19.849480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062de91-4895-4421-938b-ea681158b674 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:19.850157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062de91-4895-4421-938b-ea681158b674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:19.850458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062de91-4895-4421-938b-ea681158b674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:19.852056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062de91-4895-4421-938b-ea681158b674 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:19.853689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15f24a-89f1-4d62-a4ee-204b48e3ecbd tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:19.862445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c69164569654f90b4700e51537dcc93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a943c2901e144a297e4ae5ecd6296f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:19.862445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:19.863435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:19.863435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] PATH_INFO: `/v3/domains/4b3c8ec9174747fe9a870a9715ec4585` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:19.874276 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3487/6958] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:19.876732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062de91-4895-4421-938b-ea681158b674 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:19.886890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=4b3c8ec9174747fe9a870a9715ec4585)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:19.888648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:19.890878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062de91-4895-4421-938b-ea681158b674 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:19.898470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff01117-b0cb-46f0-af8a-59c880877111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:19.899470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff01117-b0cb-46f0-af8a-59c880877111 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:19.899903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff01117-b0cb-46f0-af8a-59c880877111 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:19.900463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff01117-b0cb-46f0-af8a-59c880877111 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:19.901492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff01117-b0cb-46f0-af8a-59c880877111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:19.903873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff01117-b0cb-46f0-af8a-59c880877111 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:19.960906 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3488/6959] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:19 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:19.964611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff01117-b0cb-46f0-af8a-59c880877111 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:19.971739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:19.979427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.087323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585', 'request_id': 'req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.112848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585', 'request_id': 'req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.122348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585', 'request_id': 'req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.129293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff01117-b0cb-46f0-af8a-59c880877111 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:20.151639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] The token cache is being invalidate because project 4b3c8ec9174747fe9a870a9715ec4585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:20.152007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.155075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.182375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585', 'request_id': 'req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.183286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585', 'request_id': 'req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.184182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '4b3c8ec9174747fe9a870a9715ec4585', 'request_id': 'req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.214192 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3472/6960] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:18:19 2022] DELETE /identity/v3/domains/4b3c8ec9174747fe9a870a9715ec4585 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:20.223379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:20.232181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:20.327248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1342f-20a7-4e49-93b3-30c0208d9256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:20.327462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1342f-20a7-4e49-93b3-30c0208d9256 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:20.327581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1342f-20a7-4e49-93b3-30c0208d9256 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:20.327828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1342f-20a7-4e49-93b3-30c0208d9256 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:20.329108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1342f-20a7-4e49-93b3-30c0208d9256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:20.332330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1342f-20a7-4e49-93b3-30c0208d9256 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:20.382249 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3489/6961] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:19 2022] POST /identity/v3/users => generated 287 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:20.385542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1342f-20a7-4e49-93b3-30c0208d9256 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:20.387548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-93f1342f-20a7-4e49-93b3-30c0208d9256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:20.412915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1342f-20a7-4e49-93b3-30c0208d9256 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:20.423858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bc9d0-409f-43ab-8caf-4aad66c9e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:20.424598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bc9d0-409f-43ab-8caf-4aad66c9e4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:20.426929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bc9d0-409f-43ab-8caf-4aad66c9e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:20.427354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bc9d0-409f-43ab-8caf-4aad66c9e4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:20.428029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bc9d0-409f-43ab-8caf-4aad66c9e4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:20.429974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bc9d0-409f-43ab-8caf-4aad66c9e4d2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:20.442525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc71fc-cb94-4b4b-b269-04a263c82a4f tempest-TestDefaultProjectId-338473406 tempest-TestDefaultProjectId-338473406-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:20.454685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:20.456671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:20.457754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:20.458918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] PATH_INFO: `/v3/users/36e9976cfdd64ddda17538745faa5a69` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:20.460962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e9976cfdd64ddda17538745faa5a69)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:20.462014 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3490/6962] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:20.467330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bc9d0-409f-43ab-8caf-4aad66c9e4d2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:20.469730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:20.482817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bc9d0-409f-43ab-8caf-4aad66c9e4d2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:20.493605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:20.494373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf668b-b869-4ba5-b909-ac4028976d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:20.494998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf668b-b869-4ba5-b909-ac4028976d45 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:20.495282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf668b-b869-4ba5-b909-ac4028976d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:20.495531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf668b-b869-4ba5-b909-ac4028976d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:20.496169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf668b-b869-4ba5-b909-ac4028976d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:20.497818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf668b-b869-4ba5-b909-ac4028976d45 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:20.512813 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3491/6963] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:20.519450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf668b-b869-4ba5-b909-ac4028976d45 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:20.540149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf668b-b869-4ba5-b909-ac4028976d45 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:20.547189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97942262-3249-4e7d-939c-cf4f935b3e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:20.547951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97942262-3249-4e7d-939c-cf4f935b3e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:20.548362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97942262-3249-4e7d-939c-cf4f935b3e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:20.549413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97942262-3249-4e7d-939c-cf4f935b3e53 admin admin] PATH_INFO: `/v3/projects/f1c695b11ee344039ae3aeef5928153b/users/b640eaf136d44e29bebc282b35dfd9ee/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:20.571617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97942262-3249-4e7d-939c-cf4f935b3e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1c695b11ee344039ae3aeef5928153b, user_id=b640eaf136d44e29bebc282b35dfd9ee, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:20.575728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97942262-3249-4e7d-939c-cf4f935b3e53 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:20.581102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:20.589560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e9976cfdd64ddda17538745faa5a69'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.605161 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3492/6964] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:20 2022] PUT /identity/v3/projects/f1c695b11ee344039ae3aeef5928153b/users/b640eaf136d44e29bebc282b35dfd9ee/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:20.617974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed5d5d-14ac-407a-8632-481f5f71d8d3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:20.618907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed5d5d-14ac-407a-8632-481f5f71d8d3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:20.618907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed5d5d-14ac-407a-8632-481f5f71d8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:20.628625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e9976cfdd64ddda17538745faa5a69'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.660994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e9976cfdd64ddda17538745faa5a69'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.675867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e9976cfdd64ddda17538745faa5a69'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.678187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c1f8f922-4fe6-4cc7-acdf-1a31c70aa338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e9976cfdd64ddda17538745faa5a69'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:20.704932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-39ed5d5d-14ac-407a-8632-481f5f71d8d3 None None] MFA Rules not processed for user `b640eaf136d44e29bebc282b35dfd9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:20.706554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-39ed5d5d-14ac-407a-8632-481f5f71d8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:20.711247 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3473/6965] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:20 2022] DELETE /identity/v3/users/36e9976cfdd64ddda17538745faa5a69 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:20.717429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89787bfc-adee-4e89-9467-8a6c3ab4753e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:20.720326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89787bfc-adee-4e89-9467-8a6c3ab4753e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:20.720326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89787bfc-adee-4e89-9467-8a6c3ab4753e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:20.780094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-89787bfc-adee-4e89-9467-8a6c3ab4753e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:20.784649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-89787bfc-adee-4e89-9467-8a6c3ab4753e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:20.865639 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3493/6966] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:20.927039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed5d5d-14ac-407a-8632-481f5f71d8d3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:20.928772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-39ed5d5d-14ac-407a-8632-481f5f71d8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:20.950978 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3474/6967] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:20.994657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89787bfc-adee-4e89-9467-8a6c3ab4753e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:20.998978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-89787bfc-adee-4e89-9467-8a6c3ab4753e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:21.073896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed5d5d-14ac-407a-8632-481f5f71d8d3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:21.180349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89787bfc-adee-4e89-9467-8a6c3ab4753e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:21.184909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b79c2a-4f1c-4af6-b062-8efbf47f62fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:21.185245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b79c2a-4f1c-4af6-b062-8efbf47f62fb service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:21.186097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b79c2a-4f1c-4af6-b062-8efbf47f62fb service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:21.186408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b79c2a-4f1c-4af6-b062-8efbf47f62fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:21.192778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b1d72-535d-427f-a757-a4566394be79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:21.193741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1d72-535d-427f-a757-a4566394be79 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:21.194045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1d72-535d-427f-a757-a4566394be79 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:21.194434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1d72-535d-427f-a757-a4566394be79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:21.213850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b1d72-535d-427f-a757-a4566394be79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:21.215388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b1d72-535d-427f-a757-a4566394be79 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:21.234954 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3475/6968] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:21.278066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:21.282627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b79c2a-4f1c-4af6-b062-8efbf47f62fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:21.284537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b79c2a-4f1c-4af6-b062-8efbf47f62fb service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:21.301083 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3494/6969] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:21.341990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b1d72-535d-427f-a757-a4566394be79 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:21.345871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a37b1d72-535d-427f-a757-a4566394be79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:21.372947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b1d72-535d-427f-a757-a4566394be79 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:21.379157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:21.379990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:21.380291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:21.380548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] PATH_INFO: `/v3/users/b640eaf136d44e29bebc282b35dfd9ee` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:21.382733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b640eaf136d44e29bebc282b35dfd9ee)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:21.384851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:21.457846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:21.512580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b640eaf136d44e29bebc282b35dfd9ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:21.528745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b640eaf136d44e29bebc282b35dfd9ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:21.550204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b640eaf136d44e29bebc282b35dfd9ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:21.586680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b640eaf136d44e29bebc282b35dfd9ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:21.588182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b640eaf136d44e29bebc282b35dfd9ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:21.609830 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3476/6970] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:21 2022] DELETE /identity/v3/users/b640eaf136d44e29bebc282b35dfd9ee => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:21.612001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08df4a-728a-40b3-8072-15bd548a7cf7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:21.612702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08df4a-728a-40b3-8072-15bd548a7cf7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:21.613309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08df4a-728a-40b3-8072-15bd548a7cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:21.663717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5e08df4a-728a-40b3-8072-15bd548a7cf7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:21.666624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5e08df4a-728a-40b3-8072-15bd548a7cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:21.787781 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3495/6971] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:21.802153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:21.803961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:21.875949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08df4a-728a-40b3-8072-15bd548a7cf7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:21.877785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5e08df4a-728a-40b3-8072-15bd548a7cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:21.898946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08df4a-728a-40b3-8072-15bd548a7cf7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:21.909535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:21.911427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:21.911807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:21.912171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] PATH_INFO: `/v3/projects/3d93feae88b74581947ed0762b881936` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:21.915275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d93feae88b74581947ed0762b881936)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:21.929020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:21.985473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b228-9038-440f-8e87-452a0bb02bf5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:22.003693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde49fd-890c-434e-a0e0-49c3138e7220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:22.005854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde49fd-890c-434e-a0e0-49c3138e7220 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:22.006226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde49fd-890c-434e-a0e0-49c3138e7220 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:22.006578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde49fd-890c-434e-a0e0-49c3138e7220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:22.020605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d93feae88b74581947ed0762b881936', 'request_id': 'req-17bb423d-aad8-449f-ab47-4d2a4f47caf1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:22.030963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde49fd-890c-434e-a0e0-49c3138e7220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:22.035237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde49fd-890c-434e-a0e0-49c3138e7220 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:22.036005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d93feae88b74581947ed0762b881936', 'request_id': 'req-17bb423d-aad8-449f-ab47-4d2a4f47caf1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:22.058734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d93feae88b74581947ed0762b881936', 'request_id': 'req-17bb423d-aad8-449f-ab47-4d2a4f47caf1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:22.070057 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3477/6972] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:18:22.077226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] The token cache is being invalidate because project 3d93feae88b74581947ed0762b881936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:22.077596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:22.079005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:22.083612 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3496/6973] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:21 2022] DELETE /identity/v3/projects/3d93feae88b74581947ed0762b881936 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:22.084804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde49fd-890c-434e-a0e0-49c3138e7220 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:22.095669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde49fd-890c-434e-a0e0-49c3138e7220 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:22.163430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:22.164516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:22.164516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:22.164687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] PATH_INFO: `/v3/users/0c69164569654f90b4700e51537dcc93` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:22.165804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c69164569654f90b4700e51537dcc93)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:22.168493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:22.304725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c69164569654f90b4700e51537dcc93'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:22.306363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:22.328738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c69164569654f90b4700e51537dcc93'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:22.353690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c69164569654f90b4700e51537dcc93'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:22.365049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c69164569654f90b4700e51537dcc93'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:22.366926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b4f7d4af-f8af-46f2-a11b-48a84275e697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c69164569654f90b4700e51537dcc93'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:22.383032 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3478/6974] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:22 2022] DELETE /identity/v3/users/0c69164569654f90b4700e51537dcc93 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:22.838469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:22.839946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:22.868443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb423d-aad8-449f-ab47-4d2a4f47caf1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:22.957944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:22.959178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:22.959627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:22.960214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] PATH_INFO: `/v3/projects/9a943c2901e144a297e4ae5ecd6296f7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:22.961741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a943c2901e144a297e4ae5ecd6296f7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:22.965492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:23.006845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:23.058887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a943c2901e144a297e4ae5ecd6296f7', 'request_id': 'req-26d107e5-98cb-4df0-918d-3cd9c7b16b87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:23.072553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a943c2901e144a297e4ae5ecd6296f7', 'request_id': 'req-26d107e5-98cb-4df0-918d-3cd9c7b16b87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:23.083350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a943c2901e144a297e4ae5ecd6296f7', 'request_id': 'req-26d107e5-98cb-4df0-918d-3cd9c7b16b87'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:23.102294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] The token cache is being invalidate because project 9a943c2901e144a297e4ae5ecd6296f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:23.102643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:23.104442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:23.106910 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3497/6975] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:22 2022] DELETE /identity/v3/projects/9a943c2901e144a297e4ae5ecd6296f7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:23.113429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b5e15-01c7-4c01-9f6e-54272e4fc3da None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:23.113926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b5e15-01c7-4c01-9f6e-54272e4fc3da None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:23.114334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b5e15-01c7-4c01-9f6e-54272e4fc3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:23.154232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3c1b5e15-01c7-4c01-9f6e-54272e4fc3da None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:23.157679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3c1b5e15-01c7-4c01-9f6e-54272e4fc3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:23.169688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:23.172576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:23.314052 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3479/6976] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:23.321441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9d3e6-3230-4e2f-8676-8e2291efdddc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:23.321943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9d3e6-3230-4e2f-8676-8e2291efdddc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:23.322336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9d3e6-3230-4e2f-8676-8e2291efdddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:23.355808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e6a9d3e6-3230-4e2f-8676-8e2291efdddc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:23.358501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e6a9d3e6-3230-4e2f-8676-8e2291efdddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:23.364404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d107e5-98cb-4df0-918d-3cd9c7b16b87 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:23.371689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:23.372522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:23.373005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:23.373463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] PATH_INFO: `/v3/projects/f1c695b11ee344039ae3aeef5928153b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:23.376528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1c695b11ee344039ae3aeef5928153b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:23.377348 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3480/6977] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:23.378410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:23.380633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9d3e6-3230-4e2f-8676-8e2291efdddc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:23.395508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9d3e6-3230-4e2f-8676-8e2291efdddc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:23.403706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e3e0-06fa-4b1e-beaa-68b24147af37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:23.404410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e3e0-06fa-4b1e-beaa-68b24147af37 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:23.404655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e3e0-06fa-4b1e-beaa-68b24147af37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:23.405095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e3e0-06fa-4b1e-beaa-68b24147af37 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:23.406572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33e3e0-06fa-4b1e-beaa-68b24147af37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:23.407203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33e3e0-06fa-4b1e-beaa-68b24147af37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:23.413706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33e3e0-06fa-4b1e-beaa-68b24147af37 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:23.432131 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3481/6978] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:23.435576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e3e0-06fa-4b1e-beaa-68b24147af37 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:23.468076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e3e0-06fa-4b1e-beaa-68b24147af37 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:23.485818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1c695b11ee344039ae3aeef5928153b', 'request_id': 'req-1f796ef2-3c89-467b-bc5d-b6490de0fe17'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:23.502013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1c695b11ee344039ae3aeef5928153b', 'request_id': 'req-1f796ef2-3c89-467b-bc5d-b6490de0fe17'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:23.512077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1c695b11ee344039ae3aeef5928153b', 'request_id': 'req-1f796ef2-3c89-467b-bc5d-b6490de0fe17'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:23.536549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] The token cache is being invalidate because project f1c695b11ee344039ae3aeef5928153b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:23.536850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:23.538056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1f796ef2-3c89-467b-bc5d-b6490de0fe17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:23.540795 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3498/6979] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:23 2022] DELETE /identity/v3/projects/f1c695b11ee344039ae3aeef5928153b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:23.546204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b798c1-ed5b-4a5b-86df-097ba7b6a929 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:23.551816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b798c1-ed5b-4a5b-86df-097ba7b6a929 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:23.552253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b798c1-ed5b-4a5b-86df-097ba7b6a929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:23.622178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-59b798c1-ed5b-4a5b-86df-097ba7b6a929 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:23.623704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa21aa7-6ea1-4bcd-9a62-7c9925c48181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:23.624511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-59b798c1-ed5b-4a5b-86df-097ba7b6a929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:23.626045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa21aa7-6ea1-4bcd-9a62-7c9925c48181 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:23.627592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa21aa7-6ea1-4bcd-9a62-7c9925c48181 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:23.627877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa21aa7-6ea1-4bcd-9a62-7c9925c48181 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:23.628561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa21aa7-6ea1-4bcd-9a62-7c9925c48181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:23.633530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa21aa7-6ea1-4bcd-9a62-7c9925c48181 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:23.670297 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3499/6980] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:23.678569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f0f1b-5a33-4ee3-9016-4e8d4c30dc7b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:23.678868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f0f1b-5a33-4ee3-9016-4e8d4c30dc7b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:23.681551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f0f1b-5a33-4ee3-9016-4e8d4c30dc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:23.716921 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3482/6981] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:23 2022] POST /identity/v3/projects => generated 385 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:23.720527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa21aa7-6ea1-4bcd-9a62-7c9925c48181 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:23.724658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-caa21aa7-6ea1-4bcd-9a62-7c9925c48181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:23.751867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-908f0f1b-5a33-4ee3-9016-4e8d4c30dc7b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:23.753389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-908f0f1b-5a33-4ee3-9016-4e8d4c30dc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:23.834902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:23.844259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:23.932857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa21aa7-6ea1-4bcd-9a62-7c9925c48181 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:23.940925 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3500/6982] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:23.942804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f0f1b-5a33-4ee3-9016-4e8d4c30dc7b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:23.954988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c666e-7b2f-4231-b55d-ea0e00930281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:23.955693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c666e-7b2f-4231-b55d-ea0e00930281 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:23.956039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c666e-7b2f-4231-b55d-ea0e00930281 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:23.956863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c666e-7b2f-4231-b55d-ea0e00930281 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:23.957702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c666e-7b2f-4231-b55d-ea0e00930281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:23.961756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f0f1b-5a33-4ee3-9016-4e8d4c30dc7b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:23.966122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c666e-7b2f-4231-b55d-ea0e00930281 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:23.986797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efd90c-b0ca-428b-bbab-4402a7605d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:23.987462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd90c-b0ca-428b-bbab-4402a7605d49 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:23.987743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd90c-b0ca-428b-bbab-4402a7605d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:23.987994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd90c-b0ca-428b-bbab-4402a7605d49 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:23.988699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efd90c-b0ca-428b-bbab-4402a7605d49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:23.988996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efd90c-b0ca-428b-bbab-4402a7605d49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:23.990736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efd90c-b0ca-428b-bbab-4402a7605d49 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:24.028667 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3483/6983] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:23 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:24.032665 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3501/6984] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:24.036597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efd90c-b0ca-428b-bbab-4402a7605d49 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.039484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c666e-7b2f-4231-b55d-ea0e00930281 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.060066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efd90c-b0ca-428b-bbab-4402a7605d49 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:24.068267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c666e-7b2f-4231-b55d-ea0e00930281 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:24.073819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253af6c5-7795-4526-8ddc-64def8cd2bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:24.074532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253af6c5-7795-4526-8ddc-64def8cd2bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.074798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253af6c5-7795-4526-8ddc-64def8cd2bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.075138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253af6c5-7795-4526-8ddc-64def8cd2bc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.078550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253af6c5-7795-4526-8ddc-64def8cd2bc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:24.083225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253af6c5-7795-4526-8ddc-64def8cd2bc1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:24.090079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4aa79a-12c5-4c7b-bf87-806b65ac0083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:24.090708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aa79a-12c5-4c7b-bf87-806b65ac0083 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.090986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aa79a-12c5-4c7b-bf87-806b65ac0083 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.091261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aa79a-12c5-4c7b-bf87-806b65ac0083 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.091965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4aa79a-12c5-4c7b-bf87-806b65ac0083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:24.094456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4aa79a-12c5-4c7b-bf87-806b65ac0083 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:24.124017 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3484/6985] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:24.128654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4aa79a-12c5-4c7b-bf87-806b65ac0083 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.144513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4aa79a-12c5-4c7b-bf87-806b65ac0083 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:24.161156 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3502/6986] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:24 2022] POST /identity/v3/projects => generated 357 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:24.162954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253af6c5-7795-4526-8ddc-64def8cd2bc1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.168012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8d6ca-dd18-4909-819e-64f47564230d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:24.169816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8d6ca-dd18-4909-819e-64f47564230d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.170140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8d6ca-dd18-4909-819e-64f47564230d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.170349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8d6ca-dd18-4909-819e-64f47564230d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.174264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8d6ca-dd18-4909-819e-64f47564230d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:24.176278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8d6ca-dd18-4909-819e-64f47564230d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:24.204674 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3485/6987] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:24.212146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8d6ca-dd18-4909-819e-64f47564230d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.237642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8d6ca-dd18-4909-819e-64f47564230d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:24.295454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253af6c5-7795-4526-8ddc-64def8cd2bc1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:24.301215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e5f6-8b9c-4d2d-8b70-16e177137214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:24.301845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e5f6-8b9c-4d2d-8b70-16e177137214 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.302115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e5f6-8b9c-4d2d-8b70-16e177137214 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.302381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e5f6-8b9c-4d2d-8b70-16e177137214 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.303139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8e5f6-8b9c-4d2d-8b70-16e177137214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:24.305241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8e5f6-8b9c-4d2d-8b70-16e177137214 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:24.343036 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3503/6988] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:24 2022] POST /identity/v3/users => generated 279 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:24.347495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e5f6-8b9c-4d2d-8b70-16e177137214 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.360278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e5f6-8b9c-4d2d-8b70-16e177137214 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:24.361948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309816b-526f-4423-9ab8-33da4a7d41d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:24.363505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309816b-526f-4423-9ab8-33da4a7d41d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.365081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309816b-526f-4423-9ab8-33da4a7d41d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.365355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309816b-526f-4423-9ab8-33da4a7d41d4 admin admin] PATH_INFO: `/v3/projects/bfae4db2e61749b3b7be1f87280b58ef/users/ec76a4b55c6d46d0bfefc7957f8d48ae/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.367439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665e811-1d8d-483f-a1b8-bff62f3d798b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:24.369606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665e811-1d8d-483f-a1b8-bff62f3d798b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.369886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665e811-1d8d-483f-a1b8-bff62f3d798b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.370242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665e811-1d8d-483f-a1b8-bff62f3d798b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.371372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665e811-1d8d-483f-a1b8-bff62f3d798b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:24.373660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665e811-1d8d-483f-a1b8-bff62f3d798b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:24.389114 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3504/6989] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:24.390862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665e811-1d8d-483f-a1b8-bff62f3d798b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.401073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309816b-526f-4423-9ab8-33da4a7d41d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfae4db2e61749b3b7be1f87280b58ef, user_id=ec76a4b55c6d46d0bfefc7957f8d48ae, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:24.404269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309816b-526f-4423-9ab8-33da4a7d41d4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:24.410206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665e811-1d8d-483f-a1b8-bff62f3d798b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:24.429929 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3486/6990] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:24 2022] PUT /identity/v3/projects/bfae4db2e61749b3b7be1f87280b58ef/users/ec76a4b55c6d46d0bfefc7957f8d48ae/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:24.439114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c755e-ad7a-4cd1-b5dd-d52f0297e085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:24.440130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c755e-ad7a-4cd1-b5dd-d52f0297e085 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.441809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c755e-ad7a-4cd1-b5dd-d52f0297e085 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.442690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb841b8-a521-4275-925c-5155ac6484d5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.442896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c755e-ad7a-4cd1-b5dd-d52f0297e085 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.443212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb841b8-a521-4275-925c-5155ac6484d5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.443588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb841b8-a521-4275-925c-5155ac6484d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.444341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c755e-ad7a-4cd1-b5dd-d52f0297e085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:24.449120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c755e-ad7a-4cd1-b5dd-d52f0297e085 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:24.471901 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3505/6991] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:24.476221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c755e-ad7a-4cd1-b5dd-d52f0297e085 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.485702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cdb841b8-a521-4275-925c-5155ac6484d5 None None] MFA Rules not processed for user `ec76a4b55c6d46d0bfefc7957f8d48ae`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:24.487323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cdb841b8-a521-4275-925c-5155ac6484d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:24.494453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c755e-ad7a-4cd1-b5dd-d52f0297e085 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:24.587840 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3487/6992] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:24.592502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb841b8-a521-4275-925c-5155ac6484d5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.611024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:24.622162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bed50-81b1-4e97-a861-55500f84426e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:24.622848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bed50-81b1-4e97-a861-55500f84426e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.623127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bed50-81b1-4e97-a861-55500f84426e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.623390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bed50-81b1-4e97-a861-55500f84426e admin admin] PATH_INFO: `/v3/projects/ee8acd6042024c0e8add968e6af51da5/users/51503af6da0e423184b9c0917f62dfbb/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.649510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:24.659971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bed50-81b1-4e97-a861-55500f84426e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8acd6042024c0e8add968e6af51da5, user_id=51503af6da0e423184b9c0917f62dfbb, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:24.663804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bed50-81b1-4e97-a861-55500f84426e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:24.682286 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3506/6993] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:24 2022] PUT /identity/v3/projects/ee8acd6042024c0e8add968e6af51da5/users/51503af6da0e423184b9c0917f62dfbb/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:24.690534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb841b8-a521-4275-925c-5155ac6484d5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:24.693389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a629c6-1e91-4e27-abab-f7a49c86c807 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.693672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a629c6-1e91-4e27-abab-f7a49c86c807 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.693936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a629c6-1e91-4e27-abab-f7a49c86c807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.759101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-01a629c6-1e91-4e27-abab-f7a49c86c807 None None] MFA Rules not processed for user `51503af6da0e423184b9c0917f62dfbb`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:24.760502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-01a629c6-1e91-4e27-abab-f7a49c86c807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:24.845163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc8c53-dd1d-4fd8-a2c9-092c29a26784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:24.845163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc8c53-dd1d-4fd8-a2c9-092c29a26784 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:24.845163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc8c53-dd1d-4fd8-a2c9-092c29a26784 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:24.845999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc8c53-dd1d-4fd8-a2c9-092c29a26784 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:24.846805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc8c53-dd1d-4fd8-a2c9-092c29a26784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:24.849966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc8c53-dd1d-4fd8-a2c9-092c29a26784 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:24.887104 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3507/6994] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:24.892709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a629c6-1e91-4e27-abab-f7a49c86c807 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.918047 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3488/6995] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:24 2022] POST /identity/v3/projects => generated 387 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:24.919034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc8c53-dd1d-4fd8-a2c9-092c29a26784 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:24.921215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a629c6-1e91-4e27-abab-f7a49c86c807 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.037142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc8c53-dd1d-4fd8-a2c9-092c29a26784 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.038257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2901afd-c3e8-4e80-83b9-bc93a582227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.039636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2901afd-c3e8-4e80-83b9-bc93a582227b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.040178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2901afd-c3e8-4e80-83b9-bc93a582227b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.041280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2901afd-c3e8-4e80-83b9-bc93a582227b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.043589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2901afd-c3e8-4e80-83b9-bc93a582227b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.045790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932930f9-3cc6-4ed5-a21b-5b82c3ef8af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.046753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932930f9-3cc6-4ed5-a21b-5b82c3ef8af8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.047392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932930f9-3cc6-4ed5-a21b-5b82c3ef8af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.047601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932930f9-3cc6-4ed5-a21b-5b82c3ef8af8 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.047759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2901afd-c3e8-4e80-83b9-bc93a582227b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.048521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932930f9-3cc6-4ed5-a21b-5b82c3ef8af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.051440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932930f9-3cc6-4ed5-a21b-5b82c3ef8af8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.094569 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3489/6996] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:24 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:25.096548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932930f9-3cc6-4ed5-a21b-5b82c3ef8af8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:25.120781 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3508/6997] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:24 2022] POST /identity/v3/projects => generated 355 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:25.121531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2901afd-c3e8-4e80-83b9-bc93a582227b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:25.129475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932930f9-3cc6-4ed5-a21b-5b82c3ef8af8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.224386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:25.336556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2901afd-c3e8-4e80-83b9-bc93a582227b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.344946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee464581-403a-4f4e-8e6c-e0863a70894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.346429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee464581-403a-4f4e-8e6c-e0863a70894a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.346914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee464581-403a-4f4e-8e6c-e0863a70894a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.348193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee464581-403a-4f4e-8e6c-e0863a70894a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.350044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee464581-403a-4f4e-8e6c-e0863a70894a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.351201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100a068-22e5-4406-a57b-3202f25206d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.351823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100a068-22e5-4406-a57b-3202f25206d2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.352085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100a068-22e5-4406-a57b-3202f25206d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.352349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100a068-22e5-4406-a57b-3202f25206d2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.352973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100a068-22e5-4406-a57b-3202f25206d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.353511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee464581-403a-4f4e-8e6c-e0863a70894a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.354875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100a068-22e5-4406-a57b-3202f25206d2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.377642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:25.382026 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3490/6998] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:25.387098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee464581-403a-4f4e-8e6c-e0863a70894a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:25.400386 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3509/6999] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:25 2022] POST /identity/v3/users => generated 284 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:25.402030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100a068-22e5-4406-a57b-3202f25206d2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:25.414430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100a068-22e5-4406-a57b-3202f25206d2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.417992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee464581-403a-4f4e-8e6c-e0863a70894a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.420746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c217d-1dc6-4695-b0fa-e795d1f7cd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.421545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c217d-1dc6-4695-b0fa-e795d1f7cd0f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.421896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c217d-1dc6-4695-b0fa-e795d1f7cd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.422227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c217d-1dc6-4695-b0fa-e795d1f7cd0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.422980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c217d-1dc6-4695-b0fa-e795d1f7cd0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.424805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c217d-1dc6-4695-b0fa-e795d1f7cd0f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.438671 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3510/7000] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:25.442558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce364a60-bbb9-4d3d-b898-53a75d47c1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.443248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c217d-1dc6-4695-b0fa-e795d1f7cd0f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:25.451206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce364a60-bbb9-4d3d-b898-53a75d47c1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.451966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce364a60-bbb9-4d3d-b898-53a75d47c1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.451966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce364a60-bbb9-4d3d-b898-53a75d47c1d2 admin admin] PATH_INFO: `/v3/projects/8ab240549a594f99b632515b1d7ca96d/users/67c72b8f4ca644ddbeefdfd7aaf74e7b/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.455368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c217d-1dc6-4695-b0fa-e795d1f7cd0f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.461091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f50db-c9e5-4dcf-80cc-55966752ba44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.461834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f50db-c9e5-4dcf-80cc-55966752ba44 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.462169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f50db-c9e5-4dcf-80cc-55966752ba44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.462499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f50db-c9e5-4dcf-80cc-55966752ba44 admin admin] PATH_INFO: `/v3/projects/e78c67f5004a4ac0954ef4783fa807d7/users/5edbb1f362ec4d329c8a29c7a6b6f408/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.475877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce364a60-bbb9-4d3d-b898-53a75d47c1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab240549a594f99b632515b1d7ca96d, user_id=67c72b8f4ca644ddbeefdfd7aaf74e7b, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.479296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce364a60-bbb9-4d3d-b898-53a75d47c1d2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.482289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f50db-c9e5-4dcf-80cc-55966752ba44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78c67f5004a4ac0954ef4783fa807d7, user_id=5edbb1f362ec4d329c8a29c7a6b6f408, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.485432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f50db-c9e5-4dcf-80cc-55966752ba44 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.515644 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3491/7001] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:25 2022] PUT /identity/v3/projects/8ab240549a594f99b632515b1d7ca96d/users/67c72b8f4ca644ddbeefdfd7aaf74e7b/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:25.517233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce364a60-bbb9-4d3d-b898-53a75d47c1d2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:25.517475 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3511/7002] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:25 2022] PUT /identity/v3/projects/e78c67f5004a4ac0954ef4783fa807d7/users/5edbb1f362ec4d329c8a29c7a6b6f408/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:25.527731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f50db-c9e5-4dcf-80cc-55966752ba44 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:25.669382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce364a60-bbb9-4d3d-b898-53a75d47c1d2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.676149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72923cd2-9660-4bb1-b5df-55e60e93cbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.676780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72923cd2-9660-4bb1-b5df-55e60e93cbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.677146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72923cd2-9660-4bb1-b5df-55e60e93cbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.677419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72923cd2-9660-4bb1-b5df-55e60e93cbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.678302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72923cd2-9660-4bb1-b5df-55e60e93cbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.681882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72923cd2-9660-4bb1-b5df-55e60e93cbc9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.683424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f50db-c9e5-4dcf-80cc-55966752ba44 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.695245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb086dcf-670b-4e02-b1a1-0a5537125af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.695862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb086dcf-670b-4e02-b1a1-0a5537125af6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.696133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb086dcf-670b-4e02-b1a1-0a5537125af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.696482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb086dcf-670b-4e02-b1a1-0a5537125af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.697014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb086dcf-670b-4e02-b1a1-0a5537125af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.700968 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3492/7003] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:25.701152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb086dcf-670b-4e02-b1a1-0a5537125af6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.702216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72923cd2-9660-4bb1-b5df-55e60e93cbc9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:25.714336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72923cd2-9660-4bb1-b5df-55e60e93cbc9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.724174 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3512/7004] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:25.725236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e44acb-cd08-489f-81be-5c0e0b7faa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.725392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb086dcf-670b-4e02-b1a1-0a5537125af6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:25.726554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e44acb-cd08-489f-81be-5c0e0b7faa0b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.726974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e44acb-cd08-489f-81be-5c0e0b7faa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.727353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e44acb-cd08-489f-81be-5c0e0b7faa0b admin admin] PATH_INFO: `/v3/projects/8ab240549a594f99b632515b1d7ca96d/users/67c72b8f4ca644ddbeefdfd7aaf74e7b/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.729709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e44acb-cd08-489f-81be-5c0e0b7faa0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab240549a594f99b632515b1d7ca96d, user_id=67c72b8f4ca644ddbeefdfd7aaf74e7b, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.734461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e44acb-cd08-489f-81be-5c0e0b7faa0b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.751741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb086dcf-670b-4e02-b1a1-0a5537125af6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.752600 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3493/7005] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:25 2022] PUT /identity/v3/projects/8ab240549a594f99b632515b1d7ca96d/users/67c72b8f4ca644ddbeefdfd7aaf74e7b/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:25.764749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3f18d-6396-4b78-abb7-0a7c9cbcd28c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.765151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3f18d-6396-4b78-abb7-0a7c9cbcd28c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.766005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3f18d-6396-4b78-abb7-0a7c9cbcd28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.813776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7fd3f18d-6396-4b78-abb7-0a7c9cbcd28c None None] MFA Rules not processed for user `67c72b8f4ca644ddbeefdfd7aaf74e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:25.815515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7fd3f18d-6396-4b78-abb7-0a7c9cbcd28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:25.855782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd188d8-110e-4036-9a47-9570d35c93f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:25.856468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd188d8-110e-4036-9a47-9570d35c93f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.856740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd188d8-110e-4036-9a47-9570d35c93f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.857010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd188d8-110e-4036-9a47-9570d35c93f8 admin admin] PATH_INFO: `/v3/projects/e78c67f5004a4ac0954ef4783fa807d7/users/5edbb1f362ec4d329c8a29c7a6b6f408/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.860260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd188d8-110e-4036-9a47-9570d35c93f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78c67f5004a4ac0954ef4783fa807d7, user_id=5edbb1f362ec4d329c8a29c7a6b6f408, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:25.863711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd188d8-110e-4036-9a47-9570d35c93f8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:25.884395 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3513/7006] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:25 2022] PUT /identity/v3/projects/e78c67f5004a4ac0954ef4783fa807d7/users/5edbb1f362ec4d329c8a29c7a6b6f408/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:25.896306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b137f-6e8f-4482-ac2f-61e3963d085c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:25.896606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b137f-6e8f-4482-ac2f-61e3963d085c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:25.896912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b137f-6e8f-4482-ac2f-61e3963d085c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:25.947693 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3494/7007] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:25.960534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3f18d-6396-4b78-abb7-0a7c9cbcd28c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:25.968098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bb0b137f-6e8f-4482-ac2f-61e3963d085c None None] MFA Rules not processed for user `5edbb1f362ec4d329c8a29c7a6b6f408`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:25.972292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bb0b137f-6e8f-4482-ac2f-61e3963d085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:25.984824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3f18d-6396-4b78-abb7-0a7c9cbcd28c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:25.998137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:26.035774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:26.068494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26590e4f-ce66-468d-878e-22fa9c349b76 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c72b8f4ca644ddbeefdfd7aaf74e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ab240549a594f99b632515b1d7ca96d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:26.069144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26590e4f-ce66-468d-878e-22fa9c349b76 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:26.069421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26590e4f-ce66-468d-878e-22fa9c349b76 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:26.069688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26590e4f-ce66-468d-878e-22fa9c349b76 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:26.070314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26590e4f-ce66-468d-878e-22fa9c349b76 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:26.073865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26590e4f-ce66-468d-878e-22fa9c349b76 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:26.100968 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3514/7008] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:26.113843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b137f-6e8f-4482-ac2f-61e3963d085c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:26.115503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bb0b137f-6e8f-4482-ac2f-61e3963d085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:26.129252 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3495/7009] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:25 2022] POST /identity/v3/projects => generated 360 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:26.130593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26590e4f-ce66-468d-878e-22fa9c349b76 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:26.214938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b137f-6e8f-4482-ac2f-61e3963d085c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:26.243924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26590e4f-ce66-468d-878e-22fa9c349b76 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:26.248784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e8ce4-8bd4-4948-9d07-b2e7e5b9fd1e tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c72b8f4ca644ddbeefdfd7aaf74e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ab240549a594f99b632515b1d7ca96d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:26.249485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e8ce4-8bd4-4948-9d07-b2e7e5b9fd1e tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:26.250281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e8ce4-8bd4-4948-9d07-b2e7e5b9fd1e tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:26.250556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e8ce4-8bd4-4948-9d07-b2e7e5b9fd1e tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] PATH_INFO: `/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags/tempest-tag-404291727` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:26.251918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e8ce4-8bd4-4948-9d07-b2e7e5b9fd1e tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorizing `identity:create_project_tag(project_id=399ada06bc2a4e73aa157fb4f8f5bc77, value=tempest-tag-404291727)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:26.254815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e8ce4-8bd4-4948-9d07-b2e7e5b9fd1e tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:26.291963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55fdd4-17d5-490e-87ae-be096a9bd474 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:26.292616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55fdd4-17d5-490e-87ae-be096a9bd474 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:26.292863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55fdd4-17d5-490e-87ae-be096a9bd474 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:26.293146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55fdd4-17d5-490e-87ae-be096a9bd474 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:26.364357 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-da2e8ce4-8bd4-4948-9d07-b2e7e5b9fd1e tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-404291727. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:18:26.364357 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:18:26.364357 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:18:26.383534 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3496/7010] 10.222.0.16 () {60 vars in 1330 bytes} [Sat Oct 15 21:18:26 2022] PUT /identity/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags/tempest-tag-404291727 => generated 0 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 290 bytes (1 switches on core 0) Oct 15 21:18:26.387259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e8ce4-8bd4-4948-9d07-b2e7e5b9fd1e tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:26.396743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55fdd4-17d5-490e-87ae-be096a9bd474 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:26.398618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55fdd4-17d5-490e-87ae-be096a9bd474 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:26.398983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e8ce4-8bd4-4948-9d07-b2e7e5b9fd1e tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:26.411805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61dc1-a530-40ab-bdda-48e57352b16a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c72b8f4ca644ddbeefdfd7aaf74e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ab240549a594f99b632515b1d7ca96d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:26.412454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61dc1-a530-40ab-bdda-48e57352b16a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:26.413064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61dc1-a530-40ab-bdda-48e57352b16a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:26.413214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3515/7011] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:26.413341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61dc1-a530-40ab-bdda-48e57352b16a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] PATH_INFO: `/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags/tempest-tag-404291727` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:26.434707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d61dc1-a530-40ab-bdda-48e57352b16a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=399ada06bc2a4e73aa157fb4f8f5bc77, value=tempest-tag-404291727)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:26.436550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d61dc1-a530-40ab-bdda-48e57352b16a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:26.453950 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3497/7012] 10.222.0.16 () {58 vars in 1311 bytes} [Sat Oct 15 21:18:26 2022] GET /identity/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags/tempest-tag-404291727 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:26.460663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55fdd4-17d5-490e-87ae-be096a9bd474 service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:26.483067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55fdd4-17d5-490e-87ae-be096a9bd474 service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:26.495384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf619f5-51a1-43d6-ae98-58f7378e229b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c72b8f4ca644ddbeefdfd7aaf74e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ab240549a594f99b632515b1d7ca96d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:26.496257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf619f5-51a1-43d6-ae98-58f7378e229b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:26.496540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf619f5-51a1-43d6-ae98-58f7378e229b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:26.496823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf619f5-51a1-43d6-ae98-58f7378e229b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] PATH_INFO: `/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:26.498376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf619f5-51a1-43d6-ae98-58f7378e229b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorizing `identity:update_project_tags(project_id=399ada06bc2a4e73aa157fb4f8f5bc77)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:26.502009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf619f5-51a1-43d6-ae98-58f7378e229b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:26.599877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61dc1-a530-40ab-bdda-48e57352b16a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:26.627441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61dc1-a530-40ab-bdda-48e57352b16a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:26.628101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:26.645181 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3516/7013] 10.222.0.16 () {60 vars in 1265 bytes} [Sat Oct 15 21:18:26 2022] PUT /identity/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags => generated 139 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:26.647721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b820586-009c-4b31-941f-fa39fe13162e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:26.648290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf619f5-51a1-43d6-ae98-58f7378e229b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:26.648681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b820586-009c-4b31-941f-fa39fe13162e service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:26.650805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b820586-009c-4b31-941f-fa39fe13162e service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:26.651081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b820586-009c-4b31-941f-fa39fe13162e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:26.667796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf619f5-51a1-43d6-ae98-58f7378e229b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:26.675788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9b3fd-320c-47f5-b8b4-e7314358d2eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c72b8f4ca644ddbeefdfd7aaf74e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ab240549a594f99b632515b1d7ca96d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:26.676593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9b3fd-320c-47f5-b8b4-e7314358d2eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:26.676928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9b3fd-320c-47f5-b8b4-e7314358d2eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:26.677383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9b3fd-320c-47f5-b8b4-e7314358d2eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] PATH_INFO: `/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:26.699117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9b3fd-320c-47f5-b8b4-e7314358d2eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=399ada06bc2a4e73aa157fb4f8f5bc77)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:26.704674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9b3fd-320c-47f5-b8b4-e7314358d2eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:26.723102 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3517/7014] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:18:26 2022] GET /identity/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags => generated 139 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:26.727136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9b3fd-320c-47f5-b8b4-e7314358d2eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:26.730621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:26.746862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9b3fd-320c-47f5-b8b4-e7314358d2eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:26.757136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030bc30-7808-458f-a511-7fc4e36da54b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c72b8f4ca644ddbeefdfd7aaf74e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ab240549a594f99b632515b1d7ca96d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:26.757921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030bc30-7808-458f-a511-7fc4e36da54b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:26.758315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030bc30-7808-458f-a511-7fc4e36da54b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:26.758608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030bc30-7808-458f-a511-7fc4e36da54b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] PATH_INFO: `/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags/tempest-tag-93071207` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:26.760074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030bc30-7808-458f-a511-7fc4e36da54b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorizing `identity:delete_project_tag(project_id=399ada06bc2a4e73aa157fb4f8f5bc77, value=tempest-tag-93071207)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:26.762542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030bc30-7808-458f-a511-7fc4e36da54b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:26.788559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b820586-009c-4b31-941f-fa39fe13162e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:26.791707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b820586-009c-4b31-941f-fa39fe13162e service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:26.815404 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3498/7015] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:26.867927 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-3030bc30-7808-458f-a511-7fc4e36da54b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-93071207. To ensure interoperability, identifiers should be a valid uuid. Oct 15 21:18:26.867927 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 15 21:18:26.867927 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:18:26.875413 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3518/7016] 10.222.0.16 () {58 vars in 1311 bytes} [Sat Oct 15 21:18:26 2022] DELETE /identity/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags/tempest-tag-93071207 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:26.879679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b820586-009c-4b31-941f-fa39fe13162e service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:26.904115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b820586-009c-4b31-941f-fa39fe13162e service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:26.910605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff93be1-12e6-42e3-9f3c-3124955a399a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c72b8f4ca644ddbeefdfd7aaf74e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ab240549a594f99b632515b1d7ca96d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:26.911269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff93be1-12e6-42e3-9f3c-3124955a399a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:26.911697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff93be1-12e6-42e3-9f3c-3124955a399a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:26.911980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff93be1-12e6-42e3-9f3c-3124955a399a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] PATH_INFO: `/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags/tempest-tag-93071207` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:26.928159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff93be1-12e6-42e3-9f3c-3124955a399a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=399ada06bc2a4e73aa157fb4f8f5bc77, value=tempest-tag-93071207)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:26.936342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff93be1-12e6-42e3-9f3c-3124955a399a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-5ff93be1-12e6-42e3-9f3c-3124955a399a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Could not find project tag: tempest-tag-93071207.: keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-93071207. Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 284, in get Oct 15 21:18:26.975087 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_project_tag(project_id, value) Oct 15 21:18:26.976100 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:18:26.976100 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:18:26.976100 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 960, in get_project_tag Oct 15 21:18:26.976100 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectTagNotFound(project_tag=tag_name) Oct 15 21:18:26.976100 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-93071207. Oct 15 21:18:26.976100 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:18:26.979552 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3499/7017] 10.222.0.16 () {58 vars in 1308 bytes} [Sat Oct 15 21:18:26 2022] GET /identity/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags/tempest-tag-93071207 => generated 105 bytes in 101 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 15 21:18:26.985621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030bc30-7808-458f-a511-7fc4e36da54b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:27.002427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030bc30-7808-458f-a511-7fc4e36da54b tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:27.008518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf05ba0-0a48-4cb7-a1c6-95e6c95951eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c72b8f4ca644ddbeefdfd7aaf74e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ab240549a594f99b632515b1d7ca96d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:27.009452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf05ba0-0a48-4cb7-a1c6-95e6c95951eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:27.009803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf05ba0-0a48-4cb7-a1c6-95e6c95951eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:27.010080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf05ba0-0a48-4cb7-a1c6-95e6c95951eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] PATH_INFO: `/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:27.011667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf05ba0-0a48-4cb7-a1c6-95e6c95951eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorizing `identity:delete_project_tags(project_id=399ada06bc2a4e73aa157fb4f8f5bc77)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:27.014477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf05ba0-0a48-4cb7-a1c6-95e6c95951eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:27.134009 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3519/7018] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:18:26 2022] DELETE /identity/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:27.135886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff93be1-12e6-42e3-9f3c-3124955a399a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:27.151613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff93be1-12e6-42e3-9f3c-3124955a399a tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:27.164635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd001805-3115-4ff8-8fc0-f8bc105ae190 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c72b8f4ca644ddbeefdfd7aaf74e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ab240549a594f99b632515b1d7ca96d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:27.165650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd001805-3115-4ff8-8fc0-f8bc105ae190 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:27.166573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd001805-3115-4ff8-8fc0-f8bc105ae190 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:27.166573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd001805-3115-4ff8-8fc0-f8bc105ae190 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] PATH_INFO: `/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:27.189101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd001805-3115-4ff8-8fc0-f8bc105ae190 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=399ada06bc2a4e73aa157fb4f8f5bc77)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:27.195230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd001805-3115-4ff8-8fc0-f8bc105ae190 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:27.222452 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3500/7019] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:18:27 2022] GET /identity/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77/tags => generated 66 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 15 21:18:27.226714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf05ba0-0a48-4cb7-a1c6-95e6c95951eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:27.243872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf05ba0-0a48-4cb7-a1c6-95e6c95951eb tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:27.267889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c72b8f4ca644ddbeefdfd7aaf74e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ab240549a594f99b632515b1d7ca96d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:27.268539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:27.268816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:27.269116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] PATH_INFO: `/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:27.270285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorizing `identity:delete_project(project_id=399ada06bc2a4e73aa157fb4f8f5bc77)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:27.276726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:27.465556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399ada06bc2a4e73aa157fb4f8f5bc77', 'request_id': 'req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:27.490193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399ada06bc2a4e73aa157fb4f8f5bc77', 'request_id': 'req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:27.508151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399ada06bc2a4e73aa157fb4f8f5bc77', 'request_id': 'req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:27.526463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] The token cache is being invalidate because project 399ada06bc2a4e73aa157fb4f8f5bc77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:27.526463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:27.529539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a0b2b0c-ca8f-4f9c-ae70-67a066b4ce60 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:27.535004 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3520/7020] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:27 2022] DELETE /identity/v3/projects/399ada06bc2a4e73aa157fb4f8f5bc77 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:27.542066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd001805-3115-4ff8-8fc0-f8bc105ae190 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:27.544064 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fd001805-3115-4ff8-8fc0-f8bc105ae190 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:27.661191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd001805-3115-4ff8-8fc0-f8bc105ae190 tempest-IdentityV3ProjectTagsTest-1251925113 tempest-IdentityV3ProjectTagsTest-1251925113-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:27.750751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:27.751674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:27.752051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:27.752410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] PATH_INFO: `/v3/users/ec76a4b55c6d46d0bfefc7957f8d48ae` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:27.754352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec76a4b55c6d46d0bfefc7957f8d48ae)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:27.756205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:27.806044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:27.825300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec76a4b55c6d46d0bfefc7957f8d48ae'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:27.849725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec76a4b55c6d46d0bfefc7957f8d48ae'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:27.864333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec76a4b55c6d46d0bfefc7957f8d48ae'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:27.875350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec76a4b55c6d46d0bfefc7957f8d48ae'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:27.876432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec76a4b55c6d46d0bfefc7957f8d48ae'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:27.890995 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3501/7021] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:27 2022] DELETE /identity/v3/users/ec76a4b55c6d46d0bfefc7957f8d48ae => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:27.891959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f68a-b608-4c36-9c0a-d9dd8cb8e303 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:27.892305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f68a-b608-4c36-9c0a-d9dd8cb8e303 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:27.892635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f68a-b608-4c36-9c0a-d9dd8cb8e303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:27.922773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-82d0f68a-b608-4c36-9c0a-d9dd8cb8e303 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:27.927665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-82d0f68a-b608-4c36-9c0a-d9dd8cb8e303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:27.990063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:28.020003 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3521/7022] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:28.037065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:28.038828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:28.121527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6761ea-f314-439b-a793-00e6a86b942f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:28.126332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00314f52-130f-4f3d-a3c0-7f14a4d6287e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:28.126993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314f52-130f-4f3d-a3c0-7f14a4d6287e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:28.127338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314f52-130f-4f3d-a3c0-7f14a4d6287e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:28.128243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314f52-130f-4f3d-a3c0-7f14a4d6287e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:28.141193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00314f52-130f-4f3d-a3c0-7f14a4d6287e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:28.142664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00314f52-130f-4f3d-a3c0-7f14a4d6287e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:28.165254 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3502/7023] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:28.695160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f68a-b608-4c36-9c0a-d9dd8cb8e303 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:28.696649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-82d0f68a-b608-4c36-9c0a-d9dd8cb8e303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:28.716007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f68a-b608-4c36-9c0a-d9dd8cb8e303 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:28.722549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:28.723191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:28.723463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:28.723733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] PATH_INFO: `/v3/projects/bfae4db2e61749b3b7be1f87280b58ef` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:28.724864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfae4db2e61749b3b7be1f87280b58ef)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:28.726803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:28.803814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfae4db2e61749b3b7be1f87280b58ef', 'request_id': 'req-9822d211-8077-4358-b179-90ba72a1a1b3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:28.813674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfae4db2e61749b3b7be1f87280b58ef', 'request_id': 'req-9822d211-8077-4358-b179-90ba72a1a1b3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:28.822163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfae4db2e61749b3b7be1f87280b58ef', 'request_id': 'req-9822d211-8077-4358-b179-90ba72a1a1b3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:28.835409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] The token cache is being invalidate because project bfae4db2e61749b3b7be1f87280b58ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:28.835714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:28.836785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:28.838201 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3522/7024] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:28 2022] DELETE /identity/v3/projects/bfae4db2e61749b3b7be1f87280b58ef => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:28.842267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00314f52-130f-4f3d-a3c0-7f14a4d6287e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:28.853444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00314f52-130f-4f3d-a3c0-7f14a4d6287e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:28.926859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:28.927529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:28.927762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:28.928029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] PATH_INFO: `/v3/users/67c72b8f4ca644ddbeefdfd7aaf74e7b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:28.929236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c72b8f4ca644ddbeefdfd7aaf74e7b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:28.931441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:28.990039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c72b8f4ca644ddbeefdfd7aaf74e7b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:29.001420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:29.008197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c72b8f4ca644ddbeefdfd7aaf74e7b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:29.020001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c72b8f4ca644ddbeefdfd7aaf74e7b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:29.028056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c72b8f4ca644ddbeefdfd7aaf74e7b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:29.028750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a02727b9-64b1-4d40-b62f-9070f1ee4d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c72b8f4ca644ddbeefdfd7aaf74e7b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:29.037591 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3503/7025] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:28 2022] DELETE /identity/v3/users/67c72b8f4ca644ddbeefdfd7aaf74e7b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:29.354116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:29.355612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:29.375477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822d211-8077-4358-b179-90ba72a1a1b3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:29.478117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:29.479208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:29.479830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:29.480375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] PATH_INFO: `/v3/projects/8ab240549a594f99b632515b1d7ca96d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:29.482009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab240549a594f99b632515b1d7ca96d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:29.484761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:29.563612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab240549a594f99b632515b1d7ca96d', 'request_id': 'req-6a41fe3a-64c5-42a0-9130-84d4cf052353'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:29.576687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab240549a594f99b632515b1d7ca96d', 'request_id': 'req-6a41fe3a-64c5-42a0-9130-84d4cf052353'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:29.588365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab240549a594f99b632515b1d7ca96d', 'request_id': 'req-6a41fe3a-64c5-42a0-9130-84d4cf052353'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:29.597623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:29.611771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] The token cache is being invalidate because project 8ab240549a594f99b632515b1d7ca96d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:29.629301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:29.632856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6a41fe3a-64c5-42a0-9130-84d4cf052353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:29.639376 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3523/7026] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:29 2022] DELETE /identity/v3/projects/8ab240549a594f99b632515b1d7ca96d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:29.644859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9951b5-4e9f-4976-8422-a0783f6ca926 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:29.645341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9951b5-4e9f-4976-8422-a0783f6ca926 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:29.645444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9951b5-4e9f-4976-8422-a0783f6ca926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:29.696417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fe9951b5-4e9f-4976-8422-a0783f6ca926 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:29.701660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fe9951b5-4e9f-4976-8422-a0783f6ca926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:29.880326 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3504/7027] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:29.889814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5aeba-3f63-43d6-ae8f-1fcb9c7a0e38 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:29.890599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5aeba-3f63-43d6-ae8f-1fcb9c7a0e38 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:29.891787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5aeba-3f63-43d6-ae8f-1fcb9c7a0e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:29.917529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-aec5aeba-3f63-43d6-ae8f-1fcb9c7a0e38 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:29.919060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aec5aeba-3f63-43d6-ae8f-1fcb9c7a0e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:29.933106 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3524/7028] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:29.938832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9951b5-4e9f-4976-8422-a0783f6ca926 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:29.951640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9951b5-4e9f-4976-8422-a0783f6ca926 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:29.957545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34993e74-edb2-4f4b-9097-576a9e001403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:29.958403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34993e74-edb2-4f4b-9097-576a9e001403 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:29.958832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34993e74-edb2-4f4b-9097-576a9e001403 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:29.958946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34993e74-edb2-4f4b-9097-576a9e001403 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:29.959605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34993e74-edb2-4f4b-9097-576a9e001403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:29.959823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34993e74-edb2-4f4b-9097-576a9e001403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:29.961597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34993e74-edb2-4f4b-9097-576a9e001403 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:29.987442 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3505/7029] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:29.993113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5aeba-3f63-43d6-ae8f-1fcb9c7a0e38 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:30.003680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5aeba-3f63-43d6-ae8f-1fcb9c7a0e38 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:30.009895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df3686-52bc-454f-be39-4926e908dc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:30.010684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3686-52bc-454f-be39-4926e908dc89 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.010854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3686-52bc-454f-be39-4926e908dc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.011134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3686-52bc-454f-be39-4926e908dc89 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.011690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df3686-52bc-454f-be39-4926e908dc89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:30.013801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df3686-52bc-454f-be39-4926e908dc89 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:30.064079 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3525/7030] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:29 2022] POST /identity/v3/projects => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:30.068869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34993e74-edb2-4f4b-9097-576a9e001403 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:30.080225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34993e74-edb2-4f4b-9097-576a9e001403 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:30.154592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81466208-98fe-49c2-8431-0713970dc15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:30.155368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466208-98fe-49c2-8431-0713970dc15f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.156463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466208-98fe-49c2-8431-0713970dc15f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.156831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466208-98fe-49c2-8431-0713970dc15f admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.157627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81466208-98fe-49c2-8431-0713970dc15f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:30.160060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81466208-98fe-49c2-8431-0713970dc15f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:30.189783 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3506/7031] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:30 2022] POST /identity/v3/users => generated 284 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:30.194117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df3686-52bc-454f-be39-4926e908dc89 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:30.205935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df3686-52bc-454f-be39-4926e908dc89 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:30.214746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbfeb3-a64c-4227-a7dd-bdf20da5138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:30.215967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbfeb3-a64c-4227-a7dd-bdf20da5138d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.216249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbfeb3-a64c-4227-a7dd-bdf20da5138d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.216520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbfeb3-a64c-4227-a7dd-bdf20da5138d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.217405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbfeb3-a64c-4227-a7dd-bdf20da5138d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:30.219257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbfeb3-a64c-4227-a7dd-bdf20da5138d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:30.233578 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3526/7032] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:30.236095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81466208-98fe-49c2-8431-0713970dc15f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:30.249193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81466208-98fe-49c2-8431-0713970dc15f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:30.254330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8792f-f401-4733-b8b0-51eba0b97afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:30.254935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8792f-f401-4733-b8b0-51eba0b97afb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.255208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8792f-f401-4733-b8b0-51eba0b97afb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.255610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8792f-f401-4733-b8b0-51eba0b97afb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.256262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8792f-f401-4733-b8b0-51eba0b97afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:30.258257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8792f-f401-4733-b8b0-51eba0b97afb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:30.275403 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3507/7033] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:30.276593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbfeb3-a64c-4227-a7dd-bdf20da5138d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:30.292168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbfeb3-a64c-4227-a7dd-bdf20da5138d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:30.302516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4d4a-b531-4247-b648-e6b68765b6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:30.303194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4d4a-b531-4247-b648-e6b68765b6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.303506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4d4a-b531-4247-b648-e6b68765b6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.303729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4d4a-b531-4247-b648-e6b68765b6e5 admin admin] PATH_INFO: `/v3/projects/076417c05d59400282acfbe8fd1c2911/users/d05528df78734a10ae1170f49d8c0457/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.327176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b4d4a-b531-4247-b648-e6b68765b6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076417c05d59400282acfbe8fd1c2911, user_id=d05528df78734a10ae1170f49d8c0457, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:30.335282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b4d4a-b531-4247-b648-e6b68765b6e5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:30.352319 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3527/7034] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:30 2022] PUT /identity/v3/projects/076417c05d59400282acfbe8fd1c2911/users/d05528df78734a10ae1170f49d8c0457/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:30.365262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b52dad-0293-45eb-b10f-8606790c987b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.365657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b52dad-0293-45eb-b10f-8606790c987b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.366254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b52dad-0293-45eb-b10f-8606790c987b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.401604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a1b52dad-0293-45eb-b10f-8606790c987b None None] MFA Rules not processed for user `d05528df78734a10ae1170f49d8c0457`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:30.403864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a1b52dad-0293-45eb-b10f-8606790c987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:30.443680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:30.504835 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3508/7035] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:30.506817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4d4a-b531-4247-b648-e6b68765b6e5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:30.612208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4d4a-b531-4247-b648-e6b68765b6e5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:30.622487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79576d-1432-47b1-85be-83cc3337e261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:30.625852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79576d-1432-47b1-85be-83cc3337e261 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.626334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79576d-1432-47b1-85be-83cc3337e261 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.627663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79576d-1432-47b1-85be-83cc3337e261 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.628626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79576d-1432-47b1-85be-83cc3337e261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:30.632491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79576d-1432-47b1-85be-83cc3337e261 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:30.688897 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3528/7036] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:30 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:30.693313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b52dad-0293-45eb-b10f-8606790c987b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:30.710012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b52dad-0293-45eb-b10f-8606790c987b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:30.815142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f27ba2-1d50-4390-a229-018b82d7e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:30.817078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f27ba2-1d50-4390-a229-018b82d7e22d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.817466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f27ba2-1d50-4390-a229-018b82d7e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.817836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f27ba2-1d50-4390-a229-018b82d7e22d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.818584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f27ba2-1d50-4390-a229-018b82d7e22d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:30.822790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f27ba2-1d50-4390-a229-018b82d7e22d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:30.854106 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3509/7037] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:30 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:30.857944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79576d-1432-47b1-85be-83cc3337e261 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:30.869112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f27ba2-1d50-4390-a229-018b82d7e22d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:30.870995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79576d-1432-47b1-85be-83cc3337e261 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:30.871318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-22f27ba2-1d50-4390-a229-018b82d7e22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:30.878594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c81af2-5060-4c3d-a0cf-9e0a08e153bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:30.879477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c81af2-5060-4c3d-a0cf-9e0a08e153bf admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.879878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c81af2-5060-4c3d-a0cf-9e0a08e153bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.880245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c81af2-5060-4c3d-a0cf-9e0a08e153bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.881021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c81af2-5060-4c3d-a0cf-9e0a08e153bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:30.883366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c81af2-5060-4c3d-a0cf-9e0a08e153bf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:30.894922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f27ba2-1d50-4390-a229-018b82d7e22d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:30.902801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:30.904088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.904370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.904641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] PATH_INFO: `/v3/users/a41804510e0746c781fa0f73bce0ecf2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.906799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a41804510e0746c781fa0f73bce0ecf2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:30.909482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:30.913739 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3529/7038] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:30.917543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c81af2-5060-4c3d-a0cf-9e0a08e153bf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:30.934116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c81af2-5060-4c3d-a0cf-9e0a08e153bf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:30.949438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabaabf-fab2-44d8-ae5a-8c562042a750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:30.950673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabaabf-fab2-44d8-ae5a-8c562042a750 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:30.951736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabaabf-fab2-44d8-ae5a-8c562042a750 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:30.952096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabaabf-fab2-44d8-ae5a-8c562042a750 admin admin] PATH_INFO: `/v3/projects/2e2d81778fdb4998ab5387602841d19b/users/d79dcf8f9a6540448d85175dc4841c67/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:30.998453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabaabf-fab2-44d8-ae5a-8c562042a750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2d81778fdb4998ab5387602841d19b, user_id=d79dcf8f9a6540448d85175dc4841c67, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:31.002796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabaabf-fab2-44d8-ae5a-8c562042a750 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:31.023317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a41804510e0746c781fa0f73bce0ecf2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:31.040086 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3530/7039] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:30 2022] PUT /identity/v3/projects/2e2d81778fdb4998ab5387602841d19b/users/d79dcf8f9a6540448d85175dc4841c67/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:31.041914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabaabf-fab2-44d8-ae5a-8c562042a750 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:31.060500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a41804510e0746c781fa0f73bce0ecf2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:31.073689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a41804510e0746c781fa0f73bce0ecf2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:31.085452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a41804510e0746c781fa0f73bce0ecf2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:31.086614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c7e6954e-1862-4ec6-ad94-09748cec9cdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a41804510e0746c781fa0f73bce0ecf2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:31.100881 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3510/7040] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:30 2022] DELETE /identity/v3/users/a41804510e0746c781fa0f73bce0ecf2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:31.102326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7742d-efb7-4a4d-b08b-e6672e1bc7be None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:31.102607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7742d-efb7-4a4d-b08b-e6672e1bc7be None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:31.102874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7742d-efb7-4a4d-b08b-e6672e1bc7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:31.112452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:31.147423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-98f7742d-efb7-4a4d-b08b-e6672e1bc7be None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:31.160139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-98f7742d-efb7-4a4d-b08b-e6672e1bc7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:31.178995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabaabf-fab2-44d8-ae5a-8c562042a750 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:31.242011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:31.308619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e7858-d795-42f8-9ab7-d6c2684530b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:31.309274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e7858-d795-42f8-9ab7-d6c2684530b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:31.309539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e7858-d795-42f8-9ab7-d6c2684530b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:31.309979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e7858-d795-42f8-9ab7-d6c2684530b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:31.310633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e7858-d795-42f8-9ab7-d6c2684530b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:31.312316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e7858-d795-42f8-9ab7-d6c2684530b4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:31.345298 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3531/7041] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:31.350310 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3511/7042] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:31.358588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e7858-d795-42f8-9ab7-d6c2684530b4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:31.362310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d02e7858-d795-42f8-9ab7-d6c2684530b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:31.365948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7742d-efb7-4a4d-b08b-e6672e1bc7be None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:31.368381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-98f7742d-efb7-4a4d-b08b-e6672e1bc7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:31.403708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e7858-d795-42f8-9ab7-d6c2684530b4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:31.425146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38141fa-26b4-46c2-97f4-33482cb3f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:31.426000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38141fa-26b4-46c2-97f4-33482cb3f5a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:31.426296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38141fa-26b4-46c2-97f4-33482cb3f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:31.426631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38141fa-26b4-46c2-97f4-33482cb3f5a6 admin admin] PATH_INFO: `/v3/projects/2e2d81778fdb4998ab5387602841d19b/users/d79dcf8f9a6540448d85175dc4841c67/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:31.429952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38141fa-26b4-46c2-97f4-33482cb3f5a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2d81778fdb4998ab5387602841d19b, user_id=d79dcf8f9a6540448d85175dc4841c67, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:31.432531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38141fa-26b4-46c2-97f4-33482cb3f5a6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:31.457743 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3532/7043] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:31 2022] PUT /identity/v3/projects/2e2d81778fdb4998ab5387602841d19b/users/d79dcf8f9a6540448d85175dc4841c67/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:31.469797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cf6e5-ce34-4c15-a55a-f3b7e27f9ebf None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:31.470315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cf6e5-ce34-4c15-a55a-f3b7e27f9ebf None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:31.471015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cf6e5-ce34-4c15-a55a-f3b7e27f9ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:31.519396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3a5cf6e5-ce34-4c15-a55a-f3b7e27f9ebf None None] MFA Rules not processed for user `d79dcf8f9a6540448d85175dc4841c67`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:31.520995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3a5cf6e5-ce34-4c15-a55a-f3b7e27f9ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:31.536914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7742d-efb7-4a4d-b08b-e6672e1bc7be None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:31.704523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcda29c-c35e-4242-a733-86d6f4d96745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:31.706777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda29c-c35e-4242-a733-86d6f4d96745 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:31.707831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda29c-c35e-4242-a733-86d6f4d96745 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:31.708050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda29c-c35e-4242-a733-86d6f4d96745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:31.797983 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3533/7044] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:31.804060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cf6e5-ce34-4c15-a55a-f3b7e27f9ebf None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:31.844123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cf6e5-ce34-4c15-a55a-f3b7e27f9ebf None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:31.868538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fded54e-8824-4e26-a8ec-7c12e9a9f5db tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:31.869199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded54e-8824-4e26-a8ec-7c12e9a9f5db tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:31.869553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded54e-8824-4e26-a8ec-7c12e9a9f5db tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:31.869766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded54e-8824-4e26-a8ec-7c12e9a9f5db tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:31.873917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fded54e-8824-4e26-a8ec-7c12e9a9f5db tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:31.876285 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-1fded54e-8824-4e26-a8ec-7c12e9a9f5db tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:18:31.876285 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:18:31.876285 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:18:31.877815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fded54e-8824-4e26-a8ec-7c12e9a9f5db tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:31.919715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcda29c-c35e-4242-a733-86d6f4d96745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:31.921588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcda29c-c35e-4242-a733-86d6f4d96745 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:31.930798 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3534/7045] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:18:31 2022] POST /identity/v3/regions => generated 224 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:31.932326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fded54e-8824-4e26-a8ec-7c12e9a9f5db tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:31.939292 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3512/7046] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:31.978044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fded54e-8824-4e26-a8ec-7c12e9a9f5db tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:31.992678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49515b-e00f-47ea-8d1e-20de01fc30f1 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:31.993663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49515b-e00f-47ea-8d1e-20de01fc30f1 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:31.994231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49515b-e00f-47ea-8d1e-20de01fc30f1 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:31.994548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49515b-e00f-47ea-8d1e-20de01fc30f1 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:31.995203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49515b-e00f-47ea-8d1e-20de01fc30f1 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:31.996798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49515b-e00f-47ea-8d1e-20de01fc30f1 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.039557 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3535/7047] 10.222.0.16 () {60 vars in 1149 bytes} [Sat Oct 15 21:18:31 2022] POST /identity/v3/regions => generated 224 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:32.040707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcda29c-c35e-4242-a733-86d6f4d96745 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.054937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcda29c-c35e-4242-a733-86d6f4d96745 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.075276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec40a-2475-416d-afc5-3081d054ec5a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.076084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec40a-2475-416d-afc5-3081d054ec5a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.076465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec40a-2475-416d-afc5-3081d054ec5a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.076854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec40a-2475-416d-afc5-3081d054ec5a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions/be610ee4-7594-4428-a7f3-28ad112d657b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.077684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ec40a-2475-416d-afc5-3081d054ec5a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:create_region(region_id=be610ee4-7594-4428-a7f3-28ad112d657b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.079289 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-ed0ec40a-2475-416d-afc5-3081d054ec5a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:18:32.079289 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:18:32.079289 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:18:32.080451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ec40a-2475-416d-afc5-3081d054ec5a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.116215 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3513/7048] 10.222.0.16 () {60 vars in 1259 bytes} [Sat Oct 15 21:18:32 2022] PUT /identity/v3/regions/be610ee4-7594-4428-a7f3-28ad112d657b => generated 232 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:32.120011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49515b-e00f-47ea-8d1e-20de01fc30f1 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.131649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:32.138811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49515b-e00f-47ea-8d1e-20de01fc30f1 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.163643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a5e2a-4ef0-4b02-b633-4e5622c96938 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.167675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a5e2a-4ef0-4b02-b633-4e5622c96938 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.168119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a5e2a-4ef0-4b02-b633-4e5622c96938 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.168392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a5e2a-4ef0-4b02-b633-4e5622c96938 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions/be610ee4-7594-4428-a7f3-28ad112d657b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.169076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a5e2a-4ef0-4b02-b633-4e5622c96938 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:delete_region(region_id=be610ee4-7594-4428-a7f3-28ad112d657b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.174177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a5e2a-4ef0-4b02-b633-4e5622c96938 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.212327 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3536/7049] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:18:32 2022] DELETE /identity/v3/regions/be610ee4-7594-4428-a7f3-28ad112d657b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:32.216071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec40a-2475-416d-afc5-3081d054ec5a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.234088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec40a-2475-416d-afc5-3081d054ec5a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.275953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:32.279450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4047d2-f54f-460e-9e81-90419118fc91 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.280263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4047d2-f54f-460e-9e81-90419118fc91 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.281644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4047d2-f54f-460e-9e81-90419118fc91 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.283168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4047d2-f54f-460e-9e81-90419118fc91 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.284071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4047d2-f54f-460e-9e81-90419118fc91 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.286825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4047d2-f54f-460e-9e81-90419118fc91 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.324929 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3514/7050] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Oct 15 21:18:32 2022] POST /identity/v3/regions => generated 253 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:32.332423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a5e2a-4ef0-4b02-b633-4e5622c96938 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.347604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a5e2a-4ef0-4b02-b633-4e5622c96938 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.371673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa398a4a-81a7-4b7e-8fe0-3aeebf99dcd3 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.372303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa398a4a-81a7-4b7e-8fe0-3aeebf99dcd3 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.373028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa398a4a-81a7-4b7e-8fe0-3aeebf99dcd3 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.373028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa398a4a-81a7-4b7e-8fe0-3aeebf99dcd3 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions/03cc011a17aa4977a668f54d63130234` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.373560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa398a4a-81a7-4b7e-8fe0-3aeebf99dcd3 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:update_region(region_id=03cc011a17aa4977a668f54d63130234)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.374687 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-fa398a4a-81a7-4b7e-8fe0-3aeebf99dcd3 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:18:32.374687 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:18:32.374687 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:18:32.375967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa398a4a-81a7-4b7e-8fe0-3aeebf99dcd3 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.409780 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3537/7051] 10.222.0.16 () {60 vars in 1250 bytes} [Sat Oct 15 21:18:32 2022] PATCH /identity/v3/regions/03cc011a17aa4977a668f54d63130234 => generated 254 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:32.410877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4047d2-f54f-460e-9e81-90419118fc91 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.422426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4047d2-f54f-460e-9e81-90419118fc91 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.443116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa398a4a-81a7-4b7e-8fe0-3aeebf99dcd3 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.446278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa398a4a-81a7-4b7e-8fe0-3aeebf99dcd3 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:32.450409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e93a8a-56f9-4a73-bce5-7dae4df7db05 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.451308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e93a8a-56f9-4a73-bce5-7dae4df7db05 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.453958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e93a8a-56f9-4a73-bce5-7dae4df7db05 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.455019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e93a8a-56f9-4a73-bce5-7dae4df7db05 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions/03cc011a17aa4977a668f54d63130234` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.455730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e93a8a-56f9-4a73-bce5-7dae4df7db05 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:get_region(region_id=03cc011a17aa4977a668f54d63130234)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.458751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e93a8a-56f9-4a73-bce5-7dae4df7db05 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.470143 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3515/7052] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:18:32 2022] GET /identity/v3/regions/03cc011a17aa4977a668f54d63130234 => generated 254 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:32.476134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e93a8a-56f9-4a73-bce5-7dae4df7db05 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.495828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa398a4a-81a7-4b7e-8fe0-3aeebf99dcd3 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.504045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e93a8a-56f9-4a73-bce5-7dae4df7db05 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.508754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.511078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.511459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.513071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] PATH_INFO: `/v3/projects/6b0324b37fb14723b8c4071057f61fb9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.515851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0324b37fb14723b8c4071057f61fb9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.520506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.522932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85d15e-6c34-4dba-aaf0-8f7e56f51e21 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.523609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85d15e-6c34-4dba-aaf0-8f7e56f51e21 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.523887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85d15e-6c34-4dba-aaf0-8f7e56f51e21 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.524174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85d15e-6c34-4dba-aaf0-8f7e56f51e21 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions/03cc011a17aa4977a668f54d63130234` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.524823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85d15e-6c34-4dba-aaf0-8f7e56f51e21 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:delete_region(region_id=03cc011a17aa4977a668f54d63130234)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.526529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85d15e-6c34-4dba-aaf0-8f7e56f51e21 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.567596 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3516/7053] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:18:32 2022] DELETE /identity/v3/regions/03cc011a17aa4977a668f54d63130234 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:32.572466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85d15e-6c34-4dba-aaf0-8f7e56f51e21 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.612915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85d15e-6c34-4dba-aaf0-8f7e56f51e21 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.639725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0324b37fb14723b8c4071057f61fb9', 'request_id': 'req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:32.652385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0324b37fb14723b8c4071057f61fb9', 'request_id': 'req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:32.669838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0324b37fb14723b8c4071057f61fb9', 'request_id': 'req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:32.696582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] The token cache is being invalidate because project 6b0324b37fb14723b8c4071057f61fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:32.697753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:32.698945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-27b7bc7c-aebe-437e-851c-cd1fdd4611d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:32.703276 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3538/7054] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:32 2022] DELETE /identity/v3/projects/6b0324b37fb14723b8c4071057f61fb9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:32.714346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9f1b4-a0c3-46a2-9439-bd33c94eb254 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.714653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9f1b4-a0c3-46a2-9439-bd33c94eb254 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.714930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9f1b4-a0c3-46a2-9439-bd33c94eb254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.726384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b254f46-31d5-4d04-b4f7-5a59419447c4 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.732233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b254f46-31d5-4d04-b4f7-5a59419447c4 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.732754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b254f46-31d5-4d04-b4f7-5a59419447c4 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.733678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b254f46-31d5-4d04-b4f7-5a59419447c4 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.734465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b254f46-31d5-4d04-b4f7-5a59419447c4 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.735735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b254f46-31d5-4d04-b4f7-5a59419447c4 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.748084 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3517/7055] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 21:18:32 2022] GET /identity/v3/regions => generated 670 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:18:32.749476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b254f46-31d5-4d04-b4f7-5a59419447c4 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.751745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4b254f46-31d5-4d04-b4f7-5a59419447c4 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:32.756342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c7c9f1b4-a0c3-46a2-9439-bd33c94eb254 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:32.757587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c7c9f1b4-a0c3-46a2-9439-bd33c94eb254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:32.767311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b254f46-31d5-4d04-b4f7-5a59419447c4 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.773547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030cd30-fc93-4f25-b3bb-377ed82c13e7 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.774887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030cd30-fc93-4f25-b3bb-377ed82c13e7 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.775319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030cd30-fc93-4f25-b3bb-377ed82c13e7 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.775758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030cd30-fc93-4f25-b3bb-377ed82c13e7 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions/03cc011a17aa4977a668f54d63130234` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.776618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030cd30-fc93-4f25-b3bb-377ed82c13e7 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:delete_region(region_id=03cc011a17aa4977a668f54d63130234)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.779271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030cd30-fc93-4f25-b3bb-377ed82c13e7 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-e030cd30-fc93-4f25-b3bb-377ed82c13e7 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Could not find region: 03cc011a17aa4977a668f54d63130234.: keystone.exception.RegionNotFound: Could not find region: 03cc011a17aa4977a668f54d63130234. Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 129, in delete_region Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ret = self.driver.delete_region(region_id) Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 152, in delete_region Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = self._get_region(session, region_id) Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 103, in _get_region Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 03cc011a17aa4977a668f54d63130234. Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:18:32.800601 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/regions.py", line 92, in delete Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return PROVIDERS.catalog_api.delete_region( Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 135, in delete_region Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 03cc011a17aa4977a668f54d63130234. Oct 15 21:18:32.801928 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:18:32.805124 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3518/7056] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:18:32 2022] DELETE /identity/v3/regions/03cc011a17aa4977a668f54d63130234 => generated 112 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:18:32.813504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030cd30-fc93-4f25-b3bb-377ed82c13e7 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.836473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030cd30-fc93-4f25-b3bb-377ed82c13e7 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.852133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1abc8-d2a6-4754-89aa-67c88872bf3a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.852762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1abc8-d2a6-4754-89aa-67c88872bf3a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.853070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1abc8-d2a6-4754-89aa-67c88872bf3a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.853343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1abc8-d2a6-4754-89aa-67c88872bf3a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.853943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1abc8-d2a6-4754-89aa-67c88872bf3a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.855193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1abc8-d2a6-4754-89aa-67c88872bf3a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.862936 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3519/7057] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 21:18:32 2022] GET /identity/v3/regions => generated 670 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:32.868199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1abc8-d2a6-4754-89aa-67c88872bf3a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.896537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1abc8-d2a6-4754-89aa-67c88872bf3a tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:32.905301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8286f2-f89e-4756-82eb-c5861eb8b379 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:32.906565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8286f2-f89e-4756-82eb-c5861eb8b379 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.906849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8286f2-f89e-4756-82eb-c5861eb8b379 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.907116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8286f2-f89e-4756-82eb-c5861eb8b379 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.907909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8286f2-f89e-4756-82eb-c5861eb8b379 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:32.910616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8286f2-f89e-4756-82eb-c5861eb8b379 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:32.940217 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3539/7058] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:32.947764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596ebae-569f-40d1-a27f-cafd2727cd97 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:32.948180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596ebae-569f-40d1-a27f-cafd2727cd97 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:32.948443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596ebae-569f-40d1-a27f-cafd2727cd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:32.962496 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3520/7059] 10.222.0.16 () {60 vars in 1150 bytes} [Sat Oct 15 21:18:32 2022] POST /identity/v3/regions => generated 253 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:32.966767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8286f2-f89e-4756-82eb-c5861eb8b379 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:32.983694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3596ebae-569f-40d1-a27f-cafd2727cd97 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:32.985994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3596ebae-569f-40d1-a27f-cafd2727cd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:33.011974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:33.043909 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3540/7060] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:33.048696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596ebae-569f-40d1-a27f-cafd2727cd97 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:33.051405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8286f2-f89e-4756-82eb-c5861eb8b379 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.068295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78ad56-5e6e-4abb-a786-22c06a92c0d9 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.069993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596ebae-569f-40d1-a27f-cafd2727cd97 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.070617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78ad56-5e6e-4abb-a786-22c06a92c0d9 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.070617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78ad56-5e6e-4abb-a786-22c06a92c0d9 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.070928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78ad56-5e6e-4abb-a786-22c06a92c0d9 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.072026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78ad56-5e6e-4abb-a786-22c06a92c0d9 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Adding query filter params (parent_region_id=3a7c95e99f174efd98b79e7b4c3a3b1c) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:33.072026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78ad56-5e6e-4abb-a786-22c06a92c0d9 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.072887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78ad56-5e6e-4abb-a786-22c06a92c0d9 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.079994 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3521/7061] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 21:18:32 2022] GET /identity/v3/regions?parent_region_id=3a7c95e99f174efd98b79e7b4c3a3b1c => generated 399 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:33.083225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3f309-fc97-4b6f-b800-415f38f592fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.083738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78ad56-5e6e-4abb-a786-22c06a92c0d9 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:33.083883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3f309-fc97-4b6f-b800-415f38f592fa admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.084109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3f309-fc97-4b6f-b800-415f38f592fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.084374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3f309-fc97-4b6f-b800-415f38f592fa admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.085139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3f309-fc97-4b6f-b800-415f38f592fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:33.085441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3f309-fc97-4b6f-b800-415f38f592fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.087654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3f309-fc97-4b6f-b800-415f38f592fa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.099562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78ad56-5e6e-4abb-a786-22c06a92c0d9 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.105872 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3541/7062] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:33.107081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3f309-fc97-4b6f-b800-415f38f592fa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:33.108648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170869d-102d-439d-93c1-99522004b15e tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.110281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170869d-102d-439d-93c1-99522004b15e tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.110649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170869d-102d-439d-93c1-99522004b15e tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.112011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170869d-102d-439d-93c1-99522004b15e tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions/9004aeb162f24391bd6e5ec972648693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.112698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170869d-102d-439d-93c1-99522004b15e tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:delete_region(region_id=9004aeb162f24391bd6e5ec972648693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.116193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170869d-102d-439d-93c1-99522004b15e tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.140378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3f309-fc97-4b6f-b800-415f38f592fa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.151554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c2e4c-883b-4838-b112-c20280899cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.152965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c2e4c-883b-4838-b112-c20280899cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.153348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c2e4c-883b-4838-b112-c20280899cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.153724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c2e4c-883b-4838-b112-c20280899cd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.154525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c2e4c-883b-4838-b112-c20280899cd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.161616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c2e4c-883b-4838-b112-c20280899cd6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.182916 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3522/7063] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:18:33 2022] DELETE /identity/v3/regions/9004aeb162f24391bd6e5ec972648693 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:33.184563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170869d-102d-439d-93c1-99522004b15e tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:33.220033 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3542/7064] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:33 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:33.228884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c2e4c-883b-4838-b112-c20280899cd6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:33.246935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170869d-102d-439d-93c1-99522004b15e tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.263237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:33.366907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446bfbe-524e-46dc-bc9c-32215f0acd36 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.367556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446bfbe-524e-46dc-bc9c-32215f0acd36 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.367856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446bfbe-524e-46dc-bc9c-32215f0acd36 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.368105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446bfbe-524e-46dc-bc9c-32215f0acd36 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions/79159dce107f49a7b1dc8913b4db4c80` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.368691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446bfbe-524e-46dc-bc9c-32215f0acd36 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:delete_region(region_id=79159dce107f49a7b1dc8913b4db4c80)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.370712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446bfbe-524e-46dc-bc9c-32215f0acd36 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.407009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c2e4c-883b-4838-b112-c20280899cd6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.410613 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3523/7065] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:18:33 2022] DELETE /identity/v3/regions/79159dce107f49a7b1dc8913b4db4c80 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:33.414843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446bfbe-524e-46dc-bc9c-32215f0acd36 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:33.448191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcda5d9-7fe0-4419-be13-6bf137677d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.449305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcda5d9-7fe0-4419-be13-6bf137677d41 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.449829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcda5d9-7fe0-4419-be13-6bf137677d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.450324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcda5d9-7fe0-4419-be13-6bf137677d41 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.451340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcda5d9-7fe0-4419-be13-6bf137677d41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.454877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcda5d9-7fe0-4419-be13-6bf137677d41 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.468824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446bfbe-524e-46dc-bc9c-32215f0acd36 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.495570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d8703-4046-46c2-9fca-9cc3334720f8 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd79dcf8f9a6540448d85175dc4841c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e2d81778fdb4998ab5387602841d19b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.496477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d8703-4046-46c2-9fca-9cc3334720f8 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.496756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d8703-4046-46c2-9fca-9cc3334720f8 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.497123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d8703-4046-46c2-9fca-9cc3334720f8 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] PATH_INFO: `/v3/regions/3a7c95e99f174efd98b79e7b4c3a3b1c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.497648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d8703-4046-46c2-9fca-9cc3334720f8 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorizing `identity:delete_region(region_id=3a7c95e99f174efd98b79e7b4c3a3b1c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.504562 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3543/7066] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:33 2022] POST /identity/v3/users => generated 285 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:33.504741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d8703-4046-46c2-9fca-9cc3334720f8 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.506757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcda5d9-7fe0-4419-be13-6bf137677d41 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:33.549921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcda5d9-7fe0-4419-be13-6bf137677d41 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.552609 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3524/7067] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:18:33 2022] DELETE /identity/v3/regions/3a7c95e99f174efd98b79e7b4c3a3b1c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:33.556516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d8703-4046-46c2-9fca-9cc3334720f8 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:33.558332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-da2d8703-4046-46c2-9fca-9cc3334720f8 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:33.582199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e9148-aa54-4402-ac0c-7febe98328be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.583132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e9148-aa54-4402-ac0c-7febe98328be admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.583292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d8703-4046-46c2-9fca-9cc3334720f8 tempest-RegionsTestJSON-118630923 tempest-RegionsTestJSON-118630923-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.583595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e9148-aa54-4402-ac0c-7febe98328be admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.583951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e9148-aa54-4402-ac0c-7febe98328be admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.584714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e9148-aa54-4402-ac0c-7febe98328be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.586657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e9148-aa54-4402-ac0c-7febe98328be admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.599308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.606759 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3544/7068] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:33.606958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.606958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.606958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] PATH_INFO: `/v3/users/d05528df78734a10ae1170f49d8c0457` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.615329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d05528df78734a10ae1170f49d8c0457)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.616656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e9148-aa54-4402-ac0c-7febe98328be admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:33.619203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.638418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e9148-aa54-4402-ac0c-7febe98328be admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.657610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e257d-3b73-4774-ad4b-0c70cc1ce36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.658403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e257d-3b73-4774-ad4b-0c70cc1ce36f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.659631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e257d-3b73-4774-ad4b-0c70cc1ce36f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.660541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e257d-3b73-4774-ad4b-0c70cc1ce36f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.661354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e257d-3b73-4774-ad4b-0c70cc1ce36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.663609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e257d-3b73-4774-ad4b-0c70cc1ce36f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.680031 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3545/7069] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:33.684578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e257d-3b73-4774-ad4b-0c70cc1ce36f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:33.696910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e257d-3b73-4774-ad4b-0c70cc1ce36f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:33.703349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b9cd0-4cd3-4a5a-b5e6-5d0ed5c796f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:33.704320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b9cd0-4cd3-4a5a-b5e6-5d0ed5c796f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.704660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b9cd0-4cd3-4a5a-b5e6-5d0ed5c796f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.705109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b9cd0-4cd3-4a5a-b5e6-5d0ed5c796f4 admin admin] PATH_INFO: `/v3/projects/0f4379801b4040c8b857afe4038e4ff7/users/1ae48516992c42578ec24ab334785ad1/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.727728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b9cd0-4cd3-4a5a-b5e6-5d0ed5c796f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4379801b4040c8b857afe4038e4ff7, user_id=1ae48516992c42578ec24ab334785ad1, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:33.730348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b9cd0-4cd3-4a5a-b5e6-5d0ed5c796f4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:33.752460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd05528df78734a10ae1170f49d8c0457'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:33.761316 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3546/7070] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:33 2022] PUT /identity/v3/projects/0f4379801b4040c8b857afe4038e4ff7/users/1ae48516992c42578ec24ab334785ad1/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:33.768443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba66ebc-a49e-45ab-bd53-eaa2f82f1575 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.768738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba66ebc-a49e-45ab-bd53-eaa2f82f1575 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.769010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba66ebc-a49e-45ab-bd53-eaa2f82f1575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.789809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd05528df78734a10ae1170f49d8c0457'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:33.807793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd05528df78734a10ae1170f49d8c0457'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:33.811003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8ba66ebc-a49e-45ab-bd53-eaa2f82f1575 None None] MFA Rules not processed for user `1ae48516992c42578ec24ab334785ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:33.814013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8ba66ebc-a49e-45ab-bd53-eaa2f82f1575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:33.829429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd05528df78734a10ae1170f49d8c0457'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:33.831194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c864f2b0-96ec-4d30-b0dc-e20420eb7095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd05528df78734a10ae1170f49d8c0457'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:33.844200 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3525/7071] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:33 2022] DELETE /identity/v3/users/d05528df78734a10ae1170f49d8c0457 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:33.849938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01189052-60db-4a6a-83fb-6f6d27a31492 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:33.850409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01189052-60db-4a6a-83fb-6f6d27a31492 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:33.850818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01189052-60db-4a6a-83fb-6f6d27a31492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:33.892265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-01189052-60db-4a6a-83fb-6f6d27a31492 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:33.893580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-01189052-60db-4a6a-83fb-6f6d27a31492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:33.909728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:34.014241 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3547/7072] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:34.017774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:34.028274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba66ebc-a49e-45ab-bd53-eaa2f82f1575 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:34.029940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8ba66ebc-a49e-45ab-bd53-eaa2f82f1575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:34.100735 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3526/7073] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:34.132822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01189052-60db-4a6a-83fb-6f6d27a31492 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:34.134491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-01189052-60db-4a6a-83fb-6f6d27a31492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:34.185374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba66ebc-a49e-45ab-bd53-eaa2f82f1575 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:34.332033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366229a-0aba-4305-8b22-d82d775f857a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:34.339595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366229a-0aba-4305-8b22-d82d775f857a service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:34.340509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366229a-0aba-4305-8b22-d82d775f857a service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:34.341388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366229a-0aba-4305-8b22-d82d775f857a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:34.381268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01189052-60db-4a6a-83fb-6f6d27a31492 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:34.396188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0af544-81b3-4fc7-baae-3bab61a03a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:34.396819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0af544-81b3-4fc7-baae-3bab61a03a5f service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:34.397604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0af544-81b3-4fc7-baae-3bab61a03a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:34.397880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0af544-81b3-4fc7-baae-3bab61a03a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:34.425899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0af544-81b3-4fc7-baae-3bab61a03a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:34.427355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0af544-81b3-4fc7-baae-3bab61a03a5f service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:34.457001 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3527/7074] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:34.552749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366229a-0aba-4305-8b22-d82d775f857a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:34.556357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366229a-0aba-4305-8b22-d82d775f857a service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:34.604925 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3548/7075] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:18:35.692356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0af544-81b3-4fc7-baae-3bab61a03a5f service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:35.694493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4b0af544-81b3-4fc7-baae-3bab61a03a5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:35.729954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0af544-81b3-4fc7-baae-3bab61a03a5f service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:35.739976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:35.742327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:35.743085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:35.743085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] PATH_INFO: `/v3/projects/076417c05d59400282acfbe8fd1c2911` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:35.744699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=076417c05d59400282acfbe8fd1c2911)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:35.747527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:35.835545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076417c05d59400282acfbe8fd1c2911', 'request_id': 'req-bb3a72d2-b81d-446f-a43a-2981672608c3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:35.851001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076417c05d59400282acfbe8fd1c2911', 'request_id': 'req-bb3a72d2-b81d-446f-a43a-2981672608c3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:35.874820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076417c05d59400282acfbe8fd1c2911', 'request_id': 'req-bb3a72d2-b81d-446f-a43a-2981672608c3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:35.906436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] The token cache is being invalidate because project 076417c05d59400282acfbe8fd1c2911 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:35.907007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:35.907780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:35.912373 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3528/7076] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:35 2022] DELETE /identity/v3/projects/076417c05d59400282acfbe8fd1c2911 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:35.913714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366229a-0aba-4305-8b22-d82d775f857a service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:35.929707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366229a-0aba-4305-8b22-d82d775f857a service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:36.002902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:36.042773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:36.043637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:36.043917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:36.044182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] PATH_INFO: `/v3/users/d79dcf8f9a6540448d85175dc4841c67` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:36.046617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79dcf8f9a6540448d85175dc4841c67)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:36.049401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:36.132426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79dcf8f9a6540448d85175dc4841c67'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:36.142811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79dcf8f9a6540448d85175dc4841c67'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:36.162933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79dcf8f9a6540448d85175dc4841c67'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:36.177070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79dcf8f9a6540448d85175dc4841c67'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:36.177070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7fef2d2f-650e-4f88-8ad0-512738e9c334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79dcf8f9a6540448d85175dc4841c67'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:36.185550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3549/7077] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:35 2022] DELETE /identity/v3/users/d79dcf8f9a6540448d85175dc4841c67 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:36.541579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:36.543015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:36.568403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a72d2-b81d-446f-a43a-2981672608c3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:36.669980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:36.685890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:36.686822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:36.687239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:36.687931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] PATH_INFO: `/v3/projects/2e2d81778fdb4998ab5387602841d19b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:36.689920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2d81778fdb4998ab5387602841d19b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:36.692367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:36.793750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2d81778fdb4998ab5387602841d19b', 'request_id': 'req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:36.812071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2d81778fdb4998ab5387602841d19b', 'request_id': 'req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:36.821328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2d81778fdb4998ab5387602841d19b', 'request_id': 'req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:36.846018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] The token cache is being invalidate because project 2e2d81778fdb4998ab5387602841d19b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:36.846374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:36.849447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3675dc58-8c6e-484f-80eb-0d5022e9a6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:36.852317 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3529/7078] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:36 2022] DELETE /identity/v3/projects/2e2d81778fdb4998ab5387602841d19b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:36.856928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7830a9-5df3-4ca2-a7e5-42b08cb9fb0a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:36.857242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7830a9-5df3-4ca2-a7e5-42b08cb9fb0a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:36.857512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7830a9-5df3-4ca2-a7e5-42b08cb9fb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:36.885596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-df7830a9-5df3-4ca2-a7e5-42b08cb9fb0a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:36.887238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-df7830a9-5df3-4ca2-a7e5-42b08cb9fb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:36.990779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3980686-1112-4d4e-b180-1d3d30aa8702 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:36.991075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3980686-1112-4d4e-b180-1d3d30aa8702 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:36.991479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3980686-1112-4d4e-b180-1d3d30aa8702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:36.994031 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3550/7079] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:37.014737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c3980686-1112-4d4e-b180-1d3d30aa8702 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:37.015656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c3980686-1112-4d4e-b180-1d3d30aa8702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:37.029360 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3530/7080] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:37.035550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7830a9-5df3-4ca2-a7e5-42b08cb9fb0a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:37.051309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7830a9-5df3-4ca2-a7e5-42b08cb9fb0a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:37.069230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e020a92-b8e7-4a2a-82b7-abb6254fac1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:37.071353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020a92-b8e7-4a2a-82b7-abb6254fac1e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:37.072221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020a92-b8e7-4a2a-82b7-abb6254fac1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:37.073069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020a92-b8e7-4a2a-82b7-abb6254fac1e admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:37.074322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e020a92-b8e7-4a2a-82b7-abb6254fac1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:37.076566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e020a92-b8e7-4a2a-82b7-abb6254fac1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:37.080914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e020a92-b8e7-4a2a-82b7-abb6254fac1e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:37.121084 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3551/7081] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:37.124759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3980686-1112-4d4e-b180-1d3d30aa8702 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:37.141627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3980686-1112-4d4e-b180-1d3d30aa8702 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:37.149269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dd823-8592-400d-8984-c9b5e77f0976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:37.150042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dd823-8592-400d-8984-c9b5e77f0976 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:37.150322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dd823-8592-400d-8984-c9b5e77f0976 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:37.150589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dd823-8592-400d-8984-c9b5e77f0976 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:37.151227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dd823-8592-400d-8984-c9b5e77f0976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:37.156807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dd823-8592-400d-8984-c9b5e77f0976 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:37.200673 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3531/7082] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:37 2022] POST /identity/v3/projects => generated 369 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:37.205205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e020a92-b8e7-4a2a-82b7-abb6254fac1e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:37.219944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e020a92-b8e7-4a2a-82b7-abb6254fac1e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:37.312448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:37.349896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7684e-b9a1-4f53-9512-04e5c4b4830b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:37.352425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7684e-b9a1-4f53-9512-04e5c4b4830b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:37.353124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7684e-b9a1-4f53-9512-04e5c4b4830b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:37.353858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7684e-b9a1-4f53-9512-04e5c4b4830b admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:37.357854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7684e-b9a1-4f53-9512-04e5c4b4830b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:37.363659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7684e-b9a1-4f53-9512-04e5c4b4830b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:37.401330 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3552/7083] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:37 2022] POST /identity/v3/users => generated 285 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:37.406297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dd823-8592-400d-8984-c9b5e77f0976 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:37.420418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dd823-8592-400d-8984-c9b5e77f0976 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:37.426052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b66c7-6aec-4e98-8f12-29c6ccecb2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:37.426935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b66c7-6aec-4e98-8f12-29c6ccecb2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:37.427286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b66c7-6aec-4e98-8f12-29c6ccecb2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:37.427633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b66c7-6aec-4e98-8f12-29c6ccecb2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:37.428613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b66c7-6aec-4e98-8f12-29c6ccecb2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:37.431109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b66c7-6aec-4e98-8f12-29c6ccecb2c2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:37.447687 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3532/7084] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:37.451546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7684e-b9a1-4f53-9512-04e5c4b4830b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:37.462602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7684e-b9a1-4f53-9512-04e5c4b4830b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:37.471763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedbc55-f421-408a-9db0-b22c0ee33479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:37.472858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedbc55-f421-408a-9db0-b22c0ee33479 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:37.473401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedbc55-f421-408a-9db0-b22c0ee33479 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:37.473623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedbc55-f421-408a-9db0-b22c0ee33479 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:37.474616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedbc55-f421-408a-9db0-b22c0ee33479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:37.476409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedbc55-f421-408a-9db0-b22c0ee33479 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:37.495367 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3553/7085] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:37.507943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b66c7-6aec-4e98-8f12-29c6ccecb2c2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:37.539190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b66c7-6aec-4e98-8f12-29c6ccecb2c2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:37.545122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9117067-0e81-4907-ac5e-dc93a4c2f563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:37.545909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9117067-0e81-4907-ac5e-dc93a4c2f563 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:37.546109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9117067-0e81-4907-ac5e-dc93a4c2f563 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:37.546620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9117067-0e81-4907-ac5e-dc93a4c2f563 admin admin] PATH_INFO: `/v3/projects/72f63c78e0e24c23a4060c561c0130bd/users/69697d06fb4f4332a16df7cb41a61134/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:37.569531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9117067-0e81-4907-ac5e-dc93a4c2f563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f63c78e0e24c23a4060c561c0130bd, user_id=69697d06fb4f4332a16df7cb41a61134, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:37.572514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9117067-0e81-4907-ac5e-dc93a4c2f563 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:37.613080 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3533/7086] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:37 2022] PUT /identity/v3/projects/72f63c78e0e24c23a4060c561c0130bd/users/69697d06fb4f4332a16df7cb41a61134/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:37.617896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadbc5d-79b0-4e6b-b033-fd9fbfeb3f6c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:37.618634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadbc5d-79b0-4e6b-b033-fd9fbfeb3f6c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:37.619115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadbc5d-79b0-4e6b-b033-fd9fbfeb3f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:37.669723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-daadbc5d-79b0-4e6b-b033-fd9fbfeb3f6c None None] MFA Rules not processed for user `69697d06fb4f4332a16df7cb41a61134`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:37.671487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-daadbc5d-79b0-4e6b-b033-fd9fbfeb3f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:37.823931 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3554/7087] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:37.836552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9117067-0e81-4907-ac5e-dc93a4c2f563 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:38.013401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9117067-0e81-4907-ac5e-dc93a4c2f563 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:38.025643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65659b-299d-4cdf-a061-2bbf81d2e2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:38.026519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65659b-299d-4cdf-a061-2bbf81d2e2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.026958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65659b-299d-4cdf-a061-2bbf81d2e2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.027087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65659b-299d-4cdf-a061-2bbf81d2e2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.027695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65659b-299d-4cdf-a061-2bbf81d2e2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:38.030448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65659b-299d-4cdf-a061-2bbf81d2e2e2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:38.077955 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3534/7088] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:37 2022] POST /identity/v3/projects => generated 369 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:38.084770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadbc5d-79b0-4e6b-b033-fd9fbfeb3f6c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:38.102443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadbc5d-79b0-4e6b-b033-fd9fbfeb3f6c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:38.194904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf4833-a151-45e8-a4f7-cdc909962390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:38.195702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf4833-a151-45e8-a4f7-cdc909962390 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.196049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf4833-a151-45e8-a4f7-cdc909962390 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.196338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf4833-a151-45e8-a4f7-cdc909962390 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.197125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf4833-a151-45e8-a4f7-cdc909962390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:38.201463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf4833-a151-45e8-a4f7-cdc909962390 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:38.237084 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3555/7089] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:38 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:38.240125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65659b-299d-4cdf-a061-2bbf81d2e2e2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:38.251041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:38.255521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65659b-299d-4cdf-a061-2bbf81d2e2e2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:38.263757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b28b39-2aa4-4dbb-a29e-70a3740c3d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:38.264455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b28b39-2aa4-4dbb-a29e-70a3740c3d13 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.264656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b28b39-2aa4-4dbb-a29e-70a3740c3d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.264865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b28b39-2aa4-4dbb-a29e-70a3740c3d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.265871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b28b39-2aa4-4dbb-a29e-70a3740c3d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:38.267863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b28b39-2aa4-4dbb-a29e-70a3740c3d13 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:38.282718 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3535/7090] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:38.287833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf4833-a151-45e8-a4f7-cdc909962390 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:38.299931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf4833-a151-45e8-a4f7-cdc909962390 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:38.305688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab3886-838e-4cf7-8eca-576c3f36558a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:38.306418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab3886-838e-4cf7-8eca-576c3f36558a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.306877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab3886-838e-4cf7-8eca-576c3f36558a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.307038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab3886-838e-4cf7-8eca-576c3f36558a admin admin] PATH_INFO: `/v3/projects/658d1ed0fea948f2b94cf941ff034726/users/988845ff0ba849c4ad1432339c890ccf/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.330460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab3886-838e-4cf7-8eca-576c3f36558a admin admin] RBAC: Authorizing `identity:create_grant(project_id=658d1ed0fea948f2b94cf941ff034726, user_id=988845ff0ba849c4ad1432339c890ccf, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:38.334090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab3886-838e-4cf7-8eca-576c3f36558a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:38.348897 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3556/7091] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:38 2022] PUT /identity/v3/projects/658d1ed0fea948f2b94cf941ff034726/users/988845ff0ba849c4ad1432339c890ccf/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:38.353023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b28b39-2aa4-4dbb-a29e-70a3740c3d13 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:38.364361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b28b39-2aa4-4dbb-a29e-70a3740c3d13 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:38.438039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64993c8d-667d-4020-8645-f9a8787ea1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:38.439200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64993c8d-667d-4020-8645-f9a8787ea1ff admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.439564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64993c8d-667d-4020-8645-f9a8787ea1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.439910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64993c8d-667d-4020-8645-f9a8787ea1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.441152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64993c8d-667d-4020-8645-f9a8787ea1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:38.443216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64993c8d-667d-4020-8645-f9a8787ea1ff admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:38.458519 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3536/7092] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:38.459925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab3886-838e-4cf7-8eca-576c3f36558a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:38.472629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab3886-838e-4cf7-8eca-576c3f36558a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:38.477724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf4085-bff6-4b6a-9883-2c7b93a10fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:38.478649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf4085-bff6-4b6a-9883-2c7b93a10fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.478921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf4085-bff6-4b6a-9883-2c7b93a10fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.479185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf4085-bff6-4b6a-9883-2c7b93a10fa6 admin admin] PATH_INFO: `/v3/projects/658d1ed0fea948f2b94cf941ff034726/users/988845ff0ba849c4ad1432339c890ccf/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.481185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf4085-bff6-4b6a-9883-2c7b93a10fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658d1ed0fea948f2b94cf941ff034726, user_id=988845ff0ba849c4ad1432339c890ccf, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:38.484683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf4085-bff6-4b6a-9883-2c7b93a10fa6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:38.497022 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3557/7093] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:38 2022] PUT /identity/v3/projects/658d1ed0fea948f2b94cf941ff034726/users/988845ff0ba849c4ad1432339c890ccf/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:38.509029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e27d1f-64a0-426c-bce0-02782e1258e0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.509348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e27d1f-64a0-426c-bce0-02782e1258e0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.509619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e27d1f-64a0-426c-bce0-02782e1258e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.542645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-37e27d1f-64a0-426c-bce0-02782e1258e0 None None] MFA Rules not processed for user `988845ff0ba849c4ad1432339c890ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:38.543869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-37e27d1f-64a0-426c-bce0-02782e1258e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:38.656341 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3537/7094] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:38.660342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf4085-bff6-4b6a-9883-2c7b93a10fa6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:38.668764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:38.687049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf4085-bff6-4b6a-9883-2c7b93a10fa6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:38.694801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651b4cc-2166-4e74-8ae4-16d8c2a3f22d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:38.695766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651b4cc-2166-4e74-8ae4-16d8c2a3f22d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.696469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651b4cc-2166-4e74-8ae4-16d8c2a3f22d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.698091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651b4cc-2166-4e74-8ae4-16d8c2a3f22d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.700020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651b4cc-2166-4e74-8ae4-16d8c2a3f22d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:38.709024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651b4cc-2166-4e74-8ae4-16d8c2a3f22d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:38.737473 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3558/7095] 10.222.0.16 () {60 vars in 1152 bytes} [Sat Oct 15 21:18:38 2022] POST /identity/v3/services => generated 240 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:38.741609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e27d1f-64a0-426c-bce0-02782e1258e0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:38.759116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e27d1f-64a0-426c-bce0-02782e1258e0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:38.787101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b659a-bef5-4507-8fcf-be1061e8521f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:38.787767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b659a-bef5-4507-8fcf-be1061e8521f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.788823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b659a-bef5-4507-8fcf-be1061e8521f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.788936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b659a-bef5-4507-8fcf-be1061e8521f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services/97ec41105a4e4d4889c4b04574301322` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.789582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b659a-bef5-4507-8fcf-be1061e8521f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:delete_service(service_id=97ec41105a4e4d4889c4b04574301322)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:38.791252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b659a-bef5-4507-8fcf-be1061e8521f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:38.841147 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3538/7096] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:38 2022] DELETE /identity/v3/services/97ec41105a4e4d4889c4b04574301322 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:38.846553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651b4cc-2166-4e74-8ae4-16d8c2a3f22d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:38.864304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651b4cc-2166-4e74-8ae4-16d8c2a3f22d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:38.910082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855ffdd-c447-4faa-b466-81194ab6592d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:38.910681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ffdd-c447-4faa-b466-81194ab6592d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.910973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ffdd-c447-4faa-b466-81194ab6592d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.911288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ffdd-c447-4faa-b466-81194ab6592d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.911934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855ffdd-c447-4faa-b466-81194ab6592d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:38.913590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855ffdd-c447-4faa-b466-81194ab6592d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:38.932691 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3559/7097] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:38 2022] POST /identity/v3/services => generated 288 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:38.936994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b659a-bef5-4507-8fcf-be1061e8521f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:38.949524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b659a-bef5-4507-8fcf-be1061e8521f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:38.972772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557bef6-68ea-4c4b-b6e4-5ae65275034f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:38.973485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557bef6-68ea-4c4b-b6e4-5ae65275034f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:38.973940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557bef6-68ea-4c4b-b6e4-5ae65275034f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:38.974274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557bef6-68ea-4c4b-b6e4-5ae65275034f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services/2afd66a298d64ce9913bf8a2a7bc9a77` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:38.975241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557bef6-68ea-4c4b-b6e4-5ae65275034f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:update_service(service_id=2afd66a298d64ce9913bf8a2a7bc9a77)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:38.976514 ncdv-master-1 devstack@keystone.service[99100]: WARNING py.warnings [None req-3557bef6-68ea-4c4b-b6e4-5ae65275034f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:18:38.976514 ncdv-master-1 devstack@keystone.service[99100]: warnings.warn(msg) Oct 15 21:18:38.976514 ncdv-master-1 devstack@keystone.service[99100]: Oct 15 21:18:38.978042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557bef6-68ea-4c4b-b6e4-5ae65275034f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.014277 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3539/7098] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Oct 15 21:18:38 2022] PATCH /identity/v3/services/2afd66a298d64ce9913bf8a2a7bc9a77 => generated 282 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:39.018681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855ffdd-c447-4faa-b466-81194ab6592d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.037909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855ffdd-c447-4faa-b466-81194ab6592d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.062453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fb604-3327-4d68-98c1-133ab40cab18 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.063367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fb604-3327-4d68-98c1-133ab40cab18 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.063724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fb604-3327-4d68-98c1-133ab40cab18 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.064092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fb604-3327-4d68-98c1-133ab40cab18 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services/2afd66a298d64ce9913bf8a2a7bc9a77` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.064794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fb604-3327-4d68-98c1-133ab40cab18 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:get_service(service_id=2afd66a298d64ce9913bf8a2a7bc9a77)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.065944 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-0d9fb604-3327-4d68-98c1-133ab40cab18 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_service": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:18:39.065944 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:18:39.065944 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:18:39.066972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fb604-3327-4d68-98c1-133ab40cab18 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.074207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3560/7099] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:18:39 2022] GET /identity/v3/services/2afd66a298d64ce9913bf8a2a7bc9a77 => generated 282 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:39.078188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557bef6-68ea-4c4b-b6e4-5ae65275034f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.090360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557bef6-68ea-4c4b-b6e4-5ae65275034f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.099332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c12553-38bf-4685-8b93-219908a20bd1 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.100286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c12553-38bf-4685-8b93-219908a20bd1 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.100765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c12553-38bf-4685-8b93-219908a20bd1 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.101515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c12553-38bf-4685-8b93-219908a20bd1 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services/2afd66a298d64ce9913bf8a2a7bc9a77` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.102129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c12553-38bf-4685-8b93-219908a20bd1 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:delete_service(service_id=2afd66a298d64ce9913bf8a2a7bc9a77)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.103774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c12553-38bf-4685-8b93-219908a20bd1 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.120360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:39.146242 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3540/7100] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:39 2022] DELETE /identity/v3/services/2afd66a298d64ce9913bf8a2a7bc9a77 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:39.150343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fb604-3327-4d68-98c1-133ab40cab18 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.168003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fb604-3327-4d68-98c1-133ab40cab18 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.194787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cef26-77f2-418b-ad90-b6c30d6ec5aa tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.195795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cef26-77f2-418b-ad90-b6c30d6ec5aa tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.196296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cef26-77f2-418b-ad90-b6c30d6ec5aa tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.196782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cef26-77f2-418b-ad90-b6c30d6ec5aa tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services/2afd66a298d64ce9913bf8a2a7bc9a77` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.201876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cef26-77f2-418b-ad90-b6c30d6ec5aa tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:get_service(service_id=2afd66a298d64ce9913bf8a2a7bc9a77)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.205515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cef26-77f2-418b-ad90-b6c30d6ec5aa tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application [None req-971cef26-77f2-418b-ad90-b6c30d6ec5aa tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Could not find service: 2afd66a298d64ce9913bf8a2a7bc9a77.: keystone.exception.ServiceNotFound: Could not find service: 2afd66a298d64ce9913bf8a2a7bc9a77. Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 2afd66a298d64ce9913bf8a2a7bc9a77. Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:18:39.227813 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:18:39.228931 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application with Lock( Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 2afd66a298d64ce9913bf8a2a7bc9a77. Oct 15 21:18:39.230433 ncdv-master-1 devstack@keystone.service[99099]: ERROR keystone.server.flask.application Oct 15 21:18:39.236353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c12553-38bf-4685-8b93-219908a20bd1 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.237524 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3561/7101] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 21:18:39 2022] GET /identity/v3/services/2afd66a298d64ce9913bf8a2a7bc9a77 => generated 113 bytes in 88 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:18:39.251714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c12553-38bf-4685-8b93-219908a20bd1 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.258204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cc4c1-25e3-4093-8926-cd0683d2f8ef tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.258913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cc4c1-25e3-4093-8926-cd0683d2f8ef tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.259189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cc4c1-25e3-4093-8926-cd0683d2f8ef tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.259455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cc4c1-25e3-4093-8926-cd0683d2f8ef tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.260568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cc4c1-25e3-4093-8926-cd0683d2f8ef tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.263335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cc4c1-25e3-4093-8926-cd0683d2f8ef tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.289078 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3541/7102] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:39 2022] POST /identity/v3/services => generated 273 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:39.292880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cef26-77f2-418b-ad90-b6c30d6ec5aa tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.305820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cef26-77f2-418b-ad90-b6c30d6ec5aa tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.324949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c90734-cb12-428d-92aa-d40918b23b0f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.326039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c90734-cb12-428d-92aa-d40918b23b0f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.326590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c90734-cb12-428d-92aa-d40918b23b0f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.327526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c90734-cb12-428d-92aa-d40918b23b0f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.328592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c90734-cb12-428d-92aa-d40918b23b0f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.330432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c90734-cb12-428d-92aa-d40918b23b0f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.354446 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3562/7103] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:39 2022] POST /identity/v3/services => generated 273 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:39.356248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cc4c1-25e3-4093-8926-cd0683d2f8ef tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.378946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cc4c1-25e3-4093-8926-cd0683d2f8ef tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.397351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109e16e-f08e-4c75-b823-e710d553c9f4 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.398223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109e16e-f08e-4c75-b823-e710d553c9f4 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.398562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109e16e-f08e-4c75-b823-e710d553c9f4 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.398917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109e16e-f08e-4c75-b823-e710d553c9f4 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.399671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109e16e-f08e-4c75-b823-e710d553c9f4 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.401600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109e16e-f08e-4c75-b823-e710d553c9f4 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.428655 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3542/7104] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:39 2022] POST /identity/v3/services => generated 274 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:39.432560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c90734-cb12-428d-92aa-d40918b23b0f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.444328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c90734-cb12-428d-92aa-d40918b23b0f tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.466773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f87102-d73d-47c4-af2b-aaa8e4b0b495 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.467752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f87102-d73d-47c4-af2b-aaa8e4b0b495 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.468100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f87102-d73d-47c4-af2b-aaa8e4b0b495 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.468436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f87102-d73d-47c4-af2b-aaa8e4b0b495 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.469137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f87102-d73d-47c4-af2b-aaa8e4b0b495 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.471104 ncdv-master-1 devstack@keystone.service[99099]: WARNING py.warnings [None req-e6f87102-d73d-47c4-af2b-aaa8e4b0b495 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 21:18:39.471104 ncdv-master-1 devstack@keystone.service[99099]: warnings.warn(msg) Oct 15 21:18:39.471104 ncdv-master-1 devstack@keystone.service[99099]: Oct 15 21:18:39.471994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f87102-d73d-47c4-af2b-aaa8e4b0b495 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.477109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109e16e-f08e-4c75-b823-e710d553c9f4 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.479097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f109e16e-f08e-4c75-b823-e710d553c9f4 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:39.486279 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3563/7105] 10.222.0.16 () {58 vars in 1131 bytes} [Sat Oct 15 21:18:39 2022] GET /identity/v3/services => generated 2975 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:39.489908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f87102-d73d-47c4-af2b-aaa8e4b0b495 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.505430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f87102-d73d-47c4-af2b-aaa8e4b0b495 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.513996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03b9dc-418b-496e-aaf4-36c47c0cc6f2 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.515139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03b9dc-418b-496e-aaf4-36c47c0cc6f2 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.515718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03b9dc-418b-496e-aaf4-36c47c0cc6f2 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.516576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03b9dc-418b-496e-aaf4-36c47c0cc6f2 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.517483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03b9dc-418b-496e-aaf4-36c47c0cc6f2 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-882715965) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:39.517947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03b9dc-418b-496e-aaf4-36c47c0cc6f2 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.520074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03b9dc-418b-496e-aaf4-36c47c0cc6f2 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.527817 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3564/7106] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Oct 15 21:18:39 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-882715965 => generated 415 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:18:39.532322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03b9dc-418b-496e-aaf4-36c47c0cc6f2 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.551221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03b9dc-418b-496e-aaf4-36c47c0cc6f2 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.557990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e058e7-96b8-4be5-b2a9-c1f373e25a01 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.558802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e058e7-96b8-4be5-b2a9-c1f373e25a01 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.559029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e058e7-96b8-4be5-b2a9-c1f373e25a01 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.559374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e058e7-96b8-4be5-b2a9-c1f373e25a01 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.560133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e058e7-96b8-4be5-b2a9-c1f373e25a01 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-448907159) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:39.560644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e058e7-96b8-4be5-b2a9-c1f373e25a01 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.563806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e058e7-96b8-4be5-b2a9-c1f373e25a01 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.570991 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3565/7107] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Oct 15 21:18:39 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-448907159 => generated 415 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:39.574512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e058e7-96b8-4be5-b2a9-c1f373e25a01 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.591653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e058e7-96b8-4be5-b2a9-c1f373e25a01 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.598890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cd58a-9c14-4aff-ba1a-0a9b7de06f08 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.599496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cd58a-9c14-4aff-ba1a-0a9b7de06f08 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.599771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cd58a-9c14-4aff-ba1a-0a9b7de06f08 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.600686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cd58a-9c14-4aff-ba1a-0a9b7de06f08 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.601372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cd58a-9c14-4aff-ba1a-0a9b7de06f08 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1327146833) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:39.601679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cd58a-9c14-4aff-ba1a-0a9b7de06f08 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.603254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cd58a-9c14-4aff-ba1a-0a9b7de06f08 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.609613 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3566/7108] 10.222.0.16 () {58 vars in 1222 bytes} [Sat Oct 15 21:18:39 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1327146833 => generated 417 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:18:39.614250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cd58a-9c14-4aff-ba1a-0a9b7de06f08 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.627686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cd58a-9c14-4aff-ba1a-0a9b7de06f08 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.629601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109e16e-f08e-4c75-b823-e710d553c9f4 tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.634078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.634957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.635237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190a4db-bc5e-45cd-9702-56ed5f3debec tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.635828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.635970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190a4db-bc5e-45cd-9702-56ed5f3debec tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.636215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190a4db-bc5e-45cd-9702-56ed5f3debec tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.636386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190a4db-bc5e-45cd-9702-56ed5f3debec tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services/1bc634bc01b64290bd3e01cb43990d4b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.636840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] PATH_INFO: `/v3/users/cfb51555eb104b03a912909cb6915077` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.636951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190a4db-bc5e-45cd-9702-56ed5f3debec tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:delete_service(service_id=1bc634bc01b64290bd3e01cb43990d4b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.638497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb51555eb104b03a912909cb6915077)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.639344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190a4db-bc5e-45cd-9702-56ed5f3debec tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.641935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.670233 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3567/7109] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:39 2022] DELETE /identity/v3/services/1bc634bc01b64290bd3e01cb43990d4b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:39.678558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190a4db-bc5e-45cd-9702-56ed5f3debec tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.713736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb51555eb104b03a912909cb6915077'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:39.726612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190a4db-bc5e-45cd-9702-56ed5f3debec tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.733867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81f8b7-6c2f-44c5-8c4c-bee3467a230d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.734777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81f8b7-6c2f-44c5-8c4c-bee3467a230d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.735092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81f8b7-6c2f-44c5-8c4c-bee3467a230d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.735363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81f8b7-6c2f-44c5-8c4c-bee3467a230d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services/088482b103a24f099d0de78ba557260d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.735946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81f8b7-6c2f-44c5-8c4c-bee3467a230d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:delete_service(service_id=088482b103a24f099d0de78ba557260d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.737696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81f8b7-6c2f-44c5-8c4c-bee3467a230d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.744674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb51555eb104b03a912909cb6915077'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:39.762740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb51555eb104b03a912909cb6915077'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:39.775355 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3568/7110] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:39 2022] DELETE /identity/v3/services/088482b103a24f099d0de78ba557260d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:39.780612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81f8b7-6c2f-44c5-8c4c-bee3467a230d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.784716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb51555eb104b03a912909cb6915077'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:39.785627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e57ae958-11a1-41bd-b8fe-32e80c1d0d59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb51555eb104b03a912909cb6915077'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:39.804085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3543/7111] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:39 2022] DELETE /identity/v3/users/cfb51555eb104b03a912909cb6915077 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:39.806510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a30a16-c8f9-40e4-a9a6-bd6c31c3de2d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.806790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a30a16-c8f9-40e4-a9a6-bd6c31c3de2d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.807063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a30a16-c8f9-40e4-a9a6-bd6c31c3de2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.826786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81f8b7-6c2f-44c5-8c4c-bee3467a230d tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:39.830449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e7a30a16-c8f9-40e4-a9a6-bd6c31c3de2d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:39.831895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e7a30a16-c8f9-40e4-a9a6-bd6c31c3de2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:39.849590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:39.870797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:39.930219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df2edd-a4a7-4488-8ac8-de714b5b92ac tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988845ff0ba849c4ad1432339c890ccf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '658d1ed0fea948f2b94cf941ff034726', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:39.931506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2edd-a4a7-4488-8ac8-de714b5b92ac tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:39.932363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2edd-a4a7-4488-8ac8-de714b5b92ac tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:39.932843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2edd-a4a7-4488-8ac8-de714b5b92ac tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] PATH_INFO: `/v3/services/0f29875fbe974d518ef243ef9d3d99ab` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:39.933718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df2edd-a4a7-4488-8ac8-de714b5b92ac tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorizing `identity:delete_service(service_id=0f29875fbe974d518ef243ef9d3d99ab)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:39.936249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df2edd-a4a7-4488-8ac8-de714b5b92ac tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:39.962691 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3544/7112] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:39 2022] POST /identity/v3/auth/tokens => generated 3405 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:39.974230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a30a16-c8f9-40e4-a9a6-bd6c31c3de2d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.976715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e7a30a16-c8f9-40e4-a9a6-bd6c31c3de2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:39.977480 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3569/7113] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:39 2022] DELETE /identity/v3/services/0f29875fbe974d518ef243ef9d3d99ab => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:39.982427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df2edd-a4a7-4488-8ac8-de714b5b92ac tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:39.985347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b5df2edd-a4a7-4488-8ac8-de714b5b92ac tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:40.015455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df2edd-a4a7-4488-8ac8-de714b5b92ac tempest-ServicesTestJSON-1530406380 tempest-ServicesTestJSON-1530406380-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:40.041468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:40.041468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:40.041468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:40.041468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] PATH_INFO: `/v3/users/69697d06fb4f4332a16df7cb41a61134` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:40.042051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] RBAC: Authorizing `identity:delete_user(user_id=69697d06fb4f4332a16df7cb41a61134)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:40.044150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:40.078270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a30a16-c8f9-40e4-a9a6-bd6c31c3de2d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:40.120454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00835fff-982b-47aa-a559-cca5085552d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:40.121327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00835fff-982b-47aa-a559-cca5085552d1 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:40.122220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00835fff-982b-47aa-a559-cca5085552d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:40.122517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00835fff-982b-47aa-a559-cca5085552d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:40.128790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69697d06fb4f4332a16df7cb41a61134'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:40.142352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00835fff-982b-47aa-a559-cca5085552d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:40.143812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00835fff-982b-47aa-a559-cca5085552d1 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:40.146108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69697d06fb4f4332a16df7cb41a61134'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:40.157349 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3545/7114] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:18:40.164796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69697d06fb4f4332a16df7cb41a61134'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:40.185297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69697d06fb4f4332a16df7cb41a61134'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:40.186034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69697d06fb4f4332a16df7cb41a61134'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:40.200728 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3570/7115] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:39 2022] DELETE /identity/v3/users/69697d06fb4f4332a16df7cb41a61134 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:40.209479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f639c-a1dc-459d-9236-69c85ccb1783 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:40.209759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f639c-a1dc-459d-9236-69c85ccb1783 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:40.210026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f639c-a1dc-459d-9236-69c85ccb1783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:40.241206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-028f639c-a1dc-459d-9236-69c85ccb1783 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:40.242409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-028f639c-a1dc-459d-9236-69c85ccb1783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:40.454721 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3546/7116] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:40.466011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:40.468719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:40.602929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa639f1e-4ba3-4ae3-97db-e244c144a61d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:40.608236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd837f-a45a-4cd0-9c9f-860dfae7a672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:40.610076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd837f-a45a-4cd0-9c9f-860dfae7a672 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:40.610380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd837f-a45a-4cd0-9c9f-860dfae7a672 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:40.610613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd837f-a45a-4cd0-9c9f-860dfae7a672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:40.622808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd837f-a45a-4cd0-9c9f-860dfae7a672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:40.624885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd837f-a45a-4cd0-9c9f-860dfae7a672 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:40.637906 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3571/7117] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:41.162674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f639c-a1dc-459d-9236-69c85ccb1783 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:41.165381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-028f639c-a1dc-459d-9236-69c85ccb1783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:41.185939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f639c-a1dc-459d-9236-69c85ccb1783 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:41.194759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:41.195882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:41.196320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:41.196734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] PATH_INFO: `/v3/projects/72f63c78e0e24c23a4060c561c0130bd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:41.199255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72f63c78e0e24c23a4060c561c0130bd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:41.204167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:41.259530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:41.268984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72f63c78e0e24c23a4060c561c0130bd', 'request_id': 'req-13289be6-6474-4dbe-9cd8-64fe40442bd8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.276973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72f63c78e0e24c23a4060c561c0130bd', 'request_id': 'req-13289be6-6474-4dbe-9cd8-64fe40442bd8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.284098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72f63c78e0e24c23a4060c561c0130bd', 'request_id': 'req-13289be6-6474-4dbe-9cd8-64fe40442bd8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.296900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] The token cache is being invalidate because project 72f63c78e0e24c23a4060c561c0130bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:41.297211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.297880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.299874 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3547/7118] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:41 2022] DELETE /identity/v3/projects/72f63c78e0e24c23a4060c561c0130bd => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:41.303386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:41.303787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd837f-a45a-4cd0-9c9f-860dfae7a672 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:41.305284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:41.306436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5bfd837f-a45a-4cd0-9c9f-860dfae7a672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:41.340116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd837f-a45a-4cd0-9c9f-860dfae7a672 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:41.350026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:41.395200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13289be6-6474-4dbe-9cd8-64fe40442bd8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:41.400672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:41.402099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:41.402519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:41.402958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] PATH_INFO: `/v3/users/988845ff0ba849c4ad1432339c890ccf` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:41.404577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=988845ff0ba849c4ad1432339c890ccf)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:41.406903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:41.413522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:41.414170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:41.414443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:41.414712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] PATH_INFO: `/v3/projects/3084ad4a90084be6a95ee563a39c9233` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:41.415847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3084ad4a90084be6a95ee563a39c9233)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:41.418142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:41.473817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988845ff0ba849c4ad1432339c890ccf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.479033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3084ad4a90084be6a95ee563a39c9233', 'request_id': 'req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.490652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988845ff0ba849c4ad1432339c890ccf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.493683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3084ad4a90084be6a95ee563a39c9233', 'request_id': 'req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.501474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3084ad4a90084be6a95ee563a39c9233', 'request_id': 'req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.505582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988845ff0ba849c4ad1432339c890ccf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.515854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] The token cache is being invalidate because project 3084ad4a90084be6a95ee563a39c9233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:41.516212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.517016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e6ed152a-c5c2-4b70-b1e7-68285c9d6434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.520330 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3572/7119] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:41 2022] DELETE /identity/v3/projects/3084ad4a90084be6a95ee563a39c9233 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:41.526269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89118d-c886-4521-8468-b0c04b9ddb92 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:41.526765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89118d-c886-4521-8468-b0c04b9ddb92 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:41.527388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988845ff0ba849c4ad1432339c890ccf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.528013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89118d-c886-4521-8468-b0c04b9ddb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:41.528907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c2970958-14d6-48a6-8b83-32b308639ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988845ff0ba849c4ad1432339c890ccf'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:41.545135 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3548/7120] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:41 2022] DELETE /identity/v3/users/988845ff0ba849c4ad1432339c890ccf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:41.560309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1b89118d-c886-4521-8468-b0c04b9ddb92 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:41.562544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1b89118d-c886-4521-8468-b0c04b9ddb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:41.665882 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3573/7121] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:41.674301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814b095-6978-435f-b859-a4b110400b38 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:41.674852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814b095-6978-435f-b859-a4b110400b38 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:41.675305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814b095-6978-435f-b859-a4b110400b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:41.697342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5814b095-6978-435f-b859-a4b110400b38 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:41.698809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5814b095-6978-435f-b859-a4b110400b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:41.710420 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3549/7122] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:41.713418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89118d-c886-4521-8468-b0c04b9ddb92 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:41.738548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89118d-c886-4521-8468-b0c04b9ddb92 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:41.743760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c99db0-bf82-47fd-81fa-e01f1ba8e640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:41.744479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c99db0-bf82-47fd-81fa-e01f1ba8e640 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:41.744755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c99db0-bf82-47fd-81fa-e01f1ba8e640 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:41.745018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c99db0-bf82-47fd-81fa-e01f1ba8e640 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:41.745871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c99db0-bf82-47fd-81fa-e01f1ba8e640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:41.746165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c99db0-bf82-47fd-81fa-e01f1ba8e640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:41.748397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c99db0-bf82-47fd-81fa-e01f1ba8e640 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:41.764732 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3574/7123] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:18:41.768840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814b095-6978-435f-b859-a4b110400b38 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:41.780998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814b095-6978-435f-b859-a4b110400b38 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:41.786943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf6542-0522-4bb3-82b6-f03603843802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:41.788065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf6542-0522-4bb3-82b6-f03603843802 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:41.788503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf6542-0522-4bb3-82b6-f03603843802 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:41.788934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf6542-0522-4bb3-82b6-f03603843802 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:41.789952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf6542-0522-4bb3-82b6-f03603843802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:41.793799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf6542-0522-4bb3-82b6-f03603843802 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:41.847418 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3550/7124] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:41 2022] POST /identity/v3/projects => generated 365 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:41.851816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c99db0-bf82-47fd-81fa-e01f1ba8e640 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:41.874805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c99db0-bf82-47fd-81fa-e01f1ba8e640 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:41.918525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:41.986112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b94555-b82a-4c6c-8b74-3b08eba6d9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:41.987001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94555-b82a-4c6c-8b74-3b08eba6d9ae admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:41.987424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94555-b82a-4c6c-8b74-3b08eba6d9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:41.990272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94555-b82a-4c6c-8b74-3b08eba6d9ae admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:41.991582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b94555-b82a-4c6c-8b74-3b08eba6d9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:41.993805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b94555-b82a-4c6c-8b74-3b08eba6d9ae admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.057342 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3575/7125] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:41 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:42.062270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf6542-0522-4bb3-82b6-f03603843802 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.071651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b94555-b82a-4c6c-8b74-3b08eba6d9ae admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.073516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-00b94555-b82a-4c6c-8b74-3b08eba6d9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:42.076851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf6542-0522-4bb3-82b6-f03603843802 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.083234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd51b26-b288-4a4c-b8a8-a33b747ccca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.084166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd51b26-b288-4a4c-b8a8-a33b747ccca3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.084502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd51b26-b288-4a4c-b8a8-a33b747ccca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.084824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd51b26-b288-4a4c-b8a8-a33b747ccca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.085552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd51b26-b288-4a4c-b8a8-a33b747ccca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.087702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd51b26-b288-4a4c-b8a8-a33b747ccca3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.110525 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3551/7126] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:42.111624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd51b26-b288-4a4c-b8a8-a33b747ccca3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.112856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b94555-b82a-4c6c-8b74-3b08eba6d9ae admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.127636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.128129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.128431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.128710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] PATH_INFO: `/v3/projects/658d1ed0fea948f2b94cf941ff034726` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.131190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] RBAC: Authorizing `identity:delete_project(project_id=658d1ed0fea948f2b94cf941ff034726)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.134900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd51b26-b288-4a4c-b8a8-a33b747ccca3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.136663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.149902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f34e-880a-4287-b81c-084e560add85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.150578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f34e-880a-4287-b81c-084e560add85 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.150801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f34e-880a-4287-b81c-084e560add85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.151100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f34e-880a-4287-b81c-084e560add85 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.151877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66f34e-880a-4287-b81c-084e560add85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.155839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66f34e-880a-4287-b81c-084e560add85 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.160525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:42.182873 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3552/7127] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:42.184122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f34e-880a-4287-b81c-084e560add85 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.207172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f34e-880a-4287-b81c-084e560add85 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.212609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53082a9f-a62c-410e-b2dc-84998aa3cf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.213899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082a9f-a62c-410e-b2dc-84998aa3cf95 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.214549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082a9f-a62c-410e-b2dc-84998aa3cf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.215543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082a9f-a62c-410e-b2dc-84998aa3cf95 admin admin] PATH_INFO: `/v3/projects/82b30bfac5e34d4fa7a797b561d34d58/users/fc7967f332c1483395accf914d75979a/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.236131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53082a9f-a62c-410e-b2dc-84998aa3cf95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b30bfac5e34d4fa7a797b561d34d58, user_id=fc7967f332c1483395accf914d75979a, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.241185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53082a9f-a62c-410e-b2dc-84998aa3cf95 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.268059 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3553/7128] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:42 2022] PUT /identity/v3/projects/82b30bfac5e34d4fa7a797b561d34d58/users/fc7967f332c1483395accf914d75979a/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:42.271618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658d1ed0fea948f2b94cf941ff034726', 'request_id': 'req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:42.278982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9e405-9223-4080-8ab7-c1ee2c33ae0c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.279315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9e405-9223-4080-8ab7-c1ee2c33ae0c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.279585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9e405-9223-4080-8ab7-c1ee2c33ae0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.290761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658d1ed0fea948f2b94cf941ff034726', 'request_id': 'req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:42.300832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658d1ed0fea948f2b94cf941ff034726', 'request_id': 'req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:42.316551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] The token cache is being invalidate because project 658d1ed0fea948f2b94cf941ff034726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:42.317294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:42.318143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ac2e8e5e-1874-4fa4-8bc5-bae2de86480a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:42.320482 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3576/7129] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:42 2022] DELETE /identity/v3/projects/658d1ed0fea948f2b94cf941ff034726 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:42.325387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ff591-ca8a-4d3e-8111-87a9e8da4ffb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.325669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ff591-ca8a-4d3e-8111-87a9e8da4ffb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.325934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ff591-ca8a-4d3e-8111-87a9e8da4ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.335861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-58f9e405-9223-4080-8ab7-c1ee2c33ae0c None None] MFA Rules not processed for user `fc7967f332c1483395accf914d75979a`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:42.337540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-58f9e405-9223-4080-8ab7-c1ee2c33ae0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:42.359627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-438ff591-ca8a-4d3e-8111-87a9e8da4ffb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:42.362349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-438ff591-ca8a-4d3e-8111-87a9e8da4ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:42.456971 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3554/7130] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:42.469220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9e405-9223-4080-8ab7-c1ee2c33ae0c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.470906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-58f9e405-9223-4080-8ab7-c1ee2c33ae0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:42.512184 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3577/7131] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:42.518285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc8fe-8e43-4131-868d-108fbd2e1b55 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.518746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc8fe-8e43-4131-868d-108fbd2e1b55 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.519202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc8fe-8e43-4131-868d-108fbd2e1b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.547374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-258fc8fe-8e43-4131-868d-108fbd2e1b55 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:42.549025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-258fc8fe-8e43-4131-868d-108fbd2e1b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:42.575697 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3578/7132] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:42.581991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc8fe-8e43-4131-868d-108fbd2e1b55 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.597472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc8fe-8e43-4131-868d-108fbd2e1b55 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.616513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f4951-d9be-45dc-9f56-46353236f238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.617160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f4951-d9be-45dc-9f56-46353236f238 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.617438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f4951-d9be-45dc-9f56-46353236f238 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.617702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f4951-d9be-45dc-9f56-46353236f238 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.618355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f4951-d9be-45dc-9f56-46353236f238 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:42.619139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f4951-d9be-45dc-9f56-46353236f238 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.620438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9e405-9223-4080-8ab7-c1ee2c33ae0c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.623438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f4951-d9be-45dc-9f56-46353236f238 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.625285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78934cc-f6d3-491f-8688-f1920cc491a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.626016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78934cc-f6d3-491f-8688-f1920cc491a2 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.626302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78934cc-f6d3-491f-8688-f1920cc491a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.626982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78934cc-f6d3-491f-8688-f1920cc491a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.637216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78934cc-f6d3-491f-8688-f1920cc491a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.638751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78934cc-f6d3-491f-8688-f1920cc491a2 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.641933 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3579/7133] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:42.642711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f4951-d9be-45dc-9f56-46353236f238 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.656815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f4951-d9be-45dc-9f56-46353236f238 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.662188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c59f58-5106-4f20-9880-88e1689a1243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.664150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c59f58-5106-4f20-9880-88e1689a1243 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.664150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c59f58-5106-4f20-9880-88e1689a1243 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.664150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c59f58-5106-4f20-9880-88e1689a1243 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.664150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c59f58-5106-4f20-9880-88e1689a1243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.665872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c59f58-5106-4f20-9880-88e1689a1243 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.669242 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3555/7134] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:18:42.698021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:42.709385 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3580/7135] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:42 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:42.711249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78934cc-f6d3-491f-8688-f1920cc491a2 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.726830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78934cc-f6d3-491f-8688-f1920cc491a2 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.785404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:42.801812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dffeb-c71a-47ab-82d5-b5f08ab891da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.802507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dffeb-c71a-47ab-82d5-b5f08ab891da admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.803346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dffeb-c71a-47ab-82d5-b5f08ab891da admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.803346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dffeb-c71a-47ab-82d5-b5f08ab891da admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.803983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784dffeb-c71a-47ab-82d5-b5f08ab891da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.805998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784dffeb-c71a-47ab-82d5-b5f08ab891da admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.833086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3556/7136] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:42 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:42.835693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c59f58-5106-4f20-9880-88e1689a1243 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.850307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c59f58-5106-4f20-9880-88e1689a1243 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.856789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b563e-b20d-48ca-b08c-b89abd3938e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.857841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b563e-b20d-48ca-b08c-b89abd3938e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.858652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b563e-b20d-48ca-b08c-b89abd3938e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.858986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b563e-b20d-48ca-b08c-b89abd3938e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.859681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b563e-b20d-48ca-b08c-b89abd3938e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.861978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b563e-b20d-48ca-b08c-b89abd3938e1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.877078 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3581/7137] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:42.879562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dffeb-c71a-47ab-82d5-b5f08ab891da admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.895965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dffeb-c71a-47ab-82d5-b5f08ab891da admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.903971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3223b-98bf-4dd5-b2b9-5e18cb914420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.904608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3223b-98bf-4dd5-b2b9-5e18cb914420 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.905773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3223b-98bf-4dd5-b2b9-5e18cb914420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.906085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3223b-98bf-4dd5-b2b9-5e18cb914420 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.906920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3223b-98bf-4dd5-b2b9-5e18cb914420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.908808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3223b-98bf-4dd5-b2b9-5e18cb914420 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:42.923317 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3557/7138] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:42.925261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b563e-b20d-48ca-b08c-b89abd3938e1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:42.939726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b563e-b20d-48ca-b08c-b89abd3938e1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:42.946787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c44f96-8fd0-4586-9f38-9b45e0eac922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:42.947590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c44f96-8fd0-4586-9f38-9b45e0eac922 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:42.947955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c44f96-8fd0-4586-9f38-9b45e0eac922 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:42.948336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c44f96-8fd0-4586-9f38-9b45e0eac922 admin admin] PATH_INFO: `/v3/projects/891505ffe7af4d07b4e4ec24b0021d9c/users/8e814fdd40e343679ad5eda9d7fbd129/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:42.966407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c44f96-8fd0-4586-9f38-9b45e0eac922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891505ffe7af4d07b4e4ec24b0021d9c, user_id=8e814fdd40e343679ad5eda9d7fbd129, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:42.969333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c44f96-8fd0-4586-9f38-9b45e0eac922 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:43.002197 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3582/7139] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:42 2022] PUT /identity/v3/projects/891505ffe7af4d07b4e4ec24b0021d9c/users/8e814fdd40e343679ad5eda9d7fbd129/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:43.105155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bccc0-9276-448f-a332-39b7a816a866 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:43.105457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bccc0-9276-448f-a332-39b7a816a866 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:43.105713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bccc0-9276-448f-a332-39b7a816a866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:43.148332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-af9bccc0-9276-448f-a332-39b7a816a866 None None] MFA Rules not processed for user `8e814fdd40e343679ad5eda9d7fbd129`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:43.150130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af9bccc0-9276-448f-a332-39b7a816a866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:43.234959 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3558/7140] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:43.239530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c44f96-8fd0-4586-9f38-9b45e0eac922 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:43.328916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c44f96-8fd0-4586-9f38-9b45e0eac922 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:43.336771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231cc65-93a8-4884-9656-92292fff1fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:43.337918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231cc65-93a8-4884-9656-92292fff1fbf admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:43.338701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231cc65-93a8-4884-9656-92292fff1fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:43.339319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231cc65-93a8-4884-9656-92292fff1fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:43.340107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231cc65-93a8-4884-9656-92292fff1fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:43.343391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231cc65-93a8-4884-9656-92292fff1fbf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:43.397094 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3583/7141] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:43 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:43.400923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bccc0-9276-448f-a332-39b7a816a866 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:43.422287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bccc0-9276-448f-a332-39b7a816a866 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:43.508156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61559cea-429e-48e5-a8cf-5adea92075af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:43.508904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61559cea-429e-48e5-a8cf-5adea92075af admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:43.509276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61559cea-429e-48e5-a8cf-5adea92075af admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:43.509606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61559cea-429e-48e5-a8cf-5adea92075af admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:43.510290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61559cea-429e-48e5-a8cf-5adea92075af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:43.513405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61559cea-429e-48e5-a8cf-5adea92075af admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:43.539268 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3559/7142] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:43 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:43.543073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231cc65-93a8-4884-9656-92292fff1fbf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:43.554002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231cc65-93a8-4884-9656-92292fff1fbf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:43.558736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b6f6e-07b9-4a9c-8ae3-3d1cb6ab8b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:43.559444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b6f6e-07b9-4a9c-8ae3-3d1cb6ab8b47 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:43.559722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b6f6e-07b9-4a9c-8ae3-3d1cb6ab8b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:43.559990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b6f6e-07b9-4a9c-8ae3-3d1cb6ab8b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:43.560614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b6f6e-07b9-4a9c-8ae3-3d1cb6ab8b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:43.562766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b6f6e-07b9-4a9c-8ae3-3d1cb6ab8b47 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:43.580415 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3584/7143] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:43.584251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61559cea-429e-48e5-a8cf-5adea92075af admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:43.595309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61559cea-429e-48e5-a8cf-5adea92075af admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:43.600544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075b332-62e8-4058-b740-5f2aebfb2e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:43.601082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075b332-62e8-4058-b740-5f2aebfb2e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:43.601362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075b332-62e8-4058-b740-5f2aebfb2e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:43.601641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075b332-62e8-4058-b740-5f2aebfb2e57 admin admin] PATH_INFO: `/v3/projects/e14f4b6714104fa8a44169447aad22b9/users/983dd47443f04fb5b868d5c1c17d7d4b/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:43.618350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075b332-62e8-4058-b740-5f2aebfb2e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14f4b6714104fa8a44169447aad22b9, user_id=983dd47443f04fb5b868d5c1c17d7d4b, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:43.621069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075b332-62e8-4058-b740-5f2aebfb2e57 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:43.639292 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3560/7144] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:43 2022] PUT /identity/v3/projects/e14f4b6714104fa8a44169447aad22b9/users/983dd47443f04fb5b868d5c1c17d7d4b/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:43.652619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b6f6e-07b9-4a9c-8ae3-3d1cb6ab8b47 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:43.664804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b6f6e-07b9-4a9c-8ae3-3d1cb6ab8b47 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:43.741765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f91612-0f00-49ad-bce7-4a18e0221a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:43.742416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f91612-0f00-49ad-bce7-4a18e0221a7b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:43.742648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f91612-0f00-49ad-bce7-4a18e0221a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:43.743092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f91612-0f00-49ad-bce7-4a18e0221a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:43.746573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f91612-0f00-49ad-bce7-4a18e0221a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:43.749255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f91612-0f00-49ad-bce7-4a18e0221a7b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:43.750341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075b332-62e8-4058-b740-5f2aebfb2e57 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:43.759024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8075b332-62e8-4058-b740-5f2aebfb2e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:43.777602 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3585/7145] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:43.778649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075b332-62e8-4058-b740-5f2aebfb2e57 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:43.781593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f91612-0f00-49ad-bce7-4a18e0221a7b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:43.787844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:43.788751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:43.789138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:43.789541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] PATH_INFO: `/v3/users/8ef16f7ada164345896bcc34d3babe66` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:43.791247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef16f7ada164345896bcc34d3babe66)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:43.792948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:43.795935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f91612-0f00-49ad-bce7-4a18e0221a7b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:43.804023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8f40f-06a3-4492-a73c-55d29d4d0e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:43.805135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8f40f-06a3-4492-a73c-55d29d4d0e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:43.807147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8f40f-06a3-4492-a73c-55d29d4d0e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:43.807147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8f40f-06a3-4492-a73c-55d29d4d0e91 admin admin] PATH_INFO: `/v3/projects/e14f4b6714104fa8a44169447aad22b9/users/983dd47443f04fb5b868d5c1c17d7d4b/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:43.808546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8f40f-06a3-4492-a73c-55d29d4d0e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14f4b6714104fa8a44169447aad22b9, user_id=983dd47443f04fb5b868d5c1c17d7d4b, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:43.812803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:INSERT=7,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:43.812955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8f40f-06a3-4492-a73c-55d29d4d0e91 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:43.837234 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3586/7146] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:43 2022] PUT /identity/v3/projects/e14f4b6714104fa8a44169447aad22b9/users/983dd47443f04fb5b868d5c1c17d7d4b/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:43.847589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c79f2-f6f6-4b3e-9588-bdb635c73d3d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:43.848561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c79f2-f6f6-4b3e-9588-bdb635c73d3d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:43.848972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c79f2-f6f6-4b3e-9588-bdb635c73d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:43.871837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef16f7ada164345896bcc34d3babe66'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:43.887933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5b3c79f2-f6f6-4b3e-9588-bdb635c73d3d None None] MFA Rules not processed for user `983dd47443f04fb5b868d5c1c17d7d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:43.889992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef16f7ada164345896bcc34d3babe66'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:43.890161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5b3c79f2-f6f6-4b3e-9588-bdb635c73d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:43.899600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef16f7ada164345896bcc34d3babe66'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:43.900610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:43.911085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef16f7ada164345896bcc34d3babe66'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:43.912051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d2276131-963a-4a13-b9a4-c3430445c5a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef16f7ada164345896bcc34d3babe66'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:43.920313 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3561/7147] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:43 2022] DELETE /identity/v3/users/8ef16f7ada164345896bcc34d3babe66 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:43.926765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de814fd5-1d77-4bea-96db-524949699360 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:43.927051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de814fd5-1d77-4bea-96db-524949699360 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:43.927753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de814fd5-1d77-4bea-96db-524949699360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:43.948605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-de814fd5-1d77-4bea-96db-524949699360 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:43.949847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-de814fd5-1d77-4bea-96db-524949699360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:43.979026 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3587/7148] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:43.984529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c79f2-f6f6-4b3e-9588-bdb635c73d3d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:43.997641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c79f2-f6f6-4b3e-9588-bdb635c73d3d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:44.056140 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3562/7149] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:44.070760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de814fd5-1d77-4bea-96db-524949699360 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:44.072150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-de814fd5-1d77-4bea-96db-524949699360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:44.076120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0cb8e-404f-4dd0-8554-14fc79bfcbd5 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983dd47443f04fb5b868d5c1c17d7d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e14f4b6714104fa8a44169447aad22b9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:44.076959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0cb8e-404f-4dd0-8554-14fc79bfcbd5 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.077515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0cb8e-404f-4dd0-8554-14fc79bfcbd5 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.078223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0cb8e-404f-4dd0-8554-14fc79bfcbd5 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.078999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0cb8e-404f-4dd0-8554-14fc79bfcbd5 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:44.081841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0cb8e-404f-4dd0-8554-14fc79bfcbd5 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:44.125073 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3588/7150] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:43 2022] POST /identity/v3/projects => generated 357 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:44.129683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0cb8e-404f-4dd0-8554-14fc79bfcbd5 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:44.154336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de814fd5-1d77-4bea-96db-524949699360 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:44.217386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:44.224914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aababf3-be44-497c-b3c8-66c895b9e4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:44.225669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0cb8e-404f-4dd0-8554-14fc79bfcbd5 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:44.226145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aababf3-be44-497c-b3c8-66c895b9e4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.226719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aababf3-be44-497c-b3c8-66c895b9e4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.227319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aababf3-be44-497c-b3c8-66c895b9e4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.241351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a92bc4-d3e8-492b-8919-0614bac07e79 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983dd47443f04fb5b868d5c1c17d7d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e14f4b6714104fa8a44169447aad22b9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:44.241351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a92bc4-d3e8-492b-8919-0614bac07e79 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.241351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a92bc4-d3e8-492b-8919-0614bac07e79 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.241673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a92bc4-d3e8-492b-8919-0614bac07e79 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.241673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a92bc4-d3e8-492b-8919-0614bac07e79 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:44.241673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a92bc4-d3e8-492b-8919-0614bac07e79 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:44.286143 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3589/7151] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:44 2022] POST /identity/v3/users => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:44.288307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a92bc4-d3e8-492b-8919-0614bac07e79 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:44.310123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aababf3-be44-497c-b3c8-66c895b9e4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:44.310690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a92bc4-d3e8-492b-8919-0614bac07e79 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:44.312959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aababf3-be44-497c-b3c8-66c895b9e4f8 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:44.316921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090404a-4689-4e6f-9ce6-9f2e49b596f7 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983dd47443f04fb5b868d5c1c17d7d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e14f4b6714104fa8a44169447aad22b9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:44.317566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090404a-4689-4e6f-9ce6-9f2e49b596f7 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.317843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090404a-4689-4e6f-9ce6-9f2e49b596f7 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.318113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090404a-4689-4e6f-9ce6-9f2e49b596f7 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.318840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090404a-4689-4e6f-9ce6-9f2e49b596f7 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:44.319139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090404a-4689-4e6f-9ce6-9f2e49b596f7 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:44.321580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090404a-4689-4e6f-9ce6-9f2e49b596f7 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:44.331318 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3563/7152] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:44.381931 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3590/7153] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Oct 15 21:18:44 2022] GET /identity/v3/users?domain_id=default => generated 7135 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:44.388202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aababf3-be44-497c-b3c8-66c895b9e4f8 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:44.405569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aababf3-be44-497c-b3c8-66c895b9e4f8 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:44.411098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc9aa1-5c44-4f1a-bc76-eb93edadad48 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983dd47443f04fb5b868d5c1c17d7d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e14f4b6714104fa8a44169447aad22b9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:44.422120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc9aa1-5c44-4f1a-bc76-eb93edadad48 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.423221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc9aa1-5c44-4f1a-bc76-eb93edadad48 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.423511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc9aa1-5c44-4f1a-bc76-eb93edadad48 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] PATH_INFO: `/v3/users/a9d0d4a92d5b45d1bf7f90d932e788fe` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.470051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc9aa1-5c44-4f1a-bc76-eb93edadad48 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorizing `identity:update_user(user_id=a9d0d4a92d5b45d1bf7f90d932e788fe)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:44.472024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc9aa1-5c44-4f1a-bc76-eb93edadad48 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:44.525255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47fc9aa1-5c44-4f1a-bc76-eb93edadad48 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a9d0d4a92d5b45d1bf7f90d932e788fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.528019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090404a-4689-4e6f-9ce6-9f2e49b596f7 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:44.530305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4090404a-4689-4e6f-9ce6-9f2e49b596f7 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:44.550105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090404a-4689-4e6f-9ce6-9f2e49b596f7 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:44.550469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47fc9aa1-5c44-4f1a-bc76-eb93edadad48 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invalidating the token cache because user a9d0d4a92d5b45d1bf7f90d932e788fe was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:44.550694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47fc9aa1-5c44-4f1a-bc76-eb93edadad48 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.552228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-47fc9aa1-5c44-4f1a-bc76-eb93edadad48 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.554508 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3564/7154] 10.222.0.16 () {60 vars in 1244 bytes} [Sat Oct 15 21:18:44 2022] PATCH /identity/v3/users/a9d0d4a92d5b45d1bf7f90d932e788fe => generated 488 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:44.557103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:44.557877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.558213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.558478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] PATH_INFO: `/v3/projects/0ee7826b3c01475db29de402244cd4aa` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.559902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee7826b3c01475db29de402244cd4aa)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:44.560575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c97f8-77ef-40a5-81ae-67b645113c50 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.561160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c97f8-77ef-40a5-81ae-67b645113c50 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.561949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c97f8-77ef-40a5-81ae-67b645113c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.563485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:44.572613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:44.603777 ncdv-master-1 devstack@keystone.service[99100]: WARNING keystone.auth.plugins.core [None req-6e0c97f8-77ef-40a5-81ae-67b645113c50 None None] User is disabled: a9d0d4a92d5b45d1bf7f90d932e788fe: AssertionError: User is disabled: a9d0d4a92d5b45d1bf7f90d932e788fe Oct 15 21:18:44.612052 ncdv-master-1 devstack@keystone.service[99100]: WARNING keystone.server.flask.application [None req-6e0c97f8-77ef-40a5-81ae-67b645113c50 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 15 21:18:44.614001 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3565/7155] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:44 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 55 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 15 21:18:44.618402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c97f8-77ef-40a5-81ae-67b645113c50 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:44.620282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e0c97f8-77ef-40a5-81ae-67b645113c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:44.630884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee7826b3c01475db29de402244cd4aa', 'request_id': 'req-60e0b71b-fc74-445f-86c7-e782254d30c1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.643196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee7826b3c01475db29de402244cd4aa', 'request_id': 'req-60e0b71b-fc74-445f-86c7-e782254d30c1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.652354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee7826b3c01475db29de402244cd4aa', 'request_id': 'req-60e0b71b-fc74-445f-86c7-e782254d30c1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.664953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] The token cache is being invalidate because project 0ee7826b3c01475db29de402244cd4aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:44.665476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.666421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-60e0b71b-fc74-445f-86c7-e782254d30c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.671529 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3591/7156] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:44 2022] DELETE /identity/v3/projects/0ee7826b3c01475db29de402244cd4aa => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:44.677258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd92b55-9629-4c7b-b4b4-4cdfa146de11 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.677540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd92b55-9629-4c7b-b4b4-4cdfa146de11 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.677807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd92b55-9629-4c7b-b4b4-4cdfa146de11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.707560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-acd92b55-9629-4c7b-b4b4-4cdfa146de11 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:44.709254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-acd92b55-9629-4c7b-b4b4-4cdfa146de11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:44.756525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c97f8-77ef-40a5-81ae-67b645113c50 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:44.777314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983dd47443f04fb5b868d5c1c17d7d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e14f4b6714104fa8a44169447aad22b9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:44.778115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.778452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.778728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] PATH_INFO: `/v3/users/a9d0d4a92d5b45d1bf7f90d932e788fe` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.801275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a9d0d4a92d5b45d1bf7f90d932e788fe)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:44.805796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:44.818732 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3592/7157] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:44.820813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eaa0e-7640-4497-ad0b-755777aedf66 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.821128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eaa0e-7640-4497-ad0b-755777aedf66 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.821411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eaa0e-7640-4497-ad0b-755777aedf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.887002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-267eaa0e-7640-4497-ad0b-755777aedf66 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:44.888237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-267eaa0e-7640-4497-ad0b-755777aedf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:44.898473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:44.899582 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3593/7158] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:44.903351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d0d4a92d5b45d1bf7f90d932e788fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.904729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eaa0e-7640-4497-ad0b-755777aedf66 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:44.929656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eaa0e-7640-4497-ad0b-755777aedf66 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:44.942259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d0d4a92d5b45d1bf7f90d932e788fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.951715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1266ebd-6974-4309-885b-44c24f5bf503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:44.954105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1266ebd-6974-4309-885b-44c24f5bf503 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:44.954619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1266ebd-6974-4309-885b-44c24f5bf503 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:44.955127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1266ebd-6974-4309-885b-44c24f5bf503 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:44.956143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1266ebd-6974-4309-885b-44c24f5bf503 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:44.956633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1266ebd-6974-4309-885b-44c24f5bf503 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:44.966002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1266ebd-6974-4309-885b-44c24f5bf503 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:44.967479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d0d4a92d5b45d1bf7f90d932e788fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.983258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d0d4a92d5b45d1bf7f90d932e788fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.985078 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3594/7159] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:44.986005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d0d4a92d5b45d1bf7f90d932e788fe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:44.988195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1266ebd-6974-4309-885b-44c24f5bf503 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:44.992023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f1266ebd-6974-4309-885b-44c24f5bf503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:44.996348 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3566/7160] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:44 2022] DELETE /identity/v3/users/a9d0d4a92d5b45d1bf7f90d932e788fe => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:45.000693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:45.002222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:45.009754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1266ebd-6974-4309-885b-44c24f5bf503 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:45.089638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93736394-c86a-4865-8c3a-4d1f77e8cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:45.090598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93736394-c86a-4865-8c3a-4d1f77e8cace admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.091156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93736394-c86a-4865-8c3a-4d1f77e8cace admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.091450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93736394-c86a-4865-8c3a-4d1f77e8cace admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.092548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93736394-c86a-4865-8c3a-4d1f77e8cace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:45.094847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93736394-c86a-4865-8c3a-4d1f77e8cace admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:45.106765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5fbca-7a85-4849-85ae-7b3275ee0e6f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:45.112453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983dd47443f04fb5b868d5c1c17d7d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e14f4b6714104fa8a44169447aad22b9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:45.113228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.113578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.114026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] PATH_INFO: `/v3/projects/af8a865c8061452f8086ce6a10f230bd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.115466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorizing `identity:delete_project(project_id=af8a865c8061452f8086ce6a10f230bd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:45.118602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:45.122120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:45.139919 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3595/7161] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:44 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:45.140949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93736394-c86a-4865-8c3a-4d1f77e8cace admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:45.194841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af8a865c8061452f8086ce6a10f230bd', 'request_id': 'req-71fa0a96-4586-4e9a-9749-619de79fd264'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:45.204486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af8a865c8061452f8086ce6a10f230bd', 'request_id': 'req-71fa0a96-4586-4e9a-9749-619de79fd264'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:45.214746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af8a865c8061452f8086ce6a10f230bd', 'request_id': 'req-71fa0a96-4586-4e9a-9749-619de79fd264'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:45.226354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93736394-c86a-4865-8c3a-4d1f77e8cace admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:45.229684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] The token cache is being invalidate because project af8a865c8061452f8086ce6a10f230bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:45.230018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:45.231174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:45.234435 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3567/7162] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:44 2022] DELETE /identity/v3/projects/af8a865c8061452f8086ce6a10f230bd => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:45.240194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:45.241514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:45.307033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:45.317519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4108-d9df-47d4-ab51-2366422b9723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:45.318642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4108-d9df-47d4-ab51-2366422b9723 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.318923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4108-d9df-47d4-ab51-2366422b9723 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.319194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4108-d9df-47d4-ab51-2366422b9723 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.320107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4108-d9df-47d4-ab51-2366422b9723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:45.323302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4108-d9df-47d4-ab51-2366422b9723 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:45.328284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa0a96-4586-4e9a-9749-619de79fd264 tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:45.336810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd35bcb-33fa-4cb5-b584-c83921963c8f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983dd47443f04fb5b868d5c1c17d7d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e14f4b6714104fa8a44169447aad22b9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:45.337490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd35bcb-33fa-4cb5-b584-c83921963c8f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.337985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd35bcb-33fa-4cb5-b584-c83921963c8f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.338470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd35bcb-33fa-4cb5-b584-c83921963c8f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.339239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd35bcb-33fa-4cb5-b584-c83921963c8f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:45.342032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd35bcb-33fa-4cb5-b584-c83921963c8f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:45.353425 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3596/7163] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:45 2022] POST /identity/v3/users => generated 284 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-cfd35bcb-33fa-4cb5-b584-c83921963c8f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Could not find domain: ebaa41cb75604a258f32254b9099a993.: keystone.exception.DomainNotFound: Could not find domain: ebaa41cb75604a258f32254b9099a993. Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: ebaa41cb75604a258f32254b9099a993. Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:18:45.357584 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 987, in create_user Oct 15 21:18:45.359624 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_domain(domain_id) Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application with Lock( Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self._enter() Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.creator() Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application created_value = creator( Oct 15 21:18:45.364531 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 15 21:18:45.365714 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 21:18:45.365714 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: ebaa41cb75604a258f32254b9099a993. Oct 15 21:18:45.365714 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:18:45.365944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4108-d9df-47d4-ab51-2366422b9723 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:45.366037 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3568/7164] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:45 2022] POST /identity/v3/users => generated 112 bytes in 121 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 15 21:18:45.366134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e5af4108-d9df-47d4-ab51-2366422b9723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:45.366234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd35bcb-33fa-4cb5-b584-c83921963c8f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:45.366816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cfd35bcb-33fa-4cb5-b584-c83921963c8f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:45.375900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4108-d9df-47d4-ab51-2366422b9723 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:45.380870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71a2ec-3598-478f-8387-96714ffd3eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:45.381285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd35bcb-33fa-4cb5-b584-c83921963c8f tempest-UsersNegativeTest-137428293 tempest-UsersNegativeTest-137428293-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:45.381530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71a2ec-3598-478f-8387-96714ffd3eae admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.381804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71a2ec-3598-478f-8387-96714ffd3eae admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.382073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71a2ec-3598-478f-8387-96714ffd3eae admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.382715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71a2ec-3598-478f-8387-96714ffd3eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:45.384299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71a2ec-3598-478f-8387-96714ffd3eae admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:45.385955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:45.386559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.386834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.387095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] PATH_INFO: `/v3/users/8e814fdd40e343679ad5eda9d7fbd129` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.388214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e814fdd40e343679ad5eda9d7fbd129)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:45.390181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:45.396866 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3597/7165] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:45.400014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71a2ec-3598-478f-8387-96714ffd3eae admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:45.410210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71a2ec-3598-478f-8387-96714ffd3eae admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:45.414502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725a096-df19-42b1-9f4b-b0343f60b90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:45.415083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725a096-df19-42b1-9f4b-b0343f60b90f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.415353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725a096-df19-42b1-9f4b-b0343f60b90f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.415631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725a096-df19-42b1-9f4b-b0343f60b90f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.416286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725a096-df19-42b1-9f4b-b0343f60b90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:45.418122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725a096-df19-42b1-9f4b-b0343f60b90f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:45.430132 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3598/7166] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:45.434591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725a096-df19-42b1-9f4b-b0343f60b90f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:45.443627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e814fdd40e343679ad5eda9d7fbd129'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:45.444985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725a096-df19-42b1-9f4b-b0343f60b90f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:45.449664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47655eef-2a23-4ffd-b7a0-40702d982e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:45.450283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47655eef-2a23-4ffd-b7a0-40702d982e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.450560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47655eef-2a23-4ffd-b7a0-40702d982e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.450828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47655eef-2a23-4ffd-b7a0-40702d982e9f admin admin] PATH_INFO: `/v3/projects/256a344e0b4b4170a56df3ca5b5b1655/users/0139d1342072434b9f904a8f4ea385fd/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.456388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e814fdd40e343679ad5eda9d7fbd129'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:45.464331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e814fdd40e343679ad5eda9d7fbd129'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:45.466733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47655eef-2a23-4ffd-b7a0-40702d982e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=256a344e0b4b4170a56df3ca5b5b1655, user_id=0139d1342072434b9f904a8f4ea385fd, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:45.469402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47655eef-2a23-4ffd-b7a0-40702d982e9f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:45.471525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e814fdd40e343679ad5eda9d7fbd129'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:45.472203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d5fb7ec1-c9be-44a7-b7b6-f6856741687f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e814fdd40e343679ad5eda9d7fbd129'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:45.479031 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3569/7167] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:45 2022] DELETE /identity/v3/users/8e814fdd40e343679ad5eda9d7fbd129 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:45.481643 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3599/7168] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:45 2022] PUT /identity/v3/projects/256a344e0b4b4170a56df3ca5b5b1655/users/0139d1342072434b9f904a8f4ea385fd/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:45.484200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f3236-14d9-4808-8256-c09c2d63e759 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.484479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f3236-14d9-4808-8256-c09c2d63e759 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.484865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f3236-14d9-4808-8256-c09c2d63e759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.505427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ca6f3236-14d9-4808-8256-c09c2d63e759 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:45.506672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ca6f3236-14d9-4808-8256-c09c2d63e759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:45.575335 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3570/7169] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:45.576192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ee4d2-b423-4f68-98d0-1dad5233b0d6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.576480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ee4d2-b423-4f68-98d0-1dad5233b0d6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.578108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ee4d2-b423-4f68-98d0-1dad5233b0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.585535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f3236-14d9-4808-8256-c09c2d63e759 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:45.586927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ca6f3236-14d9-4808-8256-c09c2d63e759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:45.607790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-178ee4d2-b423-4f68-98d0-1dad5233b0d6 None None] MFA Rules not processed for user `0139d1342072434b9f904a8f4ea385fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:45.609786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-178ee4d2-b423-4f68-98d0-1dad5233b0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:45.651691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f3236-14d9-4808-8256-c09c2d63e759 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:45.655910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:45.656188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e9299-8bce-4c4e-8bab-b168cafe043f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:45.656841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e9299-8bce-4c4e-8bab-b168cafe043f service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.657186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e9299-8bce-4c4e-8bab-b168cafe043f service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.657504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e9299-8bce-4c4e-8bab-b168cafe043f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.667467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e9299-8bce-4c4e-8bab-b168cafe043f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:45.668924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e9299-8bce-4c4e-8bab-b168cafe043f service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:45.678716 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3571/7170] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:18:45.682493 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3600/7171] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:45.693528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e9299-8bce-4c4e-8bab-b168cafe043f service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:45.694973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-829e9299-8bce-4c4e-8bab-b168cafe043f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:45.703622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:45.711709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e9299-8bce-4c4e-8bab-b168cafe043f service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:45.717127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64677aca-7f71-40d7-9c80-4fda4e84a956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:45.718279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64677aca-7f71-40d7-9c80-4fda4e84a956 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:45.718700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64677aca-7f71-40d7-9c80-4fda4e84a956 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:45.718977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64677aca-7f71-40d7-9c80-4fda4e84a956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:45.729377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64677aca-7f71-40d7-9c80-4fda4e84a956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:45.734281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64677aca-7f71-40d7-9c80-4fda4e84a956 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:45.748372 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3572/7172] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:46.120523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ee4d2-b423-4f68-98d0-1dad5233b0d6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:46.122134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-178ee4d2-b423-4f68-98d0-1dad5233b0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:46.142037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ee4d2-b423-4f68-98d0-1dad5233b0d6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:46.147543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:46.148881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:46.149375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:46.149654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] PATH_INFO: `/v3/projects/891505ffe7af4d07b4e4ec24b0021d9c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:46.150999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891505ffe7af4d07b4e4ec24b0021d9c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:46.153346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:46.276066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891505ffe7af4d07b4e4ec24b0021d9c', 'request_id': 'req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:46.297749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891505ffe7af4d07b4e4ec24b0021d9c', 'request_id': 'req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:46.307311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891505ffe7af4d07b4e4ec24b0021d9c', 'request_id': 'req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:46.330812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] The token cache is being invalidate because project 891505ffe7af4d07b4e4ec24b0021d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:46.331240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:46.332283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:46.335713 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3601/7173] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:46 2022] DELETE /identity/v3/projects/891505ffe7af4d07b4e4ec24b0021d9c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:46.341927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64677aca-7f71-40d7-9c80-4fda4e84a956 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:46.359877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64677aca-7f71-40d7-9c80-4fda4e84a956 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:46.456208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:46.456831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:46.457278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:46.457553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] PATH_INFO: `/v3/users/983dd47443f04fb5b868d5c1c17d7d4b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:46.458969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=983dd47443f04fb5b868d5c1c17d7d4b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:46.461241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:46.562663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983dd47443f04fb5b868d5c1c17d7d4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:46.581253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983dd47443f04fb5b868d5c1c17d7d4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:46.592148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983dd47443f04fb5b868d5c1c17d7d4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:46.609998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983dd47443f04fb5b868d5c1c17d7d4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:46.612297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ccfb792-224c-4c3f-af13-d83abefcd1d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983dd47443f04fb5b868d5c1c17d7d4b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:46.623246 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3573/7174] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:46 2022] DELETE /identity/v3/users/983dd47443f04fb5b868d5c1c17d7d4b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:46.999836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:47.002024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:47.029324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736b5be-7fa3-445f-aca7-ae3b9b1d1542 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:47.106241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:47.106878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:47.107147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:47.107411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] PATH_INFO: `/v3/projects/e14f4b6714104fa8a44169447aad22b9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:47.109465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e14f4b6714104fa8a44169447aad22b9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:47.111327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:47.167305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e14f4b6714104fa8a44169447aad22b9', 'request_id': 'req-cbf731d1-7867-4d22-8582-8e9b42a4a58a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:47.177537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e14f4b6714104fa8a44169447aad22b9', 'request_id': 'req-cbf731d1-7867-4d22-8582-8e9b42a4a58a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:47.184522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e14f4b6714104fa8a44169447aad22b9', 'request_id': 'req-cbf731d1-7867-4d22-8582-8e9b42a4a58a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:47.192763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:47.207695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] The token cache is being invalidate because project e14f4b6714104fa8a44169447aad22b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:47.208268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:47.208803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cbf731d1-7867-4d22-8582-8e9b42a4a58a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:47.215658 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3602/7175] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:46 2022] DELETE /identity/v3/projects/e14f4b6714104fa8a44169447aad22b9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:47.224183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ce2c8-b6fa-43c8-83d4-41ae81234110 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:47.224466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ce2c8-b6fa-43c8-83d4-41ae81234110 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:47.224739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ce2c8-b6fa-43c8-83d4-41ae81234110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:47.247272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cf5ce2c8-b6fa-43c8-83d4-41ae81234110 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:47.249427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cf5ce2c8-b6fa-43c8-83d4-41ae81234110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:47.346456 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3574/7176] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:47.349809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8b352-4dd1-489a-815d-48d5365cc0a7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:47.351151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8b352-4dd1-489a-815d-48d5365cc0a7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:47.351649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8b352-4dd1-489a-815d-48d5365cc0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:47.374691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-64a8b352-4dd1-489a-815d-48d5365cc0a7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:47.377111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-64a8b352-4dd1-489a-815d-48d5365cc0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:47.386561 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3603/7177] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:47.391438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ce2c8-b6fa-43c8-83d4-41ae81234110 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:47.419301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ce2c8-b6fa-43c8-83d4-41ae81234110 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:47.432102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34b52a-a43a-48a7-b984-4aec8521dcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:47.432720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34b52a-a43a-48a7-b984-4aec8521dcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:47.432993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34b52a-a43a-48a7-b984-4aec8521dcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:47.433683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34b52a-a43a-48a7-b984-4aec8521dcf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:47.434387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34b52a-a43a-48a7-b984-4aec8521dcf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:47.434701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34b52a-a43a-48a7-b984-4aec8521dcf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:47.438310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34b52a-a43a-48a7-b984-4aec8521dcf8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:47.452230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:47.468207 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3575/7178] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:47.472253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8b352-4dd1-489a-815d-48d5365cc0a7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:47.483742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8b352-4dd1-489a-815d-48d5365cc0a7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:47.492951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ab0d-b744-45ca-b406-b125c1d10008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:47.493888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ab0d-b744-45ca-b406-b125c1d10008 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:47.494263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ab0d-b744-45ca-b406-b125c1d10008 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:47.494704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ab0d-b744-45ca-b406-b125c1d10008 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:47.495642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3ab0d-b744-45ca-b406-b125c1d10008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:47.500117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3ab0d-b744-45ca-b406-b125c1d10008 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:47.547290 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3604/7179] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:47 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:47.552118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34b52a-a43a-48a7-b984-4aec8521dcf8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:47.566829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34b52a-a43a-48a7-b984-4aec8521dcf8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:47.645349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f978-af75-4d3c-9970-dd302644c659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:47.646215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f978-af75-4d3c-9970-dd302644c659 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:47.646640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f978-af75-4d3c-9970-dd302644c659 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:47.647252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f978-af75-4d3c-9970-dd302644c659 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:47.648053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1f978-af75-4d3c-9970-dd302644c659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:47.650896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1f978-af75-4d3c-9970-dd302644c659 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:47.683228 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3576/7180] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:47 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:47.688086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ab0d-b744-45ca-b406-b125c1d10008 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:47.705692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ab0d-b744-45ca-b406-b125c1d10008 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:47.714417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027203ac-70ba-4b49-93b9-0ef17a4d8efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:47.715660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027203ac-70ba-4b49-93b9-0ef17a4d8efd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:47.716119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027203ac-70ba-4b49-93b9-0ef17a4d8efd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:47.716661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027203ac-70ba-4b49-93b9-0ef17a4d8efd admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:47.718078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027203ac-70ba-4b49-93b9-0ef17a4d8efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:47.720531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027203ac-70ba-4b49-93b9-0ef17a4d8efd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:47.735441 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3605/7181] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:47.739699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f978-af75-4d3c-9970-dd302644c659 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:47.754894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f978-af75-4d3c-9970-dd302644c659 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:47.767513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb44852-62f3-40c2-b200-8e60cae34f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:47.768621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44852-62f3-40c2-b200-8e60cae34f6b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:47.770310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44852-62f3-40c2-b200-8e60cae34f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:47.770879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44852-62f3-40c2-b200-8e60cae34f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:47.773189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb44852-62f3-40c2-b200-8e60cae34f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:47.774897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb44852-62f3-40c2-b200-8e60cae34f6b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:47.793021 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3577/7182] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:47.796443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027203ac-70ba-4b49-93b9-0ef17a4d8efd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:47.826474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027203ac-70ba-4b49-93b9-0ef17a4d8efd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:47.839011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef729a-8b2f-4dc2-8d61-2431bbc14995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:47.841830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef729a-8b2f-4dc2-8d61-2431bbc14995 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:47.843698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef729a-8b2f-4dc2-8d61-2431bbc14995 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:47.844352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef729a-8b2f-4dc2-8d61-2431bbc14995 admin admin] PATH_INFO: `/v3/projects/8992ad73e72949fe80c3e8f630a8655d/users/ad9eb2a72a114db6b36071e6af6c71aa/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:47.876537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef729a-8b2f-4dc2-8d61-2431bbc14995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8992ad73e72949fe80c3e8f630a8655d, user_id=ad9eb2a72a114db6b36071e6af6c71aa, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:47.879536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef729a-8b2f-4dc2-8d61-2431bbc14995 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:47.913558 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3606/7183] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:47 2022] PUT /identity/v3/projects/8992ad73e72949fe80c3e8f630a8655d/users/ad9eb2a72a114db6b36071e6af6c71aa/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:47.926582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277af461-a8eb-4096-9963-1f00196f5b93 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:47.926957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277af461-a8eb-4096-9963-1f00196f5b93 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:47.927667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277af461-a8eb-4096-9963-1f00196f5b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:47.982513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-277af461-a8eb-4096-9963-1f00196f5b93 None None] MFA Rules not processed for user `ad9eb2a72a114db6b36071e6af6c71aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:47.984556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-277af461-a8eb-4096-9963-1f00196f5b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:48.188943 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3578/7184] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:48.215943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef729a-8b2f-4dc2-8d61-2431bbc14995 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:48.226476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8fef729a-8b2f-4dc2-8d61-2431bbc14995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:48.370830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef729a-8b2f-4dc2-8d61-2431bbc14995 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:48.380536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838d2cc-c534-441c-8603-25c04639880e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:48.381310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d2cc-c534-441c-8603-25c04639880e service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:48.381525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d2cc-c534-441c-8603-25c04639880e service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:48.381789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d2cc-c534-441c-8603-25c04639880e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:48.417343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838d2cc-c534-441c-8603-25c04639880e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:48.418940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838d2cc-c534-441c-8603-25c04639880e service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:48.442443 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3607/7185] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:48.833197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd479ef4-2115-4b81-9042-ab3e8779b15a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:48.833721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd479ef4-2115-4b81-9042-ab3e8779b15a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:48.834163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd479ef4-2115-4b81-9042-ab3e8779b15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:48.957892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fd479ef4-2115-4b81-9042-ab3e8779b15a None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:48.961723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fd479ef4-2115-4b81-9042-ab3e8779b15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:49.036237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:49.137832 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3579/7186] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:18:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:49.238227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838d2cc-c534-441c-8603-25c04639880e service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:49.240370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7838d2cc-c534-441c-8603-25c04639880e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:49.469677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838d2cc-c534-441c-8603-25c04639880e service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:49.496783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b67456-5c36-435e-906e-a716827c37e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:49.498107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b67456-5c36-435e-906e-a716827c37e4 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:49.499035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b67456-5c36-435e-906e-a716827c37e4 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:49.499550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b67456-5c36-435e-906e-a716827c37e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:49.517900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:49.555159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b67456-5c36-435e-906e-a716827c37e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:49.556780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b67456-5c36-435e-906e-a716827c37e4 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:49.601805 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3608/7187] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:18:50.192851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce384d1-cedb-43ad-b5ab-63ac6bc49180 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:50.194384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce384d1-cedb-43ad-b5ab-63ac6bc49180 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:50.194909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce384d1-cedb-43ad-b5ab-63ac6bc49180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:50.222555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7ce384d1-cedb-43ad-b5ab-63ac6bc49180 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:50.225747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7ce384d1-cedb-43ad-b5ab-63ac6bc49180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:50.258309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b67456-5c36-435e-906e-a716827c37e4 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:50.268155 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3580/7188] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:18:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:50.277575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b67456-5c36-435e-906e-a716827c37e4 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:50.286061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3380b80-e951-461b-8c77-14ed4701fff9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:50.286877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380b80-e951-461b-8c77-14ed4701fff9 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:50.287302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380b80-e951-461b-8c77-14ed4701fff9 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:50.287833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380b80-e951-461b-8c77-14ed4701fff9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:50.299906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3380b80-e951-461b-8c77-14ed4701fff9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:50.301594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3380b80-e951-461b-8c77-14ed4701fff9 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:50.342195 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3609/7189] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:18:50.816085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5f04e-22d6-42de-abed-22db6f92fae7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:50.816396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5f04e-22d6-42de-abed-22db6f92fae7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:50.816683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5f04e-22d6-42de-abed-22db6f92fae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:50.856392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-56d5f04e-22d6-42de-abed-22db6f92fae7 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:50.856392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-56d5f04e-22d6-42de-abed-22db6f92fae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:50.869422 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3581/7190] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:18:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:50.883010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3380b80-e951-461b-8c77-14ed4701fff9 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:50.909606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3380b80-e951-461b-8c77-14ed4701fff9 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:50.923290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af28d-17e3-4591-b407-325cf5579812 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:50.925526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af28d-17e3-4591-b407-325cf5579812 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:50.925987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af28d-17e3-4591-b407-325cf5579812 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:50.926997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af28d-17e3-4591-b407-325cf5579812 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:50.954867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37af28d-17e3-4591-b407-325cf5579812 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:50.956499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37af28d-17e3-4591-b407-325cf5579812 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:50.976384 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3610/7191] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:18:51.490616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c43b0-6653-4ca2-a6ef-94f29528a83d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:51.491577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c43b0-6653-4ca2-a6ef-94f29528a83d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:51.492026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c43b0-6653-4ca2-a6ef-94f29528a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:51.523021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-616c43b0-6653-4ca2-a6ef-94f29528a83d None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:51.540044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-616c43b0-6653-4ca2-a6ef-94f29528a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:51.564620 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3582/7192] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:18:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:51.583063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af28d-17e3-4591-b407-325cf5579812 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:51.613580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af28d-17e3-4591-b407-325cf5579812 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:51.623908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae08ae1-004a-490b-897b-33341ca9da8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:51.624678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae08ae1-004a-490b-897b-33341ca9da8b service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:51.625016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae08ae1-004a-490b-897b-33341ca9da8b service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:51.625398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae08ae1-004a-490b-897b-33341ca9da8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:51.648646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae08ae1-004a-490b-897b-33341ca9da8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:51.650631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae08ae1-004a-490b-897b-33341ca9da8b service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:51.679981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c43b0-6653-4ca2-a6ef-94f29528a83d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:51.681536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-616c43b0-6653-4ca2-a6ef-94f29528a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:51.686763 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3611/7193] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:18:51.840381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c43b0-6653-4ca2-a6ef-94f29528a83d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:51.847996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:51.848912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:51.849230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:51.849496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] PATH_INFO: `/v3/users/1ae48516992c42578ec24ab334785ad1` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:51.851778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ae48516992c42578ec24ab334785ad1)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:51.854478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:51.956493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ae48516992c42578ec24ab334785ad1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:51.978271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ae48516992c42578ec24ab334785ad1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:51.996600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ae48516992c42578ec24ab334785ad1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:52.008040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ae48516992c42578ec24ab334785ad1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:52.008956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-aa017f22-c4bb-4d71-9fff-2eb42c56bf25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ae48516992c42578ec24ab334785ad1'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:52.025732 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3583/7194] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:51 2022] DELETE /identity/v3/users/1ae48516992c42578ec24ab334785ad1 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:52.036461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8bebb-3d5e-4698-be98-b08cbc0ac3b4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:52.036742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8bebb-3d5e-4698-be98-b08cbc0ac3b4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:52.040940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8bebb-3d5e-4698-be98-b08cbc0ac3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:52.094081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-01c8bebb-3d5e-4698-be98-b08cbc0ac3b4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:52.095390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-01c8bebb-3d5e-4698-be98-b08cbc0ac3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:52.145368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4a8fa-3ed3-4571-90f0-d83326e25422 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:52.145592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4a8fa-3ed3-4571-90f0-d83326e25422 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:52.145989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4a8fa-3ed3-4571-90f0-d83326e25422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:52.223455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-45d4a8fa-3ed3-4571-90f0-d83326e25422 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:52.226617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-45d4a8fa-3ed3-4571-90f0-d83326e25422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:52.237837 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3612/7195] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:52.261755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8bebb-3d5e-4698-be98-b08cbc0ac3b4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:52.275837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-01c8bebb-3d5e-4698-be98-b08cbc0ac3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:52.345503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:52.392625 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3584/7196] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:52.410604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4a8fa-3ed3-4571-90f0-d83326e25422 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:52.414069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-45d4a8fa-3ed3-4571-90f0-d83326e25422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:52.446504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8bebb-3d5e-4698-be98-b08cbc0ac3b4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:52.452394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fe5f6-94d8-4254-959d-698a17ebe0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:52.453359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fe5f6-94d8-4254-959d-698a17ebe0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:52.453786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fe5f6-94d8-4254-959d-698a17ebe0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:52.454503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fe5f6-94d8-4254-959d-698a17ebe0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:52.466956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fe5f6-94d8-4254-959d-698a17ebe0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:52.468772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fe5f6-94d8-4254-959d-698a17ebe0a0 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:52.482565 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3613/7197] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:52.548209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4a8fa-3ed3-4571-90f0-d83326e25422 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:52.555089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5d61-0995-4282-bf85-7d93c8dbb21e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:52.556061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5d61-0995-4282-bf85-7d93c8dbb21e service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:52.556432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5d61-0995-4282-bf85-7d93c8dbb21e service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:52.556706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5d61-0995-4282-bf85-7d93c8dbb21e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:52.569864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da5d61-0995-4282-bf85-7d93c8dbb21e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:52.571781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da5d61-0995-4282-bf85-7d93c8dbb21e service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:52.603357 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3585/7198] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:18:52.956945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fe5f6-94d8-4254-959d-698a17ebe0a0 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:52.960534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a66fe5f6-94d8-4254-959d-698a17ebe0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:52.978945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fe5f6-94d8-4254-959d-698a17ebe0a0 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:52.987098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:52.990001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:52.990793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:52.993194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] PATH_INFO: `/v3/projects/0f4379801b4040c8b857afe4038e4ff7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:52.996735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f4379801b4040c8b857afe4038e4ff7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:53.002120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:53.043040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034684bb-1cc7-4765-914d-cbd12d3fdb34 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.043669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034684bb-1cc7-4765-914d-cbd12d3fdb34 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.044042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034684bb-1cc7-4765-914d-cbd12d3fdb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:53.072135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-034684bb-1cc7-4765-914d-cbd12d3fdb34 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:53.073886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-034684bb-1cc7-4765-914d-cbd12d3fdb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:53.117712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4379801b4040c8b857afe4038e4ff7', 'request_id': 'req-d893fcd4-9e49-4607-8d88-6d5fcfba739e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:53.120873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:53.133872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4379801b4040c8b857afe4038e4ff7', 'request_id': 'req-d893fcd4-9e49-4607-8d88-6d5fcfba739e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:53.143468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4379801b4040c8b857afe4038e4ff7', 'request_id': 'req-d893fcd4-9e49-4607-8d88-6d5fcfba739e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:53.167603 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3586/7199] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:53.173559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] The token cache is being invalidate because project 0f4379801b4040c8b857afe4038e4ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:53.174122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:53.175770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d893fcd4-9e49-4607-8d88-6d5fcfba739e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:53.177754 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3614/7200] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:52 2022] DELETE /identity/v3/projects/0f4379801b4040c8b857afe4038e4ff7 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:53.182112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034684bb-1cc7-4765-914d-cbd12d3fdb34 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:53.183967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f221c-9a8f-4c82-a2fd-cfa123733841 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.185405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f221c-9a8f-4c82-a2fd-cfa123733841 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.185929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f221c-9a8f-4c82-a2fd-cfa123733841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:53.224710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c31f221c-9a8f-4c82-a2fd-cfa123733841 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:53.226297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c31f221c-9a8f-4c82-a2fd-cfa123733841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:53.308296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034684bb-1cc7-4765-914d-cbd12d3fdb34 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:53.314713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cbc20-25da-4446-ad31-0fdeb415eb03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:53.315317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbc20-25da-4446-ad31-0fdeb415eb03 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.315591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbc20-25da-4446-ad31-0fdeb415eb03 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.315884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbc20-25da-4446-ad31-0fdeb415eb03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:53.317875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-415cbc20-25da-4446-ad31-0fdeb415eb03 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:53.322417 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3615/7201] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:53.325584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036c5e-cd4c-47ff-a8f8-59e016664633 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.325994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036c5e-cd4c-47ff-a8f8-59e016664633 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.326306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036c5e-cd4c-47ff-a8f8-59e016664633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:53.336272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cbc20-25da-4446-ad31-0fdeb415eb03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:53.338139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cbc20-25da-4446-ad31-0fdeb415eb03 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:53.351919 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3587/7202] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:18:53.355296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-55036c5e-cd4c-47ff-a8f8-59e016664633 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:53.357199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-55036c5e-cd4c-47ff-a8f8-59e016664633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:53.368025 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3616/7203] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:53.374070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cbc20-25da-4446-ad31-0fdeb415eb03 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:53.395651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cbc20-25da-4446-ad31-0fdeb415eb03 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:53.402625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae56cb4-3014-430a-bcfd-f754364ad366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:53.403321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56cb4-3014-430a-bcfd-f754364ad366 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.403655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56cb4-3014-430a-bcfd-f754364ad366 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.403945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56cb4-3014-430a-bcfd-f754364ad366 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:53.404619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae56cb4-3014-430a-bcfd-f754364ad366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:53.404955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae56cb4-3014-430a-bcfd-f754364ad366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:53.407829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae56cb4-3014-430a-bcfd-f754364ad366 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:53.409662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:53.438547 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3588/7204] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:18:53.443155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036c5e-cd4c-47ff-a8f8-59e016664633 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:53.458904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036c5e-cd4c-47ff-a8f8-59e016664633 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:53.475679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af534323-9ed6-4918-8e3c-2f46f66b648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:53.476290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af534323-9ed6-4918-8e3c-2f46f66b648a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.476754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af534323-9ed6-4918-8e3c-2f46f66b648a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.477213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af534323-9ed6-4918-8e3c-2f46f66b648a admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:53.477988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af534323-9ed6-4918-8e3c-2f46f66b648a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:53.480216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af534323-9ed6-4918-8e3c-2f46f66b648a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:53.544760 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3617/7205] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:53 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:53.551667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae56cb4-3014-430a-bcfd-f754364ad366 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:53.572465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae56cb4-3014-430a-bcfd-f754364ad366 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:53.742229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471a733-20a3-4721-8d33-98b893c2370f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:53.746068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471a733-20a3-4721-8d33-98b893c2370f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.746068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471a733-20a3-4721-8d33-98b893c2370f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.746068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471a733-20a3-4721-8d33-98b893c2370f admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:53.746068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471a733-20a3-4721-8d33-98b893c2370f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:53.749119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471a733-20a3-4721-8d33-98b893c2370f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:53.806175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af534323-9ed6-4918-8e3c-2f46f66b648a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:53.807509 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3589/7206] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:53 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:53.820748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af534323-9ed6-4918-8e3c-2f46f66b648a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:53.830005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906cbfc-7272-4db0-9bae-6f600fa4e1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:53.831046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906cbfc-7272-4db0-9bae-6f600fa4e1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.831558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906cbfc-7272-4db0-9bae-6f600fa4e1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.832076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906cbfc-7272-4db0-9bae-6f600fa4e1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:53.833058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906cbfc-7272-4db0-9bae-6f600fa4e1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:53.835705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906cbfc-7272-4db0-9bae-6f600fa4e1b5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:53.860532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471a733-20a3-4721-8d33-98b893c2370f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:53.861758 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3618/7207] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:53.871768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471a733-20a3-4721-8d33-98b893c2370f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:53.896178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906cbfc-7272-4db0-9bae-6f600fa4e1b5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:53.898104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e906cbfc-7272-4db0-9bae-6f600fa4e1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:53.899608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd81b0-6386-46ca-b17f-4d3769e142bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:53.900299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd81b0-6386-46ca-b17f-4d3769e142bb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.900663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd81b0-6386-46ca-b17f-4d3769e142bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.901006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd81b0-6386-46ca-b17f-4d3769e142bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:53.901840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd81b0-6386-46ca-b17f-4d3769e142bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:53.903864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd81b0-6386-46ca-b17f-4d3769e142bb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:53.923816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906cbfc-7272-4db0-9bae-6f600fa4e1b5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:53.924691 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3590/7208] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:53.929023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd81b0-6386-46ca-b17f-4d3769e142bb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:53.935131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:53.936564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.938005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.938341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] PATH_INFO: `/v3/users/51503af6da0e423184b9c0917f62dfbb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:53.942258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51503af6da0e423184b9c0917f62dfbb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:53.944347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:53.955622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd81b0-6386-46ca-b17f-4d3769e142bb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:53.966621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e14b2f-2eca-42db-bbe3-254067ad5db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:53.973407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14b2f-2eca-42db-bbe3-254067ad5db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:53.973760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14b2f-2eca-42db-bbe3-254067ad5db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:53.974047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14b2f-2eca-42db-bbe3-254067ad5db2 admin admin] PATH_INFO: `/v3/projects/2c132f2961594244a86c8a610462992e/users/026771ec28444739afc13c47339af7b0/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.025850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e14b2f-2eca-42db-bbe3-254067ad5db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c132f2961594244a86c8a610462992e, user_id=026771ec28444739afc13c47339af7b0, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:54.029924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e14b2f-2eca-42db-bbe3-254067ad5db2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:54.069087 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3591/7209] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:53 2022] PUT /identity/v3/projects/2c132f2961594244a86c8a610462992e/users/026771ec28444739afc13c47339af7b0/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:54.078246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c544ee-601f-477d-bfbe-0e10d57047ff None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:54.078533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c544ee-601f-477d-bfbe-0e10d57047ff None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:54.078805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c544ee-601f-477d-bfbe-0e10d57047ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.121187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51503af6da0e423184b9c0917f62dfbb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:54.146261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51503af6da0e423184b9c0917f62dfbb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:54.157439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c4c544ee-601f-477d-bfbe-0e10d57047ff None None] MFA Rules not processed for user `026771ec28444739afc13c47339af7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:54.159702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c4c544ee-601f-477d-bfbe-0e10d57047ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:54.162514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51503af6da0e423184b9c0917f62dfbb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:54.175657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51503af6da0e423184b9c0917f62dfbb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:54.177022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a2e4c65f-e68d-4acd-9e0b-c635b7de5a78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51503af6da0e423184b9c0917f62dfbb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:54.191901 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3619/7210] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:53 2022] DELETE /identity/v3/users/51503af6da0e423184b9c0917f62dfbb => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:54.204221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949e519-fe3a-46d0-9dec-719e7dc052c3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:54.204956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949e519-fe3a-46d0-9dec-719e7dc052c3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:54.205644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949e519-fe3a-46d0-9dec-719e7dc052c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.240477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c949e519-fe3a-46d0-9dec-719e7dc052c3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:54.242118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c949e519-fe3a-46d0-9dec-719e7dc052c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:54.274347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:54.307370 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3592/7211] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:54.313174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c544ee-601f-477d-bfbe-0e10d57047ff None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:54.315792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c4c544ee-601f-477d-bfbe-0e10d57047ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:54.401623 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3620/7212] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:54.410955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949e519-fe3a-46d0-9dec-719e7dc052c3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:54.412719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c949e519-fe3a-46d0-9dec-719e7dc052c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:54.417900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:54.461470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c544ee-601f-477d-bfbe-0e10d57047ff None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:54.471676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e8a6e-b3fa-4e0f-a344-bd4665ac617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:54.472416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8a6e-b3fa-4e0f-a344-bd4665ac617d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:54.472701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8a6e-b3fa-4e0f-a344-bd4665ac617d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:54.473120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8a6e-b3fa-4e0f-a344-bd4665ac617d admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.474049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e8a6e-b3fa-4e0f-a344-bd4665ac617d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:54.477237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e8a6e-b3fa-4e0f-a344-bd4665ac617d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:54.530461 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3593/7213] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:54 2022] POST /identity/v3/projects => generated 389 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:54.533323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949e519-fe3a-46d0-9dec-719e7dc052c3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:54.534397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e8a6e-b3fa-4e0f-a344-bd4665ac617d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:54.604475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659e002-3859-4b6a-8ba1-61a4c4b0999c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:54.605314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e002-3859-4b6a-8ba1-61a4c4b0999c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:54.605596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e002-3859-4b6a-8ba1-61a4c4b0999c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:54.606382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e002-3859-4b6a-8ba1-61a4c4b0999c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.665301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e8a6e-b3fa-4e0f-a344-bd4665ac617d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:54.673072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa69e7-aa0d-486d-834d-5a1e7b2e357d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:54.673927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa69e7-aa0d-486d-834d-5a1e7b2e357d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:54.674398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa69e7-aa0d-486d-834d-5a1e7b2e357d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:54.674845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa69e7-aa0d-486d-834d-5a1e7b2e357d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.675697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa69e7-aa0d-486d-834d-5a1e7b2e357d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:54.678087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa69e7-aa0d-486d-834d-5a1e7b2e357d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:54.715483 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3594/7214] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:54 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:54.720389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa69e7-aa0d-486d-834d-5a1e7b2e357d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:54.721229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659e002-3859-4b6a-8ba1-61a4c4b0999c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:54.722960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659e002-3859-4b6a-8ba1-61a4c4b0999c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:54.739105 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3621/7215] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:18:54.741479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa69e7-aa0d-486d-834d-5a1e7b2e357d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:54.750766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cd5c3-0f3b-4f82-92f4-4bea860bbf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:54.752016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cd5c3-0f3b-4f82-92f4-4bea860bbf70 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:54.752547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cd5c3-0f3b-4f82-92f4-4bea860bbf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:54.753055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cd5c3-0f3b-4f82-92f4-4bea860bbf70 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.753973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cd5c3-0f3b-4f82-92f4-4bea860bbf70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:54.755795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cd5c3-0f3b-4f82-92f4-4bea860bbf70 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:54.769223 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3595/7216] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:54.773103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659e002-3859-4b6a-8ba1-61a4c4b0999c service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:54.785311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659e002-3859-4b6a-8ba1-61a4c4b0999c service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:54.791348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2474fc-357f-41c0-aa44-73a4a3722f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:54.792386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2474fc-357f-41c0-aa44-73a4a3722f57 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:54.792635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2474fc-357f-41c0-aa44-73a4a3722f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:54.792961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2474fc-357f-41c0-aa44-73a4a3722f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.793689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2474fc-357f-41c0-aa44-73a4a3722f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:54.796053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2474fc-357f-41c0-aa44-73a4a3722f57 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:54.808205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3622/7217] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:54.813554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cd5c3-0f3b-4f82-92f4-4bea860bbf70 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:54.833020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cd5c3-0f3b-4f82-92f4-4bea860bbf70 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:54.838529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535b847-7f8f-43c7-9c05-8da94da8fc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:54.839263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535b847-7f8f-43c7-9c05-8da94da8fc5f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:54.839610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535b847-7f8f-43c7-9c05-8da94da8fc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:54.839941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535b847-7f8f-43c7-9c05-8da94da8fc5f admin admin] PATH_INFO: `/v3/projects/80896dfa8a424554a0c1d252de45fadd/users/3acbc3744de34684a9553e3ee5cb5133/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.859069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535b847-7f8f-43c7-9c05-8da94da8fc5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=80896dfa8a424554a0c1d252de45fadd, user_id=3acbc3744de34684a9553e3ee5cb5133, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:54.866280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535b847-7f8f-43c7-9c05-8da94da8fc5f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:54.887006 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3596/7218] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:54 2022] PUT /identity/v3/projects/80896dfa8a424554a0c1d252de45fadd/users/3acbc3744de34684a9553e3ee5cb5133/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:54.892438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ce4aa-8b88-4dc7-917b-f0a21bef0b30 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:54.892733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ce4aa-8b88-4dc7-917b-f0a21bef0b30 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:54.893071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ce4aa-8b88-4dc7-917b-f0a21bef0b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.908471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b473f-1d8c-4645-8a54-f1aba988bb89 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:54.908808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b473f-1d8c-4645-8a54-f1aba988bb89 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:54.909104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b473f-1d8c-4645-8a54-f1aba988bb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:54.917969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:54.931193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-052ce4aa-8b88-4dc7-917b-f0a21bef0b30 None None] MFA Rules not processed for user `3acbc3744de34684a9553e3ee5cb5133`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:54.933417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-052ce4aa-8b88-4dc7-917b-f0a21bef0b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:54.939135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dd3b473f-1d8c-4645-8a54-f1aba988bb89 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:54.940988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dd3b473f-1d8c-4645-8a54-f1aba988bb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:55.014497 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3597/7219] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:55.026786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b473f-1d8c-4645-8a54-f1aba988bb89 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:55.028211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dd3b473f-1d8c-4645-8a54-f1aba988bb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:55.034454 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3623/7220] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:55.039196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ce4aa-8b88-4dc7-917b-f0a21bef0b30 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:55.061930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:55.116095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ce4aa-8b88-4dc7-917b-f0a21bef0b30 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:55.123711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8d144-9d68-49ee-9312-b2fb16369690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:55.124480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8d144-9d68-49ee-9312-b2fb16369690 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:55.124778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8d144-9d68-49ee-9312-b2fb16369690 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:55.125204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8d144-9d68-49ee-9312-b2fb16369690 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:55.125870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8d144-9d68-49ee-9312-b2fb16369690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:55.126462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b473f-1d8c-4645-8a54-f1aba988bb89 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:55.128213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8d144-9d68-49ee-9312-b2fb16369690 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:55.131790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8707e-123b-4170-b041-3600b8e340c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:55.132874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8707e-123b-4170-b041-3600b8e340c5 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:55.132991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8707e-123b-4170-b041-3600b8e340c5 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:55.133206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8707e-123b-4170-b041-3600b8e340c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:55.145587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8707e-123b-4170-b041-3600b8e340c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:55.147413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8707e-123b-4170-b041-3600b8e340c5 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:55.165648 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3624/7221] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:55 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:18:55.169590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8d144-9d68-49ee-9312-b2fb16369690 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:55.174532 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3598/7222] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:18:55.269430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8707e-123b-4170-b041-3600b8e340c5 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:55.271060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-27c8707e-123b-4170-b041-3600b8e340c5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:55.278053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8d144-9d68-49ee-9312-b2fb16369690 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:55.289476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf89c0-7ae5-4de3-a1bd-07cb5272ae29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:55.290107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf89c0-7ae5-4de3-a1bd-07cb5272ae29 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:55.290384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf89c0-7ae5-4de3-a1bd-07cb5272ae29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:55.293310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf89c0-7ae5-4de3-a1bd-07cb5272ae29 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:55.293500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8707e-123b-4170-b041-3600b8e340c5 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:55.308795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf89c0-7ae5-4de3-a1bd-07cb5272ae29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:55.311985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf89c0-7ae5-4de3-a1bd-07cb5272ae29 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:55.314668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:55.315310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:55.315581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:55.315845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] PATH_INFO: `/v3/projects/ee8acd6042024c0e8add968e6af51da5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:55.318907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8acd6042024c0e8add968e6af51da5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:55.320850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:55.352622 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3625/7223] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:55 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:55.356193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf89c0-7ae5-4de3-a1bd-07cb5272ae29 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:55.369424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf89c0-7ae5-4de3-a1bd-07cb5272ae29 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:55.383122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c2de6-1105-46aa-9839-2bf2e44211f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:55.383982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c2de6-1105-46aa-9839-2bf2e44211f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:55.384438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c2de6-1105-46aa-9839-2bf2e44211f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:55.385549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c2de6-1105-46aa-9839-2bf2e44211f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:55.388252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c2de6-1105-46aa-9839-2bf2e44211f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:55.396903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c2de6-1105-46aa-9839-2bf2e44211f4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:55.423984 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3626/7224] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:55.429416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c2de6-1105-46aa-9839-2bf2e44211f4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:55.437950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8acd6042024c0e8add968e6af51da5', 'request_id': 'req-f2aa0999-ac30-43e2-88c1-9370580c72d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:55.443470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:55.458130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c2de6-1105-46aa-9839-2bf2e44211f4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:55.458830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8acd6042024c0e8add968e6af51da5', 'request_id': 'req-f2aa0999-ac30-43e2-88c1-9370580c72d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:55.468987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8acd6042024c0e8add968e6af51da5', 'request_id': 'req-f2aa0999-ac30-43e2-88c1-9370580c72d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:55.488053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] The token cache is being invalidate because project ee8acd6042024c0e8add968e6af51da5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:55.488532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:55.489263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f2aa0999-ac30-43e2-88c1-9370580c72d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:55.493940 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3599/7225] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:55 2022] DELETE /identity/v3/projects/ee8acd6042024c0e8add968e6af51da5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:18:55.493940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba2c82-257d-4923-a694-5bcf35da0ec2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:55.493940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba2c82-257d-4923-a694-5bcf35da0ec2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:55.493940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba2c82-257d-4923-a694-5bcf35da0ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:55.517536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e2ba2c82-257d-4923-a694-5bcf35da0ec2 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:55.518756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e2ba2c82-257d-4923-a694-5bcf35da0ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:55.561483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9057d3d-bf68-44a0-b6cd-131c659fcc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:55.562217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9057d3d-bf68-44a0-b6cd-131c659fcc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:55.562494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9057d3d-bf68-44a0-b6cd-131c659fcc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:55.562824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9057d3d-bf68-44a0-b6cd-131c659fcc70 admin admin] PATH_INFO: `/v3/projects/8513f1a4d8654e39a613bb0f269cdfba/users/5fd48a1a6e2e4954bf874b36cffca9c8/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:55.585392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9057d3d-bf68-44a0-b6cd-131c659fcc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8513f1a4d8654e39a613bb0f269cdfba, user_id=5fd48a1a6e2e4954bf874b36cffca9c8, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:55.589613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9057d3d-bf68-44a0-b6cd-131c659fcc70 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:55.599894 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3600/7226] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:18:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:55.600929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba2c82-257d-4923-a694-5bcf35da0ec2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:55.603209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e2ba2c82-257d-4923-a694-5bcf35da0ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:55.616673 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3627/7227] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:55 2022] PUT /identity/v3/projects/8513f1a4d8654e39a613bb0f269cdfba/users/5fd48a1a6e2e4954bf874b36cffca9c8/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:55.621806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9057d3d-bf68-44a0-b6cd-131c659fcc70 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:55.624406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f9057d3d-bf68-44a0-b6cd-131c659fcc70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:55.629853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba2c82-257d-4923-a694-5bcf35da0ec2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:55.679834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:55.706171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:55.706808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:55.707130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:55.707405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] PATH_INFO: `/v3/users/5edbb1f362ec4d329c8a29c7a6b6f408` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:55.708604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=5edbb1f362ec4d329c8a29c7a6b6f408)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:55.711335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:55.728407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9057d3d-bf68-44a0-b6cd-131c659fcc70 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:55.736241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392f6b6-ec07-43aa-93e8-f8072088fb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:55.737110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392f6b6-ec07-43aa-93e8-f8072088fb3c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:55.737373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392f6b6-ec07-43aa-93e8-f8072088fb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:55.737625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392f6b6-ec07-43aa-93e8-f8072088fb3c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:55.738287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392f6b6-ec07-43aa-93e8-f8072088fb3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:55.739842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392f6b6-ec07-43aa-93e8-f8072088fb3c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:55.756269 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3628/7228] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:55.756269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392f6b6-ec07-43aa-93e8-f8072088fb3c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:55.758472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4392f6b6-ec07-43aa-93e8-f8072088fb3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:55.782615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5edbb1f362ec4d329c8a29c7a6b6f408'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:55.800932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5edbb1f362ec4d329c8a29c7a6b6f408'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:55.815388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5edbb1f362ec4d329c8a29c7a6b6f408'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:55.825310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5edbb1f362ec4d329c8a29c7a6b6f408'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:55.826849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5edbb1f362ec4d329c8a29c7a6b6f408'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:55.835595 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3601/7229] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:55 2022] DELETE /identity/v3/users/5edbb1f362ec4d329c8a29c7a6b6f408 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:55.836264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:55.838870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:55.862759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392f6b6-ec07-43aa-93e8-f8072088fb3c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:55.899353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:55.967923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4686334-461b-44c0-aa0a-adce50b62eda admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:55.994018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002567d-75ff-41ce-bc9d-c56f3f4cd121 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:55.996545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002567d-75ff-41ce-bc9d-c56f3f4cd121 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:55.996545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002567d-75ff-41ce-bc9d-c56f3f4cd121 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:55.996545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002567d-75ff-41ce-bc9d-c56f3f4cd121 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:56.001971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44957163-e9c1-46c5-9d8f-fd2e90ad19c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:56.001971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44957163-e9c1-46c5-9d8f-fd2e90ad19c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:56.003495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7002567d-75ff-41ce-bc9d-c56f3f4cd121 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:56.005360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44957163-e9c1-46c5-9d8f-fd2e90ad19c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:56.005886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44957163-e9c1-46c5-9d8f-fd2e90ad19c6 admin admin] PATH_INFO: `/v3/projects/8513f1a4d8654e39a613bb0f269cdfba/users/5fd48a1a6e2e4954bf874b36cffca9c8/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:56.011781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44957163-e9c1-46c5-9d8f-fd2e90ad19c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8513f1a4d8654e39a613bb0f269cdfba, user_id=5fd48a1a6e2e4954bf874b36cffca9c8, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:56.018038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44957163-e9c1-46c5-9d8f-fd2e90ad19c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:56.040963 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3602/7230] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:55 2022] PUT /identity/v3/projects/8513f1a4d8654e39a613bb0f269cdfba/users/5fd48a1a6e2e4954bf874b36cffca9c8/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:56.052155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019130e-a882-4bbd-b6af-2b08d9ddfae2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:56.052733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019130e-a882-4bbd-b6af-2b08d9ddfae2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:56.053405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019130e-a882-4bbd-b6af-2b08d9ddfae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:56.101745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1019130e-a882-4bbd-b6af-2b08d9ddfae2 None None] MFA Rules not processed for user `5fd48a1a6e2e4954bf874b36cffca9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:56.103549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1019130e-a882-4bbd-b6af-2b08d9ddfae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:56.155630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002567d-75ff-41ce-bc9d-c56f3f4cd121 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:56.160523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002567d-75ff-41ce-bc9d-c56f3f4cd121 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:56.186920 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3629/7231] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:18:56.198885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:56.239326 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3603/7232] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:56.270539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002567d-75ff-41ce-bc9d-c56f3f4cd121 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:56.272225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7002567d-75ff-41ce-bc9d-c56f3f4cd121 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:56.405915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019130e-a882-4bbd-b6af-2b08d9ddfae2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:56.407800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1019130e-a882-4bbd-b6af-2b08d9ddfae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:56.452888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002567d-75ff-41ce-bc9d-c56f3f4cd121 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:56.502661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019130e-a882-4bbd-b6af-2b08d9ddfae2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:56.507506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:56.508286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:56.508630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:56.521456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] PATH_INFO: `/v3/projects/e78c67f5004a4ac0954ef4783fa807d7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:56.525016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e78c67f5004a4ac0954ef4783fa807d7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:56.527324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:56.554125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:56.574850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058ea6c-dafb-4777-b5b6-c033ba2e30c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:56.575493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058ea6c-dafb-4777-b5b6-c033ba2e30c3 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:56.576023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058ea6c-dafb-4777-b5b6-c033ba2e30c3 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:56.576905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058ea6c-dafb-4777-b5b6-c033ba2e30c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:56.583389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a058ea6c-dafb-4777-b5b6-c033ba2e30c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:56.618753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e78c67f5004a4ac0954ef4783fa807d7', 'request_id': 'req-0eb2ee80-29a2-463d-8e53-777715679214'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:56.633285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e78c67f5004a4ac0954ef4783fa807d7', 'request_id': 'req-0eb2ee80-29a2-463d-8e53-777715679214'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:56.645988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e78c67f5004a4ac0954ef4783fa807d7', 'request_id': 'req-0eb2ee80-29a2-463d-8e53-777715679214'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:56.663374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] The token cache is being invalidate because project e78c67f5004a4ac0954ef4783fa807d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:18:56.664011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:56.665085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0eb2ee80-29a2-463d-8e53-777715679214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:56.667157 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3604/7233] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:18:56 2022] DELETE /identity/v3/projects/e78c67f5004a4ac0954ef4783fa807d7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:56.668697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99da16-bb69-4b61-8695-c879d05d866c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:56.669489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99da16-bb69-4b61-8695-c879d05d866c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:56.672638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99da16-bb69-4b61-8695-c879d05d866c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:56.687848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058ea6c-dafb-4777-b5b6-c033ba2e30c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:56.689603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058ea6c-dafb-4777-b5b6-c033ba2e30c3 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:56.703362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1e99da16-bb69-4b61-8695-c879d05d866c None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:56.711261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e99da16-bb69-4b61-8695-c879d05d866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:56.721186 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3630/7234] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:18:56.722841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd201b8-15e4-45da-a9bb-d0ffb9dcfd27 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:56.723316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd201b8-15e4-45da-a9bb-d0ffb9dcfd27 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:56.723772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd201b8-15e4-45da-a9bb-d0ffb9dcfd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:56.763572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-bfd201b8-15e4-45da-a9bb-d0ffb9dcfd27 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:56.768634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bfd201b8-15e4-45da-a9bb-d0ffb9dcfd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:56.843898 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3605/7235] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:56.857918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99da16-bb69-4b61-8695-c879d05d866c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:56.872728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1e99da16-bb69-4b61-8695-c879d05d866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:56.906314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:56.938760 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3631/7236] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:56.948219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906ead6-b74a-4173-be0d-af33107ca282 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:56.948485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906ead6-b74a-4173-be0d-af33107ca282 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:56.948877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906ead6-b74a-4173-be0d-af33107ca282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:56.993290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8906ead6-b74a-4173-be0d-af33107ca282 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:56.997590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8906ead6-b74a-4173-be0d-af33107ca282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:57.014819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99da16-bb69-4b61-8695-c879d05d866c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:57.022759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2035f-46ac-4e9f-83e5-83732d2e4e80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:57.023455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2035f-46ac-4e9f-83e5-83732d2e4e80 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:57.023728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2035f-46ac-4e9f-83e5-83732d2e4e80 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:57.024030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2035f-46ac-4e9f-83e5-83732d2e4e80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:57.024716 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3632/7237] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:57.027405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906ead6-b74a-4173-be0d-af33107ca282 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:57.038093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2035f-46ac-4e9f-83e5-83732d2e4e80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:57.040307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2035f-46ac-4e9f-83e5-83732d2e4e80 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:57.053427 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3606/7238] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:18:57.057913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906ead6-b74a-4173-be0d-af33107ca282 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:57.069349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2ccff-af1d-400f-8c0c-a7fd9d6bb8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:57.070740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2ccff-af1d-400f-8c0c-a7fd9d6bb8ff admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:57.071022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2ccff-af1d-400f-8c0c-a7fd9d6bb8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:57.071317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2ccff-af1d-400f-8c0c-a7fd9d6bb8ff admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:57.071943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2ccff-af1d-400f-8c0c-a7fd9d6bb8ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:18:57.073393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2ccff-af1d-400f-8c0c-a7fd9d6bb8ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:57.075892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2ccff-af1d-400f-8c0c-a7fd9d6bb8ff admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:57.096806 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3633/7239] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:18:57.100956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2035f-46ac-4e9f-83e5-83732d2e4e80 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:57.140202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2035f-46ac-4e9f-83e5-83732d2e4e80 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:57.147296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e141248-591b-40b8-9862-9c83bba6aa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:57.148371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e141248-591b-40b8-9862-9c83bba6aa0a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:57.148700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e141248-591b-40b8-9862-9c83bba6aa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:57.149021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e141248-591b-40b8-9862-9c83bba6aa0a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:57.149804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e141248-591b-40b8-9862-9c83bba6aa0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:57.151940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e141248-591b-40b8-9862-9c83bba6aa0a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:57.233361 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3607/7240] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:57 2022] POST /identity/v3/projects => generated 401 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:57.235743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2ccff-af1d-400f-8c0c-a7fd9d6bb8ff admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:57.292433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2ccff-af1d-400f-8c0c-a7fd9d6bb8ff admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:57.366289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d31392-0fcd-4842-992c-70cc006cb014 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:57.366570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d31392-0fcd-4842-992c-70cc006cb014 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:57.367001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d31392-0fcd-4842-992c-70cc006cb014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:57.413072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b8d31392-0fcd-4842-992c-70cc006cb014 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:57.414328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b8d31392-0fcd-4842-992c-70cc006cb014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:57.439346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb042c17-6a64-49a6-adff-778a62d13344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:57.439965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb042c17-6a64-49a6-adff-778a62d13344 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:57.440242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb042c17-6a64-49a6-adff-778a62d13344 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:57.440511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb042c17-6a64-49a6-adff-778a62d13344 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:57.441190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb042c17-6a64-49a6-adff-778a62d13344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:57.443123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb042c17-6a64-49a6-adff-778a62d13344 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:57.476662 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3634/7241] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:57 2022] POST /identity/v3/users => generated 301 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:57.485103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb042c17-6a64-49a6-adff-778a62d13344 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:57.502431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:57.520664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb042c17-6a64-49a6-adff-778a62d13344 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:57.529244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef106-125e-41b3-8acf-7f34ba9bb1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:57.530268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef106-125e-41b3-8acf-7f34ba9bb1ae admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:57.530546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef106-125e-41b3-8acf-7f34ba9bb1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:57.530812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef106-125e-41b3-8acf-7f34ba9bb1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:57.531448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ef106-125e-41b3-8acf-7f34ba9bb1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:57.533905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ef106-125e-41b3-8acf-7f34ba9bb1ae admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:57.557925 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3635/7242] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:57.560437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef106-125e-41b3-8acf-7f34ba9bb1ae admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:57.561552 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3608/7243] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:57.574868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d31392-0fcd-4842-992c-70cc006cb014 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:57.586707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef106-125e-41b3-8acf-7f34ba9bb1ae admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:57.602043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf216e1-25e1-40aa-b7ec-65cc604d65e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:57.602545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf216e1-25e1-40aa-b7ec-65cc604d65e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:57.602870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf216e1-25e1-40aa-b7ec-65cc604d65e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:57.603131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf216e1-25e1-40aa-b7ec-65cc604d65e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:57.603911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf216e1-25e1-40aa-b7ec-65cc604d65e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:57.605548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf216e1-25e1-40aa-b7ec-65cc604d65e7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:57.638327 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3636/7244] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:57.639304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf216e1-25e1-40aa-b7ec-65cc604d65e7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:57.662528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf216e1-25e1-40aa-b7ec-65cc604d65e7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:57.677308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b24ac-0741-4423-853b-3fe9db305440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:57.678685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b24ac-0741-4423-853b-3fe9db305440 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:57.679202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b24ac-0741-4423-853b-3fe9db305440 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:57.679886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b24ac-0741-4423-853b-3fe9db305440 admin admin] PATH_INFO: `/v3/projects/c5471032d0284ca2923ec1a9c13fcedf/users/85affa403eeb4a3e852ca0e12cee15f8/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:57.700245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b24ac-0741-4423-853b-3fe9db305440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5471032d0284ca2923ec1a9c13fcedf, user_id=85affa403eeb4a3e852ca0e12cee15f8, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:57.703657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b24ac-0741-4423-853b-3fe9db305440 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:57.710101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d31392-0fcd-4842-992c-70cc006cb014 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:57.720139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1676ac8-4c60-4071-a631-efcd45ce28fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:57.720792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1676ac8-4c60-4071-a631-efcd45ce28fd service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:57.721121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1676ac8-4c60-4071-a631-efcd45ce28fd service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:57.721400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1676ac8-4c60-4071-a631-efcd45ce28fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:57.731551 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3637/7245] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:57 2022] PUT /identity/v3/projects/c5471032d0284ca2923ec1a9c13fcedf/users/85affa403eeb4a3e852ca0e12cee15f8/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:57.751699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7af70-fcaf-4cae-929d-e986aad25ab8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:57.751953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7af70-fcaf-4cae-929d-e986aad25ab8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:57.752259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7af70-fcaf-4cae-929d-e986aad25ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:57.766862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1676ac8-4c60-4071-a631-efcd45ce28fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:57.772383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1676ac8-4c60-4071-a631-efcd45ce28fd service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:57.797399 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3609/7246] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:18:57.807043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9af7af70-fcaf-4cae-929d-e986aad25ab8 None None] MFA Rules not processed for user `85affa403eeb4a3e852ca0e12cee15f8`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:57.813854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9af7af70-fcaf-4cae-929d-e986aad25ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:57.968237 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3638/7247] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:57.969521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1676ac8-4c60-4071-a631-efcd45ce28fd service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:57.980490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1676ac8-4c60-4071-a631-efcd45ce28fd service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:57.997648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:58.027495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76377801-b4a5-49df-977a-170e458edb38 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:58.028047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76377801-b4a5-49df-977a-170e458edb38 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:58.028395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76377801-b4a5-49df-977a-170e458edb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:58.050567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-76377801-b4a5-49df-977a-170e458edb38 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:58.052039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76377801-b4a5-49df-977a-170e458edb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:58.063646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baff7b8-7ec8-426e-b3e7-9a01d35df493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:58.064757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baff7b8-7ec8-426e-b3e7-9a01d35df493 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:58.065248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baff7b8-7ec8-426e-b3e7-9a01d35df493 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:58.065740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baff7b8-7ec8-426e-b3e7-9a01d35df493 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:58.067111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baff7b8-7ec8-426e-b3e7-9a01d35df493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:58.069944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baff7b8-7ec8-426e-b3e7-9a01d35df493 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:58.116114 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3610/7248] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:18:57 2022] POST /identity/v3/projects => generated 399 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:58.117306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baff7b8-7ec8-426e-b3e7-9a01d35df493 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:58.132832 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3639/7249] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:18:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:58.148245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76377801-b4a5-49df-977a-170e458edb38 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:58.237712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baff7b8-7ec8-426e-b3e7-9a01d35df493 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:58.245638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc713e05-a87d-42a9-b712-6b452a900354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:58.247145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc713e05-a87d-42a9-b712-6b452a900354 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:58.247583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc713e05-a87d-42a9-b712-6b452a900354 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:58.247943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc713e05-a87d-42a9-b712-6b452a900354 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:58.248859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc713e05-a87d-42a9-b712-6b452a900354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:58.251901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc713e05-a87d-42a9-b712-6b452a900354 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:58.286614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:58.292090 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3611/7250] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:18:58 2022] POST /identity/v3/users => generated 306 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:18:58.297239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc713e05-a87d-42a9-b712-6b452a900354 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:58.317999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76377801-b4a5-49df-977a-170e458edb38 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:58.322457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc713e05-a87d-42a9-b712-6b452a900354 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:58.338011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258e529-2285-4f4f-8fac-c74513cf135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:58.338960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258e529-2285-4f4f-8fac-c74513cf135f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:58.339532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258e529-2285-4f4f-8fac-c74513cf135f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:58.339978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258e529-2285-4f4f-8fac-c74513cf135f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:58.340905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258e529-2285-4f4f-8fac-c74513cf135f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:58.343447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258e529-2285-4f4f-8fac-c74513cf135f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:58.361210 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3612/7251] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:18:58.365518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258e529-2285-4f4f-8fac-c74513cf135f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:58.392638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258e529-2285-4f4f-8fac-c74513cf135f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:58.408124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd7ba8-6ced-4e36-b294-cd566959ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:58.409850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7ba8-6ced-4e36-b294-cd566959ee7f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:58.410149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7ba8-6ced-4e36-b294-cd566959ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:58.412714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7ba8-6ced-4e36-b294-cd566959ee7f admin admin] PATH_INFO: `/v3/projects/ed60daad951c4718a98a400c45794601/users/48e66d2e6f6a4f93a492d63dd8275632/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:58.456210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd7ba8-6ced-4e36-b294-cd566959ee7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed60daad951c4718a98a400c45794601, user_id=48e66d2e6f6a4f93a492d63dd8275632, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:58.460385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd7ba8-6ced-4e36-b294-cd566959ee7f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:58.484763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ac8ae-93a9-40ef-a45f-98be5c89a82f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:58.485538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ac8ae-93a9-40ef-a45f-98be5c89a82f service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:58.485812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ac8ae-93a9-40ef-a45f-98be5c89a82f service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:58.486085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ac8ae-93a9-40ef-a45f-98be5c89a82f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:58.496044 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3613/7252] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:58 2022] PUT /identity/v3/projects/ed60daad951c4718a98a400c45794601/users/48e66d2e6f6a4f93a492d63dd8275632/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:58.500517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd7ba8-6ced-4e36-b294-cd566959ee7f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:58.593735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ac8ae-93a9-40ef-a45f-98be5c89a82f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:58.595966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ac8ae-93a9-40ef-a45f-98be5c89a82f service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:58.609758 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3640/7253] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:18:58.633607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:58.644655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd7ba8-6ced-4e36-b294-cd566959ee7f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:58.660377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13a87e-3725-4646-96f9-51af568dfe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:58.660764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a87e-3725-4646-96f9-51af568dfe67 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:58.660940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a87e-3725-4646-96f9-51af568dfe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:58.661828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a87e-3725-4646-96f9-51af568dfe67 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:58.662197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a87e-3725-4646-96f9-51af568dfe67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:58.664259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a87e-3725-4646-96f9-51af568dfe67 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:58.706131 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3614/7254] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:18:58.711113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ac8ae-93a9-40ef-a45f-98be5c89a82f service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:58.727121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ac8ae-93a9-40ef-a45f-98be5c89a82f service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:58.736577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aa600-99a5-4cf7-8033-deb41533dee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:58.737990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aa600-99a5-4cf7-8033-deb41533dee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:58.738250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aa600-99a5-4cf7-8033-deb41533dee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:58.738583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aa600-99a5-4cf7-8033-deb41533dee0 admin admin] PATH_INFO: `/v3/projects/ed60daad951c4718a98a400c45794601/users/48e66d2e6f6a4f93a492d63dd8275632/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:58.742018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680aa600-99a5-4cf7-8033-deb41533dee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed60daad951c4718a98a400c45794601, user_id=48e66d2e6f6a4f93a492d63dd8275632, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:58.744807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680aa600-99a5-4cf7-8033-deb41533dee0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:58.759358 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3641/7255] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:18:58 2022] PUT /identity/v3/projects/ed60daad951c4718a98a400c45794601/users/48e66d2e6f6a4f93a492d63dd8275632/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:58.772954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad7aa0-e6c7-4a76-a014-8b0d264f821f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:58.773525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad7aa0-e6c7-4a76-a014-8b0d264f821f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:58.773660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad7aa0-e6c7-4a76-a014-8b0d264f821f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:58.820148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-98ad7aa0-e6c7-4a76-a014-8b0d264f821f None None] MFA Rules not processed for user `48e66d2e6f6a4f93a492d63dd8275632`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:58.821749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-98ad7aa0-e6c7-4a76-a014-8b0d264f821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:58.840324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db7f3-715d-4e39-98d0-bd1473f77ec1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:58.840608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db7f3-715d-4e39-98d0-bd1473f77ec1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:58.840881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db7f3-715d-4e39-98d0-bd1473f77ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:58.882040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-294db7f3-715d-4e39-98d0-bd1473f77ec1 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:58.883568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-294db7f3-715d-4e39-98d0-bd1473f77ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:58.990899 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3642/7256] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:18:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:18:58.992104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294db7f3-715d-4e39-98d0-bd1473f77ec1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:59.024088 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3615/7257] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:58 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:18:59.032620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad7aa0-e6c7-4a76-a014-8b0d264f821f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:59.035464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-98ad7aa0-e6c7-4a76-a014-8b0d264f821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:59.058115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:59.117110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294db7f3-715d-4e39-98d0-bd1473f77ec1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:59.125097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309de57-e202-4dd0-b128-13ab23b87e9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:59.125638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309de57-e202-4dd0-b128-13ab23b87e9e service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:59.125913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309de57-e202-4dd0-b128-13ab23b87e9e service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:59.126182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309de57-e202-4dd0-b128-13ab23b87e9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:59.136743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309de57-e202-4dd0-b128-13ab23b87e9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:59.138663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309de57-e202-4dd0-b128-13ab23b87e9e service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:59.150225 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3643/7258] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:18:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:18:59.180703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad7aa0-e6c7-4a76-a014-8b0d264f821f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:59.195306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe405b-a073-4d60-99dc-3571a09f8964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:59.196208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe405b-a073-4d60-99dc-3571a09f8964 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:59.196677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe405b-a073-4d60-99dc-3571a09f8964 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:59.196941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe405b-a073-4d60-99dc-3571a09f8964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:59.210283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe405b-a073-4d60-99dc-3571a09f8964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:59.212678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe405b-a073-4d60-99dc-3571a09f8964 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:59.230398 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3616/7259] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:59 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:18:59.365776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309de57-e202-4dd0-b128-13ab23b87e9e service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:59.366756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a309de57-e202-4dd0-b128-13ab23b87e9e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:59.469368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe405b-a073-4d60-99dc-3571a09f8964 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:18:59.486409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe405b-a073-4d60-99dc-3571a09f8964 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:59.492192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb281d-a376-4651-80d9-1d14aeea987f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:59.492900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb281d-a376-4651-80d9-1d14aeea987f service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:59.493268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb281d-a376-4651-80d9-1d14aeea987f service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:59.493704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb281d-a376-4651-80d9-1d14aeea987f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:59.544381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309de57-e202-4dd0-b128-13ab23b87e9e service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:18:59.551754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:18:59.552822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:59.553299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:59.553921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] PATH_INFO: `/v3/users/ad9eb2a72a114db6b36071e6af6c71aa` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:59.555368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad9eb2a72a114db6b36071e6af6c71aa)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:59.557822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:59.639154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb281d-a376-4651-80d9-1d14aeea987f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:18:59.642920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb281d-a376-4651-80d9-1d14aeea987f service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:18:59.673655 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3617/7260] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:18:59 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:18:59.677616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad9eb2a72a114db6b36071e6af6c71aa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:59.722426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad9eb2a72a114db6b36071e6af6c71aa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:59.761083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad9eb2a72a114db6b36071e6af6c71aa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:59.784517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad9eb2a72a114db6b36071e6af6c71aa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:59.785352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad9eb2a72a114db6b36071e6af6c71aa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:18:59.798350 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3644/7261] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:18:59 2022] DELETE /identity/v3/users/ad9eb2a72a114db6b36071e6af6c71aa => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:18:59.803625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddbe6d-5bab-449e-bc97-cdd7ef0e8fec None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:18:59.804966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddbe6d-5bab-449e-bc97-cdd7ef0e8fec None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:18:59.805630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddbe6d-5bab-449e-bc97-cdd7ef0e8fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:18:59.849537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b7ddbe6d-5bab-449e-bc97-cdd7ef0e8fec None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:18:59.851456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b7ddbe6d-5bab-449e-bc97-cdd7ef0e8fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:18:59.885273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:00.011109 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3618/7262] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:19:00.021491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:00.023791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:00.104395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:00.147796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dd4e6-e6c6-461e-a35d-a55ad241374d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:00.152724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d20136-6463-4532-8f07-579348530426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:00.153663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20136-6463-4532-8f07-579348530426 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:00.156016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20136-6463-4532-8f07-579348530426 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:00.157753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20136-6463-4532-8f07-579348530426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:00.172490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d20136-6463-4532-8f07-579348530426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:00.174949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d20136-6463-4532-8f07-579348530426 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:00.193436 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3645/7263] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:00.635464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddbe6d-5bab-449e-bc97-cdd7ef0e8fec None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:00.637018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b7ddbe6d-5bab-449e-bc97-cdd7ef0e8fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:00.656677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddbe6d-5bab-449e-bc97-cdd7ef0e8fec None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:00.663465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:00.663974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:00.664420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:00.664720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] PATH_INFO: `/v3/projects/8992ad73e72949fe80c3e8f630a8655d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:00.665998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8992ad73e72949fe80c3e8f630a8655d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:00.668096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:00.744373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8992ad73e72949fe80c3e8f630a8655d', 'request_id': 'req-58878b8f-d723-4062-8bb0-a45c207b6a19'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:00.754690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8992ad73e72949fe80c3e8f630a8655d', 'request_id': 'req-58878b8f-d723-4062-8bb0-a45c207b6a19'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:00.765082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8992ad73e72949fe80c3e8f630a8655d', 'request_id': 'req-58878b8f-d723-4062-8bb0-a45c207b6a19'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:00.781838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] The token cache is being invalidate because project 8992ad73e72949fe80c3e8f630a8655d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:00.782335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:00.783193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:00.785535 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3619/7264] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:00 2022] DELETE /identity/v3/projects/8992ad73e72949fe80c3e8f630a8655d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:00.790332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a4255-0d87-4c4d-bd1e-b5f27ad81340 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:00.790809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a4255-0d87-4c4d-bd1e-b5f27ad81340 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:00.790907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a4255-0d87-4c4d-bd1e-b5f27ad81340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:00.869631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:00.871318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:00.878790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6f2a4255-0d87-4c4d-bd1e-b5f27ad81340 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:00.882029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6f2a4255-0d87-4c4d-bd1e-b5f27ad81340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:00.984625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58878b8f-d723-4062-8bb0-a45c207b6a19 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:00.990096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f87cd3-18f8-4d4d-84ec-5bae99c545ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:00.990734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f87cd3-18f8-4d4d-84ec-5bae99c545ac service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:00.991010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f87cd3-18f8-4d4d-84ec-5bae99c545ac service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:00.991279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f87cd3-18f8-4d4d-84ec-5bae99c545ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:00.997643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-34f87cd3-18f8-4d4d-84ec-5bae99c545ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:01.039174 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3646/7265] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:01.048555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3076d-7c9f-402a-93d9-5a402e14c689 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:01.049264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3076d-7c9f-402a-93d9-5a402e14c689 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:01.049771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3076d-7c9f-402a-93d9-5a402e14c689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:01.091703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4bd3076d-7c9f-402a-93d9-5a402e14c689 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:01.094253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4bd3076d-7c9f-402a-93d9-5a402e14c689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:01.101971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:01.110820 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3647/7266] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:01.115598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3076d-7c9f-402a-93d9-5a402e14c689 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:01.116474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f87cd3-18f8-4d4d-84ec-5bae99c545ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:01.118260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f87cd3-18f8-4d4d-84ec-5bae99c545ac service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:01.131501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3076d-7c9f-402a-93d9-5a402e14c689 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:01.132061 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3620/7267] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:19:01.138189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65a58b-dcfd-4827-984f-2b0de5d1e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:01.139084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65a58b-dcfd-4827-984f-2b0de5d1e16c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:01.139469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65a58b-dcfd-4827-984f-2b0de5d1e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:01.139829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65a58b-dcfd-4827-984f-2b0de5d1e16c admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:01.140573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65a58b-dcfd-4827-984f-2b0de5d1e16c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:01.140966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65a58b-dcfd-4827-984f-2b0de5d1e16c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:01.142682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65a58b-dcfd-4827-984f-2b0de5d1e16c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:01.158826 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3648/7268] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:19:01.160043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f87cd3-18f8-4d4d-84ec-5bae99c545ac service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:01.173214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f87cd3-18f8-4d4d-84ec-5bae99c545ac service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:01.181708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4aac5-7c11-4d00-864c-8bebdfbc3053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:01.182568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4aac5-7c11-4d00-864c-8bebdfbc3053 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:01.183124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4aac5-7c11-4d00-864c-8bebdfbc3053 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:01.183563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4aac5-7c11-4d00-864c-8bebdfbc3053 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:01.185022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4aac5-7c11-4d00-864c-8bebdfbc3053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:01.188653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4aac5-7c11-4d00-864c-8bebdfbc3053 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:01.253497 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3621/7269] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:01 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:01.257773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65a58b-dcfd-4827-984f-2b0de5d1e16c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:01.273955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65a58b-dcfd-4827-984f-2b0de5d1e16c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:01.388325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02234a22-348b-4393-9ebc-1deb723a0b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:01.389082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234a22-348b-4393-9ebc-1deb723a0b64 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:01.389485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234a22-348b-4393-9ebc-1deb723a0b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:01.389913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234a22-348b-4393-9ebc-1deb723a0b64 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:01.390586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02234a22-348b-4393-9ebc-1deb723a0b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:01.392807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02234a22-348b-4393-9ebc-1deb723a0b64 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:01.428333 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3649/7270] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:01 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:01.432407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4aac5-7c11-4d00-864c-8bebdfbc3053 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:01.472561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4aac5-7c11-4d00-864c-8bebdfbc3053 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:01.489219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146cca5-dadf-41ce-b711-a57354752c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:01.490186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146cca5-dadf-41ce-b711-a57354752c02 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:01.490572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146cca5-dadf-41ce-b711-a57354752c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:01.490852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146cca5-dadf-41ce-b711-a57354752c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:01.491616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146cca5-dadf-41ce-b711-a57354752c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:01.493419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146cca5-dadf-41ce-b711-a57354752c02 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:01.521530 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3622/7271] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:01.525159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02234a22-348b-4393-9ebc-1deb723a0b64 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:01.541510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02234a22-348b-4393-9ebc-1deb723a0b64 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:01.548098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4d846-a541-460c-82d5-b5c00bc1e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:01.550663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4d846-a541-460c-82d5-b5c00bc1e7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:01.551050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4d846-a541-460c-82d5-b5c00bc1e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:01.551332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4d846-a541-460c-82d5-b5c00bc1e7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:01.552103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4d846-a541-460c-82d5-b5c00bc1e7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:01.554033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4d846-a541-460c-82d5-b5c00bc1e7a6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:01.575787 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3650/7272] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:01.577133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146cca5-dadf-41ce-b711-a57354752c02 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:01.595175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146cca5-dadf-41ce-b711-a57354752c02 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:01.609682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cc6d8-a333-45bb-b0b0-1ea5ee3caa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:01.610390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cc6d8-a333-45bb-b0b0-1ea5ee3caa41 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:01.610645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cc6d8-a333-45bb-b0b0-1ea5ee3caa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:01.610922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cc6d8-a333-45bb-b0b0-1ea5ee3caa41 admin admin] PATH_INFO: `/v3/projects/558be4ff2b704e3f8615a106c484b806/users/8d5f0e6c106c44da8b69993960876ef1/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:01.645829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cc6d8-a333-45bb-b0b0-1ea5ee3caa41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=558be4ff2b704e3f8615a106c484b806, user_id=8d5f0e6c106c44da8b69993960876ef1, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:01.650996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cc6d8-a333-45bb-b0b0-1ea5ee3caa41 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:01.680110 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3623/7273] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:01 2022] PUT /identity/v3/projects/558be4ff2b704e3f8615a106c484b806/users/8d5f0e6c106c44da8b69993960876ef1/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:01.692598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd427573-57dc-4162-a27a-b9cb9d95d1c1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:01.692809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd427573-57dc-4162-a27a-b9cb9d95d1c1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:01.692974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd427573-57dc-4162-a27a-b9cb9d95d1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:01.762250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fd427573-57dc-4162-a27a-b9cb9d95d1c1 None None] MFA Rules not processed for user `8d5f0e6c106c44da8b69993960876ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:01.767440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd427573-57dc-4162-a27a-b9cb9d95d1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:01.868943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:01.962950 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3651/7274] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:01.976285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cc6d8-a333-45bb-b0b0-1ea5ee3caa41 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:01.978723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-040cc6d8-a333-45bb-b0b0-1ea5ee3caa41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:02.097282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cc6d8-a333-45bb-b0b0-1ea5ee3caa41 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:02.101672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27249d42-ffe1-4cb3-85b9-48ae184f7d5f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:02.102398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27249d42-ffe1-4cb3-85b9-48ae184f7d5f service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:02.102743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27249d42-ffe1-4cb3-85b9-48ae184f7d5f service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:02.103069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27249d42-ffe1-4cb3-85b9-48ae184f7d5f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:02.117959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27249d42-ffe1-4cb3-85b9-48ae184f7d5f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:02.119526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27249d42-ffe1-4cb3-85b9-48ae184f7d5f service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:02.131600 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3624/7275] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:19:02.274498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd427573-57dc-4162-a27a-b9cb9d95d1c1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:02.385916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd427573-57dc-4162-a27a-b9cb9d95d1c1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:02.391393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:02.392047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:02.392325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:02.392595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] PATH_INFO: `/v3/users/8d5f0e6c106c44da8b69993960876ef1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:02.393892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d5f0e6c106c44da8b69993960876ef1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:02.397266 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:02.481707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d5f0e6c106c44da8b69993960876ef1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:02.495330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d5f0e6c106c44da8b69993960876ef1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:02.526190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d5f0e6c106c44da8b69993960876ef1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:02.539037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d5f0e6c106c44da8b69993960876ef1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:02.540502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d5f0e6c106c44da8b69993960876ef1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:02.560018 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3652/7276] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:02 2022] DELETE /identity/v3/users/8d5f0e6c106c44da8b69993960876ef1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:02.562331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0d449-7774-4612-9080-d56c7bc6d933 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:02.562331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0d449-7774-4612-9080-d56c7bc6d933 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:02.562559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0d449-7774-4612-9080-d56c7bc6d933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:02.588893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f5d0d449-7774-4612-9080-d56c7bc6d933 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:02.590511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f5d0d449-7774-4612-9080-d56c7bc6d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:02.679610 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3625/7277] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:02.725297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:02.732264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:02.852896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:02.877359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcedef9-feac-4ab1-b35e-e320f9bb3792 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:02.883433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6ce28-5b04-4642-a8b2-d830638f712a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:02.884286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6ce28-5b04-4642-a8b2-d830638f712a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:02.884441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6ce28-5b04-4642-a8b2-d830638f712a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:02.884623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6ce28-5b04-4642-a8b2-d830638f712a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:02.922139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6ce28-5b04-4642-a8b2-d830638f712a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:02.927774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6ce28-5b04-4642-a8b2-d830638f712a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:02.972120 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3653/7278] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:04.517847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0d449-7774-4612-9080-d56c7bc6d933 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:04.519322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f5d0d449-7774-4612-9080-d56c7bc6d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:04.543578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0d449-7774-4612-9080-d56c7bc6d933 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:04.556663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:04.557636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:04.558021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:04.558388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] PATH_INFO: `/v3/projects/558be4ff2b704e3f8615a106c484b806` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:04.561640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=558be4ff2b704e3f8615a106c484b806)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:04.566402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:04.671545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:04.689962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '558be4ff2b704e3f8615a106c484b806', 'request_id': 'req-534d8c9b-ece7-435f-a711-504def7a1d71'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:04.710154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '558be4ff2b704e3f8615a106c484b806', 'request_id': 'req-534d8c9b-ece7-435f-a711-504def7a1d71'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:04.722571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '558be4ff2b704e3f8615a106c484b806', 'request_id': 'req-534d8c9b-ece7-435f-a711-504def7a1d71'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:04.756726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] The token cache is being invalidate because project 558be4ff2b704e3f8615a106c484b806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:04.757376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:04.758180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-534d8c9b-ece7-435f-a711-504def7a1d71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:04.761084 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3626/7279] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:04 2022] DELETE /identity/v3/projects/558be4ff2b704e3f8615a106c484b806 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:04.766199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865d74a-a5e4-4a72-aebf-64195e1eb1e1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:04.766718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865d74a-a5e4-4a72-aebf-64195e1eb1e1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:04.766911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865d74a-a5e4-4a72-aebf-64195e1eb1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:04.834255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8865d74a-a5e4-4a72-aebf-64195e1eb1e1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:04.835763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8865d74a-a5e4-4a72-aebf-64195e1eb1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:04.968267 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3654/7280] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:04.976694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f5117-5333-4eec-bbdb-cf860bdea46e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:04.978967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f5117-5333-4eec-bbdb-cf860bdea46e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:04.979333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f5117-5333-4eec-bbdb-cf860bdea46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:05.007807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8f9f5117-5333-4eec-bbdb-cf860bdea46e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:05.011475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8f9f5117-5333-4eec-bbdb-cf860bdea46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:05.034669 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3627/7281] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:05.039431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865d74a-a5e4-4a72-aebf-64195e1eb1e1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:05.056360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865d74a-a5e4-4a72-aebf-64195e1eb1e1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:05.070408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a9ee1-786c-4aaa-886c-60ae3d7b1b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:05.073762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a9ee1-786c-4aaa-886c-60ae3d7b1b33 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:05.073762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a9ee1-786c-4aaa-886c-60ae3d7b1b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:05.074015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a9ee1-786c-4aaa-886c-60ae3d7b1b33 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:05.076471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a9ee1-786c-4aaa-886c-60ae3d7b1b33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:05.076888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a9ee1-786c-4aaa-886c-60ae3d7b1b33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:05.078604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a9ee1-786c-4aaa-886c-60ae3d7b1b33 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:05.103547 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3655/7282] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:19:05.107993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f5117-5333-4eec-bbdb-cf860bdea46e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:05.128515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f5117-5333-4eec-bbdb-cf860bdea46e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:05.137672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1386d-4088-4374-8fd2-67a864360e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:05.138733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1386d-4088-4374-8fd2-67a864360e5c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:05.139723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1386d-4088-4374-8fd2-67a864360e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:05.140388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1386d-4088-4374-8fd2-67a864360e5c admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:05.141487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1386d-4088-4374-8fd2-67a864360e5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:05.143445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1386d-4088-4374-8fd2-67a864360e5c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:05.195647 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3628/7283] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:05 2022] POST /identity/v3/projects => generated 359 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:19:05.199996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a9ee1-786c-4aaa-886c-60ae3d7b1b33 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:05.218398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a9ee1-786c-4aaa-886c-60ae3d7b1b33 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:05.384088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd4961-08b3-446b-b9f5-1d0fd515d7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:05.385711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd4961-08b3-446b-b9f5-1d0fd515d7c0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:05.387831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd4961-08b3-446b-b9f5-1d0fd515d7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:05.388385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd4961-08b3-446b-b9f5-1d0fd515d7c0 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:05.390449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd4961-08b3-446b-b9f5-1d0fd515d7c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:05.397231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd4961-08b3-446b-b9f5-1d0fd515d7c0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:05.481399 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3656/7284] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:05 2022] POST /identity/v3/users => generated 280 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:05.481610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1386d-4088-4374-8fd2-67a864360e5c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:05.502234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1386d-4088-4374-8fd2-67a864360e5c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:05.512213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae7efe-d51c-46fc-8208-f7e09ebdcfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:05.512948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae7efe-d51c-46fc-8208-f7e09ebdcfd9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:05.513779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae7efe-d51c-46fc-8208-f7e09ebdcfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:05.514095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae7efe-d51c-46fc-8208-f7e09ebdcfd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:05.515066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae7efe-d51c-46fc-8208-f7e09ebdcfd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:05.517023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae7efe-d51c-46fc-8208-f7e09ebdcfd9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:05.529085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3629/7285] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:19:05.534507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd4961-08b3-446b-b9f5-1d0fd515d7c0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:05.551661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd4961-08b3-446b-b9f5-1d0fd515d7c0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:05.566985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab791ef6-02ac-42fc-9cad-528d01224d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:05.567749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab791ef6-02ac-42fc-9cad-528d01224d70 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:05.568146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab791ef6-02ac-42fc-9cad-528d01224d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:05.568513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab791ef6-02ac-42fc-9cad-528d01224d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:05.569868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab791ef6-02ac-42fc-9cad-528d01224d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:05.571581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab791ef6-02ac-42fc-9cad-528d01224d70 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:05.608775 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3657/7286] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:05.609075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae7efe-d51c-46fc-8208-f7e09ebdcfd9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:05.654788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae7efe-d51c-46fc-8208-f7e09ebdcfd9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:05.674867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f79cce-9f63-4a92-9750-f8423e6c094f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:05.675834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f79cce-9f63-4a92-9750-f8423e6c094f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:05.676122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f79cce-9f63-4a92-9750-f8423e6c094f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:05.683065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f79cce-9f63-4a92-9750-f8423e6c094f admin admin] PATH_INFO: `/v3/projects/690dfd3cafc34689861d0ca355e65b01/users/8c378dc5b4724816bffb62dd72ed4ee5/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:05.757234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f79cce-9f63-4a92-9750-f8423e6c094f admin admin] RBAC: Authorizing `identity:create_grant(project_id=690dfd3cafc34689861d0ca355e65b01, user_id=8c378dc5b4724816bffb62dd72ed4ee5, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:05.760213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f79cce-9f63-4a92-9750-f8423e6c094f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:05.805120 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3630/7287] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:05 2022] PUT /identity/v3/projects/690dfd3cafc34689861d0ca355e65b01/users/8c378dc5b4724816bffb62dd72ed4ee5/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:05.818096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5d2f5-943d-47ff-a313-86fe1716b321 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:05.818578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5d2f5-943d-47ff-a313-86fe1716b321 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:05.818761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5d2f5-943d-47ff-a313-86fe1716b321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:05.873732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b2e5d2f5-943d-47ff-a313-86fe1716b321 None None] MFA Rules not processed for user `8c378dc5b4724816bffb62dd72ed4ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:05.876526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b2e5d2f5-943d-47ff-a313-86fe1716b321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:05.980091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:06.056840 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3658/7288] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:06.058314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f79cce-9f63-4a92-9750-f8423e6c094f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:06.238190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5d2f5-943d-47ff-a313-86fe1716b321 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:06.239691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b2e5d2f5-943d-47ff-a313-86fe1716b321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:06.282711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5d2f5-943d-47ff-a313-86fe1716b321 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:06.284540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f79cce-9f63-4a92-9750-f8423e6c094f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:06.295793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:06.296556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:06.297452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:06.297783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:06.298000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] PATH_INFO: `/v3/users/8c378dc5b4724816bffb62dd72ed4ee5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:06.298518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:06.298794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:06.299094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] PATH_INFO: `/v3/users/85affa403eeb4a3e852ca0e12cee15f8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:06.300842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c378dc5b4724816bffb62dd72ed4ee5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:06.302830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:06.312112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=85affa403eeb4a3e852ca0e12cee15f8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:06.322001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:06.424753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c378dc5b4724816bffb62dd72ed4ee5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:06.444194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85affa403eeb4a3e852ca0e12cee15f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:06.451780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:06.462779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85affa403eeb4a3e852ca0e12cee15f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:06.477554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c378dc5b4724816bffb62dd72ed4ee5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:06.500599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85affa403eeb4a3e852ca0e12cee15f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:06.501538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c378dc5b4724816bffb62dd72ed4ee5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:06.518354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85affa403eeb4a3e852ca0e12cee15f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:06.519215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0c5de049-512b-4da4-98f6-a1b9006ab0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85affa403eeb4a3e852ca0e12cee15f8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:06.530809 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3659/7289] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:06 2022] DELETE /identity/v3/users/85affa403eeb4a3e852ca0e12cee15f8 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:06.534641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c378dc5b4724816bffb62dd72ed4ee5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:06.535977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8ab65168-82ba-4500-9a53-44a663f0c622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c378dc5b4724816bffb62dd72ed4ee5'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:06.536486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58277290-0517-4733-b160-6f98ab051464 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:06.536767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58277290-0517-4733-b160-6f98ab051464 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:06.537065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58277290-0517-4733-b160-6f98ab051464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:06.544310 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3631/7290] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:06 2022] DELETE /identity/v3/users/8c378dc5b4724816bffb62dd72ed4ee5 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:06.548759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782685d5-a1d3-4e0e-bf39-5bbc73f71ace None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:06.550058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782685d5-a1d3-4e0e-bf39-5bbc73f71ace None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:06.550652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782685d5-a1d3-4e0e-bf39-5bbc73f71ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:06.588994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-782685d5-a1d3-4e0e-bf39-5bbc73f71ace None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:06.592934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-782685d5-a1d3-4e0e-bf39-5bbc73f71ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:06.614169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-58277290-0517-4733-b160-6f98ab051464 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:06.615638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-58277290-0517-4733-b160-6f98ab051464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:06.772154 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3660/7291] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:06.772351 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3632/7292] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:06.797011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58277290-0517-4733-b160-6f98ab051464 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:06.797679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782685d5-a1d3-4e0e-bf39-5bbc73f71ace None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:06.799068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-782685d5-a1d3-4e0e-bf39-5bbc73f71ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:06.799403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-58277290-0517-4733-b160-6f98ab051464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:06.908351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58277290-0517-4733-b160-6f98ab051464 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:06.916129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f3c1c-9585-440d-9d2f-287b6e7d3275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:06.916649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782685d5-a1d3-4e0e-bf39-5bbc73f71ace None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:06.917420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f3c1c-9585-440d-9d2f-287b6e7d3275 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:06.917877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f3c1c-9585-440d-9d2f-287b6e7d3275 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:06.918300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f3c1c-9585-440d-9d2f-287b6e7d3275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:06.925549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6e006-98a2-48d4-8662-d3a8a00fb808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:06.930525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6e006-98a2-48d4-8662-d3a8a00fb808 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:06.930852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6e006-98a2-48d4-8662-d3a8a00fb808 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:06.931132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6e006-98a2-48d4-8662-d3a8a00fb808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:06.935874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f3c1c-9585-440d-9d2f-287b6e7d3275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:06.940062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f3c1c-9585-440d-9d2f-287b6e7d3275 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:06.950045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:06.961899 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3661/7293] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:06.972099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6e006-98a2-48d4-8662-d3a8a00fb808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:06.974276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6e006-98a2-48d4-8662-d3a8a00fb808 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:06.992797 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3633/7294] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:08.220028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f3c1c-9585-440d-9d2f-287b6e7d3275 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:08.222300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-967f3c1c-9585-440d-9d2f-287b6e7d3275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:08.248886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f3c1c-9585-440d-9d2f-287b6e7d3275 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:08.261695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:08.262864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:08.262864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:08.267877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] PATH_INFO: `/v3/projects/c5471032d0284ca2923ec1a9c13fcedf` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:08.270353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5471032d0284ca2923ec1a9c13fcedf)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:08.274364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:08.373904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6e006-98a2-48d4-8662-d3a8a00fb808 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:08.376298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aee6e006-98a2-48d4-8662-d3a8a00fb808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:08.396466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5471032d0284ca2923ec1a9c13fcedf', 'request_id': 'req-5492feee-993a-4124-b33c-4afff8a175e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.411762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6e006-98a2-48d4-8662-d3a8a00fb808 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:08.426805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5471032d0284ca2923ec1a9c13fcedf', 'request_id': 'req-5492feee-993a-4124-b33c-4afff8a175e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.438649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5471032d0284ca2923ec1a9c13fcedf', 'request_id': 'req-5492feee-993a-4124-b33c-4afff8a175e5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.470241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] The token cache is being invalidate because project c5471032d0284ca2923ec1a9c13fcedf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:08.470549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.471908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.480348 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3662/7295] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:08 2022] DELETE /identity/v3/projects/c5471032d0284ca2923ec1a9c13fcedf => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:08.485330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:08.488114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:08.589934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:08.591996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:08.595423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:08.595915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] PATH_INFO: `/v3/projects/690dfd3cafc34689861d0ca355e65b01` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:08.599210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] RBAC: Authorizing `identity:delete_project(project_id=690dfd3cafc34689861d0ca355e65b01)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:08.601375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:08.608826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492feee-993a-4124-b33c-4afff8a175e5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:08.615259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:08.616343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:08.616785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:08.617403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] PATH_INFO: `/v3/users/48e66d2e6f6a4f93a492d63dd8275632` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:08.619714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e66d2e6f6a4f93a492d63dd8275632)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:08.623363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:08.656072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:08.711542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690dfd3cafc34689861d0ca355e65b01', 'request_id': 'req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.719651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e66d2e6f6a4f93a492d63dd8275632'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.729365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690dfd3cafc34689861d0ca355e65b01', 'request_id': 'req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.739722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e66d2e6f6a4f93a492d63dd8275632'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.741462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690dfd3cafc34689861d0ca355e65b01', 'request_id': 'req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.769149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] The token cache is being invalidate because project 690dfd3cafc34689861d0ca355e65b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:08.769432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.770633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa2a6ffe-a19c-4b23-b91f-f933ce85086f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.773191 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3634/7296] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:08 2022] DELETE /identity/v3/projects/690dfd3cafc34689861d0ca355e65b01 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:19:08.774115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e66d2e6f6a4f93a492d63dd8275632'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.775366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6830e-a4d3-4b75-bfc1-c25673b75b21 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:08.775782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6830e-a4d3-4b75-bfc1-c25673b75b21 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:08.776506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6830e-a4d3-4b75-bfc1-c25673b75b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:08.788848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e66d2e6f6a4f93a492d63dd8275632'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.789508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3997006a-c273-4a5f-94d2-83a807f0431b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e66d2e6f6a4f93a492d63dd8275632'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:08.798369 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3663/7297] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:08 2022] DELETE /identity/v3/users/48e66d2e6f6a4f93a492d63dd8275632 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:08.800244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e6094-694a-49cf-92b3-0b4a63fb1ea5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:08.800537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e6094-694a-49cf-92b3-0b4a63fb1ea5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:08.800977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e6094-694a-49cf-92b3-0b4a63fb1ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:08.809261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d7b6830e-a4d3-4b75-bfc1-c25673b75b21 None None] MFA Rules not processed for user `026771ec28444739afc13c47339af7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:08.810541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:08.811852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d7b6830e-a4d3-4b75-bfc1-c25673b75b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:08.833896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0a7e6094-694a-49cf-92b3-0b4a63fb1ea5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:08.835122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0a7e6094-694a-49cf-92b3-0b4a63fb1ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:08.900089 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3635/7298] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:08.930880 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3664/7299] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:08.938705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aee2db-6f9c-4103-b99f-1d8efea78f0f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:08.939050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aee2db-6f9c-4103-b99f-1d8efea78f0f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:08.940474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aee2db-6f9c-4103-b99f-1d8efea78f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:08.979754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e9aee2db-6f9c-4103-b99f-1d8efea78f0f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:08.981077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e9aee2db-6f9c-4103-b99f-1d8efea78f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:09.009005 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3636/7300] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:09.009366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e6094-694a-49cf-92b3-0b4a63fb1ea5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:09.038550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e6094-694a-49cf-92b3-0b4a63fb1ea5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:09.062393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa7d5-9f91-4154-8f2b-3fe1b3533ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:09.064549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa7d5-9f91-4154-8f2b-3fe1b3533ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:09.065426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa7d5-9f91-4154-8f2b-3fe1b3533ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:09.067012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa7d5-9f91-4154-8f2b-3fe1b3533ee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:09.070891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfa7d5-9f91-4154-8f2b-3fe1b3533ee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:09.071526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfa7d5-9f91-4154-8f2b-3fe1b3533ee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:09.073949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfa7d5-9f91-4154-8f2b-3fe1b3533ee1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:09.111904 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3665/7301] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:19:09.113262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aee2db-6f9c-4103-b99f-1d8efea78f0f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:09.130288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aee2db-6f9c-4103-b99f-1d8efea78f0f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:09.142301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69411954-1c43-4738-9bb7-0d26c911b9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:09.145206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69411954-1c43-4738-9bb7-0d26c911b9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:09.147223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69411954-1c43-4738-9bb7-0d26c911b9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:09.147642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69411954-1c43-4738-9bb7-0d26c911b9f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:09.149029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69411954-1c43-4738-9bb7-0d26c911b9f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:09.155115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69411954-1c43-4738-9bb7-0d26c911b9f0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:09.222686 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3637/7302] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:09 2022] POST /identity/v3/projects => generated 411 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:09.226924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa7d5-9f91-4154-8f2b-3fe1b3533ee1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:09.247917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa7d5-9f91-4154-8f2b-3fe1b3533ee1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:09.412719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80338840-f025-4dea-85fc-bd7cdd42e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:09.413772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80338840-f025-4dea-85fc-bd7cdd42e38c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:09.415496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80338840-f025-4dea-85fc-bd7cdd42e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:09.417173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80338840-f025-4dea-85fc-bd7cdd42e38c admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:09.418806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80338840-f025-4dea-85fc-bd7cdd42e38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:09.423335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80338840-f025-4dea-85fc-bd7cdd42e38c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:09.481307 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3666/7303] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:09 2022] POST /identity/v3/users => generated 306 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:09.486746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69411954-1c43-4738-9bb7-0d26c911b9f0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:09.504884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69411954-1c43-4738-9bb7-0d26c911b9f0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:09.521023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c6b37-72a8-4098-adf3-d703d26705fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:09.521808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c6b37-72a8-4098-adf3-d703d26705fe admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:09.522159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c6b37-72a8-4098-adf3-d703d26705fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:09.522501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c6b37-72a8-4098-adf3-d703d26705fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:09.523438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c6b37-72a8-4098-adf3-d703d26705fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:09.525638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80338840-f025-4dea-85fc-bd7cdd42e38c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:09.531633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c6b37-72a8-4098-adf3-d703d26705fe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:09.532841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80338840-f025-4dea-85fc-bd7cdd42e38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:09.566542 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3638/7304] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:09.567944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c6b37-72a8-4098-adf3-d703d26705fe admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:09.576279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80338840-f025-4dea-85fc-bd7cdd42e38c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:09.582499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c6b37-72a8-4098-adf3-d703d26705fe admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:09.584151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:09.584765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:09.585066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:09.585324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] PATH_INFO: `/v3/projects/ed60daad951c4718a98a400c45794601` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:09.586525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed60daad951c4718a98a400c45794601)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:09.594081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:09.599147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad8d2d-91fa-425b-b28a-bd273279c668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:09.600130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad8d2d-91fa-425b-b28a-bd273279c668 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:09.603103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad8d2d-91fa-425b-b28a-bd273279c668 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:09.603644 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad8d2d-91fa-425b-b28a-bd273279c668 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:09.606372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad8d2d-91fa-425b-b28a-bd273279c668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:09.610473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad8d2d-91fa-425b-b28a-bd273279c668 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:09.631929 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3639/7305] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:09.636285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad8d2d-91fa-425b-b28a-bd273279c668 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:09.638790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9fad8d2d-91fa-425b-b28a-bd273279c668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:09.714892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed60daad951c4718a98a400c45794601', 'request_id': 'req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:09.733445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed60daad951c4718a98a400c45794601', 'request_id': 'req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:09.741526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad8d2d-91fa-425b-b28a-bd273279c668 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:09.759695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed60daad951c4718a98a400c45794601', 'request_id': 'req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:09.777613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:09.783720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] The token cache is being invalidate because project ed60daad951c4718a98a400c45794601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:09.784013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:09.784696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:09.787969 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3667/7306] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:09 2022] DELETE /identity/v3/projects/ed60daad951c4718a98a400c45794601 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:09.792940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:09.795600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:09.801360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:09.870845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74191eed-455f-409d-b132-febb265493e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:09.871931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74191eed-455f-409d-b132-febb265493e9 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:09.873106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74191eed-455f-409d-b132-febb265493e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:09.873640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74191eed-455f-409d-b132-febb265493e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:09.876343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-74191eed-455f-409d-b132-febb265493e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:09.893748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74191eed-455f-409d-b132-febb265493e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:09.895863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74191eed-455f-409d-b132-febb265493e9 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:09.911359 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3640/7307] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:09.916667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e8b45-b5c5-44cc-880b-fc6cde41d1f2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:09.917304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e8b45-b5c5-44cc-880b-fc6cde41d1f2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:09.918462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e8b45-b5c5-44cc-880b-fc6cde41d1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:09.955521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf8e4b-4c7b-464f-90bb-7cace0c3cf2a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:09.967047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f568f5d-a378-4abc-9109-ffaac9283b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:09.967735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f568f5d-a378-4abc-9109-ffaac9283b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:09.968008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f568f5d-a378-4abc-9109-ffaac9283b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:09.968279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f568f5d-a378-4abc-9109-ffaac9283b39 admin admin] PATH_INFO: `/v3/projects/fb19d460a8da487c81e763cf7ca79c5e/users/b64b9f6f99394f7880ea4ce63c332193/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:09.974669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-846e8b45-b5c5-44cc-880b-fc6cde41d1f2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:09.978286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-846e8b45-b5c5-44cc-880b-fc6cde41d1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:09.994759 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3641/7308] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:10.003370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff50dd-0415-45ca-ad28-07932c4e2d16 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.004427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff50dd-0415-45ca-ad28-07932c4e2d16 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.004877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff50dd-0415-45ca-ad28-07932c4e2d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.011066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f568f5d-a378-4abc-9109-ffaac9283b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb19d460a8da487c81e763cf7ca79c5e, user_id=b64b9f6f99394f7880ea4ce63c332193, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:10.016901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f568f5d-a378-4abc-9109-ffaac9283b39 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:10.042465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b9ff50dd-0415-45ca-ad28-07932c4e2d16 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:10.045357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9ff50dd-0415-45ca-ad28-07932c4e2d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:10.051429 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3668/7309] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:09 2022] PUT /identity/v3/projects/fb19d460a8da487c81e763cf7ca79c5e/users/b64b9f6f99394f7880ea4ce63c332193/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:10.064759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1d651-5ca1-47b4-b174-2e33f24675d2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.064759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1d651-5ca1-47b4-b174-2e33f24675d2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.065027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1d651-5ca1-47b4-b174-2e33f24675d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.104409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9ea1d651-5ca1-47b4-b174-2e33f24675d2 None None] MFA Rules not processed for user `b64b9f6f99394f7880ea4ce63c332193`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:10.105682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9ea1d651-5ca1-47b4-b174-2e33f24675d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:10.153241 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3642/7310] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:19:10.155164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff50dd-0415-45ca-ad28-07932c4e2d16 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.170993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff50dd-0415-45ca-ad28-07932c4e2d16 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:10.178367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597b69-9dae-46d0-830e-edaa0c14a470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:10.179278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597b69-9dae-46d0-830e-edaa0c14a470 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.179865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597b69-9dae-46d0-830e-edaa0c14a470 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.180252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597b69-9dae-46d0-830e-edaa0c14a470 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.181148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597b69-9dae-46d0-830e-edaa0c14a470 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:10.181672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597b69-9dae-46d0-830e-edaa0c14a470 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:10.187444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597b69-9dae-46d0-830e-edaa0c14a470 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:10.200062 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3669/7311] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:10.201741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1d651-5ca1-47b4-b174-2e33f24675d2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.223369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1d651-5ca1-47b4-b174-2e33f24675d2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:10.224995 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3643/7312] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:19:10.229199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597b69-9dae-46d0-830e-edaa0c14a470 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.233236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a442ec-f2f5-4bc8-aa6f-d04408b2cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:10.233859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a442ec-f2f5-4bc8-aa6f-d04408b2cef0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.234205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a442ec-f2f5-4bc8-aa6f-d04408b2cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.234482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a442ec-f2f5-4bc8-aa6f-d04408b2cef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.235120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a442ec-f2f5-4bc8-aa6f-d04408b2cef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:10.237185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a442ec-f2f5-4bc8-aa6f-d04408b2cef0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:10.252604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597b69-9dae-46d0-830e-edaa0c14a470 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:10.264603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ab3b4-0b79-4689-8584-cdff357b5df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:10.268638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab3b4-0b79-4689-8584-cdff357b5df8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.269807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab3b4-0b79-4689-8584-cdff357b5df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.270346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab3b4-0b79-4689-8584-cdff357b5df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.272117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ab3b4-0b79-4689-8584-cdff357b5df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:10.274159 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3670/7313] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:10 2022] POST /identity/v3/projects => generated 411 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:10.279103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a442ec-f2f5-4bc8-aa6f-d04408b2cef0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.280612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ab3b4-0b79-4689-8584-cdff357b5df8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:10.364736 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3644/7314] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:10 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:10.366833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ab3b4-0b79-4689-8584-cdff357b5df8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.368962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6d8ab3b4-0b79-4689-8584-cdff357b5df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:10.378949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:10.397235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a442ec-f2f5-4bc8-aa6f-d04408b2cef0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:10.480809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ab3b4-0b79-4689-8584-cdff357b5df8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:10.491240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:10.491972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.492318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.492652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] PATH_INFO: `/v3/users/fc7967f332c1483395accf914d75979a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.494999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc7967f332c1483395accf914d75979a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:10.497531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:10.528217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562203d-ee1c-4925-8abf-741d2d741b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:10.529466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562203d-ee1c-4925-8abf-741d2d741b03 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.529805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562203d-ee1c-4925-8abf-741d2d741b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.530124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562203d-ee1c-4925-8abf-741d2d741b03 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.530817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562203d-ee1c-4925-8abf-741d2d741b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:10.532843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562203d-ee1c-4925-8abf-741d2d741b03 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:10.548690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:10.593440 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3671/7315] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:10 2022] POST /identity/v3/users => generated 312 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:10.604254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562203d-ee1c-4925-8abf-741d2d741b03 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.620700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc7967f332c1483395accf914d75979a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:10.627381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562203d-ee1c-4925-8abf-741d2d741b03 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:10.643482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc7967f332c1483395accf914d75979a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:10.644529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3eb8c-a877-46f4-86af-fa09d8435864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:10.645176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3eb8c-a877-46f4-86af-fa09d8435864 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.648373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3eb8c-a877-46f4-86af-fa09d8435864 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.648373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3eb8c-a877-46f4-86af-fa09d8435864 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.648373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3eb8c-a877-46f4-86af-fa09d8435864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:10.648373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3eb8c-a877-46f4-86af-fa09d8435864 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:10.658866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc7967f332c1483395accf914d75979a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:10.675461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc7967f332c1483395accf914d75979a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:10.676758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc7967f332c1483395accf914d75979a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:10.686794 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3645/7316] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:10 2022] DELETE /identity/v3/users/fc7967f332c1483395accf914d75979a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:10.688224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.690595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:10.696020 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3672/7317] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:10 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:10.696020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c42a6-b734-4218-9925-5f35f1efecaf None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.696020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c42a6-b734-4218-9925-5f35f1efecaf None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.696020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c42a6-b734-4218-9925-5f35f1efecaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.718639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-da2c42a6-b734-4218-9925-5f35f1efecaf None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:10.719907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-da2c42a6-b734-4218-9925-5f35f1efecaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:10.813699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab2609-ba27-44da-ab10-d8b74b95f3e7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:10.820438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0af11f-b7ac-4b28-a3e9-ac3f5b183583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:10.821061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0af11f-b7ac-4b28-a3e9-ac3f5b183583 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.821361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0af11f-b7ac-4b28-a3e9-ac3f5b183583 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.821675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0af11f-b7ac-4b28-a3e9-ac3f5b183583 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.822316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0af11f-b7ac-4b28-a3e9-ac3f5b183583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:10.824039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0af11f-b7ac-4b28-a3e9-ac3f5b183583 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:10.828638 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3673/7318] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:10.829563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c42a6-b734-4218-9925-5f35f1efecaf None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.832305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-da2c42a6-b734-4218-9925-5f35f1efecaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:10.840419 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3646/7319] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:10.844974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0af11f-b7ac-4b28-a3e9-ac3f5b183583 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.862367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c42a6-b734-4218-9925-5f35f1efecaf None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:10.864565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0af11f-b7ac-4b28-a3e9-ac3f5b183583 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:10.867701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49beca91-3814-4147-8203-d370e13747b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:10.868497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beca91-3814-4147-8203-d370e13747b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.868778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beca91-3814-4147-8203-d370e13747b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.869072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beca91-3814-4147-8203-d370e13747b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.869878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49beca91-3814-4147-8203-d370e13747b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:10.871718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49beca91-3814-4147-8203-d370e13747b3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:10.880185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce62639-3da8-41ee-ac7b-b6a64bce35a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:10.882551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce62639-3da8-41ee-ac7b-b6a64bce35a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:10.883739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce62639-3da8-41ee-ac7b-b6a64bce35a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:10.884107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce62639-3da8-41ee-ac7b-b6a64bce35a2 admin admin] PATH_INFO: `/v3/projects/9f4542951beb43e991801291b4640add/users/0f063b0db765413e837cdca254dd2374/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:10.884556 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3674/7320] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:10.885724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49beca91-3814-4147-8203-d370e13747b3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.887520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-49beca91-3814-4147-8203-d370e13747b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:10.927212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce62639-3da8-41ee-ac7b-b6a64bce35a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4542951beb43e991801291b4640add, user_id=0f063b0db765413e837cdca254dd2374, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:10.931325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce62639-3da8-41ee-ac7b-b6a64bce35a2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:10.970910 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3647/7321] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:10 2022] PUT /identity/v3/projects/9f4542951beb43e991801291b4640add/users/0f063b0db765413e837cdca254dd2374/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:10.970910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce62639-3da8-41ee-ac7b-b6a64bce35a2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:10.971209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49beca91-3814-4147-8203-d370e13747b3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:11.004692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:11.088437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625e4a0-221a-4522-9565-87b6f0b524ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:11.089368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625e4a0-221a-4522-9565-87b6f0b524ec service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:11.089718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625e4a0-221a-4522-9565-87b6f0b524ec service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:11.090123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625e4a0-221a-4522-9565-87b6f0b524ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:11.098056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce62639-3da8-41ee-ac7b-b6a64bce35a2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:11.102347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625e4a0-221a-4522-9565-87b6f0b524ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:11.104123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116acb0-1aef-400f-bb21-d1d0f0e72922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:11.104298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625e4a0-221a-4522-9565-87b6f0b524ec service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:11.104864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116acb0-1aef-400f-bb21-d1d0f0e72922 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:11.105661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116acb0-1aef-400f-bb21-d1d0f0e72922 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:11.105942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116acb0-1aef-400f-bb21-d1d0f0e72922 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:11.106666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116acb0-1aef-400f-bb21-d1d0f0e72922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:11.108541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116acb0-1aef-400f-bb21-d1d0f0e72922 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:11.116883 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3675/7322] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:11.117913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625e4a0-221a-4522-9565-87b6f0b524ec service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:11.121458 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3648/7323] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:11.127530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116acb0-1aef-400f-bb21-d1d0f0e72922 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:11.139450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625e4a0-221a-4522-9565-87b6f0b524ec service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:11.144981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116acb0-1aef-400f-bb21-d1d0f0e72922 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:11.149494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a359f2a-dd2f-485d-85c6-a560301bd04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:11.149965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a85933-e506-490b-93eb-bf5097aea272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:11.150349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a359f2a-dd2f-485d-85c6-a560301bd04d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:11.150621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a85933-e506-490b-93eb-bf5097aea272 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:11.150806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a359f2a-dd2f-485d-85c6-a560301bd04d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:11.150933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a85933-e506-490b-93eb-bf5097aea272 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:11.151149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a85933-e506-490b-93eb-bf5097aea272 admin admin] PATH_INFO: `/v3/projects/86027ace3c2c4187b73b15b96e5ee6b2/users/04c9edb9fdee444aae929fbcad8424de/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:11.152310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a359f2a-dd2f-485d-85c6-a560301bd04d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:11.153086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a359f2a-dd2f-485d-85c6-a560301bd04d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:11.155614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:11.161890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a359f2a-dd2f-485d-85c6-a560301bd04d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:11.177268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a85933-e506-490b-93eb-bf5097aea272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86027ace3c2c4187b73b15b96e5ee6b2, user_id=04c9edb9fdee444aae929fbcad8424de, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:11.182336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a85933-e506-490b-93eb-bf5097aea272 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:11.192425 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3676/7324] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:11.202885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a359f2a-dd2f-485d-85c6-a560301bd04d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:11.220762 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3649/7325] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:11 2022] PUT /identity/v3/projects/86027ace3c2c4187b73b15b96e5ee6b2/users/04c9edb9fdee444aae929fbcad8424de/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:11.221979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a359f2a-dd2f-485d-85c6-a560301bd04d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:11.234299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b816cd-66e9-49aa-b41e-b05547647659 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:11.234874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b816cd-66e9-49aa-b41e-b05547647659 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:11.235160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b816cd-66e9-49aa-b41e-b05547647659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:11.281571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-34b816cd-66e9-49aa-b41e-b05547647659 None None] MFA Rules not processed for user `04c9edb9fdee444aae929fbcad8424de`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:11.284235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-34b816cd-66e9-49aa-b41e-b05547647659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:11.341468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7521335-7bd9-435b-90e6-426d0f49207d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:11.342174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7521335-7bd9-435b-90e6-426d0f49207d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:11.342452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7521335-7bd9-435b-90e6-426d0f49207d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:11.342802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7521335-7bd9-435b-90e6-426d0f49207d admin admin] PATH_INFO: `/v3/projects/9f4542951beb43e991801291b4640add/users/0f063b0db765413e837cdca254dd2374/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:11.350977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7521335-7bd9-435b-90e6-426d0f49207d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4542951beb43e991801291b4640add, user_id=0f063b0db765413e837cdca254dd2374, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:11.355801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7521335-7bd9-435b-90e6-426d0f49207d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:11.382328 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3677/7326] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:11 2022] PUT /identity/v3/projects/9f4542951beb43e991801291b4640add/users/0f063b0db765413e837cdca254dd2374/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:11.394461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de861b05-a7e8-45e9-b372-559c909e1184 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:11.394461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de861b05-a7e8-45e9-b372-559c909e1184 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:11.394795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de861b05-a7e8-45e9-b372-559c909e1184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:11.450778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-de861b05-a7e8-45e9-b372-559c909e1184 None None] MFA Rules not processed for user `0f063b0db765413e837cdca254dd2374`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:11.456932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de861b05-a7e8-45e9-b372-559c909e1184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:11.505740 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3650/7327] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:11.506673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b816cd-66e9-49aa-b41e-b05547647659 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:11.514340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-34b816cd-66e9-49aa-b41e-b05547647659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:11.611537 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3678/7328] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:11.621024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de861b05-a7e8-45e9-b372-559c909e1184 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:11.624769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de861b05-a7e8-45e9-b372-559c909e1184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:11.694497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:11.695706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b816cd-66e9-49aa-b41e-b05547647659 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:11.741259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de861b05-a7e8-45e9-b372-559c909e1184 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:11.746873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f7d9f-5e28-4fe2-b4ec-e4372e07cbd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:11.747748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f7d9f-5e28-4fe2-b4ec-e4372e07cbd2 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:11.748261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f7d9f-5e28-4fe2-b4ec-e4372e07cbd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:11.748625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f7d9f-5e28-4fe2-b4ec-e4372e07cbd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:11.845945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc87f57-c8dd-4ad5-a7ab-34e5e9952a09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:11.846988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc87f57-c8dd-4ad5-a7ab-34e5e9952a09 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:11.847408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc87f57-c8dd-4ad5-a7ab-34e5e9952a09 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:11.847853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc87f57-c8dd-4ad5-a7ab-34e5e9952a09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:11.850495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-acc87f57-c8dd-4ad5-a7ab-34e5e9952a09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:11.861264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f7d9f-5e28-4fe2-b4ec-e4372e07cbd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:11.863968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f7d9f-5e28-4fe2-b4ec-e4372e07cbd2 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:11.868591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc87f57-c8dd-4ad5-a7ab-34e5e9952a09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:11.870985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc87f57-c8dd-4ad5-a7ab-34e5e9952a09 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:11.876814 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3679/7329] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:11.877942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f7d9f-5e28-4fe2-b4ec-e4372e07cbd2 service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:11.880595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ac7f7d9f-5e28-4fe2-b4ec-e4372e07cbd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:11.890118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:11.912400 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3651/7330] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:11.913729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc87f57-c8dd-4ad5-a7ab-34e5e9952a09 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:11.975789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f7d9f-5e28-4fe2-b4ec-e4372e07cbd2 service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:11.980974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d9cc4-8847-485f-b5d0-2aa583486293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:11.983777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d9cc4-8847-485f-b5d0-2aa583486293 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:11.984096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d9cc4-8847-485f-b5d0-2aa583486293 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:11.984509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d9cc4-8847-485f-b5d0-2aa583486293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:11.999927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d9cc4-8847-485f-b5d0-2aa583486293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:12.000767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d9cc4-8847-485f-b5d0-2aa583486293 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:12.026016 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3680/7331] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:11 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:12.030519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d9cc4-8847-485f-b5d0-2aa583486293 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:12.033097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-250d9cc4-8847-485f-b5d0-2aa583486293 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:12.055790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc87f57-c8dd-4ad5-a7ab-34e5e9952a09 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:12.067819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d9cc4-8847-485f-b5d0-2aa583486293 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:12.083952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:12.087148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:12.088429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:12.088797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] PATH_INFO: `/v3/users/04c9edb9fdee444aae929fbcad8424de` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:12.089133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:12.089790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:12.090065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:12.090337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] PATH_INFO: `/v3/projects/82b30bfac5e34d4fa7a797b561d34d58` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:12.092230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c9edb9fdee444aae929fbcad8424de)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:12.092581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=82b30bfac5e34d4fa7a797b561d34d58)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:12.096173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:12.097798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:12.245824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c9edb9fdee444aae929fbcad8424de'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:12.315139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82b30bfac5e34d4fa7a797b561d34d58', 'request_id': 'req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:12.324235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c9edb9fdee444aae929fbcad8424de'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:12.329977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82b30bfac5e34d4fa7a797b561d34d58', 'request_id': 'req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:12.340698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82b30bfac5e34d4fa7a797b561d34d58', 'request_id': 'req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:12.347397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c9edb9fdee444aae929fbcad8424de'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:12.363171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c9edb9fdee444aae929fbcad8424de'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:12.366764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-075cdaae-ccb5-476b-994e-9d26290ddef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c9edb9fdee444aae929fbcad8424de'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:12.383679 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3652/7332] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:11 2022] DELETE /identity/v3/users/04c9edb9fdee444aae929fbcad8424de => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:12.386239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf86526-c474-46c9-beff-f323d5b7fb7f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:12.386620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf86526-c474-46c9-beff-f323d5b7fb7f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:12.386969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf86526-c474-46c9-beff-f323d5b7fb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:12.387278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] The token cache is being invalidate because project 82b30bfac5e34d4fa7a797b561d34d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:12.388274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:12.389468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:12.391349 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3681/7333] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:12 2022] DELETE /identity/v3/projects/82b30bfac5e34d4fa7a797b561d34d58 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:12.440613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-aaf86526-c474-46c9-beff-f323d5b7fb7f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:12.446458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aaf86526-c474-46c9-beff-f323d5b7fb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:12.772334 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3653/7334] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:12.848952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:12.850845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:12.869239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:13.044818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a3b3d-8715-4868-b3f9-96f9e13d43ea admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:13.130190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92adaf4-b724-40b0-8556-a7c5d8dfcc79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:13.130910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92adaf4-b724-40b0-8556-a7c5d8dfcc79 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:13.131519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92adaf4-b724-40b0-8556-a7c5d8dfcc79 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:13.131789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92adaf4-b724-40b0-8556-a7c5d8dfcc79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:13.150150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92adaf4-b724-40b0-8556-a7c5d8dfcc79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:13.151977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92adaf4-b724-40b0-8556-a7c5d8dfcc79 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:13.168863 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3682/7335] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:14.431899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf86526-c474-46c9-beff-f323d5b7fb7f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:14.433273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aaf86526-c474-46c9-beff-f323d5b7fb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:14.453459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf86526-c474-46c9-beff-f323d5b7fb7f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:14.459079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:14.460304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:14.460728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:14.461265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] PATH_INFO: `/v3/projects/86027ace3c2c4187b73b15b96e5ee6b2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:14.463276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86027ace3c2c4187b73b15b96e5ee6b2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:14.465400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:14.482779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:14.529183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86027ace3c2c4187b73b15b96e5ee6b2', 'request_id': 'req-ff07dd97-22e5-4e4e-bf75-1dae50435531'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:14.538910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86027ace3c2c4187b73b15b96e5ee6b2', 'request_id': 'req-ff07dd97-22e5-4e4e-bf75-1dae50435531'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:14.548661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86027ace3c2c4187b73b15b96e5ee6b2', 'request_id': 'req-ff07dd97-22e5-4e4e-bf75-1dae50435531'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:14.561923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] The token cache is being invalidate because project 86027ace3c2c4187b73b15b96e5ee6b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:14.562232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:14.562889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff07dd97-22e5-4e4e-bf75-1dae50435531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:14.564623 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3654/7336] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:14 2022] DELETE /identity/v3/projects/86027ace3c2c4187b73b15b96e5ee6b2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:14.569359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fc89c-d87e-406b-abaa-6880fe559b12 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:14.570278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fc89c-d87e-406b-abaa-6880fe559b12 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:14.570595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fc89c-d87e-406b-abaa-6880fe559b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:14.591804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-727fc89c-d87e-406b-abaa-6880fe559b12 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:14.593220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-727fc89c-d87e-406b-abaa-6880fe559b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:14.667429 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3683/7337] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:14.676902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb4a03-ec9f-4ee0-900d-540411cbb8b2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:14.677238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb4a03-ec9f-4ee0-900d-540411cbb8b2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:14.677519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb4a03-ec9f-4ee0-900d-540411cbb8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:14.702256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bebb4a03-ec9f-4ee0-900d-540411cbb8b2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:14.703506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bebb4a03-ec9f-4ee0-900d-540411cbb8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:14.712558 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3655/7338] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:14.717836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fc89c-d87e-406b-abaa-6880fe559b12 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:14.729416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fc89c-d87e-406b-abaa-6880fe559b12 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:14.736403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9724f-f80d-458f-bd63-3c020b0b5f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:14.737281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9724f-f80d-458f-bd63-3c020b0b5f22 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:14.737281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9724f-f80d-458f-bd63-3c020b0b5f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:14.737581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9724f-f80d-458f-bd63-3c020b0b5f22 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:14.738196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9724f-f80d-458f-bd63-3c020b0b5f22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:14.738491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9724f-f80d-458f-bd63-3c020b0b5f22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:14.740981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9724f-f80d-458f-bd63-3c020b0b5f22 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:14.757090 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3684/7339] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:19:14.759306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb4a03-ec9f-4ee0-900d-540411cbb8b2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:14.769491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb4a03-ec9f-4ee0-900d-540411cbb8b2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:14.775424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc1ead-59c6-4012-8ab8-fa27c28330c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:14.776131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1ead-59c6-4012-8ab8-fa27c28330c6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:14.776465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1ead-59c6-4012-8ab8-fa27c28330c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:14.776803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1ead-59c6-4012-8ab8-fa27c28330c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:14.777537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc1ead-59c6-4012-8ab8-fa27c28330c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:14.780140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc1ead-59c6-4012-8ab8-fa27c28330c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:14.821592 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3656/7340] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:14 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:14.823056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9724f-f80d-458f-bd63-3c020b0b5f22 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:14.834226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9724f-f80d-458f-bd63-3c020b0b5f22 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:14.904744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:14.913446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db058908-47d3-4227-a02c-53ac744d50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:14.913446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db058908-47d3-4227-a02c-53ac744d50b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:14.914400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db058908-47d3-4227-a02c-53ac744d50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:14.914400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db058908-47d3-4227-a02c-53ac744d50b3 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:14.914758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db058908-47d3-4227-a02c-53ac744d50b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:14.916820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db058908-47d3-4227-a02c-53ac744d50b3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:14.948912 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3685/7341] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:14 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:14.953501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc1ead-59c6-4012-8ab8-fa27c28330c6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:14.966796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc1ead-59c6-4012-8ab8-fa27c28330c6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:14.972968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fedef-ae95-4e5e-b158-12434a5303a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:14.974072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fedef-ae95-4e5e-b158-12434a5303a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:14.974622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fedef-ae95-4e5e-b158-12434a5303a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:14.975334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fedef-ae95-4e5e-b158-12434a5303a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:14.976271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fedef-ae95-4e5e-b158-12434a5303a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:14.978632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fedef-ae95-4e5e-b158-12434a5303a9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:14.992593 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3657/7342] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:14.996561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db058908-47d3-4227-a02c-53ac744d50b3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:15.013604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db058908-47d3-4227-a02c-53ac744d50b3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:15.021600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b67cc0-7082-4a34-89b6-76c3c43a10b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:15.022478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b67cc0-7082-4a34-89b6-76c3c43a10b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:15.022816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b67cc0-7082-4a34-89b6-76c3c43a10b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:15.023160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b67cc0-7082-4a34-89b6-76c3c43a10b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:15.023918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b67cc0-7082-4a34-89b6-76c3c43a10b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:15.026086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b67cc0-7082-4a34-89b6-76c3c43a10b8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:15.039584 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3686/7343] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:15.043734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fedef-ae95-4e5e-b158-12434a5303a9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:15.058888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fedef-ae95-4e5e-b158-12434a5303a9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:15.072035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9f066-588c-4fed-a601-bce6483809b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:15.073010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9f066-588c-4fed-a601-bce6483809b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:15.073576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9f066-588c-4fed-a601-bce6483809b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:15.074089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9f066-588c-4fed-a601-bce6483809b9 admin admin] PATH_INFO: `/v3/projects/437c5b0d36db41d5ba7f5046682b6785/users/484f1214241f4939be0809e6ca6c502f/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:15.097356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9f066-588c-4fed-a601-bce6483809b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437c5b0d36db41d5ba7f5046682b6785, user_id=484f1214241f4939be0809e6ca6c502f, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:15.100709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9f066-588c-4fed-a601-bce6483809b9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:15.128375 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3658/7344] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:15 2022] PUT /identity/v3/projects/437c5b0d36db41d5ba7f5046682b6785/users/484f1214241f4939be0809e6ca6c502f/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:15.139533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dda2f-ef5c-42a8-9552-0e3b65be8149 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:15.139816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dda2f-ef5c-42a8-9552-0e3b65be8149 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:15.140081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dda2f-ef5c-42a8-9552-0e3b65be8149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:15.179509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0c7dda2f-ef5c-42a8-9552-0e3b65be8149 None None] MFA Rules not processed for user `484f1214241f4939be0809e6ca6c502f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:15.180804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0c7dda2f-ef5c-42a8-9552-0e3b65be8149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:15.328611 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3687/7345] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:15.344967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9f066-588c-4fed-a601-bce6483809b9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:15.347004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2ee9f066-588c-4fed-a601-bce6483809b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:15.444117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9f066-588c-4fed-a601-bce6483809b9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:15.449377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23811db8-bee9-41b5-9514-b2c7b37e5f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:15.450734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23811db8-bee9-41b5-9514-b2c7b37e5f19 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:15.451165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23811db8-bee9-41b5-9514-b2c7b37e5f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:15.451581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23811db8-bee9-41b5-9514-b2c7b37e5f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:15.461890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23811db8-bee9-41b5-9514-b2c7b37e5f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:15.463778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23811db8-bee9-41b5-9514-b2c7b37e5f19 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:15.477450 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3659/7346] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:16.400630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7dda2f-ef5c-42a8-9552-0e3b65be8149 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:16.405099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0c7dda2f-ef5c-42a8-9552-0e3b65be8149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:16.491233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7dda2f-ef5c-42a8-9552-0e3b65be8149 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:16.495051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c7c10-e9a5-4dc1-8c0a-fe913127efca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:16.495727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c7c10-e9a5-4dc1-8c0a-fe913127efca service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:16.496704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c7c10-e9a5-4dc1-8c0a-fe913127efca service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:16.496704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c7c10-e9a5-4dc1-8c0a-fe913127efca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:16.498901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7f6c7c10-e9a5-4dc1-8c0a-fe913127efca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:16.516099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c7c10-e9a5-4dc1-8c0a-fe913127efca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:16.518054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c7c10-e9a5-4dc1-8c0a-fe913127efca service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:16.538422 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3688/7347] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:16.599889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23811db8-bee9-41b5-9514-b2c7b37e5f19 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:16.601468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-23811db8-bee9-41b5-9514-b2c7b37e5f19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:16.650989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:16.680172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23811db8-bee9-41b5-9514-b2c7b37e5f19 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:16.685577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:16.686204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:16.686475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:16.686739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] PATH_INFO: `/v3/users/b64b9f6f99394f7880ea4ce63c332193` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:16.687878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b64b9f6f99394f7880ea4ce63c332193)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:16.689837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:16.741458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b64b9f6f99394f7880ea4ce63c332193'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:16.756362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b64b9f6f99394f7880ea4ce63c332193'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:16.766415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b64b9f6f99394f7880ea4ce63c332193'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:16.775118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b64b9f6f99394f7880ea4ce63c332193'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:16.775819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b64b9f6f99394f7880ea4ce63c332193'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:16.785105 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3660/7348] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:16 2022] DELETE /identity/v3/users/b64b9f6f99394f7880ea4ce63c332193 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:16.789375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664893a0-9a66-4b33-a3e6-acd267f5d983 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:16.790441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664893a0-9a66-4b33-a3e6-acd267f5d983 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:16.790713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664893a0-9a66-4b33-a3e6-acd267f5d983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:16.817737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-664893a0-9a66-4b33-a3e6-acd267f5d983 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:16.818955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-664893a0-9a66-4b33-a3e6-acd267f5d983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:16.944874 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3689/7349] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:16.975846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:16.979242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:17.116232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc54f2-ea55-4850-afad-64f75792afb3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:17.122393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468e199-3d35-4470-a94f-aa4648bd8768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:17.123820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468e199-3d35-4470-a94f-aa4648bd8768 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:17.124258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468e199-3d35-4470-a94f-aa4648bd8768 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:17.124629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468e199-3d35-4470-a94f-aa4648bd8768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:17.143285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468e199-3d35-4470-a94f-aa4648bd8768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:17.145799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468e199-3d35-4470-a94f-aa4648bd8768 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:17.162433 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3661/7350] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:17.579462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664893a0-9a66-4b33-a3e6-acd267f5d983 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:17.581129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-664893a0-9a66-4b33-a3e6-acd267f5d983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:17.601103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:17.782717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664893a0-9a66-4b33-a3e6-acd267f5d983 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:17.799291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e533c-286e-4bdc-8a03-cd5ef6466dd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:17.799986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e533c-286e-4bdc-8a03-cd5ef6466dd8 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:17.801140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e533c-286e-4bdc-8a03-cd5ef6466dd8 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:17.801773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e533c-286e-4bdc-8a03-cd5ef6466dd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:17.811380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cb3e533c-286e-4bdc-8a03-cd5ef6466dd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:17.832372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e533c-286e-4bdc-8a03-cd5ef6466dd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:17.834497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e533c-286e-4bdc-8a03-cd5ef6466dd8 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:17.870348 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3690/7351] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:18.133883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468e199-3d35-4470-a94f-aa4648bd8768 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:18.135339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e468e199-3d35-4470-a94f-aa4648bd8768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:18.156097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468e199-3d35-4470-a94f-aa4648bd8768 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:18.173856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:18.174598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:18.175013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:18.175377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] PATH_INFO: `/v3/projects/fb19d460a8da487c81e763cf7ca79c5e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:18.177338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb19d460a8da487c81e763cf7ca79c5e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:18.180017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:18.319827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb19d460a8da487c81e763cf7ca79c5e', 'request_id': 'req-2d29afc3-6a94-4a59-9c74-220bcc375edb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:18.334329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb19d460a8da487c81e763cf7ca79c5e', 'request_id': 'req-2d29afc3-6a94-4a59-9c74-220bcc375edb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:18.338493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:18.351642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb19d460a8da487c81e763cf7ca79c5e', 'request_id': 'req-2d29afc3-6a94-4a59-9c74-220bcc375edb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:18.376928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] The token cache is being invalidate because project fb19d460a8da487c81e763cf7ca79c5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:18.378051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:18.379133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:18.382313 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3662/7352] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:18 2022] DELETE /identity/v3/projects/fb19d460a8da487c81e763cf7ca79c5e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:18.386409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e533c-286e-4bdc-8a03-cd5ef6466dd8 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:18.401307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e533c-286e-4bdc-8a03-cd5ef6466dd8 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:18.501785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:18.502444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:18.502726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:18.503001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] PATH_INFO: `/v3/users/0f063b0db765413e837cdca254dd2374` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:18.504175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f063b0db765413e837cdca254dd2374)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:18.506155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:18.569068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f063b0db765413e837cdca254dd2374'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:18.580708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f063b0db765413e837cdca254dd2374'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:18.601903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f063b0db765413e837cdca254dd2374'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:18.612168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f063b0db765413e837cdca254dd2374'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:18.612933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ee56fc3a-6979-4bc4-b029-603cfbce6520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f063b0db765413e837cdca254dd2374'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:18.624270 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3691/7353] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:18 2022] DELETE /identity/v3/users/0f063b0db765413e837cdca254dd2374 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:18.858762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:18.860247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:18.874567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29afc3-6a94-4a59-9c74-220bcc375edb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:18.955908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:18.956595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:18.956874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:18.957167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] PATH_INFO: `/v3/projects/9f4542951beb43e991801291b4640add` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:18.958807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f4542951beb43e991801291b4640add)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:18.960872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:19.026405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f4542951beb43e991801291b4640add', 'request_id': 'req-8b886e2a-a18d-4793-a77d-f40c383f448f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:19.058168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f4542951beb43e991801291b4640add', 'request_id': 'req-8b886e2a-a18d-4793-a77d-f40c383f448f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:19.069963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f4542951beb43e991801291b4640add', 'request_id': 'req-8b886e2a-a18d-4793-a77d-f40c383f448f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:19.093207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] The token cache is being invalidate because project 9f4542951beb43e991801291b4640add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:19.093496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:19.094219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8b886e2a-a18d-4793-a77d-f40c383f448f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:19.100654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2d2e4-98cb-4f10-9c48-e403330a3824 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.100935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2d2e4-98cb-4f10-9c48-e403330a3824 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.101230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2d2e4-98cb-4f10-9c48-e403330a3824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:19.103955 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3663/7354] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:18 2022] DELETE /identity/v3/projects/9f4542951beb43e991801291b4640add => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:19.126526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-62d2d2e4-98cb-4f10-9c48-e403330a3824 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:19.128619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-62d2d2e4-98cb-4f10-9c48-e403330a3824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:19.136286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:19.211274 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3692/7355] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:19.222726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed6ccb-3603-46c8-bcc6-112389ecb051 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.224229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed6ccb-3603-46c8-bcc6-112389ecb051 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.224505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed6ccb-3603-46c8-bcc6-112389ecb051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:19.252283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e2ed6ccb-3603-46c8-bcc6-112389ecb051 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:19.253556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e2ed6ccb-3603-46c8-bcc6-112389ecb051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:19.268263 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3664/7356] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:19.269235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2d2e4-98cb-4f10-9c48-e403330a3824 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:19.279966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2d2e4-98cb-4f10-9c48-e403330a3824 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:19.287911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f99c1-ed0e-43d1-8fb5-fd8d50738f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:19.288634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f99c1-ed0e-43d1-8fb5-fd8d50738f35 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.289011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f99c1-ed0e-43d1-8fb5-fd8d50738f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.289398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f99c1-ed0e-43d1-8fb5-fd8d50738f35 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:19.290150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f99c1-ed0e-43d1-8fb5-fd8d50738f35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:19.292732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f99c1-ed0e-43d1-8fb5-fd8d50738f35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:19.298881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f99c1-ed0e-43d1-8fb5-fd8d50738f35 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:19.299845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed6ccb-3603-46c8-bcc6-112389ecb051 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:19.301295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e2ed6ccb-3603-46c8-bcc6-112389ecb051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:19.312551 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3693/7357] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:19:19.316705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f99c1-ed0e-43d1-8fb5-fd8d50738f35 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:19.324349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed6ccb-3603-46c8-bcc6-112389ecb051 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:19.331010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f99c1-ed0e-43d1-8fb5-fd8d50738f35 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:19.336655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc778d5-1f71-4910-a66d-ccca2543d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:19.337298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc778d5-1f71-4910-a66d-ccca2543d3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.337571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc778d5-1f71-4910-a66d-ccca2543d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.338637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:19.338783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc778d5-1f71-4910-a66d-ccca2543d3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:19.338871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.338962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc778d5-1f71-4910-a66d-ccca2543d3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:19.341545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc778d5-1f71-4910-a66d-ccca2543d3e0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:19.342174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.342492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] PATH_INFO: `/v3/users/484f1214241f4939be0809e6ca6c502f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:19.347103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=484f1214241f4939be0809e6ca6c502f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:19.353079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:19.400007 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3694/7358] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:19 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:19.409174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc778d5-1f71-4910-a66d-ccca2543d3e0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:19.466192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '484f1214241f4939be0809e6ca6c502f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:19.517755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:19.519744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '484f1214241f4939be0809e6ca6c502f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:19.567797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '484f1214241f4939be0809e6ca6c502f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:19.587231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '484f1214241f4939be0809e6ca6c502f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:19.588188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ab864313-f0d1-49d6-8e5f-750cabaad550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '484f1214241f4939be0809e6ca6c502f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:19.601957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc778d5-1f71-4910-a66d-ccca2543d3e0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:19.605685 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3665/7359] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:19 2022] DELETE /identity/v3/users/484f1214241f4939be0809e6ca6c502f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:19.610030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2216170-9eef-4f33-bf5a-bd6059a34e16 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.610231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2216170-9eef-4f33-bf5a-bd6059a34e16 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.610787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2216170-9eef-4f33-bf5a-bd6059a34e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:19.671885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f2216170-9eef-4f33-bf5a-bd6059a34e16 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:19.673774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f2216170-9eef-4f33-bf5a-bd6059a34e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:19.740231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c2ca5-d405-4302-acf9-818d2ce8b081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:19.740862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c2ca5-d405-4302-acf9-818d2ce8b081 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.741172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c2ca5-d405-4302-acf9-818d2ce8b081 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.745954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c2ca5-d405-4302-acf9-818d2ce8b081 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:19.747424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c2ca5-d405-4302-acf9-818d2ce8b081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:19.749398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c2ca5-d405-4302-acf9-818d2ce8b081 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:19.789089 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3695/7360] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:19 2022] POST /identity/v3/users => generated 287 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:19.790683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c2ca5-d405-4302-acf9-818d2ce8b081 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:19.794344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-998c2ca5-d405-4302-acf9-818d2ce8b081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:19.802333 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3666/7361] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:19.819573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c2ca5-d405-4302-acf9-818d2ce8b081 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:19.830679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2216170-9eef-4f33-bf5a-bd6059a34e16 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:19.832359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f2216170-9eef-4f33-bf5a-bd6059a34e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:19.833363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7049c-0276-4462-a9e1-eeace6d091b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:19.834013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7049c-0276-4462-a9e1-eeace6d091b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.834284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7049c-0276-4462-a9e1-eeace6d091b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.834547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7049c-0276-4462-a9e1-eeace6d091b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:19.835182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7049c-0276-4462-a9e1-eeace6d091b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:19.836823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7049c-0276-4462-a9e1-eeace6d091b8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:19.860050 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3696/7362] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:19:19.864450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7049c-0276-4462-a9e1-eeace6d091b8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:19.873340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:19.883617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7049c-0276-4462-a9e1-eeace6d091b8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:19.895556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8db477-3fae-44d5-af24-3c204b2ef543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:19.896286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8db477-3fae-44d5-af24-3c204b2ef543 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.896737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8db477-3fae-44d5-af24-3c204b2ef543 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.897213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8db477-3fae-44d5-af24-3c204b2ef543 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:19.898009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8db477-3fae-44d5-af24-3c204b2ef543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:19.901306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8db477-3fae-44d5-af24-3c204b2ef543 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:19.933417 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3697/7363] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:19:19.937859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8db477-3fae-44d5-af24-3c204b2ef543 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:19.956104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8db477-3fae-44d5-af24-3c204b2ef543 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:19.965076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ed93a-2cea-49d7-8af5-eb8987bbabb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:19.966691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ed93a-2cea-49d7-8af5-eb8987bbabb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.967149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ed93a-2cea-49d7-8af5-eb8987bbabb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.967567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ed93a-2cea-49d7-8af5-eb8987bbabb1 admin admin] PATH_INFO: `/v3/projects/06ff4e9366b94f8f916f458eae61e5d6/users/7514421b2f5c4380a8a67d6fd66b508b/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:19.983969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2216170-9eef-4f33-bf5a-bd6059a34e16 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:19.989717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce88feb-061f-46a4-a9db-1d6eca725241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:19.990364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce88feb-061f-46a4-a9db-1d6eca725241 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:19.990639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce88feb-061f-46a4-a9db-1d6eca725241 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:19.991106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce88feb-061f-46a4-a9db-1d6eca725241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:20.004091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ed93a-2cea-49d7-8af5-eb8987bbabb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ff4e9366b94f8f916f458eae61e5d6, user_id=7514421b2f5c4380a8a67d6fd66b508b, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:20.006335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce88feb-061f-46a4-a9db-1d6eca725241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:20.008633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ed93a-2cea-49d7-8af5-eb8987bbabb1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:20.008805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce88feb-061f-46a4-a9db-1d6eca725241 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:20.025248 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3667/7364] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:19:20.035620 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3698/7365] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:19 2022] PUT /identity/v3/projects/06ff4e9366b94f8f916f458eae61e5d6/users/7514421b2f5c4380a8a67d6fd66b508b/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:19:20.049595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc664-e01c-4c89-88ed-f557b7b95e34 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:20.050125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc664-e01c-4c89-88ed-f557b7b95e34 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:20.050551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc664-e01c-4c89-88ed-f557b7b95e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:20.095609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5c6dc664-e01c-4c89-88ed-f557b7b95e34 None None] MFA Rules not processed for user `7514421b2f5c4380a8a67d6fd66b508b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:20.097667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5c6dc664-e01c-4c89-88ed-f557b7b95e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:20.235673 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3668/7366] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:20.240367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ed93a-2cea-49d7-8af5-eb8987bbabb1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:20.355671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ed93a-2cea-49d7-8af5-eb8987bbabb1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:20.368304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412936d1-61e1-4ca2-a6e7-67c34e065756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:20.369346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412936d1-61e1-4ca2-a6e7-67c34e065756 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:20.369817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412936d1-61e1-4ca2-a6e7-67c34e065756 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:20.370476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412936d1-61e1-4ca2-a6e7-67c34e065756 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:20.371363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412936d1-61e1-4ca2-a6e7-67c34e065756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:20.375100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412936d1-61e1-4ca2-a6e7-67c34e065756 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:20.430018 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3699/7367] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:20 2022] POST /identity/v3/projects => generated 369 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:20.433959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc664-e01c-4c89-88ed-f557b7b95e34 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:20.458486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc664-e01c-4c89-88ed-f557b7b95e34 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:20.474559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:20.554209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4f99-1b4a-4216-be6c-ea4f5616f398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:20.554927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4f99-1b4a-4216-be6c-ea4f5616f398 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:20.555374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4f99-1b4a-4216-be6c-ea4f5616f398 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:20.555720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4f99-1b4a-4216-be6c-ea4f5616f398 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:20.556476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da4f99-1b4a-4216-be6c-ea4f5616f398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:20.558754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da4f99-1b4a-4216-be6c-ea4f5616f398 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:20.589453 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3669/7368] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:20 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:20.590983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412936d1-61e1-4ca2-a6e7-67c34e065756 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:20.602110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412936d1-61e1-4ca2-a6e7-67c34e065756 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:20.608686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d38f72-005a-4409-8449-3790d4968b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:20.609355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d38f72-005a-4409-8449-3790d4968b0a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:20.609632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d38f72-005a-4409-8449-3790d4968b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:20.609900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d38f72-005a-4409-8449-3790d4968b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:20.610536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d38f72-005a-4409-8449-3790d4968b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:20.612166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d38f72-005a-4409-8449-3790d4968b0a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:20.624780 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3700/7369] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:20.628999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4f99-1b4a-4216-be6c-ea4f5616f398 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:20.639845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4f99-1b4a-4216-be6c-ea4f5616f398 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:20.645049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76686c-418b-40c3-af8d-598bab1aabca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:20.645710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76686c-418b-40c3-af8d-598bab1aabca admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:20.646105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76686c-418b-40c3-af8d-598bab1aabca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:20.646380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76686c-418b-40c3-af8d-598bab1aabca admin admin] PATH_INFO: `/v3/projects/4098c64035de4b1c9bb9f8fbd941e316/users/2ad099db724b49abb5e913817130543c/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:20.662043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76686c-418b-40c3-af8d-598bab1aabca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4098c64035de4b1c9bb9f8fbd941e316, user_id=2ad099db724b49abb5e913817130543c, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:20.664778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76686c-418b-40c3-af8d-598bab1aabca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:20.685164 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3670/7370] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:20 2022] PUT /identity/v3/projects/4098c64035de4b1c9bb9f8fbd941e316/users/2ad099db724b49abb5e913817130543c/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:20.691755 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d38f72-005a-4409-8449-3790d4968b0a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:20.710444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d38f72-005a-4409-8449-3790d4968b0a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:20.749176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76686c-418b-40c3-af8d-598bab1aabca admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:20.751274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f76686c-418b-40c3-af8d-598bab1aabca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:20.785913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7426d6-46b8-4b4c-90f3-6f0e4674783e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:20.787454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7426d6-46b8-4b4c-90f3-6f0e4674783e admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:20.787670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7426d6-46b8-4b4c-90f3-6f0e4674783e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:20.787946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7426d6-46b8-4b4c-90f3-6f0e4674783e admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:20.788584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7426d6-46b8-4b4c-90f3-6f0e4674783e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:20.791111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7426d6-46b8-4b4c-90f3-6f0e4674783e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:20.808323 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3701/7371] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:20.809086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7426d6-46b8-4b4c-90f3-6f0e4674783e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:20.819899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7426d6-46b8-4b4c-90f3-6f0e4674783e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:20.826084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dea0b-5f20-4a02-9b1b-7d58e77f80f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:20.826753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dea0b-5f20-4a02-9b1b-7d58e77f80f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:20.827028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dea0b-5f20-4a02-9b1b-7d58e77f80f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:20.827314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dea0b-5f20-4a02-9b1b-7d58e77f80f0 admin admin] PATH_INFO: `/v3/projects/4098c64035de4b1c9bb9f8fbd941e316/users/2ad099db724b49abb5e913817130543c/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:20.829429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dea0b-5f20-4a02-9b1b-7d58e77f80f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4098c64035de4b1c9bb9f8fbd941e316, user_id=2ad099db724b49abb5e913817130543c, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:20.832104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dea0b-5f20-4a02-9b1b-7d58e77f80f0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:20.839819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76686c-418b-40c3-af8d-598bab1aabca admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:20.843728 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3702/7372] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:20 2022] PUT /identity/v3/projects/4098c64035de4b1c9bb9f8fbd941e316/users/2ad099db724b49abb5e913817130543c/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:20.854681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61de603-9b68-4c15-8b67-8a02e266da5d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:20.854924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61de603-9b68-4c15-8b67-8a02e266da5d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:20.855771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61de603-9b68-4c15-8b67-8a02e266da5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:20.891906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b61de603-9b68-4c15-8b67-8a02e266da5d None None] MFA Rules not processed for user `2ad099db724b49abb5e913817130543c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:20.893533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b61de603-9b68-4c15-8b67-8a02e266da5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:20.906388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:20.908107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:20.909809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:20.910075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:20.910336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] PATH_INFO: `/v3/projects/437c5b0d36db41d5ba7f5046682b6785` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:20.911588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=437c5b0d36db41d5ba7f5046682b6785)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:20.913697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:20.949945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:20.971354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437c5b0d36db41d5ba7f5046682b6785', 'request_id': 'req-b980797f-a313-4bef-ba3b-0221bf354c75'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:20.978560 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3703/7373] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:19:20.984374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61de603-9b68-4c15-8b67-8a02e266da5d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:20.987170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437c5b0d36db41d5ba7f5046682b6785', 'request_id': 'req-b980797f-a313-4bef-ba3b-0221bf354c75'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:20.994466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61de603-9b68-4c15-8b67-8a02e266da5d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:20.999562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437c5b0d36db41d5ba7f5046682b6785', 'request_id': 'req-b980797f-a313-4bef-ba3b-0221bf354c75'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:21.011946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] The token cache is being invalidate because project 437c5b0d36db41d5ba7f5046682b6785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:21.013017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:21.014590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b980797f-a313-4bef-ba3b-0221bf354c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:21.016384 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3671/7374] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:20 2022] DELETE /identity/v3/projects/437c5b0d36db41d5ba7f5046682b6785 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:19:21.021335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6dc02-f144-47ce-9cd4-e7ca8502fb9f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.021761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6dc02-f144-47ce-9cd4-e7ca8502fb9f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.022036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6dc02-f144-47ce-9cd4-e7ca8502fb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.047413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4af6dc02-f144-47ce-9cd4-e7ca8502fb9f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:21.047413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4af6dc02-f144-47ce-9cd4-e7ca8502fb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:21.086882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefc851-6118-4582-9ccb-2df6c03b8f9c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ad099db724b49abb5e913817130543c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4098c64035de4b1c9bb9f8fbd941e316', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.087513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefc851-6118-4582-9ccb-2df6c03b8f9c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.087793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefc851-6118-4582-9ccb-2df6c03b8f9c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.088065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefc851-6118-4582-9ccb-2df6c03b8f9c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.088718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefc851-6118-4582-9ccb-2df6c03b8f9c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:21.089020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefc851-6118-4582-9ccb-2df6c03b8f9c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.091370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefc851-6118-4582-9ccb-2df6c03b8f9c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.111201 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3704/7375] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:19:21.112383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefc851-6118-4582-9ccb-2df6c03b8f9c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:21.114386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0cefc851-6118-4582-9ccb-2df6c03b8f9c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:21.135363 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3672/7376] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:21.137517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72e435-9c41-4cc8-9441-d8e3c09ab864 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.137942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72e435-9c41-4cc8-9441-d8e3c09ab864 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.138397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72e435-9c41-4cc8-9441-d8e3c09ab864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.143871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefc851-6118-4582-9ccb-2df6c03b8f9c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:21.149905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430e24d-4de5-4c23-b27d-194d37bd1937 tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ad099db724b49abb5e913817130543c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4098c64035de4b1c9bb9f8fbd941e316', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.150185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430e24d-4de5-4c23-b27d-194d37bd1937 tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.150620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430e24d-4de5-4c23-b27d-194d37bd1937 tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.151035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430e24d-4de5-4c23-b27d-194d37bd1937 tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.151680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430e24d-4de5-4c23-b27d-194d37bd1937 tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.155201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430e24d-4de5-4c23-b27d-194d37bd1937 tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.167593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9e72e435-9c41-4cc8-9441-d8e3c09ab864 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:21.168971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9e72e435-9c41-4cc8-9441-d8e3c09ab864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:21.180886 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3673/7377] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:21.182120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72e435-9c41-4cc8-9441-d8e3c09ab864 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:21.193444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72e435-9c41-4cc8-9441-d8e3c09ab864 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:21.200592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da734e84-49d3-4276-8b7c-375b80d2bad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.201266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da734e84-49d3-4276-8b7c-375b80d2bad7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.201553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da734e84-49d3-4276-8b7c-375b80d2bad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.201853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da734e84-49d3-4276-8b7c-375b80d2bad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.202231 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3705/7378] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:21 2022] POST /identity/v3/projects => generated 349 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:21.206341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430e24d-4de5-4c23-b27d-194d37bd1937 tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:21.208514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da734e84-49d3-4276-8b7c-375b80d2bad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:21.208965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da734e84-49d3-4276-8b7c-375b80d2bad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.210798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da734e84-49d3-4276-8b7c-375b80d2bad7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.227564 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3674/7379] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:19:21.232018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da734e84-49d3-4276-8b7c-375b80d2bad7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:21.251549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da734e84-49d3-4276-8b7c-375b80d2bad7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:21.303537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430e24d-4de5-4c23-b27d-194d37bd1937 tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:21.310917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab85588-9236-4d3b-bc5b-cbd81b88019b tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ad099db724b49abb5e913817130543c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4098c64035de4b1c9bb9f8fbd941e316', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.311775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab85588-9236-4d3b-bc5b-cbd81b88019b tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.312229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab85588-9236-4d3b-bc5b-cbd81b88019b tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.313460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab85588-9236-4d3b-bc5b-cbd81b88019b tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.314273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab85588-9236-4d3b-bc5b-cbd81b88019b tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:21.314745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab85588-9236-4d3b-bc5b-cbd81b88019b tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.317326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab85588-9236-4d3b-bc5b-cbd81b88019b tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.332918 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3706/7380] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:19:21.339584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd95ee2-ed26-4150-b771-ef8d89569f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.340713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd95ee2-ed26-4150-b771-ef8d89569f2e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.341501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd95ee2-ed26-4150-b771-ef8d89569f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.341927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd95ee2-ed26-4150-b771-ef8d89569f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.342669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd95ee2-ed26-4150-b771-ef8d89569f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.344779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd95ee2-ed26-4150-b771-ef8d89569f2e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.353585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab85588-9236-4d3b-bc5b-cbd81b88019b tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:21.357016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8ab85588-9236-4d3b-bc5b-cbd81b88019b tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:21.387699 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3675/7381] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:21 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:21.390701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:21.392185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd95ee2-ed26-4150-b771-ef8d89569f2e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:21.435023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:21.461647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab85588-9236-4d3b-bc5b-cbd81b88019b tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:21.510791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd95ee2-ed26-4150-b771-ef8d89569f2e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:21.534208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d385a7d-b083-4ed0-a1a7-6c8effcb1125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.534889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d385a7d-b083-4ed0-a1a7-6c8effcb1125 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.535165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d385a7d-b083-4ed0-a1a7-6c8effcb1125 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.535429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d385a7d-b083-4ed0-a1a7-6c8effcb1125 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.536060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d385a7d-b083-4ed0-a1a7-6c8effcb1125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.538986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d385a7d-b083-4ed0-a1a7-6c8effcb1125 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.540913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980509db-dbe3-45f3-b905-de07803be2e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.542468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980509db-dbe3-45f3-b905-de07803be2e9 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.544744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980509db-dbe3-45f3-b905-de07803be2e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.545319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980509db-dbe3-45f3-b905-de07803be2e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.582756 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3676/7382] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:21 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:21.584090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d385a7d-b083-4ed0-a1a7-6c8effcb1125 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:21.601957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d385a7d-b083-4ed0-a1a7-6c8effcb1125 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:21.608332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353e7e0-6bd1-46b4-bd30-7ce93852817d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.609376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353e7e0-6bd1-46b4-bd30-7ce93852817d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.609733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353e7e0-6bd1-46b4-bd30-7ce93852817d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.610077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353e7e0-6bd1-46b4-bd30-7ce93852817d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.610799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353e7e0-6bd1-46b4-bd30-7ce93852817d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.613302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353e7e0-6bd1-46b4-bd30-7ce93852817d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.629087 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3677/7383] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:21.631840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353e7e0-6bd1-46b4-bd30-7ce93852817d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:21.644875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353e7e0-6bd1-46b4-bd30-7ce93852817d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:21.650104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016280eb-db0e-4cab-a672-c66443acaa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.650746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016280eb-db0e-4cab-a672-c66443acaa3c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.650995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016280eb-db0e-4cab-a672-c66443acaa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.651260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016280eb-db0e-4cab-a672-c66443acaa3c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.651901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016280eb-db0e-4cab-a672-c66443acaa3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.654264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016280eb-db0e-4cab-a672-c66443acaa3c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.661370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980509db-dbe3-45f3-b905-de07803be2e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.663544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980509db-dbe3-45f3-b905-de07803be2e9 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.669498 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3678/7384] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:21.670948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016280eb-db0e-4cab-a672-c66443acaa3c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:21.681988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016280eb-db0e-4cab-a672-c66443acaa3c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:21.682984 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3707/7385] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:19:21.688517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a934e97-7d40-499b-b2cb-a04ddd875bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.689317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a934e97-7d40-499b-b2cb-a04ddd875bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.689664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a934e97-7d40-499b-b2cb-a04ddd875bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.690171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a934e97-7d40-499b-b2cb-a04ddd875bc7 admin admin] PATH_INFO: `/v3/projects/001c1decf7ee4d3c9d5506648f5b9603/users/7439251410b640e0ad236abd28a2350c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.715806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a934e97-7d40-499b-b2cb-a04ddd875bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=001c1decf7ee4d3c9d5506648f5b9603, user_id=7439251410b640e0ad236abd28a2350c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.719496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a934e97-7d40-499b-b2cb-a04ddd875bc7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.734309 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3679/7386] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:21 2022] PUT /identity/v3/projects/001c1decf7ee4d3c9d5506648f5b9603/users/7439251410b640e0ad236abd28a2350c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:21.739636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2fa98-18dc-4fe6-a44e-4a00c38c8615 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.739925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2fa98-18dc-4fe6-a44e-4a00c38c8615 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.740197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2fa98-18dc-4fe6-a44e-4a00c38c8615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.763086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-49b2fa98-18dc-4fe6-a44e-4a00c38c8615 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:21.764283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-49b2fa98-18dc-4fe6-a44e-4a00c38c8615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:21.845945 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3708/7387] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:19:21.857021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a934e97-7d40-499b-b2cb-a04ddd875bc7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:21.859343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1a934e97-7d40-499b-b2cb-a04ddd875bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:21.945214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a934e97-7d40-499b-b2cb-a04ddd875bc7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:21.951613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320b456-b2ac-412a-af1f-c2c2c89945af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:21.952242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320b456-b2ac-412a-af1f-c2c2c89945af service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:21.952519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320b456-b2ac-412a-af1f-c2c2c89945af service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:21.952787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320b456-b2ac-412a-af1f-c2c2c89945af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:21.968023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320b456-b2ac-412a-af1f-c2c2c89945af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:21.970051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320b456-b2ac-412a-af1f-c2c2c89945af service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:21.983803 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3680/7388] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:23.750672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2fa98-18dc-4fe6-a44e-4a00c38c8615 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:23.800564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:23.847876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2fa98-18dc-4fe6-a44e-4a00c38c8615 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:23.853931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ad099db724b49abb5e913817130543c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4098c64035de4b1c9bb9f8fbd941e316', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:23.854845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:23.855299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:23.855725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] PATH_INFO: `/v3/projects/5e4d1bb3b677499088c19edd5e3f1580` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:23.858241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5e4d1bb3b677499088c19edd5e3f1580)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:23.860335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:23.920884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4d1bb3b677499088c19edd5e3f1580', 'request_id': 'req-59486c68-2ca9-46f9-ac1a-13704d78908c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:23.930561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4d1bb3b677499088c19edd5e3f1580', 'request_id': 'req-59486c68-2ca9-46f9-ac1a-13704d78908c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:23.938980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4d1bb3b677499088c19edd5e3f1580', 'request_id': 'req-59486c68-2ca9-46f9-ac1a-13704d78908c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:23.951464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] The token cache is being invalidate because project 5e4d1bb3b677499088c19edd5e3f1580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:23.951758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:23.952759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-59486c68-2ca9-46f9-ac1a-13704d78908c tempest-QuotasNegativeTest-87436178 tempest-QuotasNegativeTest-87436178-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:23.954380 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3709/7389] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:23 2022] DELETE /identity/v3/projects/5e4d1bb3b677499088c19edd5e3f1580 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:23.960224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320b456-b2ac-412a-af1f-c2c2c89945af service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:23.961731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b320b456-b2ac-412a-af1f-c2c2c89945af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:23.976180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320b456-b2ac-412a-af1f-c2c2c89945af service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:24.048799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:24.049548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:24.050002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:24.050453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] PATH_INFO: `/v3/users/7514421b2f5c4380a8a67d6fd66b508b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:24.051951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7514421b2f5c4380a8a67d6fd66b508b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:24.054224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:24.069339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:24.125993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7514421b2f5c4380a8a67d6fd66b508b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:24.142485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7514421b2f5c4380a8a67d6fd66b508b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:24.154117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7514421b2f5c4380a8a67d6fd66b508b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:24.165537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7514421b2f5c4380a8a67d6fd66b508b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:24.166410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7514421b2f5c4380a8a67d6fd66b508b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:24.174149 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3681/7390] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:23 2022] DELETE /identity/v3/users/7514421b2f5c4380a8a67d6fd66b508b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:19:24.178547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34203f7a-bda2-422e-b20e-8cca7d0fb3fb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:24.178835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34203f7a-bda2-422e-b20e-8cca7d0fb3fb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:24.179110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34203f7a-bda2-422e-b20e-8cca7d0fb3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:24.205243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-34203f7a-bda2-422e-b20e-8cca7d0fb3fb None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:24.206594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-34203f7a-bda2-422e-b20e-8cca7d0fb3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:24.283821 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3710/7391] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:24.298089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:24.303118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:24.386833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f42537-b1b8-4d51-85ad-945b6cc8fe60 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:24.391325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d91c-4869-4d03-b78a-df614a0b01a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:24.392024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d91c-4869-4d03-b78a-df614a0b01a4 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:24.392385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d91c-4869-4d03-b78a-df614a0b01a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:24.392695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d91c-4869-4d03-b78a-df614a0b01a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:24.404612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06d91c-4869-4d03-b78a-df614a0b01a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:24.406186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06d91c-4869-4d03-b78a-df614a0b01a4 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:24.419570 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3682/7392] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:24.544692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34203f7a-bda2-422e-b20e-8cca7d0fb3fb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:24.546166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-34203f7a-bda2-422e-b20e-8cca7d0fb3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:24.647847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:24.682268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34203f7a-bda2-422e-b20e-8cca7d0fb3fb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:24.704718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb972a6-57f1-4de5-a522-32aff8ad468a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:24.705727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb972a6-57f1-4de5-a522-32aff8ad468a service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:24.706802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb972a6-57f1-4de5-a522-32aff8ad468a service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:24.707403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb972a6-57f1-4de5-a522-32aff8ad468a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:24.715600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2bb972a6-57f1-4de5-a522-32aff8ad468a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:24.759097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb972a6-57f1-4de5-a522-32aff8ad468a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:24.760914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb972a6-57f1-4de5-a522-32aff8ad468a service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:24.786504 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3711/7393] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:25.195606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d91c-4869-4d03-b78a-df614a0b01a4 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:25.198402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6e06d91c-4869-4d03-b78a-df614a0b01a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:25.226960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d91c-4869-4d03-b78a-df614a0b01a4 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:25.245660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:25.246393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:25.246702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:25.246985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] PATH_INFO: `/v3/projects/06ff4e9366b94f8f916f458eae61e5d6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:25.248021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] RBAC: Authorizing `identity:delete_project(project_id=06ff4e9366b94f8f916f458eae61e5d6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:25.250047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:25.312947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06ff4e9366b94f8f916f458eae61e5d6', 'request_id': 'req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:25.326020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06ff4e9366b94f8f916f458eae61e5d6', 'request_id': 'req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:25.354650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06ff4e9366b94f8f916f458eae61e5d6', 'request_id': 'req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:25.367821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] The token cache is being invalidate because project 06ff4e9366b94f8f916f458eae61e5d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:25.368108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:25.369358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:25.371119 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3683/7394] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:25 2022] DELETE /identity/v3/projects/06ff4e9366b94f8f916f458eae61e5d6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:25.375221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb972a6-57f1-4de5-a522-32aff8ad468a service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:25.391568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb972a6-57f1-4de5-a522-32aff8ad468a service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:25.472678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:25.473454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:25.473810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:25.474137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] PATH_INFO: `/v3/users/2ad099db724b49abb5e913817130543c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:25.475545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad099db724b49abb5e913817130543c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:25.477618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:25.541415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad099db724b49abb5e913817130543c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:25.551195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad099db724b49abb5e913817130543c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:25.582559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad099db724b49abb5e913817130543c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:25.594314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad099db724b49abb5e913817130543c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:25.595597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29321499-184a-4e58-96ef-8ae2a64359c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad099db724b49abb5e913817130543c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:25.616930 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3712/7395] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:25 2022] DELETE /identity/v3/users/2ad099db724b49abb5e913817130543c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:26.417136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:26.419702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:26.452361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ec27e-29f1-4eb3-8fe1-da5701b4606d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:26.472325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:26.537661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:26.538938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:26.538938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:26.539175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] PATH_INFO: `/v3/projects/4098c64035de4b1c9bb9f8fbd941e316` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:26.540440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=4098c64035de4b1c9bb9f8fbd941e316)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:26.543732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:26.642788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4098c64035de4b1c9bb9f8fbd941e316', 'request_id': 'req-232341f0-b5d8-436a-afd4-789c41ef21ea'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:26.656897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4098c64035de4b1c9bb9f8fbd941e316', 'request_id': 'req-232341f0-b5d8-436a-afd4-789c41ef21ea'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:26.672281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4098c64035de4b1c9bb9f8fbd941e316', 'request_id': 'req-232341f0-b5d8-436a-afd4-789c41ef21ea'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:26.729307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] The token cache is being invalidate because project 4098c64035de4b1c9bb9f8fbd941e316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:26.738126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:26.741949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-232341f0-b5d8-436a-afd4-789c41ef21ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:26.748255 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3684/7396] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:26 2022] DELETE /identity/v3/projects/4098c64035de4b1c9bb9f8fbd941e316 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:26.749483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698971b-cd6f-41ae-a63b-e7e4b6817f10 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:26.750317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698971b-cd6f-41ae-a63b-e7e4b6817f10 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:26.750598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698971b-cd6f-41ae-a63b-e7e4b6817f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:26.805718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a698971b-cd6f-41ae-a63b-e7e4b6817f10 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:26.810653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a698971b-cd6f-41ae-a63b-e7e4b6817f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:26.947640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:26.977062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14140e-c42e-4d1a-acf8-78df6e2f3381 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:26.977786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14140e-c42e-4d1a-acf8-78df6e2f3381 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:26.978260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14140e-c42e-4d1a-acf8-78df6e2f3381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:26.984311 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3713/7397] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:27.003077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ef14140e-c42e-4d1a-acf8-78df6e2f3381 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:27.004848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ef14140e-c42e-4d1a-acf8-78df6e2f3381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:27.031126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698971b-cd6f-41ae-a63b-e7e4b6817f10 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:27.032462 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3685/7398] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:27.048173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698971b-cd6f-41ae-a63b-e7e4b6817f10 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:27.059403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74359e5a-c7e9-4149-beb1-038f2b7dae09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:27.060147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74359e5a-c7e9-4149-beb1-038f2b7dae09 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:27.060426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74359e5a-c7e9-4149-beb1-038f2b7dae09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:27.060692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74359e5a-c7e9-4149-beb1-038f2b7dae09 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:27.065241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74359e5a-c7e9-4149-beb1-038f2b7dae09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:27.065698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74359e5a-c7e9-4149-beb1-038f2b7dae09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:27.067343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74359e5a-c7e9-4149-beb1-038f2b7dae09 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:27.101739 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3714/7399] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:19:27.105070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14140e-c42e-4d1a-acf8-78df6e2f3381 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:27.119764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14140e-c42e-4d1a-acf8-78df6e2f3381 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:27.140520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bd23e-3e76-42f7-a546-129f061e1c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:27.142120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bd23e-3e76-42f7-a546-129f061e1c87 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:27.142120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bd23e-3e76-42f7-a546-129f061e1c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:27.145745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bd23e-3e76-42f7-a546-129f061e1c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:27.147611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bd23e-3e76-42f7-a546-129f061e1c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:27.151432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bd23e-3e76-42f7-a546-129f061e1c87 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:27.223531 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3686/7400] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:27 2022] POST /identity/v3/projects => generated 405 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:27.227493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74359e5a-c7e9-4149-beb1-038f2b7dae09 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:27.241412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74359e5a-c7e9-4149-beb1-038f2b7dae09 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:27.338594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e8d33-bbc9-46d8-b1eb-5865251387b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:27.340892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e8d33-bbc9-46d8-b1eb-5865251387b4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:27.341303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e8d33-bbc9-46d8-b1eb-5865251387b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:27.341649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e8d33-bbc9-46d8-b1eb-5865251387b4 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:27.342302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e8d33-bbc9-46d8-b1eb-5865251387b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:27.344565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e8d33-bbc9-46d8-b1eb-5865251387b4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:27.382854 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3715/7401] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:27 2022] POST /identity/v3/users => generated 303 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:27.386800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bd23e-3e76-42f7-a546-129f061e1c87 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:27.404912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bd23e-3e76-42f7-a546-129f061e1c87 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:27.410263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270bc66-7275-49bd-8d2c-17a33603d734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:27.410878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270bc66-7275-49bd-8d2c-17a33603d734 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:27.411153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270bc66-7275-49bd-8d2c-17a33603d734 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:27.411601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270bc66-7275-49bd-8d2c-17a33603d734 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:27.412253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270bc66-7275-49bd-8d2c-17a33603d734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:27.414087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270bc66-7275-49bd-8d2c-17a33603d734 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:27.429210 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3687/7402] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:19:27.433555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e8d33-bbc9-46d8-b1eb-5865251387b4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:27.451902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e8d33-bbc9-46d8-b1eb-5865251387b4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:27.459853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd62a7-7b31-4521-97d9-3ccf96d1864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:27.461348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd62a7-7b31-4521-97d9-3ccf96d1864a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:27.461881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd62a7-7b31-4521-97d9-3ccf96d1864a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:27.462846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd62a7-7b31-4521-97d9-3ccf96d1864a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:27.465804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd62a7-7b31-4521-97d9-3ccf96d1864a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:27.470670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd62a7-7b31-4521-97d9-3ccf96d1864a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:27.498739 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3716/7403] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:27.498878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270bc66-7275-49bd-8d2c-17a33603d734 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:27.514861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270bc66-7275-49bd-8d2c-17a33603d734 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:27.519828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488f272-da31-40a2-aeda-1e87b671036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:27.521724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488f272-da31-40a2-aeda-1e87b671036f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:27.522837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488f272-da31-40a2-aeda-1e87b671036f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:27.523116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488f272-da31-40a2-aeda-1e87b671036f admin admin] PATH_INFO: `/v3/projects/ec1c0bd4ca2d4e9285e66364840a1f5c/users/d2d90b0eeccf4b8498b7d863b6f60479/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:27.544654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488f272-da31-40a2-aeda-1e87b671036f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1c0bd4ca2d4e9285e66364840a1f5c, user_id=d2d90b0eeccf4b8498b7d863b6f60479, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:27.547448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488f272-da31-40a2-aeda-1e87b671036f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:27.581281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e361e-9092-41ef-ae51-72ea0ec91cf0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:27.581479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e361e-9092-41ef-ae51-72ea0ec91cf0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:27.581712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e361e-9092-41ef-ae51-72ea0ec91cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:27.590714 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3688/7404] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:27 2022] PUT /identity/v3/projects/ec1c0bd4ca2d4e9285e66364840a1f5c/users/d2d90b0eeccf4b8498b7d863b6f60479/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:19:27.657956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-415e361e-9092-41ef-ae51-72ea0ec91cf0 None None] MFA Rules not processed for user `d2d90b0eeccf4b8498b7d863b6f60479`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:27.657956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-415e361e-9092-41ef-ae51-72ea0ec91cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:27.860964 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3717/7405] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:27.866242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488f272-da31-40a2-aeda-1e87b671036f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:27.993697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:28.074340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488f272-da31-40a2-aeda-1e87b671036f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:28.096880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2346c3-fed9-40fe-a9d3-76cf16e8efee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:28.097940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2346c3-fed9-40fe-a9d3-76cf16e8efee admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:28.098434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2346c3-fed9-40fe-a9d3-76cf16e8efee admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:28.098948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2346c3-fed9-40fe-a9d3-76cf16e8efee admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:28.100153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2346c3-fed9-40fe-a9d3-76cf16e8efee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:28.102422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2346c3-fed9-40fe-a9d3-76cf16e8efee admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:28.165943 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3689/7406] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:27 2022] POST /identity/v3/projects => generated 407 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:28.170469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e361e-9092-41ef-ae51-72ea0ec91cf0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:28.200809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e361e-9092-41ef-ae51-72ea0ec91cf0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:28.320575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:28.360488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc791008-c647-451f-8307-94bee125ffb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:28.362106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc791008-c647-451f-8307-94bee125ffb8 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:28.362633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc791008-c647-451f-8307-94bee125ffb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:28.363122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc791008-c647-451f-8307-94bee125ffb8 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:28.364579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc791008-c647-451f-8307-94bee125ffb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:28.368825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc791008-c647-451f-8307-94bee125ffb8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:28.410931 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3718/7407] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:28 2022] POST /identity/v3/users => generated 310 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:19:28.414456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2346c3-fed9-40fe-a9d3-76cf16e8efee admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:28.432127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2346c3-fed9-40fe-a9d3-76cf16e8efee admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:28.439173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c61db-f07c-46a3-81b9-40441ab4117c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:28.439896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c61db-f07c-46a3-81b9-40441ab4117c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:28.440248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c61db-f07c-46a3-81b9-40441ab4117c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:28.440629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c61db-f07c-46a3-81b9-40441ab4117c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:28.441650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c61db-f07c-46a3-81b9-40441ab4117c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:28.445325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c61db-f07c-46a3-81b9-40441ab4117c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:28.473191 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3690/7408] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:28.476971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc791008-c647-451f-8307-94bee125ffb8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:28.500512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc791008-c647-451f-8307-94bee125ffb8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:28.509400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674817d3-899d-4182-99fa-93637013711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:28.510170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674817d3-899d-4182-99fa-93637013711d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:28.510515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674817d3-899d-4182-99fa-93637013711d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:28.510859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674817d3-899d-4182-99fa-93637013711d admin admin] PATH_INFO: `/v3/projects/e40ca2ce05a34f8a8a05d9876079faf3/users/18c9f03699b54ce4a901dbf2346878e6/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:28.542463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674817d3-899d-4182-99fa-93637013711d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40ca2ce05a34f8a8a05d9876079faf3, user_id=18c9f03699b54ce4a901dbf2346878e6, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:28.545972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674817d3-899d-4182-99fa-93637013711d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:28.567263 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3719/7409] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:28 2022] PUT /identity/v3/projects/e40ca2ce05a34f8a8a05d9876079faf3/users/18c9f03699b54ce4a901dbf2346878e6/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:28.571323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c61db-f07c-46a3-81b9-40441ab4117c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:28.582905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c61db-f07c-46a3-81b9-40441ab4117c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:28.657966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714de27-776a-46c2-b58b-8ab3ed74c171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:28.659031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714de27-776a-46c2-b58b-8ab3ed74c171 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:28.659031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714de27-776a-46c2-b58b-8ab3ed74c171 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:28.659235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714de27-776a-46c2-b58b-8ab3ed74c171 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:28.659782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714de27-776a-46c2-b58b-8ab3ed74c171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:28.661487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714de27-776a-46c2-b58b-8ab3ed74c171 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:28.679333 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3691/7410] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:28.683036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674817d3-899d-4182-99fa-93637013711d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:28.729687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674817d3-899d-4182-99fa-93637013711d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:28.739446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fac5c-3506-4127-adbb-a8db263bb225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:28.740403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fac5c-3506-4127-adbb-a8db263bb225 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:28.741652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fac5c-3506-4127-adbb-a8db263bb225 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:28.741925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fac5c-3506-4127-adbb-a8db263bb225 admin admin] PATH_INFO: `/v3/projects/e40ca2ce05a34f8a8a05d9876079faf3/users/18c9f03699b54ce4a901dbf2346878e6/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:28.744955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5fac5c-3506-4127-adbb-a8db263bb225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40ca2ce05a34f8a8a05d9876079faf3, user_id=18c9f03699b54ce4a901dbf2346878e6, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:28.749397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5fac5c-3506-4127-adbb-a8db263bb225 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:28.762385 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3720/7411] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:28 2022] PUT /identity/v3/projects/e40ca2ce05a34f8a8a05d9876079faf3/users/18c9f03699b54ce4a901dbf2346878e6/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:28.774015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d2750-438e-4359-8bfe-8d7ad6312c4d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:28.774548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d2750-438e-4359-8bfe-8d7ad6312c4d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:28.774931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d2750-438e-4359-8bfe-8d7ad6312c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:28.851792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3e4d2750-438e-4359-8bfe-8d7ad6312c4d None None] MFA Rules not processed for user `18c9f03699b54ce4a901dbf2346878e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:28.854339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3e4d2750-438e-4359-8bfe-8d7ad6312c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:28.994243 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3692/7412] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:28 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:29.006501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fac5c-3506-4127-adbb-a8db263bb225 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:29.008611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2c5fac5c-3506-4127-adbb-a8db263bb225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:29.102794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fac5c-3506-4127-adbb-a8db263bb225 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:29.107814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdca25c-8d3a-4cad-b23b-f9da897c5a6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:29.108930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdca25c-8d3a-4cad-b23b-f9da897c5a6b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:29.109342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdca25c-8d3a-4cad-b23b-f9da897c5a6b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:29.109701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdca25c-8d3a-4cad-b23b-f9da897c5a6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:29.206490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdca25c-8d3a-4cad-b23b-f9da897c5a6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:29.208697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdca25c-8d3a-4cad-b23b-f9da897c5a6b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:29.225049 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3721/7413] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:29.820511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5bc58-6e93-4803-bc75-3422d737d354 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:29.820981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5bc58-6e93-4803-bc75-3422d737d354 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:29.821093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5bc58-6e93-4803-bc75-3422d737d354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:29.858409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-53b5bc58-6e93-4803-bc75-3422d737d354 None None] MFA Rules not processed for user `7439251410b640e0ad236abd28a2350c`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:29.860165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-53b5bc58-6e93-4803-bc75-3422d737d354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:29.892596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:29.961864 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3693/7414] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:29.964161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdca25c-8d3a-4cad-b23b-f9da897c5a6b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:29.967984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ccdca25c-8d3a-4cad-b23b-f9da897c5a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:30.052562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdca25c-8d3a-4cad-b23b-f9da897c5a6b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:30.057192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76edbd-069a-41b5-a763-e9e0f500f9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:30.057794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76edbd-069a-41b5-a763-e9e0f500f9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:30.058069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76edbd-069a-41b5-a763-e9e0f500f9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:30.058357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76edbd-069a-41b5-a763-e9e0f500f9e4 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:30.059656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e76edbd-069a-41b5-a763-e9e0f500f9e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:30.061606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e76edbd-069a-41b5-a763-e9e0f500f9e4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:30.071079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:30.090147 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3722/7415] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:29 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:30.091526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5bc58-6e93-4803-bc75-3422d737d354 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:30.104042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5bc58-6e93-4803-bc75-3422d737d354 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:30.109334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89d6ed-a537-45c7-a8bd-19bef191211d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:30.109950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89d6ed-a537-45c7-a8bd-19bef191211d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:30.110256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89d6ed-a537-45c7-a8bd-19bef191211d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:30.110530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89d6ed-a537-45c7-a8bd-19bef191211d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:30.111204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89d6ed-a537-45c7-a8bd-19bef191211d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:30.112810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89d6ed-a537-45c7-a8bd-19bef191211d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:30.123846 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3694/7416] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:30.128334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76edbd-069a-41b5-a763-e9e0f500f9e4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:30.140697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76edbd-069a-41b5-a763-e9e0f500f9e4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:30.145845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cef8f7-8d3f-43ce-98bb-f78cda59134d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:30.146518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cef8f7-8d3f-43ce-98bb-f78cda59134d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:30.146820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cef8f7-8d3f-43ce-98bb-f78cda59134d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:30.147207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cef8f7-8d3f-43ce-98bb-f78cda59134d admin admin] PATH_INFO: `/v3/system/users/07d43d4616d644fea806df2f44628922/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:30.164923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cef8f7-8d3f-43ce-98bb-f78cda59134d admin admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=07d43d4616d644fea806df2f44628922, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:30.166609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cef8f7-8d3f-43ce-98bb-f78cda59134d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:30.176151 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3723/7417] 10.222.0.16 () {60 vars in 1378 bytes} [Sat Oct 15 21:19:30 2022] PUT /identity/v3/system/users/07d43d4616d644fea806df2f44628922/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:30.186743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad3a51-43bc-4732-b4ac-55327a968fc7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:30.187022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad3a51-43bc-4732-b4ac-55327a968fc7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:30.187288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad3a51-43bc-4732-b4ac-55327a968fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:30.194233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cef8f7-8d3f-43ce-98bb-f78cda59134d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:30.196011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-04cef8f7-8d3f-43ce-98bb-f78cda59134d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:30.229063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-31ad3a51-43bc-4732-b4ac-55327a968fc7 None None] MFA Rules not processed for user `07d43d4616d644fea806df2f44628922`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:30.231121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-31ad3a51-43bc-4732-b4ac-55327a968fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:30.328887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cef8f7-8d3f-43ce-98bb-f78cda59134d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:30.336687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8841d-c1b4-450c-8265-49b5de4522a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:30.337942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8841d-c1b4-450c-8265-49b5de4522a9 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:30.338345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8841d-c1b4-450c-8265-49b5de4522a9 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:30.338730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8841d-c1b4-450c-8265-49b5de4522a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:30.353732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8841d-c1b4-450c-8265-49b5de4522a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:30.356656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8841d-c1b4-450c-8265-49b5de4522a9 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:30.389965 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3724/7418] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:30 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:30.413144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8841d-c1b4-450c-8265-49b5de4522a9 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:30.413319 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3695/7419] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:30 2022] POST /identity/v3/auth/tokens => generated 2415 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 21:19:30.627050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8841d-c1b4-450c-8265-49b5de4522a9 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:30.631070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:30.688824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c6469-a080-4bb1-9019-3261dc3ea3f1 None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07d43d4616d644fea806df2f44628922', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:30.689852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c6469-a080-4bb1-9019-3261dc3ea3f1 None tempest-ImageQuotaTest-1489254523-system-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:30.690218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c6469-a080-4bb1-9019-3261dc3ea3f1 None tempest-ImageQuotaTest-1489254523-system-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:30.690412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c6469-a080-4bb1-9019-3261dc3ea3f1 None tempest-ImageQuotaTest-1489254523-system-admin] PATH_INFO: `/v3/services` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:30.690981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c6469-a080-4bb1-9019-3261dc3ea3f1 None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:30.692342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c6469-a080-4bb1-9019-3261dc3ea3f1 None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:30.707707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad3a51-43bc-4732-b4ac-55327a968fc7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:30.717703 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3725/7420] 10.222.0.16 () {58 vars in 1110 bytes} [Sat Oct 15 21:19:30 2022] GET /identity/v3/services => generated 2191 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:19:30.818661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:30.849124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad3a51-43bc-4732-b4ac-55327a968fc7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:30.907814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407c4df-83da-4df7-9fbd-7063d4d679c7 None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07d43d4616d644fea806df2f44628922', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:30.908538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407c4df-83da-4df7-9fbd-7063d4d679c7 None tempest-ImageQuotaTest-1489254523-system-admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:30.908909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407c4df-83da-4df7-9fbd-7063d4d679c7 None tempest-ImageQuotaTest-1489254523-system-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:30.909328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407c4df-83da-4df7-9fbd-7063d4d679c7 None tempest-ImageQuotaTest-1489254523-system-admin] PATH_INFO: `/v3/limits` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:30.910255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4407c4df-83da-4df7-9fbd-7063d4d679c7 None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: Authorizing `identity:create_limits()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:30.913297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4407c4df-83da-4df7-9fbd-7063d4d679c7 None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application [None req-4407c4df-83da-4df7-9fbd-7063d4d679c7 None tempest-ImageQuotaTest-1489254523-system-admin] Unable to create a limit that has no corresponding registered limit.: keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/limits.py", line 110, in post Oct 15 21:19:30.981746 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application refs = PROVIDERS.unified_limit_api.create_limits(limits) Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/core.py", line 116, in create_limits Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return self.driver.create_limits(limits) Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 260, in create_limits Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application target = self._check_and_fill_registered_limit_id(limit) Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 249, in _check_and_fill_registered_limit_id Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application raise exception.NoLimitReference Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Oct 15 21:19:30.983230 ncdv-master-1 devstack@keystone.service[99100]: ERROR keystone.server.flask.application Oct 15 21:19:30.995375 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3696/7421] 10.222.0.16 () {60 vars in 1126 bytes} [Sat Oct 15 21:19:30 2022] POST /identity/v3/limits => generated 124 bytes in 288 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Oct 15 21:19:31.001190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c6469-a080-4bb1-9019-3261dc3ea3f1 None tempest-ImageQuotaTest-1489254523-system-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:31.193939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c6469-a080-4bb1-9019-3261dc3ea3f1 None tempest-ImageQuotaTest-1489254523-system-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:31.258398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2565028-8e3f-4a80-b580-5da0d2b916fa None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07d43d4616d644fea806df2f44628922', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:31.259919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565028-8e3f-4a80-b580-5da0d2b916fa None tempest-ImageQuotaTest-1489254523-system-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:31.260444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565028-8e3f-4a80-b580-5da0d2b916fa None tempest-ImageQuotaTest-1489254523-system-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:31.260872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565028-8e3f-4a80-b580-5da0d2b916fa None tempest-ImageQuotaTest-1489254523-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:31.263083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2565028-8e3f-4a80-b580-5da0d2b916fa None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:31.265277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2565028-8e3f-4a80-b580-5da0d2b916fa None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:31.276669 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3726/7422] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 21:19:30 2022] GET /identity/v3/registered_limits => generated 129 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:19:31.285177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407c4df-83da-4df7-9fbd-7063d4d679c7 None tempest-ImageQuotaTest-1489254523-system-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:31.463950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407c4df-83da-4df7-9fbd-7063d4d679c7 None tempest-ImageQuotaTest-1489254523-system-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:31.540303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941a465-62d1-41c7-ab9f-4da3e8ae2d14 None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07d43d4616d644fea806df2f44628922', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:31.541117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941a465-62d1-41c7-ab9f-4da3e8ae2d14 None tempest-ImageQuotaTest-1489254523-system-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:31.541491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941a465-62d1-41c7-ab9f-4da3e8ae2d14 None tempest-ImageQuotaTest-1489254523-system-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:31.541855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941a465-62d1-41c7-ab9f-4da3e8ae2d14 None tempest-ImageQuotaTest-1489254523-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:31.542593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941a465-62d1-41c7-ab9f-4da3e8ae2d14 None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:31.544803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941a465-62d1-41c7-ab9f-4da3e8ae2d14 None tempest-ImageQuotaTest-1489254523-system-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:31.547114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:31.561089 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3697/7423] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 21:19:31 2022] GET /identity/v3/registered_limits => generated 129 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:19:39.958766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2565028-8e3f-4a80-b580-5da0d2b916fa None tempest-ImageQuotaTest-1489254523-system-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:39.975352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2565028-8e3f-4a80-b580-5da0d2b916fa None tempest-ImageQuotaTest-1489254523-system-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:39.983088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:39.983910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:39.984352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:39.984779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] PATH_INFO: `/v3/users/7439251410b640e0ad236abd28a2350c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:39.986113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7439251410b640e0ad236abd28a2350c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:39.988330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:40.066719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:40.101550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941a465-62d1-41c7-ab9f-4da3e8ae2d14 None tempest-ImageQuotaTest-1489254523-system-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:40.105626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7439251410b640e0ad236abd28a2350c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.113601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7439251410b640e0ad236abd28a2350c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.130211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941a465-62d1-41c7-ab9f-4da3e8ae2d14 None tempest-ImageQuotaTest-1489254523-system-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:40.141071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:40.141828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7439251410b640e0ad236abd28a2350c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.142250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:40.142745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:40.143249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] PATH_INFO: `/v3/users/d2d90b0eeccf4b8498b7d863b6f60479` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:40.144959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d90b0eeccf4b8498b7d863b6f60479)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:40.147518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:40.157721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7439251410b640e0ad236abd28a2350c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.158549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-88fc16a8-bdef-4d70-9b4e-9b4dd22aad81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7439251410b640e0ad236abd28a2350c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.165611 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3727/7424] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:39 2022] DELETE /identity/v3/users/7439251410b640e0ad236abd28a2350c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:40.227887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d90b0eeccf4b8498b7d863b6f60479'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.243844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d90b0eeccf4b8498b7d863b6f60479'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.257483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d90b0eeccf4b8498b7d863b6f60479'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.277159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d90b0eeccf4b8498b7d863b6f60479'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.278860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d90b0eeccf4b8498b7d863b6f60479'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.288429 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3698/7425] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:40 2022] DELETE /identity/v3/users/d2d90b0eeccf4b8498b7d863b6f60479 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:19:40.294127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a61c8f-fef3-4f0a-98e5-8cbf43669ca8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:40.294621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a61c8f-fef3-4f0a-98e5-8cbf43669ca8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:40.295482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a61c8f-fef3-4f0a-98e5-8cbf43669ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:40.318021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-54a61c8f-fef3-4f0a-98e5-8cbf43669ca8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:40.319281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-54a61c8f-fef3-4f0a-98e5-8cbf43669ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:40.407819 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3728/7426] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:40.423026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:40.427529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:40.431443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a61c8f-fef3-4f0a-98e5-8cbf43669ca8 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:40.433314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-54a61c8f-fef3-4f0a-98e5-8cbf43669ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:40.447725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a61c8f-fef3-4f0a-98e5-8cbf43669ca8 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:40.453301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:40.454141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:40.454883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:40.454883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] PATH_INFO: `/v3/projects/001c1decf7ee4d3c9d5506648f5b9603` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:40.455868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=001c1decf7ee4d3c9d5506648f5b9603)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:40.459058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:40.525502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001c1decf7ee4d3c9d5506648f5b9603', 'request_id': 'req-767dabf5-e9d7-4a51-a840-b4a29988ace5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.528361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c19ebf-c3ca-4296-be3b-63a912223d6c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:40.539017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001c1decf7ee4d3c9d5506648f5b9603', 'request_id': 'req-767dabf5-e9d7-4a51-a840-b4a29988ace5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.548295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001c1decf7ee4d3c9d5506648f5b9603', 'request_id': 'req-767dabf5-e9d7-4a51-a840-b4a29988ace5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.567842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] The token cache is being invalidate because project 001c1decf7ee4d3c9d5506648f5b9603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:40.568291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.569471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.573926 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3729/7427] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:40 2022] DELETE /identity/v3/projects/001c1decf7ee4d3c9d5506648f5b9603 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:40.576351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:40.578371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:40.617635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211d581-6518-40e2-b41e-159227fd3317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:40.618612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211d581-6518-40e2-b41e-159227fd3317 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:40.618889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211d581-6518-40e2-b41e-159227fd3317 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:40.619589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211d581-6518-40e2-b41e-159227fd3317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:40.622653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c211d581-6518-40e2-b41e-159227fd3317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:40.635061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:40.685132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dabf5-e9d7-4a51-a840-b4a29988ace5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:40.687242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:40.694060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:40.694986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:40.695461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:40.695742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] PATH_INFO: `/v3/users/07d43d4616d644fea806df2f44628922` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:40.698891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d43d4616d644fea806df2f44628922)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:40.701343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:40.718240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211d581-6518-40e2-b41e-159227fd3317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:40.728803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211d581-6518-40e2-b41e-159227fd3317 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:40.763060 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3699/7428] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:40.795553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d43d4616d644fea806df2f44628922'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.807566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d43d4616d644fea806df2f44628922'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.826005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d43d4616d644fea806df2f44628922'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.836212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d43d4616d644fea806df2f44628922'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.837520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d43d4616d644fea806df2f44628922'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:40.845727 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3730/7429] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:40 2022] DELETE /identity/v3/users/07d43d4616d644fea806df2f44628922 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:41.373509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211d581-6518-40e2-b41e-159227fd3317 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:41.376311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c211d581-6518-40e2-b41e-159227fd3317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:41.391945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211d581-6518-40e2-b41e-159227fd3317 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:41.492298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:41.493101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:41.493441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:41.493880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] PATH_INFO: `/v3/projects/ec1c0bd4ca2d4e9285e66364840a1f5c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:41.495353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec1c0bd4ca2d4e9285e66364840a1f5c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:41.500057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:41.577745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec1c0bd4ca2d4e9285e66364840a1f5c', 'request_id': 'req-d520da85-cf67-4095-abfa-ba7270f7442d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:41.591539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec1c0bd4ca2d4e9285e66364840a1f5c', 'request_id': 'req-d520da85-cf67-4095-abfa-ba7270f7442d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:41.600397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec1c0bd4ca2d4e9285e66364840a1f5c', 'request_id': 'req-d520da85-cf67-4095-abfa-ba7270f7442d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:41.614909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] The token cache is being invalidate because project ec1c0bd4ca2d4e9285e66364840a1f5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:41.615186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:41.615856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:41.617789 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3700/7430] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:41 2022] DELETE /identity/v3/projects/ec1c0bd4ca2d4e9285e66364840a1f5c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:41.622971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:41.624441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:41.639684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550822f8-9680-463b-8263-1fa0a74ab968 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:41.709703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:41.710337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:41.710610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:41.710928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] PATH_INFO: `/v3/users/18c9f03699b54ce4a901dbf2346878e6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:41.712079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18c9f03699b54ce4a901dbf2346878e6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:41.714049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:41.771072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:41.779022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18c9f03699b54ce4a901dbf2346878e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:41.789575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18c9f03699b54ce4a901dbf2346878e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:41.804531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18c9f03699b54ce4a901dbf2346878e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:41.813516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18c9f03699b54ce4a901dbf2346878e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:41.814281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a3e5b21b-471f-45f7-9ff8-a1a367fd31d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18c9f03699b54ce4a901dbf2346878e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:41.821073 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3731/7431] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:41 2022] DELETE /identity/v3/users/18c9f03699b54ce4a901dbf2346878e6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:42.027645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:42.029183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:42.049263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520da85-cf67-4095-abfa-ba7270f7442d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:42.123996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:42.124557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:42.124836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:42.125138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] PATH_INFO: `/v3/projects/e40ca2ce05a34f8a8a05d9876079faf3` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:42.126304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e40ca2ce05a34f8a8a05d9876079faf3)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:42.128246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:42.137698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:42.190932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e40ca2ce05a34f8a8a05d9876079faf3', 'request_id': 'req-e5834a84-3822-44b4-bc19-ed11f7fa7412'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:42.202369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e40ca2ce05a34f8a8a05d9876079faf3', 'request_id': 'req-e5834a84-3822-44b4-bc19-ed11f7fa7412'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:42.216045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e40ca2ce05a34f8a8a05d9876079faf3', 'request_id': 'req-e5834a84-3822-44b4-bc19-ed11f7fa7412'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:42.240068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] The token cache is being invalidate because project e40ca2ce05a34f8a8a05d9876079faf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:42.240366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:42.241326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e5834a84-3822-44b4-bc19-ed11f7fa7412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:42.243383 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3701/7432] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:42 2022] DELETE /identity/v3/projects/e40ca2ce05a34f8a8a05d9876079faf3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:42.248759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e331162-1ca5-40af-868c-5339ab095206 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:42.249149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e331162-1ca5-40af-868c-5339ab095206 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:42.249517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e331162-1ca5-40af-868c-5339ab095206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:42.272395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9e331162-1ca5-40af-868c-5339ab095206 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:42.274275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e331162-1ca5-40af-868c-5339ab095206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:42.370041 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3732/7433] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:42.378407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127a9ec-0091-4ebc-ba32-457a3876582a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:42.378844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127a9ec-0091-4ebc-ba32-457a3876582a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:42.378927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127a9ec-0091-4ebc-ba32-457a3876582a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:42.409156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5127a9ec-0091-4ebc-ba32-457a3876582a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:42.410449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5127a9ec-0091-4ebc-ba32-457a3876582a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:42.427721 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3702/7434] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:19:42.430773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e331162-1ca5-40af-868c-5339ab095206 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:42.448511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e331162-1ca5-40af-868c-5339ab095206 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:42.453927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb24820-31bc-4f63-b4ff-5948ac867cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:42.454531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb24820-31bc-4f63-b4ff-5948ac867cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:42.454792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb24820-31bc-4f63-b4ff-5948ac867cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:42.455055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb24820-31bc-4f63-b4ff-5948ac867cb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:42.455691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb24820-31bc-4f63-b4ff-5948ac867cb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:42.455983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb24820-31bc-4f63-b4ff-5948ac867cb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:42.457792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb24820-31bc-4f63-b4ff-5948ac867cb1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:42.473104 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3733/7435] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:19:42.476473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127a9ec-0091-4ebc-ba32-457a3876582a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:42.492038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127a9ec-0091-4ebc-ba32-457a3876582a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:42.497583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659f36b-ac1d-49af-a8c0-ef28f6e153dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:42.504864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659f36b-ac1d-49af-a8c0-ef28f6e153dd admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:42.504864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659f36b-ac1d-49af-a8c0-ef28f6e153dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:42.504864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659f36b-ac1d-49af-a8c0-ef28f6e153dd admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:42.504864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659f36b-ac1d-49af-a8c0-ef28f6e153dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:42.504864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659f36b-ac1d-49af-a8c0-ef28f6e153dd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:42.539875 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3703/7436] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:42 2022] POST /identity/v3/projects => generated 363 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:42.544948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb24820-31bc-4f63-b4ff-5948ac867cb1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:42.565796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb24820-31bc-4f63-b4ff-5948ac867cb1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:42.662464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda18fb8-f609-45f8-a1eb-fee355df5187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:42.663504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda18fb8-f609-45f8-a1eb-fee355df5187 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:42.664025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda18fb8-f609-45f8-a1eb-fee355df5187 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:42.664359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda18fb8-f609-45f8-a1eb-fee355df5187 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:42.665106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda18fb8-f609-45f8-a1eb-fee355df5187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:42.667095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda18fb8-f609-45f8-a1eb-fee355df5187 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:42.698003 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3734/7437] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:42 2022] POST /identity/v3/users => generated 282 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:42.700335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659f36b-ac1d-49af-a8c0-ef28f6e153dd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:42.715204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659f36b-ac1d-49af-a8c0-ef28f6e153dd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:42.720733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94888411-a1d6-43f2-a4bf-7f11e824a165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:42.721553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94888411-a1d6-43f2-a4bf-7f11e824a165 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:42.721830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94888411-a1d6-43f2-a4bf-7f11e824a165 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:42.722099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94888411-a1d6-43f2-a4bf-7f11e824a165 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:42.724131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94888411-a1d6-43f2-a4bf-7f11e824a165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:42.725793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94888411-a1d6-43f2-a4bf-7f11e824a165 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:42.738538 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3704/7438] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:42.742914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda18fb8-f609-45f8-a1eb-fee355df5187 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:42.754506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda18fb8-f609-45f8-a1eb-fee355df5187 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:42.760294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36984f93-2162-4aa7-87a9-0d13d9cddeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:42.760986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36984f93-2162-4aa7-87a9-0d13d9cddeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:42.761400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36984f93-2162-4aa7-87a9-0d13d9cddeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:42.761826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36984f93-2162-4aa7-87a9-0d13d9cddeb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:42.762864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36984f93-2162-4aa7-87a9-0d13d9cddeb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:42.764199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36984f93-2162-4aa7-87a9-0d13d9cddeb4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:42.778883 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3735/7439] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:42.785893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94888411-a1d6-43f2-a4bf-7f11e824a165 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:42.798200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94888411-a1d6-43f2-a4bf-7f11e824a165 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:42.803602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a648cc-9be6-4871-ac0e-18332bf33420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:42.804683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a648cc-9be6-4871-ac0e-18332bf33420 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:42.805084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a648cc-9be6-4871-ac0e-18332bf33420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:42.805455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a648cc-9be6-4871-ac0e-18332bf33420 admin admin] PATH_INFO: `/v3/projects/8f5e428ac4244b8dadb7dea88db74c62/users/0e531333c34b4a94af62e2b9f45402d2/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:42.829474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a648cc-9be6-4871-ac0e-18332bf33420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5e428ac4244b8dadb7dea88db74c62, user_id=0e531333c34b4a94af62e2b9f45402d2, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:42.833168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a648cc-9be6-4871-ac0e-18332bf33420 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:42.864423 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3705/7440] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:42 2022] PUT /identity/v3/projects/8f5e428ac4244b8dadb7dea88db74c62/users/0e531333c34b4a94af62e2b9f45402d2/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:42.876038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f9173-00fb-460a-a893-eed540e3ba54 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:42.876472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f9173-00fb-460a-a893-eed540e3ba54 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:42.876844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f9173-00fb-460a-a893-eed540e3ba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:42.910582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cd6f9173-00fb-460a-a893-eed540e3ba54 None None] MFA Rules not processed for user `0e531333c34b4a94af62e2b9f45402d2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:42.912011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cd6f9173-00fb-460a-a893-eed540e3ba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:42.927100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:42.987741 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3736/7441] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:42.992172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a648cc-9be6-4871-ac0e-18332bf33420 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:43.057264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a648cc-9be6-4871-ac0e-18332bf33420 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:43.062766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e157f10-d0e2-4982-9129-0608caa1c4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:43.063540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e157f10-d0e2-4982-9129-0608caa1c4d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:43.063817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e157f10-d0e2-4982-9129-0608caa1c4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:43.064090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e157f10-d0e2-4982-9129-0608caa1c4d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:43.064736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e157f10-d0e2-4982-9129-0608caa1c4d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:43.066728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e157f10-d0e2-4982-9129-0608caa1c4d6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:43.103989 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3706/7442] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:42 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:43.108415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f9173-00fb-460a-a893-eed540e3ba54 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:43.119425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f9173-00fb-460a-a893-eed540e3ba54 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:43.180953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b20c1-d056-478a-bda2-28ee69f21799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:43.181619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b20c1-d056-478a-bda2-28ee69f21799 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:43.181892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b20c1-d056-478a-bda2-28ee69f21799 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:43.182158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b20c1-d056-478a-bda2-28ee69f21799 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:43.182779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b20c1-d056-478a-bda2-28ee69f21799 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:43.184908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b20c1-d056-478a-bda2-28ee69f21799 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:43.213752 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3737/7443] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:43 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:43.215650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e157f10-d0e2-4982-9129-0608caa1c4d6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:43.226437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e157f10-d0e2-4982-9129-0608caa1c4d6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:43.231567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc401992-ea71-4d55-b0ac-1474900f2252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:43.232650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc401992-ea71-4d55-b0ac-1474900f2252 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:43.233001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc401992-ea71-4d55-b0ac-1474900f2252 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:43.233368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc401992-ea71-4d55-b0ac-1474900f2252 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:43.234210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc401992-ea71-4d55-b0ac-1474900f2252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:43.236100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc401992-ea71-4d55-b0ac-1474900f2252 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:43.248559 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3707/7444] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:43.252631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b20c1-d056-478a-bda2-28ee69f21799 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:43.263923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b20c1-d056-478a-bda2-28ee69f21799 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:43.268818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701c6e2-9775-4edc-a628-516c9fe5f399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:43.269609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701c6e2-9775-4edc-a628-516c9fe5f399 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:43.270005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701c6e2-9775-4edc-a628-516c9fe5f399 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:43.270277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701c6e2-9775-4edc-a628-516c9fe5f399 admin admin] PATH_INFO: `/v3/projects/5dc784afaf1e4230abf70ddee9f79aec/users/6e2887a319984f61805bbcd6f7e7d772/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:43.289983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701c6e2-9775-4edc-a628-516c9fe5f399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc784afaf1e4230abf70ddee9f79aec, user_id=6e2887a319984f61805bbcd6f7e7d772, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:43.293944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701c6e2-9775-4edc-a628-516c9fe5f399 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:43.312156 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3738/7445] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:43 2022] PUT /identity/v3/projects/5dc784afaf1e4230abf70ddee9f79aec/users/6e2887a319984f61805bbcd6f7e7d772/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:43.315921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc401992-ea71-4d55-b0ac-1474900f2252 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:43.327143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc401992-ea71-4d55-b0ac-1474900f2252 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:43.331148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:43.413652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f4331-8f5e-4ced-8217-33cc1ce1b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:43.413652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f4331-8f5e-4ced-8217-33cc1ce1b21d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:43.414236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f4331-8f5e-4ced-8217-33cc1ce1b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:43.414333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f4331-8f5e-4ced-8217-33cc1ce1b21d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:43.414925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f4331-8f5e-4ced-8217-33cc1ce1b21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:43.416967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f4331-8f5e-4ced-8217-33cc1ce1b21d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:43.430746 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3708/7446] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:43.434723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701c6e2-9775-4edc-a628-516c9fe5f399 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:43.447811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701c6e2-9775-4edc-a628-516c9fe5f399 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:43.453205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51706158-13b4-4129-99ba-5446d22c6bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:43.453869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51706158-13b4-4129-99ba-5446d22c6bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:43.454192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51706158-13b4-4129-99ba-5446d22c6bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:43.454596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51706158-13b4-4129-99ba-5446d22c6bc4 admin admin] PATH_INFO: `/v3/projects/5dc784afaf1e4230abf70ddee9f79aec/users/6e2887a319984f61805bbcd6f7e7d772/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:43.456654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51706158-13b4-4129-99ba-5446d22c6bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc784afaf1e4230abf70ddee9f79aec, user_id=6e2887a319984f61805bbcd6f7e7d772, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:43.459708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51706158-13b4-4129-99ba-5446d22c6bc4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:43.477627 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3739/7447] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:43 2022] PUT /identity/v3/projects/5dc784afaf1e4230abf70ddee9f79aec/users/6e2887a319984f61805bbcd6f7e7d772/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:19:43.488827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091614a5-2364-41fd-8cc6-a3c86539263b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:43.489286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091614a5-2364-41fd-8cc6-a3c86539263b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:43.489604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091614a5-2364-41fd-8cc6-a3c86539263b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:43.534004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-091614a5-2364-41fd-8cc6-a3c86539263b None None] MFA Rules not processed for user `6e2887a319984f61805bbcd6f7e7d772`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:43.535221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-091614a5-2364-41fd-8cc6-a3c86539263b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:43.677093 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3709/7448] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:19:43.686157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51706158-13b4-4129-99ba-5446d22c6bc4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:43.687929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-51706158-13b4-4129-99ba-5446d22c6bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:43.762936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51706158-13b4-4129-99ba-5446d22c6bc4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:43.766876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1700a2c-ba82-4ee7-9d6e-e4099b60bc76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:43.767516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1700a2c-ba82-4ee7-9d6e-e4099b60bc76 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:43.767789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1700a2c-ba82-4ee7-9d6e-e4099b60bc76 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:43.768054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1700a2c-ba82-4ee7-9d6e-e4099b60bc76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:43.780540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1700a2c-ba82-4ee7-9d6e-e4099b60bc76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:43.782006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1700a2c-ba82-4ee7-9d6e-e4099b60bc76 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:43.795431 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3740/7449] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:44.002869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091614a5-2364-41fd-8cc6-a3c86539263b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:44.004682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-091614a5-2364-41fd-8cc6-a3c86539263b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:44.144959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091614a5-2364-41fd-8cc6-a3c86539263b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:44.156996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ea232-1f26-4c44-9cce-a0c428c6e4a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:44.157747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ea232-1f26-4c44-9cce-a0c428c6e4a0 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:44.158043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ea232-1f26-4c44-9cce-a0c428c6e4a0 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:44.158343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ea232-1f26-4c44-9cce-a0c428c6e4a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:44.161071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0a4ea232-1f26-4c44-9cce-a0c428c6e4a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:44.213709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:44.263205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ea232-1f26-4c44-9cce-a0c428c6e4a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:44.265565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ea232-1f26-4c44-9cce-a0c428c6e4a0 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:44.288436 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3710/7450] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:47.017328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1700a2c-ba82-4ee7-9d6e-e4099b60bc76 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:47.029833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1700a2c-ba82-4ee7-9d6e-e4099b60bc76 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:47.035522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce296db-fdc5-45fc-a7ef-543f40c41784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:47.037443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce296db-fdc5-45fc-a7ef-543f40c41784 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:47.037954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce296db-fdc5-45fc-a7ef-543f40c41784 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:47.038558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce296db-fdc5-45fc-a7ef-543f40c41784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:47.048189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:47.152869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce296db-fdc5-45fc-a7ef-543f40c41784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:47.154494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce296db-fdc5-45fc-a7ef-543f40c41784 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:47.179726 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3741/7451] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:48.716131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ea232-1f26-4c44-9cce-a0c428c6e4a0 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:48.821902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ea232-1f26-4c44-9cce-a0c428c6e4a0 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:48.828223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:48.828869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:48.830017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:48.830296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] PATH_INFO: `/v3/users/0e531333c34b4a94af62e2b9f45402d2` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:48.844053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e531333c34b4a94af62e2b9f45402d2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:48.847541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:48.952108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e531333c34b4a94af62e2b9f45402d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:48.979002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e531333c34b4a94af62e2b9f45402d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:48.990331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e531333c34b4a94af62e2b9f45402d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:49.000653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e531333c34b4a94af62e2b9f45402d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:49.001975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e531333c34b4a94af62e2b9f45402d2'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:49.014623 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3711/7452] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:48 2022] DELETE /identity/v3/users/0e531333c34b4a94af62e2b9f45402d2 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:49.017077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5a7ff-3f1e-47cb-8b9e-18ed021b7b78 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:49.018408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5a7ff-3f1e-47cb-8b9e-18ed021b7b78 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:49.018674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5a7ff-3f1e-47cb-8b9e-18ed021b7b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:49.056013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a2e5a7ff-3f1e-47cb-8b9e-18ed021b7b78 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:49.057407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a2e5a7ff-3f1e-47cb-8b9e-18ed021b7b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:49.174970 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3742/7453] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:49.187640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:49.196970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:49.283211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:49.321925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81af0d6-b0ef-4232-8423-2d07b83a041a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:49.330530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a74fb-0fb2-4e26-9584-9c70036f0bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:49.331517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a74fb-0fb2-4e26-9584-9c70036f0bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:49.331959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a74fb-0fb2-4e26-9584-9c70036f0bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:49.332345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a74fb-0fb2-4e26-9584-9c70036f0bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:49.346351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a74fb-0fb2-4e26-9584-9c70036f0bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:49.348081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a74fb-0fb2-4e26-9584-9c70036f0bc3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:49.362673 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3712/7454] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:50.032287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5a7ff-3f1e-47cb-8b9e-18ed021b7b78 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:50.034883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a2e5a7ff-3f1e-47cb-8b9e-18ed021b7b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:50.051711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5a7ff-3f1e-47cb-8b9e-18ed021b7b78 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:50.058859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:50.060338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:50.060798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:50.061234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] PATH_INFO: `/v3/projects/8f5e428ac4244b8dadb7dea88db74c62` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:50.062669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5e428ac4244b8dadb7dea88db74c62)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:50.064999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:50.159098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5e428ac4244b8dadb7dea88db74c62', 'request_id': 'req-9e3b89b9-899f-4c39-8436-1ecfbde155a0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:50.175413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5e428ac4244b8dadb7dea88db74c62', 'request_id': 'req-9e3b89b9-899f-4c39-8436-1ecfbde155a0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:50.184730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5e428ac4244b8dadb7dea88db74c62', 'request_id': 'req-9e3b89b9-899f-4c39-8436-1ecfbde155a0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:50.202815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] The token cache is being invalidate because project 8f5e428ac4244b8dadb7dea88db74c62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:50.203663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:50.204499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:50.208753 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3743/7455] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:50 2022] DELETE /identity/v3/projects/8f5e428ac4244b8dadb7dea88db74c62 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:50.212769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a74fb-0fb2-4e26-9584-9c70036f0bc3 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:50.224733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a74fb-0fb2-4e26-9584-9c70036f0bc3 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:50.338250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:50.342089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:50.342089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:50.344650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] PATH_INFO: `/v3/users/6e2887a319984f61805bbcd6f7e7d772` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:50.349486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2887a319984f61805bbcd6f7e7d772)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:50.356543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:50.482902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2887a319984f61805bbcd6f7e7d772'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:50.514553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2887a319984f61805bbcd6f7e7d772'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:50.537545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2887a319984f61805bbcd6f7e7d772'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:50.562181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2887a319984f61805bbcd6f7e7d772'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:50.563435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c43cba8a-098e-4ee9-8436-897913034512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2887a319984f61805bbcd6f7e7d772'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:50.582753 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3713/7456] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:50 2022] DELETE /identity/v3/users/6e2887a319984f61805bbcd6f7e7d772 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:51.220155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:51.221811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:51.258500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b89b9-899f-4c39-8436-1ecfbde155a0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:51.312536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:51.394365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:51.395379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:51.395502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:51.395710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] PATH_INFO: `/v3/projects/5dc784afaf1e4230abf70ddee9f79aec` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:51.397126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc784afaf1e4230abf70ddee9f79aec)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:51.400579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:51.473101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc784afaf1e4230abf70ddee9f79aec', 'request_id': 'req-ab6aea44-3fd2-4104-bca1-25649aa53b19'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:51.484745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc784afaf1e4230abf70ddee9f79aec', 'request_id': 'req-ab6aea44-3fd2-4104-bca1-25649aa53b19'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:51.494499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc784afaf1e4230abf70ddee9f79aec', 'request_id': 'req-ab6aea44-3fd2-4104-bca1-25649aa53b19'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:51.511449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] The token cache is being invalidate because project 5dc784afaf1e4230abf70ddee9f79aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:51.511804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:51.513223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ab6aea44-3fd2-4104-bca1-25649aa53b19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:51.519580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb748b2-4202-4a90-8387-c10fac7408f9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:51.520170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb748b2-4202-4a90-8387-c10fac7408f9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:51.520269 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3744/7457] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:51 2022] DELETE /identity/v3/projects/5dc784afaf1e4230abf70ddee9f79aec => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:51.520365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb748b2-4202-4a90-8387-c10fac7408f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:51.562805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0fb748b2-4202-4a90-8387-c10fac7408f9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:51.563547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0fb748b2-4202-4a90-8387-c10fac7408f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:51.667854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:51.697945 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3714/7458] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:19:51.711136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e3054-95bb-4a6e-8a79-ff5cec4b9865 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:51.711632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e3054-95bb-4a6e-8a79-ff5cec4b9865 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:51.713336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e3054-95bb-4a6e-8a79-ff5cec4b9865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:51.758727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a65e3054-95bb-4a6e-8a79-ff5cec4b9865 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:51.759976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a65e3054-95bb-4a6e-8a79-ff5cec4b9865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:51.781756 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3745/7459] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:51.785634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb748b2-4202-4a90-8387-c10fac7408f9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:51.800078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb748b2-4202-4a90-8387-c10fac7408f9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:51.808915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ef139-159f-4e61-b753-7081af35f340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:51.809587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ef139-159f-4e61-b753-7081af35f340 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:51.809997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ef139-159f-4e61-b753-7081af35f340 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:51.810277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ef139-159f-4e61-b753-7081af35f340 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:51.810936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ef139-159f-4e61-b753-7081af35f340 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:19:51.811238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ef139-159f-4e61-b753-7081af35f340 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:51.812863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ef139-159f-4e61-b753-7081af35f340 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:51.838620 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3715/7460] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:19:51.840718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e3054-95bb-4a6e-8a79-ff5cec4b9865 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:51.853985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e3054-95bb-4a6e-8a79-ff5cec4b9865 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:51.872211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42297ef-d9d5-4eb7-9c99-408e1d46a76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:51.872844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42297ef-d9d5-4eb7-9c99-408e1d46a76c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:51.873148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42297ef-d9d5-4eb7-9c99-408e1d46a76c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:51.873422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42297ef-d9d5-4eb7-9c99-408e1d46a76c admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:51.874226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42297ef-d9d5-4eb7-9c99-408e1d46a76c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:51.879888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42297ef-d9d5-4eb7-9c99-408e1d46a76c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:51.951317 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3746/7461] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:51 2022] POST /identity/v3/projects => generated 393 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:51.956472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ef139-159f-4e61-b753-7081af35f340 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:51.972755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ef139-159f-4e61-b753-7081af35f340 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:52.125784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268cecd-b84e-4fb3-8ff4-3a90ad20df12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:52.128672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268cecd-b84e-4fb3-8ff4-3a90ad20df12 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:52.128861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268cecd-b84e-4fb3-8ff4-3a90ad20df12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:52.129210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268cecd-b84e-4fb3-8ff4-3a90ad20df12 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:52.130099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268cecd-b84e-4fb3-8ff4-3a90ad20df12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:52.132508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268cecd-b84e-4fb3-8ff4-3a90ad20df12 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:52.170472 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3716/7462] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:51 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:52.174285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42297ef-d9d5-4eb7-9c99-408e1d46a76c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:52.187049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42297ef-d9d5-4eb7-9c99-408e1d46a76c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:52.203206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949e114-40ae-48ac-9888-d5609c5e9b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:52.204438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949e114-40ae-48ac-9888-d5609c5e9b98 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:52.204952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949e114-40ae-48ac-9888-d5609c5e9b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:52.205508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949e114-40ae-48ac-9888-d5609c5e9b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:52.208810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949e114-40ae-48ac-9888-d5609c5e9b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:52.213897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949e114-40ae-48ac-9888-d5609c5e9b98 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:52.233347 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3747/7463] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:19:52.234579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268cecd-b84e-4fb3-8ff4-3a90ad20df12 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:52.246007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268cecd-b84e-4fb3-8ff4-3a90ad20df12 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:52.252699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cdb8a-ceaa-436c-9ce5-e002ff6aa22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:52.253595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cdb8a-ceaa-436c-9ce5-e002ff6aa22e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:52.254434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cdb8a-ceaa-436c-9ce5-e002ff6aa22e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:52.254434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cdb8a-ceaa-436c-9ce5-e002ff6aa22e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:52.255231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cdb8a-ceaa-436c-9ce5-e002ff6aa22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:52.257619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cdb8a-ceaa-436c-9ce5-e002ff6aa22e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:52.277308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949e114-40ae-48ac-9888-d5609c5e9b98 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:52.285662 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3717/7464] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:52.291784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949e114-40ae-48ac-9888-d5609c5e9b98 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:52.298305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc244ae0-d0aa-4088-8363-212f4cf05973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:52.299258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc244ae0-d0aa-4088-8363-212f4cf05973 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:52.299676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc244ae0-d0aa-4088-8363-212f4cf05973 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:52.301915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc244ae0-d0aa-4088-8363-212f4cf05973 admin admin] PATH_INFO: `/v3/projects/4213bac5947f4ee88d9368ade5de58d7/users/9b335a93fe4c44a99719cbfe03baffed/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:52.327586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc244ae0-d0aa-4088-8363-212f4cf05973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4213bac5947f4ee88d9368ade5de58d7, user_id=9b335a93fe4c44a99719cbfe03baffed, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:52.335915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc244ae0-d0aa-4088-8363-212f4cf05973 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:52.372157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a62f8-6629-4534-964d-f9e93d513cd6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:52.372585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a62f8-6629-4534-964d-f9e93d513cd6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:52.373001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a62f8-6629-4534-964d-f9e93d513cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:52.375874 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3748/7465] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:52 2022] PUT /identity/v3/projects/4213bac5947f4ee88d9368ade5de58d7/users/9b335a93fe4c44a99719cbfe03baffed/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:52.422602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-932a62f8-6629-4534-964d-f9e93d513cd6 None None] MFA Rules not processed for user `9b335a93fe4c44a99719cbfe03baffed`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:52.424389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-932a62f8-6629-4534-964d-f9e93d513cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:52.551163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc244ae0-d0aa-4088-8363-212f4cf05973 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:52.554872 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3718/7466] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:52.660599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:52.676492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc244ae0-d0aa-4088-8363-212f4cf05973 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:52.693258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085402ea-98ab-4286-a08f-0066b17b4c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:52.694118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085402ea-98ab-4286-a08f-0066b17b4c5b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:52.694765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085402ea-98ab-4286-a08f-0066b17b4c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:52.695098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085402ea-98ab-4286-a08f-0066b17b4c5b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:52.695742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085402ea-98ab-4286-a08f-0066b17b4c5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:52.699084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085402ea-98ab-4286-a08f-0066b17b4c5b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:52.744179 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3749/7467] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:19:52 2022] POST /identity/v3/projects => generated 393 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:52.747920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a62f8-6629-4534-964d-f9e93d513cd6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:52.766341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a62f8-6629-4534-964d-f9e93d513cd6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:52.869258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:52.886996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4599c4-547a-457d-88ec-af35821acb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:52.887874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4599c4-547a-457d-88ec-af35821acb31 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:52.888331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4599c4-547a-457d-88ec-af35821acb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:52.889388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4599c4-547a-457d-88ec-af35821acb31 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:52.890677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4599c4-547a-457d-88ec-af35821acb31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:52.892818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4599c4-547a-457d-88ec-af35821acb31 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:52.945261 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3719/7468] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:19:52 2022] POST /identity/v3/users => generated 303 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:19:52.950069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085402ea-98ab-4286-a08f-0066b17b4c5b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:52.971600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085402ea-98ab-4286-a08f-0066b17b4c5b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:52.978965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba890bc-cf67-4845-9a2c-7ab7288911a0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:52.979388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba890bc-cf67-4845-9a2c-7ab7288911a0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:52.979660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba890bc-cf67-4845-9a2c-7ab7288911a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:52.982947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae28f4-bccf-4722-adc2-657435582784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:52.983959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae28f4-bccf-4722-adc2-657435582784 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:52.984428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae28f4-bccf-4722-adc2-657435582784 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:52.984841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae28f4-bccf-4722-adc2-657435582784 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:52.986482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae28f4-bccf-4722-adc2-657435582784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:52.988693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae28f4-bccf-4722-adc2-657435582784 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:53.018453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0ba890bc-cf67-4845-9a2c-7ab7288911a0 None None] MFA Rules not processed for user `379c929d93b54b9a9751defeef161f57`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:53.019720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0ba890bc-cf67-4845-9a2c-7ab7288911a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:53.027309 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3750/7469] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:19:53.029739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae28f4-bccf-4722-adc2-657435582784 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:53.058895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae28f4-bccf-4722-adc2-657435582784 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:53.071306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f340c-6958-4c96-9376-4063775c387b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:53.072027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f340c-6958-4c96-9376-4063775c387b admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:53.072254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f340c-6958-4c96-9376-4063775c387b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:53.072522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f340c-6958-4c96-9376-4063775c387b admin admin] PATH_INFO: `/v3/projects/ee785f2b3b154cb490bc145f299c9b0b/users/9cb2b9affb9740369244529aa3e09246/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:53.095492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f340c-6958-4c96-9376-4063775c387b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee785f2b3b154cb490bc145f299c9b0b, user_id=9cb2b9affb9740369244529aa3e09246, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:53.098920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f340c-6958-4c96-9376-4063775c387b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:53.122879 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3751/7470] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:53 2022] PUT /identity/v3/projects/ee785f2b3b154cb490bc145f299c9b0b/users/9cb2b9affb9740369244529aa3e09246/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:53.126428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f340c-6958-4c96-9376-4063775c387b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:53.130459 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3720/7471] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:53.240015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f340c-6958-4c96-9376-4063775c387b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:53.250655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa6a65-9962-4a3e-bfd0-ae7705728d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:53.251290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa6a65-9962-4a3e-bfd0-ae7705728d6c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:53.251568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa6a65-9962-4a3e-bfd0-ae7705728d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:53.251838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa6a65-9962-4a3e-bfd0-ae7705728d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:53.252484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa6a65-9962-4a3e-bfd0-ae7705728d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:53.264496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa6a65-9962-4a3e-bfd0-ae7705728d6c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:53.281323 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3752/7472] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:19:53.284752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba890bc-cf67-4845-9a2c-7ab7288911a0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:53.304558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba890bc-cf67-4845-9a2c-7ab7288911a0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:53.312110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebf563-2eba-4137-ab19-8b0fb205a097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:53.312908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebf563-2eba-4137-ab19-8b0fb205a097 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:53.313289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebf563-2eba-4137-ab19-8b0fb205a097 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:53.313695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebf563-2eba-4137-ab19-8b0fb205a097 admin admin] PATH_INFO: `/v3/projects/ee785f2b3b154cb490bc145f299c9b0b/users/9cb2b9affb9740369244529aa3e09246/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:53.315744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bebf563-2eba-4137-ab19-8b0fb205a097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee785f2b3b154cb490bc145f299c9b0b, user_id=9cb2b9affb9740369244529aa3e09246, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:53.318622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bebf563-2eba-4137-ab19-8b0fb205a097 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:53.337147 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3721/7473] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:19:53 2022] PUT /identity/v3/projects/ee785f2b3b154cb490bc145f299c9b0b/users/9cb2b9affb9740369244529aa3e09246/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:53.349201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd710db-8e33-4b0f-bbb1-2f9a17f673cd None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:53.350012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd710db-8e33-4b0f-bbb1-2f9a17f673cd None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:53.350551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd710db-8e33-4b0f-bbb1-2f9a17f673cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:53.391768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fcd710db-8e33-4b0f-bbb1-2f9a17f673cd None None] MFA Rules not processed for user `9cb2b9affb9740369244529aa3e09246`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:53.393143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fcd710db-8e33-4b0f-bbb1-2f9a17f673cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:53.524801 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3753/7474] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:53.541744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebf563-2eba-4137-ab19-8b0fb205a097 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:53.545852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3bebf563-2eba-4137-ab19-8b0fb205a097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:53.674797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebf563-2eba-4137-ab19-8b0fb205a097 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:53.683954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75883088-c532-4c70-a5e5-666211ee442e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:53.684614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75883088-c532-4c70-a5e5-666211ee442e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:53.684896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75883088-c532-4c70-a5e5-666211ee442e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:53.685214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75883088-c532-4c70-a5e5-666211ee442e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:53.696524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75883088-c532-4c70-a5e5-666211ee442e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:53.698738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75883088-c532-4c70-a5e5-666211ee442e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:53.715090 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3722/7475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:53 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:57.474683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd710db-8e33-4b0f-bbb1-2f9a17f673cd None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:57.540481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:57.608021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd710db-8e33-4b0f-bbb1-2f9a17f673cd None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:57.613760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:57.614534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:57.614809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:57.615190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] PATH_INFO: `/v3/users/9b335a93fe4c44a99719cbfe03baffed` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:57.616401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b335a93fe4c44a99719cbfe03baffed)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:57.619128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:57.706098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b335a93fe4c44a99719cbfe03baffed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:57.725388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b335a93fe4c44a99719cbfe03baffed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:57.753338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b335a93fe4c44a99719cbfe03baffed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:57.770037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b335a93fe4c44a99719cbfe03baffed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:57.770832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b335a93fe4c44a99719cbfe03baffed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:57.789838 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3754/7476] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:57 2022] DELETE /identity/v3/users/9b335a93fe4c44a99719cbfe03baffed => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:57.794041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb32ea-503e-440b-b6d3-95c2dc6b37c2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:57.794960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb32ea-503e-440b-b6d3-95c2dc6b37c2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:57.795611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb32ea-503e-440b-b6d3-95c2dc6b37c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:57.827008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7abb32ea-503e-440b-b6d3-95c2dc6b37c2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:19:57.828973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7abb32ea-503e-440b-b6d3-95c2dc6b37c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:57.976999 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3723/7477] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:19:58.006052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:58.008407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:58.187593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8488fc5-4ea7-47f1-af35-894262661d3b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:58.191296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77fb2e-d2a0-4e3e-9e84-12ed8fbc5545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:58.191843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77fb2e-d2a0-4e3e-9e84-12ed8fbc5545 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:58.192122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77fb2e-d2a0-4e3e-9e84-12ed8fbc5545 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:58.192393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77fb2e-d2a0-4e3e-9e84-12ed8fbc5545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:58.268503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77fb2e-d2a0-4e3e-9e84-12ed8fbc5545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:58.279343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77fb2e-d2a0-4e3e-9e84-12ed8fbc5545 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:58.310845 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3755/7478] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:19:59.123244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb32ea-503e-440b-b6d3-95c2dc6b37c2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:59.124487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7abb32ea-503e-440b-b6d3-95c2dc6b37c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:19:59.143698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb32ea-503e-440b-b6d3-95c2dc6b37c2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:59.158369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:59.159315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:59.159759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:59.160183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] PATH_INFO: `/v3/projects/4213bac5947f4ee88d9368ade5de58d7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:59.161681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4213bac5947f4ee88d9368ade5de58d7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:59.164117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:59.184464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:59.271627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4213bac5947f4ee88d9368ade5de58d7', 'request_id': 'req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:59.293895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4213bac5947f4ee88d9368ade5de58d7', 'request_id': 'req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:59.302238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4213bac5947f4ee88d9368ade5de58d7', 'request_id': 'req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:59.328912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] The token cache is being invalidate because project 4213bac5947f4ee88d9368ade5de58d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:19:59.329791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:59.330722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:59.333081 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3724/7479] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:19:59 2022] DELETE /identity/v3/projects/4213bac5947f4ee88d9368ade5de58d7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:19:59.336795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77fb2e-d2a0-4e3e-9e84-12ed8fbc5545 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:19:59.351441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77fb2e-d2a0-4e3e-9e84-12ed8fbc5545 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:19:59.479406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:59.500213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:19:59.500213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:19:59.500213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:19:59.500687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] PATH_INFO: `/v3/users/9cb2b9affb9740369244529aa3e09246` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:19:59.502403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb2b9affb9740369244529aa3e09246)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:19:59.505517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:19:59.589128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb2b9affb9740369244529aa3e09246'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:59.599500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb2b9affb9740369244529aa3e09246'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:59.623577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb2b9affb9740369244529aa3e09246'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:59.636517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb2b9affb9740369244529aa3e09246'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:59.637659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5cf8dd80-06d0-4db4-bc25-83d3a398172b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb2b9affb9740369244529aa3e09246'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:19:59.652789 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3756/7480] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:19:59 2022] DELETE /identity/v3/users/9cb2b9affb9740369244529aa3e09246 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:00.004494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:00.006303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:00.026587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa7b6c-7d91-41ec-9f6b-fd53063e511d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:00.157539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:00.158206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:00.159505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:00.159946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] PATH_INFO: `/v3/projects/ee785f2b3b154cb490bc145f299c9b0b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:00.163470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee785f2b3b154cb490bc145f299c9b0b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:00.170416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:00.310682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee785f2b3b154cb490bc145f299c9b0b', 'request_id': 'req-4a00878e-c1d1-4699-a071-2985c9b4240c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:00.325931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee785f2b3b154cb490bc145f299c9b0b', 'request_id': 'req-4a00878e-c1d1-4699-a071-2985c9b4240c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:00.346586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee785f2b3b154cb490bc145f299c9b0b', 'request_id': 'req-4a00878e-c1d1-4699-a071-2985c9b4240c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:00.374727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] The token cache is being invalidate because project ee785f2b3b154cb490bc145f299c9b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:00.375021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:00.378265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a00878e-c1d1-4699-a071-2985c9b4240c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:00.380647 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3725/7481] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:00 2022] DELETE /identity/v3/projects/ee785f2b3b154cb490bc145f299c9b0b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:00.386585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f7747-30de-4b9c-a223-484ecfa2ac52 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:00.387723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f7747-30de-4b9c-a223-484ecfa2ac52 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:00.393340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f7747-30de-4b9c-a223-484ecfa2ac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:00.419349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5e9f7747-30de-4b9c-a223-484ecfa2ac52 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:00.421346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5e9f7747-30de-4b9c-a223-484ecfa2ac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:00.555227 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3757/7482] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:00.561821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6aad4-9009-4bab-9242-a06a4d5be582 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:00.562107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6aad4-9009-4bab-9242-a06a4d5be582 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:00.562392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6aad4-9009-4bab-9242-a06a4d5be582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:00.596182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a7a6aad4-9009-4bab-9242-a06a4d5be582 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:00.605816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a7a6aad4-9009-4bab-9242-a06a4d5be582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:00.627100 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3726/7483] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:00.628295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f7747-30de-4b9c-a223-484ecfa2ac52 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:00.641702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f7747-30de-4b9c-a223-484ecfa2ac52 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:00.653114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7265dac-2e4f-4909-bba4-7025bfdc4771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:00.654797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7265dac-2e4f-4909-bba4-7025bfdc4771 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:00.655161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7265dac-2e4f-4909-bba4-7025bfdc4771 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:00.655583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7265dac-2e4f-4909-bba4-7025bfdc4771 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:00.656254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7265dac-2e4f-4909-bba4-7025bfdc4771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:20:00.656632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7265dac-2e4f-4909-bba4-7025bfdc4771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:00.659450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7265dac-2e4f-4909-bba4-7025bfdc4771 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:00.683562 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3758/7484] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:20:00.685189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6aad4-9009-4bab-9242-a06a4d5be582 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:00.708591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6aad4-9009-4bab-9242-a06a4d5be582 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:00.716029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c9132-f225-48a8-9efc-24c4e26ac617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:00.716999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c9132-f225-48a8-9efc-24c4e26ac617 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:00.717603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c9132-f225-48a8-9efc-24c4e26ac617 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:00.718076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c9132-f225-48a8-9efc-24c4e26ac617 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:00.718925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c9132-f225-48a8-9efc-24c4e26ac617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:00.721655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c9132-f225-48a8-9efc-24c4e26ac617 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:00.772481 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3727/7485] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:00 2022] POST /identity/v3/projects => generated 397 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:00.775706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7265dac-2e4f-4909-bba4-7025bfdc4771 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:00.789509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7265dac-2e4f-4909-bba4-7025bfdc4771 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:00.888546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:00.905335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80526177-c81f-4960-bcd6-9559af019120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:00.906970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80526177-c81f-4960-bcd6-9559af019120 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:00.907361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80526177-c81f-4960-bcd6-9559af019120 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:00.908071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80526177-c81f-4960-bcd6-9559af019120 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:00.910154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80526177-c81f-4960-bcd6-9559af019120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:00.912583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80526177-c81f-4960-bcd6-9559af019120 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:00.961173 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3759/7486] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:00 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:00.965669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c9132-f225-48a8-9efc-24c4e26ac617 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:00.977834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:00.999049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c9132-f225-48a8-9efc-24c4e26ac617 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:01.016448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69927e53-bb03-4f5f-acf1-61b9e35a7639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:01.017244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69927e53-bb03-4f5f-acf1-61b9e35a7639 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:01.017598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69927e53-bb03-4f5f-acf1-61b9e35a7639 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:01.017937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69927e53-bb03-4f5f-acf1-61b9e35a7639 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:01.018792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69927e53-bb03-4f5f-acf1-61b9e35a7639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:01.021021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69927e53-bb03-4f5f-acf1-61b9e35a7639 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:01.042934 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3728/7487] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:01.044977 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80526177-c81f-4960-bcd6-9559af019120 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:01.064208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80526177-c81f-4960-bcd6-9559af019120 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:01.074632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599ca9f-6e39-4f06-b414-98056053be10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:01.075676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599ca9f-6e39-4f06-b414-98056053be10 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:01.075801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599ca9f-6e39-4f06-b414-98056053be10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:01.076033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599ca9f-6e39-4f06-b414-98056053be10 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:01.077726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599ca9f-6e39-4f06-b414-98056053be10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:01.081403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599ca9f-6e39-4f06-b414-98056053be10 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:01.101548 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3760/7488] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:20:01.103706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69927e53-bb03-4f5f-acf1-61b9e35a7639 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:01.118211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69927e53-bb03-4f5f-acf1-61b9e35a7639 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:01.133450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f3c78-39c9-4749-b5e4-7cf960788f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:01.135547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f3c78-39c9-4749-b5e4-7cf960788f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:01.137705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f3c78-39c9-4749-b5e4-7cf960788f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:01.142178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f3c78-39c9-4749-b5e4-7cf960788f68 admin admin] PATH_INFO: `/v3/projects/d4de7541478e491198d31312bc5dd750/users/de49589624e54ea8b12a5230d2a148c6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:01.390860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f3c78-39c9-4749-b5e4-7cf960788f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4de7541478e491198d31312bc5dd750, user_id=de49589624e54ea8b12a5230d2a148c6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:01.394666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f3c78-39c9-4749-b5e4-7cf960788f68 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:01.421235 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3729/7489] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:01 2022] PUT /identity/v3/projects/d4de7541478e491198d31312bc5dd750/users/de49589624e54ea8b12a5230d2a148c6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:01.425727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f980f-2d6b-4f4e-8fa4-7de54d40fcc0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:01.426201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f980f-2d6b-4f4e-8fa4-7de54d40fcc0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:01.428472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f980f-2d6b-4f4e-8fa4-7de54d40fcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:01.472198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-344f980f-2d6b-4f4e-8fa4-7de54d40fcc0 None None] MFA Rules not processed for user `de49589624e54ea8b12a5230d2a148c6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:01.524561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-344f980f-2d6b-4f4e-8fa4-7de54d40fcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:01.655092 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3761/7490] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:01 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:01.685785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f3c78-39c9-4749-b5e4-7cf960788f68 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:01.688534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3a5f3c78-39c9-4749-b5e4-7cf960788f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:01.817596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f3c78-39c9-4749-b5e4-7cf960788f68 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:01.831242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b371a35-e77c-424a-9f6c-417cc467ddae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:01.831866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b371a35-e77c-424a-9f6c-417cc467ddae service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:01.832135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b371a35-e77c-424a-9f6c-417cc467ddae service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:01.832396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b371a35-e77c-424a-9f6c-417cc467ddae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:01.848579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b371a35-e77c-424a-9f6c-417cc467ddae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:01.850109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b371a35-e77c-424a-9f6c-417cc467ddae service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:01.867565 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3730/7491] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:01 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:11.652325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:11.859529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:13.077268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f830e-e5c5-4ffb-913b-4b0a28029407 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:13.077850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f830e-e5c5-4ffb-913b-4b0a28029407 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:13.077850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f830e-e5c5-4ffb-913b-4b0a28029407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:13.121303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f69f830e-e5c5-4ffb-913b-4b0a28029407 None None] MFA Rules not processed for user `026771ec28444739afc13c47339af7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:13.123524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f69f830e-e5c5-4ffb-913b-4b0a28029407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:13.208530 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3762/7492] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:13.829556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b371a35-e77c-424a-9f6c-417cc467ddae service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:13.831134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2b371a35-e77c-424a-9f6c-417cc467ddae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:14.018217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b371a35-e77c-424a-9f6c-417cc467ddae service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:14.032361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9716b-5896-4913-8473-0ee17519493c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:14.033384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9716b-5896-4913-8473-0ee17519493c service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:14.033855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9716b-5896-4913-8473-0ee17519493c service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:14.034748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9716b-5896-4913-8473-0ee17519493c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:14.038512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9d9716b-5896-4913-8473-0ee17519493c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:14.068323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f830e-e5c5-4ffb-913b-4b0a28029407 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:14.070178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f69f830e-e5c5-4ffb-913b-4b0a28029407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:14.082615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9716b-5896-4913-8473-0ee17519493c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:14.088287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9716b-5896-4913-8473-0ee17519493c service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:14.094353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f830e-e5c5-4ffb-913b-4b0a28029407 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:14.110995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedbcdf-0cd4-4cb1-af8d-2d7d44727009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:14.112864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedbcdf-0cd4-4cb1-af8d-2d7d44727009 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:14.113175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedbcdf-0cd4-4cb1-af8d-2d7d44727009 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:14.113445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedbcdf-0cd4-4cb1-af8d-2d7d44727009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:14.122693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7dedbcdf-0cd4-4cb1-af8d-2d7d44727009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:14.139280 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3731/7493] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:14.151950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dedbcdf-0cd4-4cb1-af8d-2d7d44727009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:14.153710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dedbcdf-0cd4-4cb1-af8d-2d7d44727009 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:14.196768 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3763/7494] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:24.115997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=32 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:24.168576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=31 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:24.654849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9716b-5896-4913-8473-0ee17519493c service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:24.733982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9716b-5896-4913-8473-0ee17519493c service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:24.738801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:24.739554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:24.739906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:24.740264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] PATH_INFO: `/v3/users/de49589624e54ea8b12a5230d2a148c6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:24.741871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de49589624e54ea8b12a5230d2a148c6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:24.744674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:24.816133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de49589624e54ea8b12a5230d2a148c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:24.833028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de49589624e54ea8b12a5230d2a148c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:24.842365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de49589624e54ea8b12a5230d2a148c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:24.851225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de49589624e54ea8b12a5230d2a148c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:24.851929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de49589624e54ea8b12a5230d2a148c6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:24.859270 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3732/7495] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:24 2022] DELETE /identity/v3/users/de49589624e54ea8b12a5230d2a148c6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:24.863704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1b6c7-915e-4cd9-a064-13cf7c30992c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:24.863987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1b6c7-915e-4cd9-a064-13cf7c30992c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:24.864260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1b6c7-915e-4cd9-a064-13cf7c30992c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:24.885869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-76c1b6c7-915e-4cd9-a064-13cf7c30992c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:24.888921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76c1b6c7-915e-4cd9-a064-13cf7c30992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:24.960519 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3764/7496] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:24.972633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:24.974182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:25.053593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dfef5-e42a-4040-90df-95b75a30e0d3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:25.060132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff013344-8388-4e3c-9e00-5fc4b0faa5da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:25.060863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff013344-8388-4e3c-9e00-5fc4b0faa5da service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:25.061187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff013344-8388-4e3c-9e00-5fc4b0faa5da service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:25.061477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff013344-8388-4e3c-9e00-5fc4b0faa5da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:25.073203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff013344-8388-4e3c-9e00-5fc4b0faa5da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:25.075130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff013344-8388-4e3c-9e00-5fc4b0faa5da service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:25.086065 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3733/7497] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:25.338718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1b6c7-915e-4cd9-a064-13cf7c30992c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:25.341580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76c1b6c7-915e-4cd9-a064-13cf7c30992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:25.358547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1b6c7-915e-4cd9-a064-13cf7c30992c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:25.363511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:25.364279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:25.364663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:25.365053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] PATH_INFO: `/v3/projects/d4de7541478e491198d31312bc5dd750` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:25.366346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4de7541478e491198d31312bc5dd750)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:25.368419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:25.421149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4de7541478e491198d31312bc5dd750', 'request_id': 'req-b5f1e7b2-df45-4c71-8eca-e00c581047d1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:25.428991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4de7541478e491198d31312bc5dd750', 'request_id': 'req-b5f1e7b2-df45-4c71-8eca-e00c581047d1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:25.436542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4de7541478e491198d31312bc5dd750', 'request_id': 'req-b5f1e7b2-df45-4c71-8eca-e00c581047d1'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:25.447794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] The token cache is being invalidate because project d4de7541478e491198d31312bc5dd750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:25.448187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:25.448916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b5f1e7b2-df45-4c71-8eca-e00c581047d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:25.450508 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3765/7498] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:25 2022] DELETE /identity/v3/projects/d4de7541478e491198d31312bc5dd750 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:25.455412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac5423-f09b-4f56-b0e9-0f571927a841 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:25.456289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac5423-f09b-4f56-b0e9-0f571927a841 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:25.456563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac5423-f09b-4f56-b0e9-0f571927a841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:25.478783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bbac5423-f09b-4f56-b0e9-0f571927a841 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:25.480166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bbac5423-f09b-4f56-b0e9-0f571927a841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:25.498669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:25.552739 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3734/7499] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:25.560888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce061b-75af-4f98-87a1-b57ce3f06fbc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:25.561419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce061b-75af-4f98-87a1-b57ce3f06fbc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:25.561696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce061b-75af-4f98-87a1-b57ce3f06fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:25.584702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-34ce061b-75af-4f98-87a1-b57ce3f06fbc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:25.586654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-34ce061b-75af-4f98-87a1-b57ce3f06fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:25.598292 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3766/7500] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:25.603306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac5423-f09b-4f56-b0e9-0f571927a841 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:25.616598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac5423-f09b-4f56-b0e9-0f571927a841 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:25.624111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6b21c-18f6-481f-8c74-61693af26a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:25.625097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6b21c-18f6-481f-8c74-61693af26a1e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:25.625331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6b21c-18f6-481f-8c74-61693af26a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:25.625558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6b21c-18f6-481f-8c74-61693af26a1e admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:25.626200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6b21c-18f6-481f-8c74-61693af26a1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:20:25.626643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6b21c-18f6-481f-8c74-61693af26a1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:25.628245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6b21c-18f6-481f-8c74-61693af26a1e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:25.645174 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3735/7501] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:20:25.649921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce061b-75af-4f98-87a1-b57ce3f06fbc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:25.663190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce061b-75af-4f98-87a1-b57ce3f06fbc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:25.670189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c60de-4481-4f9e-9920-2c893653f2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:25.671228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c60de-4481-4f9e-9920-2c893653f2bf admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:25.671583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c60de-4481-4f9e-9920-2c893653f2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:25.671906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c60de-4481-4f9e-9920-2c893653f2bf admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:25.672757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c60de-4481-4f9e-9920-2c893653f2bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:25.674974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c60de-4481-4f9e-9920-2c893653f2bf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:25.726639 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3767/7502] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:25 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:25.728610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6b21c-18f6-481f-8c74-61693af26a1e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:25.745759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6b21c-18f6-481f-8c74-61693af26a1e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:25.826438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b2b27-4f66-42b8-8e31-e08e542760a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:25.827574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b2b27-4f66-42b8-8e31-e08e542760a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:25.827963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b2b27-4f66-42b8-8e31-e08e542760a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:25.828411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b2b27-4f66-42b8-8e31-e08e542760a6 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:25.829028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b2b27-4f66-42b8-8e31-e08e542760a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:25.831896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b2b27-4f66-42b8-8e31-e08e542760a6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:25.864795 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3736/7503] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:25 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:25.868853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c60de-4481-4f9e-9920-2c893653f2bf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:25.887811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c60de-4481-4f9e-9920-2c893653f2bf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:25.892733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11587122-ba41-4885-984e-d90ef1ff7059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:25.893636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11587122-ba41-4885-984e-d90ef1ff7059 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:25.894091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11587122-ba41-4885-984e-d90ef1ff7059 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:25.894513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11587122-ba41-4885-984e-d90ef1ff7059 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:25.895341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11587122-ba41-4885-984e-d90ef1ff7059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:25.896940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11587122-ba41-4885-984e-d90ef1ff7059 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:25.912017 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3768/7504] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:25.913332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b2b27-4f66-42b8-8e31-e08e542760a6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:25.927274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b2b27-4f66-42b8-8e31-e08e542760a6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:25.935961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe28ad-b87b-4171-bf50-04eb23e9c83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:25.937204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe28ad-b87b-4171-bf50-04eb23e9c83a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:25.937509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe28ad-b87b-4171-bf50-04eb23e9c83a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:25.937852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe28ad-b87b-4171-bf50-04eb23e9c83a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:25.938373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe28ad-b87b-4171-bf50-04eb23e9c83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:25.940090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe28ad-b87b-4171-bf50-04eb23e9c83a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:25.953290 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3737/7505] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:25.957806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11587122-ba41-4885-984e-d90ef1ff7059 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:25.970881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11587122-ba41-4885-984e-d90ef1ff7059 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:25.976044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6e984-aa22-4d6e-a3fe-6496cc8870a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:25.976709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6e984-aa22-4d6e-a3fe-6496cc8870a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:25.976986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6e984-aa22-4d6e-a3fe-6496cc8870a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:25.977298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6e984-aa22-4d6e-a3fe-6496cc8870a0 admin admin] PATH_INFO: `/v3/projects/6a8523017c7c4189b23d74aacbdb1e40/users/8bd0459c44534ab9bbea5407c2c31dde/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:25.996812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6e984-aa22-4d6e-a3fe-6496cc8870a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8523017c7c4189b23d74aacbdb1e40, user_id=8bd0459c44534ab9bbea5407c2c31dde, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:26.000980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6e984-aa22-4d6e-a3fe-6496cc8870a0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:26.004481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe28ad-b87b-4171-bf50-04eb23e9c83a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:26.006336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4cbe28ad-b87b-4171-bf50-04eb23e9c83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.021931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe28ad-b87b-4171-bf50-04eb23e9c83a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:26.027309 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3769/7506] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:25 2022] PUT /identity/v3/projects/6a8523017c7c4189b23d74aacbdb1e40/users/8bd0459c44534ab9bbea5407c2c31dde/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:26.038593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c55eda-021d-43ad-83aa-d3315f082dd1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.038889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c55eda-021d-43ad-83aa-d3315f082dd1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.039164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c55eda-021d-43ad-83aa-d3315f082dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.095869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-03c55eda-021d-43ad-83aa-d3315f082dd1 None None] MFA Rules not processed for user `8bd0459c44534ab9bbea5407c2c31dde`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:26.097151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-03c55eda-021d-43ad-83aa-d3315f082dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.103674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:26.104339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.104612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.104880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] PATH_INFO: `/v3/users/0139d1342072434b9f904a8f4ea385fd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.106111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0139d1342072434b9f904a8f4ea385fd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:26.106726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:26.108353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:26.160397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:26.176214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3770/7507] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:26.179302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0139d1342072434b9f904a8f4ea385fd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.187075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c55eda-021d-43ad-83aa-d3315f082dd1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:26.188808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-03c55eda-021d-43ad-83aa-d3315f082dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.192900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0139d1342072434b9f904a8f4ea385fd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.204291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0139d1342072434b9f904a8f4ea385fd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.212717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0139d1342072434b9f904a8f4ea385fd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.213602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-32a66cf6-6966-4f83-a4ef-043d4938a831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0139d1342072434b9f904a8f4ea385fd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.220117 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3738/7508] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:26 2022] DELETE /identity/v3/users/0139d1342072434b9f904a8f4ea385fd => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:26.224417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa057a-b293-4b37-8c2e-3bdfbaea657a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.224696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa057a-b293-4b37-8c2e-3bdfbaea657a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.224965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa057a-b293-4b37-8c2e-3bdfbaea657a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.245191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-14aa057a-b293-4b37-8c2e-3bdfbaea657a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:26.246377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-14aa057a-b293-4b37-8c2e-3bdfbaea657a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.254323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c55eda-021d-43ad-83aa-d3315f082dd1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:26.305298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873262d-6edc-4729-8c77-4a2e082abd94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:26.306355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873262d-6edc-4729-8c77-4a2e082abd94 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.306629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873262d-6edc-4729-8c77-4a2e082abd94 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.306896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873262d-6edc-4729-8c77-4a2e082abd94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.308613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b873262d-6edc-4729-8c77-4a2e082abd94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.309563 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3739/7509] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:26.319084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa057a-b293-4b37-8c2e-3bdfbaea657a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:26.320392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-14aa057a-b293-4b37-8c2e-3bdfbaea657a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.334827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa057a-b293-4b37-8c2e-3bdfbaea657a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:26.339937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cc5d1-ff74-4070-9883-311e8e077422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:26.340718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cc5d1-ff74-4070-9883-311e8e077422 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.340891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cc5d1-ff74-4070-9883-311e8e077422 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.341215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cc5d1-ff74-4070-9883-311e8e077422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.350921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cc5d1-ff74-4070-9883-311e8e077422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:26.352355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cc5d1-ff74-4070-9883-311e8e077422 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:26.362888 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3740/7510] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:20:26.370983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873262d-6edc-4729-8c77-4a2e082abd94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:26.372418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873262d-6edc-4729-8c77-4a2e082abd94 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:26.382211 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3771/7511] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:20:26.396996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cc5d1-ff74-4070-9883-311e8e077422 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:26.398448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-087cc5d1-ff74-4070-9883-311e8e077422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.412233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cc5d1-ff74-4070-9883-311e8e077422 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:26.417816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:26.418431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.418699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.418961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] PATH_INFO: `/v3/users/8bd0459c44534ab9bbea5407c2c31dde` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.420141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bd0459c44534ab9bbea5407c2c31dde)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:26.422091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:26.473204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bd0459c44534ab9bbea5407c2c31dde'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.485925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bd0459c44534ab9bbea5407c2c31dde'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.493841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bd0459c44534ab9bbea5407c2c31dde'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.500998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bd0459c44534ab9bbea5407c2c31dde'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.501696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bd0459c44534ab9bbea5407c2c31dde'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.507927 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3741/7512] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:26 2022] DELETE /identity/v3/users/8bd0459c44534ab9bbea5407c2c31dde => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:26.512992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ff810-3d55-4934-8ac9-62b1dbce9e89 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.513305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ff810-3d55-4934-8ac9-62b1dbce9e89 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.513576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ff810-3d55-4934-8ac9-62b1dbce9e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.533096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-392ff810-3d55-4934-8ac9-62b1dbce9e89 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:26.534262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-392ff810-3d55-4934-8ac9-62b1dbce9e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.547918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:26.561841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:26.563369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.574892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:26.606692 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3772/7513] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:26.617562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ff810-3d55-4934-8ac9-62b1dbce9e89 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:26.619120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-392ff810-3d55-4934-8ac9-62b1dbce9e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.644646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11123d5c-945f-4dc5-b59a-3a6c62146c45 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:26.651056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:26.651801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.652078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.652346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] PATH_INFO: `/v3/projects/256a344e0b4b4170a56df3ca5b5b1655` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.653496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=256a344e0b4b4170a56df3ca5b5b1655)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:26.655399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:26.679071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ff810-3d55-4934-8ac9-62b1dbce9e89 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:26.683673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca3984-c47f-4441-9ec0-3600c88de3cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:26.684326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca3984-c47f-4441-9ec0-3600c88de3cc service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.684641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca3984-c47f-4441-9ec0-3600c88de3cc service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.684973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca3984-c47f-4441-9ec0-3600c88de3cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.695509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca3984-c47f-4441-9ec0-3600c88de3cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:26.697487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca3984-c47f-4441-9ec0-3600c88de3cc service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:26.702702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '256a344e0b4b4170a56df3ca5b5b1655', 'request_id': 'req-04795443-b2e7-4a62-aba4-0980eae6e0bd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.707294 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3773/7514] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:20:26.710672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '256a344e0b4b4170a56df3ca5b5b1655', 'request_id': 'req-04795443-b2e7-4a62-aba4-0980eae6e0bd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.719013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '256a344e0b4b4170a56df3ca5b5b1655', 'request_id': 'req-04795443-b2e7-4a62-aba4-0980eae6e0bd'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.730307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] The token cache is being invalidate because project 256a344e0b4b4170a56df3ca5b5b1655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:26.730589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.731258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04795443-b2e7-4a62-aba4-0980eae6e0bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:26.733088 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3742/7515] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:26 2022] DELETE /identity/v3/projects/256a344e0b4b4170a56df3ca5b5b1655 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:26.738964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2cc34-7ebf-403e-a736-d18edfb3fa1f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.739333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2cc34-7ebf-403e-a736-d18edfb3fa1f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.739675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2cc34-7ebf-403e-a736-d18edfb3fa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.760211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8de2cc34-7ebf-403e-a736-d18edfb3fa1f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:26.761441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8de2cc34-7ebf-403e-a736-d18edfb3fa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.826527 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3774/7516] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:26.835010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd808f-24ee-453e-8519-44a01b8fa1d8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.835400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd808f-24ee-453e-8519-44a01b8fa1d8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.835519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd808f-24ee-453e-8519-44a01b8fa1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.857678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-44bd808f-24ee-453e-8519-44a01b8fa1d8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:26.858981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-44bd808f-24ee-453e-8519-44a01b8fa1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:26.868783 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3743/7517] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:26.873270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2cc34-7ebf-403e-a736-d18edfb3fa1f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:26.884602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2cc34-7ebf-403e-a736-d18edfb3fa1f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:26.889783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b33061-4216-44a3-a17e-604ce314c25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:26.890392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b33061-4216-44a3-a17e-604ce314c25c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.890676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b33061-4216-44a3-a17e-604ce314c25c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.890943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b33061-4216-44a3-a17e-604ce314c25c admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.891585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b33061-4216-44a3-a17e-604ce314c25c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:20:26.891886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b33061-4216-44a3-a17e-604ce314c25c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:26.893559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b33061-4216-44a3-a17e-604ce314c25c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:26.906321 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3775/7518] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:20:26.910878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd808f-24ee-453e-8519-44a01b8fa1d8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:26.921187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd808f-24ee-453e-8519-44a01b8fa1d8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:26.926782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7d16-2474-4f82-a180-a70fa3243a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:26.927401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7d16-2474-4f82-a180-a70fa3243a2c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:26.928173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7d16-2474-4f82-a180-a70fa3243a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:26.928449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7d16-2474-4f82-a180-a70fa3243a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:26.929102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a7d16-2474-4f82-a180-a70fa3243a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:26.931513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a7d16-2474-4f82-a180-a70fa3243a2c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:26.964517 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3744/7519] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:26 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:26.965834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b33061-4216-44a3-a17e-604ce314c25c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:26.976385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b33061-4216-44a3-a17e-604ce314c25c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:26.999658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7d16-2474-4f82-a180-a70fa3243a2c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.001493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-999a7d16-2474-4f82-a180-a70fa3243a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:27.038422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:27.051865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa42b3d-f394-4056-8305-005bec59c1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.052500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42b3d-f394-4056-8305-005bec59c1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.052775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42b3d-f394-4056-8305-005bec59c1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.053063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42b3d-f394-4056-8305-005bec59c1f3 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.053684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa42b3d-f394-4056-8305-005bec59c1f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.056130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa42b3d-f394-4056-8305-005bec59c1f3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.077425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7d16-2474-4f82-a180-a70fa3243a2c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.080742 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3776/7520] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:26 2022] POST /identity/v3/users => generated 303 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:27.082663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.083312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.083582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.083848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] PATH_INFO: `/v3/projects/6a8523017c7c4189b23d74aacbdb1e40` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.085683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa42b3d-f394-4056-8305-005bec59c1f3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.086396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8523017c7c4189b23d74aacbdb1e40)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.088922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.097197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa42b3d-f394-4056-8305-005bec59c1f3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.100268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:27.102037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6bacb-48c4-47c5-99d0-1ffc1d02c43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.103248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6bacb-48c4-47c5-99d0-1ffc1d02c43c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.103792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6bacb-48c4-47c5-99d0-1ffc1d02c43c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.104066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6bacb-48c4-47c5-99d0-1ffc1d02c43c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.104707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6bacb-48c4-47c5-99d0-1ffc1d02c43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.106591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6bacb-48c4-47c5-99d0-1ffc1d02c43c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.118295 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3777/7521] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:27.122785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6bacb-48c4-47c5-99d0-1ffc1d02c43c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.132804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6bacb-48c4-47c5-99d0-1ffc1d02c43c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.143980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8523017c7c4189b23d74aacbdb1e40', 'request_id': 'req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:27.152428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8523017c7c4189b23d74aacbdb1e40', 'request_id': 'req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:27.160393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8523017c7c4189b23d74aacbdb1e40', 'request_id': 'req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:27.174096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] The token cache is being invalidate because project 6a8523017c7c4189b23d74aacbdb1e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:27.174394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:27.175077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-53c75a89-c749-4982-9ebe-1b8e6a0bf42f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:27.176732 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3745/7522] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:26 2022] DELETE /identity/v3/projects/6a8523017c7c4189b23d74aacbdb1e40 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:27.181843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cb8a6-b0b6-4f6c-b7a0-c3500dda5078 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.182349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cb8a6-b0b6-4f6c-b7a0-c3500dda5078 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.182802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cb8a6-b0b6-4f6c-b7a0-c3500dda5078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.210735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7b3cb8a6-b0b6-4f6c-b7a0-c3500dda5078 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:27.212786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7b3cb8a6-b0b6-4f6c-b7a0-c3500dda5078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:27.220585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1cd7f-86b6-420e-92d5-49f63822273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.221232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1cd7f-86b6-420e-92d5-49f63822273d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.221506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1cd7f-86b6-420e-92d5-49f63822273d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.221766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1cd7f-86b6-420e-92d5-49f63822273d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.222393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1cd7f-86b6-420e-92d5-49f63822273d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.223964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1cd7f-86b6-420e-92d5-49f63822273d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.227220 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3746/7523] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:27.236204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1883601-5f8b-4b58-9309-c6ae4432f211 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.236480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1883601-5f8b-4b58-9309-c6ae4432f211 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.236749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1883601-5f8b-4b58-9309-c6ae4432f211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.240637 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3778/7524] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:20:27.241541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1cd7f-86b6-420e-92d5-49f63822273d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.243387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ffa1cd7f-86b6-420e-92d5-49f63822273d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:27.258473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1cd7f-86b6-420e-92d5-49f63822273d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.261224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e1883601-5f8b-4b58-9309-c6ae4432f211 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:27.262487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e1883601-5f8b-4b58-9309-c6ae4432f211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:27.265611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec8106-3596-44c1-9aa1-0d94c844ca45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.266756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec8106-3596-44c1-9aa1-0d94c844ca45 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.267034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec8106-3596-44c1-9aa1-0d94c844ca45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.267306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec8106-3596-44c1-9aa1-0d94c844ca45 admin admin] PATH_INFO: `/v3/projects/e77e816ccacf479093fcefa4c3c60ab3/users/e44eefd41f7d4a8e920b3da907adf013/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.277585 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3747/7525] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:27.278708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1883601-5f8b-4b58-9309-c6ae4432f211 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.285676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec8106-3596-44c1-9aa1-0d94c844ca45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77e816ccacf479093fcefa4c3c60ab3, user_id=e44eefd41f7d4a8e920b3da907adf013, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.288614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec8106-3596-44c1-9aa1-0d94c844ca45 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.290547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1883601-5f8b-4b58-9309-c6ae4432f211 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.296740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ec2cc-19bd-4b88-8cac-2dc18ce9f494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.297460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ec2cc-19bd-4b88-8cac-2dc18ce9f494 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.297926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ec2cc-19bd-4b88-8cac-2dc18ce9f494 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.298267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ec2cc-19bd-4b88-8cac-2dc18ce9f494 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.298970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ec2cc-19bd-4b88-8cac-2dc18ce9f494 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:20:27.299264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ec2cc-19bd-4b88-8cac-2dc18ce9f494 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.301003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ec2cc-19bd-4b88-8cac-2dc18ce9f494 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.313304 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3779/7526] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:27 2022] PUT /identity/v3/projects/e77e816ccacf479093fcefa4c3c60ab3/users/e44eefd41f7d4a8e920b3da907adf013/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:27.317600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae4dab-05df-4927-8fc5-26d77b77a76b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.317875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae4dab-05df-4927-8fc5-26d77b77a76b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.318236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae4dab-05df-4927-8fc5-26d77b77a76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.326330 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3748/7527] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:20:27.330671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ec2cc-19bd-4b88-8cac-2dc18ce9f494 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.347777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ec2cc-19bd-4b88-8cac-2dc18ce9f494 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.349822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-eeae4dab-05df-4927-8fc5-26d77b77a76b None None] MFA Rules not processed for user `e44eefd41f7d4a8e920b3da907adf013`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:27.351121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eeae4dab-05df-4927-8fc5-26d77b77a76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:27.407826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b61df-fdc6-411e-a32a-5b6406b38fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.408453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b61df-fdc6-411e-a32a-5b6406b38fcc admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.408724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b61df-fdc6-411e-a32a-5b6406b38fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.408984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b61df-fdc6-411e-a32a-5b6406b38fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.409685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b61df-fdc6-411e-a32a-5b6406b38fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.411758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b61df-fdc6-411e-a32a-5b6406b38fcc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.418145 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3780/7528] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:27 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:27.422554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae4dab-05df-4927-8fc5-26d77b77a76b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.434218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae4dab-05df-4927-8fc5-26d77b77a76b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.438822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f54aaf-d9c4-40bb-a565-3571b415a968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.439409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f54aaf-d9c4-40bb-a565-3571b415a968 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.439680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f54aaf-d9c4-40bb-a565-3571b415a968 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.439943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f54aaf-d9c4-40bb-a565-3571b415a968 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.440324 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3749/7529] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:27 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:27.440556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f54aaf-d9c4-40bb-a565-3571b415a968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.442453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f54aaf-d9c4-40bb-a565-3571b415a968 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.444401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b61df-fdc6-411e-a32a-5b6406b38fcc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.473101 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3781/7530] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:27 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:27.474201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f54aaf-d9c4-40bb-a565-3571b415a968 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.485120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:27.506011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:27.510594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b61df-fdc6-411e-a32a-5b6406b38fcc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.544082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f54aaf-d9c4-40bb-a565-3571b415a968 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.549315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ff3f3-787c-4b8b-b3d6-d243c8092e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.550431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ff3f3-787c-4b8b-b3d6-d243c8092e74 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.550713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ff3f3-787c-4b8b-b3d6-d243c8092e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.550985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ff3f3-787c-4b8b-b3d6-d243c8092e74 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.551638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ff3f3-787c-4b8b-b3d6-d243c8092e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.553585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ff3f3-787c-4b8b-b3d6-d243c8092e74 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.574756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3403a4-022c-4475-91b2-6d2f3c088ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.575384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3403a4-022c-4475-91b2-6d2f3c088ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.575656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3403a4-022c-4475-91b2-6d2f3c088ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.575918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3403a4-022c-4475-91b2-6d2f3c088ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.576528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3403a4-022c-4475-91b2-6d2f3c088ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.579816 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3782/7531] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:27 2022] POST /identity/v3/users => generated 309 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:27.579957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3403a4-022c-4475-91b2-6d2f3c088ba6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.581365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ff3f3-787c-4b8b-b3d6-d243c8092e74 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.593607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ff3f3-787c-4b8b-b3d6-d243c8092e74 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.599536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d327a78-9ca3-4bdb-94bf-2962d4111145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.600102 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3750/7532] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:27 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:27.600410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d327a78-9ca3-4bdb-94bf-2962d4111145 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.600741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d327a78-9ca3-4bdb-94bf-2962d4111145 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.601197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d327a78-9ca3-4bdb-94bf-2962d4111145 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.601883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d327a78-9ca3-4bdb-94bf-2962d4111145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.607934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d327a78-9ca3-4bdb-94bf-2962d4111145 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.609473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3403a4-022c-4475-91b2-6d2f3c088ba6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.619240 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3783/7533] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:27.622716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3403a4-022c-4475-91b2-6d2f3c088ba6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.623373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d327a78-9ca3-4bdb-94bf-2962d4111145 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.627582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f6edd-ee92-4fa4-b978-5235f5f3da8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.628176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f6edd-ee92-4fa4-b978-5235f5f3da8f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.628445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f6edd-ee92-4fa4-b978-5235f5f3da8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.628708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f6edd-ee92-4fa4-b978-5235f5f3da8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.629615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f6edd-ee92-4fa4-b978-5235f5f3da8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.631198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f6edd-ee92-4fa4-b978-5235f5f3da8f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.633494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d327a78-9ca3-4bdb-94bf-2962d4111145 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.638758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba175-8da6-433d-9cbe-31a00ead2da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.639388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba175-8da6-433d-9cbe-31a00ead2da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.639659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba175-8da6-433d-9cbe-31a00ead2da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.639926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba175-8da6-433d-9cbe-31a00ead2da0 admin admin] PATH_INFO: `/v3/projects/6532aeba7eca4d6084a5a57c01964222/users/ba50b4761f744e94b6e219ffa45481ab/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.642492 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3751/7534] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:27.646486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f6edd-ee92-4fa4-b978-5235f5f3da8f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.655178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ba175-8da6-433d-9cbe-31a00ead2da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6532aeba7eca4d6084a5a57c01964222, user_id=ba50b4761f744e94b6e219ffa45481ab, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.656451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f6edd-ee92-4fa4-b978-5235f5f3da8f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.657878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ba175-8da6-433d-9cbe-31a00ead2da0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.661595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc83be-d594-4b15-ba22-53ef8754116b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.662330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc83be-d594-4b15-ba22-53ef8754116b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.662655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc83be-d594-4b15-ba22-53ef8754116b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.663155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc83be-d594-4b15-ba22-53ef8754116b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.664056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc83be-d594-4b15-ba22-53ef8754116b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.665815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc83be-d594-4b15-ba22-53ef8754116b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.672289 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3784/7535] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:27 2022] PUT /identity/v3/projects/6532aeba7eca4d6084a5a57c01964222/users/ba50b4761f744e94b6e219ffa45481ab/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:27.676229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba175-8da6-433d-9cbe-31a00ead2da0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.682301 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3752/7536] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:20:27.683242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc83be-d594-4b15-ba22-53ef8754116b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.693001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc83be-d594-4b15-ba22-53ef8754116b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.740192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba175-8da6-433d-9cbe-31a00ead2da0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.746054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ffdb2-8408-4e7d-94b7-0459d25a16dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.746650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ffdb2-8408-4e7d-94b7-0459d25a16dc admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.746923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ffdb2-8408-4e7d-94b7-0459d25a16dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.747187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ffdb2-8408-4e7d-94b7-0459d25a16dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.747815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ffdb2-8408-4e7d-94b7-0459d25a16dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.749573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ffdb2-8408-4e7d-94b7-0459d25a16dc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.750820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6016e-df68-4589-9d08-9ebbe000dc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.751333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6016e-df68-4589-9d08-9ebbe000dc7e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.751616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6016e-df68-4589-9d08-9ebbe000dc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.751910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6016e-df68-4589-9d08-9ebbe000dc7e admin admin] PATH_INFO: `/v3/projects/7bf097b0f8fe47f486b68da7e7552c69/users/b4c98ad011ab494e9c9c8a14ec5a961a/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.760465 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3785/7537] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:20:27.764305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ffdb2-8408-4e7d-94b7-0459d25a16dc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.767996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6016e-df68-4589-9d08-9ebbe000dc7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf097b0f8fe47f486b68da7e7552c69, user_id=b4c98ad011ab494e9c9c8a14ec5a961a, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.770711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6016e-df68-4589-9d08-9ebbe000dc7e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.774235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ffdb2-8408-4e7d-94b7-0459d25a16dc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.779494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f57af-20ad-492a-80d4-951b7334faf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.780138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f57af-20ad-492a-80d4-951b7334faf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.780413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f57af-20ad-492a-80d4-951b7334faf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.780682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f57af-20ad-492a-80d4-951b7334faf5 admin admin] PATH_INFO: `/v3/projects/6532aeba7eca4d6084a5a57c01964222/users/ba50b4761f744e94b6e219ffa45481ab/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.782663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f57af-20ad-492a-80d4-951b7334faf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6532aeba7eca4d6084a5a57c01964222, user_id=ba50b4761f744e94b6e219ffa45481ab, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.783973 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3753/7538] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:27 2022] PUT /identity/v3/projects/7bf097b0f8fe47f486b68da7e7552c69/users/b4c98ad011ab494e9c9c8a14ec5a961a/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:27.785318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f57af-20ad-492a-80d4-951b7334faf5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.794497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885386b2-78ba-47e9-b954-cf72254d7dae None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.794781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885386b2-78ba-47e9-b954-cf72254d7dae None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.795046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885386b2-78ba-47e9-b954-cf72254d7dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.795817 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3786/7539] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:27 2022] PUT /identity/v3/projects/6532aeba7eca4d6084a5a57c01964222/users/ba50b4761f744e94b6e219ffa45481ab/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:27.806565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc432cf-ecdf-4b33-979e-cc5dc3d8d4b3 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.806990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc432cf-ecdf-4b33-979e-cc5dc3d8d4b3 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.807269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc432cf-ecdf-4b33-979e-cc5dc3d8d4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.827179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-885386b2-78ba-47e9-b954-cf72254d7dae None None] MFA Rules not processed for user `b4c98ad011ab494e9c9c8a14ec5a961a`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:27.828630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-885386b2-78ba-47e9-b954-cf72254d7dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:27.836703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4dc432cf-ecdf-4b33-979e-cc5dc3d8d4b3 None None] MFA Rules not processed for user `ba50b4761f744e94b6e219ffa45481ab`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:27.838008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4dc432cf-ecdf-4b33-979e-cc5dc3d8d4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:27.872125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:27.874539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:27.897683 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3754/7540] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:27.909146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885386b2-78ba-47e9-b954-cf72254d7dae None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.910600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-885386b2-78ba-47e9-b954-cf72254d7dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:27.911437 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3787/7541] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:27 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:27.922824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc432cf-ecdf-4b33-979e-cc5dc3d8d4b3 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:27.924177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4dc432cf-ecdf-4b33-979e-cc5dc3d8d4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:27.971834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885386b2-78ba-47e9-b954-cf72254d7dae None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.975498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8ce45-faf4-4953-869b-8b2834187936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.976086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8ce45-faf4-4953-869b-8b2834187936 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.976355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8ce45-faf4-4953-869b-8b2834187936 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.976615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8ce45-faf4-4953-869b-8b2834187936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.984850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc432cf-ecdf-4b33-979e-cc5dc3d8d4b3 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:27.985297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8ce45-faf4-4953-869b-8b2834187936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:27.986714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8ce45-faf4-4953-869b-8b2834187936 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:27.988545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aef2b6-3f9a-4058-bde5-d412708376c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:27.989174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aef2b6-3f9a-4058-bde5-d412708376c2 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:27.989453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aef2b6-3f9a-4058-bde5-d412708376c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:27.989735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aef2b6-3f9a-4058-bde5-d412708376c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:27.995552 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3755/7542] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:20:27.999317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aef2b6-3f9a-4058-bde5-d412708376c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:28.000742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aef2b6-3f9a-4058-bde5-d412708376c2 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:28.012943 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3788/7543] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:27 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:20:28.178184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8ce45-faf4-4953-869b-8b2834187936 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:28.271839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8ce45-faf4-4953-869b-8b2834187936 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:28.277549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:28.278340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:28.278660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:28.278932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] PATH_INFO: `/v3/users/e44eefd41f7d4a8e920b3da907adf013` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:28.280264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e44eefd41f7d4a8e920b3da907adf013)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:28.282257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:28.348626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e44eefd41f7d4a8e920b3da907adf013'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:28.370242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e44eefd41f7d4a8e920b3da907adf013'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:28.387060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e44eefd41f7d4a8e920b3da907adf013'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:28.398215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e44eefd41f7d4a8e920b3da907adf013'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:28.398951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e44eefd41f7d4a8e920b3da907adf013'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:28.405411 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3756/7544] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:28 2022] DELETE /identity/v3/users/e44eefd41f7d4a8e920b3da907adf013 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:28.409973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fbeed-921f-4572-8102-2128d3cd1cf2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:28.410320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fbeed-921f-4572-8102-2128d3cd1cf2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:28.410661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fbeed-921f-4572-8102-2128d3cd1cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:28.442159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c59fbeed-921f-4572-8102-2128d3cd1cf2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:28.443778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c59fbeed-921f-4572-8102-2128d3cd1cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:28.517604 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3789/7545] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:28.528459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:28.529959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:28.540845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:28.599637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0495b49-bc5d-460a-91d6-488de45d8b7b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:28.603852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63503e-16eb-4928-bdb5-b99026728c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:28.604473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63503e-16eb-4928-bdb5-b99026728c1e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:28.604747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63503e-16eb-4928-bdb5-b99026728c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:28.605011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63503e-16eb-4928-bdb5-b99026728c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:28.618560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63503e-16eb-4928-bdb5-b99026728c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:28.620050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63503e-16eb-4928-bdb5-b99026728c1e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:28.630713 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3757/7546] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:28.938975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fbeed-921f-4572-8102-2128d3cd1cf2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:28.941065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c59fbeed-921f-4572-8102-2128d3cd1cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:28.960661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fbeed-921f-4572-8102-2128d3cd1cf2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:28.973555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:28.974269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:28.974553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:28.974822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] PATH_INFO: `/v3/projects/e77e816ccacf479093fcefa4c3c60ab3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:28.976308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e77e816ccacf479093fcefa4c3c60ab3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:28.979065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:29.037641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e77e816ccacf479093fcefa4c3c60ab3', 'request_id': 'req-33c5236c-6380-42fc-9648-b047fc52a7d8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.048782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e77e816ccacf479093fcefa4c3c60ab3', 'request_id': 'req-33c5236c-6380-42fc-9648-b047fc52a7d8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.062605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e77e816ccacf479093fcefa4c3c60ab3', 'request_id': 'req-33c5236c-6380-42fc-9648-b047fc52a7d8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.071631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:29.087650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] The token cache is being invalidate because project e77e816ccacf479093fcefa4c3c60ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:29.087650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.088818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.094548 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3790/7547] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:28 2022] DELETE /identity/v3/projects/e77e816ccacf479093fcefa4c3c60ab3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:29.099494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63503e-16eb-4928-bdb5-b99026728c1e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:29.111943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63503e-16eb-4928-bdb5-b99026728c1e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:29.195949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:29.196692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:29.197013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:29.197316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] PATH_INFO: `/v3/users/ba50b4761f744e94b6e219ffa45481ab` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:29.198510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba50b4761f744e94b6e219ffa45481ab)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:29.200438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:29.254564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba50b4761f744e94b6e219ffa45481ab'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.272191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba50b4761f744e94b6e219ffa45481ab'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.284378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba50b4761f744e94b6e219ffa45481ab'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.292373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba50b4761f744e94b6e219ffa45481ab'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.293138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f00290cf-8aa7-43ba-b570-2290ff391730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba50b4761f744e94b6e219ffa45481ab'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.304134 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3758/7548] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:29 2022] DELETE /identity/v3/users/ba50b4761f744e94b6e219ffa45481ab => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:29.653859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:29.655621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:29.682956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5236c-6380-42fc-9648-b047fc52a7d8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:29.761877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:29.762641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:29.763007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:29.763513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] PATH_INFO: `/v3/projects/6532aeba7eca4d6084a5a57c01964222` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:29.764811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6532aeba7eca4d6084a5a57c01964222)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:29.770444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:29.869870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6532aeba7eca4d6084a5a57c01964222', 'request_id': 'req-e4003ff0-11a3-4456-bdf9-b696807c7698'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.882686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6532aeba7eca4d6084a5a57c01964222', 'request_id': 'req-e4003ff0-11a3-4456-bdf9-b696807c7698'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.898383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6532aeba7eca4d6084a5a57c01964222', 'request_id': 'req-e4003ff0-11a3-4456-bdf9-b696807c7698'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.924739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] The token cache is being invalidate because project 6532aeba7eca4d6084a5a57c01964222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:29.925021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.927032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4003ff0-11a3-4456-bdf9-b696807c7698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:29.929462 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3791/7549] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:29 2022] DELETE /identity/v3/projects/6532aeba7eca4d6084a5a57c01964222 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:29.936664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04768-7827-4b46-93e0-3f05daa52ec4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:29.937164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04768-7827-4b46-93e0-3f05daa52ec4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:29.937371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04768-7827-4b46-93e0-3f05daa52ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:29.972325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-15f04768-7827-4b46-93e0-3f05daa52ec4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:29.974212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-15f04768-7827-4b46-93e0-3f05daa52ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:30.004995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:30.046424 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3759/7550] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:30.054461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a0814-88a3-47f5-95dd-55e17f0ed404 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:30.054742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a0814-88a3-47f5-95dd-55e17f0ed404 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:30.054992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a0814-88a3-47f5-95dd-55e17f0ed404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:30.080097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0c4a0814-88a3-47f5-95dd-55e17f0ed404 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:30.081274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0c4a0814-88a3-47f5-95dd-55e17f0ed404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:30.090575 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3792/7551] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:30.094776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f04768-7827-4b46-93e0-3f05daa52ec4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:30.112964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f04768-7827-4b46-93e0-3f05daa52ec4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:30.118842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2484b5-8c2d-46bd-9f76-3ea7c91adf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:30.119458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2484b5-8c2d-46bd-9f76-3ea7c91adf39 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:30.120266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2484b5-8c2d-46bd-9f76-3ea7c91adf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:30.120538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2484b5-8c2d-46bd-9f76-3ea7c91adf39 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:30.121295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2484b5-8c2d-46bd-9f76-3ea7c91adf39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:20:30.121591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2484b5-8c2d-46bd-9f76-3ea7c91adf39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:30.123248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2484b5-8c2d-46bd-9f76-3ea7c91adf39 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:30.142940 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3760/7552] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:20:30.147262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a0814-88a3-47f5-95dd-55e17f0ed404 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:30.159760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a0814-88a3-47f5-95dd-55e17f0ed404 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:30.167626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bde15d-bda2-4edb-bb6b-bf519db1664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:30.168971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bde15d-bda2-4edb-bb6b-bf519db1664b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:30.169407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bde15d-bda2-4edb-bb6b-bf519db1664b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:30.169781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bde15d-bda2-4edb-bb6b-bf519db1664b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:30.170802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bde15d-bda2-4edb-bb6b-bf519db1664b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:30.172772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bde15d-bda2-4edb-bb6b-bf519db1664b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:30.228412 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3793/7553] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:30 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:30.232976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2484b5-8c2d-46bd-9f76-3ea7c91adf39 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:30.245357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2484b5-8c2d-46bd-9f76-3ea7c91adf39 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:30.308835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18357a7c-45b3-4ab1-ab64-f0623e2ff8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:30.309461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18357a7c-45b3-4ab1-ab64-f0623e2ff8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:30.309755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18357a7c-45b3-4ab1-ab64-f0623e2ff8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:30.310140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18357a7c-45b3-4ab1-ab64-f0623e2ff8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:30.310972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18357a7c-45b3-4ab1-ab64-f0623e2ff8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:30.315959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18357a7c-45b3-4ab1-ab64-f0623e2ff8c4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:30.346027 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3761/7554] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:30 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:30.350063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bde15d-bda2-4edb-bb6b-bf519db1664b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:30.363460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bde15d-bda2-4edb-bb6b-bf519db1664b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:30.369240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5358-3089-4353-a2fd-8db18352c8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:30.370992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5358-3089-4353-a2fd-8db18352c8ac admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:30.371316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5358-3089-4353-a2fd-8db18352c8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:30.371565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5358-3089-4353-a2fd-8db18352c8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:30.372876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa5358-3089-4353-a2fd-8db18352c8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:30.374694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa5358-3089-4353-a2fd-8db18352c8ac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:30.386959 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3794/7555] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:30.391271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18357a7c-45b3-4ab1-ab64-f0623e2ff8c4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:30.407016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18357a7c-45b3-4ab1-ab64-f0623e2ff8c4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:30.412605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bdb1b-6dfe-435c-8a78-baae1e357939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:30.413244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bdb1b-6dfe-435c-8a78-baae1e357939 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:30.413573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bdb1b-6dfe-435c-8a78-baae1e357939 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:30.413842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bdb1b-6dfe-435c-8a78-baae1e357939 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:30.414474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bdb1b-6dfe-435c-8a78-baae1e357939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:30.416327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bdb1b-6dfe-435c-8a78-baae1e357939 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:30.429644 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3762/7556] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:30.434619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5358-3089-4353-a2fd-8db18352c8ac admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:30.448431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5358-3089-4353-a2fd-8db18352c8ac admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:30.453420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217b2b0-1994-4fc4-a981-186477202a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:30.455000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217b2b0-1994-4fc4-a981-186477202a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:30.455355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217b2b0-1994-4fc4-a981-186477202a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:30.455723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217b2b0-1994-4fc4-a981-186477202a52 admin admin] PATH_INFO: `/v3/projects/465cd17b7a7f42dc861593a7ea741ab8/users/49c50b923f004ec8b57748595da167ea/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:30.473581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217b2b0-1994-4fc4-a981-186477202a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=465cd17b7a7f42dc861593a7ea741ab8, user_id=49c50b923f004ec8b57748595da167ea, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:30.476380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217b2b0-1994-4fc4-a981-186477202a52 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:30.492528 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3795/7557] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:30 2022] PUT /identity/v3/projects/465cd17b7a7f42dc861593a7ea741ab8/users/49c50b923f004ec8b57748595da167ea/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:30.505673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02dac-cc33-45d2-95c0-a98a368eabd4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:30.505958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02dac-cc33-45d2-95c0-a98a368eabd4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:30.506224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02dac-cc33-45d2-95c0-a98a368eabd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:30.557946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1de02dac-cc33-45d2-95c0-a98a368eabd4 None None] MFA Rules not processed for user `49c50b923f004ec8b57748595da167ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:30.559168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1de02dac-cc33-45d2-95c0-a98a368eabd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:30.673185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217b2b0-1994-4fc4-a981-186477202a52 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:30.675273 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3763/7558] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:30.708243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:30.796235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217b2b0-1994-4fc4-a981-186477202a52 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:30.803179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75ec56-73e6-4bc2-9329-4b54cd2af0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:30.803817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ec56-73e6-4bc2-9329-4b54cd2af0cf admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:30.804094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ec56-73e6-4bc2-9329-4b54cd2af0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:30.804363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ec56-73e6-4bc2-9329-4b54cd2af0cf admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:30.805056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75ec56-73e6-4bc2-9329-4b54cd2af0cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:30.807570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75ec56-73e6-4bc2-9329-4b54cd2af0cf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:30.861637 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3796/7559] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:30 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:30.865772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de02dac-cc33-45d2-95c0-a98a368eabd4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:30.879294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de02dac-cc33-45d2-95c0-a98a368eabd4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:30.929131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:30.992390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecd9a5-873b-4687-9c71-3985f44248ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:30.993350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecd9a5-873b-4687-9c71-3985f44248ce admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:30.993887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecd9a5-873b-4687-9c71-3985f44248ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:30.994313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecd9a5-873b-4687-9c71-3985f44248ce admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:30.995159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecd9a5-873b-4687-9c71-3985f44248ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:30.997256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecd9a5-873b-4687-9c71-3985f44248ce admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:31.028827 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3764/7560] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:30 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:31.032622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75ec56-73e6-4bc2-9329-4b54cd2af0cf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:31.049293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75ec56-73e6-4bc2-9329-4b54cd2af0cf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:31.054877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb6022-f2b5-4718-b3f6-bf0f363301cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:31.056594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb6022-f2b5-4718-b3f6-bf0f363301cd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:31.056739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb6022-f2b5-4718-b3f6-bf0f363301cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:31.056932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb6022-f2b5-4718-b3f6-bf0f363301cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:31.057747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb6022-f2b5-4718-b3f6-bf0f363301cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:31.059782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb6022-f2b5-4718-b3f6-bf0f363301cd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:31.074100 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3797/7561] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:20:31.077951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecd9a5-873b-4687-9c71-3985f44248ce admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:31.091893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecd9a5-873b-4687-9c71-3985f44248ce admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:31.106574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda3fc4-28ea-4452-a735-bfb4726a7d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:31.107255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda3fc4-28ea-4452-a735-bfb4726a7d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:31.107517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda3fc4-28ea-4452-a735-bfb4726a7d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:31.107786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda3fc4-28ea-4452-a735-bfb4726a7d8d admin admin] PATH_INFO: `/v3/projects/585af6eb6e56496c86838491f96c482d/users/66ddca3be5774d7286d21e17e1085ba4/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:31.136394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda3fc4-28ea-4452-a735-bfb4726a7d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=585af6eb6e56496c86838491f96c482d, user_id=66ddca3be5774d7286d21e17e1085ba4, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:31.140741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda3fc4-28ea-4452-a735-bfb4726a7d8d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:31.164197 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3765/7562] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:31 2022] PUT /identity/v3/projects/585af6eb6e56496c86838491f96c482d/users/66ddca3be5774d7286d21e17e1085ba4/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:31.168895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb6022-f2b5-4718-b3f6-bf0f363301cd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:31.179943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb6022-f2b5-4718-b3f6-bf0f363301cd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:31.281942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeedf63-d1d1-4b03-bc4c-3731c5ffe7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:31.283010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeedf63-d1d1-4b03-bc4c-3731c5ffe7eb admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:31.283509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeedf63-d1d1-4b03-bc4c-3731c5ffe7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:31.283998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeedf63-d1d1-4b03-bc4c-3731c5ffe7eb admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:31.284924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeedf63-d1d1-4b03-bc4c-3731c5ffe7eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:31.286976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeedf63-d1d1-4b03-bc4c-3731c5ffe7eb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:31.302646 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3798/7563] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:31.304154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda3fc4-28ea-4452-a735-bfb4726a7d8d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:31.318770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda3fc4-28ea-4452-a735-bfb4726a7d8d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:31.324896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d6d66-635d-42d2-8459-ec4def1da59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:31.326384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d6d66-635d-42d2-8459-ec4def1da59d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:31.326384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d6d66-635d-42d2-8459-ec4def1da59d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:31.326384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d6d66-635d-42d2-8459-ec4def1da59d admin admin] PATH_INFO: `/v3/projects/585af6eb6e56496c86838491f96c482d/users/66ddca3be5774d7286d21e17e1085ba4/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:31.328903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d6d66-635d-42d2-8459-ec4def1da59d admin admin] RBAC: Authorizing `identity:create_grant(project_id=585af6eb6e56496c86838491f96c482d, user_id=66ddca3be5774d7286d21e17e1085ba4, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:31.331920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d6d66-635d-42d2-8459-ec4def1da59d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:31.358261 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3766/7564] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:31 2022] PUT /identity/v3/projects/585af6eb6e56496c86838491f96c482d/users/66ddca3be5774d7286d21e17e1085ba4/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:31.361099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096359a8-0851-46ae-aa0d-7e1cf6c571ce None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:31.361753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096359a8-0851-46ae-aa0d-7e1cf6c571ce None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:31.362199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096359a8-0851-46ae-aa0d-7e1cf6c571ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:31.399709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-096359a8-0851-46ae-aa0d-7e1cf6c571ce None None] MFA Rules not processed for user `66ddca3be5774d7286d21e17e1085ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:31.401818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-096359a8-0851-46ae-aa0d-7e1cf6c571ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:31.519326 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3799/7565] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:31.520323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d6d66-635d-42d2-8459-ec4def1da59d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:31.522745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9d5d6d66-635d-42d2-8459-ec4def1da59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:31.616383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d6d66-635d-42d2-8459-ec4def1da59d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:31.622889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5755a7-83ea-4a05-87c4-2f497f7e5749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:31.624140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5755a7-83ea-4a05-87c4-2f497f7e5749 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:31.624510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5755a7-83ea-4a05-87c4-2f497f7e5749 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:31.624885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5755a7-83ea-4a05-87c4-2f497f7e5749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:31.644123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5755a7-83ea-4a05-87c4-2f497f7e5749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:31.646195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5755a7-83ea-4a05-87c4-2f497f7e5749 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:31.670473 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3767/7566] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:20:31.840302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096359a8-0851-46ae-aa0d-7e1cf6c571ce None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:31.862003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096359a8-0851-46ae-aa0d-7e1cf6c571ce None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:31.885857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f405d1-08ba-40f4-8443-8aa71b3b29d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:31.886929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f405d1-08ba-40f4-8443-8aa71b3b29d1 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:31.887308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f405d1-08ba-40f4-8443-8aa71b3b29d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:31.887649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f405d1-08ba-40f4-8443-8aa71b3b29d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:31.936567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:32.019530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f405d1-08ba-40f4-8443-8aa71b3b29d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:32.021904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f405d1-08ba-40f4-8443-8aa71b3b29d1 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:32.036723 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3800/7567] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:35.028027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5755a7-83ea-4a05-87c4-2f497f7e5749 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:35.172263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5755a7-83ea-4a05-87c4-2f497f7e5749 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:35.190691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:35.190884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:35.191160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:35.191593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] PATH_INFO: `/v3/users/49c50b923f004ec8b57748595da167ea` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:35.193099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c50b923f004ec8b57748595da167ea)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:35.198408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:35.245407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:35.303881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c50b923f004ec8b57748595da167ea'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:35.342449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c50b923f004ec8b57748595da167ea'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:35.361462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c50b923f004ec8b57748595da167ea'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:35.377614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c50b923f004ec8b57748595da167ea'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:35.378654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c50b923f004ec8b57748595da167ea'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:35.396524 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3768/7568] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:35 2022] DELETE /identity/v3/users/49c50b923f004ec8b57748595da167ea => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:35.398487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fcba0-399b-4e90-be6f-f7d8326453c4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:35.399025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fcba0-399b-4e90-be6f-f7d8326453c4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:35.399214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fcba0-399b-4e90-be6f-f7d8326453c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:35.423645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-807fcba0-399b-4e90-be6f-f7d8326453c4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:35.425091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-807fcba0-399b-4e90-be6f-f7d8326453c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:35.552273 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3801/7569] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:35.585288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:35.586911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:35.790949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521462f9-3412-48be-9e14-103396c2eb2f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:35.800386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c5953-f2ca-4905-8e8d-21b6e47998ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:35.801014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c5953-f2ca-4905-8e8d-21b6e47998ab service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:35.802181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c5953-f2ca-4905-8e8d-21b6e47998ab service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:35.802457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c5953-f2ca-4905-8e8d-21b6e47998ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:35.833396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c5953-f2ca-4905-8e8d-21b6e47998ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:35.834874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c5953-f2ca-4905-8e8d-21b6e47998ab service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:35.871443 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3769/7570] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:37.209006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fcba0-399b-4e90-be6f-f7d8326453c4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:37.211921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-807fcba0-399b-4e90-be6f-f7d8326453c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:37.231900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fcba0-399b-4e90-be6f-f7d8326453c4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:37.253480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:37.254144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:37.254445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:37.254697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] PATH_INFO: `/v3/projects/465cd17b7a7f42dc861593a7ea741ab8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:37.257605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=465cd17b7a7f42dc861593a7ea741ab8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:37.263539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:37.357334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465cd17b7a7f42dc861593a7ea741ab8', 'request_id': 'req-87086be6-acd2-481c-9489-c5e7c9e9b505'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:37.369556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465cd17b7a7f42dc861593a7ea741ab8', 'request_id': 'req-87086be6-acd2-481c-9489-c5e7c9e9b505'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:37.385119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465cd17b7a7f42dc861593a7ea741ab8', 'request_id': 'req-87086be6-acd2-481c-9489-c5e7c9e9b505'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:37.401860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] The token cache is being invalidate because project 465cd17b7a7f42dc861593a7ea741ab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:37.402521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:37.403656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:37.407583 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3802/7571] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:37 2022] DELETE /identity/v3/projects/465cd17b7a7f42dc861593a7ea741ab8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:37.415050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c5953-f2ca-4905-8e8d-21b6e47998ab service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:37.432450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c5953-f2ca-4905-8e8d-21b6e47998ab service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:37.552282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:37.552921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:37.553235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:37.553622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] PATH_INFO: `/v3/users/66ddca3be5774d7286d21e17e1085ba4` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:37.554818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ddca3be5774d7286d21e17e1085ba4)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:37.557446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:37.635273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:37.639541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ddca3be5774d7286d21e17e1085ba4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:37.656866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ddca3be5774d7286d21e17e1085ba4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:37.667311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ddca3be5774d7286d21e17e1085ba4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:37.676872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ddca3be5774d7286d21e17e1085ba4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:37.677819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ae13f1d3-0de1-4ea0-8524-002635ead93a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ddca3be5774d7286d21e17e1085ba4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:37.685133 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3770/7572] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:37 2022] DELETE /identity/v3/users/66ddca3be5774d7286d21e17e1085ba4 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:38.048415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:38.050609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:38.075552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87086be6-acd2-481c-9489-c5e7c9e9b505 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:38.153358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:38.182477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:38.183884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:38.183884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:38.183884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] PATH_INFO: `/v3/projects/585af6eb6e56496c86838491f96c482d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:38.185366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=585af6eb6e56496c86838491f96c482d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:38.187581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:38.259340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '585af6eb6e56496c86838491f96c482d', 'request_id': 'req-0a519655-8634-4bb6-825b-0e67cd9bba11'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:38.267391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '585af6eb6e56496c86838491f96c482d', 'request_id': 'req-0a519655-8634-4bb6-825b-0e67cd9bba11'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:38.278544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '585af6eb6e56496c86838491f96c482d', 'request_id': 'req-0a519655-8634-4bb6-825b-0e67cd9bba11'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:38.290680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] The token cache is being invalidate because project 585af6eb6e56496c86838491f96c482d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:38.291164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:38.291907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0a519655-8634-4bb6-825b-0e67cd9bba11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:38.293805 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3803/7573] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:38 2022] DELETE /identity/v3/projects/585af6eb6e56496c86838491f96c482d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:38.299172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690055bb-43e5-446c-9bf6-ec08285ad784 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:38.299563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690055bb-43e5-446c-9bf6-ec08285ad784 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:38.299899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690055bb-43e5-446c-9bf6-ec08285ad784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:38.322390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-690055bb-43e5-446c-9bf6-ec08285ad784 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:38.323912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-690055bb-43e5-446c-9bf6-ec08285ad784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:38.402628 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3771/7574] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:38.404966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9709d-ae93-4093-9fa9-717ead5ad012 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:38.408238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9709d-ae93-4093-9fa9-717ead5ad012 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:38.408238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9709d-ae93-4093-9fa9-717ead5ad012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:38.437116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-68d9709d-ae93-4093-9fa9-717ead5ad012 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:38.438622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-68d9709d-ae93-4093-9fa9-717ead5ad012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:38.457107 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3804/7575] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:38.459885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690055bb-43e5-446c-9bf6-ec08285ad784 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:38.473994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690055bb-43e5-446c-9bf6-ec08285ad784 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:38.486682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348255f-c732-4603-bf14-58cd11c5157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:38.487583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348255f-c732-4603-bf14-58cd11c5157a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:38.488011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348255f-c732-4603-bf14-58cd11c5157a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:38.488441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348255f-c732-4603-bf14-58cd11c5157a admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:38.489338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348255f-c732-4603-bf14-58cd11c5157a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:20:38.489828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348255f-c732-4603-bf14-58cd11c5157a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:38.492134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348255f-c732-4603-bf14-58cd11c5157a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:38.506829 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3772/7576] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:20:38.511876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9709d-ae93-4093-9fa9-717ead5ad012 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:38.524075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9709d-ae93-4093-9fa9-717ead5ad012 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:38.544360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045af53-2b73-463b-bcd8-3be26bc89914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:38.550708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045af53-2b73-463b-bcd8-3be26bc89914 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:38.551071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045af53-2b73-463b-bcd8-3be26bc89914 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:38.551598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045af53-2b73-463b-bcd8-3be26bc89914 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:38.553442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045af53-2b73-463b-bcd8-3be26bc89914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:38.556411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045af53-2b73-463b-bcd8-3be26bc89914 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:38.625497 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3805/7577] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:38 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:38.630003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348255f-c732-4603-bf14-58cd11c5157a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:38.644811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348255f-c732-4603-bf14-58cd11c5157a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:38.734703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19701b-ba98-44aa-95c6-57ff0c83f4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:38.735766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19701b-ba98-44aa-95c6-57ff0c83f4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:38.736299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19701b-ba98-44aa-95c6-57ff0c83f4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:38.736805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19701b-ba98-44aa-95c6-57ff0c83f4c2 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:38.737876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19701b-ba98-44aa-95c6-57ff0c83f4c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:38.740668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19701b-ba98-44aa-95c6-57ff0c83f4c2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:38.780397 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3773/7578] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:38 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:38.784474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045af53-2b73-463b-bcd8-3be26bc89914 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:38.800665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045af53-2b73-463b-bcd8-3be26bc89914 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:38.812888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277a4a8-70d0-487e-a07e-8abe7abd431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:38.813827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277a4a8-70d0-487e-a07e-8abe7abd431f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:38.814293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277a4a8-70d0-487e-a07e-8abe7abd431f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:38.814716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277a4a8-70d0-487e-a07e-8abe7abd431f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:38.815575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277a4a8-70d0-487e-a07e-8abe7abd431f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:38.830773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277a4a8-70d0-487e-a07e-8abe7abd431f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:38.848985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19701b-ba98-44aa-95c6-57ff0c83f4c2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:38.850378 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3806/7579] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:38.866396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19701b-ba98-44aa-95c6-57ff0c83f4c2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:38.880316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded12bcc-f8f3-404e-8538-e465e57b6549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:38.880863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded12bcc-f8f3-404e-8538-e465e57b6549 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:38.881163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded12bcc-f8f3-404e-8538-e465e57b6549 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:38.881431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded12bcc-f8f3-404e-8538-e465e57b6549 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:38.887132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded12bcc-f8f3-404e-8538-e465e57b6549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:38.892374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded12bcc-f8f3-404e-8538-e465e57b6549 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:38.913613 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3774/7580] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:38.917897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277a4a8-70d0-487e-a07e-8abe7abd431f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:38.931334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277a4a8-70d0-487e-a07e-8abe7abd431f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:38.951873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145feba8-1d77-4f1b-943b-a76a20473980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:38.952559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145feba8-1d77-4f1b-943b-a76a20473980 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:38.952835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145feba8-1d77-4f1b-943b-a76a20473980 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:38.953187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145feba8-1d77-4f1b-943b-a76a20473980 admin admin] PATH_INFO: `/v3/projects/4693b72376834de19759973bd8cce84d/users/ed0ed9256dd146b0b160dd845e526ec6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:38.981404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145feba8-1d77-4f1b-943b-a76a20473980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4693b72376834de19759973bd8cce84d, user_id=ed0ed9256dd146b0b160dd845e526ec6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:38.984186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145feba8-1d77-4f1b-943b-a76a20473980 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:39.005158 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3807/7581] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:38 2022] PUT /identity/v3/projects/4693b72376834de19759973bd8cce84d/users/ed0ed9256dd146b0b160dd845e526ec6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:39.019442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046efb9-f42a-455f-9196-ab6f52c177f6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:39.020318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046efb9-f42a-455f-9196-ab6f52c177f6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:39.020592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046efb9-f42a-455f-9196-ab6f52c177f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:39.108152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4046efb9-f42a-455f-9196-ab6f52c177f6 None None] MFA Rules not processed for user `ed0ed9256dd146b0b160dd845e526ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:39.111061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4046efb9-f42a-455f-9196-ab6f52c177f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:39.139349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:39.249618 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3775/7582] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:39.254292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145feba8-1d77-4f1b-943b-a76a20473980 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:39.395399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145feba8-1d77-4f1b-943b-a76a20473980 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:39.403219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ec047-3d4f-4fd9-a57b-1a5d69c843a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:39.404173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ec047-3d4f-4fd9-a57b-1a5d69c843a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:39.404700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ec047-3d4f-4fd9-a57b-1a5d69c843a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:39.405205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ec047-3d4f-4fd9-a57b-1a5d69c843a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:39.406046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ec047-3d4f-4fd9-a57b-1a5d69c843a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:39.408261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ec047-3d4f-4fd9-a57b-1a5d69c843a6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:39.460379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:39.487723 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3808/7583] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:39 2022] POST /identity/v3/projects => generated 379 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:39.487948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046efb9-f42a-455f-9196-ab6f52c177f6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:39.506696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046efb9-f42a-455f-9196-ab6f52c177f6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:39.623882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac556b19-ce15-4ed8-a7d5-2327642d8e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:39.624865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac556b19-ce15-4ed8-a7d5-2327642d8e29 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:39.625413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac556b19-ce15-4ed8-a7d5-2327642d8e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:39.625874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac556b19-ce15-4ed8-a7d5-2327642d8e29 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:39.626709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac556b19-ce15-4ed8-a7d5-2327642d8e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:39.629228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac556b19-ce15-4ed8-a7d5-2327642d8e29 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:39.658379 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3776/7584] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:39 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:39.662004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ec047-3d4f-4fd9-a57b-1a5d69c843a6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:39.683501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ec047-3d4f-4fd9-a57b-1a5d69c843a6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:39.692949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52dbc0-9cb2-4ba1-a078-58455407f48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:39.693708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52dbc0-9cb2-4ba1-a078-58455407f48a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:39.694048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52dbc0-9cb2-4ba1-a078-58455407f48a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:39.694387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52dbc0-9cb2-4ba1-a078-58455407f48a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:39.695100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52dbc0-9cb2-4ba1-a078-58455407f48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:39.700291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52dbc0-9cb2-4ba1-a078-58455407f48a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:39.733980 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3809/7585] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:20:39.737798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac556b19-ce15-4ed8-a7d5-2327642d8e29 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:39.755690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac556b19-ce15-4ed8-a7d5-2327642d8e29 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:39.780682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276e43e-b79f-45d7-81a0-58a7f54b745d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:39.781714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276e43e-b79f-45d7-81a0-58a7f54b745d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:39.781942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276e43e-b79f-45d7-81a0-58a7f54b745d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:39.782342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276e43e-b79f-45d7-81a0-58a7f54b745d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:39.783281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276e43e-b79f-45d7-81a0-58a7f54b745d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:39.785260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276e43e-b79f-45d7-81a0-58a7f54b745d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:39.812711 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3777/7586] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:20:39.817169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52dbc0-9cb2-4ba1-a078-58455407f48a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:39.835874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52dbc0-9cb2-4ba1-a078-58455407f48a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:39.853332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8006b19-5bd0-43cf-b373-c225ef9d08e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:39.854410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8006b19-5bd0-43cf-b373-c225ef9d08e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:39.855234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8006b19-5bd0-43cf-b373-c225ef9d08e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:39.855234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8006b19-5bd0-43cf-b373-c225ef9d08e2 admin admin] PATH_INFO: `/v3/projects/ffcf05e364ed45fd8f6c4d19daac2ce1/users/e3c31f3d345e4c718aa55db6ce9de88d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:39.883480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8006b19-5bd0-43cf-b373-c225ef9d08e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffcf05e364ed45fd8f6c4d19daac2ce1, user_id=e3c31f3d345e4c718aa55db6ce9de88d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:39.886197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8006b19-5bd0-43cf-b373-c225ef9d08e2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:39.908680 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3810/7587] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:39 2022] PUT /identity/v3/projects/ffcf05e364ed45fd8f6c4d19daac2ce1/users/e3c31f3d345e4c718aa55db6ce9de88d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:39.919934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d81f-b5e6-4403-b61b-92a0583182f8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:39.919934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d81f-b5e6-4403-b61b-92a0583182f8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:39.920603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d81f-b5e6-4403-b61b-92a0583182f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:39.966460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8f22d81f-b5e6-4403-b61b-92a0583182f8 None None] MFA Rules not processed for user `e3c31f3d345e4c718aa55db6ce9de88d`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:39.967699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8f22d81f-b5e6-4403-b61b-92a0583182f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:40.079123 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3778/7588] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:40.084123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8006b19-5bd0-43cf-b373-c225ef9d08e2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:40.180737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8006b19-5bd0-43cf-b373-c225ef9d08e2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:40.186002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1ba68-d9bb-409a-a5db-7276bec8cc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:40.186808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1ba68-d9bb-409a-a5db-7276bec8cc0d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:40.187239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1ba68-d9bb-409a-a5db-7276bec8cc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:40.187571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1ba68-d9bb-409a-a5db-7276bec8cc0d admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:40.188456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1ba68-d9bb-409a-a5db-7276bec8cc0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:40.202716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1ba68-d9bb-409a-a5db-7276bec8cc0d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:40.257473 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3811/7589] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:40 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:40.261917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22d81f-b5e6-4403-b61b-92a0583182f8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:40.275339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22d81f-b5e6-4403-b61b-92a0583182f8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:40.323432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:40.375432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a5fe9-37e2-40fd-ab03-06d6b8adebe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:40.376483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a5fe9-37e2-40fd-ab03-06d6b8adebe1 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:40.376755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a5fe9-37e2-40fd-ab03-06d6b8adebe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:40.377017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a5fe9-37e2-40fd-ab03-06d6b8adebe1 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:40.378235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a5fe9-37e2-40fd-ab03-06d6b8adebe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:40.380986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a5fe9-37e2-40fd-ab03-06d6b8adebe1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:40.413188 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3779/7590] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:40 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:40.416837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1ba68-d9bb-409a-a5db-7276bec8cc0d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:40.427887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1ba68-d9bb-409a-a5db-7276bec8cc0d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:40.443701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860b5d-1510-44af-91ce-16fc7f8dcfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:40.446409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860b5d-1510-44af-91ce-16fc7f8dcfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:40.446933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860b5d-1510-44af-91ce-16fc7f8dcfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:40.447469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860b5d-1510-44af-91ce-16fc7f8dcfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:40.451580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32860b5d-1510-44af-91ce-16fc7f8dcfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:40.453536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32860b5d-1510-44af-91ce-16fc7f8dcfb8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:40.469666 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3812/7591] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:40.473805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a5fe9-37e2-40fd-ab03-06d6b8adebe1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:40.490132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a5fe9-37e2-40fd-ab03-06d6b8adebe1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:40.504777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6ce28-02e6-43a7-bec6-8e5844a9560a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:40.505604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6ce28-02e6-43a7-bec6-8e5844a9560a admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:40.505957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6ce28-02e6-43a7-bec6-8e5844a9560a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:40.506307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6ce28-02e6-43a7-bec6-8e5844a9560a admin admin] PATH_INFO: `/v3/projects/eebb84ca38104598b56df57fc9e63480/users/357e2e41a4da4a3ba473ad78405ba2b8/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:40.530550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6ce28-02e6-43a7-bec6-8e5844a9560a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebb84ca38104598b56df57fc9e63480, user_id=357e2e41a4da4a3ba473ad78405ba2b8, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:40.533810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6ce28-02e6-43a7-bec6-8e5844a9560a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:40.552940 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3780/7592] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:40 2022] PUT /identity/v3/projects/eebb84ca38104598b56df57fc9e63480/users/357e2e41a4da4a3ba473ad78405ba2b8/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:40.557295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860b5d-1510-44af-91ce-16fc7f8dcfb8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:40.571800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860b5d-1510-44af-91ce-16fc7f8dcfb8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:40.689351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b8123-f8ad-4f09-9571-694bed9830f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:40.691207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8123-f8ad-4f09-9571-694bed9830f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:40.691485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8123-f8ad-4f09-9571-694bed9830f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:40.693665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8123-f8ad-4f09-9571-694bed9830f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:40.694689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b8123-f8ad-4f09-9571-694bed9830f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:40.702316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b8123-f8ad-4f09-9571-694bed9830f5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:40.729688 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3813/7593] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:40.732943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6ce28-02e6-43a7-bec6-8e5844a9560a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:40.755948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6ce28-02e6-43a7-bec6-8e5844a9560a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:40.765148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cc0a9-6a53-4410-bf8e-5f6772fa1ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:40.766335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cc0a9-6a53-4410-bf8e-5f6772fa1ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:40.767044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cc0a9-6a53-4410-bf8e-5f6772fa1ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:40.767491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cc0a9-6a53-4410-bf8e-5f6772fa1ca6 admin admin] PATH_INFO: `/v3/projects/eebb84ca38104598b56df57fc9e63480/users/357e2e41a4da4a3ba473ad78405ba2b8/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:40.772509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cc0a9-6a53-4410-bf8e-5f6772fa1ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebb84ca38104598b56df57fc9e63480, user_id=357e2e41a4da4a3ba473ad78405ba2b8, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:40.778517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cc0a9-6a53-4410-bf8e-5f6772fa1ca6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:40.810572 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3781/7594] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:40 2022] PUT /identity/v3/projects/eebb84ca38104598b56df57fc9e63480/users/357e2e41a4da4a3ba473ad78405ba2b8/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:40.821805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ccb0c-3967-4754-b6ce-0c267b2c125e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:40.822020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ccb0c-3967-4754-b6ce-0c267b2c125e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:40.822302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ccb0c-3967-4754-b6ce-0c267b2c125e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:40.875336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cc0a9-6a53-4410-bf8e-5f6772fa1ca6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:40.877147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-297cc0a9-6a53-4410-bf8e-5f6772fa1ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:40.893392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ca9ccb0c-3967-4754-b6ce-0c267b2c125e None None] MFA Rules not processed for user `357e2e41a4da4a3ba473ad78405ba2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:40.895767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ca9ccb0c-3967-4754-b6ce-0c267b2c125e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:40.919111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:41.024207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3814/7595] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:41.027322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cc0a9-6a53-4410-bf8e-5f6772fa1ca6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:41.031929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ccb0c-3967-4754-b6ce-0c267b2c125e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:41.034282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ca9ccb0c-3967-4754-b6ce-0c267b2c125e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:41.039800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f01d-d9a2-45ef-b148-477162005fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:41.040877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f01d-d9a2-45ef-b148-477162005fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:41.041495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f01d-d9a2-45ef-b148-477162005fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:41.042046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f01d-d9a2-45ef-b148-477162005fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:41.044310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b944f01d-d9a2-45ef-b148-477162005fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:41.125750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ccb0c-3967-4754-b6ce-0c267b2c125e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:41.130708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935af5cd-b322-4ae4-b464-160b568c8c4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:41.131638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935af5cd-b322-4ae4-b464-160b568c8c4f service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:41.132256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935af5cd-b322-4ae4-b464-160b568c8c4f service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:41.132783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935af5cd-b322-4ae4-b464-160b568c8c4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:41.150463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935af5cd-b322-4ae4-b464-160b568c8c4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:41.152747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935af5cd-b322-4ae4-b464-160b568c8c4f service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:41.169105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944f01d-d9a2-45ef-b148-477162005fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:41.170587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944f01d-d9a2-45ef-b148-477162005fe4 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:41.180639 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3815/7596] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:20:41.201362 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3782/7597] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:41.263738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935af5cd-b322-4ae4-b464-160b568c8c4f service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:41.265682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-935af5cd-b322-4ae4-b464-160b568c8c4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:41.283727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935af5cd-b322-4ae4-b464-160b568c8c4f service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:41.289431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a5b87-eb72-4163-b414-38dc126e482e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:41.290238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a5b87-eb72-4163-b414-38dc126e482e service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:41.290691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a5b87-eb72-4163-b414-38dc126e482e service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:41.291083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a5b87-eb72-4163-b414-38dc126e482e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:41.398077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a5b87-eb72-4163-b414-38dc126e482e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:41.400473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a5b87-eb72-4163-b414-38dc126e482e service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:41.431846 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3816/7598] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:20:44.670936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f01d-d9a2-45ef-b148-477162005fe4 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:44.694272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:44.754801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f01d-d9a2-45ef-b148-477162005fe4 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:44.761763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:44.762460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:44.762740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:44.763054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] PATH_INFO: `/v3/users/ed0ed9256dd146b0b160dd845e526ec6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:44.764461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed0ed9256dd146b0b160dd845e526ec6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:44.766801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:44.830187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed0ed9256dd146b0b160dd845e526ec6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:44.844163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed0ed9256dd146b0b160dd845e526ec6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:44.853807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed0ed9256dd146b0b160dd845e526ec6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:44.862634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed0ed9256dd146b0b160dd845e526ec6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:44.863515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed0ed9256dd146b0b160dd845e526ec6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:44.871391 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3783/7599] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:44 2022] DELETE /identity/v3/users/ed0ed9256dd146b0b160dd845e526ec6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:44.875951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d33e55-d83f-48ea-8c5f-140f62be5ae4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:44.876449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d33e55-d83f-48ea-8c5f-140f62be5ae4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:44.876903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d33e55-d83f-48ea-8c5f-140f62be5ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:44.898552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f8d33e55-d83f-48ea-8c5f-140f62be5ae4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:44.900403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f8d33e55-d83f-48ea-8c5f-140f62be5ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:44.962808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:44.975306 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3817/7600] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:44.986222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:44.987790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:45.061396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634450f-a02d-4ee8-b237-4c1299dcecd6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:45.065830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a0ab8-607e-4c45-8e50-c0752f695797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:45.066848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a0ab8-607e-4c45-8e50-c0752f695797 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:45.067193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a0ab8-607e-4c45-8e50-c0752f695797 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:45.067466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a0ab8-607e-4c45-8e50-c0752f695797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:45.079276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a0ab8-607e-4c45-8e50-c0752f695797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:45.080912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a0ab8-607e-4c45-8e50-c0752f695797 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:45.093137 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3784/7601] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:45.484994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d33e55-d83f-48ea-8c5f-140f62be5ae4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:45.487268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f8d33e55-d83f-48ea-8c5f-140f62be5ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:45.510487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d33e55-d83f-48ea-8c5f-140f62be5ae4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:45.521363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:45.522428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:45.524325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:45.525064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] PATH_INFO: `/v3/projects/4693b72376834de19759973bd8cce84d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:45.527789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4693b72376834de19759973bd8cce84d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:45.530314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:45.652137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4693b72376834de19759973bd8cce84d', 'request_id': 'req-b4c99099-c399-4252-96e4-63fda6d07b63'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:45.675463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4693b72376834de19759973bd8cce84d', 'request_id': 'req-b4c99099-c399-4252-96e4-63fda6d07b63'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:45.690937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4693b72376834de19759973bd8cce84d', 'request_id': 'req-b4c99099-c399-4252-96e4-63fda6d07b63'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:45.728760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] The token cache is being invalidate because project 4693b72376834de19759973bd8cce84d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:45.729793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:45.731383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:45.733738 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3818/7602] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:45 2022] DELETE /identity/v3/projects/4693b72376834de19759973bd8cce84d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:45.737548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a0ab8-607e-4c45-8e50-c0752f695797 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:45.751499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a0ab8-607e-4c45-8e50-c0752f695797 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:45.817221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:45.853711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:45.854348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:45.854618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:45.854911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] PATH_INFO: `/v3/users/e3c31f3d345e4c718aa55db6ce9de88d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:45.856499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c31f3d345e4c718aa55db6ce9de88d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:45.858702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:45.909271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c31f3d345e4c718aa55db6ce9de88d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:45.922869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c31f3d345e4c718aa55db6ce9de88d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:45.931129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c31f3d345e4c718aa55db6ce9de88d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:45.938277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c31f3d345e4c718aa55db6ce9de88d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:45.939533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c31f3d345e4c718aa55db6ce9de88d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:45.947313 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3785/7603] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:45 2022] DELETE /identity/v3/users/e3c31f3d345e4c718aa55db6ce9de88d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:46.609819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:46.611344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:46.627461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c99099-c399-4252-96e4-63fda6d07b63 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:46.726316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:46.727668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:46.728041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:46.728407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] PATH_INFO: `/v3/projects/ffcf05e364ed45fd8f6c4d19daac2ce1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:46.729763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffcf05e364ed45fd8f6c4d19daac2ce1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:46.732784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:46.818013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffcf05e364ed45fd8f6c4d19daac2ce1', 'request_id': 'req-83b4dfc4-a671-419e-8c88-74acd634e8fd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:46.829140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffcf05e364ed45fd8f6c4d19daac2ce1', 'request_id': 'req-83b4dfc4-a671-419e-8c88-74acd634e8fd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:46.840396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffcf05e364ed45fd8f6c4d19daac2ce1', 'request_id': 'req-83b4dfc4-a671-419e-8c88-74acd634e8fd'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:46.868331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] The token cache is being invalidate because project ffcf05e364ed45fd8f6c4d19daac2ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:46.868619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:46.869331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:46.871080 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3819/7604] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:46 2022] DELETE /identity/v3/projects/ffcf05e364ed45fd8f6c4d19daac2ce1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:46.874994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:46.876424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:46.892594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296883f6-e650-4bde-89fb-62620cdaabd7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:47.001438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:47.001749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:47.002021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:47.002441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] PATH_INFO: `/v3/users/357e2e41a4da4a3ba473ad78405ba2b8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:47.004018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=357e2e41a4da4a3ba473ad78405ba2b8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:47.006491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:47.092643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357e2e41a4da4a3ba473ad78405ba2b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:47.114818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:47.122615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357e2e41a4da4a3ba473ad78405ba2b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:47.140189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357e2e41a4da4a3ba473ad78405ba2b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:47.162685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357e2e41a4da4a3ba473ad78405ba2b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:47.167626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b9eb0d93-66af-4542-85f5-592cc54223aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357e2e41a4da4a3ba473ad78405ba2b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:47.179583 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3786/7605] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:46 2022] DELETE /identity/v3/users/357e2e41a4da4a3ba473ad78405ba2b8 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:47.910328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:47.911874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:47.925820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4dfc4-a671-419e-8c88-74acd634e8fd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:47.931505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:48.009995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:48.010642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:48.010919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:48.011186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] PATH_INFO: `/v3/projects/eebb84ca38104598b56df57fc9e63480` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:48.012556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eebb84ca38104598b56df57fc9e63480)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:48.015004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:48.096731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eebb84ca38104598b56df57fc9e63480', 'request_id': 'req-ddb798db-48a5-4590-8052-844ee28a2205'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:48.113724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eebb84ca38104598b56df57fc9e63480', 'request_id': 'req-ddb798db-48a5-4590-8052-844ee28a2205'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:48.128258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eebb84ca38104598b56df57fc9e63480', 'request_id': 'req-ddb798db-48a5-4590-8052-844ee28a2205'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:48.150209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] The token cache is being invalidate because project eebb84ca38104598b56df57fc9e63480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:48.150532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:48.151416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ddb798db-48a5-4590-8052-844ee28a2205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:48.153472 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3820/7606] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:47 2022] DELETE /identity/v3/projects/eebb84ca38104598b56df57fc9e63480 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:48.159257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e654f1-02ab-424a-9f90-94eb3a3486a5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:48.159761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e654f1-02ab-424a-9f90-94eb3a3486a5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:48.160132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e654f1-02ab-424a-9f90-94eb3a3486a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:48.208066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f4e654f1-02ab-424a-9f90-94eb3a3486a5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:48.210629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f4e654f1-02ab-424a-9f90-94eb3a3486a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:48.358494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da78d3-7df5-4050-86d8-b22a795db65f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:48.358494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da78d3-7df5-4050-86d8-b22a795db65f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:48.358494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da78d3-7df5-4050-86d8-b22a795db65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:48.360629 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3787/7607] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:48.384278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-33da78d3-7df5-4050-86d8-b22a795db65f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:48.385669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-33da78d3-7df5-4050-86d8-b22a795db65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:48.403032 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3821/7608] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:48.404254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e654f1-02ab-424a-9f90-94eb3a3486a5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:48.419147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e654f1-02ab-424a-9f90-94eb3a3486a5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:48.438371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74f823-8ebb-42cb-ae83-755c0973b3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:48.439085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f823-8ebb-42cb-ae83-755c0973b3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:48.439435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f823-8ebb-42cb-ae83-755c0973b3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:48.439781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f823-8ebb-42cb-ae83-755c0973b3a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:48.440528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74f823-8ebb-42cb-ae83-755c0973b3a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:20:48.440883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74f823-8ebb-42cb-ae83-755c0973b3a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:48.443150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74f823-8ebb-42cb-ae83-755c0973b3a7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:48.469411 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3788/7609] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:20:48.470595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da78d3-7df5-4050-86d8-b22a795db65f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:48.484030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da78d3-7df5-4050-86d8-b22a795db65f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:48.499633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a89cb-5580-42c4-a9d1-b005151e4fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:48.500393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a89cb-5580-42c4-a9d1-b005151e4fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:48.500741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a89cb-5580-42c4-a9d1-b005151e4fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:48.501535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a89cb-5580-42c4-a9d1-b005151e4fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:48.501861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a89cb-5580-42c4-a9d1-b005151e4fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:48.504183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a89cb-5580-42c4-a9d1-b005151e4fe3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:48.583175 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3822/7610] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:48 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:48.587953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74f823-8ebb-42cb-ae83-755c0973b3a7 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:48.601945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74f823-8ebb-42cb-ae83-755c0973b3a7 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:48.691140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fe733-4193-4bf7-b0e3-8029269789f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:48.692441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fe733-4193-4bf7-b0e3-8029269789f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:48.692888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fe733-4193-4bf7-b0e3-8029269789f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:48.693379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fe733-4193-4bf7-b0e3-8029269789f6 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:48.694964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fe733-4193-4bf7-b0e3-8029269789f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:48.702060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fe733-4193-4bf7-b0e3-8029269789f6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:48.753355 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3789/7611] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:48 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:48.758851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a89cb-5580-42c4-a9d1-b005151e4fe3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:48.774711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a89cb-5580-42c4-a9d1-b005151e4fe3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:48.790104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989488a-c7cb-44e2-851e-62ef7a94c4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:48.790593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989488a-c7cb-44e2-851e-62ef7a94c4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:48.791017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989488a-c7cb-44e2-851e-62ef7a94c4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:48.791397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989488a-c7cb-44e2-851e-62ef7a94c4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:48.792147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989488a-c7cb-44e2-851e-62ef7a94c4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:48.795342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989488a-c7cb-44e2-851e-62ef7a94c4f2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:48.824027 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3823/7612] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:48.827704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fe733-4193-4bf7-b0e3-8029269789f6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:48.839562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fe733-4193-4bf7-b0e3-8029269789f6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:48.847187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06afd1a-c649-49fd-9abf-2b7d9385403e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:48.848988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06afd1a-c649-49fd-9abf-2b7d9385403e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:48.849441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06afd1a-c649-49fd-9abf-2b7d9385403e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:48.849887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06afd1a-c649-49fd-9abf-2b7d9385403e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:48.850687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06afd1a-c649-49fd-9abf-2b7d9385403e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:48.854850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06afd1a-c649-49fd-9abf-2b7d9385403e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:48.873138 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3790/7613] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:20:48.877664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989488a-c7cb-44e2-851e-62ef7a94c4f2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:48.895346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989488a-c7cb-44e2-851e-62ef7a94c4f2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:48.913165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab258c8a-6f65-4d1f-a68d-40c6e31b01bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:48.914775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab258c8a-6f65-4d1f-a68d-40c6e31b01bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:48.915142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab258c8a-6f65-4d1f-a68d-40c6e31b01bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:48.915556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab258c8a-6f65-4d1f-a68d-40c6e31b01bd admin admin] PATH_INFO: `/v3/projects/9ef2c352f4304d33987428dd0f639c49/users/b7d25e22f0874d5d89007da04683ae2c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:48.948202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab258c8a-6f65-4d1f-a68d-40c6e31b01bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef2c352f4304d33987428dd0f639c49, user_id=b7d25e22f0874d5d89007da04683ae2c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:48.951733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab258c8a-6f65-4d1f-a68d-40c6e31b01bd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:48.977896 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3824/7614] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:48 2022] PUT /identity/v3/projects/9ef2c352f4304d33987428dd0f639c49/users/b7d25e22f0874d5d89007da04683ae2c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:48.989071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40623a-fb03-4f21-9bcb-d201a068a70d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:48.989361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40623a-fb03-4f21-9bcb-d201a068a70d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:48.989634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40623a-fb03-4f21-9bcb-d201a068a70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:49.019724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1b40623a-fb03-4f21-9bcb-d201a068a70d None None] MFA Rules not processed for user `b7d25e22f0874d5d89007da04683ae2c`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:49.021177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1b40623a-fb03-4f21-9bcb-d201a068a70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:49.062235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:49.111299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab258c8a-6f65-4d1f-a68d-40c6e31b01bd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:49.113551 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3791/7615] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:49.185504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:49.241713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab258c8a-6f65-4d1f-a68d-40c6e31b01bd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:49.248419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d50f29-8a27-4239-a239-a0ea3686f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:49.249182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d50f29-8a27-4239-a239-a0ea3686f3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:49.249525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d50f29-8a27-4239-a239-a0ea3686f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:49.249865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d50f29-8a27-4239-a239-a0ea3686f3f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:49.250581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d50f29-8a27-4239-a239-a0ea3686f3f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:49.253349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d50f29-8a27-4239-a239-a0ea3686f3f4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:49.323158 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3825/7616] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:49 2022] POST /identity/v3/projects => generated 379 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:49.323292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40623a-fb03-4f21-9bcb-d201a068a70d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:49.342123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40623a-fb03-4f21-9bcb-d201a068a70d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:49.475877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e896692-caeb-49a6-b1fa-1682789cab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:49.476675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e896692-caeb-49a6-b1fa-1682789cab9e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:49.477230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e896692-caeb-49a6-b1fa-1682789cab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:49.477610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e896692-caeb-49a6-b1fa-1682789cab9e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:49.478398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e896692-caeb-49a6-b1fa-1682789cab9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:49.480532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e896692-caeb-49a6-b1fa-1682789cab9e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:49.525081 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3792/7617] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:49 2022] POST /identity/v3/users => generated 296 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:49.528362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d50f29-8a27-4239-a239-a0ea3686f3f4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:49.553752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d50f29-8a27-4239-a239-a0ea3686f3f4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:49.590654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f154cf-eadb-4b9a-8d59-312e9051bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:49.594025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f154cf-eadb-4b9a-8d59-312e9051bbb2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:49.594314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f154cf-eadb-4b9a-8d59-312e9051bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:49.594590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f154cf-eadb-4b9a-8d59-312e9051bbb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:49.595260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f154cf-eadb-4b9a-8d59-312e9051bbb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:49.602981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f154cf-eadb-4b9a-8d59-312e9051bbb2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:49.644284 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3826/7618] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:49.645189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e896692-caeb-49a6-b1fa-1682789cab9e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:49.657678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e896692-caeb-49a6-b1fa-1682789cab9e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:49.665715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce2b31-2c8b-425b-887c-0f0315f4a789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:49.666926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce2b31-2c8b-425b-887c-0f0315f4a789 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:49.667182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce2b31-2c8b-425b-887c-0f0315f4a789 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:49.667434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce2b31-2c8b-425b-887c-0f0315f4a789 admin admin] PATH_INFO: `/v3/projects/81a6b0828fb04088a2ac285efbda5d1e/users/0122e1cdc3e2480a876817bb622d5060/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:49.685227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce2b31-2c8b-425b-887c-0f0315f4a789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a6b0828fb04088a2ac285efbda5d1e, user_id=0122e1cdc3e2480a876817bb622d5060, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:49.688379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce2b31-2c8b-425b-887c-0f0315f4a789 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:49.714332 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3793/7619] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:49 2022] PUT /identity/v3/projects/81a6b0828fb04088a2ac285efbda5d1e/users/0122e1cdc3e2480a876817bb622d5060/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:49.719204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f154cf-eadb-4b9a-8d59-312e9051bbb2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:49.750983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f154cf-eadb-4b9a-8d59-312e9051bbb2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:49.879536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45923c3c-80eb-4234-9216-454eec758f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:49.880153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45923c3c-80eb-4234-9216-454eec758f80 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:49.880432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45923c3c-80eb-4234-9216-454eec758f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:49.880700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45923c3c-80eb-4234-9216-454eec758f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:49.885788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45923c3c-80eb-4234-9216-454eec758f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:49.887437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45923c3c-80eb-4234-9216-454eec758f80 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:49.899736 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3827/7620] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:49.903530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce2b31-2c8b-425b-887c-0f0315f4a789 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:49.915420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce2b31-2c8b-425b-887c-0f0315f4a789 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:49.928572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394401c-08c1-4ec5-8954-7dbae502edf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:49.929379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394401c-08c1-4ec5-8954-7dbae502edf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:49.929812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394401c-08c1-4ec5-8954-7dbae502edf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:49.930145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394401c-08c1-4ec5-8954-7dbae502edf3 admin admin] PATH_INFO: `/v3/projects/81a6b0828fb04088a2ac285efbda5d1e/users/0122e1cdc3e2480a876817bb622d5060/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:49.932191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394401c-08c1-4ec5-8954-7dbae502edf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a6b0828fb04088a2ac285efbda5d1e, user_id=0122e1cdc3e2480a876817bb622d5060, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:49.935230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394401c-08c1-4ec5-8954-7dbae502edf3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:49.959432 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3794/7621] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:49 2022] PUT /identity/v3/projects/81a6b0828fb04088a2ac285efbda5d1e/users/0122e1cdc3e2480a876817bb622d5060/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:49.971110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1666a0-13f1-4f9f-a883-901c32a9d916 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:49.971616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1666a0-13f1-4f9f-a883-901c32a9d916 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:49.971616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1666a0-13f1-4f9f-a883-901c32a9d916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:50.003380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6b1666a0-13f1-4f9f-a883-901c32a9d916 None None] MFA Rules not processed for user `0122e1cdc3e2480a876817bb622d5060`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:50.004654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6b1666a0-13f1-4f9f-a883-901c32a9d916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:50.107871 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3828/7622] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:50.120115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394401c-08c1-4ec5-8954-7dbae502edf3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:50.122442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2394401c-08c1-4ec5-8954-7dbae502edf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:50.221090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394401c-08c1-4ec5-8954-7dbae502edf3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:50.227140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1666a0-13f1-4f9f-a883-901c32a9d916 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:50.229496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6b1666a0-13f1-4f9f-a883-901c32a9d916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:50.230446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3add36-6acd-4aa9-9e25-af6d2a0b3221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:50.231406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3add36-6acd-4aa9-9e25-af6d2a0b3221 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:50.231746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3add36-6acd-4aa9-9e25-af6d2a0b3221 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:50.232091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3add36-6acd-4aa9-9e25-af6d2a0b3221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:50.243619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3add36-6acd-4aa9-9e25-af6d2a0b3221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:50.245535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3add36-6acd-4aa9-9e25-af6d2a0b3221 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:50.261458 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3795/7623] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:50.293677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:50.342305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1666a0-13f1-4f9f-a883-901c32a9d916 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:50.348361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:50.349016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:50.349323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:50.349581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] PATH_INFO: `/v3/users/026771ec28444739afc13c47339af7b0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:50.350771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=026771ec28444739afc13c47339af7b0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:50.353092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:50.435737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '026771ec28444739afc13c47339af7b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.453550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '026771ec28444739afc13c47339af7b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.469671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '026771ec28444739afc13c47339af7b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.480815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '026771ec28444739afc13c47339af7b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.482025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '026771ec28444739afc13c47339af7b0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.492582 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3829/7624] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:50 2022] DELETE /identity/v3/users/026771ec28444739afc13c47339af7b0 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:50.497620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251647b2-eebb-4ccf-adf6-ebdebe18fdaa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:50.498074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251647b2-eebb-4ccf-adf6-ebdebe18fdaa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:50.498334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251647b2-eebb-4ccf-adf6-ebdebe18fdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:50.521718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-251647b2-eebb-4ccf-adf6-ebdebe18fdaa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:50.523300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-251647b2-eebb-4ccf-adf6-ebdebe18fdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:50.557361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:50.607981 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3796/7625] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:50.623925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:50.625017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:50.643726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251647b2-eebb-4ccf-adf6-ebdebe18fdaa None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:50.645362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-251647b2-eebb-4ccf-adf6-ebdebe18fdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:50.662854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251647b2-eebb-4ccf-adf6-ebdebe18fdaa None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:50.672887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:50.673614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:50.673895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:50.674160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] PATH_INFO: `/v3/users/b4c98ad011ab494e9c9c8a14ec5a961a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:50.675293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4c98ad011ab494e9c9c8a14ec5a961a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:50.677298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:50.711916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f673fe-5206-44ba-9f88-cac0ea39eae5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:50.721385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeac939-6278-4881-af0d-5debbec6071a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:50.722131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeac939-6278-4881-af0d-5debbec6071a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:50.722255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeac939-6278-4881-af0d-5debbec6071a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:50.722513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeac939-6278-4881-af0d-5debbec6071a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:50.733620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeac939-6278-4881-af0d-5debbec6071a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:50.735095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeac939-6278-4881-af0d-5debbec6071a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:50.741660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4c98ad011ab494e9c9c8a14ec5a961a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.747124 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3830/7626] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:50.748409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeac939-6278-4881-af0d-5debbec6071a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:50.749938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ceeac939-6278-4881-af0d-5debbec6071a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:50.756407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4c98ad011ab494e9c9c8a14ec5a961a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.764699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4c98ad011ab494e9c9c8a14ec5a961a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.764879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeac939-6278-4881-af0d-5debbec6071a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:50.770993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:50.771651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:50.771953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:50.772176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] PATH_INFO: `/v3/users/b7d25e22f0874d5d89007da04683ae2c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:50.772514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4c98ad011ab494e9c9c8a14ec5a961a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.773294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-59957d47-b37e-49c8-b130-4952e13b9398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4c98ad011ab494e9c9c8a14ec5a961a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.773655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d25e22f0874d5d89007da04683ae2c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:50.775814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:50.780144 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3797/7627] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:50 2022] DELETE /identity/v3/users/b4c98ad011ab494e9c9c8a14ec5a961a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:50.784454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b3659-1e53-4f4f-a45e-c760b20ce16a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:50.784809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b3659-1e53-4f4f-a45e-c760b20ce16a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:50.785319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b3659-1e53-4f4f-a45e-c760b20ce16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:50.809472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5c8b3659-1e53-4f4f-a45e-c760b20ce16a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:50.810778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5c8b3659-1e53-4f4f-a45e-c760b20ce16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:50.838493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d25e22f0874d5d89007da04683ae2c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.839581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:50.849312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d25e22f0874d5d89007da04683ae2c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.865965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d25e22f0874d5d89007da04683ae2c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.875919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d25e22f0874d5d89007da04683ae2c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.876428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b653b46c-8fb1-42fb-9d89-c17b7ab59b77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d25e22f0874d5d89007da04683ae2c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:50.883987 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3831/7628] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:50 2022] DELETE /identity/v3/users/b7d25e22f0874d5d89007da04683ae2c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:50.884890 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3798/7629] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:50.888393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291388b1-1e5d-4df8-94d7-60d6463b24ea None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:50.888568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291388b1-1e5d-4df8-94d7-60d6463b24ea None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:50.888840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291388b1-1e5d-4df8-94d7-60d6463b24ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:50.894887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b3659-1e53-4f4f-a45e-c760b20ce16a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:50.896417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5c8b3659-1e53-4f4f-a45e-c760b20ce16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:50.909926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-291388b1-1e5d-4df8-94d7-60d6463b24ea None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:50.911191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-291388b1-1e5d-4df8-94d7-60d6463b24ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:50.954718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:50.981542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b3659-1e53-4f4f-a45e-c760b20ce16a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:50.994925 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3832/7630] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:50.994925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291388b1-1e5d-4df8-94d7-60d6463b24ea None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:50.994925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-291388b1-1e5d-4df8-94d7-60d6463b24ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:51.006602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291388b1-1e5d-4df8-94d7-60d6463b24ea None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.012751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.013417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.013694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.014009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] PATH_INFO: `/v3/projects/2c132f2961594244a86c8a610462992e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.015143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c132f2961594244a86c8a610462992e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.017069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:51.041334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead86a-54e9-46ee-89e5-a84d43e5d960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.042018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead86a-54e9-46ee-89e5-a84d43e5d960 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.042367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead86a-54e9-46ee-89e5-a84d43e5d960 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.042723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead86a-54e9-46ee-89e5-a84d43e5d960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.052363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ead86a-54e9-46ee-89e5-a84d43e5d960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.053856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ead86a-54e9-46ee-89e5-a84d43e5d960 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:51.063868 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3799/7631] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:51.064764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead86a-54e9-46ee-89e5-a84d43e5d960 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:51.068739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c132f2961594244a86c8a610462992e', 'request_id': 'req-aa878c1d-74fa-42a7-bb97-0314381e1547'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.076519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead86a-54e9-46ee-89e5-a84d43e5d960 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.077276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c132f2961594244a86c8a610462992e', 'request_id': 'req-aa878c1d-74fa-42a7-bb97-0314381e1547'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.083894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c132f2961594244a86c8a610462992e', 'request_id': 'req-aa878c1d-74fa-42a7-bb97-0314381e1547'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.096660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] The token cache is being invalidate because project 2c132f2961594244a86c8a610462992e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:51.096944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.101528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.104066 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3833/7632] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:50 2022] DELETE /identity/v3/projects/2c132f2961594244a86c8a610462992e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:51.107966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:51.111643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:51.137805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c43868-1126-4e3a-b302-0c447ac123b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.138536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c43868-1126-4e3a-b302-0c447ac123b0 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.139216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c43868-1126-4e3a-b302-0c447ac123b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.139488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c43868-1126-4e3a-b302-0c447ac123b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.141939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-57c43868-1126-4e3a-b302-0c447ac123b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:51.166553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:51.187474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa878c1d-74fa-42a7-bb97-0314381e1547 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.203049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.203708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.203982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.204227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] PATH_INFO: `/v3/users/3acbc3744de34684a9553e3ee5cb5133` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.206167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3acbc3744de34684a9553e3ee5cb5133)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.208243 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:51.230641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c43868-1126-4e3a-b302-0c447ac123b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.233176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c43868-1126-4e3a-b302-0c447ac123b0 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:51.244841 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3800/7633] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:51.261805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c43868-1126-4e3a-b302-0c447ac123b0 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:51.264104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-57c43868-1126-4e3a-b302-0c447ac123b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:51.271831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3acbc3744de34684a9553e3ee5cb5133'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.279671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c43868-1126-4e3a-b302-0c447ac123b0 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.284586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.285265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3acbc3744de34684a9553e3ee5cb5133'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.285410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.285619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.286009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] PATH_INFO: `/v3/projects/7bf097b0f8fe47f486b68da7e7552c69` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.287420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf097b0f8fe47f486b68da7e7552c69)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.290005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:51.312559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3acbc3744de34684a9553e3ee5cb5133'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.321326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3acbc3744de34684a9553e3ee5cb5133'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.321988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1ffb5178-39e9-469c-90e0-5d4489af01c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3acbc3744de34684a9553e3ee5cb5133'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.330464 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3834/7634] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:51 2022] DELETE /identity/v3/users/3acbc3744de34684a9553e3ee5cb5133 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:51.357568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:51.360606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf097b0f8fe47f486b68da7e7552c69', 'request_id': 'req-04c3137a-15a7-4794-81b0-f83b4f88124b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.370803 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf097b0f8fe47f486b68da7e7552c69', 'request_id': 'req-04c3137a-15a7-4794-81b0-f83b4f88124b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.378196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf097b0f8fe47f486b68da7e7552c69', 'request_id': 'req-04c3137a-15a7-4794-81b0-f83b4f88124b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.391512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] The token cache is being invalidate because project 7bf097b0f8fe47f486b68da7e7552c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:51.391925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.393179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-04c3137a-15a7-4794-81b0-f83b4f88124b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.398907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3801/7635] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:51 2022] DELETE /identity/v3/projects/7bf097b0f8fe47f486b68da7e7552c69 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:51.400671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd79d6-a115-48fb-a955-45766ae40f6d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.400983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd79d6-a115-48fb-a955-45766ae40f6d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.401310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd79d6-a115-48fb-a955-45766ae40f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.424238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4fbd79d6-a115-48fb-a955-45766ae40f6d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:51.426098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4fbd79d6-a115-48fb-a955-45766ae40f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:51.505911 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3835/7636] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:51.513861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cf8cf-9843-402c-8aba-a6ba83d05f97 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.514378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cf8cf-9843-402c-8aba-a6ba83d05f97 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.514378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cf8cf-9843-402c-8aba-a6ba83d05f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.536695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-353cf8cf-9843-402c-8aba-a6ba83d05f97 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:51.537941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-353cf8cf-9843-402c-8aba-a6ba83d05f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:51.547514 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3802/7637] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:51.551827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd79d6-a115-48fb-a955-45766ae40f6d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:51.563417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd79d6-a115-48fb-a955-45766ae40f6d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.568305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9fea2-f614-4342-b9c3-f1c1bbba3ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.568909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9fea2-f614-4342-b9c3-f1c1bbba3ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.569223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9fea2-f614-4342-b9c3-f1c1bbba3ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.569495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9fea2-f614-4342-b9c3-f1c1bbba3ab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.570197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9fea2-f614-4342-b9c3-f1c1bbba3ab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:20:51.570497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9fea2-f614-4342-b9c3-f1c1bbba3ab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.572190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9fea2-f614-4342-b9c3-f1c1bbba3ab6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:51.585399 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3836/7638] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:20:51.590163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cf8cf-9843-402c-8aba-a6ba83d05f97 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:51.599905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cf8cf-9843-402c-8aba-a6ba83d05f97 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.604867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0bd55-3a35-483b-8e64-47c0688d696b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.605508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0bd55-3a35-483b-8e64-47c0688d696b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.605785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0bd55-3a35-483b-8e64-47c0688d696b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.606306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0bd55-3a35-483b-8e64-47c0688d696b admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.606938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0bd55-3a35-483b-8e64-47c0688d696b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.608838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0bd55-3a35-483b-8e64-47c0688d696b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:51.637449 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3803/7639] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:51 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:51.641824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9fea2-f614-4342-b9c3-f1c1bbba3ab6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:51.652420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9fea2-f614-4342-b9c3-f1c1bbba3ab6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.711184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0bd55-3a35-483b-8e64-47c0688d696b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:51.713399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5db0bd55-3a35-483b-8e64-47c0688d696b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:51.713555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:51.741686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f740cf-f4e5-4000-8779-f3e4cc399be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.743042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f740cf-f4e5-4000-8779-f3e4cc399be3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.743314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f740cf-f4e5-4000-8779-f3e4cc399be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.743687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f740cf-f4e5-4000-8779-f3e4cc399be3 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.745064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f740cf-f4e5-4000-8779-f3e4cc399be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.746927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f740cf-f4e5-4000-8779-f3e4cc399be3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:51.773872 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3837/7640] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:51 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:51.774573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f740cf-f4e5-4000-8779-f3e4cc399be3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:51.777881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-19f740cf-f4e5-4000-8779-f3e4cc399be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:51.791412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f740cf-f4e5-4000-8779-f3e4cc399be3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.796740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.797521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.797901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.798284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] PATH_INFO: `/v3/projects/80896dfa8a424554a0c1d252de45fadd` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.799576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80896dfa8a424554a0c1d252de45fadd)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.802475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:51.806450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0bd55-3a35-483b-8e64-47c0688d696b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.811131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.811746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.812018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.812282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] PATH_INFO: `/v3/projects/9ef2c352f4304d33987428dd0f639c49` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.813430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ef2c352f4304d33987428dd0f639c49)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.815329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:51.853901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80896dfa8a424554a0c1d252de45fadd', 'request_id': 'req-755fabb5-375f-471a-8b49-1e2e10d30394'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.862097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80896dfa8a424554a0c1d252de45fadd', 'request_id': 'req-755fabb5-375f-471a-8b49-1e2e10d30394'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.868312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ef2c352f4304d33987428dd0f639c49', 'request_id': 'req-317db359-044a-47df-b9ed-5a46a0edaa16'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.869552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80896dfa8a424554a0c1d252de45fadd', 'request_id': 'req-755fabb5-375f-471a-8b49-1e2e10d30394'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.875707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ef2c352f4304d33987428dd0f639c49', 'request_id': 'req-317db359-044a-47df-b9ed-5a46a0edaa16'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.881009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] The token cache is being invalidate because project 80896dfa8a424554a0c1d252de45fadd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:51.881329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.881991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.882469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ef2c352f4304d33987428dd0f639c49', 'request_id': 'req-317db359-044a-47df-b9ed-5a46a0edaa16'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.883982 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3838/7641] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:51 2022] DELETE /identity/v3/projects/80896dfa8a424554a0c1d252de45fadd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:51.885791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:51.887267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:51.894547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] The token cache is being invalidate because project 9ef2c352f4304d33987428dd0f639c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:51.895020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.895703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:51.897280 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3804/7642] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:51 2022] DELETE /identity/v3/projects/9ef2c352f4304d33987428dd0f639c49 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:51.898637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:51.901809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:51.940916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:51.955636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fabb5-375f-471a-8b49-1e2e10d30394 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.970351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317db359-044a-47df-b9ed-5a46a0edaa16 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:51.976652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:51.977605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:51.977882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:51.978148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] PATH_INFO: `/v3/users/5fd48a1a6e2e4954bf874b36cffca9c8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:51.979283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd48a1a6e2e4954bf874b36cffca9c8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:51.981907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:52.017566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa182d3-2342-4a12-8ee4-fdc232f09daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:52.018252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa182d3-2342-4a12-8ee4-fdc232f09daf admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.018536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa182d3-2342-4a12-8ee4-fdc232f09daf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.018808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa182d3-2342-4a12-8ee4-fdc232f09daf admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.019446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa182d3-2342-4a12-8ee4-fdc232f09daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:52.021067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa182d3-2342-4a12-8ee4-fdc232f09daf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:52.030002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd48a1a6e2e4954bf874b36cffca9c8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.031368 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3839/7643] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:52.032172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa182d3-2342-4a12-8ee4-fdc232f09daf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:52.034089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8aa182d3-2342-4a12-8ee4-fdc232f09daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:52.043356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd48a1a6e2e4954bf874b36cffca9c8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.047471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa182d3-2342-4a12-8ee4-fdc232f09daf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:52.051891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd48a1a6e2e4954bf874b36cffca9c8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.053885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:52.054997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.055333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.055606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] PATH_INFO: `/v3/users/0122e1cdc3e2480a876817bb622d5060` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.056778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0122e1cdc3e2480a876817bb622d5060)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:52.058754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:52.059425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd48a1a6e2e4954bf874b36cffca9c8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.060170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd48a1a6e2e4954bf874b36cffca9c8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.063904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:52.067339 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3805/7644] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:51 2022] DELETE /identity/v3/users/5fd48a1a6e2e4954bf874b36cffca9c8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:52.071230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:52.072390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:52.117801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0122e1cdc3e2480a876817bb622d5060'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.125884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0122e1cdc3e2480a876817bb622d5060'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.140157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0122e1cdc3e2480a876817bb622d5060'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.144425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68d73a-bd52-406f-98e6-893927d0deda admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:52.149461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0122e1cdc3e2480a876817bb622d5060'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.150306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0122e1cdc3e2480a876817bb622d5060'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.151007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a331f-d2c6-4530-b119-8dcf45ad6719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:52.151625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a331f-d2c6-4530-b119-8dcf45ad6719 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.151910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a331f-d2c6-4530-b119-8dcf45ad6719 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.152188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a331f-d2c6-4530-b119-8dcf45ad6719 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.152843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a331f-d2c6-4530-b119-8dcf45ad6719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:52.154562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a331f-d2c6-4530-b119-8dcf45ad6719 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:52.158148 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3840/7645] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:52 2022] DELETE /identity/v3/users/0122e1cdc3e2480a876817bb622d5060 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:52.170964 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3806/7646] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:52.172106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:52.178576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:52.277877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358961c-9b2e-49e5-8f8f-06fb372f720c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:52.285679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814c57d-7d34-4828-9f8f-61fa300d4eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:52.286578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814c57d-7d34-4828-9f8f-61fa300d4eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.286891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814c57d-7d34-4828-9f8f-61fa300d4eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.287215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814c57d-7d34-4828-9f8f-61fa300d4eaf admin admin] PATH_INFO: `/v3/projects/8f6c4d19087b41ffbd635225329910e4/users/d2c105472ebb4c188ad3148a8a264543/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.305635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814c57d-7d34-4828-9f8f-61fa300d4eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6c4d19087b41ffbd635225329910e4, user_id=d2c105472ebb4c188ad3148a8a264543, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:52.307837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a331f-d2c6-4530-b119-8dcf45ad6719 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:52.308611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814c57d-7d34-4828-9f8f-61fa300d4eaf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:52.309616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d72a331f-d2c6-4530-b119-8dcf45ad6719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:52.323971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a331f-d2c6-4530-b119-8dcf45ad6719 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:52.334575 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3841/7647] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:52 2022] PUT /identity/v3/projects/8f6c4d19087b41ffbd635225329910e4/users/d2c105472ebb4c188ad3148a8a264543/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:52.396878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:52.412436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:52.413683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.413683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.414096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] PATH_INFO: `/v3/projects/8513f1a4d8654e39a613bb0f269cdfba` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.416080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8513f1a4d8654e39a613bb0f269cdfba)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:52.418023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:52.450417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3853fc-5c98-45d8-8725-be21af95c245 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.450711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3853fc-5c98-45d8-8725-be21af95c245 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.451094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3853fc-5c98-45d8-8725-be21af95c245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.478889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8513f1a4d8654e39a613bb0f269cdfba', 'request_id': 'req-293d3ece-77a2-438e-89e7-d5f80dbaab04'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.487341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ae3853fc-5c98-45d8-8725-be21af95c245 None None] MFA Rules not processed for user `d2c105472ebb4c188ad3148a8a264543`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:52.488839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ae3853fc-5c98-45d8-8725-be21af95c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:52.493758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8513f1a4d8654e39a613bb0f269cdfba', 'request_id': 'req-293d3ece-77a2-438e-89e7-d5f80dbaab04'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.501012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8513f1a4d8654e39a613bb0f269cdfba', 'request_id': 'req-293d3ece-77a2-438e-89e7-d5f80dbaab04'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.515198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] The token cache is being invalidate because project 8513f1a4d8654e39a613bb0f269cdfba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:52.515791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.516135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.517770 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3807/7648] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:52 2022] DELETE /identity/v3/projects/8513f1a4d8654e39a613bb0f269cdfba => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:52.518540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:52.519991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:52.585305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:52.598344 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3842/7649] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:52.612292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3853fc-5c98-45d8-8725-be21af95c245 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:52.614398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ae3853fc-5c98-45d8-8725-be21af95c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:52.615763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d3ece-77a2-438e-89e7-d5f80dbaab04 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:52.621642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:52.622548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.622846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.623207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] PATH_INFO: `/v3/projects/81a6b0828fb04088a2ac285efbda5d1e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.624605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a6b0828fb04088a2ac285efbda5d1e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:52.626633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:52.687499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a6b0828fb04088a2ac285efbda5d1e', 'request_id': 'req-fcda0388-53fd-4a6c-92d6-a86dd17865d3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.699187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a6b0828fb04088a2ac285efbda5d1e', 'request_id': 'req-fcda0388-53fd-4a6c-92d6-a86dd17865d3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.702674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3853fc-5c98-45d8-8725-be21af95c245 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:52.710279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a6b0828fb04088a2ac285efbda5d1e', 'request_id': 'req-fcda0388-53fd-4a6c-92d6-a86dd17865d3'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.725673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] The token cache is being invalidate because project 81a6b0828fb04088a2ac285efbda5d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:52.726062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.727350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fcda0388-53fd-4a6c-92d6-a86dd17865d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:52.729638 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3808/7650] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:52 2022] DELETE /identity/v3/projects/81a6b0828fb04088a2ac285efbda5d1e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:52.734753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a2d37-6c7c-43ce-8a4d-00721e017702 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.735454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a2d37-6c7c-43ce-8a4d-00721e017702 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.735454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a2d37-6c7c-43ce-8a4d-00721e017702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.762416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-734a2d37-6c7c-43ce-8a4d-00721e017702 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:52.762966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39572cad-acf0-4440-a929-67426c2a9107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:52.763657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39572cad-acf0-4440-a929-67426c2a9107 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.764006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39572cad-acf0-4440-a929-67426c2a9107 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.764151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-734a2d37-6c7c-43ce-8a4d-00721e017702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:52.764407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39572cad-acf0-4440-a929-67426c2a9107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.766358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-39572cad-acf0-4440-a929-67426c2a9107 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:52.770870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:52.836944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39572cad-acf0-4440-a929-67426c2a9107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:52.838459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39572cad-acf0-4440-a929-67426c2a9107 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:52.848800 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3843/7651] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:20:52.853508 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3809/7652] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:52.855311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711a74e-b0c9-49b9-a05a-65a2696f2b9c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.855587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711a74e-b0c9-49b9-a05a-65a2696f2b9c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.855878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711a74e-b0c9-49b9-a05a-65a2696f2b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.884167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a711a74e-b0c9-49b9-a05a-65a2696f2b9c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:52.885592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a711a74e-b0c9-49b9-a05a-65a2696f2b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:52.895215 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3844/7653] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:52.900586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a2d37-6c7c-43ce-8a4d-00721e017702 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:52.911953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a2d37-6c7c-43ce-8a4d-00721e017702 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:52.918002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e2d07-eb05-4971-b839-5f314401d5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:52.918722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e2d07-eb05-4971-b839-5f314401d5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.919061 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e2d07-eb05-4971-b839-5f314401d5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.919397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e2d07-eb05-4971-b839-5f314401d5d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.920142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e2d07-eb05-4971-b839-5f314401d5d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:20:52.920507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e2d07-eb05-4971-b839-5f314401d5d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:52.922399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e2d07-eb05-4971-b839-5f314401d5d8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:52.938418 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3810/7654] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:20:52.943634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711a74e-b0c9-49b9-a05a-65a2696f2b9c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:52.954751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711a74e-b0c9-49b9-a05a-65a2696f2b9c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:52.960495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f441a31-fb66-4e17-b403-d9b9959d333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:52.961404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f441a31-fb66-4e17-b403-d9b9959d333b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:52.961762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f441a31-fb66-4e17-b403-d9b9959d333b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:52.962074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f441a31-fb66-4e17-b403-d9b9959d333b admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:52.963169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f441a31-fb66-4e17-b403-d9b9959d333b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:52.964871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f441a31-fb66-4e17-b403-d9b9959d333b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:53.004480 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3845/7655] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:52 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:53.005725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e2d07-eb05-4971-b839-5f314401d5d8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:53.017141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e2d07-eb05-4971-b839-5f314401d5d8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:53.091226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b16baa-bd0f-43f7-8a1d-803843e9717d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:53.091960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16baa-bd0f-43f7-8a1d-803843e9717d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:53.092268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16baa-bd0f-43f7-8a1d-803843e9717d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:53.092538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16baa-bd0f-43f7-8a1d-803843e9717d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:53.093319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b16baa-bd0f-43f7-8a1d-803843e9717d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:53.095425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b16baa-bd0f-43f7-8a1d-803843e9717d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:53.118994 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3811/7656] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:53 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:20:53.123027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f441a31-fb66-4e17-b403-d9b9959d333b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:53.135037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f441a31-fb66-4e17-b403-d9b9959d333b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:53.139849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878243c7-11f0-4215-8aff-d00b8751dc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:53.140544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878243c7-11f0-4215-8aff-d00b8751dc64 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:53.140891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878243c7-11f0-4215-8aff-d00b8751dc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:53.141258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878243c7-11f0-4215-8aff-d00b8751dc64 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:53.141979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878243c7-11f0-4215-8aff-d00b8751dc64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:53.143550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878243c7-11f0-4215-8aff-d00b8751dc64 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:53.154864 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3846/7657] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:53.159097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b16baa-bd0f-43f7-8a1d-803843e9717d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:53.170915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b16baa-bd0f-43f7-8a1d-803843e9717d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:53.178265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea43475-a885-4f79-9d85-e436b02aef2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:53.180094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea43475-a885-4f79-9d85-e436b02aef2e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:53.180543 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea43475-a885-4f79-9d85-e436b02aef2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:53.180874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea43475-a885-4f79-9d85-e436b02aef2e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:53.181653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea43475-a885-4f79-9d85-e436b02aef2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:53.183372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea43475-a885-4f79-9d85-e436b02aef2e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:53.202527 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3812/7658] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:53.206647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878243c7-11f0-4215-8aff-d00b8751dc64 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:53.218671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878243c7-11f0-4215-8aff-d00b8751dc64 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:53.223955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142855ea-a62b-4d17-9182-08c045369036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:53.224747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142855ea-a62b-4d17-9182-08c045369036 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:53.225138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142855ea-a62b-4d17-9182-08c045369036 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:53.225415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142855ea-a62b-4d17-9182-08c045369036 admin admin] PATH_INFO: `/v3/projects/3877567fc65349baadbfa1af9e8f5d15/users/e95c827f3b5644fba8d7f14ea009f597/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:53.241478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142855ea-a62b-4d17-9182-08c045369036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3877567fc65349baadbfa1af9e8f5d15, user_id=e95c827f3b5644fba8d7f14ea009f597, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:53.244885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142855ea-a62b-4d17-9182-08c045369036 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:53.259550 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3847/7659] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:53 2022] PUT /identity/v3/projects/3877567fc65349baadbfa1af9e8f5d15/users/e95c827f3b5644fba8d7f14ea009f597/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:20:53.264592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2307c-78ec-4169-9fd3-670f02cce076 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:53.264878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2307c-78ec-4169-9fd3-670f02cce076 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:53.265185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2307c-78ec-4169-9fd3-670f02cce076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:53.284805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3db2307c-78ec-4169-9fd3-670f02cce076 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:53.286060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3db2307c-78ec-4169-9fd3-670f02cce076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:53.350646 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3813/7660] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:53.365355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142855ea-a62b-4d17-9182-08c045369036 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:53.367294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-142855ea-a62b-4d17-9182-08c045369036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:53.373094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:53.434532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142855ea-a62b-4d17-9182-08c045369036 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:53.441876 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26127e-4f3d-4ede-8acd-9eda21c4dd64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:53.442490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26127e-4f3d-4ede-8acd-9eda21c4dd64 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:53.442768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26127e-4f3d-4ede-8acd-9eda21c4dd64 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:53.443036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26127e-4f3d-4ede-8acd-9eda21c4dd64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:53.453963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26127e-4f3d-4ede-8acd-9eda21c4dd64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:53.455422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26127e-4f3d-4ede-8acd-9eda21c4dd64 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:53.466113 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3848/7661] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:54.291021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819c8a9-4b96-42f0-9c22-d45affbf17b5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:54.291622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819c8a9-4b96-42f0-9c22-d45affbf17b5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:54.291808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819c8a9-4b96-42f0-9c22-d45affbf17b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:54.322976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f819c8a9-4b96-42f0-9c22-d45affbf17b5 None None] MFA Rules not processed for user `e95c827f3b5644fba8d7f14ea009f597`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:54.324384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f819c8a9-4b96-42f0-9c22-d45affbf17b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:54.330304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:54.415157 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3814/7662] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:54.416838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26127e-4f3d-4ede-8acd-9eda21c4dd64 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:54.427264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26127e-4f3d-4ede-8acd-9eda21c4dd64 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:54.432487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feae1b7-3965-42c2-9fa9-23f11aa5c6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:54.433215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feae1b7-3965-42c2-9fa9-23f11aa5c6fd admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:54.433666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feae1b7-3965-42c2-9fa9-23f11aa5c6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:54.433921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feae1b7-3965-42c2-9fa9-23f11aa5c6fd admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:54.434823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feae1b7-3965-42c2-9fa9-23f11aa5c6fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:54.436700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feae1b7-3965-42c2-9fa9-23f11aa5c6fd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:54.473266 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3849/7663] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:54 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:54.475640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819c8a9-4b96-42f0-9c22-d45affbf17b5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:54.489452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819c8a9-4b96-42f0-9c22-d45affbf17b5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:54.563620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471082c0-eafb-440d-994d-cd28f56483fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:54.564223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471082c0-eafb-440d-994d-cd28f56483fb admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:54.564492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471082c0-eafb-440d-994d-cd28f56483fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:54.568606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471082c0-eafb-440d-994d-cd28f56483fb admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:54.568606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471082c0-eafb-440d-994d-cd28f56483fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:54.568606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471082c0-eafb-440d-994d-cd28f56483fb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:54.597951 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3815/7664] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:54 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:54.599366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feae1b7-3965-42c2-9fa9-23f11aa5c6fd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:54.610412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feae1b7-3965-42c2-9fa9-23f11aa5c6fd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:54.614986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f96b71-876c-49bd-96b1-1bf7bebf4cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:54.615586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f96b71-876c-49bd-96b1-1bf7bebf4cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:54.615863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f96b71-876c-49bd-96b1-1bf7bebf4cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:54.616133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f96b71-876c-49bd-96b1-1bf7bebf4cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:54.617497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f96b71-876c-49bd-96b1-1bf7bebf4cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:54.619100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f96b71-876c-49bd-96b1-1bf7bebf4cb5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:54.634232 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3850/7665] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:54.635396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471082c0-eafb-440d-994d-cd28f56483fb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:54.646107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471082c0-eafb-440d-994d-cd28f56483fb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:54.651154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d2217-8973-4014-9864-826ca9c04eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:54.651857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d2217-8973-4014-9864-826ca9c04eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:54.652855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d2217-8973-4014-9864-826ca9c04eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:54.653186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d2217-8973-4014-9864-826ca9c04eaa admin admin] PATH_INFO: `/v3/projects/928b52e8bba9454f84abbcb9cf4cf1a6/users/842a24501b924da6b3bda9ca39ab19dc/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:54.669006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d2217-8973-4014-9864-826ca9c04eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=928b52e8bba9454f84abbcb9cf4cf1a6, user_id=842a24501b924da6b3bda9ca39ab19dc, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:54.671989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d2217-8973-4014-9864-826ca9c04eaa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:54.689319 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3816/7666] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:54 2022] PUT /identity/v3/projects/928b52e8bba9454f84abbcb9cf4cf1a6/users/842a24501b924da6b3bda9ca39ab19dc/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:54.690454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f96b71-876c-49bd-96b1-1bf7bebf4cb5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:54.701741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f96b71-876c-49bd-96b1-1bf7bebf4cb5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:54.772326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f143b-2e71-40d1-9d1b-fd3e99b8b445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:54.773620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f143b-2e71-40d1-9d1b-fd3e99b8b445 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:54.774023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f143b-2e71-40d1-9d1b-fd3e99b8b445 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:54.774302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f143b-2e71-40d1-9d1b-fd3e99b8b445 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:54.775037 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f143b-2e71-40d1-9d1b-fd3e99b8b445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:54.776691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f143b-2e71-40d1-9d1b-fd3e99b8b445 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:54.787166 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3851/7667] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:20:54.791281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d2217-8973-4014-9864-826ca9c04eaa admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:54.802437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d2217-8973-4014-9864-826ca9c04eaa admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:54.807889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abe663-62ad-4a98-a611-a9ba3c35a866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:54.808539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abe663-62ad-4a98-a611-a9ba3c35a866 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:54.808816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abe663-62ad-4a98-a611-a9ba3c35a866 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:54.809115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abe663-62ad-4a98-a611-a9ba3c35a866 admin admin] PATH_INFO: `/v3/projects/928b52e8bba9454f84abbcb9cf4cf1a6/users/842a24501b924da6b3bda9ca39ab19dc/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:54.811402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abe663-62ad-4a98-a611-a9ba3c35a866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=928b52e8bba9454f84abbcb9cf4cf1a6, user_id=842a24501b924da6b3bda9ca39ab19dc, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:54.814120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abe663-62ad-4a98-a611-a9ba3c35a866 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:54.825222 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3817/7668] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:54 2022] PUT /identity/v3/projects/928b52e8bba9454f84abbcb9cf4cf1a6/users/842a24501b924da6b3bda9ca39ab19dc/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:55.556959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c062c5-ad52-4c0a-9093-0a6f71e39676 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:55.557996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c062c5-ad52-4c0a-9093-0a6f71e39676 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:55.558481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c062c5-ad52-4c0a-9093-0a6f71e39676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:55.596229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f1c062c5-ad52-4c0a-9093-0a6f71e39676 None None] MFA Rules not processed for user `842a24501b924da6b3bda9ca39ab19dc`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:55.598922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f1c062c5-ad52-4c0a-9093-0a6f71e39676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:55.696515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:55.717503 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3852/7669] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:55.720133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c008a-6b25-44cc-badf-cb2f950163c6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:55.720620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c008a-6b25-44cc-badf-cb2f950163c6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:55.720724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c008a-6b25-44cc-badf-cb2f950163c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:55.742460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4a5c008a-6b25-44cc-badf-cb2f950163c6 None None] MFA Rules not processed for user `e95c827f3b5644fba8d7f14ea009f597`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:55.743677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4a5c008a-6b25-44cc-badf-cb2f950163c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:55.808994 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3818/7670] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:20:55.824778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c062c5-ad52-4c0a-9093-0a6f71e39676 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:55.826425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f1c062c5-ad52-4c0a-9093-0a6f71e39676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:55.908602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c062c5-ad52-4c0a-9093-0a6f71e39676 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:55.915218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece46f3-73c7-4986-90ef-53f73d8ca55d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:55.915939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece46f3-73c7-4986-90ef-53f73d8ca55d service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:55.916281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece46f3-73c7-4986-90ef-53f73d8ca55d service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:55.916614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece46f3-73c7-4986-90ef-53f73d8ca55d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:55.924983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece46f3-73c7-4986-90ef-53f73d8ca55d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:55.926594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece46f3-73c7-4986-90ef-53f73d8ca55d service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:55.936321 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3853/7671] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:57.827301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c008a-6b25-44cc-badf-cb2f950163c6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:57.829298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4a5c008a-6b25-44cc-badf-cb2f950163c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:57.838187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:57.913314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c008a-6b25-44cc-badf-cb2f950163c6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:57.919261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:57.919894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:57.920170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:57.920440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] PATH_INFO: `/v3/users/d2c105472ebb4c188ad3148a8a264543` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:57.921620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c105472ebb4c188ad3148a8a264543)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:57.923562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:57.973166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c105472ebb4c188ad3148a8a264543'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:57.986612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c105472ebb4c188ad3148a8a264543'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:57.995387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c105472ebb4c188ad3148a8a264543'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:58.003177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c105472ebb4c188ad3148a8a264543'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:58.003861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c105472ebb4c188ad3148a8a264543'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:58.011321 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3819/7672] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:20:57 2022] DELETE /identity/v3/users/d2c105472ebb4c188ad3148a8a264543 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:58.015969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b489-cbc9-4cab-9ec4-bb7a2423e903 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.016497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b489-cbc9-4cab-9ec4-bb7a2423e903 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.016601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b489-cbc9-4cab-9ec4-bb7a2423e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.036113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-eba7b489-cbc9-4cab-9ec4-bb7a2423e903 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:58.037255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eba7b489-cbc9-4cab-9ec4-bb7a2423e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:58.099283 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3854/7673] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:58.110080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:58.111659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:58.182341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab871ff-0aa3-4c02-9e87-dcdd8cc9461a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:58.187171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc980811-63f2-42ba-ba6e-38f262ffc334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:58.187786 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc980811-63f2-42ba-ba6e-38f262ffc334 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.188536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc980811-63f2-42ba-ba6e-38f262ffc334 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.188536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc980811-63f2-42ba-ba6e-38f262ffc334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.206063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc980811-63f2-42ba-ba6e-38f262ffc334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:58.207673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc980811-63f2-42ba-ba6e-38f262ffc334 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:58.218732 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3820/7674] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:20:58.425610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7b489-cbc9-4cab-9ec4-bb7a2423e903 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:58.426570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eba7b489-cbc9-4cab-9ec4-bb7a2423e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:58.439509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7b489-cbc9-4cab-9ec4-bb7a2423e903 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:58.445666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:58.446327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.446603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.446946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] PATH_INFO: `/v3/projects/8f6c4d19087b41ffbd635225329910e4` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.448255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6c4d19087b41ffbd635225329910e4)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:58.450408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:58.505490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6c4d19087b41ffbd635225329910e4', 'request_id': 'req-8c338372-90db-44a4-9af0-d69f4394593b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:58.515922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6c4d19087b41ffbd635225329910e4', 'request_id': 'req-8c338372-90db-44a4-9af0-d69f4394593b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:58.523209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6c4d19087b41ffbd635225329910e4', 'request_id': 'req-8c338372-90db-44a4-9af0-d69f4394593b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:58.534382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] The token cache is being invalidate because project 8f6c4d19087b41ffbd635225329910e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:20:58.534684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:58.535322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8c338372-90db-44a4-9af0-d69f4394593b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:20:58.536817 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3855/7675] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:20:58 2022] DELETE /identity/v3/projects/8f6c4d19087b41ffbd635225329910e4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:58.541704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ea25d-29bf-4678-9599-0186e5dd9467 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.541988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ea25d-29bf-4678-9599-0186e5dd9467 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.542291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ea25d-29bf-4678-9599-0186e5dd9467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.563595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-207ea25d-29bf-4678-9599-0186e5dd9467 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:58.564897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-207ea25d-29bf-4678-9599-0186e5dd9467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:58.588944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:58.631462 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3821/7676] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:58.639758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d566e21-5ed1-4c5f-a1c1-f2ab7aadeaa1 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.640190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d566e21-5ed1-4c5f-a1c1-f2ab7aadeaa1 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.640546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d566e21-5ed1-4c5f-a1c1-f2ab7aadeaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.649224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:58.663750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3d566e21-5ed1-4c5f-a1c1-f2ab7aadeaa1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:58.666649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3d566e21-5ed1-4c5f-a1c1-f2ab7aadeaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:58.673809 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3856/7677] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:58.678088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ea25d-29bf-4678-9599-0186e5dd9467 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:58.688330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ea25d-29bf-4678-9599-0186e5dd9467 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:58.692935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b498cb-2e1a-4377-852b-1824ab8f4d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:58.693559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b498cb-2e1a-4377-852b-1824ab8f4d5c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.693831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b498cb-2e1a-4377-852b-1824ab8f4d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.694095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b498cb-2e1a-4377-852b-1824ab8f4d5c admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.695419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b498cb-2e1a-4377-852b-1824ab8f4d5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:20:58.695878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b498cb-2e1a-4377-852b-1824ab8f4d5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:58.697631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b498cb-2e1a-4377-852b-1824ab8f4d5c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:58.709657 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3822/7678] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:20:58.714415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d566e21-5ed1-4c5f-a1c1-f2ab7aadeaa1 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:58.724155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d566e21-5ed1-4c5f-a1c1-f2ab7aadeaa1 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:58.729575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32a71d-d133-4571-9719-762b170c24d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:58.730177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32a71d-d133-4571-9719-762b170c24d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.730446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32a71d-d133-4571-9719-762b170c24d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.730708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32a71d-d133-4571-9719-762b170c24d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.731584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32a71d-d133-4571-9719-762b170c24d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:58.733487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32a71d-d133-4571-9719-762b170c24d6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:58.763652 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3857/7679] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:20:58 2022] POST /identity/v3/projects => generated 351 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:58.767934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b498cb-2e1a-4377-852b-1824ab8f4d5c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:58.778395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b498cb-2e1a-4377-852b-1824ab8f4d5c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:58.837513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d350c-7d77-4689-9704-4813fc5e62df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:58.838237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d350c-7d77-4689-9704-4813fc5e62df admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.838577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d350c-7d77-4689-9704-4813fc5e62df admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.838857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d350c-7d77-4689-9704-4813fc5e62df admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.839562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d350c-7d77-4689-9704-4813fc5e62df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:58.841589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d350c-7d77-4689-9704-4813fc5e62df admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:58.881480 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3823/7680] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:20:58 2022] POST /identity/v3/users => generated 276 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:20:58.887002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32a71d-d133-4571-9719-762b170c24d6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:58.899738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32a71d-d133-4571-9719-762b170c24d6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:58.905009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5b458-2128-45ec-96fc-8d7024faa112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:58.905314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5b458-2128-45ec-96fc-8d7024faa112 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.905691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5b458-2128-45ec-96fc-8d7024faa112 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.906032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5b458-2128-45ec-96fc-8d7024faa112 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.906750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5b458-2128-45ec-96fc-8d7024faa112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:58.909701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5b458-2128-45ec-96fc-8d7024faa112 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:58.920927 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3858/7681] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:58.925147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d350c-7d77-4689-9704-4813fc5e62df admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:58.935906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d350c-7d77-4689-9704-4813fc5e62df admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:58.940768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215c1c4-2d85-4066-94cf-f4c2344770d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:58.941407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215c1c4-2d85-4066-94cf-f4c2344770d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.941684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215c1c4-2d85-4066-94cf-f4c2344770d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.941952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215c1c4-2d85-4066-94cf-f4c2344770d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.942588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215c1c4-2d85-4066-94cf-f4c2344770d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:58.944242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215c1c4-2d85-4066-94cf-f4c2344770d7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:58.955920 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3824/7682] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:20:58.960173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5b458-2128-45ec-96fc-8d7024faa112 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:58.971137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5b458-2128-45ec-96fc-8d7024faa112 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:58.976993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc087603-fc65-4acc-8828-6fa51715709b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:58.977750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc087603-fc65-4acc-8828-6fa51715709b admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:58.978191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc087603-fc65-4acc-8828-6fa51715709b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:58.978771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc087603-fc65-4acc-8828-6fa51715709b admin admin] PATH_INFO: `/v3/projects/e9f0763f1a4c47c0830d6ae18c14a22d/users/fa1a710c761d4167949fc92b2265a56e/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:58.998565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc087603-fc65-4acc-8828-6fa51715709b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f0763f1a4c47c0830d6ae18c14a22d, user_id=fa1a710c761d4167949fc92b2265a56e, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:59.001593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc087603-fc65-4acc-8828-6fa51715709b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:59.016578 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3859/7683] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:20:58 2022] PUT /identity/v3/projects/e9f0763f1a4c47c0830d6ae18c14a22d/users/fa1a710c761d4167949fc92b2265a56e/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:20:59.028822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddc162-fc61-4c14-b46f-29e192dac7e0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:59.029219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddc162-fc61-4c14-b46f-29e192dac7e0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:59.029578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddc162-fc61-4c14-b46f-29e192dac7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:59.061142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a4ddc162-fc61-4c14-b46f-29e192dac7e0 None None] MFA Rules not processed for user `fa1a710c761d4167949fc92b2265a56e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:20:59.062541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a4ddc162-fc61-4c14-b46f-29e192dac7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:59.131422 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3825/7684] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:20:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:20:59.141833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc087603-fc65-4acc-8828-6fa51715709b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:20:59.145237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bc087603-fc65-4acc-8828-6fa51715709b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:20:59.218457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc087603-fc65-4acc-8828-6fa51715709b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:20:59.222244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a22ca-fc86-4d54-90cd-d58e7cac7813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:20:59.222843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a22ca-fc86-4d54-90cd-d58e7cac7813 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:20:59.223125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a22ca-fc86-4d54-90cd-d58e7cac7813 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:20:59.223481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a22ca-fc86-4d54-90cd-d58e7cac7813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:20:59.233107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a22ca-fc86-4d54-90cd-d58e7cac7813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:20:59.234407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a22ca-fc86-4d54-90cd-d58e7cac7813 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:20:59.245103 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3860/7685] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:20:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:00.829965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddc162-fc61-4c14-b46f-29e192dac7e0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:00.858721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:00.899519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddc162-fc61-4c14-b46f-29e192dac7e0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:00.904052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:00.904682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:00.904962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:00.905257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] PATH_INFO: `/v3/users/fa1a710c761d4167949fc92b2265a56e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:00.906641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa1a710c761d4167949fc92b2265a56e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:00.908304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:00.956178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa1a710c761d4167949fc92b2265a56e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:00.968867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa1a710c761d4167949fc92b2265a56e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:00.976423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa1a710c761d4167949fc92b2265a56e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:00.984187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa1a710c761d4167949fc92b2265a56e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:00.984916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa1a710c761d4167949fc92b2265a56e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:00.990861 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3826/7686] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:00 2022] DELETE /identity/v3/users/fa1a710c761d4167949fc92b2265a56e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:00.996965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d0aed-83c2-4f43-8e87-734f40636736 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:00.997864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d0aed-83c2-4f43-8e87-734f40636736 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:00.998296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d0aed-83c2-4f43-8e87-734f40636736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.018873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f90d0aed-83c2-4f43-8e87-734f40636736 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:01.020075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f90d0aed-83c2-4f43-8e87-734f40636736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:01.080600 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3861/7687] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:01.091032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:01.092367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:01.151812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108e6cb-c5ea-44a7-bd2c-1d11d1575473 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:01.155517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8855957-0240-402f-b143-ae6c283e7aeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:01.156108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8855957-0240-402f-b143-ae6c283e7aeb service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.156384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8855957-0240-402f-b143-ae6c283e7aeb service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.156653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8855957-0240-402f-b143-ae6c283e7aeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.166175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8855957-0240-402f-b143-ae6c283e7aeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:01.167604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8855957-0240-402f-b143-ae6c283e7aeb service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:01.176990 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3827/7688] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:01.355331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d0aed-83c2-4f43-8e87-734f40636736 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:01.358115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f90d0aed-83c2-4f43-8e87-734f40636736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:01.371384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d0aed-83c2-4f43-8e87-734f40636736 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:01.377682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:01.377682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.377682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.377682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] PATH_INFO: `/v3/projects/e9f0763f1a4c47c0830d6ae18c14a22d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.379010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f0763f1a4c47c0830d6ae18c14a22d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:01.380936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:01.423694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:01.433981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f0763f1a4c47c0830d6ae18c14a22d', 'request_id': 'req-1979c11b-1d8c-491e-873a-82ff64f00165'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:01.442658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f0763f1a4c47c0830d6ae18c14a22d', 'request_id': 'req-1979c11b-1d8c-491e-873a-82ff64f00165'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:01.450095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f0763f1a4c47c0830d6ae18c14a22d', 'request_id': 'req-1979c11b-1d8c-491e-873a-82ff64f00165'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:01.461813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] The token cache is being invalidate because project e9f0763f1a4c47c0830d6ae18c14a22d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:01.462095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:01.462788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1979c11b-1d8c-491e-873a-82ff64f00165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:01.464311 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3862/7689] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:01 2022] DELETE /identity/v3/projects/e9f0763f1a4c47c0830d6ae18c14a22d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:01.469357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef29c4-f888-4bb9-b3ce-bad2df310a81 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.469642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef29c4-f888-4bb9-b3ce-bad2df310a81 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.469911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef29c4-f888-4bb9-b3ce-bad2df310a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.493575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f6ef29c4-f888-4bb9-b3ce-bad2df310a81 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:01.494798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f6ef29c4-f888-4bb9-b3ce-bad2df310a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:01.541169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:01.557562 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3828/7690] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:01.565471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eed399-91d1-418e-88f3-7b85b321bf56 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.565750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eed399-91d1-418e-88f3-7b85b321bf56 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.566019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eed399-91d1-418e-88f3-7b85b321bf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.586923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c8eed399-91d1-418e-88f3-7b85b321bf56 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:01.588109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c8eed399-91d1-418e-88f3-7b85b321bf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:01.596138 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3863/7691] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:01.600235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef29c4-f888-4bb9-b3ce-bad2df310a81 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:01.610771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef29c4-f888-4bb9-b3ce-bad2df310a81 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:01.615250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e3c1f-60f9-4938-b153-f398b6db9773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:01.615826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e3c1f-60f9-4938-b153-f398b6db9773 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.616097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e3c1f-60f9-4938-b153-f398b6db9773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.616474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e3c1f-60f9-4938-b153-f398b6db9773 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.617231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e3c1f-60f9-4938-b153-f398b6db9773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:21:01.617528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e3c1f-60f9-4938-b153-f398b6db9773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:01.619104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e3c1f-60f9-4938-b153-f398b6db9773 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:01.630482 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3829/7692] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:21:01.635227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eed399-91d1-418e-88f3-7b85b321bf56 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:01.644511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eed399-91d1-418e-88f3-7b85b321bf56 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:01.650167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62845cb7-6128-450c-b7ef-7bf37e8dd887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:01.650751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62845cb7-6128-450c-b7ef-7bf37e8dd887 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.651042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62845cb7-6128-450c-b7ef-7bf37e8dd887 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.651312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62845cb7-6128-450c-b7ef-7bf37e8dd887 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.651928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62845cb7-6128-450c-b7ef-7bf37e8dd887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:01.653843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62845cb7-6128-450c-b7ef-7bf37e8dd887 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:01.681617 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3864/7693] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:01 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:01.685754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e3c1f-60f9-4938-b153-f398b6db9773 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:01.696279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e3c1f-60f9-4938-b153-f398b6db9773 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:01.752196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de1b99-2378-4976-9c84-4c9b1620a98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:01.752781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de1b99-2378-4976-9c84-4c9b1620a98b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.753072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de1b99-2378-4976-9c84-4c9b1620a98b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.753350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de1b99-2378-4976-9c84-4c9b1620a98b admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.753960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de1b99-2378-4976-9c84-4c9b1620a98b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:01.755847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de1b99-2378-4976-9c84-4c9b1620a98b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:01.777098 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3830/7694] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:01 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:01.781307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62845cb7-6128-450c-b7ef-7bf37e8dd887 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:01.791979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62845cb7-6128-450c-b7ef-7bf37e8dd887 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:01.796779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0d15f-8699-4be0-84d5-904d1b420ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:01.797409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d15f-8699-4be0-84d5-904d1b420ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.797709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d15f-8699-4be0-84d5-904d1b420ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.798001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d15f-8699-4be0-84d5-904d1b420ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.798623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0d15f-8699-4be0-84d5-904d1b420ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:01.800224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0d15f-8699-4be0-84d5-904d1b420ff2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:01.810091 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3865/7695] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:01.814138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de1b99-2378-4976-9c84-4c9b1620a98b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:01.824716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de1b99-2378-4976-9c84-4c9b1620a98b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:01.829388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073321c4-ad52-4be5-9397-6757939a7ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:01.829965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073321c4-ad52-4be5-9397-6757939a7ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.830239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073321c4-ad52-4be5-9397-6757939a7ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.830871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073321c4-ad52-4be5-9397-6757939a7ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.831158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073321c4-ad52-4be5-9397-6757939a7ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:01.832735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073321c4-ad52-4be5-9397-6757939a7ed7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:01.843769 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3831/7696] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:01.847943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0d15f-8699-4be0-84d5-904d1b420ff2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:01.858208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0d15f-8699-4be0-84d5-904d1b420ff2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:01.862821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0ed59-a7b2-4a57-8743-e144a72fb08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:01.863455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0ed59-a7b2-4a57-8743-e144a72fb08e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.863728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0ed59-a7b2-4a57-8743-e144a72fb08e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.863996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0ed59-a7b2-4a57-8743-e144a72fb08e admin admin] PATH_INFO: `/v3/projects/f0ba4ae98ca84f0599d7559ffcde62c0/users/30fadcd2de9341cab4fe73670fc52c79/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.878735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0ed59-a7b2-4a57-8743-e144a72fb08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ba4ae98ca84f0599d7559ffcde62c0, user_id=30fadcd2de9341cab4fe73670fc52c79, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:01.881729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0ed59-a7b2-4a57-8743-e144a72fb08e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:01.894922 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3866/7697] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:01 2022] PUT /identity/v3/projects/f0ba4ae98ca84f0599d7559ffcde62c0/users/30fadcd2de9341cab4fe73670fc52c79/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:01.905912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121ca17-bcf9-4194-a077-ff6610c40f94 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:01.906189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121ca17-bcf9-4194-a077-ff6610c40f94 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:01.906454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121ca17-bcf9-4194-a077-ff6610c40f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:01.933997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e121ca17-bcf9-4194-a077-ff6610c40f94 None None] MFA Rules not processed for user `30fadcd2de9341cab4fe73670fc52c79`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:01.935186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e121ca17-bcf9-4194-a077-ff6610c40f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:01.997559 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3832/7698] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:02.002090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0ed59-a7b2-4a57-8743-e144a72fb08e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:02.064363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0ed59-a7b2-4a57-8743-e144a72fb08e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:02.069094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e909d-1132-4e83-9ec6-b5781a91a63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:02.070267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e909d-1132-4e83-9ec6-b5781a91a63e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:02.070690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e909d-1132-4e83-9ec6-b5781a91a63e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:02.070959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e909d-1132-4e83-9ec6-b5781a91a63e admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:02.071599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e909d-1132-4e83-9ec6-b5781a91a63e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:02.073577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e909d-1132-4e83-9ec6-b5781a91a63e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:02.099688 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3867/7699] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:02 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:02.103663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121ca17-bcf9-4194-a077-ff6610c40f94 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:02.114761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121ca17-bcf9-4194-a077-ff6610c40f94 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:02.152664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:02.172003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a6c73-2f68-4acb-b218-63bf79e7a084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:02.172601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a6c73-2f68-4acb-b218-63bf79e7a084 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:02.172874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a6c73-2f68-4acb-b218-63bf79e7a084 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:02.173213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a6c73-2f68-4acb-b218-63bf79e7a084 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:02.173813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a6c73-2f68-4acb-b218-63bf79e7a084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:02.176235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a6c73-2f68-4acb-b218-63bf79e7a084 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:02.202284 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3833/7700] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:02 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:02.203656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e909d-1132-4e83-9ec6-b5781a91a63e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:02.214411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e909d-1132-4e83-9ec6-b5781a91a63e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:02.218988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b184d-e44b-48a9-96df-2006e1ec8caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:02.219572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b184d-e44b-48a9-96df-2006e1ec8caa admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:02.219844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b184d-e44b-48a9-96df-2006e1ec8caa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:02.220107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b184d-e44b-48a9-96df-2006e1ec8caa admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:02.220760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b184d-e44b-48a9-96df-2006e1ec8caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:02.222352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b184d-e44b-48a9-96df-2006e1ec8caa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:02.234393 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3868/7701] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:02.238660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a6c73-2f68-4acb-b218-63bf79e7a084 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:02.249280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a6c73-2f68-4acb-b218-63bf79e7a084 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:02.253956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ff244-a287-48c1-9fc7-dbfa59718b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:02.254587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ff244-a287-48c1-9fc7-dbfa59718b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:02.254861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ff244-a287-48c1-9fc7-dbfa59718b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:02.255123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ff244-a287-48c1-9fc7-dbfa59718b9c admin admin] PATH_INFO: `/v3/projects/fbb912ce82cf4939ade493d5c6496c55/users/ccc2492582144cacb8d0b9c4c093e815/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:02.270308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ff244-a287-48c1-9fc7-dbfa59718b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb912ce82cf4939ade493d5c6496c55, user_id=ccc2492582144cacb8d0b9c4c093e815, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:02.272975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ff244-a287-48c1-9fc7-dbfa59718b9c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:02.286055 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3834/7702] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:02 2022] PUT /identity/v3/projects/fbb912ce82cf4939ade493d5c6496c55/users/ccc2492582144cacb8d0b9c4c093e815/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:02.290225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b184d-e44b-48a9-96df-2006e1ec8caa admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:02.300198 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b184d-e44b-48a9-96df-2006e1ec8caa admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:02.333311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:02.359832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bedb72-90d4-490f-82cc-0f1302c55ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:02.360394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bedb72-90d4-490f-82cc-0f1302c55ebf admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:02.360667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bedb72-90d4-490f-82cc-0f1302c55ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:02.360936 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bedb72-90d4-490f-82cc-0f1302c55ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:02.361672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bedb72-90d4-490f-82cc-0f1302c55ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:02.363717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bedb72-90d4-490f-82cc-0f1302c55ebf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:02.373867 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3869/7703] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:02.378154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ff244-a287-48c1-9fc7-dbfa59718b9c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:02.389357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ff244-a287-48c1-9fc7-dbfa59718b9c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:02.394059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ce905-0bda-4a7b-b7e9-e02ef27791c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:02.394682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce905-0bda-4a7b-b7e9-e02ef27791c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:02.394960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce905-0bda-4a7b-b7e9-e02ef27791c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:02.395235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce905-0bda-4a7b-b7e9-e02ef27791c2 admin admin] PATH_INFO: `/v3/projects/fbb912ce82cf4939ade493d5c6496c55/users/ccc2492582144cacb8d0b9c4c093e815/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:02.397214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ce905-0bda-4a7b-b7e9-e02ef27791c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb912ce82cf4939ade493d5c6496c55, user_id=ccc2492582144cacb8d0b9c4c093e815, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:02.399839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ce905-0bda-4a7b-b7e9-e02ef27791c2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:02.410216 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3835/7704] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:02 2022] PUT /identity/v3/projects/fbb912ce82cf4939ade493d5c6496c55/users/ccc2492582144cacb8d0b9c4c093e815/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:02.422076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47bad0-0396-4a06-89c1-adcb22ca59e4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:02.422362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47bad0-0396-4a06-89c1-adcb22ca59e4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:02.422638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47bad0-0396-4a06-89c1-adcb22ca59e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:02.453126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0e47bad0-0396-4a06-89c1-adcb22ca59e4 None None] MFA Rules not processed for user `ccc2492582144cacb8d0b9c4c093e815`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:02.454381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0e47bad0-0396-4a06-89c1-adcb22ca59e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:02.524840 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3870/7705] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:02.537204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ce905-0bda-4a7b-b7e9-e02ef27791c2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:02.539635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-727ce905-0bda-4a7b-b7e9-e02ef27791c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:02.618527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ce905-0bda-4a7b-b7e9-e02ef27791c2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:02.623238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb5622-5de2-4def-a50f-745b1a75725e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:02.623856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb5622-5de2-4def-a50f-745b1a75725e service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:02.624288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb5622-5de2-4def-a50f-745b1a75725e service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:02.624493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb5622-5de2-4def-a50f-745b1a75725e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:02.641875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb5622-5de2-4def-a50f-745b1a75725e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:02.644255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb5622-5de2-4def-a50f-745b1a75725e service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:02.655793 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3836/7706] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:02.815468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47bad0-0396-4a06-89c1-adcb22ca59e4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:02.878630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47bad0-0396-4a06-89c1-adcb22ca59e4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:02.884269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:02.884895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:02.885213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:02.885485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] PATH_INFO: `/v3/users/30fadcd2de9341cab4fe73670fc52c79` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:02.886623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30fadcd2de9341cab4fe73670fc52c79)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:02.888559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:02.937744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30fadcd2de9341cab4fe73670fc52c79'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:02.945864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30fadcd2de9341cab4fe73670fc52c79'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:02.948189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:02.960390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30fadcd2de9341cab4fe73670fc52c79'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:02.969339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30fadcd2de9341cab4fe73670fc52c79'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:02.970019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30fadcd2de9341cab4fe73670fc52c79'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:02.976377 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3871/7707] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:02 2022] DELETE /identity/v3/users/30fadcd2de9341cab4fe73670fc52c79 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:02.981318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc97eba-0aad-4341-8f6a-d407c15e9e2f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:02.981814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc97eba-0aad-4341-8f6a-d407c15e9e2f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:02.981910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc97eba-0aad-4341-8f6a-d407c15e9e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:03.001460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dcc97eba-0aad-4341-8f6a-d407c15e9e2f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:03.002649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dcc97eba-0aad-4341-8f6a-d407c15e9e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:03.060584 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3837/7708] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:03.070907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:03.072358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:03.131075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaafaf-d9e8-4e40-9495-e14aedb0d212 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:03.135489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df428ec8-e2a6-41fa-862b-b1748537fa7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:03.136095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428ec8-e2a6-41fa-862b-b1748537fa7a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:03.136366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428ec8-e2a6-41fa-862b-b1748537fa7a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:03.136630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428ec8-e2a6-41fa-862b-b1748537fa7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:03.145812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df428ec8-e2a6-41fa-862b-b1748537fa7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:03.147211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df428ec8-e2a6-41fa-862b-b1748537fa7a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:03.156696 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3872/7709] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:03.431815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc97eba-0aad-4341-8f6a-d407c15e9e2f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:03.433363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dcc97eba-0aad-4341-8f6a-d407c15e9e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:03.449326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc97eba-0aad-4341-8f6a-d407c15e9e2f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:03.455014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:03.455517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:03.455792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:03.456153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] PATH_INFO: `/v3/projects/f0ba4ae98ca84f0599d7559ffcde62c0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:03.457410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ba4ae98ca84f0599d7559ffcde62c0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:03.459439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:03.495380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:03.510184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ba4ae98ca84f0599d7559ffcde62c0', 'request_id': 'req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:03.517362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ba4ae98ca84f0599d7559ffcde62c0', 'request_id': 'req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:03.523810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ba4ae98ca84f0599d7559ffcde62c0', 'request_id': 'req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:03.534142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] The token cache is being invalidate because project f0ba4ae98ca84f0599d7559ffcde62c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:03.534425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:03.535062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:03.536644 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3838/7710] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:03 2022] DELETE /identity/v3/projects/f0ba4ae98ca84f0599d7559ffcde62c0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:03.541562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df428ec8-e2a6-41fa-862b-b1748537fa7a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:03.551427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df428ec8-e2a6-41fa-862b-b1748537fa7a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:03.609780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:03.610412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:03.610689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:03.610952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] PATH_INFO: `/v3/users/ccc2492582144cacb8d0b9c4c093e815` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:03.612051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc2492582144cacb8d0b9c4c093e815)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:03.613991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:03.656548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:03.658076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:03.663767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc2492582144cacb8d0b9c4c093e815'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:03.671858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc2492582144cacb8d0b9c4c093e815'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:03.684408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc2492582144cacb8d0b9c4c093e815'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:03.691955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc2492582144cacb8d0b9c4c093e815'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:03.692610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc2492582144cacb8d0b9c4c093e815'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:03.694084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:03.699994 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3873/7711] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:03 2022] DELETE /identity/v3/users/ccc2492582144cacb8d0b9c4c093e815 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:03.736426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4835d-cd71-46d8-b13a-d156cc0d4e4e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:03.814614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59009f-5e71-4017-a417-3fd96123ca92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:03.815230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59009f-5e71-4017-a417-3fd96123ca92 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:03.815507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59009f-5e71-4017-a417-3fd96123ca92 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:03.816235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59009f-5e71-4017-a417-3fd96123ca92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:03.817738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5d59009f-5e71-4017-a417-3fd96123ca92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:03.833078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59009f-5e71-4017-a417-3fd96123ca92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:03.834837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59009f-5e71-4017-a417-3fd96123ca92 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:03.850531 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3839/7712] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:04.031150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:04.032721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:04.098817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4751e78-e91d-414a-96aa-3dabc6d9543d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:04.104358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:04.104975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.105276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.105541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] PATH_INFO: `/v3/projects/fbb912ce82cf4939ade493d5c6496c55` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.106883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb912ce82cf4939ade493d5c6496c55)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:04.108749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:04.159960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb912ce82cf4939ade493d5c6496c55', 'request_id': 'req-9179bb85-01b6-4d92-ace7-a05be4fa5862'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:04.167646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb912ce82cf4939ade493d5c6496c55', 'request_id': 'req-9179bb85-01b6-4d92-ace7-a05be4fa5862'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:04.174143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb912ce82cf4939ade493d5c6496c55', 'request_id': 'req-9179bb85-01b6-4d92-ace7-a05be4fa5862'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:04.196458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] The token cache is being invalidate because project fbb912ce82cf4939ade493d5c6496c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:04.196743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:04.197370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9179bb85-01b6-4d92-ace7-a05be4fa5862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:04.198992 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3874/7713] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:04 2022] DELETE /identity/v3/projects/fbb912ce82cf4939ade493d5c6496c55 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:04.204052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3ef9-866b-4ea8-9aa7-79d1dc1b826d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.204449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3ef9-866b-4ea8-9aa7-79d1dc1b826d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.204449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3ef9-866b-4ea8-9aa7-79d1dc1b826d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.223684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-cffe3ef9-866b-4ea8-9aa7-79d1dc1b826d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:04.224860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cffe3ef9-866b-4ea8-9aa7-79d1dc1b826d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:04.274078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:04.291387 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3840/7714] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:21:04.299286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6ee8c-769e-4574-a94d-66f550195461 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.299604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6ee8c-769e-4574-a94d-66f550195461 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.299877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6ee8c-769e-4574-a94d-66f550195461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.319589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d5c6ee8c-769e-4574-a94d-66f550195461 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:04.320791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d5c6ee8c-769e-4574-a94d-66f550195461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:04.328756 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3875/7715] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:04.333108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe3ef9-866b-4ea8-9aa7-79d1dc1b826d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:04.343714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe3ef9-866b-4ea8-9aa7-79d1dc1b826d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:04.348963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f1752-ffb1-4589-a69d-d2e8f1533e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:04.349589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f1752-ffb1-4589-a69d-d2e8f1533e4d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.349860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f1752-ffb1-4589-a69d-d2e8f1533e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.350123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f1752-ffb1-4589-a69d-d2e8f1533e4d admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.351395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f1752-ffb1-4589-a69d-d2e8f1533e4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:21:04.351688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f1752-ffb1-4589-a69d-d2e8f1533e4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:04.353334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f1752-ffb1-4589-a69d-d2e8f1533e4d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:04.365314 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3841/7716] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:21:04.369976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6ee8c-769e-4574-a94d-66f550195461 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:04.379337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6ee8c-769e-4574-a94d-66f550195461 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:04.385237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c647c0-3e73-4e43-9801-c1389b6b9e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:04.385826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c647c0-3e73-4e43-9801-c1389b6b9e1e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.386100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c647c0-3e73-4e43-9801-c1389b6b9e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.386377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c647c0-3e73-4e43-9801-c1389b6b9e1e admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.386996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c647c0-3e73-4e43-9801-c1389b6b9e1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:04.388913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c647c0-3e73-4e43-9801-c1389b6b9e1e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:04.417661 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3876/7717] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:04 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:04.421167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f1752-ffb1-4589-a69d-d2e8f1533e4d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:04.431576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f1752-ffb1-4589-a69d-d2e8f1533e4d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:04.490119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc60e29-27dc-43a3-a330-97209bb5dc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:04.490746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc60e29-27dc-43a3-a330-97209bb5dc55 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.491019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc60e29-27dc-43a3-a330-97209bb5dc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.491565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc60e29-27dc-43a3-a330-97209bb5dc55 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.492238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc60e29-27dc-43a3-a330-97209bb5dc55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:04.494205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc60e29-27dc-43a3-a330-97209bb5dc55 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:04.515753 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3842/7718] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:04 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:04.520241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c647c0-3e73-4e43-9801-c1389b6b9e1e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:04.531328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c647c0-3e73-4e43-9801-c1389b6b9e1e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:04.536489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c6035-c1e1-426b-aae4-e9fe40da363c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:04.537155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c6035-c1e1-426b-aae4-e9fe40da363c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.537432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c6035-c1e1-426b-aae4-e9fe40da363c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.537726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c6035-c1e1-426b-aae4-e9fe40da363c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.538941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c6035-c1e1-426b-aae4-e9fe40da363c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:04.540558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c6035-c1e1-426b-aae4-e9fe40da363c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:04.550976 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3877/7719] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:21:04.555186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc60e29-27dc-43a3-a330-97209bb5dc55 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:04.565542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc60e29-27dc-43a3-a330-97209bb5dc55 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:04.570660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23fb96-248a-4018-8bf1-1f7a974dce2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:04.571249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23fb96-248a-4018-8bf1-1f7a974dce2a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.571521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23fb96-248a-4018-8bf1-1f7a974dce2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.571839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23fb96-248a-4018-8bf1-1f7a974dce2a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.572440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23fb96-248a-4018-8bf1-1f7a974dce2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:04.574070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23fb96-248a-4018-8bf1-1f7a974dce2a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:04.585889 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3843/7720] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:04.590339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c6035-c1e1-426b-aae4-e9fe40da363c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:04.601789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c6035-c1e1-426b-aae4-e9fe40da363c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:04.606767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697e56d-11f7-446e-975f-730c5e55c55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:04.607419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e56d-11f7-446e-975f-730c5e55c55a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.608160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e56d-11f7-446e-975f-730c5e55c55a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.608160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e56d-11f7-446e-975f-730c5e55c55a admin admin] PATH_INFO: `/v3/projects/8ed96672d3d344a9b00edd3377df0ad3/users/afa9c93b9e3d41d69f92909fd98a34a2/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.626409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697e56d-11f7-446e-975f-730c5e55c55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed96672d3d344a9b00edd3377df0ad3, user_id=afa9c93b9e3d41d69f92909fd98a34a2, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:04.629058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697e56d-11f7-446e-975f-730c5e55c55a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:04.645541 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3878/7721] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:04 2022] PUT /identity/v3/projects/8ed96672d3d344a9b00edd3377df0ad3/users/afa9c93b9e3d41d69f92909fd98a34a2/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:04.651107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf6846-f157-4be8-bf35-1327ce5e9dec None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.651674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf6846-f157-4be8-bf35-1327ce5e9dec None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.652001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf6846-f157-4be8-bf35-1327ce5e9dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.673005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-1daf6846-f157-4be8-bf35-1327ce5e9dec None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:04.674147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1daf6846-f157-4be8-bf35-1327ce5e9dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:04.739495 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3844/7722] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:04.749975 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697e56d-11f7-446e-975f-730c5e55c55a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:04.751773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e697e56d-11f7-446e-975f-730c5e55c55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:04.783137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:04.818260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697e56d-11f7-446e-975f-730c5e55c55a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:04.822648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608fe9c-f9ed-400d-b902-4f206e441c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:04.823289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608fe9c-f9ed-400d-b902-4f206e441c4e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:04.823565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608fe9c-f9ed-400d-b902-4f206e441c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:04.823832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608fe9c-f9ed-400d-b902-4f206e441c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:04.832884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608fe9c-f9ed-400d-b902-4f206e441c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:04.834338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608fe9c-f9ed-400d-b902-4f206e441c4e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:04.844591 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3879/7723] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:05.490462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483376a3-98b5-4592-9491-e6905c2e11e5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:05.490983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483376a3-98b5-4592-9491-e6905c2e11e5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:05.491299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483376a3-98b5-4592-9491-e6905c2e11e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:05.520506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-483376a3-98b5-4592-9491-e6905c2e11e5 None None] MFA Rules not processed for user `afa9c93b9e3d41d69f92909fd98a34a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:05.521750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-483376a3-98b5-4592-9491-e6905c2e11e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:05.568573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:05.591183 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3845/7724] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:05.602419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608fe9c-f9ed-400d-b902-4f206e441c4e service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:05.603963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e608fe9c-f9ed-400d-b902-4f206e441c4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:05.668256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608fe9c-f9ed-400d-b902-4f206e441c4e service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:05.671899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b90c46-e20d-4369-8692-3f71a0b0fbc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:05.672497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b90c46-e20d-4369-8692-3f71a0b0fbc6 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:05.672783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b90c46-e20d-4369-8692-3f71a0b0fbc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:05.673060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b90c46-e20d-4369-8692-3f71a0b0fbc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:05.682476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b90c46-e20d-4369-8692-3f71a0b0fbc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:05.683912 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b90c46-e20d-4369-8692-3f71a0b0fbc6 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:05.693410 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3880/7725] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:07.253538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1786b-37e1-4859-b6fd-2a58ff7729ce None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:07.254021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1786b-37e1-4859-b6fd-2a58ff7729ce None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:07.254141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1786b-37e1-4859-b6fd-2a58ff7729ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:07.274229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f7d1786b-37e1-4859-b6fd-2a58ff7729ce None None] MFA Rules not processed for user `afa9c93b9e3d41d69f92909fd98a34a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:07.275423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f7d1786b-37e1-4859-b6fd-2a58ff7729ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:07.282295 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3846/7726] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:08.061956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b90c46-e20d-4369-8692-3f71a0b0fbc6 service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:08.063362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-10b90c46-e20d-4369-8692-3f71a0b0fbc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:08.078992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b90c46-e20d-4369-8692-3f71a0b0fbc6 service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:08.083643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063f39c-5cc0-4487-bbea-e94a9aa0ec78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:08.084564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063f39c-5cc0-4487-bbea-e94a9aa0ec78 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:08.085360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063f39c-5cc0-4487-bbea-e94a9aa0ec78 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:08.085625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063f39c-5cc0-4487-bbea-e94a9aa0ec78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:08.087552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a063f39c-5cc0-4487-bbea-e94a9aa0ec78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:08.164593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063f39c-5cc0-4487-bbea-e94a9aa0ec78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:08.166133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063f39c-5cc0-4487-bbea-e94a9aa0ec78 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:08.177614 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3881/7727] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:08.454746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1786b-37e1-4859-b6fd-2a58ff7729ce None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:08.456197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f7d1786b-37e1-4859-b6fd-2a58ff7729ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:08.539123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1786b-37e1-4859-b6fd-2a58ff7729ce None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:08.548479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f57ef5-65a8-4700-821c-18bfbf21c74d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:08.549126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f57ef5-65a8-4700-821c-18bfbf21c74d service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:08.549407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f57ef5-65a8-4700-821c-18bfbf21c74d service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:08.549676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f57ef5-65a8-4700-821c-18bfbf21c74d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:08.551446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-47f57ef5-65a8-4700-821c-18bfbf21c74d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:08.567893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f57ef5-65a8-4700-821c-18bfbf21c74d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:08.570209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f57ef5-65a8-4700-821c-18bfbf21c74d service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:08.585329 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3847/7728] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:18.172830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=71 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:18.577596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=41 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:25.933770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d54b42-66cd-4f52-bcb4-12d67e137f7c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:25.935309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d54b42-66cd-4f52-bcb4-12d67e137f7c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:25.935309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d54b42-66cd-4f52-bcb4-12d67e137f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:25.959440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-12d54b42-66cd-4f52-bcb4-12d67e137f7c None None] MFA Rules not processed for user `afa9c93b9e3d41d69f92909fd98a34a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:25.960680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-12d54b42-66cd-4f52-bcb4-12d67e137f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:25.967560 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3882/7729] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:28.975464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f57ef5-65a8-4700-821c-18bfbf21c74d service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:28.988267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f57ef5-65a8-4700-821c-18bfbf21c74d service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:28.992132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df0daf-8ea4-4d0d-8f8d-303683a5bbf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:28.992741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df0daf-8ea4-4d0d-8f8d-303683a5bbf2 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:28.993014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df0daf-8ea4-4d0d-8f8d-303683a5bbf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:28.993312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df0daf-8ea4-4d0d-8f8d-303683a5bbf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:28.995082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-67df0daf-8ea4-4d0d-8f8d-303683a5bbf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:29.008944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df0daf-8ea4-4d0d-8f8d-303683a5bbf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:29.010427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df0daf-8ea4-4d0d-8f8d-303683a5bbf2 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:29.022116 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3848/7730] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:29.648654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d54b42-66cd-4f52-bcb4-12d67e137f7c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:29.650145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-12d54b42-66cd-4f52-bcb4-12d67e137f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:29.664909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d54b42-66cd-4f52-bcb4-12d67e137f7c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:29.669537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:29.670628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:29.670910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:29.671409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] PATH_INFO: `/v3/users/e95c827f3b5644fba8d7f14ea009f597` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:29.672556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e95c827f3b5644fba8d7f14ea009f597)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:29.674897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:29.729597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e95c827f3b5644fba8d7f14ea009f597'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:29.737566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e95c827f3b5644fba8d7f14ea009f597'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:29.753030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e95c827f3b5644fba8d7f14ea009f597'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:29.762746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e95c827f3b5644fba8d7f14ea009f597'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:29.763405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e95c827f3b5644fba8d7f14ea009f597'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:29.771195 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3883/7731] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:29 2022] DELETE /identity/v3/users/e95c827f3b5644fba8d7f14ea009f597 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:30.149095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df0daf-8ea4-4d0d-8f8d-303683a5bbf2 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:30.162777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df0daf-8ea4-4d0d-8f8d-303683a5bbf2 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:30.285227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:30.287461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:30.288826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:30.289391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] PATH_INFO: `/v3/projects/3877567fc65349baadbfa1af9e8f5d15` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:30.291942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3877567fc65349baadbfa1af9e8f5d15)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:30.296906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:30.368443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3877567fc65349baadbfa1af9e8f5d15', 'request_id': 'req-65d66f8f-f3c8-499d-8fa0-31a43b257553'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:30.376045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3877567fc65349baadbfa1af9e8f5d15', 'request_id': 'req-65d66f8f-f3c8-499d-8fa0-31a43b257553'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:30.383362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3877567fc65349baadbfa1af9e8f5d15', 'request_id': 'req-65d66f8f-f3c8-499d-8fa0-31a43b257553'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:30.395522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] The token cache is being invalidate because project 3877567fc65349baadbfa1af9e8f5d15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:30.395880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:30.396610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:30.399028 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3849/7732] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:30 2022] DELETE /identity/v3/projects/3877567fc65349baadbfa1af9e8f5d15 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:30.403132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:30.405477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:30.420053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1de50-622e-422d-85fe-cdfe091aa3d8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:30.494952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:30.496018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:30.496524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:30.497160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] PATH_INFO: `/v3/users/842a24501b924da6b3bda9ca39ab19dc` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:30.498828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=842a24501b924da6b3bda9ca39ab19dc)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:30.501157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:30.572326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842a24501b924da6b3bda9ca39ab19dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:30.580522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842a24501b924da6b3bda9ca39ab19dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:30.591383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:30.602614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842a24501b924da6b3bda9ca39ab19dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:30.613516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842a24501b924da6b3bda9ca39ab19dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:30.615247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-64dd7845-b355-4d60-82a6-b8f4ee81a3ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842a24501b924da6b3bda9ca39ab19dc'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:30.626824 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3884/7733] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:30 2022] DELETE /identity/v3/users/842a24501b924da6b3bda9ca39ab19dc => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:30.894627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:30.896959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:30.935985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d66f8f-f3c8-499d-8fa0-31a43b257553 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:31.043449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:31.044196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:31.044750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:31.045268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] PATH_INFO: `/v3/projects/928b52e8bba9454f84abbcb9cf4cf1a6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:31.046755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=928b52e8bba9454f84abbcb9cf4cf1a6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:31.050447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:31.093394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:31.133456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '928b52e8bba9454f84abbcb9cf4cf1a6', 'request_id': 'req-a036b010-faab-41aa-8e00-26ffa0f842f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:31.149180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '928b52e8bba9454f84abbcb9cf4cf1a6', 'request_id': 'req-a036b010-faab-41aa-8e00-26ffa0f842f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:31.162334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '928b52e8bba9454f84abbcb9cf4cf1a6', 'request_id': 'req-a036b010-faab-41aa-8e00-26ffa0f842f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:31.183922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] The token cache is being invalidate because project 928b52e8bba9454f84abbcb9cf4cf1a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:31.184367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:31.185119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a036b010-faab-41aa-8e00-26ffa0f842f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:31.191318 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3850/7734] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:30 2022] DELETE /identity/v3/projects/928b52e8bba9454f84abbcb9cf4cf1a6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:31.193402 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0b4c3-ba7c-4990-b7e7-ba3171cdfa5b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:31.195077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0b4c3-ba7c-4990-b7e7-ba3171cdfa5b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:31.195461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0b4c3-ba7c-4990-b7e7-ba3171cdfa5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:31.224040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-71a0b4c3-ba7c-4990-b7e7-ba3171cdfa5b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:31.225830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-71a0b4c3-ba7c-4990-b7e7-ba3171cdfa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:31.325233 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3885/7735] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:21:31.332672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461957f4-8c92-4c78-8827-7955d8bf91d2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:31.332947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461957f4-8c92-4c78-8827-7955d8bf91d2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:31.333957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461957f4-8c92-4c78-8827-7955d8bf91d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:31.364237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-461957f4-8c92-4c78-8827-7955d8bf91d2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:31.365946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-461957f4-8c92-4c78-8827-7955d8bf91d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:31.376326 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3851/7736] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:31.381668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0b4c3-ba7c-4990-b7e7-ba3171cdfa5b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:31.393487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0b4c3-ba7c-4990-b7e7-ba3171cdfa5b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:31.411321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cb237-0099-46e6-8a13-69cd1f9163c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:31.412704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cb237-0099-46e6-8a13-69cd1f9163c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:31.413011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cb237-0099-46e6-8a13-69cd1f9163c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:31.413447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cb237-0099-46e6-8a13-69cd1f9163c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:31.414103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cb237-0099-46e6-8a13-69cd1f9163c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:21:31.414405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cb237-0099-46e6-8a13-69cd1f9163c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:31.416168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cb237-0099-46e6-8a13-69cd1f9163c3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:31.429654 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3886/7737] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:21:31.433729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461957f4-8c92-4c78-8827-7955d8bf91d2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:31.445534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461957f4-8c92-4c78-8827-7955d8bf91d2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:31.452192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e92f9d-ab02-4162-8efc-556ef84c7e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:31.452604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e92f9d-ab02-4162-8efc-556ef84c7e9e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:31.452946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e92f9d-ab02-4162-8efc-556ef84c7e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:31.453769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e92f9d-ab02-4162-8efc-556ef84c7e9e admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:31.453913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e92f9d-ab02-4162-8efc-556ef84c7e9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:31.455856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e92f9d-ab02-4162-8efc-556ef84c7e9e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:31.498918 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3852/7738] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:31 2022] POST /identity/v3/projects => generated 393 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:21:31.506057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cb237-0099-46e6-8a13-69cd1f9163c3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:31.543352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cb237-0099-46e6-8a13-69cd1f9163c3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:31.671007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1005b-baeb-4088-b8fe-5ff5c1a208b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:31.671742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1005b-baeb-4088-b8fe-5ff5c1a208b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:31.672117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1005b-baeb-4088-b8fe-5ff5c1a208b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:31.672482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1005b-baeb-4088-b8fe-5ff5c1a208b2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:31.673319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1005b-baeb-4088-b8fe-5ff5c1a208b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:31.676221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1005b-baeb-4088-b8fe-5ff5c1a208b2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:31.713435 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3887/7739] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:31 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:31.717680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e92f9d-ab02-4162-8efc-556ef84c7e9e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:31.735172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e92f9d-ab02-4162-8efc-556ef84c7e9e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:31.745833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46256a78-947c-4754-8af3-d317582fed0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:31.746473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46256a78-947c-4754-8af3-d317582fed0e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:31.746820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46256a78-947c-4754-8af3-d317582fed0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:31.747151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46256a78-947c-4754-8af3-d317582fed0e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:31.747924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46256a78-947c-4754-8af3-d317582fed0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:31.749597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46256a78-947c-4754-8af3-d317582fed0e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:31.769548 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3853/7740] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:31.772546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1005b-baeb-4088-b8fe-5ff5c1a208b2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:31.785643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1005b-baeb-4088-b8fe-5ff5c1a208b2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:31.791530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15891c18-6992-4a89-83ce-3a94344b024f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:31.792264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15891c18-6992-4a89-83ce-3a94344b024f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:31.792742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15891c18-6992-4a89-83ce-3a94344b024f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:31.793017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15891c18-6992-4a89-83ce-3a94344b024f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:31.793753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15891c18-6992-4a89-83ce-3a94344b024f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:31.796345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15891c18-6992-4a89-83ce-3a94344b024f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:31.814424 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3888/7741] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:31.815617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46256a78-947c-4754-8af3-d317582fed0e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:31.829424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46256a78-947c-4754-8af3-d317582fed0e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:31.846063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72874c10-a3cf-4097-9a12-02b2f6601f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:31.846767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72874c10-a3cf-4097-9a12-02b2f6601f72 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:31.847047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72874c10-a3cf-4097-9a12-02b2f6601f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:31.847327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72874c10-a3cf-4097-9a12-02b2f6601f72 admin admin] PATH_INFO: `/v3/projects/cdd706928da546cf8e5b0fcbf6ec94bd/users/a0d6261b46824acf958ef749425ae421/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:31.871984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72874c10-a3cf-4097-9a12-02b2f6601f72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdd706928da546cf8e5b0fcbf6ec94bd, user_id=a0d6261b46824acf958ef749425ae421, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:31.874962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72874c10-a3cf-4097-9a12-02b2f6601f72 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:31.901220 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3854/7742] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:31 2022] PUT /identity/v3/projects/cdd706928da546cf8e5b0fcbf6ec94bd/users/a0d6261b46824acf958ef749425ae421/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:21:31.902649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bcb3f-99eb-458c-a8a3-f5f94e0eb3ff None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:31.902945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bcb3f-99eb-458c-a8a3-f5f94e0eb3ff None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:31.903209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bcb3f-99eb-458c-a8a3-f5f94e0eb3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:31.935601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1a1bcb3f-99eb-458c-a8a3-f5f94e0eb3ff None None] MFA Rules not processed for user `a0d6261b46824acf958ef749425ae421`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:31.936844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1a1bcb3f-99eb-458c-a8a3-f5f94e0eb3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:31.980969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:32.008961 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3889/7743] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:21:32.013418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72874c10-a3cf-4097-9a12-02b2f6601f72 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:32.083213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72874c10-a3cf-4097-9a12-02b2f6601f72 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:32.089613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a1de0-7d04-43dd-8cfa-30c8843d5f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:32.090400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a1de0-7d04-43dd-8cfa-30c8843d5f7b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:32.090627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a1de0-7d04-43dd-8cfa-30c8843d5f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:32.091027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a1de0-7d04-43dd-8cfa-30c8843d5f7b admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:32.091670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a1de0-7d04-43dd-8cfa-30c8843d5f7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:32.093621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a1de0-7d04-43dd-8cfa-30c8843d5f7b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:32.124520 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3855/7744] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:32 2022] POST /identity/v3/projects => generated 393 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:32.128187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bcb3f-99eb-458c-a8a3-f5f94e0eb3ff None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:32.141315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bcb3f-99eb-458c-a8a3-f5f94e0eb3ff None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:32.234990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110befd-25ed-4883-80c9-c2db45b0604b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:32.236889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110befd-25ed-4883-80c9-c2db45b0604b admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:32.237378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110befd-25ed-4883-80c9-c2db45b0604b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:32.237855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110befd-25ed-4883-80c9-c2db45b0604b admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:32.239143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110befd-25ed-4883-80c9-c2db45b0604b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:32.244786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110befd-25ed-4883-80c9-c2db45b0604b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:32.324676 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3890/7745] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:32 2022] POST /identity/v3/users => generated 303 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:32.325372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a1de0-7d04-43dd-8cfa-30c8843d5f7b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:32.336937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a1de0-7d04-43dd-8cfa-30c8843d5f7b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:32.343383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178af6f5-dd85-49e2-972f-7b969d8c2661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:32.344410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178af6f5-dd85-49e2-972f-7b969d8c2661 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:32.344689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178af6f5-dd85-49e2-972f-7b969d8c2661 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:32.345015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178af6f5-dd85-49e2-972f-7b969d8c2661 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:32.346312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178af6f5-dd85-49e2-972f-7b969d8c2661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:32.351328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178af6f5-dd85-49e2-972f-7b969d8c2661 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:32.383933 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3856/7746] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:32.388405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110befd-25ed-4883-80c9-c2db45b0604b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:32.402208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110befd-25ed-4883-80c9-c2db45b0604b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:32.407596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a056d4-2506-42d8-9c85-86408cc86f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:32.408367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a056d4-2506-42d8-9c85-86408cc86f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:32.408711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a056d4-2506-42d8-9c85-86408cc86f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:32.409257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a056d4-2506-42d8-9c85-86408cc86f7b admin admin] PATH_INFO: `/v3/projects/4405aa20ee8a4987878be13acb9b0029/users/4641b6af6bf940439be4f38054c6f6fe/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:32.443462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a056d4-2506-42d8-9c85-86408cc86f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4405aa20ee8a4987878be13acb9b0029, user_id=4641b6af6bf940439be4f38054c6f6fe, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:32.447394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a056d4-2506-42d8-9c85-86408cc86f7b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:32.474195 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3891/7747] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:32 2022] PUT /identity/v3/projects/4405aa20ee8a4987878be13acb9b0029/users/4641b6af6bf940439be4f38054c6f6fe/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:32.474448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178af6f5-dd85-49e2-972f-7b969d8c2661 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:32.493383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178af6f5-dd85-49e2-972f-7b969d8c2661 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:32.513020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:32.576091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82910857-831b-4cde-b14e-de326d24a35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:32.577795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82910857-831b-4cde-b14e-de326d24a35e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:32.577795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82910857-831b-4cde-b14e-de326d24a35e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:32.577795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82910857-831b-4cde-b14e-de326d24a35e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:32.578973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82910857-831b-4cde-b14e-de326d24a35e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:32.582783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82910857-831b-4cde-b14e-de326d24a35e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:32.612782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a056d4-2506-42d8-9c85-86408cc86f7b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:32.613656 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3857/7748] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:32.653346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a056d4-2506-42d8-9c85-86408cc86f7b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:32.658332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f7cde-3038-4e2b-b9e0-119c7a6e9048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:32.659005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f7cde-3038-4e2b-b9e0-119c7a6e9048 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:32.659280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f7cde-3038-4e2b-b9e0-119c7a6e9048 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:32.659546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f7cde-3038-4e2b-b9e0-119c7a6e9048 admin admin] PATH_INFO: `/v3/projects/4405aa20ee8a4987878be13acb9b0029/users/4641b6af6bf940439be4f38054c6f6fe/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:32.662921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f7cde-3038-4e2b-b9e0-119c7a6e9048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4405aa20ee8a4987878be13acb9b0029, user_id=4641b6af6bf940439be4f38054c6f6fe, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:32.665722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f7cde-3038-4e2b-b9e0-119c7a6e9048 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:32.681751 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3892/7749] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:32 2022] PUT /identity/v3/projects/4405aa20ee8a4987878be13acb9b0029/users/4641b6af6bf940439be4f38054c6f6fe/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:32.692978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db81578-fede-4f20-a36b-f2a28719d629 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:32.693986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db81578-fede-4f20-a36b-f2a28719d629 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:32.694251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db81578-fede-4f20-a36b-f2a28719d629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:32.743597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f7cde-3038-4e2b-b9e0-119c7a6e9048 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:32.746659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e52f7cde-3038-4e2b-b9e0-119c7a6e9048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:32.771735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7db81578-fede-4f20-a36b-f2a28719d629 None None] MFA Rules not processed for user `4641b6af6bf940439be4f38054c6f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:32.772966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7db81578-fede-4f20-a36b-f2a28719d629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:32.836118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f7cde-3038-4e2b-b9e0-119c7a6e9048 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:32.841023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:32.841711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:32.841987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:32.842270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] PATH_INFO: `/v3/users/379c929d93b54b9a9751defeef161f57` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:32.843392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] RBAC: Authorizing `identity:delete_user(user_id=379c929d93b54b9a9751defeef161f57)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:32.845343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:32.870695 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3858/7750] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:21:32.886299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db81578-fede-4f20-a36b-f2a28719d629 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:32.888037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7db81578-fede-4f20-a36b-f2a28719d629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:32.901490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:32.915694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379c929d93b54b9a9751defeef161f57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:32.923381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379c929d93b54b9a9751defeef161f57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:32.936075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379c929d93b54b9a9751defeef161f57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:32.944836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379c929d93b54b9a9751defeef161f57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:32.945710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379c929d93b54b9a9751defeef161f57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:32.957258 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3893/7751] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:32 2022] DELETE /identity/v3/users/379c929d93b54b9a9751defeef161f57 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:21:32.978912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db81578-fede-4f20-a36b-f2a28719d629 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:33.030832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:33.077908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f099b-9955-4602-a720-4ee2d0047a86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:33.078663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f099b-9955-4602-a720-4ee2d0047a86 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.079044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f099b-9955-4602-a720-4ee2d0047a86 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.079400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f099b-9955-4602-a720-4ee2d0047a86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.083196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b61f099b-9955-4602-a720-4ee2d0047a86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:33.162847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:33.164267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:33.190616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f099b-9955-4602-a720-4ee2d0047a86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:33.193903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f099b-9955-4602-a720-4ee2d0047a86 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:33.207122 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3859/7752] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:21:33.248589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659f71c-358e-4d01-b0ba-804128709acd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:33.253356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:33.254089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.254450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.254719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] PATH_INFO: `/v3/projects/87333497286e48ef95cb933c074efa70` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.255880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87333497286e48ef95cb933c074efa70)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:33.257808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:33.311520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87333497286e48ef95cb933c074efa70', 'request_id': 'req-9df5d200-cc35-42c8-8ea6-2165b729c885'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:33.319467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87333497286e48ef95cb933c074efa70', 'request_id': 'req-9df5d200-cc35-42c8-8ea6-2165b729c885'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:33.326157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87333497286e48ef95cb933c074efa70', 'request_id': 'req-9df5d200-cc35-42c8-8ea6-2165b729c885'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:33.337867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] The token cache is being invalidate because project 87333497286e48ef95cb933c074efa70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:33.338160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:33.338815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9df5d200-cc35-42c8-8ea6-2165b729c885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:33.340427 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3894/7753] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:33 2022] DELETE /identity/v3/projects/87333497286e48ef95cb933c074efa70 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:21:33.346188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597818a-48eb-454f-8937-be72fd751eb4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.346720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597818a-48eb-454f-8937-be72fd751eb4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.346829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597818a-48eb-454f-8937-be72fd751eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.370487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b597818a-48eb-454f-8937-be72fd751eb4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:33.371756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b597818a-48eb-454f-8937-be72fd751eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:33.441321 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3860/7754] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:33.450358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdac0c-68c7-44d0-8221-7507dacef695 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.450736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdac0c-68c7-44d0-8221-7507dacef695 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.451124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdac0c-68c7-44d0-8221-7507dacef695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.484994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0bfdac0c-68c7-44d0-8221-7507dacef695 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:33.486503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0bfdac0c-68c7-44d0-8221-7507dacef695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:33.498367 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3895/7755] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:33.502520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597818a-48eb-454f-8937-be72fd751eb4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:33.513661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597818a-48eb-454f-8937-be72fd751eb4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:33.519221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d2194-30fa-461f-bf79-c87aa65335b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:33.519943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d2194-30fa-461f-bf79-c87aa65335b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.520448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d2194-30fa-461f-bf79-c87aa65335b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.520956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d2194-30fa-461f-bf79-c87aa65335b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.521827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d2194-30fa-461f-bf79-c87aa65335b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:21:33.522138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d2194-30fa-461f-bf79-c87aa65335b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:33.523849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d2194-30fa-461f-bf79-c87aa65335b6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:33.539665 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3861/7756] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:21:33.543738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdac0c-68c7-44d0-8221-7507dacef695 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:33.553603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdac0c-68c7-44d0-8221-7507dacef695 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:33.558739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f7cac-6b61-4c92-a087-bb6be64bdd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:33.559228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f7cac-6b61-4c92-a087-bb6be64bdd69 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.559531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f7cac-6b61-4c92-a087-bb6be64bdd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.559771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f7cac-6b61-4c92-a087-bb6be64bdd69 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.560416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f7cac-6b61-4c92-a087-bb6be64bdd69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:33.562313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f7cac-6b61-4c92-a087-bb6be64bdd69 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:33.592435 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3896/7757] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:33 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:33.596639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d2194-30fa-461f-bf79-c87aa65335b6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:33.607849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d2194-30fa-461f-bf79-c87aa65335b6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:33.676191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbf92b-6837-48a9-921b-bf639d0f87ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:33.676823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbf92b-6837-48a9-921b-bf639d0f87ff admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.678167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbf92b-6837-48a9-921b-bf639d0f87ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.678484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbf92b-6837-48a9-921b-bf639d0f87ff admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.679178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbf92b-6837-48a9-921b-bf639d0f87ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:33.681127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbf92b-6837-48a9-921b-bf639d0f87ff admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:33.690473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:33.705151 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3862/7758] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:33 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:33.709416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f7cac-6b61-4c92-a087-bb6be64bdd69 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:33.720284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f7cac-6b61-4c92-a087-bb6be64bdd69 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:33.725475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e5def-c557-4777-81a7-5a1c5a32a9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:33.726152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e5def-c557-4777-81a7-5a1c5a32a9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.726496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e5def-c557-4777-81a7-5a1c5a32a9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.726768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e5def-c557-4777-81a7-5a1c5a32a9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.727401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e5def-c557-4777-81a7-5a1c5a32a9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:33.729021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e5def-c557-4777-81a7-5a1c5a32a9a7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:33.741129 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3897/7759] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:21:33.743982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbf92b-6837-48a9-921b-bf639d0f87ff admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:33.754473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbf92b-6837-48a9-921b-bf639d0f87ff admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:33.759591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530731b-138c-4691-8582-aab975d87eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:33.760188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530731b-138c-4691-8582-aab975d87eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.760465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530731b-138c-4691-8582-aab975d87eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.760755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530731b-138c-4691-8582-aab975d87eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.761427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530731b-138c-4691-8582-aab975d87eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:33.763030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530731b-138c-4691-8582-aab975d87eb6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:33.776085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3863/7760] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:33.780201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e5def-c557-4777-81a7-5a1c5a32a9a7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:33.785622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:33.794034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e5def-c557-4777-81a7-5a1c5a32a9a7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:33.800376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21891f-3c1f-4f4a-b077-f0216816bdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:33.801046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21891f-3c1f-4f4a-b077-f0216816bdee admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.801385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21891f-3c1f-4f4a-b077-f0216816bdee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.801726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21891f-3c1f-4f4a-b077-f0216816bdee admin admin] PATH_INFO: `/v3/projects/01aeeb44218e4747b21d01776e8909f6/users/edfa2630c11d49ff8d925e71abaebf9b/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.819577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21891f-3c1f-4f4a-b077-f0216816bdee admin admin] RBAC: Authorizing `identity:create_grant(project_id=01aeeb44218e4747b21d01776e8909f6, user_id=edfa2630c11d49ff8d925e71abaebf9b, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:33.822245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21891f-3c1f-4f4a-b077-f0216816bdee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:33.836065 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3898/7761] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:33 2022] PUT /identity/v3/projects/01aeeb44218e4747b21d01776e8909f6/users/edfa2630c11d49ff8d925e71abaebf9b/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:33.846447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9deead-3da1-476d-8b54-ea79868e27f5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:33.846729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9deead-3da1-476d-8b54-ea79868e27f5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:33.846990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9deead-3da1-476d-8b54-ea79868e27f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:33.876327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-db9deead-3da1-476d-8b54-ea79868e27f5 None None] MFA Rules not processed for user `edfa2630c11d49ff8d925e71abaebf9b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:33.877623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-db9deead-3da1-476d-8b54-ea79868e27f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:33.948865 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3864/7762] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:33.960944 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21891f-3c1f-4f4a-b077-f0216816bdee admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:33.962741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f21891f-3c1f-4f4a-b077-f0216816bdee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:34.042922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21891f-3c1f-4f4a-b077-f0216816bdee admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:34.048611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e79df-b923-4481-aff6-2a355429707f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:34.049246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e79df-b923-4481-aff6-2a355429707f service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:34.049524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e79df-b923-4481-aff6-2a355429707f service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:34.049793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e79df-b923-4481-aff6-2a355429707f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:34.063692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e79df-b923-4481-aff6-2a355429707f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:34.065355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e79df-b923-4481-aff6-2a355429707f service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:34.075717 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3899/7763] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:21:36.562293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9deead-3da1-476d-8b54-ea79868e27f5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:36.563828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-db9deead-3da1-476d-8b54-ea79868e27f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:36.640662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9deead-3da1-476d-8b54-ea79868e27f5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:36.644918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e84bd-46d9-4575-bf44-6e0dc1281f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:36.645610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e84bd-46d9-4575-bf44-6e0dc1281f81 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:36.645901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e84bd-46d9-4575-bf44-6e0dc1281f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:36.646477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e84bd-46d9-4575-bf44-6e0dc1281f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:36.648370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ad6e84bd-46d9-4575-bf44-6e0dc1281f81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:36.715134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e84bd-46d9-4575-bf44-6e0dc1281f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:36.716595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e84bd-46d9-4575-bf44-6e0dc1281f81 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:36.728695 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3865/7764] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:36 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:39.635565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e79df-b923-4481-aff6-2a355429707f service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:39.754707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e79df-b923-4481-aff6-2a355429707f service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:39.763097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:39.764098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:39.764491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:39.764874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] PATH_INFO: `/v3/users/edfa2630c11d49ff8d925e71abaebf9b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:39.766449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edfa2630c11d49ff8d925e71abaebf9b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:39.770985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:39.844425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edfa2630c11d49ff8d925e71abaebf9b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:39.857330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edfa2630c11d49ff8d925e71abaebf9b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:39.885461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:39.891995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edfa2630c11d49ff8d925e71abaebf9b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:39.904148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edfa2630c11d49ff8d925e71abaebf9b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:39.905706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edfa2630c11d49ff8d925e71abaebf9b'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:39.922080 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3900/7765] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:39 2022] DELETE /identity/v3/users/edfa2630c11d49ff8d925e71abaebf9b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:39.923842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff79af0-0297-41f2-be3a-b040ed6d41cc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:39.924318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff79af0-0297-41f2-be3a-b040ed6d41cc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:39.924412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff79af0-0297-41f2-be3a-b040ed6d41cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:39.949566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8ff79af0-0297-41f2-be3a-b040ed6d41cc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:39.950906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8ff79af0-0297-41f2-be3a-b040ed6d41cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:39.986542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:40.061802 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3866/7766] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:40.075889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:40.077647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:40.153305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cbffb-438f-4d49-9449-d4bead2d1c01 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:40.157935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9049b0-840a-4237-8905-45ecc13dc254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:40.158777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9049b0-840a-4237-8905-45ecc13dc254 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:40.159255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9049b0-840a-4237-8905-45ecc13dc254 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:40.159713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9049b0-840a-4237-8905-45ecc13dc254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:40.174027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9049b0-840a-4237-8905-45ecc13dc254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:40.175460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9049b0-840a-4237-8905-45ecc13dc254 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:40.186463 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3901/7767] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:21:40.462382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd55cd9-d7ad-4483-91d1-482b15ce7c7c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:40.462790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd55cd9-d7ad-4483-91d1-482b15ce7c7c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:40.463494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd55cd9-d7ad-4483-91d1-482b15ce7c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:40.522854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9bd55cd9-d7ad-4483-91d1-482b15ce7c7c None None] MFA Rules not processed for user `a0d6261b46824acf958ef749425ae421`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:40.524076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9bd55cd9-d7ad-4483-91d1-482b15ce7c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:40.611822 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3867/7768] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:40.688353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9049b0-840a-4237-8905-45ecc13dc254 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:40.689878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8a9049b0-840a-4237-8905-45ecc13dc254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:40.716305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9049b0-840a-4237-8905-45ecc13dc254 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:40.729531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:40.730169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:40.730748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:40.730935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] PATH_INFO: `/v3/projects/01aeeb44218e4747b21d01776e8909f6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:40.732546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01aeeb44218e4747b21d01776e8909f6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:40.734570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:40.816466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01aeeb44218e4747b21d01776e8909f6', 'request_id': 'req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:40.835377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01aeeb44218e4747b21d01776e8909f6', 'request_id': 'req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:40.843024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01aeeb44218e4747b21d01776e8909f6', 'request_id': 'req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:40.874488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] The token cache is being invalidate because project 01aeeb44218e4747b21d01776e8909f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:40.874694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:40.875357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b6f481ce-fe00-48a0-959e-2c7c7dcb89c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:40.880775 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3902/7769] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:40 2022] DELETE /identity/v3/projects/01aeeb44218e4747b21d01776e8909f6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:40.882482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955b215-bd31-44c7-b57b-b47a4f409186 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:40.882783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955b215-bd31-44c7-b57b-b47a4f409186 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:40.883087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955b215-bd31-44c7-b57b-b47a4f409186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:40.914092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c955b215-bd31-44c7-b57b-b47a4f409186 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:40.916083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c955b215-bd31-44c7-b57b-b47a4f409186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:41.011907 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3868/7770] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:41.025360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a8364-fd3c-49e6-8dd5-5ee28ff9a42c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:41.025979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a8364-fd3c-49e6-8dd5-5ee28ff9a42c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:41.026435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a8364-fd3c-49e6-8dd5-5ee28ff9a42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:41.065323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-184a8364-fd3c-49e6-8dd5-5ee28ff9a42c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:41.066992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-184a8364-fd3c-49e6-8dd5-5ee28ff9a42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:41.079779 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3903/7771] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:41.085328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955b215-bd31-44c7-b57b-b47a4f409186 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:41.104394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955b215-bd31-44c7-b57b-b47a4f409186 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:41.113611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b162b3-0493-4b33-8902-b2e2d86f7c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:41.115043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b162b3-0493-4b33-8902-b2e2d86f7c88 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:41.117651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b162b3-0493-4b33-8902-b2e2d86f7c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:41.118395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b162b3-0493-4b33-8902-b2e2d86f7c88 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:41.121368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b162b3-0493-4b33-8902-b2e2d86f7c88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:21:41.121851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b162b3-0493-4b33-8902-b2e2d86f7c88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:41.125200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b162b3-0493-4b33-8902-b2e2d86f7c88 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:41.160433 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3869/7772] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:21:41.165337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a8364-fd3c-49e6-8dd5-5ee28ff9a42c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:41.179746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a8364-fd3c-49e6-8dd5-5ee28ff9a42c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:41.195318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde12cc3-3da8-4552-aa4b-8e91f970f750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:41.196398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde12cc3-3da8-4552-aa4b-8e91f970f750 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:41.196943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde12cc3-3da8-4552-aa4b-8e91f970f750 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:41.198873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde12cc3-3da8-4552-aa4b-8e91f970f750 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:41.199754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde12cc3-3da8-4552-aa4b-8e91f970f750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:41.205003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde12cc3-3da8-4552-aa4b-8e91f970f750 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:41.297888 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3904/7773] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:41 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:41.302332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b162b3-0493-4b33-8902-b2e2d86f7c88 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:41.318218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b162b3-0493-4b33-8902-b2e2d86f7c88 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:41.386706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:41.399384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fede4-0ce7-49ec-aa3b-beb3139a37cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:41.400311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fede4-0ce7-49ec-aa3b-beb3139a37cf admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:41.400757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fede4-0ce7-49ec-aa3b-beb3139a37cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:41.401256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fede4-0ce7-49ec-aa3b-beb3139a37cf admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:41.402363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fede4-0ce7-49ec-aa3b-beb3139a37cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:41.404508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fede4-0ce7-49ec-aa3b-beb3139a37cf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:41.444896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde12cc3-3da8-4552-aa4b-8e91f970f750 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:41.454695 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3870/7774] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:41 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:41.460028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde12cc3-3da8-4552-aa4b-8e91f970f750 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:41.465725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cbca1-9d35-4253-8813-f05403099ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:41.466223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cbca1-9d35-4253-8813-f05403099ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:41.466599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cbca1-9d35-4253-8813-f05403099ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:41.467038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cbca1-9d35-4253-8813-f05403099ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:41.467839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794cbca1-9d35-4253-8813-f05403099ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:41.469754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794cbca1-9d35-4253-8813-f05403099ab4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:41.498004 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3905/7775] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:41.501257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fede4-0ce7-49ec-aa3b-beb3139a37cf admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:41.512764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fede4-0ce7-49ec-aa3b-beb3139a37cf admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:41.518595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3b4b3-2e3b-4457-bc6a-c05ab138d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:41.519297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3b4b3-2e3b-4457-bc6a-c05ab138d87c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:41.519781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3b4b3-2e3b-4457-bc6a-c05ab138d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:41.520117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3b4b3-2e3b-4457-bc6a-c05ab138d87c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:41.521838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3b4b3-2e3b-4457-bc6a-c05ab138d87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:41.523849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3b4b3-2e3b-4457-bc6a-c05ab138d87c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:41.542447 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3871/7776] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:41.543261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cbca1-9d35-4253-8813-f05403099ab4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:41.559227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cbca1-9d35-4253-8813-f05403099ab4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:41.566899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f9e3b-cab2-4d09-a4b7-6551d0a79d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:41.568380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f9e3b-cab2-4d09-a4b7-6551d0a79d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:41.568727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f9e3b-cab2-4d09-a4b7-6551d0a79d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:41.569195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f9e3b-cab2-4d09-a4b7-6551d0a79d53 admin admin] PATH_INFO: `/v3/projects/8f48c35cf7e049e58cd9413b58a9b1ed/users/a4b05035bc3042d18ed6205634710a82/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:41.590386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f9e3b-cab2-4d09-a4b7-6551d0a79d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f48c35cf7e049e58cd9413b58a9b1ed, user_id=a4b05035bc3042d18ed6205634710a82, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:41.597151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f9e3b-cab2-4d09-a4b7-6551d0a79d53 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:41.639609 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3906/7777] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:41 2022] PUT /identity/v3/projects/8f48c35cf7e049e58cd9413b58a9b1ed/users/a4b05035bc3042d18ed6205634710a82/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:21:41.654693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d93521-79ef-4812-854c-60e7d9898f5f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:41.654980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d93521-79ef-4812-854c-60e7d9898f5f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:41.655253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d93521-79ef-4812-854c-60e7d9898f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:41.694772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-14d93521-79ef-4812-854c-60e7d9898f5f None None] MFA Rules not processed for user `a4b05035bc3042d18ed6205634710a82`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:41.696085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-14d93521-79ef-4812-854c-60e7d9898f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:41.854198 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3872/7778] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:41.865232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f9e3b-cab2-4d09-a4b7-6551d0a79d53 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:41.867185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cb7f9e3b-cab2-4d09-a4b7-6551d0a79d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:41.878658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:42.011077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f9e3b-cab2-4d09-a4b7-6551d0a79d53 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:42.020963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1627532-84c5-44e1-aa52-06b2c8d873ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:42.021631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627532-84c5-44e1-aa52-06b2c8d873ec service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:42.021902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627532-84c5-44e1-aa52-06b2c8d873ec service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:42.022164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627532-84c5-44e1-aa52-06b2c8d873ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:42.035091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1627532-84c5-44e1-aa52-06b2c8d873ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:42.039159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1627532-84c5-44e1-aa52-06b2c8d873ec service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:42.055820 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3907/7779] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:42.362392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d93521-79ef-4812-854c-60e7d9898f5f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:42.490313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d93521-79ef-4812-854c-60e7d9898f5f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:42.495435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:42.496173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:42.496556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:42.496963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] PATH_INFO: `/v3/users/a4b05035bc3042d18ed6205634710a82` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:42.498577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b05035bc3042d18ed6205634710a82)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:42.501744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:42.588108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b05035bc3042d18ed6205634710a82'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:42.624745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:42.626412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b05035bc3042d18ed6205634710a82'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:42.660807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b05035bc3042d18ed6205634710a82'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:42.672568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b05035bc3042d18ed6205634710a82'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:42.673385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b05035bc3042d18ed6205634710a82'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:42.681513 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3873/7780] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:42 2022] DELETE /identity/v3/users/a4b05035bc3042d18ed6205634710a82 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:42.687556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8b8a7-2f6a-4ed2-91f8-8871b0ab438d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:42.688087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8b8a7-2f6a-4ed2-91f8-8871b0ab438d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:42.688522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8b8a7-2f6a-4ed2-91f8-8871b0ab438d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:42.739879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9aa8b8a7-2f6a-4ed2-91f8-8871b0ab438d None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:42.742795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9aa8b8a7-2f6a-4ed2-91f8-8871b0ab438d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:42.834582 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3908/7781] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:42.844171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:42.845958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:42.955888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b747be-a895-480f-9384-cb3cd07752f6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:42.964771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6f79d-b210-4838-9b08-c77a608a3d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:42.965432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6f79d-b210-4838-9b08-c77a608a3d9a service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:42.965708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6f79d-b210-4838-9b08-c77a608a3d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:42.966045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6f79d-b210-4838-9b08-c77a608a3d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:42.977201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6f79d-b210-4838-9b08-c77a608a3d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:42.979135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6f79d-b210-4838-9b08-c77a608a3d9a service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:42.991104 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3874/7782] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:43.484254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8b8a7-2f6a-4ed2-91f8-8871b0ab438d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:43.486425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9aa8b8a7-2f6a-4ed2-91f8-8871b0ab438d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:43.521470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8b8a7-2f6a-4ed2-91f8-8871b0ab438d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:43.541913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:43.542632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:43.542909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:43.543415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] PATH_INFO: `/v3/projects/8f48c35cf7e049e58cd9413b58a9b1ed` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:43.544775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f48c35cf7e049e58cd9413b58a9b1ed)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:43.547154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:43.619835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f48c35cf7e049e58cd9413b58a9b1ed', 'request_id': 'req-bcf7dc9c-41fd-4e0c-8784-76294be44f85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:43.635393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f48c35cf7e049e58cd9413b58a9b1ed', 'request_id': 'req-bcf7dc9c-41fd-4e0c-8784-76294be44f85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:43.651029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f48c35cf7e049e58cd9413b58a9b1ed', 'request_id': 'req-bcf7dc9c-41fd-4e0c-8784-76294be44f85'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:43.679038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] The token cache is being invalidate because project 8f48c35cf7e049e58cd9413b58a9b1ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:43.679329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:43.680040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-bcf7dc9c-41fd-4e0c-8784-76294be44f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:43.682133 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3909/7783] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:43 2022] DELETE /identity/v3/projects/8f48c35cf7e049e58cd9413b58a9b1ed => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:43.692802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82940139-141c-4d6e-a3a1-282b2035eda7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:43.693493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82940139-141c-4d6e-a3a1-282b2035eda7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:43.693493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82940139-141c-4d6e-a3a1-282b2035eda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:43.726008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-82940139-141c-4d6e-a3a1-282b2035eda7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:43.728634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-82940139-141c-4d6e-a3a1-282b2035eda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:43.876006 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3875/7784] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:43.888617 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751b0f3-3302-4bb9-8585-95000379f6ff None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:43.888875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751b0f3-3302-4bb9-8585-95000379f6ff None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:43.889189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751b0f3-3302-4bb9-8585-95000379f6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:43.939565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8751b0f3-3302-4bb9-8585-95000379f6ff None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:43.941896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8751b0f3-3302-4bb9-8585-95000379f6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:43.970170 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3910/7785] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:43.975244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82940139-141c-4d6e-a3a1-282b2035eda7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:43.990545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82940139-141c-4d6e-a3a1-282b2035eda7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:43.996692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49312f-2fdd-4fd5-916f-b09af84e3fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:43.998615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49312f-2fdd-4fd5-916f-b09af84e3fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:43.998894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49312f-2fdd-4fd5-916f-b09af84e3fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:43.999224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49312f-2fdd-4fd5-916f-b09af84e3fb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:43.999916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49312f-2fdd-4fd5-916f-b09af84e3fb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:21:44.000224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49312f-2fdd-4fd5-916f-b09af84e3fb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:44.002997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49312f-2fdd-4fd5-916f-b09af84e3fb1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:44.019373 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3876/7786] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:21:44.024859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751b0f3-3302-4bb9-8585-95000379f6ff None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:44.036892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751b0f3-3302-4bb9-8585-95000379f6ff None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:44.058827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976e9e8-08ea-47aa-b68d-c410a7182d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:44.059482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976e9e8-08ea-47aa-b68d-c410a7182d0c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.059763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976e9e8-08ea-47aa-b68d-c410a7182d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.060035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976e9e8-08ea-47aa-b68d-c410a7182d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.060689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976e9e8-08ea-47aa-b68d-c410a7182d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:44.062817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976e9e8-08ea-47aa-b68d-c410a7182d0c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:44.074940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:44.144373 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3911/7787] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:44 2022] POST /identity/v3/projects => generated 391 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:44.147046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49312f-2fdd-4fd5-916f-b09af84e3fb1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:44.160820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49312f-2fdd-4fd5-916f-b09af84e3fb1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:44.263421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55841658-8cdb-41ee-85da-293742f76723 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.264393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55841658-8cdb-41ee-85da-293742f76723 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.264670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55841658-8cdb-41ee-85da-293742f76723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.301357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-55841658-8cdb-41ee-85da-293742f76723 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:44.303389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-55841658-8cdb-41ee-85da-293742f76723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:44.322025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605571ee-c577-4002-9a06-bae79157aeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:44.322833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605571ee-c577-4002-9a06-bae79157aeb6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.323138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605571ee-c577-4002-9a06-bae79157aeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.323410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605571ee-c577-4002-9a06-bae79157aeb6 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.325277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605571ee-c577-4002-9a06-bae79157aeb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:44.327291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605571ee-c577-4002-9a06-bae79157aeb6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:44.378639 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:44.397813 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3877/7788] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:44 2022] POST /identity/v3/users => generated 296 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:44.398817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605571ee-c577-4002-9a06-bae79157aeb6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:44.410380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605571ee-c577-4002-9a06-bae79157aeb6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:44.416835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f2a51-ebb6-4cb5-98a6-eb3ba3e95675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:44.418029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f2a51-ebb6-4cb5-98a6-eb3ba3e95675 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.418305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f2a51-ebb6-4cb5-98a6-eb3ba3e95675 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.418572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f2a51-ebb6-4cb5-98a6-eb3ba3e95675 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.419222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f2a51-ebb6-4cb5-98a6-eb3ba3e95675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:44.421383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f2a51-ebb6-4cb5-98a6-eb3ba3e95675 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:44.439765 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3878/7789] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:44.444426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f2a51-ebb6-4cb5-98a6-eb3ba3e95675 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:44.467675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f2a51-ebb6-4cb5-98a6-eb3ba3e95675 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:44.473182 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3912/7790] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:21:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:44.490606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e631c93-2118-4272-8cca-d4e3030c82af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:44.498929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e631c93-2118-4272-8cca-d4e3030c82af admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.499412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e631c93-2118-4272-8cca-d4e3030c82af admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.511577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e631c93-2118-4272-8cca-d4e3030c82af admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.512821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e631c93-2118-4272-8cca-d4e3030c82af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:44.519914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e631c93-2118-4272-8cca-d4e3030c82af admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:44.534900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55841658-8cdb-41ee-85da-293742f76723 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:44.539681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-55841658-8cdb-41ee-85da-293742f76723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:44.540600 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3879/7791] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:44.542005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e631c93-2118-4272-8cca-d4e3030c82af admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:44.561651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e631c93-2118-4272-8cca-d4e3030c82af admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:44.569558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cd4ff-d22b-46bb-9f34-3a3637a5047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:44.570783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd4ff-d22b-46bb-9f34-3a3637a5047d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.571697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd4ff-d22b-46bb-9f34-3a3637a5047d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.572349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd4ff-d22b-46bb-9f34-3a3637a5047d admin admin] PATH_INFO: `/v3/projects/d20f6b7f7df74f4994458c547cc451e5/users/7547b90326754ee7a452e76f3e088d31/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.603809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cd4ff-d22b-46bb-9f34-3a3637a5047d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d20f6b7f7df74f4994458c547cc451e5, user_id=7547b90326754ee7a452e76f3e088d31, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:44.607510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cd4ff-d22b-46bb-9f34-3a3637a5047d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:44.635756 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3880/7792] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:44 2022] PUT /identity/v3/projects/d20f6b7f7df74f4994458c547cc451e5/users/7547b90326754ee7a452e76f3e088d31/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:44.647106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a1652-68df-410d-8701-90bd8179d842 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.647356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a1652-68df-410d-8701-90bd8179d842 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.647525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a1652-68df-410d-8701-90bd8179d842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.704872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-633a1652-68df-410d-8701-90bd8179d842 None None] MFA Rules not processed for user `7547b90326754ee7a452e76f3e088d31`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:44.706180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-633a1652-68df-410d-8701-90bd8179d842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:44.734415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55841658-8cdb-41ee-85da-293742f76723 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:44.804911 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3881/7793] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:44.805979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a1652-68df-410d-8701-90bd8179d842 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:44.814863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a1652-68df-410d-8701-90bd8179d842 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:44.819381 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc58f4-6e57-43f5-a15d-75ccb5de13b6 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:44.820320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc58f4-6e57-43f5-a15d-75ccb5de13b6 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.820599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc58f4-6e57-43f5-a15d-75ccb5de13b6 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.820867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc58f4-6e57-43f5-a15d-75ccb5de13b6 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.821695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc58f4-6e57-43f5-a15d-75ccb5de13b6 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:create_application_credential(user_id=7547b90326754ee7a452e76f3e088d31)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:44.822863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc58f4-6e57-43f5-a15d-75ccb5de13b6 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:44.830193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca38fcd-9bef-4c1c-94de-6bbd9def3da6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:44.830992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca38fcd-9bef-4c1c-94de-6bbd9def3da6 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.831398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca38fcd-9bef-4c1c-94de-6bbd9def3da6 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.832414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca38fcd-9bef-4c1c-94de-6bbd9def3da6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.884081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:44.916554 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3882/7794] 10.222.0.16 () {60 vars in 1314 bytes} [Sat Oct 15 21:21:44 2022] POST /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials => generated 788 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:44.916971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca38fcd-9bef-4c1c-94de-6bbd9def3da6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:44.918783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca38fcd-9bef-4c1c-94de-6bbd9def3da6 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:44.920361 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc58f4-6e57-43f5-a15d-75ccb5de13b6 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:44.930878 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3913/7795] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:21:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:21:44.936445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc58f4-6e57-43f5-a15d-75ccb5de13b6 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:44.940929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85185c31-bb54-4ed4-80f8-37462de35ec1 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:44.941852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85185c31-bb54-4ed4-80f8-37462de35ec1 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.943181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85185c31-bb54-4ed4-80f8-37462de35ec1 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.943656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85185c31-bb54-4ed4-80f8-37462de35ec1 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/ee33fd50b4054afeaaaf5f0f409ebdee` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.955583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85185c31-bb54-4ed4-80f8-37462de35ec1 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:get_application_credential(user_id=7547b90326754ee7a452e76f3e088d31, application_credential_id=ee33fd50b4054afeaaaf5f0f409ebdee)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:44.958703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85185c31-bb54-4ed4-80f8-37462de35ec1 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:44.966558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494e03-bd69-44ab-9112-be0f335380cb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:44.968704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494e03-bd69-44ab-9112-be0f335380cb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:44.971542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494e03-bd69-44ab-9112-be0f335380cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:44.971937 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3883/7796] 10.222.0.16 () {58 vars in 1392 bytes} [Sat Oct 15 21:21:44 2022] GET /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/ee33fd50b4054afeaaaf5f0f409ebdee => generated 688 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:21:45.010481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c6494e03-bd69-44ab-9112-be0f335380cb None None] MFA Rules not processed for user `7547b90326754ee7a452e76f3e088d31`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:45.012971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c6494e03-bd69-44ab-9112-be0f335380cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:45.210388 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3914/7797] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:44 2022] POST /identity/v3/auth/tokens => generated 3424 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Oct 15 21:21:45.214937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85185c31-bb54-4ed4-80f8-37462de35ec1 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:45.228757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85185c31-bb54-4ed4-80f8-37462de35ec1 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:45.233451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595303c-5edc-497c-ab66-f42ae5a286bf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:45.234168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595303c-5edc-497c-ab66-f42ae5a286bf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:45.234510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595303c-5edc-497c-ab66-f42ae5a286bf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:45.234851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595303c-5edc-497c-ab66-f42ae5a286bf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/ee33fd50b4054afeaaaf5f0f409ebdee` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:45.236050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595303c-5edc-497c-ab66-f42ae5a286bf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:delete_application_credential(user_id=7547b90326754ee7a452e76f3e088d31, application_credential_id=ee33fd50b4054afeaaaf5f0f409ebdee)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:45.239179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595303c-5edc-497c-ab66-f42ae5a286bf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:45.277324 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3884/7798] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Oct 15 21:21:45 2022] DELETE /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/ee33fd50b4054afeaaaf5f0f409ebdee => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:45.283906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6494e03-bd69-44ab-9112-be0f335380cb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:45.299648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6494e03-bd69-44ab-9112-be0f335380cb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:45.315900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045d81f-f655-4640-a175-71879e482e86 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:45.318331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045d81f-f655-4640-a175-71879e482e86 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:45.319189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045d81f-f655-4640-a175-71879e482e86 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:45.319571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045d81f-f655-4640-a175-71879e482e86 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:45.320494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045d81f-f655-4640-a175-71879e482e86 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:create_application_credential(user_id=7547b90326754ee7a452e76f3e088d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:45.321787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045d81f-f655-4640-a175-71879e482e86 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:45.434782 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3915/7799] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Oct 15 21:21:45 2022] POST /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials => generated 921 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:45.438800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595303c-5edc-497c-ab66-f42ae5a286bf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:45.449657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:45.455889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595303c-5edc-497c-ab66-f42ae5a286bf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:45.463888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429009f9-b863-44e0-a36d-a734c6d45e36 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:45.465536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429009f9-b863-44e0-a36d-a734c6d45e36 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:45.466249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429009f9-b863-44e0-a36d-a734c6d45e36 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:45.466772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429009f9-b863-44e0-a36d-a734c6d45e36 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/33054b731f764ce2b39e891fdb6686df` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:45.484194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429009f9-b863-44e0-a36d-a734c6d45e36 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:delete_application_credential(user_id=7547b90326754ee7a452e76f3e088d31, application_credential_id=33054b731f764ce2b39e891fdb6686df)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:45.485496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429009f9-b863-44e0-a36d-a734c6d45e36 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:45.524745 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3885/7800] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Oct 15 21:21:45 2022] DELETE /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/33054b731f764ce2b39e891fdb6686df => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:45.535271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045d81f-f655-4640-a175-71879e482e86 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:45.550643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045d81f-f655-4640-a175-71879e482e86 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:45.558935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e41e6-40b5-4ffa-b2dd-d946043a9878 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:45.559762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e41e6-40b5-4ffa-b2dd-d946043a9878 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:45.560237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e41e6-40b5-4ffa-b2dd-d946043a9878 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:45.560603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e41e6-40b5-4ffa-b2dd-d946043a9878 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:45.561477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e41e6-40b5-4ffa-b2dd-d946043a9878 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:create_application_credential(user_id=7547b90326754ee7a452e76f3e088d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:45.563288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e41e6-40b5-4ffa-b2dd-d946043a9878 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:45.577778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:45.706311 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3916/7801] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Oct 15 21:21:45 2022] POST /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials => generated 813 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:45.710932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429009f9-b863-44e0-a36d-a734c6d45e36 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:45.724317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429009f9-b863-44e0-a36d-a734c6d45e36 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:45.730829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cceb63-e6b0-43ad-aac9-3d85675723aa tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:45.731651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cceb63-e6b0-43ad-aac9-3d85675723aa tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:45.732003 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cceb63-e6b0-43ad-aac9-3d85675723aa tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:45.732353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cceb63-e6b0-43ad-aac9-3d85675723aa tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/039087d2fec24f48889cb9326cd4fccc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:45.752071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cceb63-e6b0-43ad-aac9-3d85675723aa tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:delete_application_credential(user_id=7547b90326754ee7a452e76f3e088d31, application_credential_id=039087d2fec24f48889cb9326cd4fccc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:45.753990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cceb63-e6b0-43ad-aac9-3d85675723aa tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:45.788959 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3886/7802] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Oct 15 21:21:45 2022] DELETE /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/039087d2fec24f48889cb9326cd4fccc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:45.789515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e41e6-40b5-4ffa-b2dd-d946043a9878 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:45.799800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e41e6-40b5-4ffa-b2dd-d946043a9878 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:45.805304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e65948-7a4f-4395-a975-dff5e7be41a8 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:45.806209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e65948-7a4f-4395-a975-dff5e7be41a8 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:45.806587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e65948-7a4f-4395-a975-dff5e7be41a8 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:45.806950 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e65948-7a4f-4395-a975-dff5e7be41a8 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:45.807904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e65948-7a4f-4395-a975-dff5e7be41a8 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:create_application_credential(user_id=7547b90326754ee7a452e76f3e088d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:45.809611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e65948-7a4f-4395-a975-dff5e7be41a8 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:45.935925 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3917/7803] 10.222.0.16 () {60 vars in 1314 bytes} [Sat Oct 15 21:21:45 2022] POST /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials => generated 789 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:21:45.937771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cceb63-e6b0-43ad-aac9-3d85675723aa tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:45.956358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cceb63-e6b0-43ad-aac9-3d85675723aa tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:45.962847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6feb2-bb26-4656-8f38-e89b586394c7 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:45.963519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6feb2-bb26-4656-8f38-e89b586394c7 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:45.963780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6feb2-bb26-4656-8f38-e89b586394c7 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:45.964047 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6feb2-bb26-4656-8f38-e89b586394c7 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:45.964702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6feb2-bb26-4656-8f38-e89b586394c7 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:create_application_credential(user_id=7547b90326754ee7a452e76f3e088d31)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:45.967305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6feb2-bb26-4656-8f38-e89b586394c7 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:46.095068 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3887/7804] 10.222.0.16 () {60 vars in 1314 bytes} [Sat Oct 15 21:21:45 2022] POST /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials => generated 788 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:46.096365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e65948-7a4f-4395-a975-dff5e7be41a8 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:46.114411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e65948-7a4f-4395-a975-dff5e7be41a8 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:46.120444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0f66b-f6a5-4db8-84a0-ec7485633cbf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:46.121486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0f66b-f6a5-4db8-84a0-ec7485633cbf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:46.121772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0f66b-f6a5-4db8-84a0-ec7485633cbf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:46.122044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0f66b-f6a5-4db8-84a0-ec7485633cbf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:46.122826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0f66b-f6a5-4db8-84a0-ec7485633cbf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:list_application_credentials(user_id=7547b90326754ee7a452e76f3e088d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:46.125547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0f66b-f6a5-4db8-84a0-ec7485633cbf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:46.146078 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3918/7805] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 21:21:46 2022] GET /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials => generated 1501 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:46.147603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6feb2-bb26-4656-8f38-e89b586394c7 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:46.159241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6feb2-bb26-4656-8f38-e89b586394c7 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:46.166896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c42e1-c515-4e39-b883-30b131b89115 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:46.167935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c42e1-c515-4e39-b883-30b131b89115 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:46.168494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c42e1-c515-4e39-b883-30b131b89115 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:46.168900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c42e1-c515-4e39-b883-30b131b89115 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/1f90695f5d5b42f5a6f4211dc618655c` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:46.180724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c42e1-c515-4e39-b883-30b131b89115 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:delete_application_credential(user_id=7547b90326754ee7a452e76f3e088d31, application_credential_id=1f90695f5d5b42f5a6f4211dc618655c)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:46.182581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c42e1-c515-4e39-b883-30b131b89115 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:46.222830 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3888/7806] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Oct 15 21:21:46 2022] DELETE /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/1f90695f5d5b42f5a6f4211dc618655c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:46.225005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0f66b-f6a5-4db8-84a0-ec7485633cbf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:46.240473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0f66b-f6a5-4db8-84a0-ec7485633cbf tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:46.247328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac404175-e2db-4556-9361-8e1a1a49d0ca tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:46.248380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac404175-e2db-4556-9361-8e1a1a49d0ca tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:46.248713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac404175-e2db-4556-9361-8e1a1a49d0ca tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:46.249761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac404175-e2db-4556-9361-8e1a1a49d0ca tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/015e1869fec94b5ba2c9c5ff60aaa46e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:46.268192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac404175-e2db-4556-9361-8e1a1a49d0ca tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:delete_application_credential(user_id=7547b90326754ee7a452e76f3e088d31, application_credential_id=015e1869fec94b5ba2c9c5ff60aaa46e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:46.273730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac404175-e2db-4556-9361-8e1a1a49d0ca tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:46.322024 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3919/7807] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Oct 15 21:21:46 2022] DELETE /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/015e1869fec94b5ba2c9c5ff60aaa46e => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:46.323200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c42e1-c515-4e39-b883-30b131b89115 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:46.338297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c42e1-c515-4e39-b883-30b131b89115 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:46.349277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50c0c2-bf63-43c4-8ec4-4245dcec7326 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:46.351332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50c0c2-bf63-43c4-8ec4-4245dcec7326 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:46.353268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50c0c2-bf63-43c4-8ec4-4245dcec7326 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:46.354497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50c0c2-bf63-43c4-8ec4-4245dcec7326 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:46.355614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50c0c2-bf63-43c4-8ec4-4245dcec7326 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:create_application_credential(user_id=7547b90326754ee7a452e76f3e088d31)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:46.359007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50c0c2-bf63-43c4-8ec4-4245dcec7326 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:46.474252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:46.483880 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3889/7808] 10.222.0.16 () {60 vars in 1314 bytes} [Sat Oct 15 21:21:46 2022] POST /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials => generated 787 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:46.488073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac404175-e2db-4556-9361-8e1a1a49d0ca tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:46.499108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac404175-e2db-4556-9361-8e1a1a49d0ca tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:46.504822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8ecad-b03e-409c-8cb9-630dcce2784f tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:46.506239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8ecad-b03e-409c-8cb9-630dcce2784f tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:46.506586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8ecad-b03e-409c-8cb9-630dcce2784f tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:46.506924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8ecad-b03e-409c-8cb9-630dcce2784f tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:46.508406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8ecad-b03e-409c-8cb9-630dcce2784f tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:create_application_credential(user_id=7547b90326754ee7a452e76f3e088d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:46.510667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8ecad-b03e-409c-8cb9-630dcce2784f tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:46.591404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:46.614887 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3920/7809] 10.222.0.16 () {60 vars in 1314 bytes} [Sat Oct 15 21:21:46 2022] POST /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials => generated 789 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:46.618655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50c0c2-bf63-43c4-8ec4-4245dcec7326 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:46.635631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50c0c2-bf63-43c4-8ec4-4245dcec7326 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:46.640785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c19876-f0f2-4972-8164-946443fa0b19 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:46.641483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c19876-f0f2-4972-8164-946443fa0b19 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:46.641767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c19876-f0f2-4972-8164-946443fa0b19 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:46.642043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c19876-f0f2-4972-8164-946443fa0b19 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:46.642674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c19876-f0f2-4972-8164-946443fa0b19 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Adding query filter params (name=tempest-application_credential-1530721606) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:21:46.642982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c19876-f0f2-4972-8164-946443fa0b19 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:list_application_credentials(user_id=7547b90326754ee7a452e76f3e088d31)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:46.644604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c19876-f0f2-4972-8164-946443fa0b19 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:46.672283 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3890/7810] 10.222.0.16 () {58 vars in 1386 bytes} [Sat Oct 15 21:21:46 2022] GET /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials?name=tempest-application_credential-1530721606 => generated 887 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:21:46.678246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8ecad-b03e-409c-8cb9-630dcce2784f tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:46.690865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8ecad-b03e-409c-8cb9-630dcce2784f tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:46.698744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beedd21-11b1-46e8-9c9a-ebe2be5b9871 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:46.699584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beedd21-11b1-46e8-9c9a-ebe2be5b9871 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:46.699826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beedd21-11b1-46e8-9c9a-ebe2be5b9871 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:46.700117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beedd21-11b1-46e8-9c9a-ebe2be5b9871 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/9481bbe3963f4b2d8f67b4840b9c2078` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:46.711829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beedd21-11b1-46e8-9c9a-ebe2be5b9871 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:delete_application_credential(user_id=7547b90326754ee7a452e76f3e088d31, application_credential_id=9481bbe3963f4b2d8f67b4840b9c2078)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:46.713431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beedd21-11b1-46e8-9c9a-ebe2be5b9871 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:46.750785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c19876-f0f2-4972-8164-946443fa0b19 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:46.759741 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3921/7811] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Oct 15 21:21:46 2022] DELETE /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/9481bbe3963f4b2d8f67b4840b9c2078 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:46.762531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c19876-f0f2-4972-8164-946443fa0b19 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:46.773722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8915d0-59cb-45ec-a5e7-23fe700b41e5 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7547b90326754ee7a452e76f3e088d31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd20f6b7f7df74f4994458c547cc451e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:46.774493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8915d0-59cb-45ec-a5e7-23fe700b41e5 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:46.774876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8915d0-59cb-45ec-a5e7-23fe700b41e5 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:46.775265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8915d0-59cb-45ec-a5e7-23fe700b41e5 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/1936f1f924d7413998b92c396f81a8d5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:46.788034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8915d0-59cb-45ec-a5e7-23fe700b41e5 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorizing `identity:delete_application_credential(user_id=7547b90326754ee7a452e76f3e088d31, application_credential_id=1936f1f924d7413998b92c396f81a8d5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:46.790860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8915d0-59cb-45ec-a5e7-23fe700b41e5 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:46.812256 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3891/7812] 10.222.0.16 () {58 vars in 1395 bytes} [Sat Oct 15 21:21:46 2022] DELETE /identity/v3/users/7547b90326754ee7a452e76f3e088d31/application_credentials/1936f1f924d7413998b92c396f81a8d5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:46.818090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beedd21-11b1-46e8-9c9a-ebe2be5b9871 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:46.999442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beedd21-11b1-46e8-9c9a-ebe2be5b9871 tempest-ApplicationCredentialsV3Test-370279397 tempest-ApplicationCredentialsV3Test-370279397-project] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:46.999442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:46.999442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:47.000066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:47.000066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] PATH_INFO: `/v3/users/7547b90326754ee7a452e76f3e088d31` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:47.000066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7547b90326754ee7a452e76f3e088d31)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:47.001019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:47.076330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7547b90326754ee7a452e76f3e088d31'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:47.085207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7547b90326754ee7a452e76f3e088d31'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:47.119005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7547b90326754ee7a452e76f3e088d31'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:47.127405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7547b90326754ee7a452e76f3e088d31'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:47.128933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7547b90326754ee7a452e76f3e088d31'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:47.137914 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3922/7813] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:46 2022] DELETE /identity/v3/users/7547b90326754ee7a452e76f3e088d31 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:47.141732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999ad1-8583-4fdf-98c2-0108b8b30264 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:47.142259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999ad1-8583-4fdf-98c2-0108b8b30264 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:47.142754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999ad1-8583-4fdf-98c2-0108b8b30264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:47.165904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-76999ad1-8583-4fdf-98c2-0108b8b30264 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:47.171431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-76999ad1-8583-4fdf-98c2-0108b8b30264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:47.253596 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3892/7814] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:21:47.265480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:47.266979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:47.353424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=2,keystone:SELECT=86,keystone:DELETE=12 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:47.388052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fc5e3-ed2c-4864-8234-a93c79f52d21 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:47.400117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a57aa-2565-47a2-ac28-00af167c6380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:47.400945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a57aa-2565-47a2-ac28-00af167c6380 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:47.401236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a57aa-2565-47a2-ac28-00af167c6380 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:47.401501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a57aa-2565-47a2-ac28-00af167c6380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:47.415418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a57aa-2565-47a2-ac28-00af167c6380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:47.417140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a57aa-2565-47a2-ac28-00af167c6380 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:47.432137 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3923/7815] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:47.885127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999ad1-8583-4fdf-98c2-0108b8b30264 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:47.886622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-76999ad1-8583-4fdf-98c2-0108b8b30264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:47.908111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999ad1-8583-4fdf-98c2-0108b8b30264 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:47.924400 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:47.924805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:47.925112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:47.925955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] PATH_INFO: `/v3/projects/d20f6b7f7df74f4994458c547cc451e5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:47.928762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d20f6b7f7df74f4994458c547cc451e5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:47.930885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:48.034281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd20f6b7f7df74f4994458c547cc451e5', 'request_id': 'req-c9a783fa-18f3-48bc-a993-122b34c96f2e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:48.073253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd20f6b7f7df74f4994458c547cc451e5', 'request_id': 'req-c9a783fa-18f3-48bc-a993-122b34c96f2e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:48.081260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd20f6b7f7df74f4994458c547cc451e5', 'request_id': 'req-c9a783fa-18f3-48bc-a993-122b34c96f2e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:48.094441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] The token cache is being invalidate because project d20f6b7f7df74f4994458c547cc451e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:48.094734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:48.095382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c9a783fa-18f3-48bc-a993-122b34c96f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:48.097066 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3893/7816] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:47 2022] DELETE /identity/v3/projects/d20f6b7f7df74f4994458c547cc451e5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:48.101946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e5361-cfe6-4505-b9b3-393c8b71fc8a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:48.102527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e5361-cfe6-4505-b9b3-393c8b71fc8a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:48.102527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e5361-cfe6-4505-b9b3-393c8b71fc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:48.127323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6a6e5361-cfe6-4505-b9b3-393c8b71fc8a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:48.128763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6a6e5361-cfe6-4505-b9b3-393c8b71fc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:48.209366 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3924/7817] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:48.216814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052bdb6-4880-479f-a0fc-1200fa814943 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:48.217256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052bdb6-4880-479f-a0fc-1200fa814943 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:48.217603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052bdb6-4880-479f-a0fc-1200fa814943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:48.241451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a052bdb6-4880-479f-a0fc-1200fa814943 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:48.243366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a052bdb6-4880-479f-a0fc-1200fa814943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:48.253421 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3894/7818] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:48.257461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e5361-cfe6-4505-b9b3-393c8b71fc8a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:48.268457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e5361-cfe6-4505-b9b3-393c8b71fc8a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:48.276645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95576c70-8337-456c-8583-434847fefded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:48.277728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95576c70-8337-456c-8583-434847fefded admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:48.278006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95576c70-8337-456c-8583-434847fefded admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:48.278600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95576c70-8337-456c-8583-434847fefded admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:48.279293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95576c70-8337-456c-8583-434847fefded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:21:48.279593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95576c70-8337-456c-8583-434847fefded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:48.281447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95576c70-8337-456c-8583-434847fefded admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:48.298624 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3925/7819] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:21:48.300103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052bdb6-4880-479f-a0fc-1200fa814943 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:48.310637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052bdb6-4880-479f-a0fc-1200fa814943 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:48.321026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160c140-5d42-4157-bd09-b1a1f0d1a638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:48.321671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160c140-5d42-4157-bd09-b1a1f0d1a638 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:48.321947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160c140-5d42-4157-bd09-b1a1f0d1a638 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:48.322216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160c140-5d42-4157-bd09-b1a1f0d1a638 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:48.322859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160c140-5d42-4157-bd09-b1a1f0d1a638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:48.324910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160c140-5d42-4157-bd09-b1a1f0d1a638 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:48.361320 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3895/7820] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:48 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:48.363640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95576c70-8337-456c-8583-434847fefded admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:48.375990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95576c70-8337-456c-8583-434847fefded admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:48.442690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c692f63-54b2-4220-8bf2-47ee84fc1d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:48.443943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c692f63-54b2-4220-8bf2-47ee84fc1d9f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:48.444465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c692f63-54b2-4220-8bf2-47ee84fc1d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:48.445149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c692f63-54b2-4220-8bf2-47ee84fc1d9f admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:48.446022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c692f63-54b2-4220-8bf2-47ee84fc1d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:48.448295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c692f63-54b2-4220-8bf2-47ee84fc1d9f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:48.478745 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3926/7821] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:48 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:48.483453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160c140-5d42-4157-bd09-b1a1f0d1a638 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:48.495915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160c140-5d42-4157-bd09-b1a1f0d1a638 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:48.501441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297b57b-d1df-45d2-b704-c4f5c502e763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:48.502283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297b57b-d1df-45d2-b704-c4f5c502e763 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:48.502763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297b57b-d1df-45d2-b704-c4f5c502e763 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:48.503245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297b57b-d1df-45d2-b704-c4f5c502e763 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:48.504079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297b57b-d1df-45d2-b704-c4f5c502e763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:48.506797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297b57b-d1df-45d2-b704-c4f5c502e763 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:48.514580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:48.521462 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3896/7822] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:48.525904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c692f63-54b2-4220-8bf2-47ee84fc1d9f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:48.537606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c692f63-54b2-4220-8bf2-47ee84fc1d9f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:48.542861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08539f70-b274-4b83-8953-657f710824e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:48.543831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08539f70-b274-4b83-8953-657f710824e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:48.544281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08539f70-b274-4b83-8953-657f710824e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:48.544730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08539f70-b274-4b83-8953-657f710824e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:48.545555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08539f70-b274-4b83-8953-657f710824e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:48.547382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08539f70-b274-4b83-8953-657f710824e3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:48.565827 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3927/7823] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:48.570000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297b57b-d1df-45d2-b704-c4f5c502e763 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:48.590424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297b57b-d1df-45d2-b704-c4f5c502e763 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:48.598109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa682d-8698-4db3-a034-63f1a24c18b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:48.598869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa682d-8698-4db3-a034-63f1a24c18b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:48.599177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa682d-8698-4db3-a034-63f1a24c18b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:48.599392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa682d-8698-4db3-a034-63f1a24c18b4 admin admin] PATH_INFO: `/v3/projects/239b4f4f637e427188ceebdd059e0847/users/17491c055d184991a5a37d12386cceb2/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:48.631534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa682d-8698-4db3-a034-63f1a24c18b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=239b4f4f637e427188ceebdd059e0847, user_id=17491c055d184991a5a37d12386cceb2, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:48.634603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa682d-8698-4db3-a034-63f1a24c18b4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:48.657408 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3897/7824] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:48 2022] PUT /identity/v3/projects/239b4f4f637e427188ceebdd059e0847/users/17491c055d184991a5a37d12386cceb2/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:48.670011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c8b4c-6795-44bb-b089-05e42c67063b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:48.670656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c8b4c-6795-44bb-b089-05e42c67063b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:48.671152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c8b4c-6795-44bb-b089-05e42c67063b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:48.719157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d94c8b4c-6795-44bb-b089-05e42c67063b None None] MFA Rules not processed for user `17491c055d184991a5a37d12386cceb2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:48.720122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d94c8b4c-6795-44bb-b089-05e42c67063b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:48.802230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:48.845026 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3928/7825] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:48.849930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa682d-8698-4db3-a034-63f1a24c18b4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:48.943622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa682d-8698-4db3-a034-63f1a24c18b4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:48.949856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237133e8-c65c-4454-8a41-a8a433799e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:48.950615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237133e8-c65c-4454-8a41-a8a433799e40 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:48.951022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237133e8-c65c-4454-8a41-a8a433799e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:48.951394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237133e8-c65c-4454-8a41-a8a433799e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:48.952126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237133e8-c65c-4454-8a41-a8a433799e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:48.954201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237133e8-c65c-4454-8a41-a8a433799e40 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:48.988797 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3898/7826] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:48 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:48.993510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c8b4c-6795-44bb-b089-05e42c67063b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:49.005025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c8b4c-6795-44bb-b089-05e42c67063b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:49.120954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2545e-9b11-441c-abb1-c800aeaf9a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:49.121869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2545e-9b11-441c-abb1-c800aeaf9a99 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:49.122147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2545e-9b11-441c-abb1-c800aeaf9a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:49.122417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2545e-9b11-441c-abb1-c800aeaf9a99 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:49.124785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2545e-9b11-441c-abb1-c800aeaf9a99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:49.130620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2545e-9b11-441c-abb1-c800aeaf9a99 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:49.186386 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3929/7827] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:48 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:49.197601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237133e8-c65c-4454-8a41-a8a433799e40 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:49.227326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237133e8-c65c-4454-8a41-a8a433799e40 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:49.232938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b84179-1673-48f6-8c7b-94f6326789ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:49.233699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b84179-1673-48f6-8c7b-94f6326789ff admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:49.234179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b84179-1673-48f6-8c7b-94f6326789ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:49.234546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b84179-1673-48f6-8c7b-94f6326789ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:49.235286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b84179-1673-48f6-8c7b-94f6326789ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:49.238679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b84179-1673-48f6-8c7b-94f6326789ff admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:49.255525 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3899/7828] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:49.259282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2545e-9b11-441c-abb1-c800aeaf9a99 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:49.290184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2545e-9b11-441c-abb1-c800aeaf9a99 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:49.300964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b2c83-f86d-45d4-9e11-3df7f5d2b87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:49.302729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b2c83-f86d-45d4-9e11-3df7f5d2b87d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:49.303257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b2c83-f86d-45d4-9e11-3df7f5d2b87d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:49.303714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b2c83-f86d-45d4-9e11-3df7f5d2b87d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:49.304614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b2c83-f86d-45d4-9e11-3df7f5d2b87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:49.307230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b2c83-f86d-45d4-9e11-3df7f5d2b87d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:49.322969 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3930/7829] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:49.324541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b84179-1673-48f6-8c7b-94f6326789ff admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:49.341497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b84179-1673-48f6-8c7b-94f6326789ff admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:49.349060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d1dcb-ba80-450a-9363-4c7d0db3811c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:49.349898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d1dcb-ba80-450a-9363-4c7d0db3811c admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:49.350285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d1dcb-ba80-450a-9363-4c7d0db3811c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:49.350616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d1dcb-ba80-450a-9363-4c7d0db3811c admin admin] PATH_INFO: `/v3/projects/c2b8ff8fe48d40a98eca16550b72e848/users/4e62d24e9b244aefbe3b269f1758f7e8/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:49.406983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d1dcb-ba80-450a-9363-4c7d0db3811c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b8ff8fe48d40a98eca16550b72e848, user_id=4e62d24e9b244aefbe3b269f1758f7e8, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:49.415995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d1dcb-ba80-450a-9363-4c7d0db3811c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:49.441246 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3900/7830] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:49 2022] PUT /identity/v3/projects/c2b8ff8fe48d40a98eca16550b72e848/users/4e62d24e9b244aefbe3b269f1758f7e8/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:49.452256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ca466-d210-47a4-988f-a3c11381bd24 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:49.452536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ca466-d210-47a4-988f-a3c11381bd24 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:49.453547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ca466-d210-47a4-988f-a3c11381bd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:49.486275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e90ca466-d210-47a4-988f-a3c11381bd24 None None] MFA Rules not processed for user `4e62d24e9b244aefbe3b269f1758f7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:49.488023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e90ca466-d210-47a4-988f-a3c11381bd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:49.570061 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3931/7831] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:49.663357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d1dcb-ba80-450a-9363-4c7d0db3811c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:49.753773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d1dcb-ba80-450a-9363-4c7d0db3811c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:49.761476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac944d-d885-423d-98fb-f9937418c5c0 tempest-IdentityV3ProjectsTest-420343368 tempest-IdentityV3ProjectsTest-420343368-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17491c055d184991a5a37d12386cceb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '239b4f4f637e427188ceebdd059e0847', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:49.762134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac944d-d885-423d-98fb-f9937418c5c0 tempest-IdentityV3ProjectsTest-420343368 tempest-IdentityV3ProjectsTest-420343368-project] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:49.762484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac944d-d885-423d-98fb-f9937418c5c0 tempest-IdentityV3ProjectsTest-420343368 tempest-IdentityV3ProjectsTest-420343368-project] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:49.762824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac944d-d885-423d-98fb-f9937418c5c0 tempest-IdentityV3ProjectsTest-420343368 tempest-IdentityV3ProjectsTest-420343368-project] PATH_INFO: `/v3/users/17491c055d184991a5a37d12386cceb2/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:49.764067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac944d-d885-423d-98fb-f9937418c5c0 tempest-IdentityV3ProjectsTest-420343368 tempest-IdentityV3ProjectsTest-420343368-project] RBAC: Authorizing `identity:list_user_projects(user_id=17491c055d184991a5a37d12386cceb2)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:49.766234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac944d-d885-423d-98fb-f9937418c5c0 tempest-IdentityV3ProjectsTest-420343368 tempest-IdentityV3ProjectsTest-420343368-project] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:49.805304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:49.823247 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3901/7832] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:21:49 2022] GET /identity/v3/users/17491c055d184991a5a37d12386cceb2/projects => generated 515 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:21:49.823809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cdf98-81e6-4ecc-8493-a0e645f77ec9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:49.824095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cdf98-81e6-4ecc-8493-a0e645f77ec9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:49.824365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cdf98-81e6-4ecc-8493-a0e645f77ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:49.863422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e85cdf98-81e6-4ecc-8493-a0e645f77ec9 None None] MFA Rules not processed for user `17491c055d184991a5a37d12386cceb2`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:49.865924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e85cdf98-81e6-4ecc-8493-a0e645f77ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:49.876330 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3932/7833] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:49.878138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95b011-82c7-4172-bce9-f68c980b1bcc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:49.878428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95b011-82c7-4172-bce9-f68c980b1bcc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:49.878697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95b011-82c7-4172-bce9-f68c980b1bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:49.908231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ef95b011-82c7-4172-bce9-f68c980b1bcc None None] MFA Rules not processed for user `17491c055d184991a5a37d12386cceb2`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:49.909479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ef95b011-82c7-4172-bce9-f68c980b1bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:49.970604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.models.token_model [None req-ef95b011-82c7-4172-bce9-f68c980b1bcc None None] User 17491c055d184991a5a37d12386cceb2 has no access to project c2b8ff8fe48d40a98eca16550b72e848 {{(pid=99100) _validate_project_scope /opt/stack/keystone/keystone/models/token_model.py:539}} Oct 15 21:21:49.975824 ncdv-master-1 devstack@keystone.service[99100]: WARNING keystone.server.flask.application [None req-ef95b011-82c7-4172-bce9-f68c980b1bcc None None] Authorization failed. The request you have made requires authentication. from 10.222.0.16: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 15 21:21:49.982870 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3902/7834] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:49 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 100 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 15 21:21:49.984132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cdf98-81e6-4ecc-8493-a0e645f77ec9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:50.029872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:50.067502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cdf98-81e6-4ecc-8493-a0e645f77ec9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:50.072707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:50.073440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:50.073720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:50.073988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] PATH_INFO: `/v3/users/17491c055d184991a5a37d12386cceb2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:50.075114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17491c055d184991a5a37d12386cceb2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:50.077209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:50.153998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17491c055d184991a5a37d12386cceb2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:50.165414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17491c055d184991a5a37d12386cceb2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:50.180211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17491c055d184991a5a37d12386cceb2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:50.191174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17491c055d184991a5a37d12386cceb2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:50.192040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17491c055d184991a5a37d12386cceb2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:50.202793 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3933/7835] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:49 2022] DELETE /identity/v3/users/17491c055d184991a5a37d12386cceb2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:50.209669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673cc49-8efe-4c17-a187-7f8a933d28fa None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:50.209960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673cc49-8efe-4c17-a187-7f8a933d28fa None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:50.210239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673cc49-8efe-4c17-a187-7f8a933d28fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:50.246971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3673cc49-8efe-4c17-a187-7f8a933d28fa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:50.248636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3673cc49-8efe-4c17-a187-7f8a933d28fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:50.340408 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3903/7836] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:50.351437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:50.352835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:50.421535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff473d-7112-41b8-937f-bb5c144ec9d2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:50.426189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf44cf-9128-45fa-b398-a2cae0cca4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:50.427283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf44cf-9128-45fa-b398-a2cae0cca4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:50.427724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf44cf-9128-45fa-b398-a2cae0cca4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:50.428268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf44cf-9128-45fa-b398-a2cae0cca4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:50.442462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf44cf-9128-45fa-b398-a2cae0cca4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:50.444170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf44cf-9128-45fa-b398-a2cae0cca4f0 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:50.463161 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3934/7837] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:50.879680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673cc49-8efe-4c17-a187-7f8a933d28fa None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:50.883352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3673cc49-8efe-4c17-a187-7f8a933d28fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:50.908418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673cc49-8efe-4c17-a187-7f8a933d28fa None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:50.919485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:50.920192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:50.920473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:50.920743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] PATH_INFO: `/v3/projects/239b4f4f637e427188ceebdd059e0847` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:50.921955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] RBAC: Authorizing `identity:delete_project(project_id=239b4f4f637e427188ceebdd059e0847)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:50.923901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:51.020636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '239b4f4f637e427188ceebdd059e0847', 'request_id': 'req-01d78dff-637f-48a9-b4f3-c212fa4e152f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:51.040702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '239b4f4f637e427188ceebdd059e0847', 'request_id': 'req-01d78dff-637f-48a9-b4f3-c212fa4e152f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:51.090842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '239b4f4f637e427188ceebdd059e0847', 'request_id': 'req-01d78dff-637f-48a9-b4f3-c212fa4e152f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:51.105702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] The token cache is being invalidate because project 239b4f4f637e427188ceebdd059e0847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:51.105702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:51.105702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:51.108798 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3904/7838] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:50 2022] DELETE /identity/v3/projects/239b4f4f637e427188ceebdd059e0847 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:51.111695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf44cf-9128-45fa-b398-a2cae0cca4f0 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:51.130477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf44cf-9128-45fa-b398-a2cae0cca4f0 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:51.227223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:51.239729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:51.240621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:51.242172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:51.243025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] PATH_INFO: `/v3/users/4e62d24e9b244aefbe3b269f1758f7e8` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:51.244276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e62d24e9b244aefbe3b269f1758f7e8)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:51.246821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:51.313404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e62d24e9b244aefbe3b269f1758f7e8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:51.333785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e62d24e9b244aefbe3b269f1758f7e8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:51.357787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e62d24e9b244aefbe3b269f1758f7e8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:51.368609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e62d24e9b244aefbe3b269f1758f7e8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:51.369436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8a3d2bbf-e303-484b-8718-db17ffcc1209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e62d24e9b244aefbe3b269f1758f7e8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:51.381361 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3935/7839] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:51 2022] DELETE /identity/v3/users/4e62d24e9b244aefbe3b269f1758f7e8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:51.875580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:51.877347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:51.902018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d78dff-637f-48a9-b4f3-c212fa4e152f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:51.945814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:51.994783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:51.996070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:51.996562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:51.996890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] PATH_INFO: `/v3/projects/c2b8ff8fe48d40a98eca16550b72e848` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:51.998765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b8ff8fe48d40a98eca16550b72e848)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:52.001309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:52.074472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b8ff8fe48d40a98eca16550b72e848', 'request_id': 'req-ded0afa5-aeef-43d9-a24d-f8191329f8c7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:52.084147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b8ff8fe48d40a98eca16550b72e848', 'request_id': 'req-ded0afa5-aeef-43d9-a24d-f8191329f8c7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:52.091494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b8ff8fe48d40a98eca16550b72e848', 'request_id': 'req-ded0afa5-aeef-43d9-a24d-f8191329f8c7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:52.122852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] The token cache is being invalidate because project c2b8ff8fe48d40a98eca16550b72e848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:52.123140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:52.123868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ded0afa5-aeef-43d9-a24d-f8191329f8c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:52.125470 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3905/7840] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:51 2022] DELETE /identity/v3/projects/c2b8ff8fe48d40a98eca16550b72e848 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:52.131306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a0331-3d68-47e7-8a4c-b52a9894c861 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:52.131825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a0331-3d68-47e7-8a4c-b52a9894c861 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:52.132099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a0331-3d68-47e7-8a4c-b52a9894c861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:52.157886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8e3a0331-3d68-47e7-8a4c-b52a9894c861 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:52.162066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8e3a0331-3d68-47e7-8a4c-b52a9894c861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:52.268214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3936/7841] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:52.277309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2be78-0c3c-4c35-be8d-c5a474879376 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:52.277671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2be78-0c3c-4c35-be8d-c5a474879376 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:52.278029 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2be78-0c3c-4c35-be8d-c5a474879376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:52.303711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c0a2be78-0c3c-4c35-be8d-c5a474879376 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:52.305335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c0a2be78-0c3c-4c35-be8d-c5a474879376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:52.316234 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3906/7842] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:52.321234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a0331-3d68-47e7-8a4c-b52a9894c861 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:52.337469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a0331-3d68-47e7-8a4c-b52a9894c861 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:52.342746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65995d84-6abc-4e1f-a3c8-6d5ef784bc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:52.343504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995d84-6abc-4e1f-a3c8-6d5ef784bc72 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:52.343871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995d84-6abc-4e1f-a3c8-6d5ef784bc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:52.344548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995d84-6abc-4e1f-a3c8-6d5ef784bc72 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:52.345583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65995d84-6abc-4e1f-a3c8-6d5ef784bc72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:21:52.345959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65995d84-6abc-4e1f-a3c8-6d5ef784bc72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:52.348083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65995d84-6abc-4e1f-a3c8-6d5ef784bc72 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:52.365784 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3937/7843] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:21:52.368861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2be78-0c3c-4c35-be8d-c5a474879376 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:52.379895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2be78-0c3c-4c35-be8d-c5a474879376 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:52.385237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d70c3-3167-4dea-b985-89099c0e0270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:52.385941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d70c3-3167-4dea-b985-89099c0e0270 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:52.386286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d70c3-3167-4dea-b985-89099c0e0270 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:52.386621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d70c3-3167-4dea-b985-89099c0e0270 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:52.387332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d70c3-3167-4dea-b985-89099c0e0270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:52.389673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d70c3-3167-4dea-b985-89099c0e0270 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:52.430158 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3907/7844] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:52 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:52.430409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65995d84-6abc-4e1f-a3c8-6d5ef784bc72 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:52.443620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65995d84-6abc-4e1f-a3c8-6d5ef784bc72 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:52.535196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324084c0-d29e-4d83-aefc-20f36c66a3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:52.536244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324084c0-d29e-4d83-aefc-20f36c66a3ac admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:52.536527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324084c0-d29e-4d83-aefc-20f36c66a3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:52.536795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324084c0-d29e-4d83-aefc-20f36c66a3ac admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:52.537517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324084c0-d29e-4d83-aefc-20f36c66a3ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:52.539714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324084c0-d29e-4d83-aefc-20f36c66a3ac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:52.548546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:52.570717 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3938/7845] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:52 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:21:52.574837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d70c3-3167-4dea-b985-89099c0e0270 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:52.588812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d70c3-3167-4dea-b985-89099c0e0270 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:52.602924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6751505-0983-4b03-bec5-fd11c0bd833f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:52.603695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6751505-0983-4b03-bec5-fd11c0bd833f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:52.604036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6751505-0983-4b03-bec5-fd11c0bd833f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:52.604368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6751505-0983-4b03-bec5-fd11c0bd833f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:52.605376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6751505-0983-4b03-bec5-fd11c0bd833f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:52.607076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6751505-0983-4b03-bec5-fd11c0bd833f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:52.625079 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3908/7846] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:52.625317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324084c0-d29e-4d83-aefc-20f36c66a3ac admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:52.634733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324084c0-d29e-4d83-aefc-20f36c66a3ac admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:52.640063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d543c-4dc4-4dc1-a4a2-bae5db8392ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:52.640602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d543c-4dc4-4dc1-a4a2-bae5db8392ec admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:52.640748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d543c-4dc4-4dc1-a4a2-bae5db8392ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:52.641142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d543c-4dc4-4dc1-a4a2-bae5db8392ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:52.641830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d543c-4dc4-4dc1-a4a2-bae5db8392ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:52.643470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d543c-4dc4-4dc1-a4a2-bae5db8392ec admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:52.660983 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3939/7847] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:52.662086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6751505-0983-4b03-bec5-fd11c0bd833f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:52.673942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6751505-0983-4b03-bec5-fd11c0bd833f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:52.680044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fcda9-91e2-4468-ad0e-5b89d6c2b059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:52.680708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fcda9-91e2-4468-ad0e-5b89d6c2b059 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:52.680985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fcda9-91e2-4468-ad0e-5b89d6c2b059 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:52.681353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fcda9-91e2-4468-ad0e-5b89d6c2b059 admin admin] PATH_INFO: `/v3/projects/a31748bc8dd04cf981ff8a54ad14b4f0/users/675f535603f44a4283e9aa8fa7e2cd1f/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:52.697635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fcda9-91e2-4468-ad0e-5b89d6c2b059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31748bc8dd04cf981ff8a54ad14b4f0, user_id=675f535603f44a4283e9aa8fa7e2cd1f, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:52.700325 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fcda9-91e2-4468-ad0e-5b89d6c2b059 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:52.715434 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3909/7848] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:52 2022] PUT /identity/v3/projects/a31748bc8dd04cf981ff8a54ad14b4f0/users/675f535603f44a4283e9aa8fa7e2cd1f/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:21:52.726432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbb750-b5e7-4cbd-82bf-0a1e0984e2e7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:52.726802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbb750-b5e7-4cbd-82bf-0a1e0984e2e7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:52.727140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbb750-b5e7-4cbd-82bf-0a1e0984e2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:52.772110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d3fbb750-b5e7-4cbd-82bf-0a1e0984e2e7 None None] MFA Rules not processed for user `675f535603f44a4283e9aa8fa7e2cd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:52.774575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d3fbb750-b5e7-4cbd-82bf-0a1e0984e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:52.872128 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3940/7849] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:52.876514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fcda9-91e2-4468-ad0e-5b89d6c2b059 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:52.968736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fcda9-91e2-4468-ad0e-5b89d6c2b059 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:52.978936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437bfab-9d33-4a9e-955d-2d5ea6a86445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:52.980239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437bfab-9d33-4a9e-955d-2d5ea6a86445 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:52.980946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437bfab-9d33-4a9e-955d-2d5ea6a86445 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:52.981648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437bfab-9d33-4a9e-955d-2d5ea6a86445 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:52.982723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437bfab-9d33-4a9e-955d-2d5ea6a86445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:52.988340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437bfab-9d33-4a9e-955d-2d5ea6a86445 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:52.996745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:53.055562 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3910/7850] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:52 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:53.057094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbb750-b5e7-4cbd-82bf-0a1e0984e2e7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:53.072137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbb750-b5e7-4cbd-82bf-0a1e0984e2e7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:53.160181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b36a79-af85-47a9-ab78-9e4a53dd8854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:53.161139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b36a79-af85-47a9-ab78-9e4a53dd8854 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:53.161439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b36a79-af85-47a9-ab78-9e4a53dd8854 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:53.161825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b36a79-af85-47a9-ab78-9e4a53dd8854 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:53.162529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b36a79-af85-47a9-ab78-9e4a53dd8854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:53.165073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b36a79-af85-47a9-ab78-9e4a53dd8854 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:53.227581 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3941/7851] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:53 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:53.228913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437bfab-9d33-4a9e-955d-2d5ea6a86445 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:53.246000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437bfab-9d33-4a9e-955d-2d5ea6a86445 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:53.253023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ab7cf-f657-4609-becf-94bbcb4755e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:53.253758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ab7cf-f657-4609-becf-94bbcb4755e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:53.254123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ab7cf-f657-4609-becf-94bbcb4755e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:53.254468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ab7cf-f657-4609-becf-94bbcb4755e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:53.255286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ab7cf-f657-4609-becf-94bbcb4755e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:53.257244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ab7cf-f657-4609-becf-94bbcb4755e8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:53.279523 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3911/7852] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:53.280183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b36a79-af85-47a9-ab78-9e4a53dd8854 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:53.298156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b36a79-af85-47a9-ab78-9e4a53dd8854 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:53.304512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42c93c-6837-4fb1-92ad-384aeb1d7b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:53.305333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c93c-6837-4fb1-92ad-384aeb1d7b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:53.305684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c93c-6837-4fb1-92ad-384aeb1d7b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:53.306029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c93c-6837-4fb1-92ad-384aeb1d7b99 admin admin] PATH_INFO: `/v3/projects/a34ca314bda9452699474396f94420eb/users/aefd9a75bf294eb09bf5b08c04030520/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:53.331930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42c93c-6837-4fb1-92ad-384aeb1d7b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34ca314bda9452699474396f94420eb, user_id=aefd9a75bf294eb09bf5b08c04030520, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:53.336752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42c93c-6837-4fb1-92ad-384aeb1d7b99 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:53.367422 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3942/7853] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:53 2022] PUT /identity/v3/projects/a34ca314bda9452699474396f94420eb/users/aefd9a75bf294eb09bf5b08c04030520/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:21:53.371746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ab7cf-f657-4609-becf-94bbcb4755e8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:53.387988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ab7cf-f657-4609-becf-94bbcb4755e8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:53.486296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b9dc7-e03a-49c0-8721-edc8b35e42af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:53.486932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b9dc7-e03a-49c0-8721-edc8b35e42af admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:53.489612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b9dc7-e03a-49c0-8721-edc8b35e42af admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:53.489952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b9dc7-e03a-49c0-8721-edc8b35e42af admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:53.491139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b9dc7-e03a-49c0-8721-edc8b35e42af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:53.492790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b9dc7-e03a-49c0-8721-edc8b35e42af admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:53.522617 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3912/7854] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:53.526697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42c93c-6837-4fb1-92ad-384aeb1d7b99 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:53.542575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42c93c-6837-4fb1-92ad-384aeb1d7b99 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:53.549939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b710314-3cd5-4422-83be-5cc6ee877b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:53.553164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b710314-3cd5-4422-83be-5cc6ee877b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:53.553797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b710314-3cd5-4422-83be-5cc6ee877b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:53.554318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b710314-3cd5-4422-83be-5cc6ee877b5f admin admin] PATH_INFO: `/v3/projects/a34ca314bda9452699474396f94420eb/users/aefd9a75bf294eb09bf5b08c04030520/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:53.558273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b710314-3cd5-4422-83be-5cc6ee877b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34ca314bda9452699474396f94420eb, user_id=aefd9a75bf294eb09bf5b08c04030520, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:53.562522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b710314-3cd5-4422-83be-5cc6ee877b5f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:53.581135 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3943/7855] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:53 2022] PUT /identity/v3/projects/a34ca314bda9452699474396f94420eb/users/aefd9a75bf294eb09bf5b08c04030520/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:53.592674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a86f83-0b29-4d45-b47d-a2fcd46f3b37 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:53.593368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a86f83-0b29-4d45-b47d-a2fcd46f3b37 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:53.593711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a86f83-0b29-4d45-b47d-a2fcd46f3b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:53.645560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-08a86f83-0b29-4d45-b47d-a2fcd46f3b37 None None] MFA Rules not processed for user `aefd9a75bf294eb09bf5b08c04030520`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:53.647237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-08a86f83-0b29-4d45-b47d-a2fcd46f3b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:53.768147 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3913/7856] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:53.783168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b710314-3cd5-4422-83be-5cc6ee877b5f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:53.786492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6b710314-3cd5-4422-83be-5cc6ee877b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:53.843800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:53.941258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b710314-3cd5-4422-83be-5cc6ee877b5f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:53.946614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8f972-c77a-42d5-b3f1-427da8926f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:53.947476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8f972-c77a-42d5-b3f1-427da8926f27 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:53.947920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8f972-c77a-42d5-b3f1-427da8926f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:53.948417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8f972-c77a-42d5-b3f1-427da8926f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:53.959538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8f972-c77a-42d5-b3f1-427da8926f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:53.960967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8f972-c77a-42d5-b3f1-427da8926f27 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:53.975232 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3944/7857] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:56.478855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a86f83-0b29-4d45-b47d-a2fcd46f3b37 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:56.618556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:56.631964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a86f83-0b29-4d45-b47d-a2fcd46f3b37 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:56.646092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:56.646896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:56.647177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:56.647448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] PATH_INFO: `/v3/users/675f535603f44a4283e9aa8fa7e2cd1f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:56.648925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=675f535603f44a4283e9aa8fa7e2cd1f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:56.650916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:56.726924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675f535603f44a4283e9aa8fa7e2cd1f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:56.747712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675f535603f44a4283e9aa8fa7e2cd1f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:56.766530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675f535603f44a4283e9aa8fa7e2cd1f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:56.809137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675f535603f44a4283e9aa8fa7e2cd1f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:56.809969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675f535603f44a4283e9aa8fa7e2cd1f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:56.819088 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3914/7858] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:56 2022] DELETE /identity/v3/users/675f535603f44a4283e9aa8fa7e2cd1f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:56.823384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee11fd-2b03-4ed6-808c-5eb50de8c615 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:56.823987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee11fd-2b03-4ed6-808c-5eb50de8c615 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:56.823987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee11fd-2b03-4ed6-808c-5eb50de8c615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:56.878392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1aee11fd-2b03-4ed6-808c-5eb50de8c615 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:56.880199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1aee11fd-2b03-4ed6-808c-5eb50de8c615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:57.076085 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3945/7859] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:57.087106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:57.088671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:57.175345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3812b5-9ef0-43cf-aea8-9b528be3ab9b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:57.180726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6c71b-03fa-46be-8d32-630ff46ea0cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:57.183615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6c71b-03fa-46be-8d32-630ff46ea0cc service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:57.183615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6c71b-03fa-46be-8d32-630ff46ea0cc service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:57.183615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6c71b-03fa-46be-8d32-630ff46ea0cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:57.206632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6c71b-03fa-46be-8d32-630ff46ea0cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:57.208892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6c71b-03fa-46be-8d32-630ff46ea0cc service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:57.233593 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3915/7860] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:21:57.629544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee11fd-2b03-4ed6-808c-5eb50de8c615 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:57.632389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1aee11fd-2b03-4ed6-808c-5eb50de8c615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:57.660015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee11fd-2b03-4ed6-808c-5eb50de8c615 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:57.669730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:57.670452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:57.671135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:57.671418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] PATH_INFO: `/v3/projects/a31748bc8dd04cf981ff8a54ad14b4f0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:57.673311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a31748bc8dd04cf981ff8a54ad14b4f0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:57.675388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:57.766422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a31748bc8dd04cf981ff8a54ad14b4f0', 'request_id': 'req-96254fcb-7333-4501-a9e0-551b4f64a543'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:57.782150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a31748bc8dd04cf981ff8a54ad14b4f0', 'request_id': 'req-96254fcb-7333-4501-a9e0-551b4f64a543'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:57.793426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a31748bc8dd04cf981ff8a54ad14b4f0', 'request_id': 'req-96254fcb-7333-4501-a9e0-551b4f64a543'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:57.812758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] The token cache is being invalidate because project a31748bc8dd04cf981ff8a54ad14b4f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:57.813236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:57.813991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:57.819724 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3946/7861] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:57 2022] DELETE /identity/v3/projects/a31748bc8dd04cf981ff8a54ad14b4f0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:57.826352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6c71b-03fa-46be-8d32-630ff46ea0cc service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:57.841356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6c71b-03fa-46be-8d32-630ff46ea0cc service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:57.911274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:57.912366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:57.912643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:57.912900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] PATH_INFO: `/v3/users/aefd9a75bf294eb09bf5b08c04030520` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:57.914087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=aefd9a75bf294eb09bf5b08c04030520)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:57.916074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:57.948228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:57.995899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aefd9a75bf294eb09bf5b08c04030520'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:58.018106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aefd9a75bf294eb09bf5b08c04030520'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:58.038156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aefd9a75bf294eb09bf5b08c04030520'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:58.047572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aefd9a75bf294eb09bf5b08c04030520'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:58.048330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-74fe492e-0299-448e-91af-d7425cb015bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aefd9a75bf294eb09bf5b08c04030520'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:58.059303 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3916/7862] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:21:57 2022] DELETE /identity/v3/users/aefd9a75bf294eb09bf5b08c04030520 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:58.397704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:58.399410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:58.413241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96254fcb-7333-4501-a9e0-551b4f64a543 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:58.498079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:58.517538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:58.518561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:58.518844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:58.519119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] PATH_INFO: `/v3/projects/a34ca314bda9452699474396f94420eb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:58.520612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a34ca314bda9452699474396f94420eb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:58.522558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:58.597803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a34ca314bda9452699474396f94420eb', 'request_id': 'req-4785dfa2-9551-45be-9dcc-b2b6db5176a8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:58.614980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a34ca314bda9452699474396f94420eb', 'request_id': 'req-4785dfa2-9551-45be-9dcc-b2b6db5176a8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:58.643864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a34ca314bda9452699474396f94420eb', 'request_id': 'req-4785dfa2-9551-45be-9dcc-b2b6db5176a8'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:58.663263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] The token cache is being invalidate because project a34ca314bda9452699474396f94420eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:21:58.663878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:58.665509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4785dfa2-9551-45be-9dcc-b2b6db5176a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:21:58.672801 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3947/7863] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:21:58 2022] DELETE /identity/v3/projects/a34ca314bda9452699474396f94420eb => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:21:58.674243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f773839-9909-4f34-bcdf-664273701d9f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:58.674784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f773839-9909-4f34-bcdf-664273701d9f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:58.675290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f773839-9909-4f34-bcdf-664273701d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:58.742766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5f773839-9909-4f34-bcdf-664273701d9f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:58.745287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5f773839-9909-4f34-bcdf-664273701d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:58.841891 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3917/7864] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:58.850380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc685a4-ad0c-4d8e-9357-54722992b141 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:58.850655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc685a4-ad0c-4d8e-9357-54722992b141 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:58.850864 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc685a4-ad0c-4d8e-9357-54722992b141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:58.889452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2fc685a4-ad0c-4d8e-9357-54722992b141 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:58.890941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2fc685a4-ad0c-4d8e-9357-54722992b141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:58.906461 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3948/7865] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:58.907380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f773839-9909-4f34-bcdf-664273701d9f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:58.921344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f773839-9909-4f34-bcdf-664273701d9f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:58.927764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070106a-8754-4fdf-b6e2-6d35d8d44512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:58.928698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070106a-8754-4fdf-b6e2-6d35d8d44512 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:58.929613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070106a-8754-4fdf-b6e2-6d35d8d44512 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:58.929948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070106a-8754-4fdf-b6e2-6d35d8d44512 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:58.930673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070106a-8754-4fdf-b6e2-6d35d8d44512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:21:58.931173 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070106a-8754-4fdf-b6e2-6d35d8d44512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:58.932860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070106a-8754-4fdf-b6e2-6d35d8d44512 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:58.951000 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3918/7866] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:21:58.955406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc685a4-ad0c-4d8e-9357-54722992b141 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:58.970919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc685a4-ad0c-4d8e-9357-54722992b141 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:58.976879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9511450-09ad-43f1-80d6-6ea9444d66d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:58.977598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9511450-09ad-43f1-80d6-6ea9444d66d3 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:58.977948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9511450-09ad-43f1-80d6-6ea9444d66d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:58.978285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9511450-09ad-43f1-80d6-6ea9444d66d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:58.979621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9511450-09ad-43f1-80d6-6ea9444d66d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:58.984351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9511450-09ad-43f1-80d6-6ea9444d66d3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:59.017887 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3949/7867] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:58 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:59.022365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070106a-8754-4fdf-b6e2-6d35d8d44512 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:59.036200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070106a-8754-4fdf-b6e2-6d35d8d44512 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:59.129300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184d21e-733a-40a3-a7bf-141714f8233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:59.129877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184d21e-733a-40a3-a7bf-141714f8233c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:59.130154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184d21e-733a-40a3-a7bf-141714f8233c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:59.130423 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184d21e-733a-40a3-a7bf-141714f8233c admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:59.131068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184d21e-733a-40a3-a7bf-141714f8233c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:59.133318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184d21e-733a-40a3-a7bf-141714f8233c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:59.161490 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3919/7868] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:59 2022] POST /identity/v3/users => generated 284 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:21:59.165780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9511450-09ad-43f1-80d6-6ea9444d66d3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:59.177624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9511450-09ad-43f1-80d6-6ea9444d66d3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:59.182805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3c5fc-1d4e-4d0d-9ebe-f3cf6653ad58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:59.183669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3c5fc-1d4e-4d0d-9ebe-f3cf6653ad58 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:59.183947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3c5fc-1d4e-4d0d-9ebe-f3cf6653ad58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:59.184220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3c5fc-1d4e-4d0d-9ebe-f3cf6653ad58 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:59.184848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3c5fc-1d4e-4d0d-9ebe-f3cf6653ad58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:59.186604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3c5fc-1d4e-4d0d-9ebe-f3cf6653ad58 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:59.206441 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3950/7869] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:59.211380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184d21e-733a-40a3-a7bf-141714f8233c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:59.226392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184d21e-733a-40a3-a7bf-141714f8233c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:59.231998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f18f6-7397-4883-926d-57158ae5556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:59.232731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f18f6-7397-4883-926d-57158ae5556d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:59.233098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f18f6-7397-4883-926d-57158ae5556d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:59.233441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f18f6-7397-4883-926d-57158ae5556d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:59.234186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f18f6-7397-4883-926d-57158ae5556d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:59.235807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f18f6-7397-4883-926d-57158ae5556d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:59.248327 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3920/7870] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:59.252570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3c5fc-1d4e-4d0d-9ebe-f3cf6653ad58 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:59.264132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3c5fc-1d4e-4d0d-9ebe-f3cf6653ad58 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:59.270036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46e31f-ea6a-4698-9440-b21b2c828161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:59.270769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46e31f-ea6a-4698-9440-b21b2c828161 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:59.271110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46e31f-ea6a-4698-9440-b21b2c828161 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:59.271472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46e31f-ea6a-4698-9440-b21b2c828161 admin admin] PATH_INFO: `/v3/projects/684e4cca8e0541698b058aef092e8424/users/1d9ac3a33ead42ca80677f2a74a9dc51/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:59.294925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46e31f-ea6a-4698-9440-b21b2c828161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=684e4cca8e0541698b058aef092e8424, user_id=1d9ac3a33ead42ca80677f2a74a9dc51, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:59.297960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46e31f-ea6a-4698-9440-b21b2c828161 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:59.326033 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3951/7871] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:59 2022] PUT /identity/v3/projects/684e4cca8e0541698b058aef092e8424/users/1d9ac3a33ead42ca80677f2a74a9dc51/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:21:59.336796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b1e51-b986-4dab-b8d1-ac303f6579ba None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:59.337174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b1e51-b986-4dab-b8d1-ac303f6579ba None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:59.337447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b1e51-b986-4dab-b8d1-ac303f6579ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:59.345750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:59.379696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ef2b1e51-b986-4dab-b8d1-ac303f6579ba None None] MFA Rules not processed for user `1d9ac3a33ead42ca80677f2a74a9dc51`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:21:59.380965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ef2b1e51-b986-4dab-b8d1-ac303f6579ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:21:59.468916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46e31f-ea6a-4698-9440-b21b2c828161 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:59.470122 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3921/7872] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:21:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:21:59.554836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46e31f-ea6a-4698-9440-b21b2c828161 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:59.565667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d840d-9e51-4b12-afd4-1fa71b22ddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:59.567222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d840d-9e51-4b12-afd4-1fa71b22ddf1 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:59.567496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d840d-9e51-4b12-afd4-1fa71b22ddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:59.567768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d840d-9e51-4b12-afd4-1fa71b22ddf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:59.568597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d840d-9e51-4b12-afd4-1fa71b22ddf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:59.571131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d840d-9e51-4b12-afd4-1fa71b22ddf1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:59.597344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:59.610733 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3952/7873] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:21:59 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:21:59.614612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b1e51-b986-4dab-b8d1-ac303f6579ba None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:59.629636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b1e51-b986-4dab-b8d1-ac303f6579ba None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:59.709825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385199-70fe-4987-9ca5-da3af5ed9c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:59.710567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385199-70fe-4987-9ca5-da3af5ed9c4e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:59.710913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385199-70fe-4987-9ca5-da3af5ed9c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:59.711260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385199-70fe-4987-9ca5-da3af5ed9c4e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:59.711983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61385199-70fe-4987-9ca5-da3af5ed9c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:59.714125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61385199-70fe-4987-9ca5-da3af5ed9c4e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:59.745243 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3922/7874] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:21:59 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:21:59.759796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d840d-9e51-4b12-afd4-1fa71b22ddf1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:59.771323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d840d-9e51-4b12-afd4-1fa71b22ddf1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:59.776664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58c63b-b8d1-41b9-bd80-912f7a5a5042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:59.777298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58c63b-b8d1-41b9-bd80-912f7a5a5042 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:59.777574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58c63b-b8d1-41b9-bd80-912f7a5a5042 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:59.777964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58c63b-b8d1-41b9-bd80-912f7a5a5042 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:59.778627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58c63b-b8d1-41b9-bd80-912f7a5a5042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:59.780501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58c63b-b8d1-41b9-bd80-912f7a5a5042 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:59.794259 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3953/7875] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:21:59.798076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385199-70fe-4987-9ca5-da3af5ed9c4e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:59.812228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385199-70fe-4987-9ca5-da3af5ed9c4e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:59.818754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ceead-b68b-4f74-92f0-984fae757db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:59.819443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ceead-b68b-4f74-92f0-984fae757db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:59.819674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ceead-b68b-4f74-92f0-984fae757db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:59.820240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ceead-b68b-4f74-92f0-984fae757db3 admin admin] PATH_INFO: `/v3/projects/eeaef3a2587447bfb10dab8d48b6b88a/users/2b4cf63ce38645b5a8fa3d61cba4ffb0/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:59.846569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ceead-b68b-4f74-92f0-984fae757db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeaef3a2587447bfb10dab8d48b6b88a, user_id=2b4cf63ce38645b5a8fa3d61cba4ffb0, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:59.849442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ceead-b68b-4f74-92f0-984fae757db3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:59.871834 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3923/7876] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:59 2022] PUT /identity/v3/projects/eeaef3a2587447bfb10dab8d48b6b88a/users/2b4cf63ce38645b5a8fa3d61cba4ffb0/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:21:59.875726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58c63b-b8d1-41b9-bd80-912f7a5a5042 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:21:59.890373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58c63b-b8d1-41b9-bd80-912f7a5a5042 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:21:59.962704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b28cad-b09d-4da2-baff-07aeb61bc685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:21:59.963234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b28cad-b09d-4da2-baff-07aeb61bc685 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:21:59.963512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b28cad-b09d-4da2-baff-07aeb61bc685 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:21:59.963786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b28cad-b09d-4da2-baff-07aeb61bc685 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:21:59.964414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b28cad-b09d-4da2-baff-07aeb61bc685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:21:59.966042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b28cad-b09d-4da2-baff-07aeb61bc685 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:21:59.986455 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3954/7877] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:21:59.991316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ceead-b68b-4f74-92f0-984fae757db3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:00.004182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ceead-b68b-4f74-92f0-984fae757db3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:00.009932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5322a4-fd91-4139-aed1-04cd5bfe0d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:00.010677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5322a4-fd91-4139-aed1-04cd5bfe0d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:00.011079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5322a4-fd91-4139-aed1-04cd5bfe0d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:00.011412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5322a4-fd91-4139-aed1-04cd5bfe0d05 admin admin] PATH_INFO: `/v3/projects/eeaef3a2587447bfb10dab8d48b6b88a/users/2b4cf63ce38645b5a8fa3d61cba4ffb0/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:00.014845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5322a4-fd91-4139-aed1-04cd5bfe0d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeaef3a2587447bfb10dab8d48b6b88a, user_id=2b4cf63ce38645b5a8fa3d61cba4ffb0, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:00.018791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5322a4-fd91-4139-aed1-04cd5bfe0d05 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:00.046345 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3924/7878] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:21:59 2022] PUT /identity/v3/projects/eeaef3a2587447bfb10dab8d48b6b88a/users/2b4cf63ce38645b5a8fa3d61cba4ffb0/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:22:00.048306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd16753-017e-47fc-973d-041d0a33cb36 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:00.048598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd16753-017e-47fc-973d-041d0a33cb36 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:00.048874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd16753-017e-47fc-973d-041d0a33cb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:00.088494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-abd16753-017e-47fc-973d-041d0a33cb36 None None] MFA Rules not processed for user `2b4cf63ce38645b5a8fa3d61cba4ffb0`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:00.093083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-abd16753-017e-47fc-973d-041d0a33cb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:00.198514 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3955/7879] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:00.209334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5322a4-fd91-4139-aed1-04cd5bfe0d05 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:00.211268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f5322a4-fd91-4139-aed1-04cd5bfe0d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:00.389653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:00.389653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5322a4-fd91-4139-aed1-04cd5bfe0d05 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:00.389653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1dda0-6af4-4a8c-827e-79bfb38fe3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:00.389653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1dda0-6af4-4a8c-827e-79bfb38fe3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:00.391482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1dda0-6af4-4a8c-827e-79bfb38fe3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:00.391482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1dda0-6af4-4a8c-827e-79bfb38fe3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:00.391482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1dda0-6af4-4a8c-827e-79bfb38fe3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:00.391482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1dda0-6af4-4a8c-827e-79bfb38fe3b6 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:00.391482 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3925/7880] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:01.994612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd16753-017e-47fc-973d-041d0a33cb36 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:02.086661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd16753-017e-47fc-973d-041d0a33cb36 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:02.095632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:02.096460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:02.096976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:02.097378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] PATH_INFO: `/v3/users/1d9ac3a33ead42ca80677f2a74a9dc51` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:02.098654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9ac3a33ead42ca80677f2a74a9dc51)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:02.100672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:02.116494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:02.178398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9ac3a33ead42ca80677f2a74a9dc51'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:02.187056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9ac3a33ead42ca80677f2a74a9dc51'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:02.203747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9ac3a33ead42ca80677f2a74a9dc51'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:02.215285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9ac3a33ead42ca80677f2a74a9dc51'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:02.216047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9ac3a33ead42ca80677f2a74a9dc51'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:02.224553 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3956/7881] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:22:01 2022] DELETE /identity/v3/users/1d9ac3a33ead42ca80677f2a74a9dc51 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:02.228888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b323da8-a559-4b32-8af8-53591bb7da28 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:02.230882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b323da8-a559-4b32-8af8-53591bb7da28 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:02.231479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b323da8-a559-4b32-8af8-53591bb7da28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:02.262257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0b323da8-a559-4b32-8af8-53591bb7da28 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:02.263556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0b323da8-a559-4b32-8af8-53591bb7da28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:02.351599 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3926/7882] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:02.361887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:02.363990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:02.445311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eab3aa-4416-45e0-b479-5b2994ff3c2d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:02.449558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fdf46-5acd-40e0-80be-bc90e969510c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:02.450550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fdf46-5acd-40e0-80be-bc90e969510c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:02.451277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fdf46-5acd-40e0-80be-bc90e969510c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:02.451723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fdf46-5acd-40e0-80be-bc90e969510c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:02.470927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172fdf46-5acd-40e0-80be-bc90e969510c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:02.473137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172fdf46-5acd-40e0-80be-bc90e969510c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:02.487611 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3957/7883] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:02.820299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b323da8-a559-4b32-8af8-53591bb7da28 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:02.821757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0b323da8-a559-4b32-8af8-53591bb7da28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:02.862673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b323da8-a559-4b32-8af8-53591bb7da28 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:02.868985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:02.869777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:02.870151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:02.870699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] PATH_INFO: `/v3/projects/684e4cca8e0541698b058aef092e8424` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:02.871946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=684e4cca8e0541698b058aef092e8424)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:02.874042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:02.946129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '684e4cca8e0541698b058aef092e8424', 'request_id': 'req-1bc2ca43-cd34-486f-b06c-26ca23d877f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:02.958402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '684e4cca8e0541698b058aef092e8424', 'request_id': 'req-1bc2ca43-cd34-486f-b06c-26ca23d877f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:02.968559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '684e4cca8e0541698b058aef092e8424', 'request_id': 'req-1bc2ca43-cd34-486f-b06c-26ca23d877f6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:02.985993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] The token cache is being invalidate because project 684e4cca8e0541698b058aef092e8424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:22:02.986269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:02.986950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:02.989100 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3927/7884] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:22:02 2022] DELETE /identity/v3/projects/684e4cca8e0541698b058aef092e8424 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:02.992802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fdf46-5acd-40e0-80be-bc90e969510c service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:03.004847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fdf46-5acd-40e0-80be-bc90e969510c service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:03.080158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:03.080886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:03.081260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:03.081631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] PATH_INFO: `/v3/users/2b4cf63ce38645b5a8fa3d61cba4ffb0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:03.083372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4cf63ce38645b5a8fa3d61cba4ffb0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:03.085505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:03.149067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:03.175371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4cf63ce38645b5a8fa3d61cba4ffb0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:03.187031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4cf63ce38645b5a8fa3d61cba4ffb0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:03.208784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4cf63ce38645b5a8fa3d61cba4ffb0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:03.218039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4cf63ce38645b5a8fa3d61cba4ffb0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:03.218833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4c09d48e-c43e-4772-9f5b-b0a72fa7666a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4cf63ce38645b5a8fa3d61cba4ffb0'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:03.227693 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3958/7885] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:22:02 2022] DELETE /identity/v3/users/2b4cf63ce38645b5a8fa3d61cba4ffb0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:03.573222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:03.576230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:03.598212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2ca43-cd34-486f-b06c-26ca23d877f6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:03.667574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:03.668208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:03.668493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:03.668764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] PATH_INFO: `/v3/projects/eeaef3a2587447bfb10dab8d48b6b88a` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:03.670455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeaef3a2587447bfb10dab8d48b6b88a)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:03.673009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:03.687890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:03.730562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeaef3a2587447bfb10dab8d48b6b88a', 'request_id': 'req-64b87581-560a-46c0-a81b-defc864b1e38'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:03.740486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeaef3a2587447bfb10dab8d48b6b88a', 'request_id': 'req-64b87581-560a-46c0-a81b-defc864b1e38'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:03.750624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeaef3a2587447bfb10dab8d48b6b88a', 'request_id': 'req-64b87581-560a-46c0-a81b-defc864b1e38'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:03.764292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] The token cache is being invalidate because project eeaef3a2587447bfb10dab8d48b6b88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:22:03.764742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:03.765430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-64b87581-560a-46c0-a81b-defc864b1e38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:03.767812 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3928/7886] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:22:03 2022] DELETE /identity/v3/projects/eeaef3a2587447bfb10dab8d48b6b88a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:03.774042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb98b0-1267-426e-9edb-c443b303f47a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:03.774319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb98b0-1267-426e-9edb-c443b303f47a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:03.774588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb98b0-1267-426e-9edb-c443b303f47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:03.795530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b6cb98b0-1267-426e-9edb-c443b303f47a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:03.796728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b6cb98b0-1267-426e-9edb-c443b303f47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:03.865344 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3959/7887] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:03.872996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c444-bebb-47c7-80ae-63d48aa62c72 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:03.873371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c444-bebb-47c7-80ae-63d48aa62c72 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:03.873707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c444-bebb-47c7-80ae-63d48aa62c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:03.898396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3cf3c444-bebb-47c7-80ae-63d48aa62c72 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:03.899614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3cf3c444-bebb-47c7-80ae-63d48aa62c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:03.908348 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3929/7888] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:03.912701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb98b0-1267-426e-9edb-c443b303f47a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:03.922989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb98b0-1267-426e-9edb-c443b303f47a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:03.929971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b63251-8f15-40cb-a939-ebf87dfa40b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:03.930543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b63251-8f15-40cb-a939-ebf87dfa40b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:03.930816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b63251-8f15-40cb-a939-ebf87dfa40b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:03.931086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b63251-8f15-40cb-a939-ebf87dfa40b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:03.931731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b63251-8f15-40cb-a939-ebf87dfa40b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:22:03.932031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b63251-8f15-40cb-a939-ebf87dfa40b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:03.933726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b63251-8f15-40cb-a939-ebf87dfa40b7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:03.950377 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3960/7889] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:22:03.951762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3c444-bebb-47c7-80ae-63d48aa62c72 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:03.962208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3c444-bebb-47c7-80ae-63d48aa62c72 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:03.967316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47386f35-2288-4f27-bf30-accad91ea1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:03.968037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47386f35-2288-4f27-bf30-accad91ea1ea admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:03.968417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47386f35-2288-4f27-bf30-accad91ea1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:03.968789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47386f35-2288-4f27-bf30-accad91ea1ea admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:03.969547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47386f35-2288-4f27-bf30-accad91ea1ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:03.971546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47386f35-2288-4f27-bf30-accad91ea1ea admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.006612 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3930/7890] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:22:03 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:22:04.007711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b63251-8f15-40cb-a939-ebf87dfa40b7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.020094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b63251-8f15-40cb-a939-ebf87dfa40b7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.084040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187581c-9c58-46c6-bf5c-b07a49a00f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.084760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187581c-9c58-46c6-bf5c-b07a49a00f8d admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.085080 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187581c-9c58-46c6-bf5c-b07a49a00f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.085357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187581c-9c58-46c6-bf5c-b07a49a00f8d admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.085998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187581c-9c58-46c6-bf5c-b07a49a00f8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.088034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187581c-9c58-46c6-bf5c-b07a49a00f8d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.112215 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3961/7891] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:22:04 2022] POST /identity/v3/users => generated 279 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:22:04.116850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47386f35-2288-4f27-bf30-accad91ea1ea admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.128190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47386f35-2288-4f27-bf30-accad91ea1ea admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.133669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e042a6-333f-479f-a9bc-89ea7e945cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.134380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e042a6-333f-479f-a9bc-89ea7e945cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.134713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e042a6-333f-479f-a9bc-89ea7e945cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.135079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e042a6-333f-479f-a9bc-89ea7e945cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.135770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e042a6-333f-479f-a9bc-89ea7e945cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.137640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e042a6-333f-479f-a9bc-89ea7e945cc5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.149335 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3931/7892] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:22:04.153311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187581c-9c58-46c6-bf5c-b07a49a00f8d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.163930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187581c-9c58-46c6-bf5c-b07a49a00f8d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.168782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8a0f4-b3fb-4b61-b457-26416b9b5d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.169432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8a0f4-b3fb-4b61-b457-26416b9b5d8c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.169712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8a0f4-b3fb-4b61-b457-26416b9b5d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.169948 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8a0f4-b3fb-4b61-b457-26416b9b5d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.170579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8a0f4-b3fb-4b61-b457-26416b9b5d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.172188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8a0f4-b3fb-4b61-b457-26416b9b5d8c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.187135 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3962/7893] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:22:04.190458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e042a6-333f-479f-a9bc-89ea7e945cc5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.202489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e042a6-333f-479f-a9bc-89ea7e945cc5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.210204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4357f8b-b3a6-4b57-98b8-b799dd11ebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.210861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4357f8b-b3a6-4b57-98b8-b799dd11ebb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.211112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4357f8b-b3a6-4b57-98b8-b799dd11ebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.211372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4357f8b-b3a6-4b57-98b8-b799dd11ebb9 admin admin] PATH_INFO: `/v3/projects/3371446ba6734f1589667be60d160a3f/users/bd36a641d0614882831df3f3c8e0cf6c/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.231451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4357f8b-b3a6-4b57-98b8-b799dd11ebb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3371446ba6734f1589667be60d160a3f, user_id=bd36a641d0614882831df3f3c8e0cf6c, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.234181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4357f8b-b3a6-4b57-98b8-b799dd11ebb9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.250771 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3932/7894] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:22:04 2022] PUT /identity/v3/projects/3371446ba6734f1589667be60d160a3f/users/bd36a641d0614882831df3f3c8e0cf6c/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:04.261354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea6ce8-f9ce-409b-8444-7b3484831d3c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.261779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea6ce8-f9ce-409b-8444-7b3484831d3c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.262139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea6ce8-f9ce-409b-8444-7b3484831d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.294433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-08ea6ce8-f9ce-409b-8444-7b3484831d3c None None] MFA Rules not processed for user `bd36a641d0614882831df3f3c8e0cf6c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:04.295721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-08ea6ce8-f9ce-409b-8444-7b3484831d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:04.299772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:04.369266 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3963/7895] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:22:04.373887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4357f8b-b3a6-4b57-98b8-b799dd11ebb9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.436663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4357f8b-b3a6-4b57-98b8-b799dd11ebb9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.441522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c4860-a0b9-4f42-9a98-671545ea3431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.442056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c4860-a0b9-4f42-9a98-671545ea3431 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.442328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c4860-a0b9-4f42-9a98-671545ea3431 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.442593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c4860-a0b9-4f42-9a98-671545ea3431 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.443202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c4860-a0b9-4f42-9a98-671545ea3431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.445123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c4860-a0b9-4f42-9a98-671545ea3431 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.471234 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3933/7896] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:22:04 2022] POST /identity/v3/projects => generated 357 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:22:04.475988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea6ce8-f9ce-409b-8444-7b3484831d3c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.487083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea6ce8-f9ce-409b-8444-7b3484831d3c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.547012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ffcc3-882e-45ee-b7d1-0af30d3302a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.547648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ffcc3-882e-45ee-b7d1-0af30d3302a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.547923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ffcc3-882e-45ee-b7d1-0af30d3302a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.548208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ffcc3-882e-45ee-b7d1-0af30d3302a7 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.548842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ffcc3-882e-45ee-b7d1-0af30d3302a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.550811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ffcc3-882e-45ee-b7d1-0af30d3302a7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.575476 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3964/7897] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:22:04 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:22:04.576680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c4860-a0b9-4f42-9a98-671545ea3431 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.590435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c4860-a0b9-4f42-9a98-671545ea3431 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.595121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca2b6e-943a-4e9b-aa75-cd9ccca7244e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.595715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca2b6e-943a-4e9b-aa75-cd9ccca7244e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.595991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca2b6e-943a-4e9b-aa75-cd9ccca7244e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.596255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca2b6e-943a-4e9b-aa75-cd9ccca7244e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.596884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca2b6e-943a-4e9b-aa75-cd9ccca7244e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.599208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca2b6e-943a-4e9b-aa75-cd9ccca7244e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.611524 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3934/7898] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:22:04.615783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ffcc3-882e-45ee-b7d1-0af30d3302a7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.626007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ffcc3-882e-45ee-b7d1-0af30d3302a7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.631365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89a59e-ed82-4a56-99df-933ea57df31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.632014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a59e-ed82-4a56-99df-933ea57df31d admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.632289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a59e-ed82-4a56-99df-933ea57df31d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.632559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a59e-ed82-4a56-99df-933ea57df31d admin admin] PATH_INFO: `/v3/projects/6236b65c253640ac9db39751f50c031d/users/0bbaf0660475492daa6e7265039e4b57/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.641282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6da9ba-0160-4689-a539-91c47057dfec None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.642021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6da9ba-0160-4689-a539-91c47057dfec None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.642021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6da9ba-0160-4689-a539-91c47057dfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.649261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89a59e-ed82-4a56-99df-933ea57df31d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6236b65c253640ac9db39751f50c031d, user_id=0bbaf0660475492daa6e7265039e4b57, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.651974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89a59e-ed82-4a56-99df-933ea57df31d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.663525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8d6da9ba-0160-4689-a539-91c47057dfec None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:04.664730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8d6da9ba-0160-4689-a539-91c47057dfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:04.666356 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3965/7899] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:22:04 2022] PUT /identity/v3/projects/6236b65c253640ac9db39751f50c031d/users/0bbaf0660475492daa6e7265039e4b57/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:22:04.670370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:04.670880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89a59e-ed82-4a56-99df-933ea57df31d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.724446 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3935/7900] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:22:04.736925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6da9ba-0160-4689-a539-91c47057dfec None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.737973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89a59e-ed82-4a56-99df-933ea57df31d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.738473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8d6da9ba-0160-4689-a539-91c47057dfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:04.742739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15347-ac9d-4d62-bb90-57f67d36011b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.743347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15347-ac9d-4d62-bb90-57f67d36011b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.743622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15347-ac9d-4d62-bb90-57f67d36011b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.743888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15347-ac9d-4d62-bb90-57f67d36011b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.744548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb15347-ac9d-4d62-bb90-57f67d36011b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.746155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb15347-ac9d-4d62-bb90-57f67d36011b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.756874 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3966/7901] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:22:04.761092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15347-ac9d-4d62-bb90-57f67d36011b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.771469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15347-ac9d-4d62-bb90-57f67d36011b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.776105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeef42-a9cd-462a-ab1c-6974e5cf5b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.776734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeef42-a9cd-462a-ab1c-6974e5cf5b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.777008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeef42-a9cd-462a-ab1c-6974e5cf5b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.777310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeef42-a9cd-462a-ab1c-6974e5cf5b5a admin admin] PATH_INFO: `/v3/projects/6236b65c253640ac9db39751f50c031d/users/0bbaf0660475492daa6e7265039e4b57/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.779216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeef42-a9cd-462a-ab1c-6974e5cf5b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6236b65c253640ac9db39751f50c031d, user_id=0bbaf0660475492daa6e7265039e4b57, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.781857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeef42-a9cd-462a-ab1c-6974e5cf5b5a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.784096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:04.793976 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3967/7902] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:22:04 2022] PUT /identity/v3/projects/6236b65c253640ac9db39751f50c031d/users/0bbaf0660475492daa6e7265039e4b57/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:22:04.805739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5f0b7-8d67-4954-aadf-f3881b656e16 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.806024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5f0b7-8d67-4954-aadf-f3881b656e16 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.806297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5f0b7-8d67-4954-aadf-f3881b656e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.815190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6da9ba-0160-4689-a539-91c47057dfec None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.834579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-aac5f0b7-8d67-4954-aadf-f3881b656e16 None None] MFA Rules not processed for user `0bbaf0660475492daa6e7265039e4b57`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:04.835771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-aac5f0b7-8d67-4954-aadf-f3881b656e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:04.877837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51469fa4-1cfa-47e7-9750-32f08b75ee91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.878451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51469fa4-1cfa-47e7-9750-32f08b75ee91 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.879204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51469fa4-1cfa-47e7-9750-32f08b75ee91 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.879476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51469fa4-1cfa-47e7-9750-32f08b75ee91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.907129 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3968/7903] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:22:04.912345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5f0b7-8d67-4954-aadf-f3881b656e16 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.921638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5f0b7-8d67-4954-aadf-f3881b656e16 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.927743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f24b62-7b2a-425f-b912-0c1256afc2c7 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbaf0660475492daa6e7265039e4b57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6236b65c253640ac9db39751f50c031d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.928336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24b62-7b2a-425f-b912-0c1256afc2c7 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.928607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24b62-7b2a-425f-b912-0c1256afc2c7 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.928873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24b62-7b2a-425f-b912-0c1256afc2c7 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.929559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f24b62-7b2a-425f-b912-0c1256afc2c7 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:22:04.929956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f24b62-7b2a-425f-b912-0c1256afc2c7 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.931607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f24b62-7b2a-425f-b912-0c1256afc2c7 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.939601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51469fa4-1cfa-47e7-9750-32f08b75ee91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.941395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51469fa4-1cfa-47e7-9750-32f08b75ee91 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:04.943690 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3969/7904] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:22:04.946222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:04.948119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f24b62-7b2a-425f-b912-0c1256afc2c7 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:04.951112 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3936/7905] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:22:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:22:04.958544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f24b62-7b2a-425f-b912-0c1256afc2c7 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:04.963590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827acf7-82da-44f0-a8bd-2e2d1cd0980a tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbaf0660475492daa6e7265039e4b57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6236b65c253640ac9db39751f50c031d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:04.964211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827acf7-82da-44f0-a8bd-2e2d1cd0980a tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:04.964483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827acf7-82da-44f0-a8bd-2e2d1cd0980a tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:04.964748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827acf7-82da-44f0-a8bd-2e2d1cd0980a tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:04.965399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827acf7-82da-44f0-a8bd-2e2d1cd0980a tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:04.967302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827acf7-82da-44f0-a8bd-2e2d1cd0980a tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:05.001082 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3970/7906] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:22:04 2022] POST /identity/v3/projects => generated 349 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:22:05.004253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51469fa4-1cfa-47e7-9750-32f08b75ee91 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:05.014746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51469fa4-1cfa-47e7-9750-32f08b75ee91 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:05.077820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f269b36-7366-435d-bbc5-94defb68e790 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbaf0660475492daa6e7265039e4b57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6236b65c253640ac9db39751f50c031d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:05.078509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f269b36-7366-435d-bbc5-94defb68e790 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:05.078798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f269b36-7366-435d-bbc5-94defb68e790 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:05.079069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f269b36-7366-435d-bbc5-94defb68e790 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:05.079722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f269b36-7366-435d-bbc5-94defb68e790 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:22:05.080050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f269b36-7366-435d-bbc5-94defb68e790 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:05.082321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f269b36-7366-435d-bbc5-94defb68e790 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:05.094659 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3937/7907] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:22:05.105425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827acf7-82da-44f0-a8bd-2e2d1cd0980a tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:05.109925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0827acf7-82da-44f0-a8bd-2e2d1cd0980a tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:05.176813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827acf7-82da-44f0-a8bd-2e2d1cd0980a tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:05.182244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467612ad-aaf4-43b1-bb81-c2faf2586806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:05.182926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467612ad-aaf4-43b1-bb81-c2faf2586806 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:05.183205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467612ad-aaf4-43b1-bb81-c2faf2586806 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:05.183479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467612ad-aaf4-43b1-bb81-c2faf2586806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:05.196014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467612ad-aaf4-43b1-bb81-c2faf2586806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:05.197589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467612ad-aaf4-43b1-bb81-c2faf2586806 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:05.209008 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3971/7908] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:22:05.340517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f269b36-7366-435d-bbc5-94defb68e790 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:05.351778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f269b36-7366-435d-bbc5-94defb68e790 tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:05.356918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbaf0660475492daa6e7265039e4b57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6236b65c253640ac9db39751f50c031d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:05.357696 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:05.358123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:05.358405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] PATH_INFO: `/v3/projects/4a5553d944e14d4ebdfeac80f6400c6f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:05.359527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4a5553d944e14d4ebdfeac80f6400c6f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:05.361521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:05.422823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5553d944e14d4ebdfeac80f6400c6f', 'request_id': 'req-a1a670f0-a943-486f-8251-3d6529cc052b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:05.431545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5553d944e14d4ebdfeac80f6400c6f', 'request_id': 'req-a1a670f0-a943-486f-8251-3d6529cc052b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:05.440559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5553d944e14d4ebdfeac80f6400c6f', 'request_id': 'req-a1a670f0-a943-486f-8251-3d6529cc052b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:05.452021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] The token cache is being invalidate because project 4a5553d944e14d4ebdfeac80f6400c6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:22:05.452301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:05.452962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-a1a670f0-a943-486f-8251-3d6529cc052b tempest-QuotasTest-1648600821 tempest-QuotasTest-1648600821-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:05.454662 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3938/7909] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:22:05 2022] DELETE /identity/v3/projects/4a5553d944e14d4ebdfeac80f6400c6f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:05.554223 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467612ad-aaf4-43b1-bb81-c2faf2586806 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:05.610497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:05.629916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467612ad-aaf4-43b1-bb81-c2faf2586806 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:05.705578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:05.706275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:05.706619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:05.707111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] PATH_INFO: `/v3/users/bd36a641d0614882831df3f3c8e0cf6c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:05.708615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd36a641d0614882831df3f3c8e0cf6c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:05.710690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:05.764467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd36a641d0614882831df3f3c8e0cf6c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:05.772588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd36a641d0614882831df3f3c8e0cf6c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:05.787640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd36a641d0614882831df3f3c8e0cf6c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:05.795744 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd36a641d0614882831df3f3c8e0cf6c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:05.796468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd36a641d0614882831df3f3c8e0cf6c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:05.802838 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3972/7910] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:22:05 2022] DELETE /identity/v3/users/bd36a641d0614882831df3f3c8e0cf6c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:05.807313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091b4fb-b65c-4b75-91fc-9e55d6b93676 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:05.807833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091b4fb-b65c-4b75-91fc-9e55d6b93676 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:05.807833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091b4fb-b65c-4b75-91fc-9e55d6b93676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:05.827330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9091b4fb-b65c-4b75-91fc-9e55d6b93676 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:05.828765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9091b4fb-b65c-4b75-91fc-9e55d6b93676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:05.896339 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3939/7911] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:05.906258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:05.907862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:05.973134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62972470-6186-4e34-b71f-eac22e6932e4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:05.976816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4668d2-5de5-49df-81f1-012a2b7d11db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:05.977455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4668d2-5de5-49df-81f1-012a2b7d11db service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:05.977733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4668d2-5de5-49df-81f1-012a2b7d11db service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:05.978003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4668d2-5de5-49df-81f1-012a2b7d11db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:05.987485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4668d2-5de5-49df-81f1-012a2b7d11db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:05.989592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4668d2-5de5-49df-81f1-012a2b7d11db service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:05.999036 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3973/7912] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:22:06.269827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091b4fb-b65c-4b75-91fc-9e55d6b93676 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:06.271373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9091b4fb-b65c-4b75-91fc-9e55d6b93676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:06.287938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091b4fb-b65c-4b75-91fc-9e55d6b93676 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:06.292686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:06.293343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:06.293619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:06.293872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] PATH_INFO: `/v3/projects/3371446ba6734f1589667be60d160a3f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:06.294987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3371446ba6734f1589667be60d160a3f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:06.296902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:06.309362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:06.345240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3371446ba6734f1589667be60d160a3f', 'request_id': 'req-8d66b77e-c3b4-46b3-a24e-10e020819bd6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:06.355167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3371446ba6734f1589667be60d160a3f', 'request_id': 'req-8d66b77e-c3b4-46b3-a24e-10e020819bd6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:06.361861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3371446ba6734f1589667be60d160a3f', 'request_id': 'req-8d66b77e-c3b4-46b3-a24e-10e020819bd6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:06.374174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] The token cache is being invalidate because project 3371446ba6734f1589667be60d160a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:22:06.374461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:06.375145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:06.376659 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3940/7913] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:22:06 2022] DELETE /identity/v3/projects/3371446ba6734f1589667be60d160a3f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:06.381025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4668d2-5de5-49df-81f1-012a2b7d11db service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:06.391446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4668d2-5de5-49df-81f1-012a2b7d11db service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:06.403901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:06.455252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:06.455963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:06.456241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:06.457122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] PATH_INFO: `/v3/users/0bbaf0660475492daa6e7265039e4b57` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:06.458329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bbaf0660475492daa6e7265039e4b57)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:06.460599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:06.515144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bbaf0660475492daa6e7265039e4b57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:06.522597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bbaf0660475492daa6e7265039e4b57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:06.535858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bbaf0660475492daa6e7265039e4b57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:06.545067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bbaf0660475492daa6e7265039e4b57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:06.545794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bbaf0660475492daa6e7265039e4b57'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:06.553105 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3974/7914] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:22:06 2022] DELETE /identity/v3/users/0bbaf0660475492daa6e7265039e4b57 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:07.014752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.016290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:07.032702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66b77e-c3b4-46b3-a24e-10e020819bd6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.068660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.070297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:07.125751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.126291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.126568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.126832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] PATH_INFO: `/v3/projects/6236b65c253640ac9db39751f50c031d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.127941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6236b65c253640ac9db39751f50c031d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.130437 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.142123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9bd24-e56d-4e35-a18e-557f9fb6e145 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.147662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.148289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.148562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.148827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] PATH_INFO: `/v3/users/afa9c93b9e3d41d69f92909fd98a34a2` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.149966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] RBAC: Authorizing `identity:delete_user(user_id=afa9c93b9e3d41d69f92909fd98a34a2)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.151860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.180838 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6236b65c253640ac9db39751f50c031d', 'request_id': 'req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.189094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6236b65c253640ac9db39751f50c031d', 'request_id': 'req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.201222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:07.202542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6236b65c253640ac9db39751f50c031d', 'request_id': 'req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.212442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afa9c93b9e3d41d69f92909fd98a34a2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.216251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] The token cache is being invalidate because project 6236b65c253640ac9db39751f50c031d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:22:07.216558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.217274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd2f5caf-e949-4b33-9dfe-6e3ba6854f21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.218828 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3941/7915] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:22:07 2022] DELETE /identity/v3/projects/6236b65c253640ac9db39751f50c031d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:07.220504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afa9c93b9e3d41d69f92909fd98a34a2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.224433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df5ae-b5c8-43ab-a6f7-96222ecead8c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.224710 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df5ae-b5c8-43ab-a6f7-96222ecead8c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.224976 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df5ae-b5c8-43ab-a6f7-96222ecead8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.234575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afa9c93b9e3d41d69f92909fd98a34a2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.243470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afa9c93b9e3d41d69f92909fd98a34a2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.244219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-cb7ff980-cd10-4158-bb98-f7ca5b5e432f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afa9c93b9e3d41d69f92909fd98a34a2'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.245357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5b0df5ae-b5c8-43ab-a6f7-96222ecead8c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:07.246556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5b0df5ae-b5c8-43ab-a6f7-96222ecead8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:07.250908 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3975/7916] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:22:07 2022] DELETE /identity/v3/users/afa9c93b9e3d41d69f92909fd98a34a2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:22:07.276145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:07.316639 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3942/7917] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:22:07.324786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef99e9-f086-4208-a8ee-7c28080018d9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.325218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef99e9-f086-4208-a8ee-7c28080018d9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.325518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef99e9-f086-4208-a8ee-7c28080018d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.346916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e7ef99e9-f086-4208-a8ee-7c28080018d9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:07.348785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e7ef99e9-f086-4208-a8ee-7c28080018d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:07.423098 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3976/7918] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:22:07.428866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0df5ae-b5c8-43ab-a6f7-96222ecead8c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.439283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef99e9-f086-4208-a8ee-7c28080018d9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.440686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e7ef99e9-f086-4208-a8ee-7c28080018d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:07.443371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0df5ae-b5c8-43ab-a6f7-96222ecead8c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.448805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468b399-d787-4b35-b095-c5a13c130ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.449648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468b399-d787-4b35-b095-c5a13c130ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.450023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468b399-d787-4b35-b095-c5a13c130ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.450860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468b399-d787-4b35-b095-c5a13c130ce1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.451387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468b399-d787-4b35-b095-c5a13c130ce1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:22:07.451779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468b399-d787-4b35-b095-c5a13c130ce1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.453572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468b399-d787-4b35-b095-c5a13c130ce1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.453709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef99e9-f086-4208-a8ee-7c28080018d9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.460305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.460942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.461403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.461527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] PATH_INFO: `/v3/projects/8ed96672d3d344a9b00edd3377df0ad3` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.463228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed96672d3d344a9b00edd3377df0ad3)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.465997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.470604 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3943/7919] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:22:07.471648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468b399-d787-4b35-b095-c5a13c130ce1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.484630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468b399-d787-4b35-b095-c5a13c130ce1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.490257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ff29-1549-4532-a87e-6b2e90ce78f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.490851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ff29-1549-4532-a87e-6b2e90ce78f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.491121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ff29-1549-4532-a87e-6b2e90ce78f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.491399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ff29-1549-4532-a87e-6b2e90ce78f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.491991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259ff29-1549-4532-a87e-6b2e90ce78f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.493939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259ff29-1549-4532-a87e-6b2e90ce78f4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.520389 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed96672d3d344a9b00edd3377df0ad3', 'request_id': 'req-3a5b5944-eeaf-464b-b577-4aaba0578ad3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.527919 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3944/7920] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:22:07 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:22:07.529234 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ff29-1549-4532-a87e-6b2e90ce78f4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.529384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed96672d3d344a9b00edd3377df0ad3', 'request_id': 'req-3a5b5944-eeaf-464b-b577-4aaba0578ad3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.536115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed96672d3d344a9b00edd3377df0ad3', 'request_id': 'req-3a5b5944-eeaf-464b-b577-4aaba0578ad3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.547379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] The token cache is being invalidate because project 8ed96672d3d344a9b00edd3377df0ad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:22:07.547694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.548364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-3a5b5944-eeaf-464b-b577-4aaba0578ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:07.549927 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3977/7921] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:22:07 2022] DELETE /identity/v3/projects/8ed96672d3d344a9b00edd3377df0ad3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:22:07.555982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9f7cb-336e-4fc5-8553-c9b770eea6bc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.556921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9f7cb-336e-4fc5-8553-c9b770eea6bc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.557103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9f7cb-336e-4fc5-8553-c9b770eea6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.578735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-38c9f7cb-336e-4fc5-8553-c9b770eea6bc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:07.579964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-38c9f7cb-336e-4fc5-8553-c9b770eea6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:07.595623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ff29-1549-4532-a87e-6b2e90ce78f4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.601510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15f079-fda1-477f-85df-daf6a2705af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.602110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15f079-fda1-477f-85df-daf6a2705af6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.602384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15f079-fda1-477f-85df-daf6a2705af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.602654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15f079-fda1-477f-85df-daf6a2705af6 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.603334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15f079-fda1-477f-85df-daf6a2705af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.605528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15f079-fda1-477f-85df-daf6a2705af6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.613366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:07.629743 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3945/7922] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:22:07 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:22:07.633320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15f079-fda1-477f-85df-daf6a2705af6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.634941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ba15f079-fda1-477f-85df-daf6a2705af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:07.646448 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3978/7923] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:22:07.647455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15f079-fda1-477f-85df-daf6a2705af6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.652091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015aee7-4005-4021-84e8-b74a2d2c3e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.652678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015aee7-4005-4021-84e8-b74a2d2c3e58 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.652953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015aee7-4005-4021-84e8-b74a2d2c3e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.653251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015aee7-4005-4021-84e8-b74a2d2c3e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.653881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015aee7-4005-4021-84e8-b74a2d2c3e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.654605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b50a-18a0-4fac-a516-d81183f0188a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.654891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b50a-18a0-4fac-a516-d81183f0188a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.655165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b50a-18a0-4fac-a516-d81183f0188a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.655467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015aee7-4005-4021-84e8-b74a2d2c3e58 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.668346 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3946/7924] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:22:07.672253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015aee7-4005-4021-84e8-b74a2d2c3e58 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.676490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1046b50a-18a0-4fac-a516-d81183f0188a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:07.677767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1046b50a-18a0-4fac-a516-d81183f0188a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:07.683977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015aee7-4005-4021-84e8-b74a2d2c3e58 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.686183 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3979/7925] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:22:07.688565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0e45f-2712-45cd-b54b-4baea46298dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.689365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0e45f-2712-45cd-b54b-4baea46298dc admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.689507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0e45f-2712-45cd-b54b-4baea46298dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.689733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0e45f-2712-45cd-b54b-4baea46298dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.690162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046b50a-18a0-4fac-a516-d81183f0188a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.690354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0e45f-2712-45cd-b54b-4baea46298dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.692577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0e45f-2712-45cd-b54b-4baea46298dc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.699518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046b50a-18a0-4fac-a516-d81183f0188a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.703338 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3947/7926] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:22:07.704183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef1cc0-7cec-4752-b535-050d58c66ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.704768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef1cc0-7cec-4752-b535-050d58c66ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.705057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef1cc0-7cec-4752-b535-050d58c66ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.705374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef1cc0-7cec-4752-b535-050d58c66ba3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.706006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef1cc0-7cec-4752-b535-050d58c66ba3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:22:07.706327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef1cc0-7cec-4752-b535-050d58c66ba3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.707640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0e45f-2712-45cd-b54b-4baea46298dc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.708043 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef1cc0-7cec-4752-b535-050d58c66ba3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.717808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0e45f-2712-45cd-b54b-4baea46298dc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.724256 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3980/7927] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:22:07.724447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefdec0-d755-4eb2-aead-26b200a9f3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.724588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefdec0-d755-4eb2-aead-26b200a9f3bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.724809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefdec0-d755-4eb2-aead-26b200a9f3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.725118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefdec0-d755-4eb2-aead-26b200a9f3bd admin admin] PATH_INFO: `/v3/projects/b10af6f804204512a191cfd026eb7b9d/users/303a08b0ba924da0a8ce9eed05768f03/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.725277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef1cc0-7cec-4752-b535-050d58c66ba3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.735050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef1cc0-7cec-4752-b535-050d58c66ba3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.740252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ce176-a427-4d21-abb9-17fc2dc05f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.740825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ce176-a427-4d21-abb9-17fc2dc05f21 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.741131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ce176-a427-4d21-abb9-17fc2dc05f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.741318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefdec0-d755-4eb2-aead-26b200a9f3bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10af6f804204512a191cfd026eb7b9d, user_id=303a08b0ba924da0a8ce9eed05768f03, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.741470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ce176-a427-4d21-abb9-17fc2dc05f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.741986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ce176-a427-4d21-abb9-17fc2dc05f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.743840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ce176-a427-4d21-abb9-17fc2dc05f21 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.744011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefdec0-d755-4eb2-aead-26b200a9f3bd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.757221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3948/7928] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:22:07 2022] PUT /identity/v3/projects/b10af6f804204512a191cfd026eb7b9d/users/303a08b0ba924da0a8ce9eed05768f03/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:07.768459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16f1b9-d1cd-4b0b-bd9b-65c9f9b5187a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.768863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16f1b9-d1cd-4b0b-bd9b-65c9f9b5187a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.769372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16f1b9-d1cd-4b0b-bd9b-65c9f9b5187a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.776257 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3981/7929] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:22:07 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:22:07.780520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ce176-a427-4d21-abb9-17fc2dc05f21 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.799296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3c16f1b9-d1cd-4b0b-bd9b-65c9f9b5187a None None] MFA Rules not processed for user `303a08b0ba924da0a8ce9eed05768f03`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:07.800857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3c16f1b9-d1cd-4b0b-bd9b-65c9f9b5187a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:07.831031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:07.841976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ce176-a427-4d21-abb9-17fc2dc05f21 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.846595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b11b43-7375-40a3-b4fd-5f46cbfbc51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.847420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b11b43-7375-40a3-b4fd-5f46cbfbc51c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.847677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b11b43-7375-40a3-b4fd-5f46cbfbc51c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.847942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b11b43-7375-40a3-b4fd-5f46cbfbc51c admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.848554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b11b43-7375-40a3-b4fd-5f46cbfbc51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.850484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b11b43-7375-40a3-b4fd-5f46cbfbc51c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.870221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3949/7930] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:22:07.871742 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3982/7931] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:22:07 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:22:07.876336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16f1b9-d1cd-4b0b-bd9b-65c9f9b5187a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.883268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b11b43-7375-40a3-b4fd-5f46cbfbc51c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.885241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e2b11b43-7375-40a3-b4fd-5f46cbfbc51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:07.888294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16f1b9-d1cd-4b0b-bd9b-65c9f9b5187a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.892999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec69d8-4442-4341-9edb-a132284b1471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.893692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec69d8-4442-4341-9edb-a132284b1471 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.894648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec69d8-4442-4341-9edb-a132284b1471 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.895193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec69d8-4442-4341-9edb-a132284b1471 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.895636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec69d8-4442-4341-9edb-a132284b1471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.897280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec69d8-4442-4341-9edb-a132284b1471 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.907417 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3950/7932] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:22:07.911562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec69d8-4442-4341-9edb-a132284b1471 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.920967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec69d8-4442-4341-9edb-a132284b1471 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.925799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ab047-3ab3-4dae-8c5a-0fc9616f787f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.926363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab047-3ab3-4dae-8c5a-0fc9616f787f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.926635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab047-3ab3-4dae-8c5a-0fc9616f787f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.926898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab047-3ab3-4dae-8c5a-0fc9616f787f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.927507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ab047-3ab3-4dae-8c5a-0fc9616f787f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.929153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ab047-3ab3-4dae-8c5a-0fc9616f787f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.939085 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3951/7933] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:22:07.943548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ab047-3ab3-4dae-8c5a-0fc9616f787f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:07.950486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b11b43-7375-40a3-b4fd-5f46cbfbc51c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.953104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ab047-3ab3-4dae-8c5a-0fc9616f787f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:07.954233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd39463-9395-4f9c-bf66-06bfb3419bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.954831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd39463-9395-4f9c-bf66-06bfb3419bfe service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.955101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd39463-9395-4f9c-bf66-06bfb3419bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.955365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd39463-9395-4f9c-bf66-06bfb3419bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.958303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03f94b-48ba-44de-8072-4f4664e41697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:07.958923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03f94b-48ba-44de-8072-4f4664e41697 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:07.959198 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03f94b-48ba-44de-8072-4f4664e41697 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:07.959472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03f94b-48ba-44de-8072-4f4664e41697 admin admin] PATH_INFO: `/v3/projects/b64eab9ca65b4247bef1740b0041a80e/users/e70e53f89b2d4b73abfe41829a61fc92/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:07.963942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd39463-9395-4f9c-bf66-06bfb3419bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.965392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd39463-9395-4f9c-bf66-06bfb3419bfe service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.974979 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3983/7934] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:22:07.975206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03f94b-48ba-44de-8072-4f4664e41697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b64eab9ca65b4247bef1740b0041a80e, user_id=e70e53f89b2d4b73abfe41829a61fc92, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:07.977935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03f94b-48ba-44de-8072-4f4664e41697 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:07.992748 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3952/7935] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:22:07 2022] PUT /identity/v3/projects/b64eab9ca65b4247bef1740b0041a80e/users/e70e53f89b2d4b73abfe41829a61fc92/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:22:08.001983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd39463-9395-4f9c-bf66-06bfb3419bfe service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:08.003080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd383ae-a700-4240-b1d2-43ab695c1507 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:08.003438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd383ae-a700-4240-b1d2-43ab695c1507 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:08.003849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd383ae-a700-4240-b1d2-43ab695c1507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:08.013064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd39463-9395-4f9c-bf66-06bfb3419bfe service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:08.031466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9bd383ae-a700-4240-b1d2-43ab695c1507 None None] MFA Rules not processed for user `e70e53f89b2d4b73abfe41829a61fc92`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:08.032947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9bd383ae-a700-4240-b1d2-43ab695c1507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:08.040112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:08.073989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:08.074642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:08.074926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:08.075174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] PATH_INFO: `/v3/users/303a08b0ba924da0a8ce9eed05768f03` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:08.076282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303a08b0ba924da0a8ce9eed05768f03)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:08.078500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:08.101955 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3953/7936] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:22:08.114295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd383ae-a700-4240-b1d2-43ab695c1507 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:08.116108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9bd383ae-a700-4240-b1d2-43ab695c1507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:08.130460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303a08b0ba924da0a8ce9eed05768f03'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:08.137716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303a08b0ba924da0a8ce9eed05768f03'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:08.151332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303a08b0ba924da0a8ce9eed05768f03'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:08.160676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303a08b0ba924da0a8ce9eed05768f03'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:08.161392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b4e50bf1-5fe9-4fe6-bb9d-1da8bcbec530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303a08b0ba924da0a8ce9eed05768f03'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:08.167689 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3984/7937] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:22:08 2022] DELETE /identity/v3/users/303a08b0ba924da0a8ce9eed05768f03 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:08.173979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11993189-6ec6-41de-893f-5b79b504f5da None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:08.174246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11993189-6ec6-41de-893f-5b79b504f5da None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:08.174520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11993189-6ec6-41de-893f-5b79b504f5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:08.177353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd383ae-a700-4240-b1d2-43ab695c1507 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:08.194538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-11993189-6ec6-41de-893f-5b79b504f5da None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:08.196252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-11993189-6ec6-41de-893f-5b79b504f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:08.206077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:08.233688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c8a8e-e2bd-4843-bbaa-beb081fc895c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:08.234287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c8a8e-e2bd-4843-bbaa-beb081fc895c service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:08.234557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c8a8e-e2bd-4843-bbaa-beb081fc895c service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:08.234826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c8a8e-e2bd-4843-bbaa-beb081fc895c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:08.236601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-897c8a8e-e2bd-4843-bbaa-beb081fc895c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:08.264750 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3985/7938] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:08.275667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11993189-6ec6-41de-893f-5b79b504f5da None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:08.277022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-11993189-6ec6-41de-893f-5b79b504f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:08.302096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c8a8e-e2bd-4843-bbaa-beb081fc895c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:08.303556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c8a8e-e2bd-4843-bbaa-beb081fc895c service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:08.313454 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3954/7939] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:22:08.330768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:08.339202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11993189-6ec6-41de-893f-5b79b504f5da None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:08.343656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6926a-c802-4b87-bd63-4d35a579422b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:08.344262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6926a-c802-4b87-bd63-4d35a579422b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:08.344536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6926a-c802-4b87-bd63-4d35a579422b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:08.344917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6926a-c802-4b87-bd63-4d35a579422b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:08.355932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6926a-c802-4b87-bd63-4d35a579422b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:08.357434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6926a-c802-4b87-bd63-4d35a579422b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:08.368188 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3986/7940] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:22:08.697462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c8a8e-e2bd-4843-bbaa-beb081fc895c service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:08.699151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-897c8a8e-e2bd-4843-bbaa-beb081fc895c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:08.729590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c8a8e-e2bd-4843-bbaa-beb081fc895c service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:08.734996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:08.735691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:08.736046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:08.736388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] PATH_INFO: `/v3/projects/b10af6f804204512a191cfd026eb7b9d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:08.737720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10af6f804204512a191cfd026eb7b9d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:08.742909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:08.811328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10af6f804204512a191cfd026eb7b9d', 'request_id': 'req-282d108e-d7ee-43b0-95d7-0e9452bafc5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:08.821375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10af6f804204512a191cfd026eb7b9d', 'request_id': 'req-282d108e-d7ee-43b0-95d7-0e9452bafc5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:08.829845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10af6f804204512a191cfd026eb7b9d', 'request_id': 'req-282d108e-d7ee-43b0-95d7-0e9452bafc5f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:08.843806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] The token cache is being invalidate because project b10af6f804204512a191cfd026eb7b9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:22:08.844993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:08.844993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-282d108e-d7ee-43b0-95d7-0e9452bafc5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:08.848094 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3955/7941] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:22:08 2022] DELETE /identity/v3/projects/b10af6f804204512a191cfd026eb7b9d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:08.852771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aded43e-b2b9-4be2-af74-73e6088f9929 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:08.853367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aded43e-b2b9-4be2-af74-73e6088f9929 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:08.853515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aded43e-b2b9-4be2-af74-73e6088f9929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:08.877875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2aded43e-b2b9-4be2-af74-73e6088f9929 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:08.879632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2aded43e-b2b9-4be2-af74-73e6088f9929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:08.953153 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3987/7942] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:08.961562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02babc59-b2c7-4423-b312-acca70266ecf None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:08.961846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02babc59-b2c7-4423-b312-acca70266ecf None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:08.962116 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02babc59-b2c7-4423-b312-acca70266ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:08.989592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-02babc59-b2c7-4423-b312-acca70266ecf None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:08.990841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-02babc59-b2c7-4423-b312-acca70266ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:08.999787 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3956/7943] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:09.004327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aded43e-b2b9-4be2-af74-73e6088f9929 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:09.014737 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aded43e-b2b9-4be2-af74-73e6088f9929 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:09.020354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72f955-0586-4df1-9c28-03f7e7d967a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:09.021064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72f955-0586-4df1-9c28-03f7e7d967a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:09.021285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72f955-0586-4df1-9c28-03f7e7d967a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:09.021535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72f955-0586-4df1-9c28-03f7e7d967a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:09.022176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72f955-0586-4df1-9c28-03f7e7d967a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:22:09.022484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72f955-0586-4df1-9c28-03f7e7d967a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:09.024681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72f955-0586-4df1-9c28-03f7e7d967a1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:09.038367 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3988/7944] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:22:09.043143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02babc59-b2c7-4423-b312-acca70266ecf None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:09.054661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02babc59-b2c7-4423-b312-acca70266ecf None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:09.059406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320ce11-4dad-4b97-bcdc-2572849fcdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:09.059973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320ce11-4dad-4b97-bcdc-2572849fcdfa admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:09.060340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320ce11-4dad-4b97-bcdc-2572849fcdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:09.060613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320ce11-4dad-4b97-bcdc-2572849fcdfa admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:09.061314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320ce11-4dad-4b97-bcdc-2572849fcdfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:09.063217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320ce11-4dad-4b97-bcdc-2572849fcdfa admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:09.098079 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3957/7945] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:22:09 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:22:09.102170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72f955-0586-4df1-9c28-03f7e7d967a1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:09.113404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72f955-0586-4df1-9c28-03f7e7d967a1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:09.174908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64783d59-5246-4ea5-85f8-483974736479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:09.175440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64783d59-5246-4ea5-85f8-483974736479 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:09.175715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64783d59-5246-4ea5-85f8-483974736479 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:09.175986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64783d59-5246-4ea5-85f8-483974736479 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:09.176640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64783d59-5246-4ea5-85f8-483974736479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:09.178597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64783d59-5246-4ea5-85f8-483974736479 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:09.203316 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3989/7946] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:22:09 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:22:09.209262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320ce11-4dad-4b97-bcdc-2572849fcdfa admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:09.220203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320ce11-4dad-4b97-bcdc-2572849fcdfa admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:09.226368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381430a8-376c-424d-a057-26a151b0399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:09.227501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381430a8-376c-424d-a057-26a151b0399c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:09.227501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381430a8-376c-424d-a057-26a151b0399c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:09.227501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381430a8-376c-424d-a057-26a151b0399c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:09.228171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381430a8-376c-424d-a057-26a151b0399c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:09.230207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381430a8-376c-424d-a057-26a151b0399c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:09.244472 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3958/7947] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:22:09.248932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64783d59-5246-4ea5-85f8-483974736479 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:09.261579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64783d59-5246-4ea5-85f8-483974736479 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:09.266730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5ac8a-5d93-4f53-9924-64f4c19c15c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:09.267330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5ac8a-5d93-4f53-9924-64f4c19c15c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:09.267601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5ac8a-5d93-4f53-9924-64f4c19c15c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:09.267868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5ac8a-5d93-4f53-9924-64f4c19c15c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:09.268495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5ac8a-5d93-4f53-9924-64f4c19c15c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:09.270987 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5ac8a-5d93-4f53-9924-64f4c19c15c7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:09.282365 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3990/7948] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:22:09.286720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381430a8-376c-424d-a057-26a151b0399c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:09.297371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381430a8-376c-424d-a057-26a151b0399c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:09.302067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e4f1c-f7e2-4fc6-b010-1b00a3074634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:09.302693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e4f1c-f7e2-4fc6-b010-1b00a3074634 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:09.302972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e4f1c-f7e2-4fc6-b010-1b00a3074634 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:09.303261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e4f1c-f7e2-4fc6-b010-1b00a3074634 admin admin] PATH_INFO: `/v3/projects/a8d303e1d1b9482d99655f130cc231db/users/55b36b56e72a48dda48976dd3ee45e02/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:09.318624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e4f1c-f7e2-4fc6-b010-1b00a3074634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d303e1d1b9482d99655f130cc231db, user_id=55b36b56e72a48dda48976dd3ee45e02, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:09.321292 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e4f1c-f7e2-4fc6-b010-1b00a3074634 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:09.334736 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3959/7949] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:22:09 2022] PUT /identity/v3/projects/a8d303e1d1b9482d99655f130cc231db/users/55b36b56e72a48dda48976dd3ee45e02/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:09.345173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e827fe-bdb1-474a-bc48-98d6d448cbaa None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:09.345743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e827fe-bdb1-474a-bc48-98d6d448cbaa None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:09.345959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e827fe-bdb1-474a-bc48-98d6d448cbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:09.374250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f0e827fe-bdb1-474a-bc48-98d6d448cbaa None None] MFA Rules not processed for user `55b36b56e72a48dda48976dd3ee45e02`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:09.375443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f0e827fe-bdb1-474a-bc48-98d6d448cbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:09.424376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:09.446345 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3991/7950] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:09.456237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e4f1c-f7e2-4fc6-b010-1b00a3074634 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:09.458063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-762e4f1c-f7e2-4fc6-b010-1b00a3074634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:09.463318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:09.520470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e4f1c-f7e2-4fc6-b010-1b00a3074634 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:09.524243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd656ebf-6e8e-46de-917f-977ef318ec61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:09.524837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd656ebf-6e8e-46de-917f-977ef318ec61 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:09.525153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd656ebf-6e8e-46de-917f-977ef318ec61 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:09.525421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd656ebf-6e8e-46de-917f-977ef318ec61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:09.533766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd656ebf-6e8e-46de-917f-977ef318ec61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:09.535827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd656ebf-6e8e-46de-917f-977ef318ec61 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:09.544778 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3960/7951] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:10.971501 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e827fe-bdb1-474a-bc48-98d6d448cbaa None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:10.973903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f0e827fe-bdb1-474a-bc48-98d6d448cbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:11.046520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e827fe-bdb1-474a-bc48-98d6d448cbaa None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:11.051234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:11.051874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:11.052179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:11.052448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] PATH_INFO: `/v3/users/e70e53f89b2d4b73abfe41829a61fc92` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:11.053635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e70e53f89b2d4b73abfe41829a61fc92)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:11.055549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:11.109407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e70e53f89b2d4b73abfe41829a61fc92'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:11.117398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e70e53f89b2d4b73abfe41829a61fc92'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:11.130307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e70e53f89b2d4b73abfe41829a61fc92'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:11.139376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e70e53f89b2d4b73abfe41829a61fc92'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:11.140093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e70e53f89b2d4b73abfe41829a61fc92'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:11.147338 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3992/7952] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:22:10 2022] DELETE /identity/v3/users/e70e53f89b2d4b73abfe41829a61fc92 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:11.151936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799897e-ffd8-4605-a433-b736f540f954 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:11.152410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799897e-ffd8-4605-a433-b736f540f954 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:11.152544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799897e-ffd8-4605-a433-b736f540f954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:11.172968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b799897e-ffd8-4605-a433-b736f540f954 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:11.174232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b799897e-ffd8-4605-a433-b736f540f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:11.240357 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3961/7953] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:22:11.250270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:11.251684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:11.318168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0d20c-32d9-486d-8994-d46b5b2e54c6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:11.322007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d402aa0-878a-4bac-85b7-a9ce90629a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:11.322614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d402aa0-878a-4bac-85b7-a9ce90629a71 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:11.322886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d402aa0-878a-4bac-85b7-a9ce90629a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:11.323152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d402aa0-878a-4bac-85b7-a9ce90629a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:11.333005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d402aa0-878a-4bac-85b7-a9ce90629a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:11.334452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d402aa0-878a-4bac-85b7-a9ce90629a71 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:11.344373 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3993/7954] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:11.975370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799897e-ffd8-4605-a433-b736f540f954 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:11.977790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b799897e-ffd8-4605-a433-b736f540f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:11.995743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799897e-ffd8-4605-a433-b736f540f954 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:12.004571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:12.005340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:12.006289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:12.006629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] PATH_INFO: `/v3/projects/b64eab9ca65b4247bef1740b0041a80e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:12.007860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b64eab9ca65b4247bef1740b0041a80e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:12.010027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:12.064606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b64eab9ca65b4247bef1740b0041a80e', 'request_id': 'req-6daf22ac-1019-4a25-8c1f-b5da1234643e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:12.073510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b64eab9ca65b4247bef1740b0041a80e', 'request_id': 'req-6daf22ac-1019-4a25-8c1f-b5da1234643e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:12.080666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b64eab9ca65b4247bef1740b0041a80e', 'request_id': 'req-6daf22ac-1019-4a25-8c1f-b5da1234643e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:12.093096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] The token cache is being invalidate because project b64eab9ca65b4247bef1740b0041a80e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:22:12.093392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:12.094065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:12.095546 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3962/7955] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:22:11 2022] DELETE /identity/v3/projects/b64eab9ca65b4247bef1740b0041a80e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:21.339985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:22.089898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:28.605767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d402aa0-878a-4bac-85b7-a9ce90629a71 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:28.607251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6d402aa0-878a-4bac-85b7-a9ce90629a71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:28.718451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d402aa0-878a-4bac-85b7-a9ce90629a71 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:28.786100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad5647-f8c0-41e1-97c4-09e6fb765860 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:28.786785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5647-f8c0-41e1-97c4-09e6fb765860 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:28.787059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5647-f8c0-41e1-97c4-09e6fb765860 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:28.787442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5647-f8c0-41e1-97c4-09e6fb765860 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:28.789557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4cad5647-f8c0-41e1-97c4-09e6fb765860 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:28.812245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad5647-f8c0-41e1-97c4-09e6fb765860 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:28.813946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad5647-f8c0-41e1-97c4-09e6fb765860 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:28.828100 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3994/7956] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:29.066474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:29.068599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:29.172679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf22ac-1019-4a25-8c1f-b5da1234643e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:29.182566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ec1b9-caee-46a2-82d5-49c41f948051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:29.183308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ec1b9-caee-46a2-82d5-49c41f948051 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:29.183673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ec1b9-caee-46a2-82d5-49c41f948051 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:29.184099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ec1b9-caee-46a2-82d5-49c41f948051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:29.186333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a26ec1b9-caee-46a2-82d5-49c41f948051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:29.206334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ec1b9-caee-46a2-82d5-49c41f948051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:29.208266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ec1b9-caee-46a2-82d5-49c41f948051 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:29.220604 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3963/7957] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:29.349615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad5647-f8c0-41e1-97c4-09e6fb765860 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:29.351154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4cad5647-f8c0-41e1-97c4-09e6fb765860 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:29.369791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad5647-f8c0-41e1-97c4-09e6fb765860 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:29.375270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd061d8-0def-4ee8-930d-2e2dd8ebce50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:29.375945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd061d8-0def-4ee8-930d-2e2dd8ebce50 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:29.376291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd061d8-0def-4ee8-930d-2e2dd8ebce50 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:29.376630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd061d8-0def-4ee8-930d-2e2dd8ebce50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:29.378533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2dd061d8-0def-4ee8-930d-2e2dd8ebce50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:29.394111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd061d8-0def-4ee8-930d-2e2dd8ebce50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:29.395797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd061d8-0def-4ee8-930d-2e2dd8ebce50 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:29.409812 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3995/7958] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:29.621802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ec1b9-caee-46a2-82d5-49c41f948051 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:29.642642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ec1b9-caee-46a2-82d5-49c41f948051 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:29.649333 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d6347-4144-49b9-adee-c94d9af8bbb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:29.650632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d6347-4144-49b9-adee-c94d9af8bbb8 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:29.651138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d6347-4144-49b9-adee-c94d9af8bbb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:29.652471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d6347-4144-49b9-adee-c94d9af8bbb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:29.654678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b32d6347-4144-49b9-adee-c94d9af8bbb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:29.745678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d6347-4144-49b9-adee-c94d9af8bbb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:29.747186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d6347-4144-49b9-adee-c94d9af8bbb8 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:29.764286 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3964/7959] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:39.402062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=79 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:39.756840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=62 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:57.409167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd061d8-0def-4ee8-930d-2e2dd8ebce50 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:57.410713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2dd061d8-0def-4ee8-930d-2e2dd8ebce50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:57.497505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd061d8-0def-4ee8-930d-2e2dd8ebce50 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:57.502004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:57.502622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:57.502890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:57.503152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] PATH_INFO: `/v3/users/55b36b56e72a48dda48976dd3ee45e02` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:57.504254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=55b36b56e72a48dda48976dd3ee45e02)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:57.506162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:57.553177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55b36b56e72a48dda48976dd3ee45e02'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:57.560024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55b36b56e72a48dda48976dd3ee45e02'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:57.572241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55b36b56e72a48dda48976dd3ee45e02'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:57.579660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55b36b56e72a48dda48976dd3ee45e02'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:57.581098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55b36b56e72a48dda48976dd3ee45e02'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:57.586858 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3996/7960] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:22:57 2022] DELETE /identity/v3/users/55b36b56e72a48dda48976dd3ee45e02 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:57.591538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e314fe-cf3c-4855-9ea8-55cb6001dc4f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:57.592026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e314fe-cf3c-4855-9ea8-55cb6001dc4f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:57.592134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e314fe-cf3c-4855-9ea8-55cb6001dc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:57.611783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-90e314fe-cf3c-4855-9ea8-55cb6001dc4f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:57.612960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-90e314fe-cf3c-4855-9ea8-55cb6001dc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:57.671292 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3965/7961] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:57.681329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:57.682712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:57.746225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:57.760542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4e39c-d62f-4461-b2e7-eb5afe273a8b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:57.764153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb474b71-c80e-40af-b32a-b8dcebca6625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:57.764745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb474b71-c80e-40af-b32a-b8dcebca6625 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:57.765014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb474b71-c80e-40af-b32a-b8dcebca6625 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:57.765303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb474b71-c80e-40af-b32a-b8dcebca6625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:57.774690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb474b71-c80e-40af-b32a-b8dcebca6625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:57.776094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb474b71-c80e-40af-b32a-b8dcebca6625 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:57.785497 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3997/7962] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:57.957731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e314fe-cf3c-4855-9ea8-55cb6001dc4f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:57.960215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-90e314fe-cf3c-4855-9ea8-55cb6001dc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:57.977624 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e314fe-cf3c-4855-9ea8-55cb6001dc4f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:57.982766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:57.983391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:57.983693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:57.983960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] PATH_INFO: `/v3/projects/a8d303e1d1b9482d99655f130cc231db` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:57.985180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d303e1d1b9482d99655f130cc231db)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:57.987916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:58.035120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d303e1d1b9482d99655f130cc231db', 'request_id': 'req-69712aa4-57a4-4b99-9755-39e9f54eff12'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:58.042654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d303e1d1b9482d99655f130cc231db', 'request_id': 'req-69712aa4-57a4-4b99-9755-39e9f54eff12'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:58.049533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d303e1d1b9482d99655f130cc231db', 'request_id': 'req-69712aa4-57a4-4b99-9755-39e9f54eff12'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:58.059779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] The token cache is being invalidate because project a8d303e1d1b9482d99655f130cc231db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:22:58.060056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:58.060679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-69712aa4-57a4-4b99-9755-39e9f54eff12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:58.062176 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3966/7963] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:22:57 2022] DELETE /identity/v3/projects/a8d303e1d1b9482d99655f130cc231db => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:58.067652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42cc69-5c54-43c9-af37-053cc90f9567 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:58.067942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42cc69-5c54-43c9-af37-053cc90f9567 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:58.068234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42cc69-5c54-43c9-af37-053cc90f9567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:58.087900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1e42cc69-5c54-43c9-af37-053cc90f9567 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:58.089121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1e42cc69-5c54-43c9-af37-053cc90f9567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:58.148726 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3998/7964] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:58.157680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a23b7f-2030-46fe-8b6b-b2546067a54a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:58.157961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a23b7f-2030-46fe-8b6b-b2546067a54a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:58.158731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a23b7f-2030-46fe-8b6b-b2546067a54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:58.178538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b4a23b7f-2030-46fe-8b6b-b2546067a54a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:58.179729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b4a23b7f-2030-46fe-8b6b-b2546067a54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:58.187613 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3967/7965] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:58.192386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42cc69-5c54-43c9-af37-053cc90f9567 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:58.202379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42cc69-5c54-43c9-af37-053cc90f9567 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:58.207622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4cf9f-489a-4d6b-8546-4d2daf34ba87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:58.208209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4cf9f-489a-4d6b-8546-4d2daf34ba87 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:58.208513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4cf9f-489a-4d6b-8546-4d2daf34ba87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:58.208781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4cf9f-489a-4d6b-8546-4d2daf34ba87 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:58.209447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4cf9f-489a-4d6b-8546-4d2daf34ba87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:22:58.209745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4cf9f-489a-4d6b-8546-4d2daf34ba87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:58.211351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4cf9f-489a-4d6b-8546-4d2daf34ba87 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:58.222989 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 3999/7966] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:22:58.227313 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a23b7f-2030-46fe-8b6b-b2546067a54a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:58.237211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a23b7f-2030-46fe-8b6b-b2546067a54a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:58.242066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337fcdb-7f17-4abe-8257-94f5c377820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:58.242661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337fcdb-7f17-4abe-8257-94f5c377820f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:58.242963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337fcdb-7f17-4abe-8257-94f5c377820f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:58.243254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337fcdb-7f17-4abe-8257-94f5c377820f admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:58.243882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337fcdb-7f17-4abe-8257-94f5c377820f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:58.245871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337fcdb-7f17-4abe-8257-94f5c377820f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:58.274336 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3968/7967] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:22:58 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:22:58.278727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4cf9f-489a-4d6b-8546-4d2daf34ba87 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:58.288896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4cf9f-489a-4d6b-8546-4d2daf34ba87 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:58.345471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ca698-4dce-463e-be76-02eac2231569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:58.346113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca698-4dce-463e-be76-02eac2231569 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:58.346451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca698-4dce-463e-be76-02eac2231569 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:58.346717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca698-4dce-463e-be76-02eac2231569 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:58.347337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ca698-4dce-463e-be76-02eac2231569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:58.349252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ca698-4dce-463e-be76-02eac2231569 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:58.371746 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4000/7968] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:22:58 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:22:58.375792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337fcdb-7f17-4abe-8257-94f5c377820f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:58.386308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337fcdb-7f17-4abe-8257-94f5c377820f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:58.390952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509070b-210a-46a6-abc3-0995639e7d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:58.391535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509070b-210a-46a6-abc3-0995639e7d49 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:58.391807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509070b-210a-46a6-abc3-0995639e7d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:58.392071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509070b-210a-46a6-abc3-0995639e7d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:58.393243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509070b-210a-46a6-abc3-0995639e7d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:58.394828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509070b-210a-46a6-abc3-0995639e7d49 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:58.404689 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3969/7969] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:22:58.408581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ca698-4dce-463e-be76-02eac2231569 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:58.418473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ca698-4dce-463e-be76-02eac2231569 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:58.423129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edc5c1-b4c3-4133-b6ef-16cc4edbccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:58.423710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edc5c1-b4c3-4133-b6ef-16cc4edbccf8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:58.424502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edc5c1-b4c3-4133-b6ef-16cc4edbccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:58.424763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edc5c1-b4c3-4133-b6ef-16cc4edbccf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:58.425424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edc5c1-b4c3-4133-b6ef-16cc4edbccf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:58.426997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edc5c1-b4c3-4133-b6ef-16cc4edbccf8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:58.437593 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4001/7970] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:22:58.442131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509070b-210a-46a6-abc3-0995639e7d49 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:58.452236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509070b-210a-46a6-abc3-0995639e7d49 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:58.456971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db985999-8eb4-4264-81f6-50f3b8448eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:58.457577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db985999-8eb4-4264-81f6-50f3b8448eca admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:58.457855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db985999-8eb4-4264-81f6-50f3b8448eca admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:58.458128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db985999-8eb4-4264-81f6-50f3b8448eca admin admin] PATH_INFO: `/v3/projects/9d43e6d89d0c420a8ef85636beaa6cfd/users/c752f560511e4b59a687691d3c910f2d/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:58.472863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db985999-8eb4-4264-81f6-50f3b8448eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d43e6d89d0c420a8ef85636beaa6cfd, user_id=c752f560511e4b59a687691d3c910f2d, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:58.475556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db985999-8eb4-4264-81f6-50f3b8448eca admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:58.489545 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3970/7971] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:22:58 2022] PUT /identity/v3/projects/9d43e6d89d0c420a8ef85636beaa6cfd/users/c752f560511e4b59a687691d3c910f2d/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:22:58.500082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac4bd8-6115-4783-bc19-d9f5709f7fff None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:58.500358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac4bd8-6115-4783-bc19-d9f5709f7fff None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:58.500645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac4bd8-6115-4783-bc19-d9f5709f7fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:58.528462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-07ac4bd8-6115-4783-bc19-d9f5709f7fff None None] MFA Rules not processed for user `c752f560511e4b59a687691d3c910f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:58.529688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-07ac4bd8-6115-4783-bc19-d9f5709f7fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:58.573170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:58.596768 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4002/7972] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:58.607810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db985999-8eb4-4264-81f6-50f3b8448eca admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:58.609513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-db985999-8eb4-4264-81f6-50f3b8448eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:58.631942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:58.672006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db985999-8eb4-4264-81f6-50f3b8448eca admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:58.676293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bffba-4247-4d3e-97d0-7da1ecda9794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:58.676877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bffba-4247-4d3e-97d0-7da1ecda9794 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:58.677190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bffba-4247-4d3e-97d0-7da1ecda9794 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:58.677462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bffba-4247-4d3e-97d0-7da1ecda9794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:58.685713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bffba-4247-4d3e-97d0-7da1ecda9794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:58.687131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bffba-4247-4d3e-97d0-7da1ecda9794 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:58.695959 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3971/7973] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:59.016456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac4bd8-6115-4783-bc19-d9f5709f7fff None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:59.079291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac4bd8-6115-4783-bc19-d9f5709f7fff None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:59.084817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:59.085473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:59.085746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:59.086012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] PATH_INFO: `/v3/users/c752f560511e4b59a687691d3c910f2d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:59.087107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c752f560511e4b59a687691d3c910f2d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:59.088995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:59.135538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c752f560511e4b59a687691d3c910f2d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:59.143126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c752f560511e4b59a687691d3c910f2d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:59.155657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c752f560511e4b59a687691d3c910f2d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:59.163661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c752f560511e4b59a687691d3c910f2d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:59.164365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c752f560511e4b59a687691d3c910f2d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:59.170610 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4003/7974] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:22:59 2022] DELETE /identity/v3/users/c752f560511e4b59a687691d3c910f2d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:59.175109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02f4f9-08aa-4004-b4ba-811da4c1dfce None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:59.175392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02f4f9-08aa-4004-b4ba-811da4c1dfce None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:59.175664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02f4f9-08aa-4004-b4ba-811da4c1dfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:59.195154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bb02f4f9-08aa-4004-b4ba-811da4c1dfce None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:59.196376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bb02f4f9-08aa-4004-b4ba-811da4c1dfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:59.256437 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3972/7975] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:59.266250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:59.267843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:59.329851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd7f56-f81f-4f13-8ba2-2688aae7b436 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:59.333668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b433e3-cbe3-4804-a2a7-9bc3ff629755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:59.334261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b433e3-cbe3-4804-a2a7-9bc3ff629755 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:59.334530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b433e3-cbe3-4804-a2a7-9bc3ff629755 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:59.334794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b433e3-cbe3-4804-a2a7-9bc3ff629755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:59.344150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b433e3-cbe3-4804-a2a7-9bc3ff629755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:59.345603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b433e3-cbe3-4804-a2a7-9bc3ff629755 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:59.356068 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4004/7976] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:22:59.624397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02f4f9-08aa-4004-b4ba-811da4c1dfce None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:59.625968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bb02f4f9-08aa-4004-b4ba-811da4c1dfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:59.641450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02f4f9-08aa-4004-b4ba-811da4c1dfce None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:59.647658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:59.648330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:59.648542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:59.648812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] PATH_INFO: `/v3/projects/9d43e6d89d0c420a8ef85636beaa6cfd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:59.650165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d43e6d89d0c420a8ef85636beaa6cfd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:59.652177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:59.702017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d43e6d89d0c420a8ef85636beaa6cfd', 'request_id': 'req-ec302cbf-00b5-4514-8967-c03372b35939'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:59.709482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d43e6d89d0c420a8ef85636beaa6cfd', 'request_id': 'req-ec302cbf-00b5-4514-8967-c03372b35939'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:59.715836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d43e6d89d0c420a8ef85636beaa6cfd', 'request_id': 'req-ec302cbf-00b5-4514-8967-c03372b35939'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:59.727627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] The token cache is being invalidate because project 9d43e6d89d0c420a8ef85636beaa6cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:22:59.727912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:59.728551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ec302cbf-00b5-4514-8967-c03372b35939 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:22:59.730358 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3973/7977] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:22:59 2022] DELETE /identity/v3/projects/9d43e6d89d0c420a8ef85636beaa6cfd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:22:59.735135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f1b80-7db6-4c2c-9442-cc7e04bca0c6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:59.735645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f1b80-7db6-4c2c-9442-cc7e04bca0c6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:59.735645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f1b80-7db6-4c2c-9442-cc7e04bca0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:59.745213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:59.758184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-906f1b80-7db6-4c2c-9442-cc7e04bca0c6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:59.759561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-906f1b80-7db6-4c2c-9442-cc7e04bca0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:59.820886 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4005/7978] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:59.828536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea459950-baa1-4eea-be5b-50154d7e96e1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:59.828815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea459950-baa1-4eea-be5b-50154d7e96e1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:59.829111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea459950-baa1-4eea-be5b-50154d7e96e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:59.849762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ea459950-baa1-4eea-be5b-50154d7e96e1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:22:59.850954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ea459950-baa1-4eea-be5b-50154d7e96e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:22:59.859008 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3974/7979] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:22:59.863739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f1b80-7db6-4c2c-9442-cc7e04bca0c6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:59.873588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f1b80-7db6-4c2c-9442-cc7e04bca0c6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:59.878934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b849c-479b-4e4d-8a32-9b1cbffe9be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:59.879520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b849c-479b-4e4d-8a32-9b1cbffe9be4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:59.879792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b849c-479b-4e4d-8a32-9b1cbffe9be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:59.880058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b849c-479b-4e4d-8a32-9b1cbffe9be4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:59.880690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b849c-479b-4e4d-8a32-9b1cbffe9be4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:22:59.881648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b849c-479b-4e4d-8a32-9b1cbffe9be4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:59.883257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b849c-479b-4e4d-8a32-9b1cbffe9be4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:59.894793 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4006/7980] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:22:59.898933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea459950-baa1-4eea-be5b-50154d7e96e1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:59.908293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea459950-baa1-4eea-be5b-50154d7e96e1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:22:59.913274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d476498-9ed9-4180-a09f-a36edda65f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:22:59.913876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d476498-9ed9-4180-a09f-a36edda65f5f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:22:59.914143 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d476498-9ed9-4180-a09f-a36edda65f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:22:59.914407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d476498-9ed9-4180-a09f-a36edda65f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:22:59.915033 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d476498-9ed9-4180-a09f-a36edda65f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:22:59.916923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d476498-9ed9-4180-a09f-a36edda65f5f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:22:59.945514 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3975/7981] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:22:59 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:22:59.949731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b849c-479b-4e4d-8a32-9b1cbffe9be4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:22:59.959726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b849c-479b-4e4d-8a32-9b1cbffe9be4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:00.017837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c405cc-c4d9-4756-997e-310079154797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:00.018547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c405cc-c4d9-4756-997e-310079154797 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:00.018820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c405cc-c4d9-4756-997e-310079154797 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:00.019082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c405cc-c4d9-4756-997e-310079154797 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:00.019698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c405cc-c4d9-4756-997e-310079154797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:00.021621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c405cc-c4d9-4756-997e-310079154797 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:00.042509 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4007/7982] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:22:59 2022] POST /identity/v3/users => generated 279 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:23:00.046679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d476498-9ed9-4180-a09f-a36edda65f5f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:00.057553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d476498-9ed9-4180-a09f-a36edda65f5f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:00.062121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bfcea-fd9f-4e6a-9fb4-03a96334051d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:00.062702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bfcea-fd9f-4e6a-9fb4-03a96334051d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:00.062974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bfcea-fd9f-4e6a-9fb4-03a96334051d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:00.063238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bfcea-fd9f-4e6a-9fb4-03a96334051d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:00.063864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bfcea-fd9f-4e6a-9fb4-03a96334051d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:00.066109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bfcea-fd9f-4e6a-9fb4-03a96334051d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:00.076341 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3976/7983] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:00.080235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c405cc-c4d9-4756-997e-310079154797 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:00.090227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c405cc-c4d9-4756-997e-310079154797 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:00.094814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af885c2b-1690-40e5-a14e-7754a7e6269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:00.095393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af885c2b-1690-40e5-a14e-7754a7e6269f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:00.095751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af885c2b-1690-40e5-a14e-7754a7e6269f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:00.096019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af885c2b-1690-40e5-a14e-7754a7e6269f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:00.096634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af885c2b-1690-40e5-a14e-7754a7e6269f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:00.098227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af885c2b-1690-40e5-a14e-7754a7e6269f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:00.109080 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4008/7984] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:00.113482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bfcea-fd9f-4e6a-9fb4-03a96334051d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:00.123533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bfcea-fd9f-4e6a-9fb4-03a96334051d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:00.128812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32783427-255e-4a24-96a4-ef9075fcd6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:00.129480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32783427-255e-4a24-96a4-ef9075fcd6eb admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:00.129776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32783427-255e-4a24-96a4-ef9075fcd6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:00.130035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32783427-255e-4a24-96a4-ef9075fcd6eb admin admin] PATH_INFO: `/v3/projects/fcdae76d98814bd0a8573d61596dedc0/users/1336f6ced44845b1b41f10207ec6b21a/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:00.132609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:00.148220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32783427-255e-4a24-96a4-ef9075fcd6eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcdae76d98814bd0a8573d61596dedc0, user_id=1336f6ced44845b1b41f10207ec6b21a, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:00.150936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32783427-255e-4a24-96a4-ef9075fcd6eb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:00.164375 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3977/7985] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:00 2022] PUT /identity/v3/projects/fcdae76d98814bd0a8573d61596dedc0/users/1336f6ced44845b1b41f10207ec6b21a/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:00.175482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf7b19-7e22-4eed-a830-ea594bb47371 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:00.175762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf7b19-7e22-4eed-a830-ea594bb47371 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:00.176027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf7b19-7e22-4eed-a830-ea594bb47371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:00.204724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cdbf7b19-7e22-4eed-a830-ea594bb47371 None None] MFA Rules not processed for user `1336f6ced44845b1b41f10207ec6b21a`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:00.206854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cdbf7b19-7e22-4eed-a830-ea594bb47371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:00.267159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:00.272510 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4009/7986] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:00.282226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32783427-255e-4a24-96a4-ef9075fcd6eb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:00.283928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-32783427-255e-4a24-96a4-ef9075fcd6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:00.343107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32783427-255e-4a24-96a4-ef9075fcd6eb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:00.346734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5395765-a66b-48b8-8552-89d33e1dbb2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:00.347320 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395765-a66b-48b8-8552-89d33e1dbb2d service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:00.347604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395765-a66b-48b8-8552-89d33e1dbb2d service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:00.347868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395765-a66b-48b8-8552-89d33e1dbb2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:00.358201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5395765-a66b-48b8-8552-89d33e1dbb2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:00.359958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5395765-a66b-48b8-8552-89d33e1dbb2d service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:00.368928 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3978/7987] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:07.141292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf7b19-7e22-4eed-a830-ea594bb47371 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:07.213274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf7b19-7e22-4eed-a830-ea594bb47371 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:07.217866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:07.218444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:07.218721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:07.218995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] PATH_INFO: `/v3/users/1336f6ced44845b1b41f10207ec6b21a` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:07.220095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1336f6ced44845b1b41f10207ec6b21a)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:07.222540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:07.269088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1336f6ced44845b1b41f10207ec6b21a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:07.277181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1336f6ced44845b1b41f10207ec6b21a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:07.290044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1336f6ced44845b1b41f10207ec6b21a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:07.297533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1336f6ced44845b1b41f10207ec6b21a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:07.298607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1336f6ced44845b1b41f10207ec6b21a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:07.305088 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4010/7988] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:23:07 2022] DELETE /identity/v3/users/1336f6ced44845b1b41f10207ec6b21a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:07.309650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab7a78-cead-468f-9d61-0cf9cfd69a84 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:07.310146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab7a78-cead-468f-9d61-0cf9cfd69a84 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:07.310266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab7a78-cead-468f-9d61-0cf9cfd69a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:07.328956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-96ab7a78-cead-468f-9d61-0cf9cfd69a84 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:07.330645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-96ab7a78-cead-468f-9d61-0cf9cfd69a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:07.389462 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3979/7989] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:07.401488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:07.402965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:07.464853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c72ff-bb6b-4fee-bca0-63816bb46599 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:07.468564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41642cf5-a5b8-46b3-a923-2f08aad47fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:07.469208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642cf5-a5b8-46b3-a923-2f08aad47fea service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:07.469483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642cf5-a5b8-46b3-a923-2f08aad47fea service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:07.469747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642cf5-a5b8-46b3-a923-2f08aad47fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:07.479929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41642cf5-a5b8-46b3-a923-2f08aad47fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:07.482031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41642cf5-a5b8-46b3-a923-2f08aad47fea service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:07.493125 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4011/7990] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:07.669817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab7a78-cead-468f-9d61-0cf9cfd69a84 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:07.671391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-96ab7a78-cead-468f-9d61-0cf9cfd69a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:07.686290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab7a78-cead-468f-9d61-0cf9cfd69a84 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:07.691892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:07.692531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:07.693159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:07.693159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] PATH_INFO: `/v3/projects/fcdae76d98814bd0a8573d61596dedc0` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:07.694425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcdae76d98814bd0a8573d61596dedc0)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:07.696541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:07.746296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcdae76d98814bd0a8573d61596dedc0', 'request_id': 'req-c07d94d8-a8e8-4c06-9aa7-e03279e92014'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:07.755200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcdae76d98814bd0a8573d61596dedc0', 'request_id': 'req-c07d94d8-a8e8-4c06-9aa7-e03279e92014'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:07.761736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcdae76d98814bd0a8573d61596dedc0', 'request_id': 'req-c07d94d8-a8e8-4c06-9aa7-e03279e92014'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:07.773374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] The token cache is being invalidate because project fcdae76d98814bd0a8573d61596dedc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:23:07.773660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:07.774318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c07d94d8-a8e8-4c06-9aa7-e03279e92014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:07.775831 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3980/7991] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:23:07 2022] DELETE /identity/v3/projects/fcdae76d98814bd0a8573d61596dedc0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:07.781148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cc61d-8573-4e8e-9cc0-98ba565ba9b4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:07.781433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cc61d-8573-4e8e-9cc0-98ba565ba9b4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:07.781704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cc61d-8573-4e8e-9cc0-98ba565ba9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:07.801966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9d9cc61d-8573-4e8e-9cc0-98ba565ba9b4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:07.803103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9d9cc61d-8573-4e8e-9cc0-98ba565ba9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:07.821083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:07.866557 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4012/7992] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:07.874530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f209b-fdac-4b8a-8c95-8b26f6bf3bbd None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:07.874797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f209b-fdac-4b8a-8c95-8b26f6bf3bbd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:07.875067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f209b-fdac-4b8a-8c95-8b26f6bf3bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:07.895698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-085f209b-fdac-4b8a-8c95-8b26f6bf3bbd None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:07.896877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-085f209b-fdac-4b8a-8c95-8b26f6bf3bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:07.904695 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3981/7993] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:07.909366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cc61d-8573-4e8e-9cc0-98ba565ba9b4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:07.919096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cc61d-8573-4e8e-9cc0-98ba565ba9b4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:07.923850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910375c-76e9-4234-ab11-b0ca13175ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:07.924437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910375c-76e9-4234-ab11-b0ca13175ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:07.924713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910375c-76e9-4234-ab11-b0ca13175ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:07.924980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910375c-76e9-4234-ab11-b0ca13175ae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:07.925649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910375c-76e9-4234-ab11-b0ca13175ae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:23:07.925949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910375c-76e9-4234-ab11-b0ca13175ae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:07.927524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910375c-76e9-4234-ab11-b0ca13175ae8 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:07.939433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4013/7994] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:23:07.944289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f209b-fdac-4b8a-8c95-8b26f6bf3bbd None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:07.948638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:07.955825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f209b-fdac-4b8a-8c95-8b26f6bf3bbd None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:07.961600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4479b9b-3ab1-4da0-9883-4f49ed457ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:07.962183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4479b9b-3ab1-4da0-9883-4f49ed457ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:07.962453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4479b9b-3ab1-4da0-9883-4f49ed457ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:07.962717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4479b9b-3ab1-4da0-9883-4f49ed457ed6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:07.963328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4479b9b-3ab1-4da0-9883-4f49ed457ed6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:07.965240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4479b9b-3ab1-4da0-9883-4f49ed457ed6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:07.992000 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3982/7995] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:23:07 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:07.996188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910375c-76e9-4234-ab11-b0ca13175ae8 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.007510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910375c-76e9-4234-ab11-b0ca13175ae8 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.064946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0a04e-8d74-4294-b58e-6183b60562f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.065506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0a04e-8d74-4294-b58e-6183b60562f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.065776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0a04e-8d74-4294-b58e-6183b60562f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.066039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0a04e-8d74-4294-b58e-6183b60562f7 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.066659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0a04e-8d74-4294-b58e-6183b60562f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.068539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0a04e-8d74-4294-b58e-6183b60562f7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.088743 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4014/7996] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:23:07 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:08.093015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4479b9b-3ab1-4da0-9883-4f49ed457ed6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.103566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4479b9b-3ab1-4da0-9883-4f49ed457ed6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.108453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c86a86-3c4f-4aaa-874f-341193475c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.109059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c86a86-3c4f-4aaa-874f-341193475c49 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.109348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c86a86-3c4f-4aaa-874f-341193475c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.109615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c86a86-3c4f-4aaa-874f-341193475c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.110244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c86a86-3c4f-4aaa-874f-341193475c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.112380 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c86a86-3c4f-4aaa-874f-341193475c49 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.122752 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3983/7997] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:08.127779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0a04e-8d74-4294-b58e-6183b60562f7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.138051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0a04e-8d74-4294-b58e-6183b60562f7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.142882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d811dc-ce93-4f83-8496-78aa3f5a4bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.143599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d811dc-ce93-4f83-8496-78aa3f5a4bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.143886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d811dc-ce93-4f83-8496-78aa3f5a4bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.144155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d811dc-ce93-4f83-8496-78aa3f5a4bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.144779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d811dc-ce93-4f83-8496-78aa3f5a4bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.146397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d811dc-ce93-4f83-8496-78aa3f5a4bf5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.157000 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4015/7998] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:23:08.161241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c86a86-3c4f-4aaa-874f-341193475c49 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.171315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c86a86-3c4f-4aaa-874f-341193475c49 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.176540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89f7e2-5f6c-4622-9e45-6e1ffdfcdfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.177218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89f7e2-5f6c-4622-9e45-6e1ffdfcdfd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.177497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89f7e2-5f6c-4622-9e45-6e1ffdfcdfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.177768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89f7e2-5f6c-4622-9e45-6e1ffdfcdfd2 admin admin] PATH_INFO: `/v3/projects/7b722dda022e4459b471341ab11f550f/users/b40b97939477498ca194cad7664432bf/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.194421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89f7e2-5f6c-4622-9e45-6e1ffdfcdfd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b722dda022e4459b471341ab11f550f, user_id=b40b97939477498ca194cad7664432bf, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.197141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89f7e2-5f6c-4622-9e45-6e1ffdfcdfd2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.210925 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3984/7999] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:08 2022] PUT /identity/v3/projects/7b722dda022e4459b471341ab11f550f/users/b40b97939477498ca194cad7664432bf/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:08.222812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b620d08-e479-463f-98a1-e699456ff8b7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.223120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b620d08-e479-463f-98a1-e699456ff8b7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.223415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b620d08-e479-463f-98a1-e699456ff8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.251555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-3b620d08-e479-463f-98a1-e699456ff8b7 None None] MFA Rules not processed for user `b40b97939477498ca194cad7664432bf`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:08.252846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3b620d08-e479-463f-98a1-e699456ff8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:08.319029 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4016/8000] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:23:08.323054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89f7e2-5f6c-4622-9e45-6e1ffdfcdfd2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.388681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89f7e2-5f6c-4622-9e45-6e1ffdfcdfd2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.393476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100fc0e-339b-476b-9c82-f7f764f2d135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.394142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100fc0e-339b-476b-9c82-f7f764f2d135 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.394336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100fc0e-339b-476b-9c82-f7f764f2d135 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.394612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100fc0e-339b-476b-9c82-f7f764f2d135 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.395798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100fc0e-339b-476b-9c82-f7f764f2d135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.397749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100fc0e-339b-476b-9c82-f7f764f2d135 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.424767 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3985/8001] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:23:08 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:08.428958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b620d08-e479-463f-98a1-e699456ff8b7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.439740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b620d08-e479-463f-98a1-e699456ff8b7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.452649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:08.498679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdebe0-b34d-472c-bbf5-3d55ec3ce153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.499275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdebe0-b34d-472c-bbf5-3d55ec3ce153 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.499544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdebe0-b34d-472c-bbf5-3d55ec3ce153 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.499804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdebe0-b34d-472c-bbf5-3d55ec3ce153 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.500420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdebe0-b34d-472c-bbf5-3d55ec3ce153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.502375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdebe0-b34d-472c-bbf5-3d55ec3ce153 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.525217 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4017/8002] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:23:08 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:08.529204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100fc0e-339b-476b-9c82-f7f764f2d135 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.539649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100fc0e-339b-476b-9c82-f7f764f2d135 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.544277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ec6ca-a141-4c54-bbcd-079af9a41c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.545177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ec6ca-a141-4c54-bbcd-079af9a41c8f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.545474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ec6ca-a141-4c54-bbcd-079af9a41c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.545722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ec6ca-a141-4c54-bbcd-079af9a41c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.546349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ec6ca-a141-4c54-bbcd-079af9a41c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.547955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ec6ca-a141-4c54-bbcd-079af9a41c8f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.558655 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3986/8003] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:08.562761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdebe0-b34d-472c-bbf5-3d55ec3ce153 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.575595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdebe0-b34d-472c-bbf5-3d55ec3ce153 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.580314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0fd2f-6690-4d7e-824a-0965217f3442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.580958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0fd2f-6690-4d7e-824a-0965217f3442 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.581272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0fd2f-6690-4d7e-824a-0965217f3442 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.581540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0fd2f-6690-4d7e-824a-0965217f3442 admin admin] PATH_INFO: `/v3/projects/b67789ec74b84849a9910d8b3bba8fed/users/10ab81d1c8cc4cc28f3359d2900f9a8f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.597188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0fd2f-6690-4d7e-824a-0965217f3442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67789ec74b84849a9910d8b3bba8fed, user_id=10ab81d1c8cc4cc28f3359d2900f9a8f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.600210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0fd2f-6690-4d7e-824a-0965217f3442 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.612809 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4018/8004] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:08 2022] PUT /identity/v3/projects/b67789ec74b84849a9910d8b3bba8fed/users/10ab81d1c8cc4cc28f3359d2900f9a8f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:08.617181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ec6ca-a141-4c54-bbcd-079af9a41c8f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.627066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ec6ca-a141-4c54-bbcd-079af9a41c8f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.683565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e86539-adca-4fea-a1b0-c159decde55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.684147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e86539-adca-4fea-a1b0-c159decde55e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.684416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e86539-adca-4fea-a1b0-c159decde55e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.684677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e86539-adca-4fea-a1b0-c159decde55e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.685324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e86539-adca-4fea-a1b0-c159decde55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.686917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e86539-adca-4fea-a1b0-c159decde55e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.696918 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3987/8005] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:08.701116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0fd2f-6690-4d7e-824a-0965217f3442 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.711888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0fd2f-6690-4d7e-824a-0965217f3442 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.716532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdbeb4-80b6-4484-833d-99f5ab1104be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.717208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdbeb4-80b6-4484-833d-99f5ab1104be admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.717484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdbeb4-80b6-4484-833d-99f5ab1104be admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.717752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdbeb4-80b6-4484-833d-99f5ab1104be admin admin] PATH_INFO: `/v3/projects/b67789ec74b84849a9910d8b3bba8fed/users/10ab81d1c8cc4cc28f3359d2900f9a8f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.719639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdbeb4-80b6-4484-833d-99f5ab1104be admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67789ec74b84849a9910d8b3bba8fed, user_id=10ab81d1c8cc4cc28f3359d2900f9a8f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.722308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdbeb4-80b6-4484-833d-99f5ab1104be admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.732989 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4019/8006] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:08 2022] PUT /identity/v3/projects/b67789ec74b84849a9910d8b3bba8fed/users/10ab81d1c8cc4cc28f3359d2900f9a8f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:08.743927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4387-85e8-42a0-924b-d623b560b97d None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.744205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4387-85e8-42a0-924b-d623b560b97d None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.744477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4387-85e8-42a0-924b-d623b560b97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.772592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-07ba4387-85e8-42a0-924b-d623b560b97d None None] MFA Rules not processed for user `10ab81d1c8cc4cc28f3359d2900f9a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:08.773847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-07ba4387-85e8-42a0-924b-d623b560b97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:08.777845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:08.843148 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3988/8007] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:08.855032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdbeb4-80b6-4484-833d-99f5ab1104be admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:08.856800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-97fdbeb4-80b6-4484-833d-99f5ab1104be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:08.920093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdbeb4-80b6-4484-833d-99f5ab1104be admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:08.923973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45de69-e7e5-410e-a462-572ce3d08cec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:08.924571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45de69-e7e5-410e-a462-572ce3d08cec service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:08.924843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45de69-e7e5-410e-a462-572ce3d08cec service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:08.925137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45de69-e7e5-410e-a462-572ce3d08cec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:08.934293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45de69-e7e5-410e-a462-572ce3d08cec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:08.935699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45de69-e7e5-410e-a462-572ce3d08cec service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:08.945454 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4020/8008] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:09.021028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba4387-85e8-42a0-924b-d623b560b97d None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:09.033204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba4387-85e8-42a0-924b-d623b560b97d None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:09.037935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce528a2-51e3-470b-a98d-6660bd62c0fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:09.038520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce528a2-51e3-470b-a98d-6660bd62c0fc service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:09.038794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce528a2-51e3-470b-a98d-6660bd62c0fc service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:09.039058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce528a2-51e3-470b-a98d-6660bd62c0fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:09.096149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce528a2-51e3-470b-a98d-6660bd62c0fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:09.097558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce528a2-51e3-470b-a98d-6660bd62c0fc service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:09.106691 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3989/8009] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:18.940800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:19.102532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=55 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:40.348672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45de69-e7e5-410e-a462-572ce3d08cec service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:40.412401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45de69-e7e5-410e-a462-572ce3d08cec service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:40.417603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:40.418221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:40.418500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:40.418770 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] PATH_INFO: `/v3/users/b40b97939477498ca194cad7664432bf` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:40.419883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b40b97939477498ca194cad7664432bf)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:40.421815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:40.467701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b40b97939477498ca194cad7664432bf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:40.474749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b40b97939477498ca194cad7664432bf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:40.486896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b40b97939477498ca194cad7664432bf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:40.494382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b40b97939477498ca194cad7664432bf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:40.494953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b40b97939477498ca194cad7664432bf'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:40.500834 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4021/8010] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:23:40 2022] DELETE /identity/v3/users/b40b97939477498ca194cad7664432bf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:40.505321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca018c1-f500-4c01-9af5-dbe7ef8602ca None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:40.505834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca018c1-f500-4c01-9af5-dbe7ef8602ca None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:40.505940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca018c1-f500-4c01-9af5-dbe7ef8602ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:40.533774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2ca018c1-f500-4c01-9af5-dbe7ef8602ca None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:40.534979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2ca018c1-f500-4c01-9af5-dbe7ef8602ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:40.598602 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3990/8011] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:23:40.609171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:40.610563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:40.682598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818dc30-7668-47f9-91ee-2f6555fcfdd4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:40.686235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53765bef-cef3-4791-8520-07c753380771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:40.686827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53765bef-cef3-4791-8520-07c753380771 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:40.687129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53765bef-cef3-4791-8520-07c753380771 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:40.687395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53765bef-cef3-4791-8520-07c753380771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:40.696739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53765bef-cef3-4791-8520-07c753380771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:40.698187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53765bef-cef3-4791-8520-07c753380771 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:40.707433 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4022/8012] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:40.983820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca018c1-f500-4c01-9af5-dbe7ef8602ca None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:40.985360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2ca018c1-f500-4c01-9af5-dbe7ef8602ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:40.998573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca018c1-f500-4c01-9af5-dbe7ef8602ca None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:41.003330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:41.003959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:41.004236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:41.004508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] PATH_INFO: `/v3/projects/7b722dda022e4459b471341ab11f550f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:41.006329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b722dda022e4459b471341ab11f550f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:41.009212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:41.057891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b722dda022e4459b471341ab11f550f', 'request_id': 'req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.065049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b722dda022e4459b471341ab11f550f', 'request_id': 'req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.071444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b722dda022e4459b471341ab11f550f', 'request_id': 'req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.082158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] The token cache is being invalidate because project 7b722dda022e4459b471341ab11f550f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:23:41.082443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.083076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.084554 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3991/8013] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:23:40 2022] DELETE /identity/v3/projects/7b722dda022e4459b471341ab11f550f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:23:41.089768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53765bef-cef3-4791-8520-07c753380771 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:41.100955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53765bef-cef3-4791-8520-07c753380771 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:41.111195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:41.162686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:41.163732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:41.164016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:41.164286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] PATH_INFO: `/v3/users/10ab81d1c8cc4cc28f3359d2900f9a8f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:41.165424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10ab81d1c8cc4cc28f3359d2900f9a8f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:41.167320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:41.220982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10ab81d1c8cc4cc28f3359d2900f9a8f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.228675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10ab81d1c8cc4cc28f3359d2900f9a8f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.240750 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10ab81d1c8cc4cc28f3359d2900f9a8f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.248691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10ab81d1c8cc4cc28f3359d2900f9a8f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.249324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4eea1933-db44-42dd-8cd5-d8dc30fba1f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10ab81d1c8cc4cc28f3359d2900f9a8f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.255290 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4023/8014] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:23:41 2022] DELETE /identity/v3/users/10ab81d1c8cc4cc28f3359d2900f9a8f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:41.525982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:41.527451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:41.546653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac39f67-c8db-4bc7-a8b8-a1586cca0985 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:41.607768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:41.608321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:41.608592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:41.608859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] PATH_INFO: `/v3/projects/b67789ec74b84849a9910d8b3bba8fed` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:41.610017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b67789ec74b84849a9910d8b3bba8fed)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:41.611992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:41.632688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:41.659420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67789ec74b84849a9910d8b3bba8fed', 'request_id': 'req-4718c4b3-22f3-4f29-a796-1fb4413e0641'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.667227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67789ec74b84849a9910d8b3bba8fed', 'request_id': 'req-4718c4b3-22f3-4f29-a796-1fb4413e0641'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.673487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67789ec74b84849a9910d8b3bba8fed', 'request_id': 'req-4718c4b3-22f3-4f29-a796-1fb4413e0641'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.683691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] The token cache is being invalidate because project b67789ec74b84849a9910d8b3bba8fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:23:41.683970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.684592 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4718c4b3-22f3-4f29-a796-1fb4413e0641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:41.686104 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3992/8015] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:23:41 2022] DELETE /identity/v3/projects/b67789ec74b84849a9910d8b3bba8fed => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:41.690989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fc957-5353-4790-8e2a-a81a7fac4c2a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:41.691494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fc957-5353-4790-8e2a-a81a7fac4c2a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:41.691629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fc957-5353-4790-8e2a-a81a7fac4c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:41.721559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b18fc957-5353-4790-8e2a-a81a7fac4c2a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:41.722759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b18fc957-5353-4790-8e2a-a81a7fac4c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:41.781625 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4024/8016] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:41.791500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4067c1-11ab-4480-adf3-dfded8136b15 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:41.791823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4067c1-11ab-4480-adf3-dfded8136b15 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:41.792100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4067c1-11ab-4480-adf3-dfded8136b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:41.812089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fc4067c1-11ab-4480-adf3-dfded8136b15 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:41.813337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fc4067c1-11ab-4480-adf3-dfded8136b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:41.821229 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3993/8017] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:41.825385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fc957-5353-4790-8e2a-a81a7fac4c2a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:41.834918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fc957-5353-4790-8e2a-a81a7fac4c2a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:41.840089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ab244-7def-4c0c-b720-3d415abe415a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:41.840666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ab244-7def-4c0c-b720-3d415abe415a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:41.840935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ab244-7def-4c0c-b720-3d415abe415a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:41.841226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ab244-7def-4c0c-b720-3d415abe415a admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:41.841853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ab244-7def-4c0c-b720-3d415abe415a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:23:41.842144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ab244-7def-4c0c-b720-3d415abe415a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:41.843696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ab244-7def-4c0c-b720-3d415abe415a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:41.855896 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4025/8018] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:23:41.860070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4067c1-11ab-4480-adf3-dfded8136b15 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:41.869619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4067c1-11ab-4480-adf3-dfded8136b15 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:41.874239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019efa57-9080-4f6f-9648-0b7afc9df38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:41.874828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019efa57-9080-4f6f-9648-0b7afc9df38b admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:41.875101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019efa57-9080-4f6f-9648-0b7afc9df38b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:41.875367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019efa57-9080-4f6f-9648-0b7afc9df38b admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:41.875991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019efa57-9080-4f6f-9648-0b7afc9df38b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:41.877913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019efa57-9080-4f6f-9648-0b7afc9df38b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:41.905373 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3994/8019] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:23:41 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:41.909397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ab244-7def-4c0c-b720-3d415abe415a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:41.920016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ab244-7def-4c0c-b720-3d415abe415a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:41.924698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:41.977056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c0c05-c066-4535-8c75-b47ecf3da504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:41.977673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c0c05-c066-4535-8c75-b47ecf3da504 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:41.977949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c0c05-c066-4535-8c75-b47ecf3da504 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:41.978217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c0c05-c066-4535-8c75-b47ecf3da504 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:41.979086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c0c05-c066-4535-8c75-b47ecf3da504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:41.981018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c0c05-c066-4535-8c75-b47ecf3da504 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.001631 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4026/8020] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:23:41 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:23:42.005880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019efa57-9080-4f6f-9648-0b7afc9df38b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:42.016865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019efa57-9080-4f6f-9648-0b7afc9df38b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:42.021447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539e7fa-ff40-45fc-8543-4cd091a8a3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:42.022025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539e7fa-ff40-45fc-8543-4cd091a8a3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.022298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539e7fa-ff40-45fc-8543-4cd091a8a3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.023309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539e7fa-ff40-45fc-8543-4cd091a8a3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.023943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539e7fa-ff40-45fc-8543-4cd091a8a3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:42.025560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539e7fa-ff40-45fc-8543-4cd091a8a3f1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.035275 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3995/8021] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:42.039262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c0c05-c066-4535-8c75-b47ecf3da504 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:42.049175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c0c05-c066-4535-8c75-b47ecf3da504 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:42.053787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda45b0-a494-44a9-aa31-8822e12f2254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:42.054374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda45b0-a494-44a9-aa31-8822e12f2254 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.054643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda45b0-a494-44a9-aa31-8822e12f2254 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.054909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda45b0-a494-44a9-aa31-8822e12f2254 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.055530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda45b0-a494-44a9-aa31-8822e12f2254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:42.057124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda45b0-a494-44a9-aa31-8822e12f2254 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.067571 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4027/8022] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:42.072162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539e7fa-ff40-45fc-8543-4cd091a8a3f1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:42.082137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539e7fa-ff40-45fc-8543-4cd091a8a3f1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:42.086765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49f328-8260-4fba-be79-6c62463414ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:42.087388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49f328-8260-4fba-be79-6c62463414ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.087662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49f328-8260-4fba-be79-6c62463414ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.087927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49f328-8260-4fba-be79-6c62463414ff admin admin] PATH_INFO: `/v3/projects/362818a79ae14a8a852c56a6490e9faa/users/8dd016ce35fb4d66bf845cd22774f280/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.102905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49f328-8260-4fba-be79-6c62463414ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=362818a79ae14a8a852c56a6490e9faa, user_id=8dd016ce35fb4d66bf845cd22774f280, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:42.106071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49f328-8260-4fba-be79-6c62463414ff admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.119345 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3996/8023] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:42 2022] PUT /identity/v3/projects/362818a79ae14a8a852c56a6490e9faa/users/8dd016ce35fb4d66bf845cd22774f280/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:23:42.129478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42eacf-6c05-4e6f-b232-0d2120501a9f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.129762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42eacf-6c05-4e6f-b232-0d2120501a9f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.130029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42eacf-6c05-4e6f-b232-0d2120501a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.158375 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7c42eacf-6c05-4e6f-b232-0d2120501a9f None None] MFA Rules not processed for user `8dd016ce35fb4d66bf845cd22774f280`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:42.159566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7c42eacf-6c05-4e6f-b232-0d2120501a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:42.223645 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4028/8024] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:42.228170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49f328-8260-4fba-be79-6c62463414ff admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:42.297336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49f328-8260-4fba-be79-6c62463414ff admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:42.302102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8a12-9262-48e6-b703-bf6de67d5d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:42.302693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8a12-9262-48e6-b703-bf6de67d5d40 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.302966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8a12-9262-48e6-b703-bf6de67d5d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.303233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8a12-9262-48e6-b703-bf6de67d5d40 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.304559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a8a12-9262-48e6-b703-bf6de67d5d40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:42.306526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a8a12-9262-48e6-b703-bf6de67d5d40 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.332588 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3997/8025] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:23:42 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:42.337164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42eacf-6c05-4e6f-b232-0d2120501a9f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:42.347960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42eacf-6c05-4e6f-b232-0d2120501a9f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:42.404933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a9908-0a9c-4dc2-bab8-f10b6d240ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:42.405571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a9908-0a9c-4dc2-bab8-f10b6d240ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.405841 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a9908-0a9c-4dc2-bab8-f10b6d240ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.406103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a9908-0a9c-4dc2-bab8-f10b6d240ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.406711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a9908-0a9c-4dc2-bab8-f10b6d240ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:42.408577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a9908-0a9c-4dc2-bab8-f10b6d240ab1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.430044 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4029/8026] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:23:42 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:42.434053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8a12-9262-48e6-b703-bf6de67d5d40 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:42.444267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8a12-9262-48e6-b703-bf6de67d5d40 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:42.448847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b2a4a-fedd-48b9-a867-1bd501c6e31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:42.449456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b2a4a-fedd-48b9-a867-1bd501c6e31d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.449727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b2a4a-fedd-48b9-a867-1bd501c6e31d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.449988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b2a4a-fedd-48b9-a867-1bd501c6e31d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.450610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b2a4a-fedd-48b9-a867-1bd501c6e31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:42.452189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b2a4a-fedd-48b9-a867-1bd501c6e31d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.463245 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3998/8027] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:42.467236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a9908-0a9c-4dc2-bab8-f10b6d240ab1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:42.478628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a9908-0a9c-4dc2-bab8-f10b6d240ab1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:42.483228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777352c-0cd7-4ab0-b2cc-e17f436c5161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:42.483855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777352c-0cd7-4ab0-b2cc-e17f436c5161 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.484129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777352c-0cd7-4ab0-b2cc-e17f436c5161 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.484396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777352c-0cd7-4ab0-b2cc-e17f436c5161 admin admin] PATH_INFO: `/v3/projects/4e404078277d4994a37699f4fc0aaff0/users/98be9ae0d34b489ba75f59b6f9cc1e65/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.486580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:42.501148 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777352c-0cd7-4ab0-b2cc-e17f436c5161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e404078277d4994a37699f4fc0aaff0, user_id=98be9ae0d34b489ba75f59b6f9cc1e65, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:42.503763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777352c-0cd7-4ab0-b2cc-e17f436c5161 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.516047 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4030/8028] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:42 2022] PUT /identity/v3/projects/4e404078277d4994a37699f4fc0aaff0/users/98be9ae0d34b489ba75f59b6f9cc1e65/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:42.519920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b2a4a-fedd-48b9-a867-1bd501c6e31d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:42.530979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b2a4a-fedd-48b9-a867-1bd501c6e31d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:42.544125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:42.595154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657e88d-8dd5-408b-aac4-55d45d54693a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:42.595743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657e88d-8dd5-408b-aac4-55d45d54693a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.596012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657e88d-8dd5-408b-aac4-55d45d54693a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.596274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657e88d-8dd5-408b-aac4-55d45d54693a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.596891 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657e88d-8dd5-408b-aac4-55d45d54693a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:42.598636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657e88d-8dd5-408b-aac4-55d45d54693a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.609547 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 3999/8029] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:42.613734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777352c-0cd7-4ab0-b2cc-e17f436c5161 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:42.624091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777352c-0cd7-4ab0-b2cc-e17f436c5161 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:42.628842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d77fa-c17f-4728-9607-87de3d324937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:42.629512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d77fa-c17f-4728-9607-87de3d324937 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.629787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d77fa-c17f-4728-9607-87de3d324937 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.630054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d77fa-c17f-4728-9607-87de3d324937 admin admin] PATH_INFO: `/v3/projects/4e404078277d4994a37699f4fc0aaff0/users/98be9ae0d34b489ba75f59b6f9cc1e65/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.632500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d77fa-c17f-4728-9607-87de3d324937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e404078277d4994a37699f4fc0aaff0, user_id=98be9ae0d34b489ba75f59b6f9cc1e65, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:42.635250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d77fa-c17f-4728-9607-87de3d324937 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.645312 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4031/8030] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:42 2022] PUT /identity/v3/projects/4e404078277d4994a37699f4fc0aaff0/users/98be9ae0d34b489ba75f59b6f9cc1e65/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:42.655978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a1c50-94b7-4c3f-8978-9277180b02db None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.656201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a1c50-94b7-4c3f-8978-9277180b02db None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.656469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a1c50-94b7-4c3f-8978-9277180b02db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.684269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-696a1c50-94b7-4c3f-8978-9277180b02db None None] MFA Rules not processed for user `98be9ae0d34b489ba75f59b6f9cc1e65`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:42.685433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-696a1c50-94b7-4c3f-8978-9277180b02db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:42.754636 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4000/8031] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:42.766568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d77fa-c17f-4728-9607-87de3d324937 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:42.768316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-058d77fa-c17f-4728-9607-87de3d324937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:42.829544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d77fa-c17f-4728-9607-87de3d324937 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:42.833232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ecd6f-d1e7-421a-becb-079637ddfa2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:42.833829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ecd6f-d1e7-421a-becb-079637ddfa2c service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:42.834101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ecd6f-d1e7-421a-becb-079637ddfa2c service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:42.834366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ecd6f-d1e7-421a-becb-079637ddfa2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:42.844113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ecd6f-d1e7-421a-becb-079637ddfa2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:42.845577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ecd6f-d1e7-421a-becb-079637ddfa2c service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:42.855328 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4032/8032] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:43.273121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a1c50-94b7-4c3f-8978-9277180b02db None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:43.296262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ecd6f-d1e7-421a-becb-079637ddfa2c service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:43.297674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4a0ecd6f-d1e7-421a-becb-079637ddfa2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:43.338172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a1c50-94b7-4c3f-8978-9277180b02db None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:43.343342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:43.343959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:43.344231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:43.344497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] PATH_INFO: `/v3/users/8dd016ce35fb4d66bf845cd22774f280` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:43.345804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd016ce35fb4d66bf845cd22774f280)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:43.347897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:43.359158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ecd6f-d1e7-421a-becb-079637ddfa2c service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:43.363856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da093cd-081d-4bab-a429-180f008de7ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:43.364465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da093cd-081d-4bab-a429-180f008de7ae service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:43.364739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da093cd-081d-4bab-a429-180f008de7ae service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:43.365076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da093cd-081d-4bab-a429-180f008de7ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:43.367320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6da093cd-081d-4bab-a429-180f008de7ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:43.391178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:43.397190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd016ce35fb4d66bf845cd22774f280'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:43.410475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd016ce35fb4d66bf845cd22774f280'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:43.418238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd016ce35fb4d66bf845cd22774f280'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:43.425303 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd016ce35fb4d66bf845cd22774f280'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:43.425896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f3a9e87f-40ff-40b5-91db-1199327727e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd016ce35fb4d66bf845cd22774f280'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:43.431819 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4001/8033] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:23:43 2022] DELETE /identity/v3/users/8dd016ce35fb4d66bf845cd22774f280 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:43.436036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee3656-c660-47bf-8a79-713b7c3b4d0f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:43.436312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee3656-c660-47bf-8a79-713b7c3b4d0f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:43.436581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee3656-c660-47bf-8a79-713b7c3b4d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:43.447840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da093cd-081d-4bab-a429-180f008de7ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:43.449316 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da093cd-081d-4bab-a429-180f008de7ae service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:43.455501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-96ee3656-c660-47bf-8a79-713b7c3b4d0f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:43.456676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-96ee3656-c660-47bf-8a79-713b7c3b4d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:43.465847 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4033/8034] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:23:43.526035 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4002/8035] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:23:43.535793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da093cd-081d-4bab-a429-180f008de7ae service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:43.537218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6da093cd-081d-4bab-a429-180f008de7ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:43.552144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da093cd-081d-4bab-a429-180f008de7ae service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:43.562231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:43.611911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a971eb4-2aec-4023-8f96-e14d9bdf1bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:43.612554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a971eb4-2aec-4023-8f96-e14d9bdf1bec service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:43.612822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a971eb4-2aec-4023-8f96-e14d9bdf1bec service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:43.613131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a971eb4-2aec-4023-8f96-e14d9bdf1bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:43.623457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a971eb4-2aec-4023-8f96-e14d9bdf1bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:43.624996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a971eb4-2aec-4023-8f96-e14d9bdf1bec service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:43.635357 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4034/8036] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:44.086077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee3656-c660-47bf-8a79-713b7c3b4d0f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:44.087693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-96ee3656-c660-47bf-8a79-713b7c3b4d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:44.102238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee3656-c660-47bf-8a79-713b7c3b4d0f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:44.107576 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:44.108280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:44.108627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:44.108962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] PATH_INFO: `/v3/projects/362818a79ae14a8a852c56a6490e9faa` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:44.110340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=362818a79ae14a8a852c56a6490e9faa)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:44.112414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:44.164876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362818a79ae14a8a852c56a6490e9faa', 'request_id': 'req-e3623f17-ddc7-401f-a55b-8401a70b89c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:44.172606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362818a79ae14a8a852c56a6490e9faa', 'request_id': 'req-e3623f17-ddc7-401f-a55b-8401a70b89c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:44.181907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362818a79ae14a8a852c56a6490e9faa', 'request_id': 'req-e3623f17-ddc7-401f-a55b-8401a70b89c4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:44.199366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] The token cache is being invalidate because project 362818a79ae14a8a852c56a6490e9faa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:23:44.199739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:44.200450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:44.205094 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4003/8037] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:23:44 2022] DELETE /identity/v3/projects/362818a79ae14a8a852c56a6490e9faa => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:44.206623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a971eb4-2aec-4023-8f96-e14d9bdf1bec service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:44.216334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a971eb4-2aec-4023-8f96-e14d9bdf1bec service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:44.274937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:44.275590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:44.275858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:44.276123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] PATH_INFO: `/v3/users/98be9ae0d34b489ba75f59b6f9cc1e65` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:44.277255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98be9ae0d34b489ba75f59b6f9cc1e65)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:44.279145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:44.329442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98be9ae0d34b489ba75f59b6f9cc1e65'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:44.338365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98be9ae0d34b489ba75f59b6f9cc1e65'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:44.350549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98be9ae0d34b489ba75f59b6f9cc1e65'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:44.359596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98be9ae0d34b489ba75f59b6f9cc1e65'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:44.360342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98be9ae0d34b489ba75f59b6f9cc1e65'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:44.367258 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4035/8038] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:23:44 2022] DELETE /identity/v3/users/98be9ae0d34b489ba75f59b6f9cc1e65 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:44.930852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:44.932923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:44.972349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:44.974465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:45.025935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:45.040101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:45.075576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec75d8-ec08-4ec2-bd78-0682244c8ea3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:45.080781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:45.081591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.081890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.082171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] PATH_INFO: `/v3/projects/4e404078277d4994a37699f4fc0aaff0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.083789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e404078277d4994a37699f4fc0aaff0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:45.084734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3623f17-ddc7-401f-a55b-8401a70b89c4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:45.086533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:45.160917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e404078277d4994a37699f4fc0aaff0', 'request_id': 'req-2b06ef32-790f-4efb-99e8-02abc11d940a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:45.172949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e404078277d4994a37699f4fc0aaff0', 'request_id': 'req-2b06ef32-790f-4efb-99e8-02abc11d940a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:45.181417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e404078277d4994a37699f4fc0aaff0', 'request_id': 'req-2b06ef32-790f-4efb-99e8-02abc11d940a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:45.207996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231605a-b3e7-4fd0-be7e-5fac04dbea77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:45.208397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231605a-b3e7-4fd0-be7e-5fac04dbea77 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.208531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231605a-b3e7-4fd0-be7e-5fac04dbea77 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.208787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231605a-b3e7-4fd0-be7e-5fac04dbea77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.211947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b231605a-b3e7-4fd0-be7e-5fac04dbea77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:45.212963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] The token cache is being invalidate because project 4e404078277d4994a37699f4fc0aaff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:23:45.213839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:45.214865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b06ef32-790f-4efb-99e8-02abc11d940a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:45.217257 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4036/8039] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:23:44 2022] DELETE /identity/v3/projects/4e404078277d4994a37699f4fc0aaff0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:45.221915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7d3ba-2de9-4ad9-8f1d-fadfdb1b130e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.222470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7d3ba-2de9-4ad9-8f1d-fadfdb1b130e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.223292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7d3ba-2de9-4ad9-8f1d-fadfdb1b130e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.238025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231605a-b3e7-4fd0-be7e-5fac04dbea77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:45.239809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231605a-b3e7-4fd0-be7e-5fac04dbea77 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:45.250009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4bf7d3ba-2de9-4ad9-8f1d-fadfdb1b130e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:45.251415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4bf7d3ba-2de9-4ad9-8f1d-fadfdb1b130e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:45.262424 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4004/8040] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:45.336230 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4037/8041] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:45.343902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6ee47-e872-4652-923a-8dcaa1404337 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.344182 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6ee47-e872-4652-923a-8dcaa1404337 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.344450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6ee47-e872-4652-923a-8dcaa1404337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.367514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-dfe6ee47-e872-4652-923a-8dcaa1404337 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:45.368630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dfe6ee47-e872-4652-923a-8dcaa1404337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:45.376912 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4005/8042] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:45.381173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7d3ba-2de9-4ad9-8f1d-fadfdb1b130e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:45.391146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7d3ba-2de9-4ad9-8f1d-fadfdb1b130e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:45.396125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb6035-43d6-4e0f-a74e-d084fe43d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:45.396348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb6035-43d6-4e0f-a74e-d084fe43d754 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.396654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb6035-43d6-4e0f-a74e-d084fe43d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.396923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb6035-43d6-4e0f-a74e-d084fe43d754 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.398147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb6035-43d6-4e0f-a74e-d084fe43d754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:23:45.398448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb6035-43d6-4e0f-a74e-d084fe43d754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:45.400053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb6035-43d6-4e0f-a74e-d084fe43d754 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:45.415450 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4038/8043] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:23:45.416549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6ee47-e872-4652-923a-8dcaa1404337 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:45.425921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6ee47-e872-4652-923a-8dcaa1404337 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:45.431516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e172-a8b5-4d71-a118-85ba792ca756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:45.432107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e172-a8b5-4d71-a118-85ba792ca756 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.432404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e172-a8b5-4d71-a118-85ba792ca756 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.432671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e172-a8b5-4d71-a118-85ba792ca756 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.433462 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75e172-a8b5-4d71-a118-85ba792ca756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:45.435449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75e172-a8b5-4d71-a118-85ba792ca756 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:45.467597 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4006/8044] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:23:45 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:23:45.468740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb6035-43d6-4e0f-a74e-d084fe43d754 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:45.480036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb6035-43d6-4e0f-a74e-d084fe43d754 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:45.529970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:45.541329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9018c9c-8098-4027-9279-936a181f4106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:45.542188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9018c9c-8098-4027-9279-936a181f4106 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.542483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9018c9c-8098-4027-9279-936a181f4106 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.542749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9018c9c-8098-4027-9279-936a181f4106 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.543481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9018c9c-8098-4027-9279-936a181f4106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:45.545480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9018c9c-8098-4027-9279-936a181f4106 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:45.568249 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4039/8045] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:23:45 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:23:45.572809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e172-a8b5-4d71-a118-85ba792ca756 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:45.583229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e172-a8b5-4d71-a118-85ba792ca756 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:45.588408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d7508-90a9-4f62-b4a2-0f3c7dfde940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:45.589009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d7508-90a9-4f62-b4a2-0f3c7dfde940 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.589326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d7508-90a9-4f62-b4a2-0f3c7dfde940 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.589579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d7508-90a9-4f62-b4a2-0f3c7dfde940 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.590204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d7508-90a9-4f62-b4a2-0f3c7dfde940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:45.591870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d7508-90a9-4f62-b4a2-0f3c7dfde940 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:45.603535 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4007/8046] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:45.607566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9018c9c-8098-4027-9279-936a181f4106 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:45.618410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9018c9c-8098-4027-9279-936a181f4106 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:45.622996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d17a31-8040-4199-bb74-5a034f023031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:45.623594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d17a31-8040-4199-bb74-5a034f023031 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.623883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d17a31-8040-4199-bb74-5a034f023031 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.624132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d17a31-8040-4199-bb74-5a034f023031 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.624761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d17a31-8040-4199-bb74-5a034f023031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:45.626380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d17a31-8040-4199-bb74-5a034f023031 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:45.640100 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4040/8047] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:23:45.641278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d7508-90a9-4f62-b4a2-0f3c7dfde940 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:45.652324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d7508-90a9-4f62-b4a2-0f3c7dfde940 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:45.658928 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b8f95-6b91-4480-a439-5f98d7716f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:45.659619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b8f95-6b91-4480-a439-5f98d7716f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.659889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b8f95-6b91-4480-a439-5f98d7716f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.660153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b8f95-6b91-4480-a439-5f98d7716f63 admin admin] PATH_INFO: `/v3/projects/ca3089cd257d441f91dde57d69b8f0b6/users/be6a46dc9d414db7b2983801dc6adac7/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.676177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b8f95-6b91-4480-a439-5f98d7716f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca3089cd257d441f91dde57d69b8f0b6, user_id=be6a46dc9d414db7b2983801dc6adac7, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:45.678983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b8f95-6b91-4480-a439-5f98d7716f63 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:45.692235 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4008/8048] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:45 2022] PUT /identity/v3/projects/ca3089cd257d441f91dde57d69b8f0b6/users/be6a46dc9d414db7b2983801dc6adac7/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:45.703218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4d4eb-09ed-4287-8ec6-f3fb36011afc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.703587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4d4eb-09ed-4287-8ec6-f3fb36011afc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.703931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4d4eb-09ed-4287-8ec6-f3fb36011afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.733845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a7a4d4eb-09ed-4287-8ec6-f3fb36011afc None None] MFA Rules not processed for user `be6a46dc9d414db7b2983801dc6adac7`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:45.735062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a7a4d4eb-09ed-4287-8ec6-f3fb36011afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:45.803343 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4041/8049] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:45.810758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b8f95-6b91-4480-a439-5f98d7716f63 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:45.880608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b8f95-6b91-4480-a439-5f98d7716f63 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:45.886004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf1a06-6444-47b1-a97d-1aec8989e200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:45.886602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf1a06-6444-47b1-a97d-1aec8989e200 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.886882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf1a06-6444-47b1-a97d-1aec8989e200 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.887151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf1a06-6444-47b1-a97d-1aec8989e200 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.887770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf1a06-6444-47b1-a97d-1aec8989e200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:45.889734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf1a06-6444-47b1-a97d-1aec8989e200 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:45.895023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:45.918441 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4009/8050] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:23:45 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:45.922405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4d4eb-09ed-4287-8ec6-f3fb36011afc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:45.934118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4d4eb-09ed-4287-8ec6-f3fb36011afc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:45.994153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7b341-8e39-4e4b-894d-d3906c7ea4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:45.994952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7b341-8e39-4e4b-894d-d3906c7ea4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:45.995230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7b341-8e39-4e4b-894d-d3906c7ea4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:45.995498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7b341-8e39-4e4b-894d-d3906c7ea4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:45.996168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7b341-8e39-4e4b-894d-d3906c7ea4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:45.998152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7b341-8e39-4e4b-894d-d3906c7ea4a4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:46.023726 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4042/8051] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:23:45 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:23:46.027806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf1a06-6444-47b1-a97d-1aec8989e200 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:46.040030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf1a06-6444-47b1-a97d-1aec8989e200 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:46.044799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8743403-30ca-4936-9e53-d4fb0569efb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:46.045443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8743403-30ca-4936-9e53-d4fb0569efb0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:46.046365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8743403-30ca-4936-9e53-d4fb0569efb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:46.046641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8743403-30ca-4936-9e53-d4fb0569efb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:46.048098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8743403-30ca-4936-9e53-d4fb0569efb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:46.049774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8743403-30ca-4936-9e53-d4fb0569efb0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:46.063838 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4010/8052] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:23:46.065511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7b341-8e39-4e4b-894d-d3906c7ea4a4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:46.077276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7b341-8e39-4e4b-894d-d3906c7ea4a4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:46.082020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c80397-b4c6-46ef-b35b-6cd5e1e85b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:46.082674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c80397-b4c6-46ef-b35b-6cd5e1e85b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:46.083009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c80397-b4c6-46ef-b35b-6cd5e1e85b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:46.083277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c80397-b4c6-46ef-b35b-6cd5e1e85b7c admin admin] PATH_INFO: `/v3/projects/f3919333ea33409987046c7d579ed7c5/users/ec6183e1070241a6bb6b507ee0813d33/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:46.102655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c80397-b4c6-46ef-b35b-6cd5e1e85b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3919333ea33409987046c7d579ed7c5, user_id=ec6183e1070241a6bb6b507ee0813d33, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:46.106396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c80397-b4c6-46ef-b35b-6cd5e1e85b7c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:46.125220 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4043/8053] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:46 2022] PUT /identity/v3/projects/f3919333ea33409987046c7d579ed7c5/users/ec6183e1070241a6bb6b507ee0813d33/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:23:46.128207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8743403-30ca-4936-9e53-d4fb0569efb0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:46.141763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8743403-30ca-4936-9e53-d4fb0569efb0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:46.202156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54867d-0e6c-4b9b-97b6-503d7997b58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:46.202759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54867d-0e6c-4b9b-97b6-503d7997b58e admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:46.203037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54867d-0e6c-4b9b-97b6-503d7997b58e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:46.203298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54867d-0e6c-4b9b-97b6-503d7997b58e admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:46.203936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54867d-0e6c-4b9b-97b6-503d7997b58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:46.205559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54867d-0e6c-4b9b-97b6-503d7997b58e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:46.216719 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4011/8054] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:46.220883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c80397-b4c6-46ef-b35b-6cd5e1e85b7c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:46.231694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c80397-b4c6-46ef-b35b-6cd5e1e85b7c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:46.236561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80045da6-2b77-4ece-8584-034c286e4bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:46.237327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80045da6-2b77-4ece-8584-034c286e4bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:46.237602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80045da6-2b77-4ece-8584-034c286e4bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:46.237888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80045da6-2b77-4ece-8584-034c286e4bc2 admin admin] PATH_INFO: `/v3/projects/f3919333ea33409987046c7d579ed7c5/users/ec6183e1070241a6bb6b507ee0813d33/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:46.239937 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80045da6-2b77-4ece-8584-034c286e4bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3919333ea33409987046c7d579ed7c5, user_id=ec6183e1070241a6bb6b507ee0813d33, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:46.243465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80045da6-2b77-4ece-8584-034c286e4bc2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:46.254476 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4044/8055] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:46 2022] PUT /identity/v3/projects/f3919333ea33409987046c7d579ed7c5/users/ec6183e1070241a6bb6b507ee0813d33/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:46.265850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca22ea-daac-431b-9ebb-3de6af0286e9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:46.266135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca22ea-daac-431b-9ebb-3de6af0286e9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:46.266404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca22ea-daac-431b-9ebb-3de6af0286e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:46.295595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7fca22ea-daac-431b-9ebb-3de6af0286e9 None None] MFA Rules not processed for user `ec6183e1070241a6bb6b507ee0813d33`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:46.296798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7fca22ea-daac-431b-9ebb-3de6af0286e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:46.387971 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4012/8056] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:46.399998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80045da6-2b77-4ece-8584-034c286e4bc2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:46.401712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80045da6-2b77-4ece-8584-034c286e4bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:46.416493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:46.466802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80045da6-2b77-4ece-8584-034c286e4bc2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:46.470528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2abc6d-630e-48f8-b80b-4b7b96dfde3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:46.471124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2abc6d-630e-48f8-b80b-4b7b96dfde3d service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:46.471396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2abc6d-630e-48f8-b80b-4b7b96dfde3d service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:46.471662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2abc6d-630e-48f8-b80b-4b7b96dfde3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:46.482169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2abc6d-630e-48f8-b80b-4b7b96dfde3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:46.483641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2abc6d-630e-48f8-b80b-4b7b96dfde3d service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:46.494618 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4045/8057] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:23:46.576018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca22ea-daac-431b-9ebb-3de6af0286e9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:46.588353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca22ea-daac-431b-9ebb-3de6af0286e9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:46.592058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2567d7-d078-4e07-a179-d8b811bc6455 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:46.592769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2567d7-d078-4e07-a179-d8b811bc6455 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:46.593211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2567d7-d078-4e07-a179-d8b811bc6455 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:46.593527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2567d7-d078-4e07-a179-d8b811bc6455 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:46.657122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:46.659552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2567d7-d078-4e07-a179-d8b811bc6455 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:46.661700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2567d7-d078-4e07-a179-d8b811bc6455 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:46.672592 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4013/8058] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:49.430969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2abc6d-630e-48f8-b80b-4b7b96dfde3d service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:49.494416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2abc6d-630e-48f8-b80b-4b7b96dfde3d service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:49.498961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:49.499585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:49.499860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:49.500125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] PATH_INFO: `/v3/users/be6a46dc9d414db7b2983801dc6adac7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:49.501278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=be6a46dc9d414db7b2983801dc6adac7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:49.503174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:49.552319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be6a46dc9d414db7b2983801dc6adac7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:49.560352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be6a46dc9d414db7b2983801dc6adac7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:49.577810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be6a46dc9d414db7b2983801dc6adac7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:49.587234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be6a46dc9d414db7b2983801dc6adac7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:49.587928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be6a46dc9d414db7b2983801dc6adac7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:49.594969 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4046/8059] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:23:49 2022] DELETE /identity/v3/users/be6a46dc9d414db7b2983801dc6adac7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:49.599614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f14b0b-87ef-4c36-839f-290a8d2f0aef None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:49.600130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f14b0b-87ef-4c36-839f-290a8d2f0aef None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:49.601378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f14b0b-87ef-4c36-839f-290a8d2f0aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:49.627088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-41f14b0b-87ef-4c36-839f-290a8d2f0aef None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:49.628477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-41f14b0b-87ef-4c36-839f-290a8d2f0aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:49.702620 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4014/8060] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:23:49.713195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:49.714631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:49.761881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:49.790669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf1ddd-1c77-4237-8700-495655cb6c2c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:49.794475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601509b-1fe3-4b26-bae7-1e6d46372b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:49.795087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601509b-1fe3-4b26-bae7-1e6d46372b87 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:49.795365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601509b-1fe3-4b26-bae7-1e6d46372b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:49.795637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601509b-1fe3-4b26-bae7-1e6d46372b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:49.805810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601509b-1fe3-4b26-bae7-1e6d46372b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:49.807253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601509b-1fe3-4b26-bae7-1e6d46372b87 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:49.818081 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4047/8061] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:23:50.117280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f14b0b-87ef-4c36-839f-290a8d2f0aef None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:50.118843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-41f14b0b-87ef-4c36-839f-290a8d2f0aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:50.132453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f14b0b-87ef-4c36-839f-290a8d2f0aef None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:50.140656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:50.141344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:50.141753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:50.141968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] PATH_INFO: `/v3/projects/ca3089cd257d441f91dde57d69b8f0b6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:50.143022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca3089cd257d441f91dde57d69b8f0b6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:50.145102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:50.229868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca3089cd257d441f91dde57d69b8f0b6', 'request_id': 'req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:50.244207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca3089cd257d441f91dde57d69b8f0b6', 'request_id': 'req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:50.252886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca3089cd257d441f91dde57d69b8f0b6', 'request_id': 'req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:50.266458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] The token cache is being invalidate because project ca3089cd257d441f91dde57d69b8f0b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:23:50.266751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:50.267413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:50.272248 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4015/8062] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:23:50 2022] DELETE /identity/v3/projects/ca3089cd257d441f91dde57d69b8f0b6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:50.275724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601509b-1fe3-4b26-bae7-1e6d46372b87 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:50.295122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601509b-1fe3-4b26-bae7-1e6d46372b87 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:50.386132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:50.386902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:50.387228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:50.387505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] PATH_INFO: `/v3/users/ec6183e1070241a6bb6b507ee0813d33` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:50.388831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec6183e1070241a6bb6b507ee0813d33)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:50.390898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:50.469346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec6183e1070241a6bb6b507ee0813d33'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:50.484842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec6183e1070241a6bb6b507ee0813d33'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:50.501408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec6183e1070241a6bb6b507ee0813d33'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:50.521452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec6183e1070241a6bb6b507ee0813d33'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:50.522618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9a612979-96cd-4318-a56b-682db316c712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec6183e1070241a6bb6b507ee0813d33'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:50.536979 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4048/8063] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:23:50 2022] DELETE /identity/v3/users/ec6183e1070241a6bb6b507ee0813d33 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:50.913056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:50.914623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:50.928965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeb8bd-1287-40f3-aafa-171c5ba6fc0b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:51.000605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:51.001645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.001750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.001968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] PATH_INFO: `/v3/projects/f3919333ea33409987046c7d579ed7c5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.004565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3919333ea33409987046c7d579ed7c5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:51.008944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:51.044872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:51.074499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3919333ea33409987046c7d579ed7c5', 'request_id': 'req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:51.085840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3919333ea33409987046c7d579ed7c5', 'request_id': 'req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:51.095701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3919333ea33409987046c7d579ed7c5', 'request_id': 'req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:51.113625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] The token cache is being invalidate because project f3919333ea33409987046c7d579ed7c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:23:51.113847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:51.114547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-eb4f769a-68e4-4b6d-91f6-efa054c10dc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:51.116554 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4016/8064] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:23:50 2022] DELETE /identity/v3/projects/f3919333ea33409987046c7d579ed7c5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:51.121989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19b32a-a978-4fb5-80b1-68907e1c7d9f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.122341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19b32a-a978-4fb5-80b1-68907e1c7d9f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.122710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19b32a-a978-4fb5-80b1-68907e1c7d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.157921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cb19b32a-a978-4fb5-80b1-68907e1c7d9f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:51.159543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cb19b32a-a978-4fb5-80b1-68907e1c7d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:51.246020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:51.261675 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4049/8065] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:51.269190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750456a-1391-4580-9ce4-5e80e100b507 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.269551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750456a-1391-4580-9ce4-5e80e100b507 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.269900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750456a-1391-4580-9ce4-5e80e100b507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.295034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2750456a-1391-4580-9ce4-5e80e100b507 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:51.296468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2750456a-1391-4580-9ce4-5e80e100b507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:51.308563 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4017/8066] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:51.309479 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19b32a-a978-4fb5-80b1-68907e1c7d9f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:51.321278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19b32a-a978-4fb5-80b1-68907e1c7d9f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:51.326226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d4cb3-83c3-4aa9-a967-69e224644081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:51.326835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d4cb3-83c3-4aa9-a967-69e224644081 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.327100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d4cb3-83c3-4aa9-a967-69e224644081 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.327364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d4cb3-83c3-4aa9-a967-69e224644081 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.328035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d4cb3-83c3-4aa9-a967-69e224644081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:23:51.328333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d4cb3-83c3-4aa9-a967-69e224644081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:51.329939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d4cb3-83c3-4aa9-a967-69e224644081 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:51.345095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4050/8067] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:23:51.347537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750456a-1391-4580-9ce4-5e80e100b507 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:51.358445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750456a-1391-4580-9ce4-5e80e100b507 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:51.365245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d97d7-4c00-4110-b661-5d5ba2d1c601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:51.366599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d97d7-4c00-4110-b661-5d5ba2d1c601 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.366984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d97d7-4c00-4110-b661-5d5ba2d1c601 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.367389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d97d7-4c00-4110-b661-5d5ba2d1c601 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.368069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d97d7-4c00-4110-b661-5d5ba2d1c601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:51.374286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d97d7-4c00-4110-b661-5d5ba2d1c601 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:51.409150 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4018/8068] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:23:51 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:51.412815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d4cb3-83c3-4aa9-a967-69e224644081 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:51.424276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d4cb3-83c3-4aa9-a967-69e224644081 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:51.491675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fe6b7-4a35-4dfc-97ee-efdb5b7cf8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:51.492671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fe6b7-4a35-4dfc-97ee-efdb5b7cf8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.492671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fe6b7-4a35-4dfc-97ee-efdb5b7cf8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.493014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fe6b7-4a35-4dfc-97ee-efdb5b7cf8d0 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.493618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fe6b7-4a35-4dfc-97ee-efdb5b7cf8d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:51.495591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fe6b7-4a35-4dfc-97ee-efdb5b7cf8d0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:51.536621 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4051/8069] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:23:51 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:23:51.541637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d97d7-4c00-4110-b661-5d5ba2d1c601 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:51.556557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d97d7-4c00-4110-b661-5d5ba2d1c601 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:51.561479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ad429-6805-491e-9f20-c6d0ecd08bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:51.562089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ad429-6805-491e-9f20-c6d0ecd08bcb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.562558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ad429-6805-491e-9f20-c6d0ecd08bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.562836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ad429-6805-491e-9f20-c6d0ecd08bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.563487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ad429-6805-491e-9f20-c6d0ecd08bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:51.565211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ad429-6805-491e-9f20-c6d0ecd08bcb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:51.576782 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4019/8070] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:51.580867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fe6b7-4a35-4dfc-97ee-efdb5b7cf8d0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:51.593362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fe6b7-4a35-4dfc-97ee-efdb5b7cf8d0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:51.598238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69315593-b92a-4733-997f-ef1c1552b3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:51.599125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69315593-b92a-4733-997f-ef1c1552b3bd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.599125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69315593-b92a-4733-997f-ef1c1552b3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.599364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69315593-b92a-4733-997f-ef1c1552b3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.599999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69315593-b92a-4733-997f-ef1c1552b3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:51.602762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69315593-b92a-4733-997f-ef1c1552b3bd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:51.617386 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4052/8071] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:51.620405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ad429-6805-491e-9f20-c6d0ecd08bcb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:51.632082 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ad429-6805-491e-9f20-c6d0ecd08bcb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:51.637226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ea4e1-4373-4572-bfef-fafbe0789b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:51.637894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ea4e1-4373-4572-bfef-fafbe0789b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.638168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ea4e1-4373-4572-bfef-fafbe0789b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.638439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ea4e1-4373-4572-bfef-fafbe0789b6e admin admin] PATH_INFO: `/v3/projects/c80f8fc65d7e45cead07577070327ba7/users/63271157e9284825bd0244dd07d60d23/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.656167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ea4e1-4373-4572-bfef-fafbe0789b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80f8fc65d7e45cead07577070327ba7, user_id=63271157e9284825bd0244dd07d60d23, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:51.659078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ea4e1-4373-4572-bfef-fafbe0789b6e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:51.679765 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4020/8072] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:51 2022] PUT /identity/v3/projects/c80f8fc65d7e45cead07577070327ba7/users/63271157e9284825bd0244dd07d60d23/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:51.690495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad5694-f19c-4007-9da8-7d051460dd69 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.690879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad5694-f19c-4007-9da8-7d051460dd69 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.691145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad5694-f19c-4007-9da8-7d051460dd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.738665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-59ad5694-f19c-4007-9da8-7d051460dd69 None None] MFA Rules not processed for user `63271157e9284825bd0244dd07d60d23`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:51.740809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-59ad5694-f19c-4007-9da8-7d051460dd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:51.780757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ea4e1-4373-4572-bfef-fafbe0789b6e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:51.782623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2e2ea4e1-4373-4572-bfef-fafbe0789b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:51.832355 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4053/8073] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:23:51.837172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad5694-f19c-4007-9da8-7d051460dd69 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:51.892775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ea4e1-4373-4572-bfef-fafbe0789b6e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:51.898318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3daf68-ea0f-480a-8bcc-28e54930b37b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:51.899512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3daf68-ea0f-480a-8bcc-28e54930b37b service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.899787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3daf68-ea0f-480a-8bcc-28e54930b37b service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.900053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3daf68-ea0f-480a-8bcc-28e54930b37b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.902805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1d3daf68-ea0f-480a-8bcc-28e54930b37b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:51.916122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:51.921317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3daf68-ea0f-480a-8bcc-28e54930b37b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:51.923948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3daf68-ea0f-480a-8bcc-28e54930b37b service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:51.933242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad5694-f19c-4007-9da8-7d051460dd69 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:51.940118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df601f67-064e-4e39-bcda-a9292d573099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:51.941158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df601f67-064e-4e39-bcda-a9292d573099 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:51.941480 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4021/8074] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:23:51.943605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df601f67-064e-4e39-bcda-a9292d573099 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:51.944607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df601f67-064e-4e39-bcda-a9292d573099 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:51.945505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df601f67-064e-4e39-bcda-a9292d573099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:51.953166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df601f67-064e-4e39-bcda-a9292d573099 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:51.995135 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4054/8075] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:23:51 2022] POST /identity/v3/projects => generated 391 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:51.996322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3daf68-ea0f-480a-8bcc-28e54930b37b service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:52.013528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3daf68-ea0f-480a-8bcc-28e54930b37b service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:52.048191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:52.078632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b7171-dd56-46c0-86f0-a64f562e37f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:52.079334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b7171-dd56-46c0-86f0-a64f562e37f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:52.079494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b7171-dd56-46c0-86f0-a64f562e37f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:52.079758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b7171-dd56-46c0-86f0-a64f562e37f5 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:52.080534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b7171-dd56-46c0-86f0-a64f562e37f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:52.082497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b7171-dd56-46c0-86f0-a64f562e37f5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:52.106873 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4022/8076] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:23:51 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:23:52.110783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df601f67-064e-4e39-bcda-a9292d573099 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:52.126928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df601f67-064e-4e39-bcda-a9292d573099 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:52.134926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aab2e3-4e77-4ea3-b1d9-b0df83cd044b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:52.135535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aab2e3-4e77-4ea3-b1d9-b0df83cd044b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:52.135815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aab2e3-4e77-4ea3-b1d9-b0df83cd044b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:52.136081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aab2e3-4e77-4ea3-b1d9-b0df83cd044b admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:52.136829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aab2e3-4e77-4ea3-b1d9-b0df83cd044b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:52.138464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aab2e3-4e77-4ea3-b1d9-b0df83cd044b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:52.151660 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4055/8077] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:52.155782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b7171-dd56-46c0-86f0-a64f562e37f5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:52.166332 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b7171-dd56-46c0-86f0-a64f562e37f5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:52.171038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff181ae-30f3-4baf-baef-5481bbea448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:52.171746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff181ae-30f3-4baf-baef-5481bbea448f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:52.172083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff181ae-30f3-4baf-baef-5481bbea448f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:52.172456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff181ae-30f3-4baf-baef-5481bbea448f admin admin] PATH_INFO: `/v3/projects/e02b1542a13040c0a8b58d51f544936e/users/b767899dc55e44e29a61c4b5b916943f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:52.188667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff181ae-30f3-4baf-baef-5481bbea448f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02b1542a13040c0a8b58d51f544936e, user_id=b767899dc55e44e29a61c4b5b916943f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:52.191456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff181ae-30f3-4baf-baef-5481bbea448f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:52.206177 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4023/8078] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:52 2022] PUT /identity/v3/projects/e02b1542a13040c0a8b58d51f544936e/users/b767899dc55e44e29a61c4b5b916943f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:52.211526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aab2e3-4e77-4ea3-b1d9-b0df83cd044b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:52.221754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aab2e3-4e77-4ea3-b1d9-b0df83cd044b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:52.287280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303603af-255f-499a-85a8-8c2c95983681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:52.288373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303603af-255f-499a-85a8-8c2c95983681 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:52.288647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303603af-255f-499a-85a8-8c2c95983681 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:52.288915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303603af-255f-499a-85a8-8c2c95983681 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:52.289588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303603af-255f-499a-85a8-8c2c95983681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:52.291180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303603af-255f-499a-85a8-8c2c95983681 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:52.302001 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4056/8079] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:52.306734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff181ae-30f3-4baf-baef-5481bbea448f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:52.318622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff181ae-30f3-4baf-baef-5481bbea448f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:52.325009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883bb99-3eb0-4515-922e-fcd8cbdec408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:52.325550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883bb99-3eb0-4515-922e-fcd8cbdec408 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:52.326167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883bb99-3eb0-4515-922e-fcd8cbdec408 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:52.326304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883bb99-3eb0-4515-922e-fcd8cbdec408 admin admin] PATH_INFO: `/v3/projects/e02b1542a13040c0a8b58d51f544936e/users/b767899dc55e44e29a61c4b5b916943f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:52.328766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883bb99-3eb0-4515-922e-fcd8cbdec408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02b1542a13040c0a8b58d51f544936e, user_id=b767899dc55e44e29a61c4b5b916943f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:52.331465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883bb99-3eb0-4515-922e-fcd8cbdec408 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:52.353886 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4024/8080] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:52 2022] PUT /identity/v3/projects/e02b1542a13040c0a8b58d51f544936e/users/b767899dc55e44e29a61c4b5b916943f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:52.462579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210136-ad49-42c0-a51d-7e1231a6dfd0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:52.462949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210136-ad49-42c0-a51d-7e1231a6dfd0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:52.463225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210136-ad49-42c0-a51d-7e1231a6dfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:52.498319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-03210136-ad49-42c0-a51d-7e1231a6dfd0 None None] MFA Rules not processed for user `b767899dc55e44e29a61c4b5b916943f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:52.500506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-03210136-ad49-42c0-a51d-7e1231a6dfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:52.585093 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4057/8081] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:52 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:52.594112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883bb99-3eb0-4515-922e-fcd8cbdec408 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:52.595830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2883bb99-3eb0-4515-922e-fcd8cbdec408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:52.660236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883bb99-3eb0-4515-922e-fcd8cbdec408 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:52.664667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336d1b2-b647-445b-b07e-c8aa58682e40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:52.665315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336d1b2-b647-445b-b07e-c8aa58682e40 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:52.665591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336d1b2-b647-445b-b07e-c8aa58682e40 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:52.665856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336d1b2-b647-445b-b07e-c8aa58682e40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:52.675800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336d1b2-b647-445b-b07e-c8aa58682e40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:52.677298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336d1b2-b647-445b-b07e-c8aa58682e40 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:52.687609 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4025/8082] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:52 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:52.741548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03210136-ad49-42c0-a51d-7e1231a6dfd0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:52.743156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-03210136-ad49-42c0-a51d-7e1231a6dfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:52.759472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03210136-ad49-42c0-a51d-7e1231a6dfd0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:52.763695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc901-6b02-4d77-9a0f-226d16c590c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:52.764373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc901-6b02-4d77-9a0f-226d16c590c6 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:52.764720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc901-6b02-4d77-9a0f-226d16c590c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:52.765429 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc901-6b02-4d77-9a0f-226d16c590c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:52.818645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:52.833397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc901-6b02-4d77-9a0f-226d16c590c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:52.835978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc901-6b02-4d77-9a0f-226d16c590c6 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:52.845403 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4058/8083] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:56.333372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336d1b2-b647-445b-b07e-c8aa58682e40 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:56.425440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336d1b2-b647-445b-b07e-c8aa58682e40 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:56.431376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:56.432108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:56.432457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:56.432776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] PATH_INFO: `/v3/users/63271157e9284825bd0244dd07d60d23` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:56.434041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63271157e9284825bd0244dd07d60d23)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:56.436054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:56.490407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:56.516006 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63271157e9284825bd0244dd07d60d23'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:56.540490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63271157e9284825bd0244dd07d60d23'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:56.555822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63271157e9284825bd0244dd07d60d23'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:56.564538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63271157e9284825bd0244dd07d60d23'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:56.565497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63271157e9284825bd0244dd07d60d23'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:56.573657 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4026/8084] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:23:56 2022] DELETE /identity/v3/users/63271157e9284825bd0244dd07d60d23 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:56.579094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d86c34-6175-41ff-a02a-9866d2d87b2f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:56.579541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d86c34-6175-41ff-a02a-9866d2d87b2f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:56.579798 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d86c34-6175-41ff-a02a-9866d2d87b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:56.608060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-72d86c34-6175-41ff-a02a-9866d2d87b2f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:56.609405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-72d86c34-6175-41ff-a02a-9866d2d87b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:56.713280 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4059/8085] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:23:56.735515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:56.737095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:56.848682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0caabe-9cef-4f03-8b39-2e7d074b40f4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:56.852773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b60872-11c8-471c-8959-b395331cb8ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:56.854275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b60872-11c8-471c-8959-b395331cb8ef service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:56.854275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b60872-11c8-471c-8959-b395331cb8ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:56.854275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b60872-11c8-471c-8959-b395331cb8ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:56.883286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b60872-11c8-471c-8959-b395331cb8ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:56.884766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b60872-11c8-471c-8959-b395331cb8ef service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:56.901123 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4027/8086] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:23:57.368972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d86c34-6175-41ff-a02a-9866d2d87b2f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:57.370731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-72d86c34-6175-41ff-a02a-9866d2d87b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:57.391154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d86c34-6175-41ff-a02a-9866d2d87b2f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:57.405029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:57.405769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:57.406041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:57.406304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] PATH_INFO: `/v3/projects/c80f8fc65d7e45cead07577070327ba7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:57.407442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80f8fc65d7e45cead07577070327ba7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:57.409391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:57.492580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80f8fc65d7e45cead07577070327ba7', 'request_id': 'req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:57.504532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80f8fc65d7e45cead07577070327ba7', 'request_id': 'req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:57.513592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80f8fc65d7e45cead07577070327ba7', 'request_id': 'req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:57.533777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] The token cache is being invalidate because project c80f8fc65d7e45cead07577070327ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:23:57.534071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:57.534832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:57.536658 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4060/8087] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:23:57 2022] DELETE /identity/v3/projects/c80f8fc65d7e45cead07577070327ba7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:57.541072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b60872-11c8-471c-8959-b395331cb8ef service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:57.553571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b60872-11c8-471c-8959-b395331cb8ef service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:57.644835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:57.645531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:57.645814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:57.646062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] PATH_INFO: `/v3/users/b767899dc55e44e29a61c4b5b916943f` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:57.647752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b767899dc55e44e29a61c4b5b916943f)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:57.650119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:57.716432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b767899dc55e44e29a61c4b5b916943f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:57.723068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:57.731962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b767899dc55e44e29a61c4b5b916943f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:57.742308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b767899dc55e44e29a61c4b5b916943f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:57.756824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b767899dc55e44e29a61c4b5b916943f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:57.758041 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c1a56900-c85c-4a2e-bac3-5d51a87474f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b767899dc55e44e29a61c4b5b916943f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:57.766479 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4028/8088] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:23:57 2022] DELETE /identity/v3/users/b767899dc55e44e29a61c4b5b916943f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:58.124681 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:58.130045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:58.152952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2c3fd-5081-42b6-84e7-d6e9f2767af3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:58.249399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:58.250480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:58.250973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:58.253450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] PATH_INFO: `/v3/projects/e02b1542a13040c0a8b58d51f544936e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:58.254795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02b1542a13040c0a8b58d51f544936e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:58.257193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:58.272412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:58.310694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29742a4-0695-4e5a-922c-a044a98caa7c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:58.311926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29742a4-0695-4e5a-922c-a044a98caa7c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:58.312152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29742a4-0695-4e5a-922c-a044a98caa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:58.336682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f29742a4-0695-4e5a-922c-a044a98caa7c None None] MFA Rules not processed for user `a0d6261b46824acf958ef749425ae421`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:58.336911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02b1542a13040c0a8b58d51f544936e', 'request_id': 'req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:58.338152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f29742a4-0695-4e5a-922c-a044a98caa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:58.349334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02b1542a13040c0a8b58d51f544936e', 'request_id': 'req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:58.387783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02b1542a13040c0a8b58d51f544936e', 'request_id': 'req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:58.416782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] The token cache is being invalidate because project e02b1542a13040c0a8b58d51f544936e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:23:58.418109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:58.424382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e7cc5a9d-6230-44d8-b138-bb47dead1ffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:23:58.427733 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4061/8089] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:23:58 2022] DELETE /identity/v3/projects/e02b1542a13040c0a8b58d51f544936e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:58.444472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89443939-ef0d-4b0b-a8f9-a6081e235e9b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:58.444472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89443939-ef0d-4b0b-a8f9-a6081e235e9b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:58.444825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89443939-ef0d-4b0b-a8f9-a6081e235e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:58.465814 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4029/8090] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:23:58.483166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-89443939-ef0d-4b0b-a8f9-a6081e235e9b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:58.484226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-89443939-ef0d-4b0b-a8f9-a6081e235e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:58.611656 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4062/8091] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:58.619555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7368f1-ac9a-47e0-a64b-e742fc9b82f3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:58.620057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7368f1-ac9a-47e0-a64b-e742fc9b82f3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:58.620184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7368f1-ac9a-47e0-a64b-e742fc9b82f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:58.647350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-5b7368f1-ac9a-47e0-a64b-e742fc9b82f3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:58.648831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5b7368f1-ac9a-47e0-a64b-e742fc9b82f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:58.665722 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4030/8092] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:58.670949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89443939-ef0d-4b0b-a8f9-a6081e235e9b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:58.681435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89443939-ef0d-4b0b-a8f9-a6081e235e9b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:58.689734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b0ba-ab78-498a-8eb6-8d4e0ab5d544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:58.690540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b0ba-ab78-498a-8eb6-8d4e0ab5d544 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:58.690680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b0ba-ab78-498a-8eb6-8d4e0ab5d544 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:58.690935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b0ba-ab78-498a-8eb6-8d4e0ab5d544 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:58.691587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596b0ba-ab78-498a-8eb6-8d4e0ab5d544 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:23:58.691880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596b0ba-ab78-498a-8eb6-8d4e0ab5d544 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:58.697788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596b0ba-ab78-498a-8eb6-8d4e0ab5d544 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:58.715861 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4063/8093] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:23:58.720302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7368f1-ac9a-47e0-a64b-e742fc9b82f3 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:58.733219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7368f1-ac9a-47e0-a64b-e742fc9b82f3 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:58.739276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d766e6-9dbf-40ee-847e-0f8c81adee51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:58.739895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d766e6-9dbf-40ee-847e-0f8c81adee51 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:58.740172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d766e6-9dbf-40ee-847e-0f8c81adee51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:58.740511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d766e6-9dbf-40ee-847e-0f8c81adee51 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:58.741185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d766e6-9dbf-40ee-847e-0f8c81adee51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:58.744224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d766e6-9dbf-40ee-847e-0f8c81adee51 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:58.791192 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4031/8094] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:23:58 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:58.795594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b0ba-ab78-498a-8eb6-8d4e0ab5d544 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:58.808326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b0ba-ab78-498a-8eb6-8d4e0ab5d544 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:58.894238 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2cf5f-4a01-40dc-b718-871e0e8e87c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:58.894861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2cf5f-4a01-40dc-b718-871e0e8e87c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:58.895142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2cf5f-4a01-40dc-b718-871e0e8e87c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:58.895416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2cf5f-4a01-40dc-b718-871e0e8e87c2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:58.896209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2cf5f-4a01-40dc-b718-871e0e8e87c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:58.899817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2cf5f-4a01-40dc-b718-871e0e8e87c2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:58.941420 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4064/8095] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:23:58 2022] POST /identity/v3/users => generated 283 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:23:58.941607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d766e6-9dbf-40ee-847e-0f8c81adee51 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:58.944658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b1d766e6-9dbf-40ee-847e-0f8c81adee51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:58.945686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2cf5f-4a01-40dc-b718-871e0e8e87c2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:58.956650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2cf5f-4a01-40dc-b718-871e0e8e87c2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:58.962522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adb427-48ca-403e-8a85-fb561feab59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:58.963124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adb427-48ca-403e-8a85-fb561feab59d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:58.963441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adb427-48ca-403e-8a85-fb561feab59d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:58.963687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adb427-48ca-403e-8a85-fb561feab59d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:58.964303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19adb427-48ca-403e-8a85-fb561feab59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:58.966133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19adb427-48ca-403e-8a85-fb561feab59d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:58.979049 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4065/8096] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:58.984331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adb427-48ca-403e-8a85-fb561feab59d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:58.996686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adb427-48ca-403e-8a85-fb561feab59d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:59.001893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944989a-4ced-4dae-b28e-4f62b34a2f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:59.002621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944989a-4ced-4dae-b28e-4f62b34a2f4a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.002904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944989a-4ced-4dae-b28e-4f62b34a2f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.003174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944989a-4ced-4dae-b28e-4f62b34a2f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.003960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944989a-4ced-4dae-b28e-4f62b34a2f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:59.005922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944989a-4ced-4dae-b28e-4f62b34a2f4a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:59.007765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:59.019392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d766e6-9dbf-40ee-847e-0f8c81adee51 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:59.023124 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4066/8097] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:59.025013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6652a7d-c157-4fb5-ac6c-d986a1ab2d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:59.025318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944989a-4ced-4dae-b28e-4f62b34a2f4a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:59.026074 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6652a7d-c157-4fb5-ac6c-d986a1ab2d6a service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.026346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6652a7d-c157-4fb5-ac6c-d986a1ab2d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.026622 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6652a7d-c157-4fb5-ac6c-d986a1ab2d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.029108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e6652a7d-c157-4fb5-ac6c-d986a1ab2d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:59.040656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944989a-4ced-4dae-b28e-4f62b34a2f4a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:59.044156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9372ec-7865-4d14-89f9-5edb6004a7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:59.044439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6652a7d-c157-4fb5-ac6c-d986a1ab2d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:59.044940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9372ec-7865-4d14-89f9-5edb6004a7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.045258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9372ec-7865-4d14-89f9-5edb6004a7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.045661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9372ec-7865-4d14-89f9-5edb6004a7c0 admin admin] PATH_INFO: `/v3/projects/ea59890f8bec47ca9b9784ffe65566bf/users/d3dd57cb3e7643ef8ccfaf7aa6609bde/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.045919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6652a7d-c157-4fb5-ac6c-d986a1ab2d6a service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:59.057311 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4032/8098] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:23:59.065220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9372ec-7865-4d14-89f9-5edb6004a7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea59890f8bec47ca9b9784ffe65566bf, user_id=d3dd57cb3e7643ef8ccfaf7aa6609bde, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:59.068010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9372ec-7865-4d14-89f9-5edb6004a7c0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:59.084952 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4067/8099] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:59 2022] PUT /identity/v3/projects/ea59890f8bec47ca9b9784ffe65566bf/users/d3dd57cb3e7643ef8ccfaf7aa6609bde/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:59.095446 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0cecb-cce9-43e3-94ee-2f4e4fb45727 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.095748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0cecb-cce9-43e3-94ee-2f4e4fb45727 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.096118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0cecb-cce9-43e3-94ee-2f4e4fb45727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.128268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3da0cecb-cce9-43e3-94ee-2f4e4fb45727 None None] MFA Rules not processed for user `d3dd57cb3e7643ef8ccfaf7aa6609bde`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:59.129660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3da0cecb-cce9-43e3-94ee-2f4e4fb45727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:59.180021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:59.221383 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4033/8100] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:59.223146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9372ec-7865-4d14-89f9-5edb6004a7c0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:59.298591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9372ec-7865-4d14-89f9-5edb6004a7c0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:59.305554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203da27-9018-4297-adea-cd7381bb1b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:59.306458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203da27-9018-4297-adea-cd7381bb1b27 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.306940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203da27-9018-4297-adea-cd7381bb1b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.307772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203da27-9018-4297-adea-cd7381bb1b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.308720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203da27-9018-4297-adea-cd7381bb1b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:59.311002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203da27-9018-4297-adea-cd7381bb1b27 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:59.362801 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4068/8101] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:23:59 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:59.364769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0cecb-cce9-43e3-94ee-2f4e4fb45727 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:59.377566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0cecb-cce9-43e3-94ee-2f4e4fb45727 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:59.486091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e24bd-fcd7-48d6-bd69-6f7cfd8f4c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:59.486708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e24bd-fcd7-48d6-bd69-6f7cfd8f4c82 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.486986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e24bd-fcd7-48d6-bd69-6f7cfd8f4c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.487256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e24bd-fcd7-48d6-bd69-6f7cfd8f4c82 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.488081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e24bd-fcd7-48d6-bd69-6f7cfd8f4c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:59.490858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e24bd-fcd7-48d6-bd69-6f7cfd8f4c82 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:59.521029 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4034/8102] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:23:59 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:23:59.525139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203da27-9018-4297-adea-cd7381bb1b27 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:59.536129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203da27-9018-4297-adea-cd7381bb1b27 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:59.544360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdab06-84cb-434e-8233-26e9d7b3be91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:59.544971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdab06-84cb-434e-8233-26e9d7b3be91 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.545275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdab06-84cb-434e-8233-26e9d7b3be91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.546852 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdab06-84cb-434e-8233-26e9d7b3be91 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.547512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdab06-84cb-434e-8233-26e9d7b3be91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:59.549288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdab06-84cb-434e-8233-26e9d7b3be91 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:59.565099 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4069/8103] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:23:59.566152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e24bd-fcd7-48d6-bd69-6f7cfd8f4c82 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:59.578401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e24bd-fcd7-48d6-bd69-6f7cfd8f4c82 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:59.583305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ceb16-dd67-4da4-a2f1-de2f830ff6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:59.583898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ceb16-dd67-4da4-a2f1-de2f830ff6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.584326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ceb16-dd67-4da4-a2f1-de2f830ff6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.584713 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ceb16-dd67-4da4-a2f1-de2f830ff6e4 admin admin] PATH_INFO: `/v3/projects/0a4405556d7c44f18a8676ed12c6252b/users/1a902e954a6640bc81cc9d52bf9354b8/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.601245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ceb16-dd67-4da4-a2f1-de2f830ff6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4405556d7c44f18a8676ed12c6252b, user_id=1a902e954a6640bc81cc9d52bf9354b8, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:59.604330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ceb16-dd67-4da4-a2f1-de2f830ff6e4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:59.619624 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4035/8104] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:59 2022] PUT /identity/v3/projects/0a4405556d7c44f18a8676ed12c6252b/users/1a902e954a6640bc81cc9d52bf9354b8/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:23:59.623553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdab06-84cb-434e-8233-26e9d7b3be91 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:59.634619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdab06-84cb-434e-8233-26e9d7b3be91 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:59.698905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2e070-28d8-4fce-9d8d-3de5f0e6c58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:59.699477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2e070-28d8-4fce-9d8d-3de5f0e6c58d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.699659 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2e070-28d8-4fce-9d8d-3de5f0e6c58d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.699921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2e070-28d8-4fce-9d8d-3de5f0e6c58d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.700553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2e070-28d8-4fce-9d8d-3de5f0e6c58d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:59.702729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2e070-28d8-4fce-9d8d-3de5f0e6c58d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:59.717800 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4070/8105] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:23:59.719086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ceb16-dd67-4da4-a2f1-de2f830ff6e4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:59.731092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ceb16-dd67-4da4-a2f1-de2f830ff6e4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:59.737355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf82520-85b6-4d6f-a8a3-ef7dc783f86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:59.738221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82520-85b6-4d6f-a8a3-ef7dc783f86a admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.738525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82520-85b6-4d6f-a8a3-ef7dc783f86a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.738943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82520-85b6-4d6f-a8a3-ef7dc783f86a admin admin] PATH_INFO: `/v3/projects/0a4405556d7c44f18a8676ed12c6252b/users/1a902e954a6640bc81cc9d52bf9354b8/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.741151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf82520-85b6-4d6f-a8a3-ef7dc783f86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4405556d7c44f18a8676ed12c6252b, user_id=1a902e954a6640bc81cc9d52bf9354b8, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:23:59.744157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf82520-85b6-4d6f-a8a3-ef7dc783f86a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:23:59.757509 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4036/8106] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:23:59 2022] PUT /identity/v3/projects/0a4405556d7c44f18a8676ed12c6252b/users/1a902e954a6640bc81cc9d52bf9354b8/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:23:59.769207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d36b0-558b-482a-a103-93fd249671a6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.769498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d36b0-558b-482a-a103-93fd249671a6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.769773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d36b0-558b-482a-a103-93fd249671a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:23:59.804081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-210d36b0-558b-482a-a103-93fd249671a6 None None] MFA Rules not processed for user `1a902e954a6640bc81cc9d52bf9354b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:23:59.805419 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-210d36b0-558b-482a-a103-93fd249671a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:59.845409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:59.895523 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4071/8107] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:23:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:23:59.907818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf82520-85b6-4d6f-a8a3-ef7dc783f86a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:23:59.909863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cdf82520-85b6-4d6f-a8a3-ef7dc783f86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:23:59.983475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf82520-85b6-4d6f-a8a3-ef7dc783f86a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:23:59.987212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6f6e7-c33e-43a6-ac19-b07905280b34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:23:59.987970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6f6e7-c33e-43a6-ac19-b07905280b34 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:23:59.988237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6f6e7-c33e-43a6-ac19-b07905280b34 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:23:59.988510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6f6e7-c33e-43a6-ac19-b07905280b34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:00.000485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6f6e7-c33e-43a6-ac19-b07905280b34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:00.002032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6f6e7-c33e-43a6-ac19-b07905280b34 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:00.013285 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4037/8108] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:23:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:00.110205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d36b0-558b-482a-a103-93fd249671a6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:00.125134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d36b0-558b-482a-a103-93fd249671a6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:00.129076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa67778-0bb1-47c2-ac70-0c192850795f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:00.130401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa67778-0bb1-47c2-ac70-0c192850795f service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:00.130779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa67778-0bb1-47c2-ac70-0c192850795f service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:00.131790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa67778-0bb1-47c2-ac70-0c192850795f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:00.209137 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa67778-0bb1-47c2-ac70-0c192850795f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:00.210567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa67778-0bb1-47c2-ac70-0c192850795f service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:00.220663 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4072/8109] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:05.350473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6f6e7-c33e-43a6-ac19-b07905280b34 service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:05.391904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:05.422648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6f6e7-c33e-43a6-ac19-b07905280b34 service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:05.429186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:05.429848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:05.430119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:05.430404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] PATH_INFO: `/v3/users/d3dd57cb3e7643ef8ccfaf7aa6609bde` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:05.431852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3dd57cb3e7643ef8ccfaf7aa6609bde)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:05.433950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:05.492315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3dd57cb3e7643ef8ccfaf7aa6609bde'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:05.509245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3dd57cb3e7643ef8ccfaf7aa6609bde'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:05.519319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3dd57cb3e7643ef8ccfaf7aa6609bde'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:05.528304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3dd57cb3e7643ef8ccfaf7aa6609bde'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:05.528985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3dd57cb3e7643ef8ccfaf7aa6609bde'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:05.534940 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4038/8110] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:24:05 2022] DELETE /identity/v3/users/d3dd57cb3e7643ef8ccfaf7aa6609bde => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:05.539434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd510-3605-4239-b643-b5ddde6a3b23 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:05.539965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd510-3605-4239-b643-b5ddde6a3b23 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:05.539965 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd510-3605-4239-b643-b5ddde6a3b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:05.560329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-dd3bd510-3605-4239-b643-b5ddde6a3b23 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:05.561578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dd3bd510-3605-4239-b643-b5ddde6a3b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:05.626110 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4073/8111] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:05.637480 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:05.638884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:05.701072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa71fe-15a6-452c-978a-a10bcd34b15e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:05.704659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79ca03-c911-4b20-8de5-11fad70d3e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:05.705281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79ca03-c911-4b20-8de5-11fad70d3e55 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:05.705555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79ca03-c911-4b20-8de5-11fad70d3e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:05.705820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79ca03-c911-4b20-8de5-11fad70d3e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:05.714994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79ca03-c911-4b20-8de5-11fad70d3e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:05.716432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79ca03-c911-4b20-8de5-11fad70d3e55 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:05.728465 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4039/8112] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:06.001342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd510-3605-4239-b643-b5ddde6a3b23 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:06.002766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dd3bd510-3605-4239-b643-b5ddde6a3b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:06.018416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd510-3605-4239-b643-b5ddde6a3b23 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:06.024769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:06.025604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:06.025879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:06.026146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] PATH_INFO: `/v3/projects/ea59890f8bec47ca9b9784ffe65566bf` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:06.027372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea59890f8bec47ca9b9784ffe65566bf)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:06.029330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:06.076466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea59890f8bec47ca9b9784ffe65566bf', 'request_id': 'req-6fdd2eb1-e8a4-497f-897e-33163198e25a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.084425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea59890f8bec47ca9b9784ffe65566bf', 'request_id': 'req-6fdd2eb1-e8a4-497f-897e-33163198e25a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.090670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea59890f8bec47ca9b9784ffe65566bf', 'request_id': 'req-6fdd2eb1-e8a4-497f-897e-33163198e25a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.091998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:06.103894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] The token cache is being invalidate because project ea59890f8bec47ca9b9784ffe65566bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:24:06.104179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.104833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.106380 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4074/8113] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:24:06 2022] DELETE /identity/v3/projects/ea59890f8bec47ca9b9784ffe65566bf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:06.110428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79ca03-c911-4b20-8de5-11fad70d3e55 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:06.120471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79ca03-c911-4b20-8de5-11fad70d3e55 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:06.171801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:06.178796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:06.179585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:06.180066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:06.180542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] PATH_INFO: `/v3/users/1a902e954a6640bc81cc9d52bf9354b8` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:06.181856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a902e954a6640bc81cc9d52bf9354b8)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:06.183790 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:06.234944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a902e954a6640bc81cc9d52bf9354b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.247042 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a902e954a6640bc81cc9d52bf9354b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.254737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a902e954a6640bc81cc9d52bf9354b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.262683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a902e954a6640bc81cc9d52bf9354b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.263322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-e2adec69-35bf-44ce-af1c-682ec933a038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a902e954a6640bc81cc9d52bf9354b8'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.269153 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4040/8114] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:24:06 2022] DELETE /identity/v3/users/1a902e954a6640bc81cc9d52bf9354b8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:06.556383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:06.557891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:06.574499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd2eb1-e8a4-497f-897e-33163198e25a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:06.635254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:06.635922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:06.636192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:06.636460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] PATH_INFO: `/v3/projects/0a4405556d7c44f18a8676ed12c6252b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:06.637621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4405556d7c44f18a8676ed12c6252b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:06.639550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:06.687487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4405556d7c44f18a8676ed12c6252b', 'request_id': 'req-98ceab68-5716-44df-a684-1ada7a34b799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.696006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4405556d7c44f18a8676ed12c6252b', 'request_id': 'req-98ceab68-5716-44df-a684-1ada7a34b799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.702584 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4405556d7c44f18a8676ed12c6252b', 'request_id': 'req-98ceab68-5716-44df-a684-1ada7a34b799'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.713968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] The token cache is being invalidate because project 0a4405556d7c44f18a8676ed12c6252b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:24:06.714332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.715028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-98ceab68-5716-44df-a684-1ada7a34b799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:06.716625 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4075/8115] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:24:06 2022] DELETE /identity/v3/projects/0a4405556d7c44f18a8676ed12c6252b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:06.721374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88450d-753c-4893-a27c-adbd218f7ba0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:06.721855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88450d-753c-4893-a27c-adbd218f7ba0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:06.721991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88450d-753c-4893-a27c-adbd218f7ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:06.743208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3f88450d-753c-4893-a27c-adbd218f7ba0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:06.744403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3f88450d-753c-4893-a27c-adbd218f7ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:06.808247 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4041/8116] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:06.815972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8c0fd-3e72-4515-99f6-74a88a8b663c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:06.816284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8c0fd-3e72-4515-99f6-74a88a8b663c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:06.816559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8c0fd-3e72-4515-99f6-74a88a8b663c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:06.837493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ffb8c0fd-3e72-4515-99f6-74a88a8b663c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:06.838709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ffb8c0fd-3e72-4515-99f6-74a88a8b663c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:06.847299 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4076/8117] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:06.852286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88450d-753c-4893-a27c-adbd218f7ba0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:06.863528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88450d-753c-4893-a27c-adbd218f7ba0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:06.868321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19900329-d284-4281-9adf-ea0791331839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:06.868907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19900329-d284-4281-9adf-ea0791331839 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:06.869224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19900329-d284-4281-9adf-ea0791331839 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:06.869850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19900329-d284-4281-9adf-ea0791331839 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:06.870497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19900329-d284-4281-9adf-ea0791331839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:24:06.870798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19900329-d284-4281-9adf-ea0791331839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:06.872660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19900329-d284-4281-9adf-ea0791331839 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:06.884883 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4042/8118] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:24:06.889130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8c0fd-3e72-4515-99f6-74a88a8b663c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:06.899076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8c0fd-3e72-4515-99f6-74a88a8b663c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:06.903791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b266f4-60c0-41b8-8e7a-0b464949ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:06.904371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b266f4-60c0-41b8-8e7a-0b464949ff8f admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:06.904641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b266f4-60c0-41b8-8e7a-0b464949ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:06.904903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b266f4-60c0-41b8-8e7a-0b464949ff8f admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:06.905544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b266f4-60c0-41b8-8e7a-0b464949ff8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:06.907435 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b266f4-60c0-41b8-8e7a-0b464949ff8f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:06.934753 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4077/8119] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:24:06 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:24:06.939331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19900329-d284-4281-9adf-ea0791331839 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:06.951337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19900329-d284-4281-9adf-ea0791331839 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.010181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cb57d-c27b-47dd-b869-a99088cbf75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.010884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cb57d-c27b-47dd-b869-a99088cbf75d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.011227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cb57d-c27b-47dd-b869-a99088cbf75d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.011503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cb57d-c27b-47dd-b869-a99088cbf75d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.012133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cb57d-c27b-47dd-b869-a99088cbf75d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.014100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cb57d-c27b-47dd-b869-a99088cbf75d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.025083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:07.041067 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4043/8120] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:24:06 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:07.043942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b266f4-60c0-41b8-8e7a-0b464949ff8f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:07.054799 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b266f4-60c0-41b8-8e7a-0b464949ff8f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.059594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16819e72-596d-4c95-a915-4b6e9471a2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.060290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16819e72-596d-4c95-a915-4b6e9471a2bc admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.060670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16819e72-596d-4c95-a915-4b6e9471a2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.061183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16819e72-596d-4c95-a915-4b6e9471a2bc admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.061918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16819e72-596d-4c95-a915-4b6e9471a2bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.063634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16819e72-596d-4c95-a915-4b6e9471a2bc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.074017 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4078/8121] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:24:07.078098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cb57d-c27b-47dd-b869-a99088cbf75d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:07.089310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cb57d-c27b-47dd-b869-a99088cbf75d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.094038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995a7bb-81ca-43aa-9bc7-493520c778ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.094621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995a7bb-81ca-43aa-9bc7-493520c778ed admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.094895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995a7bb-81ca-43aa-9bc7-493520c778ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.095163 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995a7bb-81ca-43aa-9bc7-493520c778ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.095793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995a7bb-81ca-43aa-9bc7-493520c778ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.097435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995a7bb-81ca-43aa-9bc7-493520c778ed admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.108534 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4044/8122] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:07.112650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16819e72-596d-4c95-a915-4b6e9471a2bc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:07.122683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16819e72-596d-4c95-a915-4b6e9471a2bc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.127239 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508efbc-e9c6-4b6d-820a-2755b4390dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.127868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508efbc-e9c6-4b6d-820a-2755b4390dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.128139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508efbc-e9c6-4b6d-820a-2755b4390dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.128403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508efbc-e9c6-4b6d-820a-2755b4390dd9 admin admin] PATH_INFO: `/v3/projects/43d1ef8ebe1f472eb24e9c264e27ce5d/users/2d03f395bf5d429492621d270aab90a9/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.143077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508efbc-e9c6-4b6d-820a-2755b4390dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d1ef8ebe1f472eb24e9c264e27ce5d, user_id=2d03f395bf5d429492621d270aab90a9, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.145729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508efbc-e9c6-4b6d-820a-2755b4390dd9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.158928 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4079/8123] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:24:07 2022] PUT /identity/v3/projects/43d1ef8ebe1f472eb24e9c264e27ce5d/users/2d03f395bf5d429492621d270aab90a9/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:07.170494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2274d-7c58-44d7-a3b9-2df34b675eee None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.170772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2274d-7c58-44d7-a3b9-2df34b675eee None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.171040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2274d-7c58-44d7-a3b9-2df34b675eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.200546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bcf2274d-7c58-44d7-a3b9-2df34b675eee None None] MFA Rules not processed for user `2d03f395bf5d429492621d270aab90a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:07.201816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bcf2274d-7c58-44d7-a3b9-2df34b675eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:07.273126 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4045/8124] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:07.274740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508efbc-e9c6-4b6d-820a-2755b4390dd9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:07.295315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:07.347846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508efbc-e9c6-4b6d-820a-2755b4390dd9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.353645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc65c4a-9954-4a2d-b69e-d86a986dc4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.354277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc65c4a-9954-4a2d-b69e-d86a986dc4a5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.354554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc65c4a-9954-4a2d-b69e-d86a986dc4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.354822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc65c4a-9954-4a2d-b69e-d86a986dc4a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.355452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc65c4a-9954-4a2d-b69e-d86a986dc4a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.357629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc65c4a-9954-4a2d-b69e-d86a986dc4a5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.387666 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4080/8125] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:24:07 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:24:07.392084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2274d-7c58-44d7-a3b9-2df34b675eee None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:07.404758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2274d-7c58-44d7-a3b9-2df34b675eee None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.472832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67d338-b04e-4771-9080-a589f357445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.473481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67d338-b04e-4771-9080-a589f357445e admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.473754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67d338-b04e-4771-9080-a589f357445e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.474023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67d338-b04e-4771-9080-a589f357445e admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.475013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67d338-b04e-4771-9080-a589f357445e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.477144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67d338-b04e-4771-9080-a589f357445e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.503148 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4046/8126] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:24:07 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:07.507875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc65c4a-9954-4a2d-b69e-d86a986dc4a5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:07.519279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc65c4a-9954-4a2d-b69e-d86a986dc4a5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.524819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19e3b0-e706-4343-bec8-9a6aaf486758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.525535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19e3b0-e706-4343-bec8-9a6aaf486758 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.525818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19e3b0-e706-4343-bec8-9a6aaf486758 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.526090 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19e3b0-e706-4343-bec8-9a6aaf486758 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.526723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19e3b0-e706-4343-bec8-9a6aaf486758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.528314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19e3b0-e706-4343-bec8-9a6aaf486758 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.540666 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4081/8127] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:07.544810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67d338-b04e-4771-9080-a589f357445e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:07.556606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67d338-b04e-4771-9080-a589f357445e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.561376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c995dd-4a62-40f4-95e3-81ee3c454f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.562025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c995dd-4a62-40f4-95e3-81ee3c454f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.562300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c995dd-4a62-40f4-95e3-81ee3c454f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.562565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c995dd-4a62-40f4-95e3-81ee3c454f96 admin admin] PATH_INFO: `/v3/projects/8f4a0310145d47aeb8b4b0047c841a43/users/45c767d0256d43b38df82a228120843c/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.578527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c995dd-4a62-40f4-95e3-81ee3c454f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f4a0310145d47aeb8b4b0047c841a43, user_id=45c767d0256d43b38df82a228120843c, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.581226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c995dd-4a62-40f4-95e3-81ee3c454f96 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.596017 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4047/8128] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:24:07 2022] PUT /identity/v3/projects/8f4a0310145d47aeb8b4b0047c841a43/users/45c767d0256d43b38df82a228120843c/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:24:07.600032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19e3b0-e706-4343-bec8-9a6aaf486758 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:07.610600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19e3b0-e706-4343-bec8-9a6aaf486758 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.667600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfa21c-49c0-44b6-b257-b9a978adb4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.668193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa21c-49c0-44b6-b257-b9a978adb4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.668465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa21c-49c0-44b6-b257-b9a978adb4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.668728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa21c-49c0-44b6-b257-b9a978adb4f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.669380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfa21c-49c0-44b6-b257-b9a978adb4f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.670982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfa21c-49c0-44b6-b257-b9a978adb4f3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.681205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4082/8129] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:07.685356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c995dd-4a62-40f4-95e3-81ee3c454f96 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:07.696354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c995dd-4a62-40f4-95e3-81ee3c454f96 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.700923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809d02f-21fb-4287-bea4-580dc49e1562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.701585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d02f-21fb-4287-bea4-580dc49e1562 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.701854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d02f-21fb-4287-bea4-580dc49e1562 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.702117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d02f-21fb-4287-bea4-580dc49e1562 admin admin] PATH_INFO: `/v3/projects/8f4a0310145d47aeb8b4b0047c841a43/users/45c767d0256d43b38df82a228120843c/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.703988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809d02f-21fb-4287-bea4-580dc49e1562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f4a0310145d47aeb8b4b0047c841a43, user_id=45c767d0256d43b38df82a228120843c, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.706643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809d02f-21fb-4287-bea4-580dc49e1562 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.716726 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4048/8130] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:24:07 2022] PUT /identity/v3/projects/8f4a0310145d47aeb8b4b0047c841a43/users/45c767d0256d43b38df82a228120843c/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:07.730448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8a046-6102-4160-8668-dda49f3cf77d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.730882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8a046-6102-4160-8668-dda49f3cf77d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.731158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8a046-6102-4160-8668-dda49f3cf77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.760105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-91e8a046-6102-4160-8668-dda49f3cf77d None None] MFA Rules not processed for user `45c767d0256d43b38df82a228120843c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:07.761399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-91e8a046-6102-4160-8668-dda49f3cf77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:07.830503 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4083/8131] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:07.841760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809d02f-21fb-4287-bea4-580dc49e1562 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:07.843646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2809d02f-21fb-4287-bea4-580dc49e1562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:07.877463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:07.913144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809d02f-21fb-4287-bea4-580dc49e1562 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:07.916860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f523e-c4c5-440f-b497-e5ef80eff3b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:07.917497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f523e-c4c5-440f-b497-e5ef80eff3b8 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:07.917770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f523e-c4c5-440f-b497-e5ef80eff3b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:07.918037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f523e-c4c5-440f-b497-e5ef80eff3b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:07.977473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f523e-c4c5-440f-b497-e5ef80eff3b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:07.979289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f523e-c4c5-440f-b497-e5ef80eff3b8 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:07.988221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4049/8132] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:10.574030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8a046-6102-4160-8668-dda49f3cf77d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:10.589749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:10.644109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8a046-6102-4160-8668-dda49f3cf77d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:10.649003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:10.649692 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:10.649972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:10.650248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] PATH_INFO: `/v3/users/2d03f395bf5d429492621d270aab90a9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:10.651395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d03f395bf5d429492621d270aab90a9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:10.653450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:10.703314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d03f395bf5d429492621d270aab90a9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:10.710691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d03f395bf5d429492621d270aab90a9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:10.723078 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d03f395bf5d429492621d270aab90a9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:10.732377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d03f395bf5d429492621d270aab90a9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:10.733461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d03f395bf5d429492621d270aab90a9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:10.740256 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4084/8133] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:24:10 2022] DELETE /identity/v3/users/2d03f395bf5d429492621d270aab90a9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:10.745092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bbd76-99e3-42a1-8c5f-4eac59cca587 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:10.745545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bbd76-99e3-42a1-8c5f-4eac59cca587 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:10.745625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bbd76-99e3-42a1-8c5f-4eac59cca587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:10.765967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-633bbd76-99e3-42a1-8c5f-4eac59cca587 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:10.767169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-633bbd76-99e3-42a1-8c5f-4eac59cca587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:10.829458 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4050/8134] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:24:10.839421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:10.840995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:10.902985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c76a1-196d-4935-a76f-42125188787b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:10.906858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eb3e9-6264-42b0-ac4c-f9fda96e6a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:10.907462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb3e9-6264-42b0-ac4c-f9fda96e6a4d service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:10.907731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb3e9-6264-42b0-ac4c-f9fda96e6a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:10.907994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb3e9-6264-42b0-ac4c-f9fda96e6a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:10.918118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eb3e9-6264-42b0-ac4c-f9fda96e6a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:10.919529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eb3e9-6264-42b0-ac4c-f9fda96e6a4d service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:10.929801 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4085/8135] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:11.694894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bbd76-99e3-42a1-8c5f-4eac59cca587 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:11.696383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-633bbd76-99e3-42a1-8c5f-4eac59cca587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:11.708340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bbd76-99e3-42a1-8c5f-4eac59cca587 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:11.720906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:11.721774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:11.722648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:11.723759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] PATH_INFO: `/v3/projects/43d1ef8ebe1f472eb24e9c264e27ce5d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:11.725865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d1ef8ebe1f472eb24e9c264e27ce5d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:11.729599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:11.817356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d1ef8ebe1f472eb24e9c264e27ce5d', 'request_id': 'req-5118bf78-324b-466d-a565-8f94cde11453'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:11.838442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d1ef8ebe1f472eb24e9c264e27ce5d', 'request_id': 'req-5118bf78-324b-466d-a565-8f94cde11453'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:11.847109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d1ef8ebe1f472eb24e9c264e27ce5d', 'request_id': 'req-5118bf78-324b-466d-a565-8f94cde11453'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:11.862060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] The token cache is being invalidate because project 43d1ef8ebe1f472eb24e9c264e27ce5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:24:11.862647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:11.863556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:11.866093 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4051/8136] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:24:11 2022] DELETE /identity/v3/projects/43d1ef8ebe1f472eb24e9c264e27ce5d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:11.871067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eb3e9-6264-42b0-ac4c-f9fda96e6a4d service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:11.883181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eb3e9-6264-42b0-ac4c-f9fda96e6a4d service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:11.919109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:11.957333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:11.957976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:11.958313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:11.958636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] PATH_INFO: `/v3/users/45c767d0256d43b38df82a228120843c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:11.959767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c767d0256d43b38df82a228120843c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:11.961781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:12.018434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c767d0256d43b38df82a228120843c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:12.026722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c767d0256d43b38df82a228120843c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:12.042809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c767d0256d43b38df82a228120843c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:12.051346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c767d0256d43b38df82a228120843c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:12.052456 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-9ebf8ebd-fa60-49f8-9080-b526619ef0ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c767d0256d43b38df82a228120843c'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:12.058866 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4086/8137] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:24:11 2022] DELETE /identity/v3/users/45c767d0256d43b38df82a228120843c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:24:12.606225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:12.607588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:12.616009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:12.625887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118bf78-324b-466d-a565-8f94cde11453 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:12.684769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:12.685412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:12.685686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:12.685950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] PATH_INFO: `/v3/projects/8f4a0310145d47aeb8b4b0047c841a43` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:12.687036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f4a0310145d47aeb8b4b0047c841a43)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:12.688974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:12.735306 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f4a0310145d47aeb8b4b0047c841a43', 'request_id': 'req-b1696e88-4d89-4e61-b5f3-d493371951ad'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:12.743351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f4a0310145d47aeb8b4b0047c841a43', 'request_id': 'req-b1696e88-4d89-4e61-b5f3-d493371951ad'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:12.749776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f4a0310145d47aeb8b4b0047c841a43', 'request_id': 'req-b1696e88-4d89-4e61-b5f3-d493371951ad'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:12.760657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] The token cache is being invalidate because project 8f4a0310145d47aeb8b4b0047c841a43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:24:12.760949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:12.761601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b1696e88-4d89-4e61-b5f3-d493371951ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:12.763082 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4052/8138] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:24:12 2022] DELETE /identity/v3/projects/8f4a0310145d47aeb8b4b0047c841a43 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:12.767913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8918a-132c-4212-815e-b3ced2ea3021 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:12.768246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8918a-132c-4212-815e-b3ced2ea3021 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:12.768522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8918a-132c-4212-815e-b3ced2ea3021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:12.788789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e5d8918a-132c-4212-815e-b3ced2ea3021 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:12.789999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e5d8918a-132c-4212-815e-b3ced2ea3021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:12.849330 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4087/8139] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:12.857412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9195d2-ea57-43f9-bbbb-670cdc733b11 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:12.857881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9195d2-ea57-43f9-bbbb-670cdc733b11 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:12.857960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9195d2-ea57-43f9-bbbb-670cdc733b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:12.878326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ad9195d2-ea57-43f9-bbbb-670cdc733b11 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:12.879518 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ad9195d2-ea57-43f9-bbbb-670cdc733b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:12.888184 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4053/8140] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:12.892313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8918a-132c-4212-815e-b3ced2ea3021 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:12.902500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8918a-132c-4212-815e-b3ced2ea3021 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:12.907093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fd910-6583-4d23-afa0-6c7e10474ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:12.907695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fd910-6583-4d23-afa0-6c7e10474ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:12.907968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fd910-6583-4d23-afa0-6c7e10474ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:12.908231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fd910-6583-4d23-afa0-6c7e10474ca6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:12.908868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fd910-6583-4d23-afa0-6c7e10474ca6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:24:12.909244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fd910-6583-4d23-afa0-6c7e10474ca6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:12.910839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fd910-6583-4d23-afa0-6c7e10474ca6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:12.922942 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4088/8141] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:24:12.928123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9195d2-ea57-43f9-bbbb-670cdc733b11 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:12.937849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9195d2-ea57-43f9-bbbb-670cdc733b11 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:12.942587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85820dc3-f647-443f-9304-804044ade999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:12.943980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85820dc3-f647-443f-9304-804044ade999 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:12.944305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85820dc3-f647-443f-9304-804044ade999 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:12.944574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85820dc3-f647-443f-9304-804044ade999 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:12.945238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85820dc3-f647-443f-9304-804044ade999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:12.947181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85820dc3-f647-443f-9304-804044ade999 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:12.981352 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4054/8142] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:24:12 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:12.985491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fd910-6583-4d23-afa0-6c7e10474ca6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:12.995979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fd910-6583-4d23-afa0-6c7e10474ca6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.035929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:13.057418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9710411-38c4-4da6-bbe8-6d4f463facbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.058017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9710411-38c4-4da6-bbe8-6d4f463facbf admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.058310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9710411-38c4-4da6-bbe8-6d4f463facbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.058577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9710411-38c4-4da6-bbe8-6d4f463facbf admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.059188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9710411-38c4-4da6-bbe8-6d4f463facbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.061105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9710411-38c4-4da6-bbe8-6d4f463facbf admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.082760 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4089/8143] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:24:12 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:13.086868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85820dc3-f647-443f-9304-804044ade999 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.097066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85820dc3-f647-443f-9304-804044ade999 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.102439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f4201-f7fa-474f-9e30-57968297291d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.103228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f4201-f7fa-474f-9e30-57968297291d admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.103500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f4201-f7fa-474f-9e30-57968297291d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.103787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f4201-f7fa-474f-9e30-57968297291d admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.104415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f4201-f7fa-474f-9e30-57968297291d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.106037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f4201-f7fa-474f-9e30-57968297291d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.116656 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4055/8144] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:13.120502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9710411-38c4-4da6-bbe8-6d4f463facbf admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.130758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9710411-38c4-4da6-bbe8-6d4f463facbf admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.135747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dad519-d607-4744-88c5-ce336a26a490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.136335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dad519-d607-4744-88c5-ce336a26a490 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.136611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dad519-d607-4744-88c5-ce336a26a490 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.136882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dad519-d607-4744-88c5-ce336a26a490 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.137547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dad519-d607-4744-88c5-ce336a26a490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.139122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dad519-d607-4744-88c5-ce336a26a490 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.149841 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4090/8145] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:13.154889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f4201-f7fa-474f-9e30-57968297291d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.165820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f4201-f7fa-474f-9e30-57968297291d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.170374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ac78d-5c5f-4cd7-828a-475f52d01962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.170997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ac78d-5c5f-4cd7-828a-475f52d01962 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.171267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ac78d-5c5f-4cd7-828a-475f52d01962 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.171536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ac78d-5c5f-4cd7-828a-475f52d01962 admin admin] PATH_INFO: `/v3/projects/8bdeccacc3f54c70bf0a6276553b5463/users/70f10eefc38e451e89d9eb73d6e5b331/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.186700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ac78d-5c5f-4cd7-828a-475f52d01962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdeccacc3f54c70bf0a6276553b5463, user_id=70f10eefc38e451e89d9eb73d6e5b331, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.189521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ac78d-5c5f-4cd7-828a-475f52d01962 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.208325 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4056/8146] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:24:13 2022] PUT /identity/v3/projects/8bdeccacc3f54c70bf0a6276553b5463/users/70f10eefc38e451e89d9eb73d6e5b331/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:13.219352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db49fdb-85fe-4875-b96d-0d5c18514a8e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.219631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db49fdb-85fe-4875-b96d-0d5c18514a8e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.219896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db49fdb-85fe-4875-b96d-0d5c18514a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.248809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-5db49fdb-85fe-4875-b96d-0d5c18514a8e None None] MFA Rules not processed for user `70f10eefc38e451e89d9eb73d6e5b331`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:13.250012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5db49fdb-85fe-4875-b96d-0d5c18514a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:13.313357 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4091/8147] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:13.317901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ac78d-5c5f-4cd7-828a-475f52d01962 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.347854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:13.381218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ac78d-5c5f-4cd7-828a-475f52d01962 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.385868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe516af-cda6-410f-ba67-2e9eead9bf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.386477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe516af-cda6-410f-ba67-2e9eead9bf67 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.386756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe516af-cda6-410f-ba67-2e9eead9bf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.387027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe516af-cda6-410f-ba67-2e9eead9bf67 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.387707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe516af-cda6-410f-ba67-2e9eead9bf67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.389735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe516af-cda6-410f-ba67-2e9eead9bf67 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.416504 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4057/8148] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:24:13 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:13.420533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db49fdb-85fe-4875-b96d-0d5c18514a8e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.431986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db49fdb-85fe-4875-b96d-0d5c18514a8e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.489278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87994ab1-ad19-4e9e-8b5e-30745b06279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.490523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87994ab1-ad19-4e9e-8b5e-30745b06279a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.490803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87994ab1-ad19-4e9e-8b5e-30745b06279a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.491068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87994ab1-ad19-4e9e-8b5e-30745b06279a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.492355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87994ab1-ad19-4e9e-8b5e-30745b06279a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.494474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87994ab1-ad19-4e9e-8b5e-30745b06279a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.518860 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4092/8149] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:24:13 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:13.520651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe516af-cda6-410f-ba67-2e9eead9bf67 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.531434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe516af-cda6-410f-ba67-2e9eead9bf67 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.536326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecac5a1-deaf-484e-b39f-2109f2f979d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.537547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecac5a1-deaf-484e-b39f-2109f2f979d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.537817 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecac5a1-deaf-484e-b39f-2109f2f979d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.538079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecac5a1-deaf-484e-b39f-2109f2f979d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.538709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecac5a1-deaf-484e-b39f-2109f2f979d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.540297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecac5a1-deaf-484e-b39f-2109f2f979d5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.554846 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4058/8150] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:13.556010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87994ab1-ad19-4e9e-8b5e-30745b06279a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.566188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87994ab1-ad19-4e9e-8b5e-30745b06279a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.571103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccbd4f-342a-4917-ab96-13ec98ccc379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.571704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccbd4f-342a-4917-ab96-13ec98ccc379 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.571979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccbd4f-342a-4917-ab96-13ec98ccc379 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.572247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccbd4f-342a-4917-ab96-13ec98ccc379 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.572886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccbd4f-342a-4917-ab96-13ec98ccc379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.574567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccbd4f-342a-4917-ab96-13ec98ccc379 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.585284 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4093/8151] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:13.589253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecac5a1-deaf-484e-b39f-2109f2f979d5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.599780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecac5a1-deaf-484e-b39f-2109f2f979d5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.604702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa837cc5-d66a-4afc-916f-b94ce21d7771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.605382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa837cc5-d66a-4afc-916f-b94ce21d7771 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.605656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa837cc5-d66a-4afc-916f-b94ce21d7771 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.605926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa837cc5-d66a-4afc-916f-b94ce21d7771 admin admin] PATH_INFO: `/v3/projects/68cce36bb57441eda2368a9931405ecd/users/4ab039085219424b9da0a7956a4347a7/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.620411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa837cc5-d66a-4afc-916f-b94ce21d7771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68cce36bb57441eda2368a9931405ecd, user_id=4ab039085219424b9da0a7956a4347a7, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.623081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa837cc5-d66a-4afc-916f-b94ce21d7771 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.635331 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4059/8152] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:24:13 2022] PUT /identity/v3/projects/68cce36bb57441eda2368a9931405ecd/users/4ab039085219424b9da0a7956a4347a7/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:13.646303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2d4a9-54e6-48f8-bc42-7d0dd0bdd031 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.646672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2d4a9-54e6-48f8-bc42-7d0dd0bdd031 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.646949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2d4a9-54e6-48f8-bc42-7d0dd0bdd031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.675517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-11a2d4a9-54e6-48f8-bc42-7d0dd0bdd031 None None] MFA Rules not processed for user `4ab039085219424b9da0a7956a4347a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:13.676765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-11a2d4a9-54e6-48f8-bc42-7d0dd0bdd031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:13.690805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:13.747347 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4094/8153] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:13.751666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa837cc5-d66a-4afc-916f-b94ce21d7771 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.818546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa837cc5-d66a-4afc-916f-b94ce21d7771 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.823172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b411892-efce-4dca-ab1b-b29d9a3145c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.823758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b411892-efce-4dca-ab1b-b29d9a3145c8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.824032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b411892-efce-4dca-ab1b-b29d9a3145c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.824430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b411892-efce-4dca-ab1b-b29d9a3145c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.825291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b411892-efce-4dca-ab1b-b29d9a3145c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.827186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b411892-efce-4dca-ab1b-b29d9a3145c8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.854161 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4060/8154] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:24:13 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:24:13.858492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2d4a9-54e6-48f8-bc42-7d0dd0bdd031 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.869432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2d4a9-54e6-48f8-bc42-7d0dd0bdd031 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.926803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c439a1-c59d-47a0-9a46-31d139cad7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.927399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c439a1-c59d-47a0-9a46-31d139cad7cb admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.927596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c439a1-c59d-47a0-9a46-31d139cad7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.927866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c439a1-c59d-47a0-9a46-31d139cad7cb admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.928497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c439a1-c59d-47a0-9a46-31d139cad7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.930434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c439a1-c59d-47a0-9a46-31d139cad7cb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.954201 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4095/8155] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:24:13 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:13.955442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b411892-efce-4dca-ab1b-b29d9a3145c8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:13.966726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b411892-efce-4dca-ab1b-b29d9a3145c8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:13.971739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9d085-cac1-4f5c-a5df-29c978b666ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:13.972339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9d085-cac1-4f5c-a5df-29c978b666ae admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:13.972751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9d085-cac1-4f5c-a5df-29c978b666ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:13.973028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9d085-cac1-4f5c-a5df-29c978b666ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:13.973695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9d085-cac1-4f5c-a5df-29c978b666ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:13.975962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9d085-cac1-4f5c-a5df-29c978b666ae admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:13.988261 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4061/8156] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:13.993120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c439a1-c59d-47a0-9a46-31d139cad7cb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:14.003117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c439a1-c59d-47a0-9a46-31d139cad7cb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:14.007945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b6d0a-2465-40f6-bb8e-432e9d4122cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:14.008585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b6d0a-2465-40f6-bb8e-432e9d4122cd admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:14.008855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b6d0a-2465-40f6-bb8e-432e9d4122cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:14.009179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b6d0a-2465-40f6-bb8e-432e9d4122cd admin admin] PATH_INFO: `/v3/projects/3f19b0ba82784090aecf6411e5c27288/users/85641bc8a2874736b2af9486d99341b5/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:14.023942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b6d0a-2465-40f6-bb8e-432e9d4122cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f19b0ba82784090aecf6411e5c27288, user_id=85641bc8a2874736b2af9486d99341b5, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:14.026589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b6d0a-2465-40f6-bb8e-432e9d4122cd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:14.043445 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4096/8157] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:24:13 2022] PUT /identity/v3/projects/3f19b0ba82784090aecf6411e5c27288/users/85641bc8a2874736b2af9486d99341b5/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:14.047345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9d085-cac1-4f5c-a5df-29c978b666ae admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:14.058367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9d085-cac1-4f5c-a5df-29c978b666ae admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:14.105999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:14.116338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a644277-f526-429a-b3da-965487866a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:14.116923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a644277-f526-429a-b3da-965487866a98 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:14.117231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a644277-f526-429a-b3da-965487866a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:14.117498 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a644277-f526-429a-b3da-965487866a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:14.118121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a644277-f526-429a-b3da-965487866a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:14.119707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a644277-f526-429a-b3da-965487866a98 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:14.129374 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4062/8158] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:14.134095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b6d0a-2465-40f6-bb8e-432e9d4122cd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:14.144166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b6d0a-2465-40f6-bb8e-432e9d4122cd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:14.148875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728519b1-bc56-4778-a324-146df3fc7f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:14.149551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728519b1-bc56-4778-a324-146df3fc7f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:14.149823 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728519b1-bc56-4778-a324-146df3fc7f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:14.150086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728519b1-bc56-4778-a324-146df3fc7f87 admin admin] PATH_INFO: `/v3/projects/3f19b0ba82784090aecf6411e5c27288/users/85641bc8a2874736b2af9486d99341b5/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:14.151968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728519b1-bc56-4778-a324-146df3fc7f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f19b0ba82784090aecf6411e5c27288, user_id=85641bc8a2874736b2af9486d99341b5, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:14.154644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728519b1-bc56-4778-a324-146df3fc7f87 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:14.165151 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4097/8159] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:24:14 2022] PUT /identity/v3/projects/3f19b0ba82784090aecf6411e5c27288/users/85641bc8a2874736b2af9486d99341b5/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:14.176230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76e8a7-cd72-4b10-b13f-c7e9daf9858f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:14.176511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76e8a7-cd72-4b10-b13f-c7e9daf9858f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:14.176780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76e8a7-cd72-4b10-b13f-c7e9daf9858f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:14.206568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ce76e8a7-cd72-4b10-b13f-c7e9daf9858f None None] MFA Rules not processed for user `85641bc8a2874736b2af9486d99341b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:14.207768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ce76e8a7-cd72-4b10-b13f-c7e9daf9858f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:14.275757 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4063/8160] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:14.287920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728519b1-bc56-4778-a324-146df3fc7f87 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:14.289674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-728519b1-bc56-4778-a324-146df3fc7f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:14.350660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728519b1-bc56-4778-a324-146df3fc7f87 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:14.354572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e3c09-c81f-4476-b844-2855faba58fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:14.355171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e3c09-c81f-4476-b844-2855faba58fc service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:14.355444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e3c09-c81f-4476-b844-2855faba58fc service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:14.355709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e3c09-c81f-4476-b844-2855faba58fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:14.378446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:14.419868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e3c09-c81f-4476-b844-2855faba58fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:14.421848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e3c09-c81f-4476-b844-2855faba58fc service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:14.432095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4098/8161] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:15.941671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76e8a7-cd72-4b10-b13f-c7e9daf9858f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:15.943121 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ce76e8a7-cd72-4b10-b13f-c7e9daf9858f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:15.957280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76e8a7-cd72-4b10-b13f-c7e9daf9858f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:15.961410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1f89b-41a6-46f4-b93a-72a6b68abfee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:15.962016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1f89b-41a6-46f4-b93a-72a6b68abfee service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:15.962284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1f89b-41a6-46f4-b93a-72a6b68abfee service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:15.962547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1f89b-41a6-46f4-b93a-72a6b68abfee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:16.017965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1f89b-41a6-46f4-b93a-72a6b68abfee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:16.019364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1f89b-41a6-46f4-b93a-72a6b68abfee service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:16.028750 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4064/8162] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:16.192979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e3c09-c81f-4476-b844-2855faba58fc service cinder] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:16.195788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bd6e3c09-c81f-4476-b844-2855faba58fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:16.268712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e3c09-c81f-4476-b844-2855faba58fc service cinder] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:16.273166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5de3b4-e7ee-407a-96fa-c6b1b30894ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:16.273773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5de3b4-e7ee-407a-96fa-c6b1b30894ca service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:16.274046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5de3b4-e7ee-407a-96fa-c6b1b30894ca service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:16.274369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5de3b4-e7ee-407a-96fa-c6b1b30894ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:16.276392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-5d5de3b4-e7ee-407a-96fa-c6b1b30894ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:16.290164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5de3b4-e7ee-407a-96fa-c6b1b30894ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:16.291600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5de3b4-e7ee-407a-96fa-c6b1b30894ca service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:16.309264 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4099/8163] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:16.580860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1f89b-41a6-46f4-b93a-72a6b68abfee service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:16.593642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1f89b-41a6-46f4-b93a-72a6b68abfee service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:16.597579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349c81b-2be3-4401-9994-f562956c723c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:16.598311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349c81b-2be3-4401-9994-f562956c723c service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:16.598611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349c81b-2be3-4401-9994-f562956c723c service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:16.599559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349c81b-2be3-4401-9994-f562956c723c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:16.609679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349c81b-2be3-4401-9994-f562956c723c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:16.611102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349c81b-2be3-4401-9994-f562956c723c service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:16.621201 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4065/8164] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:20.506432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5de3b4-e7ee-407a-96fa-c6b1b30894ca service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:20.569482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5de3b4-e7ee-407a-96fa-c6b1b30894ca service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:20.574138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:20.574763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:20.575038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:20.575336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] PATH_INFO: `/v3/users/70f10eefc38e451e89d9eb73d6e5b331` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:20.576518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f10eefc38e451e89d9eb73d6e5b331)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:20.578465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:20.626228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f10eefc38e451e89d9eb73d6e5b331'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:20.633497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f10eefc38e451e89d9eb73d6e5b331'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:20.641234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:20.646854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f10eefc38e451e89d9eb73d6e5b331'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:20.654426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f10eefc38e451e89d9eb73d6e5b331'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:20.655036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f10eefc38e451e89d9eb73d6e5b331'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:20.661064 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4100/8165] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:24:20 2022] DELETE /identity/v3/users/70f10eefc38e451e89d9eb73d6e5b331 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:20.666181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df116e06-79eb-461a-89c4-2c87c032f3d1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:20.666661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df116e06-79eb-461a-89c4-2c87c032f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:20.666748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df116e06-79eb-461a-89c4-2c87c032f3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:20.685839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-df116e06-79eb-461a-89c4-2c87c032f3d1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:20.687012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-df116e06-79eb-461a-89c4-2c87c032f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:20.738735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:20.747943 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4066/8166] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:20.757650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:20.759069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:20.820038 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a330f-baf6-4a05-b292-5fdb0977768c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:20.823939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cabbf-697b-47ed-ac84-0a9eda134630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:20.824534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cabbf-697b-47ed-ac84-0a9eda134630 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:20.824810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cabbf-697b-47ed-ac84-0a9eda134630 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:20.825181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cabbf-697b-47ed-ac84-0a9eda134630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:20.835103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cabbf-697b-47ed-ac84-0a9eda134630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:20.836503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cabbf-697b-47ed-ac84-0a9eda134630 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:20.846491 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4101/8167] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:21.122135 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df116e06-79eb-461a-89c4-2c87c032f3d1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:21.123572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-df116e06-79eb-461a-89c4-2c87c032f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:21.135936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df116e06-79eb-461a-89c4-2c87c032f3d1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:21.140858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:21.141629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:21.141901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:21.142138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] PATH_INFO: `/v3/projects/8bdeccacc3f54c70bf0a6276553b5463` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:21.143421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bdeccacc3f54c70bf0a6276553b5463)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:21.145401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:21.193463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bdeccacc3f54c70bf0a6276553b5463', 'request_id': 'req-670c376e-5017-4e4d-85ee-435a2790b3f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.201591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bdeccacc3f54c70bf0a6276553b5463', 'request_id': 'req-670c376e-5017-4e4d-85ee-435a2790b3f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.208366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bdeccacc3f54c70bf0a6276553b5463', 'request_id': 'req-670c376e-5017-4e4d-85ee-435a2790b3f0'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.218809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] The token cache is being invalidate because project 8bdeccacc3f54c70bf0a6276553b5463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:24:21.219089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.219722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.221536 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4067/8168] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:24:21 2022] DELETE /identity/v3/projects/8bdeccacc3f54c70bf0a6276553b5463 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:21.226052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cabbf-697b-47ed-ac84-0a9eda134630 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:21.236074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cabbf-697b-47ed-ac84-0a9eda134630 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:21.293832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:21.294392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:21.294647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:21.294913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] PATH_INFO: `/v3/users/4ab039085219424b9da0a7956a4347a7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:21.296012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab039085219424b9da0a7956a4347a7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:21.297947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:21.344453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab039085219424b9da0a7956a4347a7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.351732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab039085219424b9da0a7956a4347a7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.365541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab039085219424b9da0a7956a4347a7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.372990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab039085219424b9da0a7956a4347a7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.373688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab039085219424b9da0a7956a4347a7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.379689 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4102/8169] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:24:21 2022] DELETE /identity/v3/users/4ab039085219424b9da0a7956a4347a7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:24:21.659362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:21.660849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:21.674706 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c376e-5017-4e4d-85ee-435a2790b3f0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:21.733242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:21.733870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:21.734144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:21.734411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] PATH_INFO: `/v3/projects/68cce36bb57441eda2368a9931405ecd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:21.735534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68cce36bb57441eda2368a9931405ecd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:21.737479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:21.785444 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68cce36bb57441eda2368a9931405ecd', 'request_id': 'req-2d11a080-2fa2-4b37-b390-067f63df6c40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.793629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68cce36bb57441eda2368a9931405ecd', 'request_id': 'req-2d11a080-2fa2-4b37-b390-067f63df6c40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.799949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68cce36bb57441eda2368a9931405ecd', 'request_id': 'req-2d11a080-2fa2-4b37-b390-067f63df6c40'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.811130 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] The token cache is being invalidate because project 68cce36bb57441eda2368a9931405ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:24:21.811507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.812478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.814023 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4068/8170] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:24:21 2022] DELETE /identity/v3/projects/68cce36bb57441eda2368a9931405ecd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:21.818117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:21.819548 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:21.833014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0455937-27b9-4ad1-a107-412c86fc3191 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:21.836541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:21.893131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:21.893760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:21.894033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:21.894298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] PATH_INFO: `/v3/users/85641bc8a2874736b2af9486d99341b5` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:21.895427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85641bc8a2874736b2af9486d99341b5)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:21.897426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:21.943021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85641bc8a2874736b2af9486d99341b5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.950032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85641bc8a2874736b2af9486d99341b5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.962658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85641bc8a2874736b2af9486d99341b5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.969299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85641bc8a2874736b2af9486d99341b5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.969980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-636a51ff-eb11-46f1-b225-c2962374101f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85641bc8a2874736b2af9486d99341b5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:21.976959 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4103/8171] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:24:21 2022] DELETE /identity/v3/users/85641bc8a2874736b2af9486d99341b5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:22.250864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:22.252449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:22.270673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11a080-2fa2-4b37-b390-067f63df6c40 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:22.281501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:22.332435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:22.333085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:22.333362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:22.334174 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] PATH_INFO: `/v3/projects/3f19b0ba82784090aecf6411e5c27288` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:22.335330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f19b0ba82784090aecf6411e5c27288)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:22.337281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:22.384769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f19b0ba82784090aecf6411e5c27288', 'request_id': 'req-9aba00ce-9bbb-4616-be37-03a155df741d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:22.391919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f19b0ba82784090aecf6411e5c27288', 'request_id': 'req-9aba00ce-9bbb-4616-be37-03a155df741d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:22.398785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f19b0ba82784090aecf6411e5c27288', 'request_id': 'req-9aba00ce-9bbb-4616-be37-03a155df741d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:22.408937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] The token cache is being invalidate because project 3f19b0ba82784090aecf6411e5c27288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:24:22.409248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:22.409888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-9aba00ce-9bbb-4616-be37-03a155df741d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:22.411360 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4069/8172] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:24:22 2022] DELETE /identity/v3/projects/3f19b0ba82784090aecf6411e5c27288 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:22.416926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3f567-96f1-416e-a550-00eb6ae33c3a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:22.417270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3f567-96f1-416e-a550-00eb6ae33c3a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:22.417545 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3f567-96f1-416e-a550-00eb6ae33c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:22.437753 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b2d3f567-96f1-416e-a550-00eb6ae33c3a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:22.438942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b2d3f567-96f1-416e-a550-00eb6ae33c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:22.498262 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4104/8173] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:22.506690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f5706-4e90-4661-b709-91a4259345ae None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:22.506966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f5706-4e90-4661-b709-91a4259345ae None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:22.507235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f5706-4e90-4661-b709-91a4259345ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:22.527469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d04f5706-4e90-4661-b709-91a4259345ae None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:22.528665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d04f5706-4e90-4661-b709-91a4259345ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:22.536852 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4070/8174] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:22.541135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3f567-96f1-416e-a550-00eb6ae33c3a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:22.550916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3f567-96f1-416e-a550-00eb6ae33c3a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:22.555720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a5449-9cf5-437e-8b58-2bdd542f0f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:22.556309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a5449-9cf5-437e-8b58-2bdd542f0f11 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:22.556585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a5449-9cf5-437e-8b58-2bdd542f0f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:22.556858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a5449-9cf5-437e-8b58-2bdd542f0f11 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:22.557524 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a5449-9cf5-437e-8b58-2bdd542f0f11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:24:22.557826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a5449-9cf5-437e-8b58-2bdd542f0f11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:22.559404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a5449-9cf5-437e-8b58-2bdd542f0f11 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:22.571482 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4105/8175] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:24:22.575449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f5706-4e90-4661-b709-91a4259345ae None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:22.585387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f5706-4e90-4661-b709-91a4259345ae None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:22.590035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf372924-54e7-4daa-8362-f33853bb40b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:22.590616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf372924-54e7-4daa-8362-f33853bb40b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:22.590892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf372924-54e7-4daa-8362-f33853bb40b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:22.591161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf372924-54e7-4daa-8362-f33853bb40b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:22.591789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf372924-54e7-4daa-8362-f33853bb40b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:22.593747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf372924-54e7-4daa-8362-f33853bb40b0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:22.621376 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4071/8176] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:24:22 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:22.625348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a5449-9cf5-437e-8b58-2bdd542f0f11 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:22.636192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a5449-9cf5-437e-8b58-2bdd542f0f11 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:22.644691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:22.694769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c74c0-743f-4739-b56f-6fc0b129e988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:22.695359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c74c0-743f-4739-b56f-6fc0b129e988 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:22.695652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c74c0-743f-4739-b56f-6fc0b129e988 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:22.695916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c74c0-743f-4739-b56f-6fc0b129e988 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:22.696522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c74c0-743f-4739-b56f-6fc0b129e988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:22.698431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c74c0-743f-4739-b56f-6fc0b129e988 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:22.718791 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4106/8177] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:24:22 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:22.722981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf372924-54e7-4daa-8362-f33853bb40b0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:22.739559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf372924-54e7-4daa-8362-f33853bb40b0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:22.745023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458f4d1-1d2d-4636-888f-410ae3ede254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:22.745645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458f4d1-1d2d-4636-888f-410ae3ede254 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:22.745921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458f4d1-1d2d-4636-888f-410ae3ede254 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:22.746190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458f4d1-1d2d-4636-888f-410ae3ede254 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:22.746825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458f4d1-1d2d-4636-888f-410ae3ede254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:22.748433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458f4d1-1d2d-4636-888f-410ae3ede254 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:22.758965 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4072/8178] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:22.762954 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c74c0-743f-4739-b56f-6fc0b129e988 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:22.772976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c74c0-743f-4739-b56f-6fc0b129e988 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:22.777788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667cff7-ce20-4a62-a18f-63cf84706808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:22.778421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667cff7-ce20-4a62-a18f-63cf84706808 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:22.778675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667cff7-ce20-4a62-a18f-63cf84706808 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:22.778960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667cff7-ce20-4a62-a18f-63cf84706808 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:22.779583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667cff7-ce20-4a62-a18f-63cf84706808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:22.781214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667cff7-ce20-4a62-a18f-63cf84706808 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:22.791824 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4107/8179] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:22.795875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458f4d1-1d2d-4636-888f-410ae3ede254 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:22.806099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458f4d1-1d2d-4636-888f-410ae3ede254 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:22.810774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124134c-435e-424c-9196-ca36e780768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:22.811404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124134c-435e-424c-9196-ca36e780768b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:22.811678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124134c-435e-424c-9196-ca36e780768b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:22.811946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124134c-435e-424c-9196-ca36e780768b admin admin] PATH_INFO: `/v3/projects/ed7fd7cd6e0e44fbb6417fc21d360c9d/users/d49bc5df78e94ee8ab3cfebcc9a8e340/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:22.827125 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124134c-435e-424c-9196-ca36e780768b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7fd7cd6e0e44fbb6417fc21d360c9d, user_id=d49bc5df78e94ee8ab3cfebcc9a8e340, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:22.829910 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124134c-435e-424c-9196-ca36e780768b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:22.843044 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4073/8180] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:24:22 2022] PUT /identity/v3/projects/ed7fd7cd6e0e44fbb6417fc21d360c9d/users/d49bc5df78e94ee8ab3cfebcc9a8e340/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:22.853854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ef542-4c08-4943-a11d-1c54380c44a0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:22.854298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ef542-4c08-4943-a11d-1c54380c44a0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:22.854480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ef542-4c08-4943-a11d-1c54380c44a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:22.884010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9f7ef542-4c08-4943-a11d-1c54380c44a0 None None] MFA Rules not processed for user `d49bc5df78e94ee8ab3cfebcc9a8e340`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:22.885244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9f7ef542-4c08-4943-a11d-1c54380c44a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:22.947173 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4108/8181] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:24:22.957966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124134c-435e-424c-9196-ca36e780768b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:22.959851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a124134c-435e-424c-9196-ca36e780768b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:23.003384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:23.021526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124134c-435e-424c-9196-ca36e780768b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:23.025237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf200bc-028e-4f5a-a477-ee9374a1ae16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:23.025829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf200bc-028e-4f5a-a477-ee9374a1ae16 service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:23.026100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf200bc-028e-4f5a-a477-ee9374a1ae16 service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:23.026363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf200bc-028e-4f5a-a477-ee9374a1ae16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:23.034559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf200bc-028e-4f5a-a477-ee9374a1ae16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:23.035974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf200bc-028e-4f5a-a477-ee9374a1ae16 service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:23.045846 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4074/8182] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:23.071069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ef542-4c08-4943-a11d-1c54380c44a0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:23.133994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ef542-4c08-4943-a11d-1c54380c44a0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:23.138554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:23.139170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:23.139452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:23.139718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] PATH_INFO: `/v3/users/d49bc5df78e94ee8ab3cfebcc9a8e340` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:23.140835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d49bc5df78e94ee8ab3cfebcc9a8e340)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:23.142760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:23.165360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:23.190730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd49bc5df78e94ee8ab3cfebcc9a8e340'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:23.199647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd49bc5df78e94ee8ab3cfebcc9a8e340'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:23.212111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd49bc5df78e94ee8ab3cfebcc9a8e340'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:23.219821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd49bc5df78e94ee8ab3cfebcc9a8e340'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:23.220522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd49bc5df78e94ee8ab3cfebcc9a8e340'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:23.227333 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4109/8183] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:24:23 2022] DELETE /identity/v3/users/d49bc5df78e94ee8ab3cfebcc9a8e340 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:24:23.232123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7ef91-eace-479d-97c5-b5497cdf98f9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:23.232405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7ef91-eace-479d-97c5-b5497cdf98f9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:23.232673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7ef91-eace-479d-97c5-b5497cdf98f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:23.251962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a5b7ef91-eace-479d-97c5-b5497cdf98f9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:23.253172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a5b7ef91-eace-479d-97c5-b5497cdf98f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:23.312999 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4075/8184] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:23.322949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:23.324348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:23.385378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857288-58ec-4a95-978d-27c794776098 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:23.389056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b088194-3afa-4672-9637-e3cbcdc9b521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:23.389666 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b088194-3afa-4672-9637-e3cbcdc9b521 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:23.389943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b088194-3afa-4672-9637-e3cbcdc9b521 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:23.390211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b088194-3afa-4672-9637-e3cbcdc9b521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:23.399325 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b088194-3afa-4672-9637-e3cbcdc9b521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:23.400763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b088194-3afa-4672-9637-e3cbcdc9b521 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:23.410205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4110/8185] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:23.673906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7ef91-eace-479d-97c5-b5497cdf98f9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:23.675419 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a5b7ef91-eace-479d-97c5-b5497cdf98f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:23.688529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7ef91-eace-479d-97c5-b5497cdf98f9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:23.693737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:23.694357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:23.694638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:23.694907 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] PATH_INFO: `/v3/projects/ed7fd7cd6e0e44fbb6417fc21d360c9d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:23.696268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7fd7cd6e0e44fbb6417fc21d360c9d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:23.698843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:23.747885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7fd7cd6e0e44fbb6417fc21d360c9d', 'request_id': 'req-2f97c60a-0289-4f55-abe4-55a4615ea63d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:23.756630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7fd7cd6e0e44fbb6417fc21d360c9d', 'request_id': 'req-2f97c60a-0289-4f55-abe4-55a4615ea63d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:23.763066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7fd7cd6e0e44fbb6417fc21d360c9d', 'request_id': 'req-2f97c60a-0289-4f55-abe4-55a4615ea63d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:23.773312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] The token cache is being invalidate because project ed7fd7cd6e0e44fbb6417fc21d360c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:24:23.773589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:23.774220 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-2f97c60a-0289-4f55-abe4-55a4615ea63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:24:23.775691 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4076/8186] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:24:23 2022] DELETE /identity/v3/projects/ed7fd7cd6e0e44fbb6417fc21d360c9d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:23.780496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b14f3e-980b-4f50-844e-bb7aaea2516c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:23.780764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b14f3e-980b-4f50-844e-bb7aaea2516c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:23.781030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b14f3e-980b-4f50-844e-bb7aaea2516c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:23.804985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a8b14f3e-980b-4f50-844e-bb7aaea2516c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:23.806196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a8b14f3e-980b-4f50-844e-bb7aaea2516c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:23.865319 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4111/8187] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:23.873887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47a5bf-b5ff-4571-8849-e63595d4039f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:23.874168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47a5bf-b5ff-4571-8849-e63595d4039f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:23.874438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47a5bf-b5ff-4571-8849-e63595d4039f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:23.894718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3e47a5bf-b5ff-4571-8849-e63595d4039f None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:23.895899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3e47a5bf-b5ff-4571-8849-e63595d4039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:23.903757 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4077/8188] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:23.907861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b14f3e-980b-4f50-844e-bb7aaea2516c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:23.917919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b14f3e-980b-4f50-844e-bb7aaea2516c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:23.922484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c412a8-9153-4761-a3a2-bfa1bac688f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:23.923077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c412a8-9153-4761-a3a2-bfa1bac688f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:23.923355 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c412a8-9153-4761-a3a2-bfa1bac688f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:23.923625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c412a8-9153-4761-a3a2-bfa1bac688f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:23.924259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c412a8-9153-4761-a3a2-bfa1bac688f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:24:23.924558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c412a8-9153-4761-a3a2-bfa1bac688f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:23.926177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c412a8-9153-4761-a3a2-bfa1bac688f5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:23.938056 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4112/8189] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:24:23.942315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47a5bf-b5ff-4571-8849-e63595d4039f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:23.952195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47a5bf-b5ff-4571-8849-e63595d4039f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:23.956828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6616f4-5a60-45e4-a1df-174b83e1fa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:23.957460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6616f4-5a60-45e4-a1df-174b83e1fa59 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:23.957737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6616f4-5a60-45e4-a1df-174b83e1fa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:23.958007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6616f4-5a60-45e4-a1df-174b83e1fa59 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:23.958789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6616f4-5a60-45e4-a1df-174b83e1fa59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:23.960721 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6616f4-5a60-45e4-a1df-174b83e1fa59 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:23.990374 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4078/8190] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:24:23 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:23.992782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c412a8-9153-4761-a3a2-bfa1bac688f5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:24.002875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c412a8-9153-4761-a3a2-bfa1bac688f5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:24.030685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:24.061695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e1ab8-bf50-491a-a5b9-fce93ee682a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:24.062284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e1ab8-bf50-491a-a5b9-fce93ee682a1 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:24.062555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e1ab8-bf50-491a-a5b9-fce93ee682a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:24.062818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e1ab8-bf50-491a-a5b9-fce93ee682a1 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:24.063424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e1ab8-bf50-491a-a5b9-fce93ee682a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:24.065330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e1ab8-bf50-491a-a5b9-fce93ee682a1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:24.086109 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4113/8191] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:24:23 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:24:24.090362 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6616f4-5a60-45e4-a1df-174b83e1fa59 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:24.101287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6616f4-5a60-45e4-a1df-174b83e1fa59 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:24.106128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd1a41-7df3-4b7d-be9e-790c90777f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:24.106716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd1a41-7df3-4b7d-be9e-790c90777f88 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:24.106990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd1a41-7df3-4b7d-be9e-790c90777f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:24.107256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd1a41-7df3-4b7d-be9e-790c90777f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:24.107927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd1a41-7df3-4b7d-be9e-790c90777f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:24.109549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd1a41-7df3-4b7d-be9e-790c90777f88 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:24.120377 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4079/8192] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:24.124192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e1ab8-bf50-491a-a5b9-fce93ee682a1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:24.134169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e1ab8-bf50-491a-a5b9-fce93ee682a1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:24.138788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be38913-dd0f-44a3-bc64-edee21178bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:24.139369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be38913-dd0f-44a3-bc64-edee21178bea admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:24.139643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be38913-dd0f-44a3-bc64-edee21178bea admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:24.139909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be38913-dd0f-44a3-bc64-edee21178bea admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:24.140534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be38913-dd0f-44a3-bc64-edee21178bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:24.142125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be38913-dd0f-44a3-bc64-edee21178bea admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:24.152461 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4114/8193] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:24:24.156385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd1a41-7df3-4b7d-be9e-790c90777f88 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:24.167357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd1a41-7df3-4b7d-be9e-790c90777f88 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:24.171898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e75112-4391-4cc9-84f3-340c173df9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:24.172525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e75112-4391-4cc9-84f3-340c173df9e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:24.172799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e75112-4391-4cc9-84f3-340c173df9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:24.173081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e75112-4391-4cc9-84f3-340c173df9e8 admin admin] PATH_INFO: `/v3/projects/73d7ccfabbb44b5d8b1aa1253b2048e1/users/d159c11f06c4482ebf4e600305bfe0ee/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:24.187975 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e75112-4391-4cc9-84f3-340c173df9e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d7ccfabbb44b5d8b1aa1253b2048e1, user_id=d159c11f06c4482ebf4e600305bfe0ee, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:24.190645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e75112-4391-4cc9-84f3-340c173df9e8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:24.205983 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4080/8194] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:24:24 2022] PUT /identity/v3/projects/73d7ccfabbb44b5d8b1aa1253b2048e1/users/d159c11f06c4482ebf4e600305bfe0ee/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:24:24.211322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd732c-05d1-4930-aee9-976aabc1e1f9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:24.211607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd732c-05d1-4930-aee9-976aabc1e1f9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:24.211880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd732c-05d1-4930-aee9-976aabc1e1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:24.231731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d7bd732c-05d1-4930-aee9-976aabc1e1f9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:24.232925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d7bd732c-05d1-4930-aee9-976aabc1e1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:24.292654 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4115/8195] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:24.302279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e75112-4391-4cc9-84f3-340c173df9e8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:24.304209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e6e75112-4391-4cc9-84f3-340c173df9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:24.309501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:24.370792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e75112-4391-4cc9-84f3-340c173df9e8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:24.374497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a5679-c1f8-483f-b8a7-b5180fcaa6a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:24.375095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a5679-c1f8-483f-b8a7-b5180fcaa6a9 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:24.375371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a5679-c1f8-483f-b8a7-b5180fcaa6a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:24.375637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a5679-c1f8-483f-b8a7-b5180fcaa6a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:24.385649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a5679-c1f8-483f-b8a7-b5180fcaa6a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:24.387101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a5679-c1f8-483f-b8a7-b5180fcaa6a9 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:24.397060 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4081/8196] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:24:25.003331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc915b-3e50-4899-a17e-baa951bcd636 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:25.003833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc915b-3e50-4899-a17e-baa951bcd636 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:25.003916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc915b-3e50-4899-a17e-baa951bcd636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:25.033127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-90cc915b-3e50-4899-a17e-baa951bcd636 None None] MFA Rules not processed for user `d159c11f06c4482ebf4e600305bfe0ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:25.034324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-90cc915b-3e50-4899-a17e-baa951bcd636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:25.103056 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4116/8197] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:25.116293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a5679-c1f8-483f-b8a7-b5180fcaa6a9 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:25.117954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cd1a5679-c1f8-483f-b8a7-b5180fcaa6a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:25.178215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a5679-c1f8-483f-b8a7-b5180fcaa6a9 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:25.182213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9ee8c-860f-44ac-af0b-64164f91065a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:25.183580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9ee8c-860f-44ac-af0b-64164f91065a service cinder] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:25.184225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9ee8c-860f-44ac-af0b-64164f91065a service cinder] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:25.184505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9ee8c-860f-44ac-af0b-64164f91065a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:25.194322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9ee8c-860f-44ac-af0b-64164f91065a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:25.195777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9ee8c-860f-44ac-af0b-64164f91065a service cinder] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:25.205357 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4082/8198] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:35.123693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:35.201196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=51 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:39.404815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e4f7a-c2bf-4e72-93c0-a59e171620a4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:39.405358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e4f7a-c2bf-4e72-93c0-a59e171620a4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:39.405443 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e4f7a-c2bf-4e72-93c0-a59e171620a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:39.426830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0b3e4f7a-c2bf-4e72-93c0-a59e171620a4 None None] MFA Rules not processed for user `d159c11f06c4482ebf4e600305bfe0ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:24:39.436582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0b3e4f7a-c2bf-4e72-93c0-a59e171620a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:39.444266 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4117/8199] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:24:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:24:41.024460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9ee8c-860f-44ac-af0b-64164f91065a service cinder] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:41.025929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b0a9ee8c-860f-44ac-af0b-64164f91065a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:41.111318 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9ee8c-860f-44ac-af0b-64164f91065a service cinder] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:41.116490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f7332-09e9-4f6e-81af-3ef9736ea928 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:41.117123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f7332-09e9-4f6e-81af-3ef9736ea928 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:41.117397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f7332-09e9-4f6e-81af-3ef9736ea928 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:41.117663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f7332-09e9-4f6e-81af-3ef9736ea928 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:41.119388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-446f7332-09e9-4f6e-81af-3ef9736ea928 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:41.133296 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f7332-09e9-4f6e-81af-3ef9736ea928 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:41.134868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f7332-09e9-4f6e-81af-3ef9736ea928 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:41.147631 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4083/8200] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:42.833296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e4f7a-c2bf-4e72-93c0-a59e171620a4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:42.834700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0b3e4f7a-c2bf-4e72-93c0-a59e171620a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:42.849600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e4f7a-c2bf-4e72-93c0-a59e171620a4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:42.853474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3dc5a-0346-4f51-91a1-187c2e0c5823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:42.854058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3dc5a-0346-4f51-91a1-187c2e0c5823 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:42.854348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3dc5a-0346-4f51-91a1-187c2e0c5823 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:42.854595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3dc5a-0346-4f51-91a1-187c2e0c5823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:42.856249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9fb3dc5a-0346-4f51-91a1-187c2e0c5823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:42.868520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3dc5a-0346-4f51-91a1-187c2e0c5823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:42.869968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3dc5a-0346-4f51-91a1-187c2e0c5823 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:42.880583 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4118/8201] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:43.613500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f7332-09e9-4f6e-81af-3ef9736ea928 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:43.615589 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-446f7332-09e9-4f6e-81af-3ef9736ea928 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:43.702856 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f7332-09e9-4f6e-81af-3ef9736ea928 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:43.709128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a5e14-3771-484e-969c-cbe49a85ee81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:43.709673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a5e14-3771-484e-969c-cbe49a85ee81 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:43.709950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a5e14-3771-484e-969c-cbe49a85ee81 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:43.710222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a5e14-3771-484e-969c-cbe49a85ee81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:43.711985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3f0a5e14-3771-484e-969c-cbe49a85ee81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:43.727183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a5e14-3771-484e-969c-cbe49a85ee81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:43.729055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a5e14-3771-484e-969c-cbe49a85ee81 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:43.743722 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4084/8202] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:45.018026 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3dc5a-0346-4f51-91a1-187c2e0c5823 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:45.029564 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3dc5a-0346-4f51-91a1-187c2e0c5823 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:45.033986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40320f6e-3529-42c0-99d4-590ebdeb1de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:45.034691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40320f6e-3529-42c0-99d4-590ebdeb1de4 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:45.035036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40320f6e-3529-42c0-99d4-590ebdeb1de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:45.035409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40320f6e-3529-42c0-99d4-590ebdeb1de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:45.037856 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-40320f6e-3529-42c0-99d4-590ebdeb1de4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:45.050662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40320f6e-3529-42c0-99d4-590ebdeb1de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:45.052777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40320f6e-3529-42c0-99d4-590ebdeb1de4 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:45.063699 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4119/8203] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:45.326931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a5e14-3771-484e-969c-cbe49a85ee81 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:45.339705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a5e14-3771-484e-969c-cbe49a85ee81 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:45.343341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c036db-091b-4391-ad26-f86c67bd542b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:45.343924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c036db-091b-4391-ad26-f86c67bd542b service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:45.344195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c036db-091b-4391-ad26-f86c67bd542b service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:45.344458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c036db-091b-4391-ad26-f86c67bd542b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:45.346165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d3c036db-091b-4391-ad26-f86c67bd542b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:45.362698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c036db-091b-4391-ad26-f86c67bd542b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:45.364666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c036db-091b-4391-ad26-f86c67bd542b service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:45.378354 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4085/8204] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:50.607396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40320f6e-3529-42c0-99d4-590ebdeb1de4 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:24:50.623771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40320f6e-3529-42c0-99d4-590ebdeb1de4 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:24:50.629371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9af76d-2b48-4c64-aef1-440481328610 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:24:50.630657 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9af76d-2b48-4c64-aef1-440481328610 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:24:50.630935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9af76d-2b48-4c64-aef1-440481328610 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:24:50.631204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9af76d-2b48-4c64-aef1-440481328610 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:24:50.632947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1e9af76d-2b48-4c64-aef1-440481328610 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:24:50.647917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9af76d-2b48-4c64-aef1-440481328610 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:24:50.649538 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9af76d-2b48-4c64-aef1-440481328610 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:24:50.663297 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4120/8205] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:24:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:24:55.370901 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=72 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:25:00.656005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=21 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:25:47.912995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c036db-091b-4391-ad26-f86c67bd542b service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:25:47.925265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c036db-091b-4391-ad26-f86c67bd542b service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:25:47.929615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76648828-8e87-4344-994c-462f4c2db6ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:25:47.930222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76648828-8e87-4344-994c-462f4c2db6ae service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:25:47.930503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76648828-8e87-4344-994c-462f4c2db6ae service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:25:47.930850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76648828-8e87-4344-994c-462f4c2db6ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:25:47.932680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-76648828-8e87-4344-994c-462f4c2db6ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:25:47.950456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76648828-8e87-4344-994c-462f4c2db6ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:25:47.951913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76648828-8e87-4344-994c-462f4c2db6ae service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:25:47.964088 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4086/8206] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:25:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:25:54.775773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9af76d-2b48-4c64-aef1-440481328610 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:25:54.777336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1e9af76d-2b48-4c64-aef1-440481328610 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:25:54.794718 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9af76d-2b48-4c64-aef1-440481328610 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:25:54.800330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b4f85-d5a6-403f-969a-9bca625c09b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:25:54.800946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b4f85-d5a6-403f-969a-9bca625c09b1 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:25:54.801267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b4f85-d5a6-403f-969a-9bca625c09b1 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:25:54.801537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b4f85-d5a6-403f-969a-9bca625c09b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:25:54.803273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3e5b4f85-d5a6-403f-969a-9bca625c09b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:25:54.820270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b4f85-d5a6-403f-969a-9bca625c09b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:25:54.821904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b4f85-d5a6-403f-969a-9bca625c09b1 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:25:54.839149 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4121/8207] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:25:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:25:57.958117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:00.848073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79d3ee-69d2-4a71-8bf1-57f784570cf7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:26:00.848678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79d3ee-69d2-4a71-8bf1-57f784570cf7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:26:00.848678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79d3ee-69d2-4a71-8bf1-57f784570cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:26:00.873417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3d79d3ee-69d2-4a71-8bf1-57f784570cf7 None None] MFA Rules not processed for user `a0d6261b46824acf958ef749425ae421`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:26:00.874955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3d79d3ee-69d2-4a71-8bf1-57f784570cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:26:00.940915 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4087/8208] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:26:04.830003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:10.933526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=25 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:11.592345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b4f85-d5a6-403f-969a-9bca625c09b1 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:26:11.606517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b4f85-d5a6-403f-969a-9bca625c09b1 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:26:11.611560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42c508-7a18-400c-9aca-986413d0f7d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:26:11.612141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42c508-7a18-400c-9aca-986413d0f7d8 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:26:11.612407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42c508-7a18-400c-9aca-986413d0f7d8 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:26:11.612668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42c508-7a18-400c-9aca-986413d0f7d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:26:11.614373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-be42c508-7a18-400c-9aca-986413d0f7d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:26:11.640425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42c508-7a18-400c-9aca-986413d0f7d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:26:11.642046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42c508-7a18-400c-9aca-986413d0f7d8 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:26:11.655226 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4122/8209] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:26:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:26:15.768887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79d3ee-69d2-4a71-8bf1-57f784570cf7 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:26:15.785689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79d3ee-69d2-4a71-8bf1-57f784570cf7 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:26:15.794798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424e84a-c560-4396-89ee-c5e38ae1e23f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:26:15.794798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424e84a-c560-4396-89ee-c5e38ae1e23f service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:26:15.794798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424e84a-c560-4396-89ee-c5e38ae1e23f service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:26:15.794798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424e84a-c560-4396-89ee-c5e38ae1e23f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:26:15.795087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9424e84a-c560-4396-89ee-c5e38ae1e23f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:26:15.809995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424e84a-c560-4396-89ee-c5e38ae1e23f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:26:15.811631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424e84a-c560-4396-89ee-c5e38ae1e23f service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:26:15.824943 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4088/8210] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:26:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:26:21.647568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=11 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:25.817487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:35.895878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42c508-7a18-400c-9aca-986413d0f7d8 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:26:35.898381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-be42c508-7a18-400c-9aca-986413d0f7d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:26:35.916187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42c508-7a18-400c-9aca-986413d0f7d8 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:26:35.922272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08693c-ec80-4a83-bfdb-279773448eb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:26:35.922855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08693c-ec80-4a83-bfdb-279773448eb7 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:26:35.923124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08693c-ec80-4a83-bfdb-279773448eb7 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:26:35.923387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08693c-ec80-4a83-bfdb-279773448eb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:26:35.925024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9a08693c-ec80-4a83-bfdb-279773448eb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:26:35.935542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08693c-ec80-4a83-bfdb-279773448eb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:26:35.937108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08693c-ec80-4a83-bfdb-279773448eb7 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:26:35.945978 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4123/8211] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:26:35 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:26:42.933847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424e84a-c560-4396-89ee-c5e38ae1e23f service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:26:42.956992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424e84a-c560-4396-89ee-c5e38ae1e23f service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:26:42.962610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70451-b334-4c9b-b66a-96d20bccd1e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:26:42.963200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70451-b334-4c9b-b66a-96d20bccd1e8 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:26:42.963477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70451-b334-4c9b-b66a-96d20bccd1e8 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:26:42.963748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70451-b334-4c9b-b66a-96d20bccd1e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:26:42.965470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-66f70451-b334-4c9b-b66a-96d20bccd1e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:26:43.030692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f70451-b334-4c9b-b66a-96d20bccd1e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:26:43.032279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f70451-b334-4c9b-b66a-96d20bccd1e8 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:26:43.043127 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4089/8212] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:26:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:26:44.211693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995676d-da38-4ff7-b6ab-3b18114f4fb8 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:26:44.212184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995676d-da38-4ff7-b6ab-3b18114f4fb8 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:26:44.212290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995676d-da38-4ff7-b6ab-3b18114f4fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:26:44.232515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c995676d-da38-4ff7-b6ab-3b18114f4fb8 None None] MFA Rules not processed for user `d159c11f06c4482ebf4e600305bfe0ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:26:44.233920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c995676d-da38-4ff7-b6ab-3b18114f4fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:26:44.240912 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4124/8213] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:26:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:26:44.485880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70451-b334-4c9b-b66a-96d20bccd1e8 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:26:44.498377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70451-b334-4c9b-b66a-96d20bccd1e8 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:26:44.502001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8153a-12e7-4526-b360-b6b0c476ef83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:26:44.502596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8153a-12e7-4526-b360-b6b0c476ef83 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:26:44.502963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8153a-12e7-4526-b360-b6b0c476ef83 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:26:44.503232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8153a-12e7-4526-b360-b6b0c476ef83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:26:44.505736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-42c8153a-12e7-4526-b360-b6b0c476ef83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:26:44.519626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8153a-12e7-4526-b360-b6b0c476ef83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:26:44.521280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8153a-12e7-4526-b360-b6b0c476ef83 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:26:44.534575 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4090/8214] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:26:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:26:54.227152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=9 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:54.526271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=39 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:27:15.501343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd142ce-a3b9-4ca4-85ab-25e2c373622f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:27:15.502185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd142ce-a3b9-4ca4-85ab-25e2c373622f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:27:15.502592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd142ce-a3b9-4ca4-85ab-25e2c373622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:27:15.525896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-abd142ce-a3b9-4ca4-85ab-25e2c373622f None None] MFA Rules not processed for user `a0d6261b46824acf958ef749425ae421`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:27:15.527134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-abd142ce-a3b9-4ca4-85ab-25e2c373622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:27:15.541519 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4125/8215] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:27:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:27:25.519660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:53.213599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8153a-12e7-4526-b360-b6b0c476ef83 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:53.237133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8153a-12e7-4526-b360-b6b0c476ef83 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:53.241057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b15b0a-c517-4376-b55d-81da230e38dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:53.241677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b15b0a-c517-4376-b55d-81da230e38dc service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:53.241952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b15b0a-c517-4376-b55d-81da230e38dc service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:53.242222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b15b0a-c517-4376-b55d-81da230e38dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:53.244073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-46b15b0a-c517-4376-b55d-81da230e38dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:28:53.274126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b15b0a-c517-4376-b55d-81da230e38dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:53.275532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b15b0a-c517-4376-b55d-81da230e38dc service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:53.289431 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4091/8216] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:28:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:28:58.297891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd142ce-a3b9-4ca4-85ab-25e2c373622f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:58.299318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-abd142ce-a3b9-4ca4-85ab-25e2c373622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:28:58.323345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd142ce-a3b9-4ca4-85ab-25e2c373622f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:58.327907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:58.328523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:58.328796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:58.329134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] PATH_INFO: `/v3/users/a0d6261b46824acf958ef749425ae421` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:58.330227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d6261b46824acf958ef749425ae421)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:58.332118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:58.378768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d6261b46824acf958ef749425ae421'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:58.386701 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d6261b46824acf958ef749425ae421'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:58.399300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d6261b46824acf958ef749425ae421'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:58.406715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d6261b46824acf958ef749425ae421'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:58.407414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d6261b46824acf958ef749425ae421'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:58.413545 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4126/8217] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:28:58 2022] DELETE /identity/v3/users/a0d6261b46824acf958ef749425ae421 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:28:58.419086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bb575-400c-4a25-ad58-85a9d1b640f4 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:58.419553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bb575-400c-4a25-ad58-85a9d1b640f4 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:58.419651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bb575-400c-4a25-ad58-85a9d1b640f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:58.440586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fb7bb575-400c-4a25-ad58-85a9d1b640f4 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:28:58.441753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fb7bb575-400c-4a25-ad58-85a9d1b640f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:28:58.506734 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4092/8218] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:28:58.517135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:58.518733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:28:58.580452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0a81-912d-462f-9647-abd6206f77a4 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:58.584061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc79e3a-f144-4b47-b7ec-495bb45bd38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:58.584658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc79e3a-f144-4b47-b7ec-495bb45bd38e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:58.584930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc79e3a-f144-4b47-b7ec-495bb45bd38e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:58.585228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc79e3a-f144-4b47-b7ec-495bb45bd38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:58.594871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc79e3a-f144-4b47-b7ec-495bb45bd38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:58.596289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc79e3a-f144-4b47-b7ec-495bb45bd38e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:58.606757 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4127/8219] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:28:58.779103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bb575-400c-4a25-ad58-85a9d1b640f4 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:58.780539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fb7bb575-400c-4a25-ad58-85a9d1b640f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:28:58.793969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bb575-400c-4a25-ad58-85a9d1b640f4 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:58.798811 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:58.799627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:58.799908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:58.800180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] PATH_INFO: `/v3/projects/cdd706928da546cf8e5b0fcbf6ec94bd` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:58.801344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdd706928da546cf8e5b0fcbf6ec94bd)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:58.803272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:58.850649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'request_id': 'req-c64e3dc4-4440-4a0e-8fda-fb460aead31c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:58.860605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'request_id': 'req-c64e3dc4-4440-4a0e-8fda-fb460aead31c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:58.868371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'request_id': 'req-c64e3dc4-4440-4a0e-8fda-fb460aead31c'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:58.883659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] The token cache is being invalidate because project cdd706928da546cf8e5b0fcbf6ec94bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:28:58.883946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:58.884641 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:58.886990 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4093/8220] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:28:58 2022] DELETE /identity/v3/projects/cdd706928da546cf8e5b0fcbf6ec94bd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:28:58.891401 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc79e3a-f144-4b47-b7ec-495bb45bd38e service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:58.903070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc79e3a-f144-4b47-b7ec-495bb45bd38e service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:58.965281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:58.966028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:58.966372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:58.966677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] PATH_INFO: `/v3/users/4641b6af6bf940439be4f38054c6f6fe` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:58.967991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4641b6af6bf940439be4f38054c6f6fe)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:58.970123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:58.975510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:59.021344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4641b6af6bf940439be4f38054c6f6fe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:59.028409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4641b6af6bf940439be4f38054c6f6fe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:59.040672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4641b6af6bf940439be4f38054c6f6fe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:59.048014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4641b6af6bf940439be4f38054c6f6fe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:59.048651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-379fad2d-4ea3-44e7-88bb-da23c582252d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4641b6af6bf940439be4f38054c6f6fe'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:59.054851 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4128/8221] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:28:58 2022] DELETE /identity/v3/users/4641b6af6bf940439be4f38054c6f6fe => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:28:59.251301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:59.253026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:28:59.270177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e3dc4-4440-4a0e-8fda-fb460aead31c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:59.327672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:59.343757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:59.344473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:59.344778 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:59.345447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] PATH_INFO: `/v3/projects/4405aa20ee8a4987878be13acb9b0029` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:59.346812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4405aa20ee8a4987878be13acb9b0029)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:59.349151 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:59.404019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4405aa20ee8a4987878be13acb9b0029', 'request_id': 'req-3d9c2175-125b-4c5c-8ee0-ba9037963524'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:59.413304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4405aa20ee8a4987878be13acb9b0029', 'request_id': 'req-3d9c2175-125b-4c5c-8ee0-ba9037963524'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:59.422008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4405aa20ee8a4987878be13acb9b0029', 'request_id': 'req-3d9c2175-125b-4c5c-8ee0-ba9037963524'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:59.434122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] The token cache is being invalidate because project 4405aa20ee8a4987878be13acb9b0029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:28:59.434411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:59.435070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3d9c2175-125b-4c5c-8ee0-ba9037963524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:28:59.436616 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4094/8222] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:28:59 2022] DELETE /identity/v3/projects/4405aa20ee8a4987878be13acb9b0029 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:28:59.441761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58401e-b8ce-4ce6-9a86-bee6d7587cc0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:59.442219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58401e-b8ce-4ce6-9a86-bee6d7587cc0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:59.442323 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58401e-b8ce-4ce6-9a86-bee6d7587cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:59.484105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ef58401e-b8ce-4ce6-9a86-bee6d7587cc0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:28:59.485702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ef58401e-b8ce-4ce6-9a86-bee6d7587cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:28:59.553383 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4129/8223] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:28:59.562572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375c63a-4558-43dc-94b2-5c54ebffb431 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:59.562858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375c63a-4558-43dc-94b2-5c54ebffb431 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:59.563128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375c63a-4558-43dc-94b2-5c54ebffb431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:59.589004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7375c63a-4558-43dc-94b2-5c54ebffb431 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:28:59.590279 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7375c63a-4558-43dc-94b2-5c54ebffb431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:28:59.598277 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4095/8224] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:28:59.603060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58401e-b8ce-4ce6-9a86-bee6d7587cc0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:59.618509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58401e-b8ce-4ce6-9a86-bee6d7587cc0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:59.624237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c26e1-470f-40e5-9688-ba54ca1f128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:59.624843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c26e1-470f-40e5-9688-ba54ca1f128c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:59.625178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c26e1-470f-40e5-9688-ba54ca1f128c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:59.625480 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c26e1-470f-40e5-9688-ba54ca1f128c admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:59.626121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c26e1-470f-40e5-9688-ba54ca1f128c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:28:59.626421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c26e1-470f-40e5-9688-ba54ca1f128c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:59.628007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c26e1-470f-40e5-9688-ba54ca1f128c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:59.640310 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4130/8225] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:28:59.644810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375c63a-4558-43dc-94b2-5c54ebffb431 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:59.655044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375c63a-4558-43dc-94b2-5c54ebffb431 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:59.659802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09214dba-9f1c-4017-9b9b-3e753b62b8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:59.660390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09214dba-9f1c-4017-9b9b-3e753b62b8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:59.660664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09214dba-9f1c-4017-9b9b-3e753b62b8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:59.660930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09214dba-9f1c-4017-9b9b-3e753b62b8d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:59.661616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09214dba-9f1c-4017-9b9b-3e753b62b8d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:59.663541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09214dba-9f1c-4017-9b9b-3e753b62b8d0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:59.700147 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4096/8226] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:28:59 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:28:59.704204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c26e1-470f-40e5-9688-ba54ca1f128c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:59.714840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c26e1-470f-40e5-9688-ba54ca1f128c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:59.764866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:59.772079 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb76a5-b781-4621-a737-1b356bd9c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:59.772782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb76a5-b781-4621-a737-1b356bd9c07c admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:59.773291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb76a5-b781-4621-a737-1b356bd9c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:59.773781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb76a5-b781-4621-a737-1b356bd9c07c admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:59.774625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb76a5-b781-4621-a737-1b356bd9c07c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:59.777094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb76a5-b781-4621-a737-1b356bd9c07c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:59.797639 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4131/8227] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:28:59 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:28:59.801857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09214dba-9f1c-4017-9b9b-3e753b62b8d0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:59.813241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09214dba-9f1c-4017-9b9b-3e753b62b8d0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:59.818014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1161672-40e3-4353-82db-ce9df06211f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:59.818603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1161672-40e3-4353-82db-ce9df06211f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:59.818880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1161672-40e3-4353-82db-ce9df06211f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:59.819205 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1161672-40e3-4353-82db-ce9df06211f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:59.819848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1161672-40e3-4353-82db-ce9df06211f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:59.821503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1161672-40e3-4353-82db-ce9df06211f3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:59.832384 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4097/8228] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:28:59.836258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb76a5-b781-4621-a737-1b356bd9c07c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:59.846571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb76a5-b781-4621-a737-1b356bd9c07c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:59.851288 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494051fd-00aa-43d5-a4ed-9d4d50787935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:59.851870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494051fd-00aa-43d5-a4ed-9d4d50787935 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:59.852143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494051fd-00aa-43d5-a4ed-9d4d50787935 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:59.852523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494051fd-00aa-43d5-a4ed-9d4d50787935 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:59.853177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494051fd-00aa-43d5-a4ed-9d4d50787935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:59.855315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494051fd-00aa-43d5-a4ed-9d4d50787935 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:59.867430 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4132/8229] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:28:59.872270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1161672-40e3-4353-82db-ce9df06211f3 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:28:59.883420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1161672-40e3-4353-82db-ce9df06211f3 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:28:59.888708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cd677-45d6-4da0-bee6-e8de441e727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:28:59.889397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cd677-45d6-4da0-bee6-e8de441e727b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:59.889675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cd677-45d6-4da0-bee6-e8de441e727b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:59.889946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cd677-45d6-4da0-bee6-e8de441e727b admin admin] PATH_INFO: `/v3/projects/a82cb785a32d473db8e973a212715528/users/3df1b449adeb42c58878a8344c9ddd66/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:59.904994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cd677-45d6-4da0-bee6-e8de441e727b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82cb785a32d473db8e973a212715528, user_id=3df1b449adeb42c58878a8344c9ddd66, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:28:59.907715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cd677-45d6-4da0-bee6-e8de441e727b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:28:59.920824 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4098/8230] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:28:59 2022] PUT /identity/v3/projects/a82cb785a32d473db8e973a212715528/users/3df1b449adeb42c58878a8344c9ddd66/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:28:59.926156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fff2a-1c8a-4646-910a-62810a8c65fa None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:28:59.926508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fff2a-1c8a-4646-910a-62810a8c65fa None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:28:59.926783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fff2a-1c8a-4646-910a-62810a8c65fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:28:59.946653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f82fff2a-1c8a-4646-910a-62810a8c65fa None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:28:59.948994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f82fff2a-1c8a-4646-910a-62810a8c65fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:00.007870 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4133/8231] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:00.018123 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cd677-45d6-4da0-bee6-e8de441e727b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:00.019877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7c0cd677-45d6-4da0-bee6-e8de441e727b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:00.078611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cd677-45d6-4da0-bee6-e8de441e727b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:00.082323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a70dc-aca4-4918-b236-6e3f8db390be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:00.082922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a70dc-aca4-4918-b236-6e3f8db390be service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:00.083200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a70dc-aca4-4918-b236-6e3f8db390be service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:00.083472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a70dc-aca4-4918-b236-6e3f8db390be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:00.092948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a70dc-aca4-4918-b236-6e3f8db390be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:00.094398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a70dc-aca4-4918-b236-6e3f8db390be service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:00.104304 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4099/8232] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:29:00.806764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b991a-c694-4cce-989d-e23b7a167bb7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:00.807905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b991a-c694-4cce-989d-e23b7a167bb7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:00.807905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b991a-c694-4cce-989d-e23b7a167bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:00.815896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a70dc-aca4-4918-b236-6e3f8db390be service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:00.821217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-860a70dc-aca4-4918-b236-6e3f8db390be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:00.836237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a70dc-aca4-4918-b236-6e3f8db390be service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:00.840689 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6d4b991a-c694-4cce-989d-e23b7a167bb7 None None] MFA Rules not processed for user `3df1b449adeb42c58878a8344c9ddd66`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:00.841872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:00.842115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6d4b991a-c694-4cce-989d-e23b7a167bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:00.842908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:00.843324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:00.843735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] PATH_INFO: `/v3/users/d159c11f06c4482ebf4e600305bfe0ee` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:00.845154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d159c11f06c4482ebf4e600305bfe0ee)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:00.847441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:00.853539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:00.911736 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd159c11f06c4482ebf4e600305bfe0ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:00.927513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd159c11f06c4482ebf4e600305bfe0ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:00.933183 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4134/8233] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:00.937158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b991a-c694-4cce-989d-e23b7a167bb7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:00.937373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd159c11f06c4482ebf4e600305bfe0ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:00.945892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd159c11f06c4482ebf4e600305bfe0ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:00.946640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd159c11f06c4482ebf4e600305bfe0ee'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:00.950301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b991a-c694-4cce-989d-e23b7a167bb7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:00.954908 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4100/8234] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:29:00 2022] DELETE /identity/v3/users/d159c11f06c4482ebf4e600305bfe0ee => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:00.955319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe5f72-c6d3-40f7-8861-b53393d61c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:00.955932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe5f72-c6d3-40f7-8861-b53393d61c6e admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:00.956207 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe5f72-c6d3-40f7-8861-b53393d61c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:00.956474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe5f72-c6d3-40f7-8861-b53393d61c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:00.957143 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe5f72-c6d3-40f7-8861-b53393d61c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:00.959047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe5f72-c6d3-40f7-8861-b53393d61c6e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:00.993424 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4135/8235] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:29:00 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:29:00.995185 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:00.996931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:01.089485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be73170-892b-442c-84f6-252192a4a8bd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.156442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184c335-d0f3-4621-86b0-5015b751cb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.157315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184c335-d0f3-4621-86b0-5015b751cb96 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.157662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184c335-d0f3-4621-86b0-5015b751cb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.157986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184c335-d0f3-4621-86b0-5015b751cb96 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.158731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184c335-d0f3-4621-86b0-5015b751cb96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.161005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184c335-d0f3-4621-86b0-5015b751cb96 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.171557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe5f72-c6d3-40f7-8861-b53393d61c6e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.174700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-59fe5f72-c6d3-40f7-8861-b53393d61c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:01.190659 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4101/8236] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:29:00 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:29:01.191613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe5f72-c6d3-40f7-8861-b53393d61c6e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.191720 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184c335-d0f3-4621-86b0-5015b751cb96 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.196433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.197170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.197512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.198403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] PATH_INFO: `/v3/projects/73d7ccfabbb44b5d8b1aa1253b2048e1` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.200130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73d7ccfabbb44b5d8b1aa1253b2048e1)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.202209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.214763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184c335-d0f3-4621-86b0-5015b751cb96 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.219629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf02c4-b552-4081-a144-a03f8d679856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.220225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf02c4-b552-4081-a144-a03f8d679856 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.220497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf02c4-b552-4081-a144-a03f8d679856 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.220764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf02c4-b552-4081-a144-a03f8d679856 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.221434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf02c4-b552-4081-a144-a03f8d679856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.223035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf02c4-b552-4081-a144-a03f8d679856 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.234022 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4102/8237] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:29:01.238253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf02c4-b552-4081-a144-a03f8d679856 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.248270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:01.248497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf02c4-b552-4081-a144-a03f8d679856 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.254890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:01.258039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'request_id': 'req-d3f4e276-1f75-482c-8596-abf2dbae62c6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:01.267838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'request_id': 'req-d3f4e276-1f75-482c-8596-abf2dbae62c6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:01.274918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'request_id': 'req-d3f4e276-1f75-482c-8596-abf2dbae62c6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:01.286069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] The token cache is being invalidate because project 73d7ccfabbb44b5d8b1aa1253b2048e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:29:01.286357 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:01.287016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d3f4e276-1f75-482c-8596-abf2dbae62c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:01.292856 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4136/8238] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:29:01 2022] DELETE /identity/v3/projects/73d7ccfabbb44b5d8b1aa1253b2048e1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:29:01.294420 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf34a3-2458-45c6-94bd-f5a7c4f83ab7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.294698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf34a3-2458-45c6-94bd-f5a7c4f83ab7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.294967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf34a3-2458-45c6-94bd-f5a7c4f83ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.314796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-cedf34a3-2458-45c6-94bd-f5a7c4f83ab7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:01.316030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cedf34a3-2458-45c6-94bd-f5a7c4f83ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:01.317356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf3a2b-af1c-4cd4-bcd3-06e9c7898d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.318022 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf3a2b-af1c-4cd4-bcd3-06e9c7898d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.318290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf3a2b-af1c-4cd4-bcd3-06e9c7898d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.318559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf3a2b-af1c-4cd4-bcd3-06e9c7898d99 admin admin] PATH_INFO: `/v3/projects/d4e1b847f5c44a65a7daca0fb163c7f9/users/96f813ef9dad48b99493bd24c3f2a1d7/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.325111 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4137/8239] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:29:01.333925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92f65e-f953-4f6d-bd80-ba63375a321c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.334295 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92f65e-f953-4f6d-bd80-ba63375a321c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.334607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf3a2b-af1c-4cd4-bcd3-06e9c7898d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e1b847f5c44a65a7daca0fb163c7f9, user_id=96f813ef9dad48b99493bd24c3f2a1d7, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.334713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92f65e-f953-4f6d-bd80-ba63375a321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.337315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf3a2b-af1c-4cd4-bcd3-06e9c7898d99 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.355568 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4103/8240] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:29:01 2022] PUT /identity/v3/projects/d4e1b847f5c44a65a7daca0fb163c7f9/users/96f813ef9dad48b99493bd24c3f2a1d7/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:29:01.355763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fc92f65e-f953-4f6d-bd80-ba63375a321c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:01.356782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fc92f65e-f953-4f6d-bd80-ba63375a321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:01.359549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf3a2b-af1c-4cd4-bcd3-06e9c7898d99 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.361140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9faf3a2b-af1c-4cd4-bcd3-06e9c7898d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:01.419227 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4138/8241] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:29:01.423554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92f65e-f953-4f6d-bd80-ba63375a321c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.433138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92f65e-f953-4f6d-bd80-ba63375a321c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.437927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf3a2b-af1c-4cd4-bcd3-06e9c7898d99 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.438310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c456aed-9976-4e67-bd69-3f9342c68a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.438920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c456aed-9976-4e67-bd69-3f9342c68a1c admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.439322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c456aed-9976-4e67-bd69-3f9342c68a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.439598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c456aed-9976-4e67-bd69-3f9342c68a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.440270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c456aed-9976-4e67-bd69-3f9342c68a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:29:01.440571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c456aed-9976-4e67-bd69-3f9342c68a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.442184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c456aed-9976-4e67-bd69-3f9342c68a1c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.442746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc32ad-0280-4c8c-80e2-6dad4cf0102f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.443385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc32ad-0280-4c8c-80e2-6dad4cf0102f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.443822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc32ad-0280-4c8c-80e2-6dad4cf0102f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.444092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc32ad-0280-4c8c-80e2-6dad4cf0102f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.445232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc32ad-0280-4c8c-80e2-6dad4cf0102f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.446824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc32ad-0280-4c8c-80e2-6dad4cf0102f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.458000 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4139/8242] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:29:01.458865 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4104/8243] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:29:01.459086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c456aed-9976-4e67-bd69-3f9342c68a1c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.462775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc32ad-0280-4c8c-80e2-6dad4cf0102f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.469294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c456aed-9976-4e67-bd69-3f9342c68a1c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.472780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc32ad-0280-4c8c-80e2-6dad4cf0102f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.473920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50224b-9fe0-42c5-b07b-c53fbe5d6765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.474504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50224b-9fe0-42c5-b07b-c53fbe5d6765 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.474781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50224b-9fe0-42c5-b07b-c53fbe5d6765 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.475052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50224b-9fe0-42c5-b07b-c53fbe5d6765 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.475667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50224b-9fe0-42c5-b07b-c53fbe5d6765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.477365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1841cc-90ec-42f0-bd0f-11ca5282af7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.477587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50224b-9fe0-42c5-b07b-c53fbe5d6765 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.477981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1841cc-90ec-42f0-bd0f-11ca5282af7a admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.478255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1841cc-90ec-42f0-bd0f-11ca5282af7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.478544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1841cc-90ec-42f0-bd0f-11ca5282af7a admin admin] PATH_INFO: `/v3/projects/d4e1b847f5c44a65a7daca0fb163c7f9/users/96f813ef9dad48b99493bd24c3f2a1d7/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.481169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1841cc-90ec-42f0-bd0f-11ca5282af7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e1b847f5c44a65a7daca0fb163c7f9, user_id=96f813ef9dad48b99493bd24c3f2a1d7, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.484373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1841cc-90ec-42f0-bd0f-11ca5282af7a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.496697 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4105/8244] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:29:01 2022] PUT /identity/v3/projects/d4e1b847f5c44a65a7daca0fb163c7f9/users/96f813ef9dad48b99493bd24c3f2a1d7/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:29:01.510605 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4140/8245] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:29:01 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:29:01.511729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1841cc-90ec-42f0-bd0f-11ca5282af7a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.579816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1841cc-90ec-42f0-bd0f-11ca5282af7a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.599904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:01.643329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5953ebd-16f0-4f84-8bc7-70b0a1ee7a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.643941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953ebd-16f0-4f84-8bc7-70b0a1ee7a2a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.644217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953ebd-16f0-4f84-8bc7-70b0a1ee7a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.644486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953ebd-16f0-4f84-8bc7-70b0a1ee7a2a admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.645165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5953ebd-16f0-4f84-8bc7-70b0a1ee7a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.647073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5953ebd-16f0-4f84-8bc7-70b0a1ee7a2a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.673064 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4106/8246] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:29:01 2022] POST /identity/v3/users => generated 288 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:29:01.674332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50224b-9fe0-42c5-b07b-c53fbe5d6765 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.686284 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50224b-9fe0-42c5-b07b-c53fbe5d6765 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.690966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ed6f4-e3b3-43c8-85c8-e7772a2361a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.691560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ed6f4-e3b3-43c8-85c8-e7772a2361a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.691833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ed6f4-e3b3-43c8-85c8-e7772a2361a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.692126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ed6f4-e3b3-43c8-85c8-e7772a2361a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.692792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ed6f4-e3b3-43c8-85c8-e7772a2361a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.694409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ed6f4-e3b3-43c8-85c8-e7772a2361a1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.707161 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4141/8247] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:29:01.711288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5953ebd-16f0-4f84-8bc7-70b0a1ee7a2a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.722656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5953ebd-16f0-4f84-8bc7-70b0a1ee7a2a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.727680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7957916-db2d-4286-bb5f-5d2c19cd6c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.728285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7957916-db2d-4286-bb5f-5d2c19cd6c9f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.728565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7957916-db2d-4286-bb5f-5d2c19cd6c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.728924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7957916-db2d-4286-bb5f-5d2c19cd6c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.729950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7957916-db2d-4286-bb5f-5d2c19cd6c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.731889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7957916-db2d-4286-bb5f-5d2c19cd6c9f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.743343 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4107/8248] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:29:01.747899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ed6f4-e3b3-43c8-85c8-e7772a2361a1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.758611 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ed6f4-e3b3-43c8-85c8-e7772a2361a1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:01.764181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153494a7-9b19-4aed-9741-0febf781e898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:01.764972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153494a7-9b19-4aed-9741-0febf781e898 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.765309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153494a7-9b19-4aed-9741-0febf781e898 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.765582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153494a7-9b19-4aed-9741-0febf781e898 admin admin] PATH_INFO: `/v3/projects/e2bfe1e4fbfd48998884d5e8ef8a3e57/users/bd14dd0707a1423d82328ac3a54ccb28/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.782135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153494a7-9b19-4aed-9741-0febf781e898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2bfe1e4fbfd48998884d5e8ef8a3e57, user_id=bd14dd0707a1423d82328ac3a54ccb28, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:01.784792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153494a7-9b19-4aed-9741-0febf781e898 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:01.800200 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4142/8249] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:29:01 2022] PUT /identity/v3/projects/e2bfe1e4fbfd48998884d5e8ef8a3e57/users/bd14dd0707a1423d82328ac3a54ccb28/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:01.811564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39b66-e2a6-4eaf-88a4-b3a40cb8057e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:01.811847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39b66-e2a6-4eaf-88a4-b3a40cb8057e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:01.812120 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39b66-e2a6-4eaf-88a4-b3a40cb8057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:01.842932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9ce39b66-e2a6-4eaf-88a4-b3a40cb8057e None None] MFA Rules not processed for user `bd14dd0707a1423d82328ac3a54ccb28`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:01.844160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9ce39b66-e2a6-4eaf-88a4-b3a40cb8057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:01.923878 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4108/8250] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:01.928522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153494a7-9b19-4aed-9741-0febf781e898 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:01.986279 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:01.999978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153494a7-9b19-4aed-9741-0febf781e898 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:02.005067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624f949-0e41-49ba-8f94-f4f3d3f7bcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:02.005665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624f949-0e41-49ba-8f94-f4f3d3f7bcd7 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.005959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624f949-0e41-49ba-8f94-f4f3d3f7bcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.006297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624f949-0e41-49ba-8f94-f4f3d3f7bcd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.006934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624f949-0e41-49ba-8f94-f4f3d3f7bcd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:02.008818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624f949-0e41-49ba-8f94-f4f3d3f7bcd7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:02.040807 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4143/8251] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:29:01 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:29:02.042395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce39b66-e2a6-4eaf-88a4-b3a40cb8057e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:02.053115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce39b66-e2a6-4eaf-88a4-b3a40cb8057e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:02.114342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa1f65-d861-48da-bf18-478fd62300ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:02.114986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1f65-d861-48da-bf18-478fd62300ef admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.115270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1f65-d861-48da-bf18-478fd62300ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.115618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1f65-d861-48da-bf18-478fd62300ef admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.116314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa1f65-d861-48da-bf18-478fd62300ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:02.118685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa1f65-d861-48da-bf18-478fd62300ef admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:02.145367 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4109/8252] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:29:02 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:29:02.149259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624f949-0e41-49ba-8f94-f4f3d3f7bcd7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:02.161547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624f949-0e41-49ba-8f94-f4f3d3f7bcd7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:02.166931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34de59-6afe-46e5-9a8d-52f09c01f444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:02.167835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34de59-6afe-46e5-9a8d-52f09c01f444 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.168144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34de59-6afe-46e5-9a8d-52f09c01f444 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.168405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedfe3a-59e1-47bb-aac0-55ea7e1e3161 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.168513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34de59-6afe-46e5-9a8d-52f09c01f444 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.168704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedfe3a-59e1-47bb-aac0-55ea7e1e3161 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.168958 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedfe3a-59e1-47bb-aac0-55ea7e1e3161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.170953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34de59-6afe-46e5-9a8d-52f09c01f444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:02.172326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34de59-6afe-46e5-9a8d-52f09c01f444 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:02.188112 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4144/8253] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:29:02.189001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34de59-6afe-46e5-9a8d-52f09c01f444 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:02.203161 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8eedfe3a-59e1-47bb-aac0-55ea7e1e3161 None None] MFA Rules not processed for user `96f813ef9dad48b99493bd24c3f2a1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:02.204372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8eedfe3a-59e1-47bb-aac0-55ea7e1e3161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:02.213336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34de59-6afe-46e5-9a8d-52f09c01f444 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:02.213875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:02.219893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5050134-9194-4140-a171-53ba3075ef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:02.220868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5050134-9194-4140-a171-53ba3075ef05 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.221008 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5050134-9194-4140-a171-53ba3075ef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.221305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5050134-9194-4140-a171-53ba3075ef05 admin admin] PATH_INFO: `/v3/projects/00952acabdec4fa98e0a86743091655f/users/82154bd4c8164aeb9c010b8f506038bb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.238287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5050134-9194-4140-a171-53ba3075ef05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00952acabdec4fa98e0a86743091655f, user_id=82154bd4c8164aeb9c010b8f506038bb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:02.241713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5050134-9194-4140-a171-53ba3075ef05 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:02.257721 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4145/8254] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:29:02 2022] PUT /identity/v3/projects/00952acabdec4fa98e0a86743091655f/users/82154bd4c8164aeb9c010b8f506038bb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:02.262764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5050134-9194-4140-a171-53ba3075ef05 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:02.289235 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4110/8255] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:02.300363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc1e53-f0fb-43bc-b8f4-d626a3dcde34 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.300675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc1e53-f0fb-43bc-b8f4-d626a3dcde34 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.300945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc1e53-f0fb-43bc-b8f4-d626a3dcde34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.322243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ebcc1e53-f0fb-43bc-b8f4-d626a3dcde34 None None] MFA Rules not processed for user `3df1b449adeb42c58878a8344c9ddd66`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:02.324941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ebcc1e53-f0fb-43bc-b8f4-d626a3dcde34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:02.334869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5050134-9194-4140-a171-53ba3075ef05 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:02.340423 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6a60a-8929-4f3c-9f2d-ddbdbaa311a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:02.341044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6a60a-8929-4f3c-9f2d-ddbdbaa311a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.341338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6a60a-8929-4f3c-9f2d-ddbdbaa311a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.341607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6a60a-8929-4f3c-9f2d-ddbdbaa311a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.342278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6a60a-8929-4f3c-9f2d-ddbdbaa311a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:02.344471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6a60a-8929-4f3c-9f2d-ddbdbaa311a5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:02.360607 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4146/8256] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:29:02.365946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6a60a-8929-4f3c-9f2d-ddbdbaa311a5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:02.378366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6a60a-8929-4f3c-9f2d-ddbdbaa311a5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:02.383045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfe823-c8e9-489e-bef1-d562418d1c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:02.383694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfe823-c8e9-489e-bef1-d562418d1c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.384022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfe823-c8e9-489e-bef1-d562418d1c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.384286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfe823-c8e9-489e-bef1-d562418d1c75 admin admin] PATH_INFO: `/v3/projects/00952acabdec4fa98e0a86743091655f/users/82154bd4c8164aeb9c010b8f506038bb/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.388163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfe823-c8e9-489e-bef1-d562418d1c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00952acabdec4fa98e0a86743091655f, user_id=82154bd4c8164aeb9c010b8f506038bb, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:02.392893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfe823-c8e9-489e-bef1-d562418d1c75 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:02.404608 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4147/8257] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:29:02 2022] PUT /identity/v3/projects/00952acabdec4fa98e0a86743091655f/users/82154bd4c8164aeb9c010b8f506038bb/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:02.413486 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4111/8258] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:29:02.415578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbec5bb-8add-4a3a-9f83-f4a767855fe5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.415938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbec5bb-8add-4a3a-9f83-f4a767855fe5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.416210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbec5bb-8add-4a3a-9f83-f4a767855fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.425148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc1e53-f0fb-43bc-b8f4-d626a3dcde34 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:02.426517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ebcc1e53-f0fb-43bc-b8f4-d626a3dcde34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:02.448969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1bbec5bb-8add-4a3a-9f83-f4a767855fe5 None None] MFA Rules not processed for user `82154bd4c8164aeb9c010b8f506038bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:02.450213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1bbec5bb-8add-4a3a-9f83-f4a767855fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:02.520254 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4148/8259] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:02.523224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc1e53-f0fb-43bc-b8f4-d626a3dcde34 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:02.531705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbec5bb-8add-4a3a-9f83-f4a767855fe5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:02.533165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1bbec5bb-8add-4a3a-9f83-f4a767855fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:02.544022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:02.563718 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:02.590740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e202b2-78d6-40d3-9e78-27b4cb9a130c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:02.591343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e202b2-78d6-40d3-9e78-27b4cb9a130c service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.591628 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e202b2-78d6-40d3-9e78-27b4cb9a130c service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.591895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e202b2-78d6-40d3-9e78-27b4cb9a130c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.593690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e3e202b2-78d6-40d3-9e78-27b4cb9a130c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:02.596073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbec5bb-8add-4a3a-9f83-f4a767855fe5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:02.599743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7ebc9-115a-4d71-94a0-ad824733fce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:02.600347 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7ebc9-115a-4d71-94a0-ad824733fce6 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:02.600623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7ebc9-115a-4d71-94a0-ad824733fce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:02.600889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7ebc9-115a-4d71-94a0-ad824733fce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:02.653752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e202b2-78d6-40d3-9e78-27b4cb9a130c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:02.655348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e202b2-78d6-40d3-9e78-27b4cb9a130c service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:02.660490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7ebc9-115a-4d71-94a0-ad824733fce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:02.661962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7ebc9-115a-4d71-94a0-ad824733fce6 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:02.664278 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4112/8260] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:29:02.672684 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4149/8261] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:29:04.805781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e202b2-78d6-40d3-9e78-27b4cb9a130c service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:04.820058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e202b2-78d6-40d3-9e78-27b4cb9a130c service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:04.824894 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648b5c4-2288-4fdd-8a5e-cee8d9968366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:04.825548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648b5c4-2288-4fdd-8a5e-cee8d9968366 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:04.825821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648b5c4-2288-4fdd-8a5e-cee8d9968366 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:04.826086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648b5c4-2288-4fdd-8a5e-cee8d9968366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:04.836391 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648b5c4-2288-4fdd-8a5e-cee8d9968366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:04.837949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648b5c4-2288-4fdd-8a5e-cee8d9968366 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:04.849333 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4113/8262] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:29:10.021774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dfc4bb-076a-4a73-9f84-f07925ed697b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:10.022241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dfc4bb-076a-4a73-9f84-f07925ed697b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:10.022438 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dfc4bb-076a-4a73-9f84-f07925ed697b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:10.042738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-49dfc4bb-076a-4a73-9f84-f07925ed697b None None] MFA Rules not processed for user `bd14dd0707a1423d82328ac3a54ccb28`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:10.044287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-49dfc4bb-076a-4a73-9f84-f07925ed697b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:10.051251 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4150/8263] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:10.411479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648b5c4-2288-4fdd-8a5e-cee8d9968366 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:10.412957 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5648b5c4-2288-4fdd-8a5e-cee8d9968366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:10.428603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648b5c4-2288-4fdd-8a5e-cee8d9968366 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:10.433636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ca133-55d5-4279-8fce-cc0762e30c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:10.434249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ca133-55d5-4279-8fce-cc0762e30c45 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:10.434524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ca133-55d5-4279-8fce-cc0762e30c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:10.434791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ca133-55d5-4279-8fce-cc0762e30c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:10.436530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-347ca133-55d5-4279-8fce-cc0762e30c45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:10.449984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ca133-55d5-4279-8fce-cc0762e30c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:10.451424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ca133-55d5-4279-8fce-cc0762e30c45 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:10.462882 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4114/8264] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:29:18.906098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dfc4bb-076a-4a73-9f84-f07925ed697b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:18.918816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dfc4bb-076a-4a73-9f84-f07925ed697b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:18.922597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dd3c3-0d18-4364-b163-27531d1823b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:18.923190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dd3c3-0d18-4364-b163-27531d1823b3 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:18.923468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dd3c3-0d18-4364-b163-27531d1823b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:18.923735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dd3c3-0d18-4364-b163-27531d1823b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:18.925511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-585dd3c3-0d18-4364-b163-27531d1823b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:18.938609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dd3c3-0d18-4364-b163-27531d1823b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:18.940028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dd3c3-0d18-4364-b163-27531d1823b3 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:18.951310 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4151/8265] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:29:20.456210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=50 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:22.890510 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ca133-55d5-4279-8fce-cc0762e30c45 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:22.893272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-347ca133-55d5-4279-8fce-cc0762e30c45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:22.914617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ca133-55d5-4279-8fce-cc0762e30c45 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:22.920431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c47bea-63f8-450f-9dde-04d17a856404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:22.921057 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c47bea-63f8-450f-9dde-04d17a856404 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:22.921364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c47bea-63f8-450f-9dde-04d17a856404 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:22.921636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c47bea-63f8-450f-9dde-04d17a856404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:22.923426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-76c47bea-63f8-450f-9dde-04d17a856404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:22.940096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c47bea-63f8-450f-9dde-04d17a856404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:22.941766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c47bea-63f8-450f-9dde-04d17a856404 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:22.956297 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4115/8266] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:29:27.883983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dd3c3-0d18-4364-b163-27531d1823b3 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:27.885463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-585dd3c3-0d18-4364-b163-27531d1823b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:27.985377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dd3c3-0d18-4364-b163-27531d1823b3 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:27.990609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3d4a5-ba8d-4b07-b4c1-07b4ef99de5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:27.991467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d4a5-ba8d-4b07-b4c1-07b4ef99de5d service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:27.991746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d4a5-ba8d-4b07-b4c1-07b4ef99de5d service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:27.992016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d4a5-ba8d-4b07-b4c1-07b4ef99de5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:27.993904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-08e3d4a5-ba8d-4b07-b4c1-07b4ef99de5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:28.011283 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3d4a5-ba8d-4b07-b4c1-07b4ef99de5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:28.013147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3d4a5-ba8d-4b07-b4c1-07b4ef99de5d service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:28.026040 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4152/8267] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:29:28.468651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c47bea-63f8-450f-9dde-04d17a856404 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:28.533429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c47bea-63f8-450f-9dde-04d17a856404 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:28.537895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:28.538511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:28.538785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:28.539053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] PATH_INFO: `/v3/users/3df1b449adeb42c58878a8344c9ddd66` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:28.540149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3df1b449adeb42c58878a8344c9ddd66)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:28.542618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:28.587559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3df1b449adeb42c58878a8344c9ddd66'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:28.599473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3df1b449adeb42c58878a8344c9ddd66'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:28.606750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3df1b449adeb42c58878a8344c9ddd66'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:28.613683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3df1b449adeb42c58878a8344c9ddd66'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:28.614309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3df1b449adeb42c58878a8344c9ddd66'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:28.620611 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4116/8268] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:29:28 2022] DELETE /identity/v3/users/3df1b449adeb42c58878a8344c9ddd66 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:28.792565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3d4a5-ba8d-4b07-b4c1-07b4ef99de5d service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:28.803646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3d4a5-ba8d-4b07-b4c1-07b4ef99de5d service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:28.825264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:28.862062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:28.862688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:28.862962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:28.863231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] PATH_INFO: `/v3/projects/a82cb785a32d473db8e973a212715528` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:28.864334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a82cb785a32d473db8e973a212715528)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:28.866522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:28.912612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a82cb785a32d473db8e973a212715528', 'request_id': 'req-2877fdc4-98b3-4f48-bbec-f142d2538b0f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:28.919874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a82cb785a32d473db8e973a212715528', 'request_id': 'req-2877fdc4-98b3-4f48-bbec-f142d2538b0f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:28.926214 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a82cb785a32d473db8e973a212715528', 'request_id': 'req-2877fdc4-98b3-4f48-bbec-f142d2538b0f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:28.937224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] The token cache is being invalidate because project a82cb785a32d473db8e973a212715528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:29:28.937514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:28.938162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:28.939689 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4153/8269] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:29:28 2022] DELETE /identity/v3/projects/a82cb785a32d473db8e973a212715528 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:28.943773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:28.945189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:28.959474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b8ba4-eb89-430d-be92-d75744bf9a74 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:29.016115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:29.016732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.017005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.017300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] PATH_INFO: `/v3/users/96f813ef9dad48b99493bd24c3f2a1d7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:29.018389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f813ef9dad48b99493bd24c3f2a1d7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:29.020295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:29.025529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:29.067994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f813ef9dad48b99493bd24c3f2a1d7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:29.080943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f813ef9dad48b99493bd24c3f2a1d7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:29.088442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f813ef9dad48b99493bd24c3f2a1d7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:29.099876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f813ef9dad48b99493bd24c3f2a1d7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:29.100585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-737bda99-3860-4a17-bbaa-0d64f28e43eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f813ef9dad48b99493bd24c3f2a1d7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:29.107449 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4117/8270] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:29:28 2022] DELETE /identity/v3/users/96f813ef9dad48b99493bd24c3f2a1d7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:29.288713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:29.290291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:29.305153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877fdc4-98b3-4f48-bbec-f142d2538b0f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:29.363306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:29.363928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.364196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.364458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] PATH_INFO: `/v3/projects/d4e1b847f5c44a65a7daca0fb163c7f9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:29.365585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e1b847f5c44a65a7daca0fb163c7f9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:29.367729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:29.414522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e1b847f5c44a65a7daca0fb163c7f9', 'request_id': 'req-6b243dd9-481d-457c-a498-92bcbcea6ff7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:29.421929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e1b847f5c44a65a7daca0fb163c7f9', 'request_id': 'req-6b243dd9-481d-457c-a498-92bcbcea6ff7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:29.423051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:29.429813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e1b847f5c44a65a7daca0fb163c7f9', 'request_id': 'req-6b243dd9-481d-457c-a498-92bcbcea6ff7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:29.441110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] The token cache is being invalidate because project d4e1b847f5c44a65a7daca0fb163c7f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:29:29.441394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:29.442025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-6b243dd9-481d-457c-a498-92bcbcea6ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:29:29.443579 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4154/8271] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:29:29 2022] DELETE /identity/v3/projects/d4e1b847f5c44a65a7daca0fb163c7f9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:29.449404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffe9e3-5b5f-45f2-8244-aa14d532ca0b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.449683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffe9e3-5b5f-45f2-8244-aa14d532ca0b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.449946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffe9e3-5b5f-45f2-8244-aa14d532ca0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:29.469742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-52ffe9e3-5b5f-45f2-8244-aa14d532ca0b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:29.470993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-52ffe9e3-5b5f-45f2-8244-aa14d532ca0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:29.530234 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4118/8272] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:29.650413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377fae-37a5-438f-bf3b-2320e4d142a7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.650688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377fae-37a5-438f-bf3b-2320e4d142a7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.650956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377fae-37a5-438f-bf3b-2320e4d142a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:29.670482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-f5377fae-37a5-438f-bf3b-2320e4d142a7 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:29.671705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f5377fae-37a5-438f-bf3b-2320e4d142a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:29.679639 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4155/8273] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:29.684681 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffe9e3-5b5f-45f2-8244-aa14d532ca0b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:29.694568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffe9e3-5b5f-45f2-8244-aa14d532ca0b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:29.699788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafd145-8f86-4266-a69b-a0cec20645a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:29.700429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafd145-8f86-4266-a69b-a0cec20645a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.700787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafd145-8f86-4266-a69b-a0cec20645a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.700966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafd145-8f86-4266-a69b-a0cec20645a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:29.701601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafd145-8f86-4266-a69b-a0cec20645a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:29:29.701899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafd145-8f86-4266-a69b-a0cec20645a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:29.703495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafd145-8f86-4266-a69b-a0cec20645a9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:29.715238 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4119/8274] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:29:29.719587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5377fae-37a5-438f-bf3b-2320e4d142a7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:29.728771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5377fae-37a5-438f-bf3b-2320e4d142a7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:29.733459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd5887-597f-4ce5-8c67-08f33899a6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:29.734035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd5887-597f-4ce5-8c67-08f33899a6be admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.734303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd5887-597f-4ce5-8c67-08f33899a6be admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.734562 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd5887-597f-4ce5-8c67-08f33899a6be admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:29.735197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd5887-597f-4ce5-8c67-08f33899a6be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:29.737068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd5887-597f-4ce5-8c67-08f33899a6be admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:29.765025 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4156/8275] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:29:29 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:29:29.768881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafd145-8f86-4266-a69b-a0cec20645a9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:29.778769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafd145-8f86-4266-a69b-a0cec20645a9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:29.834943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d8d61-81c3-438f-af18-ec6afebf5023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:29.835559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8d61-81c3-438f-af18-ec6afebf5023 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.835846 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8d61-81c3-438f-af18-ec6afebf5023 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.836108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8d61-81c3-438f-af18-ec6afebf5023 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:29.836717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d8d61-81c3-438f-af18-ec6afebf5023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:29.838660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d8d61-81c3-438f-af18-ec6afebf5023 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:29.860736 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4120/8276] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:29:29 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:29:29.864859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd5887-597f-4ce5-8c67-08f33899a6be admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:29.876462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd5887-597f-4ce5-8c67-08f33899a6be admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:29.881149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47837f37-e75d-498a-b5ee-8ef5ff71324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:29.881736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47837f37-e75d-498a-b5ee-8ef5ff71324d admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.882011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47837f37-e75d-498a-b5ee-8ef5ff71324d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.882278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47837f37-e75d-498a-b5ee-8ef5ff71324d admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:29.882906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47837f37-e75d-498a-b5ee-8ef5ff71324d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:29.884514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47837f37-e75d-498a-b5ee-8ef5ff71324d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:29.894578 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4157/8277] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:29:29.898619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d8d61-81c3-438f-af18-ec6afebf5023 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:29.903911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:29.910274 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d8d61-81c3-438f-af18-ec6afebf5023 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:29.915880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da733ef-69d5-40aa-b7e3-c139c5640dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:29.916493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da733ef-69d5-40aa-b7e3-c139c5640dbd admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.916764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da733ef-69d5-40aa-b7e3-c139c5640dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.917588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da733ef-69d5-40aa-b7e3-c139c5640dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:29.918235 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da733ef-69d5-40aa-b7e3-c139c5640dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:29.919857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da733ef-69d5-40aa-b7e3-c139c5640dbd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:29.930366 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4121/8278] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:29:29.934994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47837f37-e75d-498a-b5ee-8ef5ff71324d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:29.945010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47837f37-e75d-498a-b5ee-8ef5ff71324d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:29.950758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964eb581-11ce-4e9d-8bed-6fa444e9a140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:29.951395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eb581-11ce-4e9d-8bed-6fa444e9a140 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.951670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eb581-11ce-4e9d-8bed-6fa444e9a140 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.951939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eb581-11ce-4e9d-8bed-6fa444e9a140 admin admin] PATH_INFO: `/v3/projects/6c1026ed03744eaaa999b2d54aac2250/users/32a68a0c1d8f45149c42699b2367f108/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:29.967053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964eb581-11ce-4e9d-8bed-6fa444e9a140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1026ed03744eaaa999b2d54aac2250, user_id=32a68a0c1d8f45149c42699b2367f108, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:29.969742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964eb581-11ce-4e9d-8bed-6fa444e9a140 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:29.982706 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4158/8279] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:29:29 2022] PUT /identity/v3/projects/6c1026ed03744eaaa999b2d54aac2250/users/32a68a0c1d8f45149c42699b2367f108/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:29.987693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90e645-e8b7-4607-b51d-7ad53515066c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:29.987971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90e645-e8b7-4607-b51d-7ad53515066c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:29.988264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90e645-e8b7-4607-b51d-7ad53515066c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:30.008845 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-2e90e645-e8b7-4607-b51d-7ad53515066c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:30.010053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2e90e645-e8b7-4607-b51d-7ad53515066c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:30.068314 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4122/8280] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:30.077935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964eb581-11ce-4e9d-8bed-6fa444e9a140 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:30.079627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-964eb581-11ce-4e9d-8bed-6fa444e9a140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:30.140379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964eb581-11ce-4e9d-8bed-6fa444e9a140 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:30.144051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e769e-f1ef-4706-80e1-9aba41809c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:30.144654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e769e-f1ef-4706-80e1-9aba41809c6a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:30.144929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e769e-f1ef-4706-80e1-9aba41809c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:30.145263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e769e-f1ef-4706-80e1-9aba41809c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:30.154880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e769e-f1ef-4706-80e1-9aba41809c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:30.156232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e769e-f1ef-4706-80e1-9aba41809c6a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:30.165759 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4159/8281] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:29:34.307761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca29b32-6f39-4ec2-bc35-07589027b03c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:34.308823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca29b32-6f39-4ec2-bc35-07589027b03c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:34.309126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca29b32-6f39-4ec2-bc35-07589027b03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:34.345572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0ca29b32-6f39-4ec2-bc35-07589027b03c None None] MFA Rules not processed for user `32a68a0c1d8f45149c42699b2367f108`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:34.346948 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0ca29b32-6f39-4ec2-bc35-07589027b03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:34.419851 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4123/8282] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:34.424449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e769e-f1ef-4706-80e1-9aba41809c6a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:34.434797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e769e-f1ef-4706-80e1-9aba41809c6a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:34.440040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91957b-06f2-49fd-90db-56eaf4fe6cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:34.440170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91957b-06f2-49fd-90db-56eaf4fe6cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:34.440450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91957b-06f2-49fd-90db-56eaf4fe6cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:34.440698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91957b-06f2-49fd-90db-56eaf4fe6cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:34.441346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91957b-06f2-49fd-90db-56eaf4fe6cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:34.443312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91957b-06f2-49fd-90db-56eaf4fe6cc2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:34.491818 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4160/8283] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:29:34 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:29:34.496167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca29b32-6f39-4ec2-bc35-07589027b03c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:34.513142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca29b32-6f39-4ec2-bc35-07589027b03c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:34.583929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0cc37-c709-43e0-9190-81cb1ab2a571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:34.584642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0cc37-c709-43e0-9190-81cb1ab2a571 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:34.584984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0cc37-c709-43e0-9190-81cb1ab2a571 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:34.585349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0cc37-c709-43e0-9190-81cb1ab2a571 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:34.586096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0cc37-c709-43e0-9190-81cb1ab2a571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:34.588131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0cc37-c709-43e0-9190-81cb1ab2a571 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:34.620102 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4124/8284] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:29:34 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:29:34.622113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91957b-06f2-49fd-90db-56eaf4fe6cc2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:34.633104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91957b-06f2-49fd-90db-56eaf4fe6cc2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:34.644971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8b106-44f2-4224-9a3b-54e7a6450d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:34.645808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8b106-44f2-4224-9a3b-54e7a6450d10 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:34.646568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8b106-44f2-4224-9a3b-54e7a6450d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:34.646568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8b106-44f2-4224-9a3b-54e7a6450d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:34.646988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8b106-44f2-4224-9a3b-54e7a6450d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:34.649966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8b106-44f2-4224-9a3b-54e7a6450d10 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:34.662236 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4161/8285] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:29:34.666411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0cc37-c709-43e0-9190-81cb1ab2a571 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:34.677339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0cc37-c709-43e0-9190-81cb1ab2a571 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:34.682557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508d0cb-169f-43ec-ba95-647674b6f11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:34.683780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508d0cb-169f-43ec-ba95-647674b6f11f admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:34.684052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508d0cb-169f-43ec-ba95-647674b6f11f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:34.684417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508d0cb-169f-43ec-ba95-647674b6f11f admin admin] PATH_INFO: `/v3/projects/2935f96dd64e43c787b1a87ee6bae4f6/users/4bd4c58eea264d7e81d63c593a46122f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:34.694619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:34.703684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508d0cb-169f-43ec-ba95-647674b6f11f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2935f96dd64e43c787b1a87ee6bae4f6, user_id=4bd4c58eea264d7e81d63c593a46122f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:34.707556 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508d0cb-169f-43ec-ba95-647674b6f11f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:34.722647 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4125/8286] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:29:34 2022] PUT /identity/v3/projects/2935f96dd64e43c787b1a87ee6bae4f6/users/4bd4c58eea264d7e81d63c593a46122f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:34.727093 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8b106-44f2-4224-9a3b-54e7a6450d10 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:34.739415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8b106-44f2-4224-9a3b-54e7a6450d10 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:34.790889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:34.803577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244014d-06c0-4dcc-855f-c68c7c3fd8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:34.804246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244014d-06c0-4dcc-855f-c68c7c3fd8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:34.805348 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244014d-06c0-4dcc-855f-c68c7c3fd8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:34.805781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244014d-06c0-4dcc-855f-c68c7c3fd8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:34.806475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244014d-06c0-4dcc-855f-c68c7c3fd8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:34.808304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244014d-06c0-4dcc-855f-c68c7c3fd8f4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:34.818763 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4162/8287] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:29:34.823239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508d0cb-169f-43ec-ba95-647674b6f11f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:34.834365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508d0cb-169f-43ec-ba95-647674b6f11f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:34.839731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ff8b0-5592-4af9-9a8b-587a07066584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:34.840397 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ff8b0-5592-4af9-9a8b-587a07066584 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:34.840672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ff8b0-5592-4af9-9a8b-587a07066584 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:34.840940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ff8b0-5592-4af9-9a8b-587a07066584 admin admin] PATH_INFO: `/v3/projects/2935f96dd64e43c787b1a87ee6bae4f6/users/4bd4c58eea264d7e81d63c593a46122f/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:34.842935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ff8b0-5592-4af9-9a8b-587a07066584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2935f96dd64e43c787b1a87ee6bae4f6, user_id=4bd4c58eea264d7e81d63c593a46122f, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:34.845674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ff8b0-5592-4af9-9a8b-587a07066584 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:34.857617 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4126/8288] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:29:34 2022] PUT /identity/v3/projects/2935f96dd64e43c787b1a87ee6bae4f6/users/4bd4c58eea264d7e81d63c593a46122f/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:29:39.320777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb8817-c7ef-4519-9a18-da7320694344 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:39.321305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb8817-c7ef-4519-9a18-da7320694344 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:39.321464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb8817-c7ef-4519-9a18-da7320694344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:39.353650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-18fb8817-c7ef-4519-9a18-da7320694344 None None] MFA Rules not processed for user `4bd4c58eea264d7e81d63c593a46122f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:39.354894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-18fb8817-c7ef-4519-9a18-da7320694344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:39.428512 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4163/8289] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:39.444594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ff8b0-5592-4af9-9a8b-587a07066584 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:39.446603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4a6ff8b0-5592-4af9-9a8b-587a07066584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:39.531136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ff8b0-5592-4af9-9a8b-587a07066584 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:39.534883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13142a-4d31-4eb7-86e6-342c3017c83f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:39.535482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13142a-4d31-4eb7-86e6-342c3017c83f service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:39.535879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13142a-4d31-4eb7-86e6-342c3017c83f service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:39.536011 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13142a-4d31-4eb7-86e6-342c3017c83f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:39.603963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13142a-4d31-4eb7-86e6-342c3017c83f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:39.606014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13142a-4d31-4eb7-86e6-342c3017c83f service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:39.616776 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4127/8290] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:29:49.425153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=40 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:49.611914 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=67,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:57.945182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bf2c2-750d-496c-b912-62874e5150b7 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:57.945747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bf2c2-750d-496c-b912-62874e5150b7 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:57.945747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bf2c2-750d-496c-b912-62874e5150b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:57.973281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fb4bf2c2-750d-496c-b912-62874e5150b7 None None] MFA Rules not processed for user `bd14dd0707a1423d82328ac3a54ccb28`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:57.974507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fb4bf2c2-750d-496c-b912-62874e5150b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:58.040461 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4164/8291] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:58.860512 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c540c-cdd1-4b6d-95ab-ef35fffd296b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:58.862911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c540c-cdd1-4b6d-95ab-ef35fffd296b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:58.863359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c540c-cdd1-4b6d-95ab-ef35fffd296b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:58.894393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f45c540c-cdd1-4b6d-95ab-ef35fffd296b None None] MFA Rules not processed for user `32a68a0c1d8f45149c42699b2367f108`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:29:58.895225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bf2c2-750d-496c-b912-62874e5150b7 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:29:58.895662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f45c540c-cdd1-4b6d-95ab-ef35fffd296b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:58.897292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fb4bf2c2-750d-496c-b912-62874e5150b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:58.904799 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4128/8292] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:29:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:29:59.027519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bf2c2-750d-496c-b912-62874e5150b7 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:29:59.045966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db820c-93dd-4f08-baff-d348bfa4208d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:29:59.046758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db820c-93dd-4f08-baff-d348bfa4208d service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:29:59.047132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db820c-93dd-4f08-baff-d348bfa4208d service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:29:59.047525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db820c-93dd-4f08-baff-d348bfa4208d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:29:59.050060 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76db820c-93dd-4f08-baff-d348bfa4208d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:29:59.066828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db820c-93dd-4f08-baff-d348bfa4208d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:29:59.068600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db820c-93dd-4f08-baff-d348bfa4208d service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:29:59.085953 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4165/8293] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:29:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:30:01.093343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c540c-cdd1-4b6d-95ab-ef35fffd296b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:30:01.174697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c540c-cdd1-4b6d-95ab-ef35fffd296b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:30:01.179058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7338a-310e-4e3e-b183-0a39c4c4c896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:30:01.180019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7338a-310e-4e3e-b183-0a39c4c4c896 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:30:01.180449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7338a-310e-4e3e-b183-0a39c4c4c896 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:30:01.180867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7338a-310e-4e3e-b183-0a39c4c4c896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:30:01.182971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-06d7338a-310e-4e3e-b183-0a39c4c4c896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:30:01.211986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7338a-310e-4e3e-b183-0a39c4c4c896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:30:01.213605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7338a-310e-4e3e-b183-0a39c4c4c896 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:30:01.227238 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4129/8294] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:30:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:30:03.039694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db820c-93dd-4f08-baff-d348bfa4208d service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:30:03.041368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76db820c-93dd-4f08-baff-d348bfa4208d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:30:03.120394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db820c-93dd-4f08-baff-d348bfa4208d service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:30:03.125923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ab17d-40fa-4634-9ca2-f3597283036f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:30:03.126534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ab17d-40fa-4634-9ca2-f3597283036f service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:30:03.126792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ab17d-40fa-4634-9ca2-f3597283036f service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:30:03.127539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ab17d-40fa-4634-9ca2-f3597283036f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:30:03.129560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b74ab17d-40fa-4634-9ca2-f3597283036f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:30:03.145687 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ab17d-40fa-4634-9ca2-f3597283036f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:30:03.147260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ab17d-40fa-4634-9ca2-f3597283036f service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:30:03.161180 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4166/8295] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:30:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:30:09.875546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7338a-310e-4e3e-b183-0a39c4c4c896 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:30:09.892697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7338a-310e-4e3e-b183-0a39c4c4c896 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:30:09.899150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f367c-6458-4425-9f0e-29502e93c6f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:30:09.899779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f367c-6458-4425-9f0e-29502e93c6f6 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:30:09.900194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f367c-6458-4425-9f0e-29502e93c6f6 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:30:09.900493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f367c-6458-4425-9f0e-29502e93c6f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:30:09.902344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3b2f367c-6458-4425-9f0e-29502e93c6f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:30:09.917412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f367c-6458-4425-9f0e-29502e93c6f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:30:09.919068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f367c-6458-4425-9f0e-29502e93c6f6 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:30:09.933340 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4130/8296] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:30:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:30:13.153671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:30:14.433811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ab17d-40fa-4634-9ca2-f3597283036f service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:30:14.446302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ab17d-40fa-4634-9ca2-f3597283036f service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:30:14.449861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c0d2b-cc18-408d-97b7-64529f9304ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:30:14.450453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c0d2b-cc18-408d-97b7-64529f9304ee service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:30:14.450726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c0d2b-cc18-408d-97b7-64529f9304ee service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:30:14.450992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c0d2b-cc18-408d-97b7-64529f9304ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:30:14.460909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c0d2b-cc18-408d-97b7-64529f9304ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:30:14.462358 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c0d2b-cc18-408d-97b7-64529f9304ee service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:30:14.472095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4167/8297] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:30:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:30:19.927986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=35 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:30:24.467851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:30:48.849906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f367c-6458-4425-9f0e-29502e93c6f6 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:30:48.867638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f367c-6458-4425-9f0e-29502e93c6f6 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:30:48.873068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7aebd-fb85-4fef-b22b-76a614810b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:30:48.873802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7aebd-fb85-4fef-b22b-76a614810b85 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:30:48.874139 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7aebd-fb85-4fef-b22b-76a614810b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:30:48.874384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7aebd-fb85-4fef-b22b-76a614810b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:30:48.876663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-14c7aebd-fb85-4fef-b22b-76a614810b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:30:48.907073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7aebd-fb85-4fef-b22b-76a614810b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:30:48.909475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7aebd-fb85-4fef-b22b-76a614810b85 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:30:48.928522 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4131/8298] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:30:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:30:58.920531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:06.456027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90ab12-9ce4-487c-a0ee-fb9b2a1dbcd4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:06.456541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90ab12-9ce4-487c-a0ee-fb9b2a1dbcd4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:06.456783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90ab12-9ce4-487c-a0ee-fb9b2a1dbcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:06.476427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-9f90ab12-9ce4-487c-a0ee-fb9b2a1dbcd4 None None] MFA Rules not processed for user `bd14dd0707a1423d82328ac3a54ccb28`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:31:06.477736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9f90ab12-9ce4-487c-a0ee-fb9b2a1dbcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:06.485657 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4168/8299] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:31:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:31:14.476466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7aebd-fb85-4fef-b22b-76a614810b85 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:14.493714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7aebd-fb85-4fef-b22b-76a614810b85 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:14.499725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95125e46-03d3-447d-b34a-5e196954824a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:14.500553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95125e46-03d3-447d-b34a-5e196954824a service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:14.500999 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95125e46-03d3-447d-b34a-5e196954824a service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:14.501440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95125e46-03d3-447d-b34a-5e196954824a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:14.503359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-95125e46-03d3-447d-b34a-5e196954824a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:14.543603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95125e46-03d3-447d-b34a-5e196954824a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:14.545533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95125e46-03d3-447d-b34a-5e196954824a service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:14.565257 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4132/8300] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:31:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:31:16.077533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90ab12-9ce4-487c-a0ee-fb9b2a1dbcd4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:16.151829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90ab12-9ce4-487c-a0ee-fb9b2a1dbcd4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:16.156453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:16.157107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:16.157386 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:16.157652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] PATH_INFO: `/v3/users/32a68a0c1d8f45149c42699b2367f108` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:16.158768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a68a0c1d8f45149c42699b2367f108)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:16.162164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:16.227780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a68a0c1d8f45149c42699b2367f108'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.235592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a68a0c1d8f45149c42699b2367f108'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.255494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a68a0c1d8f45149c42699b2367f108'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.263244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a68a0c1d8f45149c42699b2367f108'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.264616 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a68a0c1d8f45149c42699b2367f108'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.272109 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4169/8301] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:31:16 2022] DELETE /identity/v3/users/32a68a0c1d8f45149c42699b2367f108 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:31:16.451722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95125e46-03d3-447d-b34a-5e196954824a service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:16.463769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95125e46-03d3-447d-b34a-5e196954824a service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:16.526701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:16.527261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:16.527547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:16.527814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] PATH_INFO: `/v3/projects/6c1026ed03744eaaa999b2d54aac2250` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:16.528995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1026ed03744eaaa999b2d54aac2250)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:16.530983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:16.592393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1026ed03744eaaa999b2d54aac2250', 'request_id': 'req-608b1b52-5a45-4d3f-b9a3-ae8319cde707'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.601154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1026ed03744eaaa999b2d54aac2250', 'request_id': 'req-608b1b52-5a45-4d3f-b9a3-ae8319cde707'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.609693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1026ed03744eaaa999b2d54aac2250', 'request_id': 'req-608b1b52-5a45-4d3f-b9a3-ae8319cde707'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.622961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] The token cache is being invalidate because project 6c1026ed03744eaaa999b2d54aac2250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:31:16.623248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.623917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.630393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:16.631209 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4133/8302] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:31:16 2022] DELETE /identity/v3/projects/6c1026ed03744eaaa999b2d54aac2250 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:31:16.632089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:16.646643 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060b404-c97e-496b-af9f-9e7f4666e2dd admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:16.711262 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:16.711520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:16.711729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:16.712053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] PATH_INFO: `/v3/users/4bd4c58eea264d7e81d63c593a46122f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:16.713817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd4c58eea264d7e81d63c593a46122f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:16.715405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:16.731847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:16.769846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd4c58eea264d7e81d63c593a46122f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.776979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd4c58eea264d7e81d63c593a46122f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.790454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd4c58eea264d7e81d63c593a46122f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.802787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd4c58eea264d7e81d63c593a46122f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.803404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-d4ac2613-ded0-4b80-beeb-ada0b690e8ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd4c58eea264d7e81d63c593a46122f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:16.809781 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4170/8303] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:31:16 2022] DELETE /identity/v3/users/4bd4c58eea264d7e81d63c593a46122f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:31:16.984187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:16.985633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:16.999478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b1b52-5a45-4d3f-b9a3-ae8319cde707 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:17.060157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:17.060727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.061001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.061298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] PATH_INFO: `/v3/projects/2935f96dd64e43c787b1a87ee6bae4f6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.062412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2935f96dd64e43c787b1a87ee6bae4f6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:17.064350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:17.085893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:17.111514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2935f96dd64e43c787b1a87ee6bae4f6', 'request_id': 'req-d4d6235b-80ca-4489-af53-3cec76198cbe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:17.119617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2935f96dd64e43c787b1a87ee6bae4f6', 'request_id': 'req-d4d6235b-80ca-4489-af53-3cec76198cbe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:17.126426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2935f96dd64e43c787b1a87ee6bae4f6', 'request_id': 'req-d4d6235b-80ca-4489-af53-3cec76198cbe'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:17.137171 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] The token cache is being invalidate because project 2935f96dd64e43c787b1a87ee6bae4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:31:17.137449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:17.138072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-d4d6235b-80ca-4489-af53-3cec76198cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:31:17.139534 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4134/8304] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:31:16 2022] DELETE /identity/v3/projects/2935f96dd64e43c787b1a87ee6bae4f6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:31:17.144888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd76c5-10c3-4b6e-8ee4-b79d6a19215b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.145403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd76c5-10c3-4b6e-8ee4-b79d6a19215b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.145496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd76c5-10c3-4b6e-8ee4-b79d6a19215b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.165247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-34cd76c5-10c3-4b6e-8ee4-b79d6a19215b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:31:17.166466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-34cd76c5-10c3-4b6e-8ee4-b79d6a19215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:17.226231 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4171/8305] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:31:17.236474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4162269-4325-417f-9deb-214d4fb7774c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.236758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4162269-4325-417f-9deb-214d4fb7774c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.237054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4162269-4325-417f-9deb-214d4fb7774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.261690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e4162269-4325-417f-9deb-214d4fb7774c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:31:17.262970 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e4162269-4325-417f-9deb-214d4fb7774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:17.271243 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4135/8306] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:31:17.275550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd76c5-10c3-4b6e-8ee4-b79d6a19215b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:17.286006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd76c5-10c3-4b6e-8ee4-b79d6a19215b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:17.290516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e96c06-d7fb-4696-998f-302260682135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:17.291101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e96c06-d7fb-4696-998f-302260682135 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.291376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e96c06-d7fb-4696-998f-302260682135 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.291640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e96c06-d7fb-4696-998f-302260682135 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.292273 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e96c06-d7fb-4696-998f-302260682135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:31:17.292567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e96c06-d7fb-4696-998f-302260682135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:17.294169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e96c06-d7fb-4696-998f-302260682135 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:17.306531 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4172/8307] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:31:17.310919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4162269-4325-417f-9deb-214d4fb7774c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:17.320146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4162269-4325-417f-9deb-214d4fb7774c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:17.324814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb583b-c509-4128-8273-ae808f60a085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:17.325434 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb583b-c509-4128-8273-ae808f60a085 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.325708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb583b-c509-4128-8273-ae808f60a085 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.325974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb583b-c509-4128-8273-ae808f60a085 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.326581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb583b-c509-4128-8273-ae808f60a085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:17.328484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb583b-c509-4128-8273-ae808f60a085 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:17.356338 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4136/8308] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:31:17 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:31:17.360573 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e96c06-d7fb-4696-998f-302260682135 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:17.371688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e96c06-d7fb-4696-998f-302260682135 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:17.429539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7861b3-0494-4bba-9d08-ba176ef7a547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:17.430136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7861b3-0494-4bba-9d08-ba176ef7a547 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.430406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7861b3-0494-4bba-9d08-ba176ef7a547 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.430671 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7861b3-0494-4bba-9d08-ba176ef7a547 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.431276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7861b3-0494-4bba-9d08-ba176ef7a547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:17.433255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7861b3-0494-4bba-9d08-ba176ef7a547 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:17.455810 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4173/8309] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:31:17 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:31:17.459365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb583b-c509-4128-8273-ae808f60a085 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:17.469560 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb583b-c509-4128-8273-ae808f60a085 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:17.478180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce8260-f492-4c39-84e0-88065e164ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:17.478998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce8260-f492-4c39-84e0-88065e164ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.479276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce8260-f492-4c39-84e0-88065e164ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.479544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce8260-f492-4c39-84e0-88065e164ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.480175 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce8260-f492-4c39-84e0-88065e164ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:17.481809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce8260-f492-4c39-84e0-88065e164ac5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:17.492221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4137/8310] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:31:17.496236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7861b3-0494-4bba-9d08-ba176ef7a547 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:17.508047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7861b3-0494-4bba-9d08-ba176ef7a547 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:17.512697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69a1e6-869e-4e77-80ff-ff5e30ca7e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:17.513324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69a1e6-869e-4e77-80ff-ff5e30ca7e34 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.513595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69a1e6-869e-4e77-80ff-ff5e30ca7e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.513860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69a1e6-869e-4e77-80ff-ff5e30ca7e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.514508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69a1e6-869e-4e77-80ff-ff5e30ca7e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:17.516074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69a1e6-869e-4e77-80ff-ff5e30ca7e34 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:17.518286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:17.530473 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4174/8311] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:31:17.534448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce8260-f492-4c39-84e0-88065e164ac5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:17.544789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce8260-f492-4c39-84e0-88065e164ac5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:17.549443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e107-4efd-4bb7-a6c2-3b9a95efd5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:17.550072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e107-4efd-4bb7-a6c2-3b9a95efd5ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.550409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e107-4efd-4bb7-a6c2-3b9a95efd5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.550740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e107-4efd-4bb7-a6c2-3b9a95efd5ff admin admin] PATH_INFO: `/v3/projects/b11f869e7a3e49e59e3236788206474b/users/32a6ad0f3d3b44d48df46e91d80abc95/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.567547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032e107-4efd-4bb7-a6c2-3b9a95efd5ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11f869e7a3e49e59e3236788206474b, user_id=32a6ad0f3d3b44d48df46e91d80abc95, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:17.570147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032e107-4efd-4bb7-a6c2-3b9a95efd5ff admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:17.584849 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4138/8312] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:31:17 2022] PUT /identity/v3/projects/b11f869e7a3e49e59e3236788206474b/users/32a6ad0f3d3b44d48df46e91d80abc95/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:31:17.589988 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98ba9e-dd46-4012-afad-426e9239886b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.590267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98ba9e-dd46-4012-afad-426e9239886b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.590531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98ba9e-dd46-4012-afad-426e9239886b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.618447 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-df98ba9e-dd46-4012-afad-426e9239886b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:31:17.619632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-df98ba9e-dd46-4012-afad-426e9239886b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:17.688165 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4175/8313] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:31:17.698084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e107-4efd-4bb7-a6c2-3b9a95efd5ff admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:17.699765 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0032e107-4efd-4bb7-a6c2-3b9a95efd5ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:17.761488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e107-4efd-4bb7-a6c2-3b9a95efd5ff admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:17.765080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad57f86-c6d8-48ec-aa20-a8c13cdaf857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:17.765677 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad57f86-c6d8-48ec-aa20-a8c13cdaf857 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:17.765951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad57f86-c6d8-48ec-aa20-a8c13cdaf857 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:17.766218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad57f86-c6d8-48ec-aa20-a8c13cdaf857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:17.775650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad57f86-c6d8-48ec-aa20-a8c13cdaf857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:17.777110 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad57f86-c6d8-48ec-aa20-a8c13cdaf857 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:17.787172 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4139/8314] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:31:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:31:21.768637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98ba9e-dd46-4012-afad-426e9239886b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:21.771884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-df98ba9e-dd46-4012-afad-426e9239886b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:21.899862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98ba9e-dd46-4012-afad-426e9239886b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:21.913670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5f06b-d8d5-490d-8f23-6ac462b7d016 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:21.914820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f06b-d8d5-490d-8f23-6ac462b7d016 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:21.915477 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f06b-d8d5-490d-8f23-6ac462b7d016 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:21.915967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f06b-d8d5-490d-8f23-6ac462b7d016 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:21.918679 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-63c5f06b-d8d5-490d-8f23-6ac462b7d016 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:21.940430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5f06b-d8d5-490d-8f23-6ac462b7d016 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:21.944046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5f06b-d8d5-490d-8f23-6ac462b7d016 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:21.975957 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4176/8315] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:31:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:31:22.795059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77c1e4-99dd-4431-b38c-16ec2ce35a41 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:22.795532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77c1e4-99dd-4431-b38c-16ec2ce35a41 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:22.795627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77c1e4-99dd-4431-b38c-16ec2ce35a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:22.829468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ea77c1e4-99dd-4431-b38c-16ec2ce35a41 None None] MFA Rules not processed for user `32a6ad0f3d3b44d48df46e91d80abc95`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:31:22.830831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ea77c1e4-99dd-4431-b38c-16ec2ce35a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:22.871932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:22.911788 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4140/8316] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:31:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:31:22.916844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5f06b-d8d5-490d-8f23-6ac462b7d016 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:22.929809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5f06b-d8d5-490d-8f23-6ac462b7d016 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:22.935092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3f6ad-849a-426e-aedf-41dfeaadeff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:22.935720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3f6ad-849a-426e-aedf-41dfeaadeff2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:22.935992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3f6ad-849a-426e-aedf-41dfeaadeff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:22.936416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3f6ad-849a-426e-aedf-41dfeaadeff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:22.937199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3f6ad-849a-426e-aedf-41dfeaadeff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:22.940144 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3f6ad-849a-426e-aedf-41dfeaadeff2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:22.973678 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4177/8317] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:31:22 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:31:22.977927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77c1e4-99dd-4431-b38c-16ec2ce35a41 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:22.990253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77c1e4-99dd-4431-b38c-16ec2ce35a41 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:23.066291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae9597-1f20-4026-8d2e-7d03a1bf4071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:23.067076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae9597-1f20-4026-8d2e-7d03a1bf4071 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:23.067414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae9597-1f20-4026-8d2e-7d03a1bf4071 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:23.067749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae9597-1f20-4026-8d2e-7d03a1bf4071 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:23.068439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae9597-1f20-4026-8d2e-7d03a1bf4071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:23.070404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae9597-1f20-4026-8d2e-7d03a1bf4071 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:23.094768 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4141/8318] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:31:22 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:31:23.099874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3f6ad-849a-426e-aedf-41dfeaadeff2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:23.110630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3f6ad-849a-426e-aedf-41dfeaadeff2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:23.115752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a3552-ac1e-43b0-bac5-e5e3ef977979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:23.116308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a3552-ac1e-43b0-bac5-e5e3ef977979 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:23.116684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a3552-ac1e-43b0-bac5-e5e3ef977979 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:23.116881 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a3552-ac1e-43b0-bac5-e5e3ef977979 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:23.117811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a3552-ac1e-43b0-bac5-e5e3ef977979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:23.119424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a3552-ac1e-43b0-bac5-e5e3ef977979 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:23.133187 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4178/8319] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:31:23.134501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae9597-1f20-4026-8d2e-7d03a1bf4071 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:23.145650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae9597-1f20-4026-8d2e-7d03a1bf4071 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:23.150548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bbc53-3bbb-4a0a-a96d-2ebfc9b6ef7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:23.151244 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bbc53-3bbb-4a0a-a96d-2ebfc9b6ef7e admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:23.151671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bbc53-3bbb-4a0a-a96d-2ebfc9b6ef7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:23.152071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bbc53-3bbb-4a0a-a96d-2ebfc9b6ef7e admin admin] PATH_INFO: `/v3/projects/22fd84ef105949f98fb0226bf6ca2485/users/ff92b45b4784481b8bdf30e923d4f4ce/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:23.171676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bbc53-3bbb-4a0a-a96d-2ebfc9b6ef7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=22fd84ef105949f98fb0226bf6ca2485, user_id=ff92b45b4784481b8bdf30e923d4f4ce, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:23.174824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bbc53-3bbb-4a0a-a96d-2ebfc9b6ef7e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:23.192020 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4142/8320] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:31:23 2022] PUT /identity/v3/projects/22fd84ef105949f98fb0226bf6ca2485/users/ff92b45b4784481b8bdf30e923d4f4ce/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:31:23.196173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a3552-ac1e-43b0-bac5-e5e3ef977979 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:23.211575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a3552-ac1e-43b0-bac5-e5e3ef977979 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:23.269955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:23.278381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe7176-c30b-4ae1-a52b-58ba43a28454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:23.278994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe7176-c30b-4ae1-a52b-58ba43a28454 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:23.279267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe7176-c30b-4ae1-a52b-58ba43a28454 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:23.279531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe7176-c30b-4ae1-a52b-58ba43a28454 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:23.280165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe7176-c30b-4ae1-a52b-58ba43a28454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:23.281804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe7176-c30b-4ae1-a52b-58ba43a28454 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:23.293058 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4179/8321] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:31:23.297739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bbc53-3bbb-4a0a-a96d-2ebfc9b6ef7e admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:23.308316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bbc53-3bbb-4a0a-a96d-2ebfc9b6ef7e admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:23.312946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec9d75-c30e-4535-ad8a-fa6e19d57453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:23.313611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec9d75-c30e-4535-ad8a-fa6e19d57453 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:23.313884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec9d75-c30e-4535-ad8a-fa6e19d57453 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:23.314148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec9d75-c30e-4535-ad8a-fa6e19d57453 admin admin] PATH_INFO: `/v3/projects/22fd84ef105949f98fb0226bf6ca2485/users/ff92b45b4784481b8bdf30e923d4f4ce/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:23.316549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec9d75-c30e-4535-ad8a-fa6e19d57453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22fd84ef105949f98fb0226bf6ca2485, user_id=ff92b45b4784481b8bdf30e923d4f4ce, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:23.319240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec9d75-c30e-4535-ad8a-fa6e19d57453 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:23.329905 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4143/8322] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:31:23 2022] PUT /identity/v3/projects/22fd84ef105949f98fb0226bf6ca2485/users/ff92b45b4784481b8bdf30e923d4f4ce/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:31:27.856353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003e9c-ef4c-4d0a-ac1e-86d1f5a20d7b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:27.856903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003e9c-ef4c-4d0a-ac1e-86d1f5a20d7b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:27.856903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003e9c-ef4c-4d0a-ac1e-86d1f5a20d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:27.891383 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7b003e9c-ef4c-4d0a-ac1e-86d1f5a20d7b None None] MFA Rules not processed for user `ff92b45b4784481b8bdf30e923d4f4ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:31:27.892385 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7b003e9c-ef4c-4d0a-ac1e-86d1f5a20d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:27.986245 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4180/8323] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:31:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:31:28.000937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec9d75-c30e-4535-ad8a-fa6e19d57453 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:31:28.003321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-31ec9d75-c30e-4535-ad8a-fa6e19d57453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:28.091972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec9d75-c30e-4535-ad8a-fa6e19d57453 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:31:28.096339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc3836-c6d1-4aeb-b69f-2850b55da78b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:31:28.096945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc3836-c6d1-4aeb-b69f-2850b55da78b service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:28.097436 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc3836-c6d1-4aeb-b69f-2850b55da78b service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:28.097691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc3836-c6d1-4aeb-b69f-2850b55da78b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:28.148561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:28.167040 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc3836-c6d1-4aeb-b69f-2850b55da78b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:31:28.168413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc3836-c6d1-4aeb-b69f-2850b55da78b service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:31:28.185261 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4144/8324] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:31:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:31:32.173464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7d43c-0e6c-46e2-8b28-45ce36ff9c86 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:32.173992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7d43c-0e6c-46e2-8b28-45ce36ff9c86 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:32.174098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7d43c-0e6c-46e2-8b28-45ce36ff9c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:32.195446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-19e7d43c-0e6c-46e2-8b28-45ce36ff9c86 None None] MFA Rules not processed for user `32a6ad0f3d3b44d48df46e91d80abc95`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:31:32.196702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-19e7d43c-0e6c-46e2-8b28-45ce36ff9c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:32.203786 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4181/8325] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:31:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:31:38.179450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=8 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:42.189430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=41 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:53.861393 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cf0b5-2011-4f79-8f9a-1e2ff610be89 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:31:53.861946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cf0b5-2011-4f79-8f9a-1e2ff610be89 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:31:53.862056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cf0b5-2011-4f79-8f9a-1e2ff610be89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:31:53.882767 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-808cf0b5-2011-4f79-8f9a-1e2ff610be89 None None] MFA Rules not processed for user `32a6ad0f3d3b44d48df46e91d80abc95`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:31:53.884265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-808cf0b5-2011-4f79-8f9a-1e2ff610be89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:31:53.892476 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4145/8326] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:31:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:32:03.876354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:06.140331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaec7ae-1432-4d97-b74a-27a8eafe99e5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:06.140832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaec7ae-1432-4d97-b74a-27a8eafe99e5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:06.140929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaec7ae-1432-4d97-b74a-27a8eafe99e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:06.201291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6eaec7ae-1432-4d97-b74a-27a8eafe99e5 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:32:06.202530 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6eaec7ae-1432-4d97-b74a-27a8eafe99e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:06.267028 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4182/8327] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:32:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:32:06.358654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cf0b5-2011-4f79-8f9a-1e2ff610be89 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:06.360075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-808cf0b5-2011-4f79-8f9a-1e2ff610be89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:06.455167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cf0b5-2011-4f79-8f9a-1e2ff610be89 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:06.460322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c6119-3ddf-4cb8-b2e1-dd42a058355f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:06.461344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c6119-3ddf-4cb8-b2e1-dd42a058355f service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:06.461807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c6119-3ddf-4cb8-b2e1-dd42a058355f service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:06.462278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c6119-3ddf-4cb8-b2e1-dd42a058355f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:06.471730 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c6119-3ddf-4cb8-b2e1-dd42a058355f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:06.473340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c6119-3ddf-4cb8-b2e1-dd42a058355f service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:06.481645 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4146/8328] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:32:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:32:07.748794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb3166-e773-433c-af13-42854edaea11 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:07.749566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb3166-e773-433c-af13-42854edaea11 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:07.749663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb3166-e773-433c-af13-42854edaea11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:07.769021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-76cb3166-e773-433c-af13-42854edaea11 None None] MFA Rules not processed for user `32a6ad0f3d3b44d48df46e91d80abc95`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:32:07.770242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76cb3166-e773-433c-af13-42854edaea11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:07.777737 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4183/8329] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:32:08.035951 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c6119-3ddf-4cb8-b2e1-dd42a058355f service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:08.092676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c6119-3ddf-4cb8-b2e1-dd42a058355f service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:08.096251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95037-d551-4ce1-8120-774c3a2de6db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:08.096834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95037-d551-4ce1-8120-774c3a2de6db service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:08.097131 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95037-d551-4ce1-8120-774c3a2de6db service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:08.097413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95037-d551-4ce1-8120-774c3a2de6db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:08.099118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-70a95037-d551-4ce1-8120-774c3a2de6db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:08.111052 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95037-d551-4ce1-8120-774c3a2de6db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:08.112461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95037-d551-4ce1-8120-774c3a2de6db service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:08.123447 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4147/8330] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:32:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:32:17.764132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=47 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:18.117385 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=67 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:31.435528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb3166-e773-433c-af13-42854edaea11 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:31.437340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-76cb3166-e773-433c-af13-42854edaea11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:31.534780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95037-d551-4ce1-8120-774c3a2de6db service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:31.536997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-70a95037-d551-4ce1-8120-774c3a2de6db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:31.592672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb3166-e773-433c-af13-42854edaea11 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:31.598585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028d865-0401-4a61-8076-23c408f04d16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:31.599379 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d865-0401-4a61-8076-23c408f04d16 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:31.599925 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d865-0401-4a61-8076-23c408f04d16 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:31.600199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d865-0401-4a61-8076-23c408f04d16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:31.608039 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d028d865-0401-4a61-8076-23c408f04d16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:31.626690 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028d865-0401-4a61-8076-23c408f04d16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:31.628551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028d865-0401-4a61-8076-23c408f04d16 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:31.642842 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4184/8331] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:32:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:32:31.651691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95037-d551-4ce1-8120-774c3a2de6db service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:31.657442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:31.659408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:31.659787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:31.660128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] PATH_INFO: `/v3/users/bd14dd0707a1423d82328ac3a54ccb28` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:31.661404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd14dd0707a1423d82328ac3a54ccb28)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:31.663413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:31.722808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd14dd0707a1423d82328ac3a54ccb28'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:31.738726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd14dd0707a1423d82328ac3a54ccb28'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:31.749142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd14dd0707a1423d82328ac3a54ccb28'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:31.758493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd14dd0707a1423d82328ac3a54ccb28'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:31.759293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd14dd0707a1423d82328ac3a54ccb28'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:31.766063 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4148/8332] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:32:31 2022] DELETE /identity/v3/users/bd14dd0707a1423d82328ac3a54ccb28 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:32:31.770567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664aad7-8efe-4ab6-9c54-30f8a2c406c2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:31.770978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664aad7-8efe-4ab6-9c54-30f8a2c406c2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:31.771099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664aad7-8efe-4ab6-9c54-30f8a2c406c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:31.794586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d664aad7-8efe-4ab6-9c54-30f8a2c406c2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:32:31.795986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d664aad7-8efe-4ab6-9c54-30f8a2c406c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:31.873189 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4185/8333] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:32:31.884184 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:31.886023 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:31.953724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03cf3c-9281-4314-b125-210fd999f910 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:31.957860 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d37e38-224a-4887-aadb-63dd254c47ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:31.958433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d37e38-224a-4887-aadb-63dd254c47ce service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:31.958711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d37e38-224a-4887-aadb-63dd254c47ce service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:31.959005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d37e38-224a-4887-aadb-63dd254c47ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:31.970881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d37e38-224a-4887-aadb-63dd254c47ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:31.972542 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d37e38-224a-4887-aadb-63dd254c47ce service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:31.986463 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4149/8334] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:32:32.205073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664aad7-8efe-4ab6-9c54-30f8a2c406c2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:32.206356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d664aad7-8efe-4ab6-9c54-30f8a2c406c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:32.221888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664aad7-8efe-4ab6-9c54-30f8a2c406c2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:32.227551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:32.228623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:32.229509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:32.229947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] PATH_INFO: `/v3/projects/e2bfe1e4fbfd48998884d5e8ef8a3e57` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:32.231334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2bfe1e4fbfd48998884d5e8ef8a3e57)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:32.233642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:32.304983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'request_id': 'req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.308724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:32.315506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'request_id': 'req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.326734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'request_id': 'req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.340953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] The token cache is being invalidate because project e2bfe1e4fbfd48998884d5e8ef8a3e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:32:32.341268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.341980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.344854 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4186/8335] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:32:32 2022] DELETE /identity/v3/projects/e2bfe1e4fbfd48998884d5e8ef8a3e57 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:32:32.349012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d37e38-224a-4887-aadb-63dd254c47ce service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:32.359798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d37e38-224a-4887-aadb-63dd254c47ce service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:32.364821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:32.425336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:32.426218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:32.426649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:32.426940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] PATH_INFO: `/v3/users/82154bd4c8164aeb9c010b8f506038bb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:32.428394 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=82154bd4c8164aeb9c010b8f506038bb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:32.430615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:32.487301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82154bd4c8164aeb9c010b8f506038bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.501293 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82154bd4c8164aeb9c010b8f506038bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.510523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82154bd4c8164aeb9c010b8f506038bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.520552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82154bd4c8164aeb9c010b8f506038bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.521472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c94b4d25-f7ab-4efe-a29d-8d03dc65cb9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82154bd4c8164aeb9c010b8f506038bb'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.529823 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4150/8336] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:32:32 2022] DELETE /identity/v3/users/82154bd4c8164aeb9c010b8f506038bb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:32:32.850331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:32.852161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:32.867354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b3447-bff2-4c66-acf4-6768cee2ed8a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:32.929097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:32.929732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:32.930009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:32.930285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] PATH_INFO: `/v3/projects/00952acabdec4fa98e0a86743091655f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:32.931495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00952acabdec4fa98e0a86743091655f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:32.933595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:32.981947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00952acabdec4fa98e0a86743091655f', 'request_id': 'req-101a967b-aaf4-498a-8837-0d54e578d777'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.989096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00952acabdec4fa98e0a86743091655f', 'request_id': 'req-101a967b-aaf4-498a-8837-0d54e578d777'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:32.995432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00952acabdec4fa98e0a86743091655f', 'request_id': 'req-101a967b-aaf4-498a-8837-0d54e578d777'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:33.006062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] The token cache is being invalidate because project 00952acabdec4fa98e0a86743091655f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:32:33.006343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:33.006983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-101a967b-aaf4-498a-8837-0d54e578d777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:32:33.008462 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4187/8337] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:32:32 2022] DELETE /identity/v3/projects/00952acabdec4fa98e0a86743091655f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:32:33.014172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909032-7c45-4cfd-bea8-8cc80cdf7c27 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:33.014694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909032-7c45-4cfd-bea8-8cc80cdf7c27 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:33.015740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909032-7c45-4cfd-bea8-8cc80cdf7c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:33.035328 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-89909032-7c45-4cfd-bea8-8cc80cdf7c27 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:32:33.036550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-89909032-7c45-4cfd-bea8-8cc80cdf7c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:33.096608 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4151/8338] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:32:33.104843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89414bff-f97f-4c98-bb68-4a20fc9a8374 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:33.105159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89414bff-f97f-4c98-bb68-4a20fc9a8374 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:33.105434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89414bff-f97f-4c98-bb68-4a20fc9a8374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:33.124939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-89414bff-f97f-4c98-bb68-4a20fc9a8374 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:32:33.126416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-89414bff-f97f-4c98-bb68-4a20fc9a8374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:33.134674 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4188/8339] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:32:33.139669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909032-7c45-4cfd-bea8-8cc80cdf7c27 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:33.149441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909032-7c45-4cfd-bea8-8cc80cdf7c27 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:33.154655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df0993-54b3-449b-bb31-7f03a5d2306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:33.155260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df0993-54b3-449b-bb31-7f03a5d2306a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:33.155536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df0993-54b3-449b-bb31-7f03a5d2306a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:33.155807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df0993-54b3-449b-bb31-7f03a5d2306a admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:33.156459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df0993-54b3-449b-bb31-7f03a5d2306a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:32:33.156764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df0993-54b3-449b-bb31-7f03a5d2306a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:33.158412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df0993-54b3-449b-bb31-7f03a5d2306a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:33.170195 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4152/8340] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:32:33.174205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89414bff-f97f-4c98-bb68-4a20fc9a8374 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:33.184052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89414bff-f97f-4c98-bb68-4a20fc9a8374 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:33.189201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b9a85-3f0f-422f-826e-7886aacfb503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:33.189846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b9a85-3f0f-422f-826e-7886aacfb503 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:33.190092 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b9a85-3f0f-422f-826e-7886aacfb503 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:33.190513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b9a85-3f0f-422f-826e-7886aacfb503 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:33.191149 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b9a85-3f0f-422f-826e-7886aacfb503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:33.193054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b9a85-3f0f-422f-826e-7886aacfb503 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:33.226034 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4189/8341] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:32:33 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:32:33.227201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df0993-54b3-449b-bb31-7f03a5d2306a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:33.237562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df0993-54b3-449b-bb31-7f03a5d2306a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:33.250504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:33.301021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58501ad0-4e98-4407-a291-e6a61f3a188c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:33.301650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58501ad0-4e98-4407-a291-e6a61f3a188c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:33.301929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58501ad0-4e98-4407-a291-e6a61f3a188c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:33.302199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58501ad0-4e98-4407-a291-e6a61f3a188c admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:33.302816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58501ad0-4e98-4407-a291-e6a61f3a188c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:33.304744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58501ad0-4e98-4407-a291-e6a61f3a188c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:33.330127 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4153/8342] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:32:33 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:32:33.331475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b9a85-3f0f-422f-826e-7886aacfb503 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:33.341952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b9a85-3f0f-422f-826e-7886aacfb503 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:33.347025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ff551-946b-4e06-bd00-2db4456409b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:33.347618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ff551-946b-4e06-bd00-2db4456409b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:33.347898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ff551-946b-4e06-bd00-2db4456409b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:33.348164 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ff551-946b-4e06-bd00-2db4456409b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:33.348794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ff551-946b-4e06-bd00-2db4456409b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:33.350587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ff551-946b-4e06-bd00-2db4456409b6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:33.361208 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4190/8343] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:32:33.365559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58501ad0-4e98-4407-a291-e6a61f3a188c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:33.377231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58501ad0-4e98-4407-a291-e6a61f3a188c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:33.382597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa037d-d573-4f9a-85e8-d285f9ef5a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:33.383294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa037d-d573-4f9a-85e8-d285f9ef5a0a admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:33.383572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa037d-d573-4f9a-85e8-d285f9ef5a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:33.383869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa037d-d573-4f9a-85e8-d285f9ef5a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:33.384578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa037d-d573-4f9a-85e8-d285f9ef5a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:33.387141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa037d-d573-4f9a-85e8-d285f9ef5a0a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:33.398187 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4154/8344] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:32:33.402547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ff551-946b-4e06-bd00-2db4456409b6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:33.412857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ff551-946b-4e06-bd00-2db4456409b6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:33.417587 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c436f-9b06-4340-b565-8d61a110647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:33.418234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c436f-9b06-4340-b565-8d61a110647c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:33.418517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c436f-9b06-4340-b565-8d61a110647c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:33.418784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c436f-9b06-4340-b565-8d61a110647c admin admin] PATH_INFO: `/v3/projects/84ddbf82d0764067ad376f9626516cb9/users/98a36b3c55b1462ea9cc65bfad1b967b/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:33.434837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c436f-9b06-4340-b565-8d61a110647c admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ddbf82d0764067ad376f9626516cb9, user_id=98a36b3c55b1462ea9cc65bfad1b967b, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:33.437708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c436f-9b06-4340-b565-8d61a110647c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:33.453644 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4191/8345] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:32:33 2022] PUT /identity/v3/projects/84ddbf82d0764067ad376f9626516cb9/users/98a36b3c55b1462ea9cc65bfad1b967b/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:32:33.459399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e107e-9ece-45e6-ad34-b039d256f302 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:33.459719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e107e-9ece-45e6-ad34-b039d256f302 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:33.460043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e107e-9ece-45e6-ad34-b039d256f302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:33.481255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-299e107e-9ece-45e6-ad34-b039d256f302 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:32:33.482481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-299e107e-9ece-45e6-ad34-b039d256f302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:33.543847 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4155/8346] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:32:33.554621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c436f-9b06-4340-b565-8d61a110647c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:33.556384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f15c436f-9b06-4340-b565-8d61a110647c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:33.566726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:33.620676 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c436f-9b06-4340-b565-8d61a110647c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:33.624213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc3fd1-b390-481d-8287-4f1e4d143ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:33.624810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc3fd1-b390-481d-8287-4f1e4d143ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:33.625434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc3fd1-b390-481d-8287-4f1e4d143ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:33.625704 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc3fd1-b390-481d-8287-4f1e4d143ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:33.634788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc3fd1-b390-481d-8287-4f1e4d143ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:33.636285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc3fd1-b390-481d-8287-4f1e4d143ee2 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:33.645789 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4192/8347] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:32:38.104683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b38fb-3a74-4d18-b3b9-55a16b7c4302 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:38.105370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b38fb-3a74-4d18-b3b9-55a16b7c4302 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:38.105370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b38fb-3a74-4d18-b3b9-55a16b7c4302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:38.140672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bb9b38fb-3a74-4d18-b3b9-55a16b7c4302 None None] MFA Rules not processed for user `98a36b3c55b1462ea9cc65bfad1b967b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:32:38.141997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bb9b38fb-3a74-4d18-b3b9-55a16b7c4302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:38.234777 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4156/8348] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:32:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:32:38.239812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc3fd1-b390-481d-8287-4f1e4d143ee2 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:38.253426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc3fd1-b390-481d-8287-4f1e4d143ee2 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:38.259175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23086e6f-67bd-4a6d-92d4-7d354b57da18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:38.260041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23086e6f-67bd-4a6d-92d4-7d354b57da18 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:38.261124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23086e6f-67bd-4a6d-92d4-7d354b57da18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:38.262696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23086e6f-67bd-4a6d-92d4-7d354b57da18 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:38.263528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23086e6f-67bd-4a6d-92d4-7d354b57da18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:38.266097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23086e6f-67bd-4a6d-92d4-7d354b57da18 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:38.305213 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4193/8349] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:32:38 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:32:38.308402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b38fb-3a74-4d18-b3b9-55a16b7c4302 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:38.320401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b38fb-3a74-4d18-b3b9-55a16b7c4302 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:38.346888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:38.395614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b764b-b101-418c-8314-e0016ae169c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:38.396227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b764b-b101-418c-8314-e0016ae169c6 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:38.396508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b764b-b101-418c-8314-e0016ae169c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:38.396770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b764b-b101-418c-8314-e0016ae169c6 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:38.397441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b764b-b101-418c-8314-e0016ae169c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:38.399601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b764b-b101-418c-8314-e0016ae169c6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:38.425097 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4157/8350] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:32:38 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:32:38.429381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23086e6f-67bd-4a6d-92d4-7d354b57da18 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:38.441608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23086e6f-67bd-4a6d-92d4-7d354b57da18 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:38.447464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb966d30-a466-47ab-b69f-7d48edba7d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:38.448722 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb966d30-a466-47ab-b69f-7d48edba7d20 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:38.449160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb966d30-a466-47ab-b69f-7d48edba7d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:38.449441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb966d30-a466-47ab-b69f-7d48edba7d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:38.450087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb966d30-a466-47ab-b69f-7d48edba7d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:38.451994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb966d30-a466-47ab-b69f-7d48edba7d20 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:38.473089 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4194/8351] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:32:38.475986 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b764b-b101-418c-8314-e0016ae169c6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:38.487884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b764b-b101-418c-8314-e0016ae169c6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:38.493735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a15601-2301-4293-83de-d9e3b87d358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:38.495698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a15601-2301-4293-83de-d9e3b87d358d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:38.495698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a15601-2301-4293-83de-d9e3b87d358d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:38.495698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a15601-2301-4293-83de-d9e3b87d358d admin admin] PATH_INFO: `/v3/projects/9f176587de28499590dfb6c222c6879d/users/481890bf10ba412eac4c85554da00e5e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:38.516950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a15601-2301-4293-83de-d9e3b87d358d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f176587de28499590dfb6c222c6879d, user_id=481890bf10ba412eac4c85554da00e5e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:38.519159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a15601-2301-4293-83de-d9e3b87d358d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:38.536339 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4158/8352] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:32:38 2022] PUT /identity/v3/projects/9f176587de28499590dfb6c222c6879d/users/481890bf10ba412eac4c85554da00e5e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:32:38.540414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb966d30-a466-47ab-b69f-7d48edba7d20 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:38.552017 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb966d30-a466-47ab-b69f-7d48edba7d20 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:38.619801 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9925af5-a9ff-4d63-8d8a-b607868bbc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:38.621306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9925af5-a9ff-4d63-8d8a-b607868bbc23 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:38.621637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9925af5-a9ff-4d63-8d8a-b607868bbc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:38.623509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9925af5-a9ff-4d63-8d8a-b607868bbc23 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:38.624248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9925af5-a9ff-4d63-8d8a-b607868bbc23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:38.627371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9925af5-a9ff-4d63-8d8a-b607868bbc23 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:38.639769 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4195/8353] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:32:38.644008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a15601-2301-4293-83de-d9e3b87d358d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:38.657663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a15601-2301-4293-83de-d9e3b87d358d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:38.663908 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04012e3-01e2-4867-ba95-8bfa455cff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:38.664405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04012e3-01e2-4867-ba95-8bfa455cff5d admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:38.664869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04012e3-01e2-4867-ba95-8bfa455cff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:38.665260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04012e3-01e2-4867-ba95-8bfa455cff5d admin admin] PATH_INFO: `/v3/projects/9f176587de28499590dfb6c222c6879d/users/481890bf10ba412eac4c85554da00e5e/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:38.667594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04012e3-01e2-4867-ba95-8bfa455cff5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f176587de28499590dfb6c222c6879d, user_id=481890bf10ba412eac4c85554da00e5e, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:38.670872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04012e3-01e2-4867-ba95-8bfa455cff5d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:38.684642 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4159/8354] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:32:38 2022] PUT /identity/v3/projects/9f176587de28499590dfb6c222c6879d/users/481890bf10ba412eac4c85554da00e5e/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:32:44.065540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc92f-1404-47ce-af90-3e05e1a7ed64 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:44.066110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc92f-1404-47ce-af90-3e05e1a7ed64 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:44.066110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc92f-1404-47ce-af90-3e05e1a7ed64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:44.143550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7a8bc92f-1404-47ce-af90-3e05e1a7ed64 None None] MFA Rules not processed for user `481890bf10ba412eac4c85554da00e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:32:44.144763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7a8bc92f-1404-47ce-af90-3e05e1a7ed64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:44.240763 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:44.248595 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4196/8355] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:32:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:32:44.270044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04012e3-01e2-4867-ba95-8bfa455cff5d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:44.274152 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b04012e3-01e2-4867-ba95-8bfa455cff5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:44.368813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04012e3-01e2-4867-ba95-8bfa455cff5d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:44.375521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148d09d-277e-45a5-8e9a-74b728e2e187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:44.376404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148d09d-277e-45a5-8e9a-74b728e2e187 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:44.376690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148d09d-277e-45a5-8e9a-74b728e2e187 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:44.376965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148d09d-277e-45a5-8e9a-74b728e2e187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:44.450272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148d09d-277e-45a5-8e9a-74b728e2e187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:44.452504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148d09d-277e-45a5-8e9a-74b728e2e187 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:44.464207 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4160/8356] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:32:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:32:45.743991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380f641-b06c-4db8-bb44-3cb328abfadb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:45.744529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380f641-b06c-4db8-bb44-3cb328abfadb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:45.744627 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380f641-b06c-4db8-bb44-3cb328abfadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:45.767790 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6380f641-b06c-4db8-bb44-3cb328abfadb None None] MFA Rules not processed for user `98a36b3c55b1462ea9cc65bfad1b967b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:32:45.769416 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6380f641-b06c-4db8-bb44-3cb328abfadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:32:45.778478 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4197/8357] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:32:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:32:54.458396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:55.762297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:57.799409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148d09d-277e-45a5-8e9a-74b728e2e187 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:32:57.879356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148d09d-277e-45a5-8e9a-74b728e2e187 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:32:57.883262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb952a-5d9a-404d-9591-289f31c4bbec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:32:57.883746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb952a-5d9a-404d-9591-289f31c4bbec service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:32:57.884056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb952a-5d9a-404d-9591-289f31c4bbec service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:32:57.884389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb952a-5d9a-404d-9591-289f31c4bbec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:32:57.895454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb952a-5d9a-404d-9591-289f31c4bbec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:32:57.897344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb952a-5d9a-404d-9591-289f31c4bbec service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:32:57.910154 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4161/8358] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:32:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:33:03.482362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19a73c-e47f-48f4-bc57-576fa048ddad None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:33:03.482836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19a73c-e47f-48f4-bc57-576fa048ddad None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:33:03.482945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19a73c-e47f-48f4-bc57-576fa048ddad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:33:03.502286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-af19a73c-e47f-48f4-bc57-576fa048ddad None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:33:03.503461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-af19a73c-e47f-48f4-bc57-576fa048ddad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:33:03.556753 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4198/8359] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:33:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:33:03.569572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb952a-5d9a-404d-9591-289f31c4bbec service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:33:03.571024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c1bb952a-5d9a-404d-9591-289f31c4bbec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:33:03.642797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb952a-5d9a-404d-9591-289f31c4bbec service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:33:03.647822 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b98622-c33d-47cf-a755-bf52a3402eb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:33:03.648417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b98622-c33d-47cf-a755-bf52a3402eb9 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:33:03.648694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b98622-c33d-47cf-a755-bf52a3402eb9 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:33:03.648965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b98622-c33d-47cf-a755-bf52a3402eb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:33:03.657107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b98622-c33d-47cf-a755-bf52a3402eb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:33:03.658690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b98622-c33d-47cf-a755-bf52a3402eb9 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:33:03.666766 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4162/8360] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:33:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:33:06.501665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388b960-b5a8-426b-ae0d-59cc7c1dc89b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:33:06.502187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388b960-b5a8-426b-ae0d-59cc7c1dc89b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:33:06.502187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388b960-b5a8-426b-ae0d-59cc7c1dc89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:33:06.520910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0388b960-b5a8-426b-ae0d-59cc7c1dc89b None None] MFA Rules not processed for user `32a6ad0f3d3b44d48df46e91d80abc95`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:33:06.522117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0388b960-b5a8-426b-ae0d-59cc7c1dc89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:33:06.575490 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4199/8361] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:33:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:33:13.663432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=65 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:33:16.569002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=50 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:33:17.451783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b98622-c33d-47cf-a755-bf52a3402eb9 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:33:17.464531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b98622-c33d-47cf-a755-bf52a3402eb9 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:33:17.468068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1262c8e-118d-4ed1-9241-2633d6775ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:33:17.468757 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1262c8e-118d-4ed1-9241-2633d6775ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:33:17.469028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1262c8e-118d-4ed1-9241-2633d6775ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:33:17.469365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1262c8e-118d-4ed1-9241-2633d6775ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:33:17.471070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d1262c8e-118d-4ed1-9241-2633d6775ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:33:17.533836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1262c8e-118d-4ed1-9241-2633d6775ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:33:17.535178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1262c8e-118d-4ed1-9241-2633d6775ee3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:33:17.544639 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4163/8362] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:33:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:33:27.540453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=30 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:34:05.582403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388b960-b5a8-426b-ae0d-59cc7c1dc89b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:34:05.594614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388b960-b5a8-426b-ae0d-59cc7c1dc89b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:34:05.598231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d516143-64ab-4a8e-b7ee-b38a694b0d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:34:05.598904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d516143-64ab-4a8e-b7ee-b38a694b0d98 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:34:05.599176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d516143-64ab-4a8e-b7ee-b38a694b0d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:34:05.599440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d516143-64ab-4a8e-b7ee-b38a694b0d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:34:05.608634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1d516143-64ab-4a8e-b7ee-b38a694b0d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:34:05.634462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d516143-64ab-4a8e-b7ee-b38a694b0d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:34:05.635898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d516143-64ab-4a8e-b7ee-b38a694b0d98 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:34:05.646344 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4200/8363] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:34:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:34:15.640236 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=11 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:34:27.212238 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1262c8e-118d-4ed1-9241-2633d6775ee3 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:34:27.223443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1262c8e-118d-4ed1-9241-2633d6775ee3 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:34:27.227247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827fa52-80d5-44c1-872e-19fbdc6a0817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:34:27.227834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827fa52-80d5-44c1-872e-19fbdc6a0817 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:34:27.228108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827fa52-80d5-44c1-872e-19fbdc6a0817 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:34:27.228379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827fa52-80d5-44c1-872e-19fbdc6a0817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:34:27.230140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b827fa52-80d5-44c1-872e-19fbdc6a0817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:34:27.242734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827fa52-80d5-44c1-872e-19fbdc6a0817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:34:27.244156 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827fa52-80d5-44c1-872e-19fbdc6a0817 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:34:27.254839 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4164/8364] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:34:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:34:37.248517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:35:25.426919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d516143-64ab-4a8e-b7ee-b38a694b0d98 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:35:25.438883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d516143-64ab-4a8e-b7ee-b38a694b0d98 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:35:25.442531 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8773f93-3b09-4352-b5a9-6a21065b25c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:35:25.443122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8773f93-3b09-4352-b5a9-6a21065b25c4 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:35:25.443398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8773f93-3b09-4352-b5a9-6a21065b25c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:35:25.443664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8773f93-3b09-4352-b5a9-6a21065b25c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:35:25.445384 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a8773f93-3b09-4352-b5a9-6a21065b25c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:35:25.462133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8773f93-3b09-4352-b5a9-6a21065b25c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:35:25.463511 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8773f93-3b09-4352-b5a9-6a21065b25c4 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:35:25.477104 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4201/8365] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:35:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:35:26.566193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827fa52-80d5-44c1-872e-19fbdc6a0817 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:35:26.590830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827fa52-80d5-44c1-872e-19fbdc6a0817 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:35:26.597917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3521a-1b7a-4a87-b2e5-9b8879574fb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:35:26.598525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3521a-1b7a-4a87-b2e5-9b8879574fb0 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:35:26.598800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3521a-1b7a-4a87-b2e5-9b8879574fb0 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:35:26.599066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3521a-1b7a-4a87-b2e5-9b8879574fb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:35:26.600784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9a3521a-1b7a-4a87-b2e5-9b8879574fb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:35:26.641132 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3521a-1b7a-4a87-b2e5-9b8879574fb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:35:26.642812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3521a-1b7a-4a87-b2e5-9b8879574fb0 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:35:26.681704 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4165/8366] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:35:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:35:27.828395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cba84-f510-4984-a46f-4e2cebb3b7ae None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:35:27.828867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cba84-f510-4984-a46f-4e2cebb3b7ae None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:35:27.828969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cba84-f510-4984-a46f-4e2cebb3b7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:35:27.848719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-353cba84-f510-4984-a46f-4e2cebb3b7ae None None] MFA Rules not processed for user `32a6ad0f3d3b44d48df46e91d80abc95`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:35:27.849943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-353cba84-f510-4984-a46f-4e2cebb3b7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:35:27.856688 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4202/8367] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:35:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:35:28.619206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3521a-1b7a-4a87-b2e5-9b8879574fb0 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:35:28.620651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b9a3521a-1b7a-4a87-b2e5-9b8879574fb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:35:28.695799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3521a-1b7a-4a87-b2e5-9b8879574fb0 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:35:28.700472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc4a7b-ee2c-4325-bd10-ee19fd1d5a8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:35:28.701084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc4a7b-ee2c-4325-bd10-ee19fd1d5a8b service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:35:28.701359 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc4a7b-ee2c-4325-bd10-ee19fd1d5a8b service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:35:28.702189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc4a7b-ee2c-4325-bd10-ee19fd1d5a8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:35:28.703888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-14bc4a7b-ee2c-4325-bd10-ee19fd1d5a8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:35:28.718230 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc4a7b-ee2c-4325-bd10-ee19fd1d5a8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:35:28.719781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc4a7b-ee2c-4325-bd10-ee19fd1d5a8b service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:35:28.731915 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4166/8368] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:35:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:35:33.503066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cba84-f510-4984-a46f-4e2cebb3b7ae None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:35:33.515489 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cba84-f510-4984-a46f-4e2cebb3b7ae None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:35:33.519022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea3221-a287-4039-bc71-be493cd1166d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:35:33.519631 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea3221-a287-4039-bc71-be493cd1166d service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:35:33.519897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea3221-a287-4039-bc71-be493cd1166d service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:35:33.520158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea3221-a287-4039-bc71-be493cd1166d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:35:33.521872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3dea3221-a287-4039-bc71-be493cd1166d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:35:33.534176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea3221-a287-4039-bc71-be493cd1166d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:35:33.535575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea3221-a287-4039-bc71-be493cd1166d service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:35:33.546609 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4203/8369] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:35:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:35:36.527060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc4a7b-ee2c-4325-bd10-ee19fd1d5a8b service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:35:36.542912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc4a7b-ee2c-4325-bd10-ee19fd1d5a8b service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:35:36.548947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55d0d4-b0a8-403a-87fc-912e7a435d8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:35:36.549571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55d0d4-b0a8-403a-87fc-912e7a435d8e service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:35:36.549853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55d0d4-b0a8-403a-87fc-912e7a435d8e service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:35:36.550129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55d0d4-b0a8-403a-87fc-912e7a435d8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:35:36.551807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af55d0d4-b0a8-403a-87fc-912e7a435d8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:35:36.566945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55d0d4-b0a8-403a-87fc-912e7a435d8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:35:36.568511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55d0d4-b0a8-403a-87fc-912e7a435d8e service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:35:36.582443 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4167/8370] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:35:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:35:43.539913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=15 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:35:46.575080 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=44 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:36:18.465004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea3221-a287-4039-bc71-be493cd1166d service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:36:18.479404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea3221-a287-4039-bc71-be493cd1166d service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:36:18.484294 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321b1d1-28c2-452f-82a1-ac22024dfb62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:36:18.484875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321b1d1-28c2-452f-82a1-ac22024dfb62 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:36:18.485179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321b1d1-28c2-452f-82a1-ac22024dfb62 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:36:18.485444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321b1d1-28c2-452f-82a1-ac22024dfb62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:36:18.487096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7321b1d1-28c2-452f-82a1-ac22024dfb62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:36:18.513898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321b1d1-28c2-452f-82a1-ac22024dfb62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:36:18.515453 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321b1d1-28c2-452f-82a1-ac22024dfb62 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:36:18.528594 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4204/8371] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:36:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:36:23.721392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55d0d4-b0a8-403a-87fc-912e7a435d8e service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:36:23.722865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-af55d0d4-b0a8-403a-87fc-912e7a435d8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:36:23.740187 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55d0d4-b0a8-403a-87fc-912e7a435d8e service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:36:23.745140 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98cf6c-da11-4c9d-af7c-35672243d1b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:36:23.745724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98cf6c-da11-4c9d-af7c-35672243d1b5 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:36:23.745994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98cf6c-da11-4c9d-af7c-35672243d1b5 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:36:23.746256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98cf6c-da11-4c9d-af7c-35672243d1b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:36:23.747903 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1d98cf6c-da11-4c9d-af7c-35672243d1b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:36:23.761714 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98cf6c-da11-4c9d-af7c-35672243d1b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:36:23.763270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98cf6c-da11-4c9d-af7c-35672243d1b5 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:36:23.775445 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4168/8372] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:36:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:36:28.522250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=11 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:36:33.768826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:28.312971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321b1d1-28c2-452f-82a1-ac22024dfb62 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:37:28.314534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7321b1d1-28c2-452f-82a1-ac22024dfb62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:37:28.398018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321b1d1-28c2-452f-82a1-ac22024dfb62 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:37:28.401630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9668265-f5a9-4429-ad67-bd2dfe88cb82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa85b647a4be4d81b966bb50691bd3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:37:28.402227 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9668265-f5a9-4429-ad67-bd2dfe88cb82 service cinder] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:37:28.402500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9668265-f5a9-4429-ad67-bd2dfe88cb82 service cinder] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:37:28.402961 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9668265-f5a9-4429-ad67-bd2dfe88cb82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:37:28.407395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a9668265-f5a9-4429-ad67-bd2dfe88cb82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:37:28.419985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9668265-f5a9-4429-ad67-bd2dfe88cb82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:37:28.421493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9668265-f5a9-4429-ad67-bd2dfe88cb82 service cinder] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:37:28.430742 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4205/8373] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:37:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:37:31.121549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc4de8-a906-42af-9c1a-d5621a36ebbd None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:37:31.122077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc4de8-a906-42af-9c1a-d5621a36ebbd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:37:31.122077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc4de8-a906-42af-9c1a-d5621a36ebbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:37:31.144366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b6cc4de8-a906-42af-9c1a-d5621a36ebbd None None] MFA Rules not processed for user `32a6ad0f3d3b44d48df46e91d80abc95`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:37:31.145601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b6cc4de8-a906-42af-9c1a-d5621a36ebbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:37:31.153486 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4169/8374] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:37:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:37:31.491469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8674-1528-4179-a181-870c3c3f681d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:37:31.492009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8674-1528-4179-a181-870c3c3f681d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:37:31.492009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8674-1528-4179-a181-870c3c3f681d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:37:31.541135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a48e8674-1528-4179-a181-870c3c3f681d None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:37:31.542265 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a48e8674-1528-4179-a181-870c3c3f681d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:37:31.548742 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4206/8375] 10.222.0.16 () {58 vars in 1061 bytes} [Sat Oct 15 21:37:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:37:31.554654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc4de8-a906-42af-9c1a-d5621a36ebbd None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:37:31.564458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc4de8-a906-42af-9c1a-d5621a36ebbd None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:37:31.567944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c34b6-8e70-42d6-a597-d7ba37ac8fee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:37:31.568521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c34b6-8e70-42d6-a597-d7ba37ac8fee service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:37:31.568791 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c34b6-8e70-42d6-a597-d7ba37ac8fee service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:37:31.569071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c34b6-8e70-42d6-a597-d7ba37ac8fee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:37:31.570747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-508c34b6-8e70-42d6-a597-d7ba37ac8fee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:37:31.583068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c34b6-8e70-42d6-a597-d7ba37ac8fee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:37:31.584471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c34b6-8e70-42d6-a597-d7ba37ac8fee service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:37:31.594886 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4170/8376] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:37:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:37:32.064784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e8674-1528-4179-a181-870c3c3f681d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:37:32.072838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e8674-1528-4179-a181-870c3c3f681d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:37:32.077276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20618-7b4a-417b-8a4a-e0fa4937667d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:37:32.077882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20618-7b4a-417b-8a4a-e0fa4937667d service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:37:32.078151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20618-7b4a-417b-8a4a-e0fa4937667d service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:37:32.078413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20618-7b4a-417b-8a4a-e0fa4937667d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:37:32.080087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-47f20618-7b4a-417b-8a4a-e0fa4937667d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:37:32.101365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f20618-7b4a-417b-8a4a-e0fa4937667d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:37:32.102777 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f20618-7b4a-417b-8a4a-e0fa4937667d service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:37:32.114464 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4207/8377] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:37:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:37:33.051796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f1367-5910-40b7-8485-27409edfaef5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:37:33.052223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f1367-5910-40b7-8485-27409edfaef5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:37:33.052327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f1367-5910-40b7-8485-27409edfaef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:37:33.089798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-aa1f1367-5910-40b7-8485-27409edfaef5 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:37:33.091485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aa1f1367-5910-40b7-8485-27409edfaef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:37:33.100505 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4171/8378] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 21:37:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:37:33.113372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20618-7b4a-417b-8a4a-e0fa4937667d service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:37:33.127301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20618-7b4a-417b-8a4a-e0fa4937667d service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:37:33.132290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4bf336-4e9c-4bc9-be74-9f1c60e19898 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:37:33.132882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4bf336-4e9c-4bc9-be74-9f1c60e19898 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:37:33.133190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4bf336-4e9c-4bc9-be74-9f1c60e19898 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:37:33.133461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4bf336-4e9c-4bc9-be74-9f1c60e19898 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:37:33.142220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4bf336-4e9c-4bc9-be74-9f1c60e19898 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:37:33.143772 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4bf336-4e9c-4bc9-be74-9f1c60e19898 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:37:33.153952 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4208/8379] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:37:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:37:43.087450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=18 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:43.149366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=63 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:10.225885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f1367-5910-40b7-8485-27409edfaef5 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:10.228089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aa1f1367-5910-40b7-8485-27409edfaef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:10.249093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f1367-5910-40b7-8485-27409edfaef5 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:10.255551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112e82-cfaa-4378-bcf6-78313273145a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:10.256271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112e82-cfaa-4378-bcf6-78313273145a service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:10.256637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112e82-cfaa-4378-bcf6-78313273145a service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:10.256934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112e82-cfaa-4378-bcf6-78313273145a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:10.267112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112e82-cfaa-4378-bcf6-78313273145a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:10.268733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112e82-cfaa-4378-bcf6-78313273145a service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:10.279639 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4172/8380] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:38:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:38:13.573150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4bf336-4e9c-4bc9-be74-9f1c60e19898 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:13.583007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4bf336-4e9c-4bc9-be74-9f1c60e19898 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:13.587040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8f0bf-8f54-435e-8e6b-9bc17bae8a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:13.587648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8f0bf-8f54-435e-8e6b-9bc17bae8a3d service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:13.587910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8f0bf-8f54-435e-8e6b-9bc17bae8a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:13.588170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8f0bf-8f54-435e-8e6b-9bc17bae8a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:13.659566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8f0bf-8f54-435e-8e6b-9bc17bae8a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:13.660978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8f0bf-8f54-435e-8e6b-9bc17bae8a3d service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:13.670422 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4209/8381] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:38:16.891931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112e82-cfaa-4378-bcf6-78313273145a service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:16.916472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112e82-cfaa-4378-bcf6-78313273145a service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:16.923801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dade2e9-f1b1-4928-b2f1-d9368483df4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:16.924702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dade2e9-f1b1-4928-b2f1-d9368483df4c service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:16.925949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dade2e9-f1b1-4928-b2f1-d9368483df4c service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:16.925949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dade2e9-f1b1-4928-b2f1-d9368483df4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:16.928464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3dade2e9-f1b1-4928-b2f1-d9368483df4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:16.947049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dade2e9-f1b1-4928-b2f1-d9368483df4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:16.949345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dade2e9-f1b1-4928-b2f1-d9368483df4c service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:16.970595 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4173/8382] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:38:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:38:20.145655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8f0bf-8f54-435e-8e6b-9bc17bae8a3d service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:20.160728 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8f0bf-8f54-435e-8e6b-9bc17bae8a3d service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:20.166199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd33414-0eda-4663-b237-97e68d00649b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:20.166889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd33414-0eda-4663-b237-97e68d00649b service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:20.167228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd33414-0eda-4663-b237-97e68d00649b service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:20.167563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd33414-0eda-4663-b237-97e68d00649b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:20.169412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-9dd33414-0eda-4663-b237-97e68d00649b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:20.186434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd33414-0eda-4663-b237-97e68d00649b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:20.188006 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd33414-0eda-4663-b237-97e68d00649b service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:20.204129 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4210/8383] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:38:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:38:20.217422 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dade2e9-f1b1-4928-b2f1-d9368483df4c service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:20.229154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dade2e9-f1b1-4928-b2f1-d9368483df4c service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:20.234200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:20.234930 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:20.235302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:20.235667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] PATH_INFO: `/v3/users/98a36b3c55b1462ea9cc65bfad1b967b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:20.237155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98a36b3c55b1462ea9cc65bfad1b967b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:20.239158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:20.286660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98a36b3c55b1462ea9cc65bfad1b967b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.299263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98a36b3c55b1462ea9cc65bfad1b967b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.306514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98a36b3c55b1462ea9cc65bfad1b967b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.313626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98a36b3c55b1462ea9cc65bfad1b967b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.314242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98a36b3c55b1462ea9cc65bfad1b967b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.320110 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4174/8384] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:38:20 2022] DELETE /identity/v3/users/98a36b3c55b1462ea9cc65bfad1b967b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:38:20.513752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd33414-0eda-4663-b237-97e68d00649b service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:20.525969 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd33414-0eda-4663-b237-97e68d00649b service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:20.584655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:20.585232 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:20.585505 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:20.585767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] PATH_INFO: `/v3/projects/84ddbf82d0764067ad376f9626516cb9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:20.587570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ddbf82d0764067ad376f9626516cb9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:20.589592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:20.644063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ddbf82d0764067ad376f9626516cb9', 'request_id': 'req-f7fdd11b-d39e-4b3b-b991-74722f9810e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.652990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ddbf82d0764067ad376f9626516cb9', 'request_id': 'req-f7fdd11b-d39e-4b3b-b991-74722f9810e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.660332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ddbf82d0764067ad376f9626516cb9', 'request_id': 'req-f7fdd11b-d39e-4b3b-b991-74722f9810e6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.673551 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:20.690469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] The token cache is being invalidate because project 84ddbf82d0764067ad376f9626516cb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:38:20.691052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.691802 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.693488 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4211/8385] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:38:20 2022] DELETE /identity/v3/projects/84ddbf82d0764067ad376f9626516cb9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:38:20.698254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:20.699870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:20.714195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39187e6-981d-47a8-89ea-b883b0418021 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:20.774936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:20.775584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:20.775859 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:20.776126 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] PATH_INFO: `/v3/users/481890bf10ba412eac4c85554da00e5e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:20.777281 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=481890bf10ba412eac4c85554da00e5e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:20.779276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:20.832634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481890bf10ba412eac4c85554da00e5e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.833783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:20.849630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481890bf10ba412eac4c85554da00e5e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.857697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481890bf10ba412eac4c85554da00e5e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.864924 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481890bf10ba412eac4c85554da00e5e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.865584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ff38b8e0-ab56-4110-8396-b68363de1441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481890bf10ba412eac4c85554da00e5e'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:20.872415 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4175/8386] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:38:20 2022] DELETE /identity/v3/users/481890bf10ba412eac4c85554da00e5e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:38:21.046574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:21.048030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:21.062314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdd11b-d39e-4b3b-b991-74722f9810e6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:21.118928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:21.119490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.119764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.120030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] PATH_INFO: `/v3/projects/9f176587de28499590dfb6c222c6879d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.121167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f176587de28499590dfb6c222c6879d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:21.123064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:21.172725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f176587de28499590dfb6c222c6879d', 'request_id': 'req-5aafc250-74f3-485f-8a96-9b16985d47fa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:21.181521 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f176587de28499590dfb6c222c6879d', 'request_id': 'req-5aafc250-74f3-485f-8a96-9b16985d47fa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:21.188888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f176587de28499590dfb6c222c6879d', 'request_id': 'req-5aafc250-74f3-485f-8a96-9b16985d47fa'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:21.200376 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] The token cache is being invalidate because project 9f176587de28499590dfb6c222c6879d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:38:21.200663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:21.201351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-5aafc250-74f3-485f-8a96-9b16985d47fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:38:21.202904 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4212/8387] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:38:21 2022] DELETE /identity/v3/projects/9f176587de28499590dfb6c222c6879d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:38:21.207921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f7377-edc2-40ea-bc4d-3e9c0b8e118e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.208199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f7377-edc2-40ea-bc4d-3e9c0b8e118e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.208499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f7377-edc2-40ea-bc4d-3e9c0b8e118e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.236571 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8f7f7377-edc2-40ea-bc4d-3e9c0b8e118e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:38:21.237793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8f7f7377-edc2-40ea-bc4d-3e9c0b8e118e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:21.298320 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4176/8388] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:38:21.306378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f02401-0b85-489b-9546-6ea5e0a7af0b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.306664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f02401-0b85-489b-9546-6ea5e0a7af0b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.306934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f02401-0b85-489b-9546-6ea5e0a7af0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.326942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c5f02401-0b85-489b-9546-6ea5e0a7af0b None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:38:21.328123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c5f02401-0b85-489b-9546-6ea5e0a7af0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:21.335884 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4213/8389] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:38:21.340795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f7377-edc2-40ea-bc4d-3e9c0b8e118e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:21.350698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f7377-edc2-40ea-bc4d-3e9c0b8e118e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:21.355401 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43d3c2-037f-403d-be68-266004cb4471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:21.355974 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43d3c2-037f-403d-be68-266004cb4471 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.356253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43d3c2-037f-403d-be68-266004cb4471 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.356552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43d3c2-037f-403d-be68-266004cb4471 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.357231 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43d3c2-037f-403d-be68-266004cb4471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:38:21.357535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43d3c2-037f-403d-be68-266004cb4471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:21.359141 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43d3c2-037f-403d-be68-266004cb4471 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:21.370727 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4177/8390] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:38:21.374953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f02401-0b85-489b-9546-6ea5e0a7af0b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:21.384939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f02401-0b85-489b-9546-6ea5e0a7af0b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:21.389822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e6c97-2748-4903-8cfd-33f86c69a676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:21.390399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e6c97-2748-4903-8cfd-33f86c69a676 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.390672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e6c97-2748-4903-8cfd-33f86c69a676 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.390970 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e6c97-2748-4903-8cfd-33f86c69a676 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.391593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e6c97-2748-4903-8cfd-33f86c69a676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:21.393506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e6c97-2748-4903-8cfd-33f86c69a676 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:21.421967 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4214/8391] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:38:21 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:38:21.426236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43d3c2-037f-403d-be68-266004cb4471 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:21.437027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43d3c2-037f-403d-be68-266004cb4471 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:21.494167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ed9d4-953b-4e89-925e-36637c08096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:21.494761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ed9d4-953b-4e89-925e-36637c08096d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.495035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ed9d4-953b-4e89-925e-36637c08096d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.495307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ed9d4-953b-4e89-925e-36637c08096d admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.495934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ed9d4-953b-4e89-925e-36637c08096d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:21.497969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ed9d4-953b-4e89-925e-36637c08096d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:21.522680 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4178/8392] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:38:21 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:38:21.523871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e6c97-2748-4903-8cfd-33f86c69a676 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:21.534899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e6c97-2748-4903-8cfd-33f86c69a676 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:21.539470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a59561-7a3c-4ab0-b365-2fa3f0d7a828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:21.540087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a59561-7a3c-4ab0-b365-2fa3f0d7a828 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.540436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a59561-7a3c-4ab0-b365-2fa3f0d7a828 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.540703 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a59561-7a3c-4ab0-b365-2fa3f0d7a828 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.541345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a59561-7a3c-4ab0-b365-2fa3f0d7a828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:21.542931 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a59561-7a3c-4ab0-b365-2fa3f0d7a828 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:21.552975 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4215/8393] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:38:21.557105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ed9d4-953b-4e89-925e-36637c08096d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:21.567154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ed9d4-953b-4e89-925e-36637c08096d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:21.572389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df2e75-db46-4586-a96b-4d3ede28d1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:21.572979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df2e75-db46-4586-a96b-4d3ede28d1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.573289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df2e75-db46-4586-a96b-4d3ede28d1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.573561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df2e75-db46-4586-a96b-4d3ede28d1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.574194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df2e75-db46-4586-a96b-4d3ede28d1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:21.575780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df2e75-db46-4586-a96b-4d3ede28d1f1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:21.586467 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4179/8394] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:38:21.591374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a59561-7a3c-4ab0-b365-2fa3f0d7a828 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:21.601393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a59561-7a3c-4ab0-b365-2fa3f0d7a828 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:21.606195 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355daed6-5a3b-46f3-a2f5-77262340a057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:21.606830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355daed6-5a3b-46f3-a2f5-77262340a057 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.607103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355daed6-5a3b-46f3-a2f5-77262340a057 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.607369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355daed6-5a3b-46f3-a2f5-77262340a057 admin admin] PATH_INFO: `/v3/projects/079b9450a9ce4a2f9518bce05c72c49e/users/be7e10dfac9d4ab5a74d803b56cc33db/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.622466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355daed6-5a3b-46f3-a2f5-77262340a057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079b9450a9ce4a2f9518bce05c72c49e, user_id=be7e10dfac9d4ab5a74d803b56cc33db, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:21.625075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355daed6-5a3b-46f3-a2f5-77262340a057 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:21.639008 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4216/8395] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:38:21 2022] PUT /identity/v3/projects/079b9450a9ce4a2f9518bce05c72c49e/users/be7e10dfac9d4ab5a74d803b56cc33db/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:38:21.643981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b67072-9339-46a8-9376-3f8e510b3f55 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.644268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b67072-9339-46a8-9376-3f8e510b3f55 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.644541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b67072-9339-46a8-9376-3f8e510b3f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.663456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-96b67072-9339-46a8-9376-3f8e510b3f55 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:38:21.664659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-96b67072-9339-46a8-9376-3f8e510b3f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:21.683505 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:21.729429 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4180/8396] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:38:21.738997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355daed6-5a3b-46f3-a2f5-77262340a057 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:21.740658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-355daed6-5a3b-46f3-a2f5-77262340a057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:21.761290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:21.799431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355daed6-5a3b-46f3-a2f5-77262340a057 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:21.803141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7d617-409d-4c67-a270-04754b867b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:21.803774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7d617-409d-4c67-a270-04754b867b92 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:21.804047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7d617-409d-4c67-a270-04754b867b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:21.804314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7d617-409d-4c67-a270-04754b867b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:21.813773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7d617-409d-4c67-a270-04754b867b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:21.815194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7d617-409d-4c67-a270-04754b867b92 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:21.825987 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4217/8397] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:38:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:38:25.749545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aaf2c-a7c3-4a20-b4ae-b09fa6bfc199 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:25.750109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aaf2c-a7c3-4a20-b4ae-b09fa6bfc199 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:25.750109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aaf2c-a7c3-4a20-b4ae-b09fa6bfc199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:25.783673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-331aaf2c-a7c3-4a20-b4ae-b09fa6bfc199 None None] MFA Rules not processed for user `be7e10dfac9d4ab5a74d803b56cc33db`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:38:25.785087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-331aaf2c-a7c3-4a20-b4ae-b09fa6bfc199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:25.883334 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4181/8398] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:38:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:38:25.888968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7d617-409d-4c67-a270-04754b867b92 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:25.904887 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7d617-409d-4c67-a270-04754b867b92 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:25.911974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938a163-f038-43df-85b6-4f2f272b7868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:25.912817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938a163-f038-43df-85b6-4f2f272b7868 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:25.913138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938a163-f038-43df-85b6-4f2f272b7868 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:25.913405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938a163-f038-43df-85b6-4f2f272b7868 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:25.914082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938a163-f038-43df-85b6-4f2f272b7868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:25.916175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938a163-f038-43df-85b6-4f2f272b7868 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:25.951770 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4218/8399] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:38:25 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:38:25.953606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aaf2c-a7c3-4a20-b4ae-b09fa6bfc199 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:25.965953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aaf2c-a7c3-4a20-b4ae-b09fa6bfc199 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:26.032259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9119c4c-1350-4fca-baad-5a04addf86f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:26.032871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9119c4c-1350-4fca-baad-5a04addf86f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:26.033177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9119c4c-1350-4fca-baad-5a04addf86f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:26.033452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9119c4c-1350-4fca-baad-5a04addf86f5 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:26.034081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9119c4c-1350-4fca-baad-5a04addf86f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:26.036020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9119c4c-1350-4fca-baad-5a04addf86f5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:26.063260 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4182/8400] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:38:25 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:38:26.067500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938a163-f038-43df-85b6-4f2f272b7868 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:26.079800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938a163-f038-43df-85b6-4f2f272b7868 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:26.084660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6abfa5-e637-4ae2-9e5e-57d8d05155e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:26.085296 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6abfa5-e637-4ae2-9e5e-57d8d05155e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:26.085570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6abfa5-e637-4ae2-9e5e-57d8d05155e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:26.085833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6abfa5-e637-4ae2-9e5e-57d8d05155e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:26.086496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6abfa5-e637-4ae2-9e5e-57d8d05155e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:26.089158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6abfa5-e637-4ae2-9e5e-57d8d05155e9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:26.104057 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4219/8401] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:38:26.110788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9119c4c-1350-4fca-baad-5a04addf86f5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:26.123945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9119c4c-1350-4fca-baad-5a04addf86f5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:26.129850 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6a9f8-75a0-4dea-8dc2-b0fe228b9e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:26.130519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a9f8-75a0-4dea-8dc2-b0fe228b9e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:26.130798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a9f8-75a0-4dea-8dc2-b0fe228b9e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:26.131137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a9f8-75a0-4dea-8dc2-b0fe228b9e68 admin admin] PATH_INFO: `/v3/projects/8ae615766ce94a5e87b5e65a40075deb/users/a0b6605213214338a02e96aee7ce7cec/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:26.151915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6a9f8-75a0-4dea-8dc2-b0fe228b9e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae615766ce94a5e87b5e65a40075deb, user_id=a0b6605213214338a02e96aee7ce7cec, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:26.155345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6a9f8-75a0-4dea-8dc2-b0fe228b9e68 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:26.171425 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4183/8402] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:38:26 2022] PUT /identity/v3/projects/8ae615766ce94a5e87b5e65a40075deb/users/a0b6605213214338a02e96aee7ce7cec/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:38:26.175781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6abfa5-e637-4ae2-9e5e-57d8d05155e9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:26.187593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6abfa5-e637-4ae2-9e5e-57d8d05155e9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:26.275606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91aca0b-27da-4208-b827-5a9fb3e1283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:26.276224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91aca0b-27da-4208-b827-5a9fb3e1283a admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:26.276497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91aca0b-27da-4208-b827-5a9fb3e1283a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:26.276761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91aca0b-27da-4208-b827-5a9fb3e1283a admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:26.277424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91aca0b-27da-4208-b827-5a9fb3e1283a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:26.279005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91aca0b-27da-4208-b827-5a9fb3e1283a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:26.292880 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4220/8403] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 21:38:26.294392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6a9f8-75a0-4dea-8dc2-b0fe228b9e68 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:26.307370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6a9f8-75a0-4dea-8dc2-b0fe228b9e68 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:26.312689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dffb80-dc81-4d72-8556-0a0c6e3699ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:26.313415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dffb80-dc81-4d72-8556-0a0c6e3699ce admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:26.313693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dffb80-dc81-4d72-8556-0a0c6e3699ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:26.313966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dffb80-dc81-4d72-8556-0a0c6e3699ce admin admin] PATH_INFO: `/v3/projects/8ae615766ce94a5e87b5e65a40075deb/users/a0b6605213214338a02e96aee7ce7cec/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:26.315962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dffb80-dc81-4d72-8556-0a0c6e3699ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae615766ce94a5e87b5e65a40075deb, user_id=a0b6605213214338a02e96aee7ce7cec, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:26.318739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dffb80-dc81-4d72-8556-0a0c6e3699ce admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:26.330866 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4184/8404] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:38:26 2022] PUT /identity/v3/projects/8ae615766ce94a5e87b5e65a40075deb/users/a0b6605213214338a02e96aee7ce7cec/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:38:31.099717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97a16c-36d2-4d26-823e-e80d4fd07c80 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:31.100215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97a16c-36d2-4d26-823e-e80d4fd07c80 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:31.100314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97a16c-36d2-4d26-823e-e80d4fd07c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:31.134713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8f97a16c-36d2-4d26-823e-e80d4fd07c80 None None] MFA Rules not processed for user `a0b6605213214338a02e96aee7ce7cec`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:38:31.136377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8f97a16c-36d2-4d26-823e-e80d4fd07c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:31.239405 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4221/8405] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:38:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:38:31.255448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dffb80-dc81-4d72-8556-0a0c6e3699ce admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:31.257538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e9dffb80-dc81-4d72-8556-0a0c6e3699ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:31.273081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:31.357525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dffb80-dc81-4d72-8556-0a0c6e3699ce admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:31.364334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da587e6f-4042-4588-8777-1348628cc944 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:31.365476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da587e6f-4042-4588-8777-1348628cc944 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:31.365962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da587e6f-4042-4588-8777-1348628cc944 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:31.366245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da587e6f-4042-4588-8777-1348628cc944 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:31.377504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da587e6f-4042-4588-8777-1348628cc944 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:31.379138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da587e6f-4042-4588-8777-1348628cc944 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:31.393785 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4185/8406] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:38:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:38:31.450495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b0cd8-aaf0-4952-97ce-f44ee64f9f2b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:31.450813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b0cd8-aaf0-4952-97ce-f44ee64f9f2b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:31.451040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b0cd8-aaf0-4952-97ce-f44ee64f9f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:31.474786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-251b0cd8-aaf0-4952-97ce-f44ee64f9f2b None None] MFA Rules not processed for user `be7e10dfac9d4ab5a74d803b56cc33db`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:38:31.476012 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-251b0cd8-aaf0-4952-97ce-f44ee64f9f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:31.481172 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:31.545596 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4222/8407] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:38:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:38:31.828875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da587e6f-4042-4588-8777-1348628cc944 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:31.883038 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da587e6f-4042-4588-8777-1348628cc944 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:31.887067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9fbbc-5906-4d44-b674-57b7e264bd5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:31.887654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9fbbc-5906-4d44-b674-57b7e264bd5e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:31.887926 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9fbbc-5906-4d44-b674-57b7e264bd5e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:31.888191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9fbbc-5906-4d44-b674-57b7e264bd5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:31.889934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-61e9fbbc-5906-4d44-b674-57b7e264bd5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:31.902239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9fbbc-5906-4d44-b674-57b7e264bd5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:31.903602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9fbbc-5906-4d44-b674-57b7e264bd5e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:31.914429 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4186/8408] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:38:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:38:33.292615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05fd80-ad22-49ef-8ca7-e9c91e95c609 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:33.297905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05fd80-ad22-49ef-8ca7-e9c91e95c609 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:33.299023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05fd80-ad22-49ef-8ca7-e9c91e95c609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:33.330150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8c05fd80-ad22-49ef-8ca7-e9c91e95c609 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:38:33.332582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8c05fd80-ad22-49ef-8ca7-e9c91e95c609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:33.417863 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4223/8409] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 21:38:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:38:33.431111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9fbbc-5906-4d44-b674-57b7e264bd5e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:38:33.432798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-61e9fbbc-5906-4d44-b674-57b7e264bd5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:33.450949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9fbbc-5906-4d44-b674-57b7e264bd5e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:38:33.457257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5706289-ba23-43bb-bb1f-9be81faef895 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:38:33.457810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5706289-ba23-43bb-bb1f-9be81faef895 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:33.458068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5706289-ba23-43bb-bb1f-9be81faef895 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:33.458329 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5706289-ba23-43bb-bb1f-9be81faef895 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:33.468428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5706289-ba23-43bb-bb1f-9be81faef895 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:38:33.470034 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5706289-ba23-43bb-bb1f-9be81faef895 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:38:33.481837 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4187/8410] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:38:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:38:43.407762 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=50 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:43.425257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac2d8-8b94-4cd7-93f1-26b42b70c1be None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:38:43.425588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac2d8-8b94-4cd7-93f1-26b42b70c1be None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:38:43.425916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac2d8-8b94-4cd7-93f1-26b42b70c1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:38:43.447047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-407ac2d8-8b94-4cd7-93f1-26b42b70c1be None None] MFA Rules not processed for user `be7e10dfac9d4ab5a74d803b56cc33db`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:38:43.448268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-407ac2d8-8b94-4cd7-93f1-26b42b70c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:38:43.455899 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4224/8411] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:38:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:38:43.476717 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=63 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:53.441583 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:39:07.821284 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5706289-ba23-43bb-bb1f-9be81faef895 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:39:07.899499 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5706289-ba23-43bb-bb1f-9be81faef895 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:39:07.904032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:39:07.904648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:39:07.904920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:39:07.905218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] PATH_INFO: `/v3/users/be7e10dfac9d4ab5a74d803b56cc33db` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:39:07.906309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=be7e10dfac9d4ab5a74d803b56cc33db)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:39:07.908224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:39:07.957304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be7e10dfac9d4ab5a74d803b56cc33db'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:07.970875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be7e10dfac9d4ab5a74d803b56cc33db'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:07.978815 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be7e10dfac9d4ab5a74d803b56cc33db'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:07.986299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be7e10dfac9d4ab5a74d803b56cc33db'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:07.987321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be7e10dfac9d4ab5a74d803b56cc33db'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:07.993753 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4188/8412] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:39:07 2022] DELETE /identity/v3/users/be7e10dfac9d4ab5a74d803b56cc33db => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:39:08.173378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ac2d8-8b94-4cd7-93f1-26b42b70c1be None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:39:08.184522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ac2d8-8b94-4cd7-93f1-26b42b70c1be None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:39:08.261537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:39:08.262636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:39:08.263010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:39:08.263370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] PATH_INFO: `/v3/projects/079b9450a9ce4a2f9518bce05c72c49e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:39:08.264829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=079b9450a9ce4a2f9518bce05c72c49e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:39:08.267211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:39:08.326800 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079b9450a9ce4a2f9518bce05c72c49e', 'request_id': 'req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.336738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079b9450a9ce4a2f9518bce05c72c49e', 'request_id': 'req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.343886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079b9450a9ce4a2f9518bce05c72c49e', 'request_id': 'req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.354561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] The token cache is being invalidate because project 079b9450a9ce4a2f9518bce05c72c49e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:39:08.354843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.355478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.357151 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4225/8413] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:39:08 2022] DELETE /identity/v3/projects/079b9450a9ce4a2f9518bce05c72c49e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:39:08.361188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:39:08.363092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:39:08.376294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49407904-dac8-48fe-91d3-2a3d1f97e8fb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:39:08.433399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:39:08.433962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:39:08.434268 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:39:08.434575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] PATH_INFO: `/v3/users/a0b6605213214338a02e96aee7ce7cec` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:39:08.436037 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b6605213214338a02e96aee7ce7cec)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:39:08.438366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:39:08.451525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:39:08.486338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b6605213214338a02e96aee7ce7cec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.498967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b6605213214338a02e96aee7ce7cec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.506987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b6605213214338a02e96aee7ce7cec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.514363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b6605213214338a02e96aee7ce7cec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.514985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b6605213214338a02e96aee7ce7cec'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.520816 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4189/8414] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:39:08 2022] DELETE /identity/v3/users/a0b6605213214338a02e96aee7ce7cec => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:39:08.695610 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:39:08.697062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:39:08.713487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc1111-4c10-44eb-b9d1-041ec0668a7d admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:39:08.773363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:39:08.773999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:39:08.774269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:39:08.774590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] PATH_INFO: `/v3/projects/8ae615766ce94a5e87b5e65a40075deb` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:39:08.775715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae615766ce94a5e87b5e65a40075deb)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:39:08.777636 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:39:08.813350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:39:08.827894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae615766ce94a5e87b5e65a40075deb', 'request_id': 'req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.835482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae615766ce94a5e87b5e65a40075deb', 'request_id': 'req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.846862 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae615766ce94a5e87b5e65a40075deb', 'request_id': 'req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.858635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] The token cache is being invalidate because project 8ae615766ce94a5e87b5e65a40075deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:39:08.858922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.859606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:39:08.861165 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4226/8415] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:39:08 2022] DELETE /identity/v3/projects/8ae615766ce94a5e87b5e65a40075deb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:39:18.517331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=22,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:39:18.855312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:39:33.198222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:39:33.199715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:39:33.267095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0387410-48ce-441f-86d9-fae6533d10e8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:39:33.318192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0044d2c-8d76-49ed-8950-0222baadba89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:39:33.318979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0044d2c-8d76-49ed-8950-0222baadba89 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:39:33.319258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0044d2c-8d76-49ed-8950-0222baadba89 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:39:33.319528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0044d2c-8d76-49ed-8950-0222baadba89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:39:33.321307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d0044d2c-8d76-49ed-8950-0222baadba89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:39:33.335915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0044d2c-8d76-49ed-8950-0222baadba89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:39:33.337591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0044d2c-8d76-49ed-8950-0222baadba89 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:39:33.349014 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4190/8416] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:39:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:39:36.469421 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:39:36.472329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:39:36.598527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69ec8-c7c3-4aab-8b87-c20a0a10e83f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:39:36.605615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e3709-a2d7-44e0-b643-819fa28c30ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:39:36.606371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e3709-a2d7-44e0-b643-819fa28c30ff service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:39:36.606707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e3709-a2d7-44e0-b643-819fa28c30ff service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:39:36.607034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e3709-a2d7-44e0-b643-819fa28c30ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:39:36.609329 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e13e3709-a2d7-44e0-b643-819fa28c30ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:39:36.626360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e3709-a2d7-44e0-b643-819fa28c30ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:39:36.628027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e3709-a2d7-44e0-b643-819fa28c30ff service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:39:36.644981 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4227/8417] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:39:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:39:43.342370 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=50 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:39:46.636669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=43 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:19.324807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0044d2c-8d76-49ed-8950-0222baadba89 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:19.326280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d0044d2c-8d76-49ed-8950-0222baadba89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:19.336506 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0044d2c-8d76-49ed-8950-0222baadba89 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:19.340070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecace6-8ebd-4fdf-82d2-780121f84896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:19.340649 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecace6-8ebd-4fdf-82d2-780121f84896 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:19.340923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecace6-8ebd-4fdf-82d2-780121f84896 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:19.341226 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecace6-8ebd-4fdf-82d2-780121f84896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:19.342892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4ecace6-8ebd-4fdf-82d2-780121f84896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:19.405631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecace6-8ebd-4fdf-82d2-780121f84896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:19.407075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecace6-8ebd-4fdf-82d2-780121f84896 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:19.416883 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4191/8418] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:40:25.752371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e3709-a2d7-44e0-b643-819fa28c30ff service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:25.753978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e13e3709-a2d7-44e0-b643-819fa28c30ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:25.771028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e3709-a2d7-44e0-b643-819fa28c30ff service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:25.778150 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:25.778781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:25.779052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:25.779314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] PATH_INFO: `/v3/users/32a6ad0f3d3b44d48df46e91d80abc95` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:25.780448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a6ad0f3d3b44d48df46e91d80abc95)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:25.782414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:25.834945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a6ad0f3d3b44d48df46e91d80abc95'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:25.843145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a6ad0f3d3b44d48df46e91d80abc95'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:25.862497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a6ad0f3d3b44d48df46e91d80abc95'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:25.873255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a6ad0f3d3b44d48df46e91d80abc95'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:25.873907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a6ad0f3d3b44d48df46e91d80abc95'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:25.880638 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4228/8419] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:40:25 2022] DELETE /identity/v3/users/32a6ad0f3d3b44d48df46e91d80abc95 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:26.060821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecace6-8ebd-4fdf-82d2-780121f84896 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:26.071264 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecace6-8ebd-4fdf-82d2-780121f84896 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:26.131802 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:26.132424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:26.132695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:26.132961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] PATH_INFO: `/v3/projects/b11f869e7a3e49e59e3236788206474b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:26.134192 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b11f869e7a3e49e59e3236788206474b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:26.136115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:26.184128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b11f869e7a3e49e59e3236788206474b', 'request_id': 'req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.191565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b11f869e7a3e49e59e3236788206474b', 'request_id': 'req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.198735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b11f869e7a3e49e59e3236788206474b', 'request_id': 'req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.209587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] The token cache is being invalidate because project b11f869e7a3e49e59e3236788206474b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:40:26.209871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.210523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.212706 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4192/8420] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:40:26 2022] DELETE /identity/v3/projects/b11f869e7a3e49e59e3236788206474b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:26.216743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:26.218191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:26.231694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7a134-b451-485f-83cf-4248e3132bbb admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:26.288280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:26.288905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:26.289210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:26.289473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] PATH_INFO: `/v3/users/ff92b45b4784481b8bdf30e923d4f4ce` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:26.290581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff92b45b4784481b8bdf30e923d4f4ce)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:26.292496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:26.338171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff92b45b4784481b8bdf30e923d4f4ce'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.345245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff92b45b4784481b8bdf30e923d4f4ce'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.357362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff92b45b4784481b8bdf30e923d4f4ce'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.358967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:26.365465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff92b45b4784481b8bdf30e923d4f4ce'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.366557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-29e6c335-9791-4223-9c54-3a33de44bea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff92b45b4784481b8bdf30e923d4f4ce'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.373669 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4229/8421] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:40:26 2022] DELETE /identity/v3/users/ff92b45b4784481b8bdf30e923d4f4ce => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:26.545550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:26.547085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:26.560626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1225e8-18fe-4569-8db7-cf38c9fb7e18 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:26.582127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:26.622936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:26.623865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:26.624145 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:26.624416 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] PATH_INFO: `/v3/projects/22fd84ef105949f98fb0226bf6ca2485` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:26.625582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=22fd84ef105949f98fb0226bf6ca2485)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:26.627538 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:26.673982 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22fd84ef105949f98fb0226bf6ca2485', 'request_id': 'req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.682129 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22fd84ef105949f98fb0226bf6ca2485', 'request_id': 'req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.688550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22fd84ef105949f98fb0226bf6ca2485', 'request_id': 'req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.699180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] The token cache is being invalidate because project 22fd84ef105949f98fb0226bf6ca2485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:40:26.699467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.700122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-ca609bfc-7e27-4c6d-bb66-a169d2a1cd6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:26.701647 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4193/8422] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:40:26 2022] DELETE /identity/v3/projects/22fd84ef105949f98fb0226bf6ca2485 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:36.370558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:36.696027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:36.730003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b641-495c-4340-961d-d9f7a5962cdf None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:36.730282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b641-495c-4340-961d-d9f7a5962cdf None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:36.730557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b641-495c-4340-961d-d9f7a5962cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:36.769915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-2f08b641-495c-4340-961d-d9f7a5962cdf None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:36.771113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2f08b641-495c-4340-961d-d9f7a5962cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:36.829113 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4230/8423] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:36.840430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb00ac9-b2a7-434a-989b-79f8986b6efd None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:36.840708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb00ac9-b2a7-434a-989b-79f8986b6efd None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:36.840973 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb00ac9-b2a7-434a-989b-79f8986b6efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:36.861253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-fcb00ac9-b2a7-434a-989b-79f8986b6efd None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:36.862445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fcb00ac9-b2a7-434a-989b-79f8986b6efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:36.870148 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4194/8424] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:36.874782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08b641-495c-4340-961d-d9f7a5962cdf None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:36.884321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08b641-495c-4340-961d-d9f7a5962cdf None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:36.890180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecda6f5-3fba-423f-8139-c606a549d633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:36.890776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecda6f5-3fba-423f-8139-c606a549d633 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:36.891052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecda6f5-3fba-423f-8139-c606a549d633 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:36.891321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecda6f5-3fba-423f-8139-c606a549d633 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:36.891962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecda6f5-3fba-423f-8139-c606a549d633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:40:36.892264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecda6f5-3fba-423f-8139-c606a549d633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:36.893883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecda6f5-3fba-423f-8139-c606a549d633 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:36.906129 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4231/8425] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:40:36.919989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb00ac9-b2a7-434a-989b-79f8986b6efd None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:36.929157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb00ac9-b2a7-434a-989b-79f8986b6efd None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:36.933779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40e1a1-4b88-4e96-abea-04e1907a5096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:36.934358 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e1a1-4b88-4e96-abea-04e1907a5096 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:36.934629 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e1a1-4b88-4e96-abea-04e1907a5096 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:36.934893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e1a1-4b88-4e96-abea-04e1907a5096 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:36.935507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40e1a1-4b88-4e96-abea-04e1907a5096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:36.937544 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40e1a1-4b88-4e96-abea-04e1907a5096 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:36.973695 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4195/8426] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:40:36 2022] POST /identity/v3/projects => generated 393 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:40:36.977915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecda6f5-3fba-423f-8139-c606a549d633 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:36.988528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecda6f5-3fba-423f-8139-c606a549d633 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:37.046885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc5d00-137c-439c-8a7d-614de7fde4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:37.047487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc5d00-137c-439c-8a7d-614de7fde4a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.047764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc5d00-137c-439c-8a7d-614de7fde4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.048033 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc5d00-137c-439c-8a7d-614de7fde4a6 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.048650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc5d00-137c-439c-8a7d-614de7fde4a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:37.050578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc5d00-137c-439c-8a7d-614de7fde4a6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:37.071988 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4232/8427] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:40:36 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:40:37.076579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40e1a1-4b88-4e96-abea-04e1907a5096 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:37.086922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40e1a1-4b88-4e96-abea-04e1907a5096 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:37.092167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fba499-533a-48b5-90ba-f04c9ab4e0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:37.092745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fba499-533a-48b5-90ba-f04c9ab4e0eb admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.093019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fba499-533a-48b5-90ba-f04c9ab4e0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.093316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fba499-533a-48b5-90ba-f04c9ab4e0eb admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.093937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fba499-533a-48b5-90ba-f04c9ab4e0eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:37.095530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fba499-533a-48b5-90ba-f04c9ab4e0eb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:37.105717 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4196/8428] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:40:37.109834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc5d00-137c-439c-8a7d-614de7fde4a6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:37.119795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc5d00-137c-439c-8a7d-614de7fde4a6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:37.124425 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2ad9d-3c9f-4f22-b17d-687b689ed5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:37.125009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2ad9d-3c9f-4f22-b17d-687b689ed5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.125313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2ad9d-3c9f-4f22-b17d-687b689ed5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.125585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2ad9d-3c9f-4f22-b17d-687b689ed5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.126212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2ad9d-3c9f-4f22-b17d-687b689ed5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:37.127828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2ad9d-3c9f-4f22-b17d-687b689ed5d9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:37.139164 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4233/8429] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:40:37.143298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fba499-533a-48b5-90ba-f04c9ab4e0eb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:37.154282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fba499-533a-48b5-90ba-f04c9ab4e0eb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:37.158864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4818243-9ea2-4642-b893-0489c4d68418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:37.159487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4818243-9ea2-4642-b893-0489c4d68418 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.159788 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4818243-9ea2-4642-b893-0489c4d68418 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.160053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4818243-9ea2-4642-b893-0489c4d68418 admin admin] PATH_INFO: `/v3/projects/e41304ad412e441eb0941973df577f98/users/11e52af42d814aed8ea14a256678bce9/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.175346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4818243-9ea2-4642-b893-0489c4d68418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41304ad412e441eb0941973df577f98, user_id=11e52af42d814aed8ea14a256678bce9, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:37.178091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4818243-9ea2-4642-b893-0489c4d68418 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:37.191151 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4197/8430] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:40:37 2022] PUT /identity/v3/projects/e41304ad412e441eb0941973df577f98/users/11e52af42d814aed8ea14a256678bce9/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:37.202761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c0a35-eda0-4caa-9ce8-84b42f27b48e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.203154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c0a35-eda0-4caa-9ce8-84b42f27b48e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.203427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c0a35-eda0-4caa-9ce8-84b42f27b48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.231547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ff6c0a35-eda0-4caa-9ce8-84b42f27b48e None None] MFA Rules not processed for user `11e52af42d814aed8ea14a256678bce9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:37.232787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ff6c0a35-eda0-4caa-9ce8-84b42f27b48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:37.273743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:37.299420 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4234/8431] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:37.303991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4818243-9ea2-4642-b893-0489c4d68418 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:37.366046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4818243-9ea2-4642-b893-0489c4d68418 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:37.371193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883f82d-0fad-4654-8e79-8d081add2094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:37.371783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883f82d-0fad-4654-8e79-8d081add2094 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.372067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883f82d-0fad-4654-8e79-8d081add2094 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.372336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883f82d-0fad-4654-8e79-8d081add2094 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.372962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883f82d-0fad-4654-8e79-8d081add2094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:37.374913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883f82d-0fad-4654-8e79-8d081add2094 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:37.402076 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4198/8432] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:40:37 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:40:37.406098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c0a35-eda0-4caa-9ce8-84b42f27b48e None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:37.416822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c0a35-eda0-4caa-9ce8-84b42f27b48e None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:37.474312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99db2ab-8481-4ac9-9485-2fe8e414be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:37.474911 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99db2ab-8481-4ac9-9485-2fe8e414be34 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.475181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99db2ab-8481-4ac9-9485-2fe8e414be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.475444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99db2ab-8481-4ac9-9485-2fe8e414be34 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.476059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99db2ab-8481-4ac9-9485-2fe8e414be34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:37.478005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99db2ab-8481-4ac9-9485-2fe8e414be34 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:37.499533 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4235/8433] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:40:37 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:40:37.503789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883f82d-0fad-4654-8e79-8d081add2094 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:37.514922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883f82d-0fad-4654-8e79-8d081add2094 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:37.519493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39949528-b2df-4da0-a62c-30c89ee9f17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:37.520071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39949528-b2df-4da0-a62c-30c89ee9f17b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.520345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39949528-b2df-4da0-a62c-30c89ee9f17b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.520610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39949528-b2df-4da0-a62c-30c89ee9f17b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.521294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39949528-b2df-4da0-a62c-30c89ee9f17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:37.522878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39949528-b2df-4da0-a62c-30c89ee9f17b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:37.533338 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4199/8434] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:40:37.537574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99db2ab-8481-4ac9-9485-2fe8e414be34 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:37.547599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99db2ab-8481-4ac9-9485-2fe8e414be34 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:37.552271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36651d5c-17d2-4ddd-b881-bc6e55c2520e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:37.552907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36651d5c-17d2-4ddd-b881-bc6e55c2520e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.553220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36651d5c-17d2-4ddd-b881-bc6e55c2520e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.553492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36651d5c-17d2-4ddd-b881-bc6e55c2520e admin admin] PATH_INFO: `/v3/projects/6e13cade98794076a20e38791ad0cddc/users/e4c0698f643f4e7cb3f898c453f51fab/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.568686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36651d5c-17d2-4ddd-b881-bc6e55c2520e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e13cade98794076a20e38791ad0cddc, user_id=e4c0698f643f4e7cb3f898c453f51fab, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:37.571339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36651d5c-17d2-4ddd-b881-bc6e55c2520e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:37.584961 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4236/8435] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:40:37 2022] PUT /identity/v3/projects/6e13cade98794076a20e38791ad0cddc/users/e4c0698f643f4e7cb3f898c453f51fab/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:37.589663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39949528-b2df-4da0-a62c-30c89ee9f17b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:37.599584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39949528-b2df-4da0-a62c-30c89ee9f17b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:37.657048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a214-2e3a-458f-8048-cf183f9ba6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:37.657653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a214-2e3a-458f-8048-cf183f9ba6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.657938 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a214-2e3a-458f-8048-cf183f9ba6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.658209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a214-2e3a-458f-8048-cf183f9ba6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.658848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a214-2e3a-458f-8048-cf183f9ba6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:37.660453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a214-2e3a-458f-8048-cf183f9ba6a7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:37.662660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:37.672425 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4200/8436] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:40:37.676441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36651d5c-17d2-4ddd-b881-bc6e55c2520e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:37.687463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36651d5c-17d2-4ddd-b881-bc6e55c2520e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:37.692994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428a0ee-db09-4b2f-a6b7-4e66cad2196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:37.693660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428a0ee-db09-4b2f-a6b7-4e66cad2196e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.693929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428a0ee-db09-4b2f-a6b7-4e66cad2196e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.694193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428a0ee-db09-4b2f-a6b7-4e66cad2196e admin admin] PATH_INFO: `/v3/projects/6e13cade98794076a20e38791ad0cddc/users/e4c0698f643f4e7cb3f898c453f51fab/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.696066 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428a0ee-db09-4b2f-a6b7-4e66cad2196e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e13cade98794076a20e38791ad0cddc, user_id=e4c0698f643f4e7cb3f898c453f51fab, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:37.698725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428a0ee-db09-4b2f-a6b7-4e66cad2196e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:37.708661 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4237/8437] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:40:37 2022] PUT /identity/v3/projects/6e13cade98794076a20e38791ad0cddc/users/e4c0698f643f4e7cb3f898c453f51fab/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:37.718781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13072dae-0994-452d-a4d4-00c00085df57 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.719252 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13072dae-0994-452d-a4d4-00c00085df57 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.719347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13072dae-0994-452d-a4d4-00c00085df57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.747884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-13072dae-0994-452d-a4d4-00c00085df57 None None] MFA Rules not processed for user `e4c0698f643f4e7cb3f898c453f51fab`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:37.749122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-13072dae-0994-452d-a4d4-00c00085df57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:37.817299 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4201/8438] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:37 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:37.830624 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf41f-5245-443e-9105-e03925ceba13 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:37.830901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf41f-5245-443e-9105-e03925ceba13 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:37.831165 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf41f-5245-443e-9105-e03925ceba13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:37.850075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fccaf41f-5245-443e-9105-e03925ceba13 None None] MFA Rules not processed for user `11e52af42d814aed8ea14a256678bce9`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:37.851252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fccaf41f-5245-443e-9105-e03925ceba13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:37.908260 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4238/8439] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:37.922005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13072dae-0994-452d-a4d4-00c00085df57 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:37.923533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-13072dae-0994-452d-a4d4-00c00085df57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:37.999740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13072dae-0994-452d-a4d4-00c00085df57 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:38.005343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490dc0c-e3b3-45a9-a439-13ac23d9df1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:38.005934 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490dc0c-e3b3-45a9-a439-13ac23d9df1b service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:38.006207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490dc0c-e3b3-45a9-a439-13ac23d9df1b service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:38.006472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490dc0c-e3b3-45a9-a439-13ac23d9df1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:38.015078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490dc0c-e3b3-45a9-a439-13ac23d9df1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:38.016657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490dc0c-e3b3-45a9-a439-13ac23d9df1b service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:38.025555 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4202/8440] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:40:37 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:40:38.056135 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcc232-290c-4b3b-8a31-bf4882dafa90 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:38.056417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcc232-290c-4b3b-8a31-bf4882dafa90 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:38.056688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcc232-290c-4b3b-8a31-bf4882dafa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:38.106633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d2bcc232-290c-4b3b-8a31-bf4882dafa90 None None] MFA Rules not processed for user `af8591ebfa174c91853b673360326a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:38.107820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d2bcc232-290c-4b3b-8a31-bf4882dafa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:38.116197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:38.170007 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4239/8441] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Oct 15 21:40:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:38.176241 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490dc0c-e3b3-45a9-a439-13ac23d9df1b service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:38.185594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490dc0c-e3b3-45a9-a439-13ac23d9df1b service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:38.189426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fbcb8-7fd6-4821-acd4-0b574b23df1a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:38.190009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fbcb8-7fd6-4821-acd4-0b574b23df1a service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:38.190302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fbcb8-7fd6-4821-acd4-0b574b23df1a service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:38.190567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fbcb8-7fd6-4821-acd4-0b574b23df1a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:38.193600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b20fbcb8-7fd6-4821-acd4-0b574b23df1a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:38.210642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fbcb8-7fd6-4821-acd4-0b574b23df1a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:38.212070 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fbcb8-7fd6-4821-acd4-0b574b23df1a service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:38.226435 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4203/8442] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:40:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:40:41.502474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcc232-290c-4b3b-8a31-bf4882dafa90 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:41.503933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d2bcc232-290c-4b3b-8a31-bf4882dafa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:41.565570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcc232-290c-4b3b-8a31-bf4882dafa90 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:41.569261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1771634-b769-4857-86ed-7f51b2a3554c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:41.569972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771634-b769-4857-86ed-7f51b2a3554c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:41.570307 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771634-b769-4857-86ed-7f51b2a3554c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:41.570639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771634-b769-4857-86ed-7f51b2a3554c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:41.572412 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b1771634-b769-4857-86ed-7f51b2a3554c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:41.585281 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1771634-b769-4857-86ed-7f51b2a3554c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:41.586733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1771634-b769-4857-86ed-7f51b2a3554c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:41.599172 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4240/8443] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:40:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:40:42.330911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fbcb8-7fd6-4821-acd4-0b574b23df1a service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:42.348507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fbcb8-7fd6-4821-acd4-0b574b23df1a service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:42.355426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeabd11-78ec-4fa9-aad4-6ac2770bdd0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:42.357209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeabd11-78ec-4fa9-aad4-6ac2770bdd0e service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:42.357609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeabd11-78ec-4fa9-aad4-6ac2770bdd0e service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:42.357881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeabd11-78ec-4fa9-aad4-6ac2770bdd0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:42.359664 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-caeabd11-78ec-4fa9-aad4-6ac2770bdd0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:42.375759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeabd11-78ec-4fa9-aad4-6ac2770bdd0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:42.377633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeabd11-78ec-4fa9-aad4-6ac2770bdd0e service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:42.391786 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4204/8444] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:40:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:40:47.076512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1771634-b769-4857-86ed-7f51b2a3554c service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:47.079333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b1771634-b769-4857-86ed-7f51b2a3554c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:47.101886 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1771634-b769-4857-86ed-7f51b2a3554c service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:47.107866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c5eac-45c8-453c-a894-6717e8cb1b6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:47.109163 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c5eac-45c8-453c-a894-6717e8cb1b6f service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:47.109442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c5eac-45c8-453c-a894-6717e8cb1b6f service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:47.109712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c5eac-45c8-453c-a894-6717e8cb1b6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:47.111461 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fd6c5eac-45c8-453c-a894-6717e8cb1b6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:47.127062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c5eac-45c8-453c-a894-6717e8cb1b6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:47.128674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c5eac-45c8-453c-a894-6717e8cb1b6f service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:47.143314 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4241/8445] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:40:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:40:49.208477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeabd11-78ec-4fa9-aad4-6ac2770bdd0e service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:49.222667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeabd11-78ec-4fa9-aad4-6ac2770bdd0e service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:49.228170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c1140-22ed-4164-9816-cf7d730654ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:49.228750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c1140-22ed-4164-9816-cf7d730654ca service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:49.229024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c1140-22ed-4164-9816-cf7d730654ca service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:49.229316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c1140-22ed-4164-9816-cf7d730654ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:49.237782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c1140-22ed-4164-9816-cf7d730654ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:49.239343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c1140-22ed-4164-9816-cf7d730654ca service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:49.249637 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4205/8446] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:40:49 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:40:49.264665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c53285-34c4-4a92-8bf0-6dc00b1a0fe5 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:49.265155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c53285-34c4-4a92-8bf0-6dc00b1a0fe5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:49.265268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c53285-34c4-4a92-8bf0-6dc00b1a0fe5 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:49.266563 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4242/8447] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 21:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:40:49.271530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c1140-22ed-4164-9816-cf7d730654ca service nova] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 21:40:49.285939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c1140-22ed-4164-9816-cf7d730654ca service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:49.294158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c1140-22ed-4164-9816-cf7d730654ca service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:49.299376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbda04c-0dcf-4859-8969-ce55d00304d8 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4c0698f643f4e7cb3f898c453f51fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e13cade98794076a20e38791ad0cddc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:49.299981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbda04c-0dcf-4859-8969-ce55d00304d8 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:49.300256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbda04c-0dcf-4859-8969-ce55d00304d8 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:49.300525 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbda04c-0dcf-4859-8969-ce55d00304d8 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] PATH_INFO: `/v3/projects/6e13cade98794076a20e38791ad0cddc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:49.301657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbda04c-0dcf-4859-8969-ce55d00304d8 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] RBAC: Authorizing `identity:get_project(project_id=6e13cade98794076a20e38791ad0cddc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:49.303425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbda04c-0dcf-4859-8969-ce55d00304d8 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:49.305019 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4206/8448] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 21:40:49 2022] GET /identity/v3/projects/6e13cade98794076a20e38791ad0cddc => generated 393 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:40:49.607302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddece0-370d-46cc-9bcf-0a636a7c6945 None None] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:49.608184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddece0-370d-46cc-9bcf-0a636a7c6945 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:49.608606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddece0-370d-46cc-9bcf-0a636a7c6945 None None] PATH_INFO: `/` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:49.610025 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4243/8449] 10.222.0.16 () {54 vars in 912 bytes} [Sat Oct 15 21:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 21:40:49.614729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbda04c-0dcf-4859-8969-ce55d00304d8 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Authenticating service token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 15 21:40:49.630530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbda04c-0dcf-4859-8969-ce55d00304d8 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:49.639829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbda04c-0dcf-4859-8969-ce55d00304d8 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:49.644421 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3da86-f129-461c-8dbb-6b118252e4ea tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4c0698f643f4e7cb3f898c453f51fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e13cade98794076a20e38791ad0cddc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:49.645097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3da86-f129-461c-8dbb-6b118252e4ea tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:49.645379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3da86-f129-461c-8dbb-6b118252e4ea tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:49.645652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3da86-f129-461c-8dbb-6b118252e4ea tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] PATH_INFO: `/v3/projects/6e13cade98794076a20e38791ad0cddc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:49.646769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3da86-f129-461c-8dbb-6b118252e4ea tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] RBAC: Authorizing `identity:get_project(project_id=6e13cade98794076a20e38791ad0cddc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:49.648591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3da86-f129-461c-8dbb-6b118252e4ea tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:49.650258 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4207/8450] 10.222.0.16 () {58 vars in 1443 bytes} [Sat Oct 15 21:40:49 2022] GET /identity/v3/projects/6e13cade98794076a20e38791ad0cddc => generated 393 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:40:52.449557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddece0-370d-46cc-9bcf-0a636a7c6945 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:52.513626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddece0-370d-46cc-9bcf-0a636a7c6945 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:52.518816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:52.519437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:52.519707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:52.519968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] PATH_INFO: `/v3/users/11e52af42d814aed8ea14a256678bce9` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:52.521106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e52af42d814aed8ea14a256678bce9)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:52.523002 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:52.566346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:52.568082 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e52af42d814aed8ea14a256678bce9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:52.575789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e52af42d814aed8ea14a256678bce9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:52.588071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e52af42d814aed8ea14a256678bce9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:52.595403 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e52af42d814aed8ea14a256678bce9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:52.596052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e52af42d814aed8ea14a256678bce9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:52.602013 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4244/8451] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:40:52 2022] DELETE /identity/v3/users/11e52af42d814aed8ea14a256678bce9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:52.606812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82a9fd-f7a2-4d5b-8bb4-f1156a3b388e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:52.607270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82a9fd-f7a2-4d5b-8bb4-f1156a3b388e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:52.607465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82a9fd-f7a2-4d5b-8bb4-f1156a3b388e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:52.627682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8e82a9fd-f7a2-4d5b-8bb4-f1156a3b388e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:52.628890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8e82a9fd-f7a2-4d5b-8bb4-f1156a3b388e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:52.634297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:52.689689 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4208/8452] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:52.699473 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:52.700859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:52.757673 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb19be-de62-4506-9910-4e339917c139 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:52.761305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e98b6-5ac7-4cbf-b92b-d28009ac735f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:52.761894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e98b6-5ac7-4cbf-b92b-d28009ac735f service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:52.762162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e98b6-5ac7-4cbf-b92b-d28009ac735f service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:52.762686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e98b6-5ac7-4cbf-b92b-d28009ac735f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:52.771577 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e98b6-5ac7-4cbf-b92b-d28009ac735f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:52.772976 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e98b6-5ac7-4cbf-b92b-d28009ac735f service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:52.782250 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4245/8453] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:40:52.954195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82a9fd-f7a2-4d5b-8bb4-f1156a3b388e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:52.955632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8e82a9fd-f7a2-4d5b-8bb4-f1156a3b388e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:52.967471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82a9fd-f7a2-4d5b-8bb4-f1156a3b388e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:52.973866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:52.974513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:52.974828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:52.975411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] PATH_INFO: `/v3/projects/e41304ad412e441eb0941973df577f98` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:52.976598 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e41304ad412e441eb0941973df577f98)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:52.978563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:53.029796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41304ad412e441eb0941973df577f98', 'request_id': 'req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.037747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41304ad412e441eb0941973df577f98', 'request_id': 'req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.044117 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41304ad412e441eb0941973df577f98', 'request_id': 'req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.054425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] The token cache is being invalidate because project e41304ad412e441eb0941973df577f98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:40:53.054705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.055330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.056805 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4209/8454] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:40:52 2022] DELETE /identity/v3/projects/e41304ad412e441eb0941973df577f98 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:53.060903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e98b6-5ac7-4cbf-b92b-d28009ac735f service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:53.072094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e98b6-5ac7-4cbf-b92b-d28009ac735f service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:53.131188 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:53.131821 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:53.132098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:53.132368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] PATH_INFO: `/v3/users/e4c0698f643f4e7cb3f898c453f51fab` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:53.133513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c0698f643f4e7cb3f898c453f51fab)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:53.135407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:53.185388 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c0698f643f4e7cb3f898c453f51fab'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.193134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c0698f643f4e7cb3f898c453f51fab'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.201061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:53.207097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c0698f643f4e7cb3f898c453f51fab'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.217571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c0698f643f4e7cb3f898c453f51fab'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.218193 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-ff35295c-4ab1-4c01-9a3e-076efc0c4f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c0698f643f4e7cb3f898c453f51fab'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.224348 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4246/8455] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:40:53 2022] DELETE /identity/v3/users/e4c0698f643f4e7cb3f898c453f51fab => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:53.502357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:53.504107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:53.520352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a8a0c-00e2-4467-9ae1-1107ebca7027 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:53.578747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:53.579339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:53.579607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:53.579873 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] PATH_INFO: `/v3/projects/6e13cade98794076a20e38791ad0cddc` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:53.580968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e13cade98794076a20e38791ad0cddc)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:53.582896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:53.619941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:53.630435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e13cade98794076a20e38791ad0cddc', 'request_id': 'req-b5ca00bc-1369-4220-b336-0ea4225d9137'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.637582 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e13cade98794076a20e38791ad0cddc', 'request_id': 'req-b5ca00bc-1369-4220-b336-0ea4225d9137'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.644874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e13cade98794076a20e38791ad0cddc', 'request_id': 'req-b5ca00bc-1369-4220-b336-0ea4225d9137'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.655282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] The token cache is being invalidate because project 6e13cade98794076a20e38791ad0cddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:40:53.655591 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.656495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-b5ca00bc-1369-4220-b336-0ea4225d9137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:40:53.658064 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4210/8456] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:40:53 2022] DELETE /identity/v3/projects/6e13cade98794076a20e38791ad0cddc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:53.664034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535c0e5-fad3-4424-b304-7d132c6d27dc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:53.664502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535c0e5-fad3-4424-b304-7d132c6d27dc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:53.664615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535c0e5-fad3-4424-b304-7d132c6d27dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:53.684289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7535c0e5-fad3-4424-b304-7d132c6d27dc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:53.685506 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7535c0e5-fad3-4424-b304-7d132c6d27dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:53.747517 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4247/8457] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:40:53.755881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d888e-9065-47bf-96ce-ea22a573c29a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:53.756164 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d888e-9065-47bf-96ce-ea22a573c29a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:53.756435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d888e-9065-47bf-96ce-ea22a573c29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:53.776877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-537d888e-9065-47bf-96ce-ea22a573c29a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:53.778091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-537d888e-9065-47bf-96ce-ea22a573c29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:53.787057 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4211/8458] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:53.791212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535c0e5-fad3-4424-b304-7d132c6d27dc None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:53.801746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535c0e5-fad3-4424-b304-7d132c6d27dc None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:53.806228 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae169587-726a-4091-b491-53d2a7fb3fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:53.806808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae169587-726a-4091-b491-53d2a7fb3fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:53.807076 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae169587-726a-4091-b491-53d2a7fb3fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:53.807337 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae169587-726a-4091-b491-53d2a7fb3fb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:53.807963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae169587-726a-4091-b491-53d2a7fb3fb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:40:53.808254 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae169587-726a-4091-b491-53d2a7fb3fb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:53.809850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae169587-726a-4091-b491-53d2a7fb3fb6 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:53.821361 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4248/8459] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 21:40:53.826492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d888e-9065-47bf-96ce-ea22a573c29a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:53.836031 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d888e-9065-47bf-96ce-ea22a573c29a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:53.840711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd443d-7217-4bd9-b4e4-11c096795e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:53.841330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd443d-7217-4bd9-b4e4-11c096795e52 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:53.841609 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd443d-7217-4bd9-b4e4-11c096795e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:53.841877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd443d-7217-4bd9-b4e4-11c096795e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:53.842495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd443d-7217-4bd9-b4e4-11c096795e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:53.844906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd443d-7217-4bd9-b4e4-11c096795e52 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:53.872492 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4212/8460] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:40:53 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:40:53.876504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae169587-726a-4091-b491-53d2a7fb3fb6 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:53.886373 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae169587-726a-4091-b491-53d2a7fb3fb6 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:53.941942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d948-4934-4018-907f-143b7721bb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:53.942533 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d948-4934-4018-907f-143b7721bb8a admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:53.942805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d948-4934-4018-907f-143b7721bb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:53.943070 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d948-4934-4018-907f-143b7721bb8a admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:53.943680 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287d948-4934-4018-907f-143b7721bb8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:53.945595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287d948-4934-4018-907f-143b7721bb8a admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:53.966724 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4249/8461] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:40:53 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:40:53.971322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd443d-7217-4bd9-b4e4-11c096795e52 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:53.981997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd443d-7217-4bd9-b4e4-11c096795e52 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:53.987337 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902e384-fa43-4ce5-a9ae-f0f29e18a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:53.987915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902e384-fa43-4ce5-a9ae-f0f29e18a1ad admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:53.988189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902e384-fa43-4ce5-a9ae-f0f29e18a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:53.988457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902e384-fa43-4ce5-a9ae-f0f29e18a1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:53.989122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902e384-fa43-4ce5-a9ae-f0f29e18a1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:53.990723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902e384-fa43-4ce5-a9ae-f0f29e18a1ad admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:54.000978 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4213/8462] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:40:54.004932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d948-4934-4018-907f-143b7721bb8a admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:54.016146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d948-4934-4018-907f-143b7721bb8a admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:54.020743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b989bfb-bb31-4a48-98a2-9a454e3cb251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:54.021359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b989bfb-bb31-4a48-98a2-9a454e3cb251 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:54.021633 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b989bfb-bb31-4a48-98a2-9a454e3cb251 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:54.021901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b989bfb-bb31-4a48-98a2-9a454e3cb251 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:54.022520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b989bfb-bb31-4a48-98a2-9a454e3cb251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:54.024069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b989bfb-bb31-4a48-98a2-9a454e3cb251 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:54.035273 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4250/8463] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:40:54.039552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902e384-fa43-4ce5-a9ae-f0f29e18a1ad admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:54.050094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902e384-fa43-4ce5-a9ae-f0f29e18a1ad admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:54.054857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb746c0a-d1cb-492c-9f66-13bd298393ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:54.055519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb746c0a-d1cb-492c-9f66-13bd298393ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:54.055795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb746c0a-d1cb-492c-9f66-13bd298393ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:54.056063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb746c0a-d1cb-492c-9f66-13bd298393ee admin admin] PATH_INFO: `/v3/projects/512ac462334c424eb45066ea7647a3a9/users/c1bffd7ffed94cf595a69eaa7fa758ed/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:54.070732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb746c0a-d1cb-492c-9f66-13bd298393ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=512ac462334c424eb45066ea7647a3a9, user_id=c1bffd7ffed94cf595a69eaa7fa758ed, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:54.073428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb746c0a-d1cb-492c-9f66-13bd298393ee admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:54.086574 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4214/8464] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:40:54 2022] PUT /identity/v3/projects/512ac462334c424eb45066ea7647a3a9/users/c1bffd7ffed94cf595a69eaa7fa758ed/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:40:54.091544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f2d29-1ec6-47bc-bb28-d7c11229713a None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:54.091871 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f2d29-1ec6-47bc-bb28-d7c11229713a None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:54.092117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f2d29-1ec6-47bc-bb28-d7c11229713a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:54.112353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-b41f2d29-1ec6-47bc-bb28-d7c11229713a None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:54.113576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b41f2d29-1ec6-47bc-bb28-d7c11229713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:54.157714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:54.173127 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4251/8465] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:54.182652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7aa67-b06e-417e-9935-830ff6a62f2b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:54.182931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7aa67-b06e-417e-9935-830ff6a62f2b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:54.183224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7aa67-b06e-417e-9935-830ff6a62f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:54.204703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-18d7aa67-b06e-417e-9935-830ff6a62f2b None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:54.205963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-18d7aa67-b06e-417e-9935-830ff6a62f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:54.255872 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4215/8466] 10.222.0.16 () {58 vars in 1061 bytes} [Sat Oct 15 21:40:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:40:54.262063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f2d29-1ec6-47bc-bb28-d7c11229713a None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:54.272398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f2d29-1ec6-47bc-bb28-d7c11229713a None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:54.276024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399fce1-1beb-4066-969b-de435a438f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:54.276604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399fce1-1beb-4066-969b-de435a438f33 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:54.276878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399fce1-1beb-4066-969b-de435a438f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:54.277176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399fce1-1beb-4066-969b-de435a438f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:54.286813 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399fce1-1beb-4066-969b-de435a438f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:54.288250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399fce1-1beb-4066-969b-de435a438f33 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:54.297821 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4252/8467] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:40:58.654917 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eabf62-fde8-4d11-b71c-4a8a5a37f943 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:58.655402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eabf62-fde8-4d11-b71c-4a8a5a37f943 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:58.655402 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eabf62-fde8-4d11-b71c-4a8a5a37f943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:58.686260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-92eabf62-fde8-4d11-b71c-4a8a5a37f943 None None] MFA Rules not processed for user `c1bffd7ffed94cf595a69eaa7fa758ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:58.687833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-92eabf62-fde8-4d11-b71c-4a8a5a37f943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:58.764694 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:58.779535 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4216/8468] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:58.781994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e84363-752e-41a5-a693-325ac73052c6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:58.782486 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e84363-752e-41a5-a693-325ac73052c6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:58.782585 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e84363-752e-41a5-a693-325ac73052c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:58.803187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-60e84363-752e-41a5-a693-325ac73052c6 None None] MFA Rules not processed for user `c1bffd7ffed94cf595a69eaa7fa758ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:40:58.804537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-60e84363-752e-41a5-a693-325ac73052c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:58.811756 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4253/8469] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:40:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:40:58.823678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eabf62-fde8-4d11-b71c-4a8a5a37f943 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:40:58.825153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-92eabf62-fde8-4d11-b71c-4a8a5a37f943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:40:58.915532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eabf62-fde8-4d11-b71c-4a8a5a37f943 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:40:58.921837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b167c-d739-433e-a4b4-81ef4c32b176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:40:58.922590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b167c-d739-433e-a4b4-81ef4c32b176 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:40:58.922972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b167c-d739-433e-a4b4-81ef4c32b176 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:40:58.923346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b167c-d739-433e-a4b4-81ef4c32b176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:40:58.932549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b167c-d739-433e-a4b4-81ef4c32b176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:40:58.934531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b167c-d739-433e-a4b4-81ef4c32b176 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:40:58.950004 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4217/8470] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:40:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:41:08.797341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=13 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:08.941519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=34 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:09.594102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e84363-752e-41a5-a693-325ac73052c6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:09.603733 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e84363-752e-41a5-a693-325ac73052c6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:09.607353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729ac3b-36da-47b9-a1e9-4f4d0c355e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:09.607946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729ac3b-36da-47b9-a1e9-4f4d0c355e21 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:09.608251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729ac3b-36da-47b9-a1e9-4f4d0c355e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:09.608518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729ac3b-36da-47b9-a1e9-4f4d0c355e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:09.610895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2729ac3b-36da-47b9-a1e9-4f4d0c355e21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:09.625641 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729ac3b-36da-47b9-a1e9-4f4d0c355e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:09.627087 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729ac3b-36da-47b9-a1e9-4f4d0c355e21 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:09.638479 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4254/8471] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:41:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:41:09.725403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b26e35-2bd9-4f88-a5e4-5d670b512ec0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:09.725606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b26e35-2bd9-4f88-a5e4-5d670b512ec0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:09.725872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b26e35-2bd9-4f88-a5e4-5d670b512ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:09.745307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b8b26e35-2bd9-4f88-a5e4-5d670b512ec0 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:41:09.746495 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b8b26e35-2bd9-4f88-a5e4-5d670b512ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:09.820293 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4218/8472] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 21:41:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:41:09.831857 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729ac3b-36da-47b9-a1e9-4f4d0c355e21 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:09.833579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-2729ac3b-36da-47b9-a1e9-4f4d0c355e21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:09.850694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729ac3b-36da-47b9-a1e9-4f4d0c355e21 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:09.855615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c6286-429e-4ca6-9af1-7588f488ffa0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:09.856201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c6286-429e-4ca6-9af1-7588f488ffa0 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:09.856470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c6286-429e-4ca6-9af1-7588f488ffa0 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:09.856738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c6286-429e-4ca6-9af1-7588f488ffa0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:09.865413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c6286-429e-4ca6-9af1-7588f488ffa0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:09.866968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c6286-429e-4ca6-9af1-7588f488ffa0 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:09.877064 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4255/8473] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:41:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:41:10.057702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b26e35-2bd9-4f88-a5e4-5d670b512ec0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:10.059210 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b8b26e35-2bd9-4f88-a5e4-5d670b512ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:10.070921 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b26e35-2bd9-4f88-a5e4-5d670b512ec0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:10.074418 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee61d50-06f8-4408-85f3-dfb713eec45b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:10.075000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee61d50-06f8-4408-85f3-dfb713eec45b service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:10.075270 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee61d50-06f8-4408-85f3-dfb713eec45b service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:10.075533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee61d50-06f8-4408-85f3-dfb713eec45b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:10.077208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eee61d50-06f8-4408-85f3-dfb713eec45b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:10.093977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee61d50-06f8-4408-85f3-dfb713eec45b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:10.095433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee61d50-06f8-4408-85f3-dfb713eec45b service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:10.108904 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4219/8474] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:41:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:41:19.872426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=12 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:20.101634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=39 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:23.442434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c6286-429e-4ca6-9af1-7588f488ffa0 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:23.456788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c6286-429e-4ca6-9af1-7588f488ffa0 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:23.462413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2c000-0c89-425a-854b-f3a97f127eb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:23.463032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2c000-0c89-425a-854b-f3a97f127eb2 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:23.463301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2c000-0c89-425a-854b-f3a97f127eb2 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:23.463567 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2c000-0c89-425a-854b-f3a97f127eb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:23.465291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8bd2c000-0c89-425a-854b-f3a97f127eb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:23.480074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2c000-0c89-425a-854b-f3a97f127eb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:23.481652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2c000-0c89-425a-854b-f3a97f127eb2 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:23.494309 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4256/8475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:41:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:41:28.762991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee61d50-06f8-4408-85f3-dfb713eec45b service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:28.764382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-eee61d50-06f8-4408-85f3-dfb713eec45b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:28.791236 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee61d50-06f8-4408-85f3-dfb713eec45b service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:28.796600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58e899-f78d-4696-9be1-6cef052b4d27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:28.797229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58e899-f78d-4696-9be1-6cef052b4d27 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:28.797500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58e899-f78d-4696-9be1-6cef052b4d27 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:28.797759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58e899-f78d-4696-9be1-6cef052b4d27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:28.800251 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ea58e899-f78d-4696-9be1-6cef052b4d27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:28.820315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58e899-f78d-4696-9be1-6cef052b4d27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:28.824616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58e899-f78d-4696-9be1-6cef052b4d27 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:28.838964 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4220/8476] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:41:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:41:33.487682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:38.830890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:48.970450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2c000-0c89-425a-854b-f3a97f127eb2 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:48.984132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2c000-0c89-425a-854b-f3a97f127eb2 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:48.989761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:48.990392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:48.990664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:48.990926 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] PATH_INFO: `/v3/users/c1bffd7ffed94cf595a69eaa7fa758ed` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:48.992056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1bffd7ffed94cf595a69eaa7fa758ed)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:48.994171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:49.044668 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1bffd7ffed94cf595a69eaa7fa758ed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:41:49.052123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1bffd7ffed94cf595a69eaa7fa758ed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:41:49.065556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1bffd7ffed94cf595a69eaa7fa758ed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:41:49.072748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1bffd7ffed94cf595a69eaa7fa758ed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:41:49.073410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-2b96c433-1c71-4ede-8602-bead58b2c852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1bffd7ffed94cf595a69eaa7fa758ed'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:41:49.079518 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4257/8477] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:41:48 2022] DELETE /identity/v3/users/c1bffd7ffed94cf595a69eaa7fa758ed => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:41:49.258648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58e899-f78d-4696-9be1-6cef052b4d27 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:49.270758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58e899-f78d-4696-9be1-6cef052b4d27 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:49.329555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:49.330905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:49.331219 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:49.331489 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] PATH_INFO: `/v3/projects/512ac462334c424eb45066ea7647a3a9` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:49.332615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] RBAC: Authorizing `identity:delete_project(project_id=512ac462334c424eb45066ea7647a3a9)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:49.334618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:49.379968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512ac462334c424eb45066ea7647a3a9', 'request_id': 'req-05021eb4-64de-4e6f-92a4-d595f3b7702a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:41:49.387389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512ac462334c424eb45066ea7647a3a9', 'request_id': 'req-05021eb4-64de-4e6f-92a4-d595f3b7702a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:41:49.394501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512ac462334c424eb45066ea7647a3a9', 'request_id': 'req-05021eb4-64de-4e6f-92a4-d595f3b7702a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:41:49.404939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] The token cache is being invalidate because project 512ac462334c424eb45066ea7647a3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:41:49.405250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:41:49.406039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-05021eb4-64de-4e6f-92a4-d595f3b7702a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:41:49.407585 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4221/8478] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:41:49 2022] DELETE /identity/v3/projects/512ac462334c424eb45066ea7647a3a9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:41:49.412837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29db00-bdc3-4876-81d6-74c4d005e473 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:49.413161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29db00-bdc3-4876-81d6-74c4d005e473 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:49.413437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29db00-bdc3-4876-81d6-74c4d005e473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:49.432933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-df29db00-bdc3-4876-81d6-74c4d005e473 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:41:49.434085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-df29db00-bdc3-4876-81d6-74c4d005e473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:49.494213 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4258/8479] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:41:49.503496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368a451-63c2-41ed-bce4-9bf4ae6b72d0 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:49.503775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368a451-63c2-41ed-bce4-9bf4ae6b72d0 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:49.504048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368a451-63c2-41ed-bce4-9bf4ae6b72d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:49.525354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8368a451-63c2-41ed-bce4-9bf4ae6b72d0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:41:49.526566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8368a451-63c2-41ed-bce4-9bf4ae6b72d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:49.534980 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4222/8480] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:41:49.539271 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29db00-bdc3-4876-81d6-74c4d005e473 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:49.549870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29db00-bdc3-4876-81d6-74c4d005e473 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:49.554485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f0476-f85a-4c15-90c6-ebfeac732dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:49.555124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f0476-f85a-4c15-90c6-ebfeac732dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:49.555514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f0476-f85a-4c15-90c6-ebfeac732dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:49.555809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f0476-f85a-4c15-90c6-ebfeac732dc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:49.556445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f0476-f85a-4c15-90c6-ebfeac732dc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:41:49.556745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f0476-f85a-4c15-90c6-ebfeac732dc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:49.558396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f0476-f85a-4c15-90c6-ebfeac732dc7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:49.569888 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4259/8481] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:41:49.574342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368a451-63c2-41ed-bce4-9bf4ae6b72d0 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:49.583731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368a451-63c2-41ed-bce4-9bf4ae6b72d0 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:49.588398 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1167513-0785-41d9-9ae4-4bc3030ec643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:49.589009 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1167513-0785-41d9-9ae4-4bc3030ec643 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:49.589315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1167513-0785-41d9-9ae4-4bc3030ec643 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:49.589583 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1167513-0785-41d9-9ae4-4bc3030ec643 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:49.590208 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1167513-0785-41d9-9ae4-4bc3030ec643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:49.592104 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1167513-0785-41d9-9ae4-4bc3030ec643 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:49.620515 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4223/8482] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:41:49 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:41:49.624448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f0476-f85a-4c15-90c6-ebfeac732dc7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:49.634682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f0476-f85a-4c15-90c6-ebfeac732dc7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:49.691993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26142a32-7dda-45ea-b8be-37fa14945122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:49.692502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26142a32-7dda-45ea-b8be-37fa14945122 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:49.692771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26142a32-7dda-45ea-b8be-37fa14945122 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:49.693052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26142a32-7dda-45ea-b8be-37fa14945122 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:49.693677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26142a32-7dda-45ea-b8be-37fa14945122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:49.695559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26142a32-7dda-45ea-b8be-37fa14945122 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:49.703352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:49.717417 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4260/8483] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:41:49 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:41:49.722094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1167513-0785-41d9-9ae4-4bc3030ec643 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:49.732354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1167513-0785-41d9-9ae4-4bc3030ec643 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:49.737616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671834c-417b-453f-ad51-a87dadc30f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:49.738199 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671834c-417b-453f-ad51-a87dadc30f10 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:49.738472 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671834c-417b-453f-ad51-a87dadc30f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:49.738768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671834c-417b-453f-ad51-a87dadc30f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:49.739399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671834c-417b-453f-ad51-a87dadc30f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:49.741024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671834c-417b-453f-ad51-a87dadc30f10 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:49.750859 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4224/8484] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:41:49.754861 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26142a32-7dda-45ea-b8be-37fa14945122 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:49.765019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26142a32-7dda-45ea-b8be-37fa14945122 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:49.769991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2b16d-094d-4e9f-a465-38722d6128cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:49.770570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2b16d-094d-4e9f-a465-38722d6128cd admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:49.770845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2b16d-094d-4e9f-a465-38722d6128cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:49.771112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2b16d-094d-4e9f-a465-38722d6128cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:49.771735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2b16d-094d-4e9f-a465-38722d6128cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:49.773322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2b16d-094d-4e9f-a465-38722d6128cd admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:49.783892 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4261/8485] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:41:49.788269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671834c-417b-453f-ad51-a87dadc30f10 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:49.798429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671834c-417b-453f-ad51-a87dadc30f10 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:49.803267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd69db-4dc2-42c9-ba3d-a38af1160f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:49.803896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd69db-4dc2-42c9-ba3d-a38af1160f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:49.804170 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd69db-4dc2-42c9-ba3d-a38af1160f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:49.804438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd69db-4dc2-42c9-ba3d-a38af1160f76 admin admin] PATH_INFO: `/v3/projects/9059ee7869df4e5abed98104bae1ef7d/users/6d864b55e1d94113bed6ec710c0c0fc6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:49.820373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd69db-4dc2-42c9-ba3d-a38af1160f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9059ee7869df4e5abed98104bae1ef7d, user_id=6d864b55e1d94113bed6ec710c0c0fc6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:49.823059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd69db-4dc2-42c9-ba3d-a38af1160f76 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:49.836835 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4225/8486] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:41:49 2022] PUT /identity/v3/projects/9059ee7869df4e5abed98104bae1ef7d/users/6d864b55e1d94113bed6ec710c0c0fc6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:41:49.842057 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25f01a-b6c1-4627-8e28-86a00e963f98 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:49.842342 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25f01a-b6c1-4627-8e28-86a00e963f98 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:49.842613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25f01a-b6c1-4627-8e28-86a00e963f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:49.862159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7c25f01a-b6c1-4627-8e28-86a00e963f98 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:41:49.863470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7c25f01a-b6c1-4627-8e28-86a00e963f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:49.921996 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4262/8487] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:41:49.932146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd69db-4dc2-42c9-ba3d-a38af1160f76 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:49.933876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-07cd69db-4dc2-42c9-ba3d-a38af1160f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:49.966659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:49.996726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd69db-4dc2-42c9-ba3d-a38af1160f76 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:50.000443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9dd30-aeb7-4bc3-b3ab-bd079fd6352d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:50.001054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9dd30-aeb7-4bc3-b3ab-bd079fd6352d service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:50.001347 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9dd30-aeb7-4bc3-b3ab-bd079fd6352d service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:50.001607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9dd30-aeb7-4bc3-b3ab-bd079fd6352d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:50.011464 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9dd30-aeb7-4bc3-b3ab-bd079fd6352d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:50.013172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9dd30-aeb7-4bc3-b3ab-bd079fd6352d service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:50.022902 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4226/8488] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:41:54.061655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cb267-7f5f-4a17-a7c7-a5e32ed30d48 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:54.062186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cb267-7f5f-4a17-a7c7-a5e32ed30d48 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:54.062186 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cb267-7f5f-4a17-a7c7-a5e32ed30d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:54.092030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-687cb267-7f5f-4a17-a7c7-a5e32ed30d48 None None] MFA Rules not processed for user `6d864b55e1d94113bed6ec710c0c0fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:41:54.093222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-687cb267-7f5f-4a17-a7c7-a5e32ed30d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:54.163412 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4263/8489] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:41:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:41:54.174342 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b7555-c0a8-46c6-9af6-a6d7804ab5eb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:54.174923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b7555-c0a8-46c6-9af6-a6d7804ab5eb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:54.174923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b7555-c0a8-46c6-9af6-a6d7804ab5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:54.194872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-938b7555-c0a8-46c6-9af6-a6d7804ab5eb None None] MFA Rules not processed for user `6d864b55e1d94113bed6ec710c0c0fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:41:54.196311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-938b7555-c0a8-46c6-9af6-a6d7804ab5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:54.204030 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4227/8490] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:41:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:41:54.217147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cb267-7f5f-4a17-a7c7-a5e32ed30d48 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:41:54.218786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-687cb267-7f5f-4a17-a7c7-a5e32ed30d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:41:54.304945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cb267-7f5f-4a17-a7c7-a5e32ed30d48 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:41:54.310642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d65f7f-d944-4691-89c2-ab057742fa4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:41:54.311257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d65f7f-d944-4691-89c2-ab057742fa4e service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:41:54.311539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d65f7f-d944-4691-89c2-ab057742fa4e service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:41:54.311825 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d65f7f-d944-4691-89c2-ab057742fa4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:41:54.316816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:54.327109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d65f7f-d944-4691-89c2-ab057742fa4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:41:54.328396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d65f7f-d944-4691-89c2-ab057742fa4e service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:41:54.356709 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4264/8491] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:41:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:42:04.188445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=17 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:04.348919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:35.497814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b7555-c0a8-46c6-9af6-a6d7804ab5eb None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:42:35.510153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b7555-c0a8-46c6-9af6-a6d7804ab5eb None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:42:35.513978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcab29-0233-4a6d-a7ac-38585037988f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:42:35.514553 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcab29-0233-4a6d-a7ac-38585037988f service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:42:35.514818 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcab29-0233-4a6d-a7ac-38585037988f service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:42:35.515077 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcab29-0233-4a6d-a7ac-38585037988f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:42:35.516740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3fbcab29-0233-4a6d-a7ac-38585037988f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:42:35.561960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcab29-0233-4a6d-a7ac-38585037988f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:42:35.563384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcab29-0233-4a6d-a7ac-38585037988f service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:42:35.575613 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4228/8492] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:42:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:42:42.652046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f4614-ac8b-4df9-8789-3d24eb3e326d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:42:42.652575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f4614-ac8b-4df9-8789-3d24eb3e326d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:42:42.652575 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f4614-ac8b-4df9-8789-3d24eb3e326d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:42:42.672250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7d7f4614-ac8b-4df9-8789-3d24eb3e326d None None] MFA Rules not processed for user `6d864b55e1d94113bed6ec710c0c0fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:42:42.673469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7d7f4614-ac8b-4df9-8789-3d24eb3e326d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:42:42.681907 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4265/8493] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:42:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:42:43.261746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcab29-0233-4a6d-a7ac-38585037988f service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:42:43.263256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3fbcab29-0233-4a6d-a7ac-38585037988f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:42:43.339330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcab29-0233-4a6d-a7ac-38585037988f service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:42:43.344280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b400a-1d90-495b-a2b4-1c18a6d637da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:42:43.344869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b400a-1d90-495b-a2b4-1c18a6d637da service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:42:43.345204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b400a-1d90-495b-a2b4-1c18a6d637da service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:42:43.346094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b400a-1d90-495b-a2b4-1c18a6d637da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:42:43.347871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-530b400a-1d90-495b-a2b4-1c18a6d637da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:42:43.358996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b400a-1d90-495b-a2b4-1c18a6d637da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:42:43.360557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b400a-1d90-495b-a2b4-1c18a6d637da service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:42:43.370600 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4229/8494] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:42:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:42:52.667467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:53.366441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=53 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:37.532190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f4614-ac8b-4df9-8789-3d24eb3e326d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:37.534334 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7d7f4614-ac8b-4df9-8789-3d24eb3e326d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:37.573691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f4614-ac8b-4df9-8789-3d24eb3e326d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:37.579884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3929a-6549-4f15-aa85-7c0c7591117b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:37.580503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3929a-6549-4f15-aa85-7c0c7591117b service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:37.580780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3929a-6549-4f15-aa85-7c0c7591117b service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:37.586309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3929a-6549-4f15-aa85-7c0c7591117b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:37.588474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-add3929a-6549-4f15-aa85-7c0c7591117b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:37.594826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b400a-1d90-495b-a2b4-1c18a6d637da service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:37.605593 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3929a-6549-4f15-aa85-7c0c7591117b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:37.607365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3929a-6549-4f15-aa85-7c0c7591117b service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:37.610242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b400a-1d90-495b-a2b4-1c18a6d637da service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:37.617410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:37.618314 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:37.618594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:37.618998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] PATH_INFO: `/v3/users/6d864b55e1d94113bed6ec710c0c0fc6` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:37.620134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d864b55e1d94113bed6ec710c0c0fc6)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:37.622256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:37.625942 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4266/8495] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:43:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:43:37.674701 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d864b55e1d94113bed6ec710c0c0fc6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:43:37.687275 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d864b55e1d94113bed6ec710c0c0fc6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:43:37.694065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d864b55e1d94113bed6ec710c0c0fc6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:43:37.701194 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d864b55e1d94113bed6ec710c0c0fc6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:43:37.701816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-5ef44831-25bd-4b51-8b5f-b4e45a91492e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d864b55e1d94113bed6ec710c0c0fc6'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:43:37.707592 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4230/8496] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:43:37 2022] DELETE /identity/v3/users/6d864b55e1d94113bed6ec710c0c0fc6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:43:37.884218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3929a-6549-4f15-aa85-7c0c7591117b service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:37.896992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3929a-6549-4f15-aa85-7c0c7591117b service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:37.958839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:37.959517 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:37.959793 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:37.960058 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] PATH_INFO: `/v3/projects/9059ee7869df4e5abed98104bae1ef7d` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:37.961215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] RBAC: Authorizing `identity:delete_project(project_id=9059ee7869df4e5abed98104bae1ef7d)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:37.963119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:38.010788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9059ee7869df4e5abed98104bae1ef7d', 'request_id': 'req-f472f6e9-34c0-4c1a-b121-e32b84da25be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:43:38.018106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9059ee7869df4e5abed98104bae1ef7d', 'request_id': 'req-f472f6e9-34c0-4c1a-b121-e32b84da25be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:43:38.025241 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9059ee7869df4e5abed98104bae1ef7d', 'request_id': 'req-f472f6e9-34c0-4c1a-b121-e32b84da25be'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:43:38.035544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] The token cache is being invalidate because project 9059ee7869df4e5abed98104bae1ef7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:43:38.035832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:43:38.036686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-f472f6e9-34c0-4c1a-b121-e32b84da25be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:43:38.038294 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4267/8497] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:43:37 2022] DELETE /identity/v3/projects/9059ee7869df4e5abed98104bae1ef7d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:43:38.043261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b0b47-00a1-4f6f-81fb-dd66f0428535 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:38.043541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b0b47-00a1-4f6f-81fb-dd66f0428535 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:38.043808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b0b47-00a1-4f6f-81fb-dd66f0428535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:38.063658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-8a4b0b47-00a1-4f6f-81fb-dd66f0428535 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:43:38.064834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8a4b0b47-00a1-4f6f-81fb-dd66f0428535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:38.123152 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4231/8498] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:43:38.132602 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4fae3-8cbc-4035-ae48-eb2645420cc6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:38.132882 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4fae3-8cbc-4035-ae48-eb2645420cc6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:38.133215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4fae3-8cbc-4035-ae48-eb2645420cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:38.152991 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-69f4fae3-8cbc-4035-ae48-eb2645420cc6 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:43:38.154200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-69f4fae3-8cbc-4035-ae48-eb2645420cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:38.162836 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4268/8499] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:43:38.167025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b0b47-00a1-4f6f-81fb-dd66f0428535 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:38.178090 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b0b47-00a1-4f6f-81fb-dd66f0428535 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:38.182702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be11301-dec9-4a1a-aef3-ccdb5f1053b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:38.183286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be11301-dec9-4a1a-aef3-ccdb5f1053b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:38.183561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be11301-dec9-4a1a-aef3-ccdb5f1053b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:38.183829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be11301-dec9-4a1a-aef3-ccdb5f1053b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:38.184465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be11301-dec9-4a1a-aef3-ccdb5f1053b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:43:38.184760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be11301-dec9-4a1a-aef3-ccdb5f1053b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:38.186431 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be11301-dec9-4a1a-aef3-ccdb5f1053b2 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:38.199329 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4232/8500] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:43:38.203529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4fae3-8cbc-4035-ae48-eb2645420cc6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:38.213251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4fae3-8cbc-4035-ae48-eb2645420cc6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:38.218003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf46e68-a749-4aa3-890d-3fa0751c7cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:38.218589 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf46e68-a749-4aa3-890d-3fa0751c7cee admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:38.218867 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf46e68-a749-4aa3-890d-3fa0751c7cee admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:38.219138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf46e68-a749-4aa3-890d-3fa0751c7cee admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:38.219767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf46e68-a749-4aa3-890d-3fa0751c7cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:38.221702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf46e68-a749-4aa3-890d-3fa0751c7cee admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:38.249452 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4269/8501] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:43:38 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:43:38.253602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be11301-dec9-4a1a-aef3-ccdb5f1053b2 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:38.263684 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be11301-dec9-4a1a-aef3-ccdb5f1053b2 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:38.319993 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee110dc-4855-4488-9d7a-4f7a89a66f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:38.320585 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee110dc-4855-4488-9d7a-4f7a89a66f92 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:38.320857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee110dc-4855-4488-9d7a-4f7a89a66f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:38.321155 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee110dc-4855-4488-9d7a-4f7a89a66f92 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:38.321762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee110dc-4855-4488-9d7a-4f7a89a66f92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:38.323690 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee110dc-4855-4488-9d7a-4f7a89a66f92 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:38.331617 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:38.347300 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4233/8502] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:43:38 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:43:38.352034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf46e68-a749-4aa3-890d-3fa0751c7cee admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:38.362478 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf46e68-a749-4aa3-890d-3fa0751c7cee admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:38.367216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47cb90-568b-45b0-a5ee-8c915d4dda36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:38.367804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47cb90-568b-45b0-a5ee-8c915d4dda36 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:38.368075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47cb90-568b-45b0-a5ee-8c915d4dda36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:38.368338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47cb90-568b-45b0-a5ee-8c915d4dda36 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:38.368959 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47cb90-568b-45b0-a5ee-8c915d4dda36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:38.370547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47cb90-568b-45b0-a5ee-8c915d4dda36 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:38.380849 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4270/8503] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:43:38.384725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee110dc-4855-4488-9d7a-4f7a89a66f92 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:38.394874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee110dc-4855-4488-9d7a-4f7a89a66f92 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:38.400032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b9bf7-f254-4cd6-af39-805dfa7564c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:38.400614 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b9bf7-f254-4cd6-af39-805dfa7564c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:38.400888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b9bf7-f254-4cd6-af39-805dfa7564c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:38.401202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b9bf7-f254-4cd6-af39-805dfa7564c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:38.401828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b9bf7-f254-4cd6-af39-805dfa7564c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:38.403405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b9bf7-f254-4cd6-af39-805dfa7564c7 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:38.414443 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4234/8504] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:43:38.418559 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47cb90-568b-45b0-a5ee-8c915d4dda36 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:38.428945 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47cb90-568b-45b0-a5ee-8c915d4dda36 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:38.433522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d55524-cf87-4fc1-8a69-203c80270fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:38.434154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d55524-cf87-4fc1-8a69-203c80270fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:38.434428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d55524-cf87-4fc1-8a69-203c80270fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:38.434695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d55524-cf87-4fc1-8a69-203c80270fd3 admin admin] PATH_INFO: `/v3/projects/d42197537d0b407882fc410a5d359e1b/users/4165cc6b781b4df3abd6a227170d4e46/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:38.450035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d55524-cf87-4fc1-8a69-203c80270fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42197537d0b407882fc410a5d359e1b, user_id=4165cc6b781b4df3abd6a227170d4e46, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:38.452632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d55524-cf87-4fc1-8a69-203c80270fd3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:38.460966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:38.467974 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4271/8505] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:43:38 2022] PUT /identity/v3/projects/d42197537d0b407882fc410a5d359e1b/users/4165cc6b781b4df3abd6a227170d4e46/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:43:38.476272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325295f-3ede-45e8-930d-886d9297a9d3 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:38.476558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325295f-3ede-45e8-930d-886d9297a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:38.476833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325295f-3ede-45e8-930d-886d9297a9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:38.496588 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b325295f-3ede-45e8-930d-886d9297a9d3 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:43:38.497827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b325295f-3ede-45e8-930d-886d9297a9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:38.556590 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4235/8506] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:43:38.566313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d55524-cf87-4fc1-8a69-203c80270fd3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:38.568133 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-51d55524-cf87-4fc1-8a69-203c80270fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:38.624291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d55524-cf87-4fc1-8a69-203c80270fd3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:38.628185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14dd9b-e0b4-45cc-9d08-1e2af0109f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:38.628843 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14dd9b-e0b4-45cc-9d08-1e2af0109f82 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:38.629147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14dd9b-e0b4-45cc-9d08-1e2af0109f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:38.629415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14dd9b-e0b4-45cc-9d08-1e2af0109f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:38.638566 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14dd9b-e0b4-45cc-9d08-1e2af0109f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:38.640233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14dd9b-e0b4-45cc-9d08-1e2af0109f82 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:38.650176 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4272/8507] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:43:42.749204 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72b471-782b-4225-b28b-9b27cd04a22a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:42.749729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72b471-782b-4225-b28b-9b27cd04a22a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:42.749729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72b471-782b-4225-b28b-9b27cd04a22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:42.781892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bb72b471-782b-4225-b28b-9b27cd04a22a None None] MFA Rules not processed for user `4165cc6b781b4df3abd6a227170d4e46`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:43:42.783134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bb72b471-782b-4225-b28b-9b27cd04a22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:42.863140 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4236/8508] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:43:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:43:42.877609 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14dd9b-e0b4-45cc-9d08-1e2af0109f82 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:42.879091 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-df14dd9b-e0b4-45cc-9d08-1e2af0109f82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:42.961558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14dd9b-e0b4-45cc-9d08-1e2af0109f82 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:42.967248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d096dd-795e-42d1-ad23-bd7469fae5df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:42.967764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d096dd-795e-42d1-ad23-bd7469fae5df service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:42.968040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d096dd-795e-42d1-ad23-bd7469fae5df service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:42.968306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d096dd-795e-42d1-ad23-bd7469fae5df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:42.976972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d096dd-795e-42d1-ad23-bd7469fae5df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:42.979086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d096dd-795e-42d1-ad23-bd7469fae5df service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:42.989991 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4273/8509] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:43:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:43:44.341555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e9b9a-aad5-4572-9c0f-a43e6baf5cb8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:44.342019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e9b9a-aad5-4572-9c0f-a43e6baf5cb8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:44.342112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e9b9a-aad5-4572-9c0f-a43e6baf5cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:44.362477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f21e9b9a-aad5-4572-9c0f-a43e6baf5cb8 None None] MFA Rules not processed for user `4165cc6b781b4df3abd6a227170d4e46`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:43:44.363379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f21e9b9a-aad5-4572-9c0f-a43e6baf5cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:44.370871 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4237/8510] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:43:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:43:44.693328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d096dd-795e-42d1-ad23-bd7469fae5df service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:44.694850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-42d096dd-795e-42d1-ad23-bd7469fae5df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:44.706637 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d096dd-795e-42d1-ad23-bd7469fae5df service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:44.711194 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5688c30-5816-465d-a66f-8232a6c3c1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:44.712028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5688c30-5816-465d-a66f-8232a6c3c1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:44.712304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5688c30-5816-465d-a66f-8232a6c3c1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:44.712570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5688c30-5816-465d-a66f-8232a6c3c1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:44.714339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c5688c30-5816-465d-a66f-8232a6c3c1e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:44.727557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5688c30-5816-465d-a66f-8232a6c3c1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:44.729056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5688c30-5816-465d-a66f-8232a6c3c1e8 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:44.740717 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4274/8511] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:43:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:43:45.630654 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8ce11-b6c7-4d45-8939-5eac4d7464a8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:45.631630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8ce11-b6c7-4d45-8939-5eac4d7464a8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:45.631630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8ce11-b6c7-4d45-8939-5eac4d7464a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:45.663754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-63f8ce11-b6c7-4d45-8939-5eac4d7464a8 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:43:45.664880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-63f8ce11-b6c7-4d45-8939-5eac4d7464a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:45.716382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:45.734329 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4238/8512] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Oct 15 21:43:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:43:45.747879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea38641-3b7c-45a5-9abe-0b3939facd01 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:45.748408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea38641-3b7c-45a5-9abe-0b3939facd01 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:45.748408 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea38641-3b7c-45a5-9abe-0b3939facd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:45.769697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-eea38641-3b7c-45a5-9abe-0b3939facd01 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:43:45.771519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eea38641-3b7c-45a5-9abe-0b3939facd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:43:45.780194 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4275/8513] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Oct 15 21:43:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:43:45.786521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8ce11-b6c7-4d45-8939-5eac4d7464a8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:43:45.798366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8ce11-b6c7-4d45-8939-5eac4d7464a8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:43:45.803652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e92251-d096-4800-83f4-f84b03db9477 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:43:45.804258 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e92251-d096-4800-83f4-f84b03db9477 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:43:45.804531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e92251-d096-4800-83f4-f84b03db9477 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:43:45.804799 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e92251-d096-4800-83f4-f84b03db9477 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:43:45.813807 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e92251-d096-4800-83f4-f84b03db9477 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:43:45.815409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e92251-d096-4800-83f4-f84b03db9477 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:43:45.826371 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4239/8514] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:43:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:43:55.763640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=70 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:55.821945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=11 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:44:24.445097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da931d43-164d-4cd8-ac93-962d77465d7c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:44:24.445702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da931d43-164d-4cd8-ac93-962d77465d7c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:44:24.445702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da931d43-164d-4cd8-ac93-962d77465d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:44:24.469859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-da931d43-164d-4cd8-ac93-962d77465d7c None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:44:24.470995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-da931d43-164d-4cd8-ac93-962d77465d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:44:24.480053 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4276/8515] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Oct 15 21:44:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:44:24.492341 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3f43f-3cb1-4109-9747-e3b6c3f38f13 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:44:24.492796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3f43f-3cb1-4109-9747-e3b6c3f38f13 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:44:24.492904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3f43f-3cb1-4109-9747-e3b6c3f38f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:44:24.512824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-36a3f43f-3cb1-4109-9747-e3b6c3f38f13 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:44:24.514283 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-36a3f43f-3cb1-4109-9747-e3b6c3f38f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:44:24.522877 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4240/8516] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Oct 15 21:44:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:44:24.528990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da931d43-164d-4cd8-ac93-962d77465d7c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:44:24.539644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da931d43-164d-4cd8-ac93-962d77465d7c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:44:24.544626 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcab442-deb3-4ede-9537-2355b9407266 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:44:24.545249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcab442-deb3-4ede-9537-2355b9407266 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:44:24.545522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcab442-deb3-4ede-9537-2355b9407266 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:44:24.545814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcab442-deb3-4ede-9537-2355b9407266 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:44:24.554483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcab442-deb3-4ede-9537-2355b9407266 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:44:24.556024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcab442-deb3-4ede-9537-2355b9407266 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:44:24.565568 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4277/8517] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:44:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:44:34.507996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:44:34.561426 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=9 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:02.339343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3f43f-3cb1-4109-9747-e3b6c3f38f13 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:45:02.349797 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3f43f-3cb1-4109-9747-e3b6c3f38f13 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:45:02.354772 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2512d-60ff-43a1-bfb2-ff96bca44e47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:45:02.355356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2512d-60ff-43a1-bfb2-ff96bca44e47 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:45:02.355631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2512d-60ff-43a1-bfb2-ff96bca44e47 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:45:02.355898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2512d-60ff-43a1-bfb2-ff96bca44e47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:45:02.357632 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-79a2512d-60ff-43a1-bfb2-ff96bca44e47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:45:02.369308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2512d-60ff-43a1-bfb2-ff96bca44e47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:45:02.370864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2512d-60ff-43a1-bfb2-ff96bca44e47 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:45:02.381424 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4241/8518] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:45:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:45:12.376653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:16.038998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcab442-deb3-4ede-9537-2355b9407266 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:45:16.048361 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcab442-deb3-4ede-9537-2355b9407266 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:45:16.052571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948ffd8-5449-4afb-9689-d38e9f5ad81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:45:16.053200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948ffd8-5449-4afb-9689-d38e9f5ad81f service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:45:16.053491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948ffd8-5449-4afb-9689-d38e9f5ad81f service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:45:16.053741 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948ffd8-5449-4afb-9689-d38e9f5ad81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:45:16.055514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8948ffd8-5449-4afb-9689-d38e9f5ad81f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:45:16.082632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948ffd8-5449-4afb-9689-d38e9f5ad81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:45:16.084122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948ffd8-5449-4afb-9689-d38e9f5ad81f service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:45:16.094861 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4278/8519] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:45:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:45:25.294883 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fb024-d4db-486c-8938-1d2c7ce38d49 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:45:25.295640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fb024-d4db-486c-8938-1d2c7ce38d49 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:45:25.295640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fb024-d4db-486c-8938-1d2c7ce38d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:45:25.317633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6a0fb024-d4db-486c-8938-1d2c7ce38d49 None None] MFA Rules not processed for user `4165cc6b781b4df3abd6a227170d4e46`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:45:25.318784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6a0fb024-d4db-486c-8938-1d2c7ce38d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:45:25.325920 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4242/8520] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:45:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:45:26.088897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=11 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:35.311665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:12.106138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71a4f-875b-402a-bd60-41fb33dd874b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:12.106706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71a4f-875b-402a-bd60-41fb33dd874b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:12.106706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71a4f-875b-402a-bd60-41fb33dd874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:12.147989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-fbe71a4f-875b-402a-bd60-41fb33dd874b None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:46:12.149242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-fbe71a4f-875b-402a-bd60-41fb33dd874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:12.158547 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4279/8521] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Oct 15 21:46:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:46:13.347169 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73549d6b-3dd3-41c7-88f5-dc07e36e6216 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:13.347746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73549d6b-3dd3-41c7-88f5-dc07e36e6216 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:13.347746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73549d6b-3dd3-41c7-88f5-dc07e36e6216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:13.368094 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-73549d6b-3dd3-41c7-88f5-dc07e36e6216 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:46:13.369315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-73549d6b-3dd3-41c7-88f5-dc07e36e6216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:13.378403 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4243/8522] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Oct 15 21:46:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:46:13.384304 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828095-a611-40fa-8c22-bac88a345834 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:13.384788 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828095-a611-40fa-8c22-bac88a345834 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:13.384897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828095-a611-40fa-8c22-bac88a345834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:13.404460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c3828095-a611-40fa-8c22-bac88a345834 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:46:13.405670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c3828095-a611-40fa-8c22-bac88a345834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:13.415276 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4280/8523] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:46:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:46:13.425834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73549d6b-3dd3-41c7-88f5-dc07e36e6216 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:46:13.435812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73549d6b-3dd3-41c7-88f5-dc07e36e6216 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:46:13.439404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79a294-4fdd-4ede-9159-c14fa760a9ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:46:13.439991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79a294-4fdd-4ede-9159-c14fa760a9ac service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:13.440261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79a294-4fdd-4ede-9159-c14fa760a9ac service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:13.440524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79a294-4fdd-4ede-9159-c14fa760a9ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:13.450565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79a294-4fdd-4ede-9159-c14fa760a9ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:46:13.451989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79a294-4fdd-4ede-9159-c14fa760a9ac service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:46:13.462614 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4244/8524] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:46:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:46:23.399691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=12 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:23.458096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:27.866014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3828095-a611-40fa-8c22-bac88a345834 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:46:27.875436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3828095-a611-40fa-8c22-bac88a345834 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:46:27.879183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e9a65-9d50-4476-b59f-588cc0496277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:46:27.879783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e9a65-9d50-4476-b59f-588cc0496277 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:27.880056 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e9a65-9d50-4476-b59f-588cc0496277 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:27.880321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e9a65-9d50-4476-b59f-588cc0496277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:27.882083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-973e9a65-9d50-4476-b59f-588cc0496277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:27.896048 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e9a65-9d50-4476-b59f-588cc0496277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:46:27.897595 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e9a65-9d50-4476-b59f-588cc0496277 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:46:27.908342 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4281/8525] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:46:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:46:27.989819 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79a294-4fdd-4ede-9159-c14fa760a9ac service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:46:28.001550 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79a294-4fdd-4ede-9159-c14fa760a9ac service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:46:28.007119 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69643c85-d968-4c2d-92f4-e543a5084afc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:46:28.007728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69643c85-d968-4c2d-92f4-e543a5084afc service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:28.008005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69643c85-d968-4c2d-92f4-e543a5084afc service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:28.008256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69643c85-d968-4c2d-92f4-e543a5084afc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:28.009912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-69643c85-d968-4c2d-92f4-e543a5084afc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:28.033428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69643c85-d968-4c2d-92f4-e543a5084afc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:46:28.035181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69643c85-d968-4c2d-92f4-e543a5084afc service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:46:28.045164 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4245/8526] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:46:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:46:28.222065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e9a65-9d50-4476-b59f-588cc0496277 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:46:28.230842 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e9a65-9d50-4476-b59f-588cc0496277 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:46:28.234618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2744ec-47e7-48bd-b7a4-faf91dbbc13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:46:28.235208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2744ec-47e7-48bd-b7a4-faf91dbbc13a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:28.235665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2744ec-47e7-48bd-b7a4-faf91dbbc13a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:28.235820 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2744ec-47e7-48bd-b7a4-faf91dbbc13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:28.237596 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3f2744ec-47e7-48bd-b7a4-faf91dbbc13a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:28.254618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2744ec-47e7-48bd-b7a4-faf91dbbc13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:46:28.256124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2744ec-47e7-48bd-b7a4-faf91dbbc13a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:46:28.270118 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4282/8527] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:46:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:46:29.139466 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bc05a-7005-4c7b-afbe-0f86aaa15c07 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:29.140346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bc05a-7005-4c7b-afbe-0f86aaa15c07 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:29.140346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bc05a-7005-4c7b-afbe-0f86aaa15c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:29.160745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7b8bc05a-7005-4c7b-afbe-0f86aaa15c07 None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:46:29.161946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7b8bc05a-7005-4c7b-afbe-0f86aaa15c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:29.169103 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4246/8528] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Oct 15 21:46:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:46:29.181855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2744ec-47e7-48bd-b7a4-faf91dbbc13a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:46:29.190430 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2744ec-47e7-48bd-b7a4-faf91dbbc13a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:46:29.194769 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308d97e-3211-415c-8592-3e18e71a0e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:46:29.196003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308d97e-3211-415c-8592-3e18e71a0e8a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:29.196278 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308d97e-3211-415c-8592-3e18e71a0e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:29.196542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308d97e-3211-415c-8592-3e18e71a0e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:29.203910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308d97e-3211-415c-8592-3e18e71a0e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:46:29.205351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308d97e-3211-415c-8592-3e18e71a0e8a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:46:29.213828 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4283/8529] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:46:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:46:31.687746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3d646-59e1-4c85-92e0-649002f3db6b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:31.688351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3d646-59e1-4c85-92e0-649002f3db6b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:31.688351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3d646-59e1-4c85-92e0-649002f3db6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:31.708276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b4d3d646-59e1-4c85-92e0-649002f3db6b None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:46:31.709500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b4d3d646-59e1-4c85-92e0-649002f3db6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:31.715873 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4247/8530] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Oct 15 21:46:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:46:31.728231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308d97e-3211-415c-8592-3e18e71a0e8a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:46:31.736898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308d97e-3211-415c-8592-3e18e71a0e8a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:46:31.741170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0f8ad-61f8-47ba-a2b6-4e2e2645c1f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:46:31.741759 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0f8ad-61f8-47ba-a2b6-4e2e2645c1f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:31.742027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0f8ad-61f8-47ba-a2b6-4e2e2645c1f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:31.742317 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0f8ad-61f8-47ba-a2b6-4e2e2645c1f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:31.749406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0f8ad-61f8-47ba-a2b6-4e2e2645c1f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:46:31.750809 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0f8ad-61f8-47ba-a2b6-4e2e2645c1f8 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:46:31.759047 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4284/8531] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:46:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:46:32.140109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab7f0a-8252-4648-8f5e-ebf14a8e44fb None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:32.140323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab7f0a-8252-4648-8f5e-ebf14a8e44fb None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:32.140593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab7f0a-8252-4648-8f5e-ebf14a8e44fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:32.162454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f1ab7f0a-8252-4648-8f5e-ebf14a8e44fb None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:46:32.163700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f1ab7f0a-8252-4648-8f5e-ebf14a8e44fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:32.173391 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4248/8532] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 21:46:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:46:32.227491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0f8ad-61f8-47ba-a2b6-4e2e2645c1f8 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:46:32.240326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0f8ad-61f8-47ba-a2b6-4e2e2645c1f8 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:46:32.246929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9ef1f-c9ba-42a2-a4a4-fd445d613cda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:46:32.247832 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9ef1f-c9ba-42a2-a4a4-fd445d613cda service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:32.248127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9ef1f-c9ba-42a2-a4a4-fd445d613cda service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:32.248485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9ef1f-c9ba-42a2-a4a4-fd445d613cda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:32.259485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9ef1f-c9ba-42a2-a4a4-fd445d613cda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:46:32.261083 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9ef1f-c9ba-42a2-a4a4-fd445d613cda service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:46:32.272823 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4285/8533] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:46:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:46:40.546223 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1b75-bd74-464e-8560-f8cddc61c484 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:40.546742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1b75-bd74-464e-8560-f8cddc61c484 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:40.546742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1b75-bd74-464e-8560-f8cddc61c484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:40.568985 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c5ec1b75-bd74-464e-8560-f8cddc61c484 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:46:40.570645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c5ec1b75-bd74-464e-8560-f8cddc61c484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:40.580372 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4249/8534] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 21:46:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:46:40.597097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9ef1f-c9ba-42a2-a4a4-fd445d613cda service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:46:40.608971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9ef1f-c9ba-42a2-a4a4-fd445d613cda service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:46:40.614974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78025b0-191e-47fd-9cb7-654a51581c01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:46:40.615569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78025b0-191e-47fd-9cb7-654a51581c01 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:40.615920 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78025b0-191e-47fd-9cb7-654a51581c01 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:40.616096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78025b0-191e-47fd-9cb7-654a51581c01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:40.625312 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78025b0-191e-47fd-9cb7-654a51581c01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:46:40.626901 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78025b0-191e-47fd-9cb7-654a51581c01 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:46:40.638071 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4286/8535] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:46:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:46:43.580477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce504a09-58e9-4037-b6f0-e1b8b6f4092b None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:43.580947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce504a09-58e9-4037-b6f0-e1b8b6f4092b None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:43.581154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce504a09-58e9-4037-b6f0-e1b8b6f4092b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:43.601376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ce504a09-58e9-4037-b6f0-e1b8b6f4092b None None] MFA Rules not processed for user `4165cc6b781b4df3abd6a227170d4e46`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:46:43.602573 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ce504a09-58e9-4037-b6f0-e1b8b6f4092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:43.609496 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4250/8536] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:46:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:46:43.908738 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06926b-81b1-4a60-ad56-c3b6e419c29b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:43.909263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06926b-81b1-4a60-ad56-c3b6e419c29b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:43.909369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06926b-81b1-4a60-ad56-c3b6e419c29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:43.929893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-da06926b-81b1-4a60-ad56-c3b6e419c29b None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:46:43.931074 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-da06926b-81b1-4a60-ad56-c3b6e419c29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:46:43.937450 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4287/8537] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:46:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:46:43.949640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce504a09-58e9-4037-b6f0-e1b8b6f4092b None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:46:43.959267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce504a09-58e9-4037-b6f0-e1b8b6f4092b None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:46:43.962852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6664f-99b8-4222-80b2-929a03234142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:46:43.963439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6664f-99b8-4222-80b2-929a03234142 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:46:43.963732 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6664f-99b8-4222-80b2-929a03234142 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:46:43.963995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6664f-99b8-4222-80b2-929a03234142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:46:43.971030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6664f-99b8-4222-80b2-929a03234142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:46:43.972455 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6664f-99b8-4222-80b2-929a03234142 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:46:43.980482 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4251/8538] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:46:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:46:53.924218 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=39 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:53.977191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=32 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:08.228632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06926b-81b1-4a60-ad56-c3b6e419c29b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:48:08.237981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06926b-81b1-4a60-ad56-c3b6e419c29b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:48:08.241729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da067223-1734-4405-bc42-31e05184e081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:48:08.242586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da067223-1734-4405-bc42-31e05184e081 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:48:08.242933 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da067223-1734-4405-bc42-31e05184e081 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:48:08.243292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da067223-1734-4405-bc42-31e05184e081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:48:08.245122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-da067223-1734-4405-bc42-31e05184e081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:48:08.267892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da067223-1734-4405-bc42-31e05184e081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:48:08.269460 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da067223-1734-4405-bc42-31e05184e081 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:48:08.284377 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4288/8539] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:48:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:48:08.464784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e9ab5-8fe5-4668-b721-821414e4ebb5 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:48:08.465242 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e9ab5-8fe5-4668-b721-821414e4ebb5 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:48:08.465365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e9ab5-8fe5-4668-b721-821414e4ebb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:48:08.496079 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c59e9ab5-8fe5-4668-b721-821414e4ebb5 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:48:08.497467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c59e9ab5-8fe5-4668-b721-821414e4ebb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:48:08.506581 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4252/8540] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 21:48:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:48:08.516716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da067223-1734-4405-bc42-31e05184e081 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:48:08.525458 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da067223-1734-4405-bc42-31e05184e081 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:48:08.529120 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c7c34-b945-41df-bf8a-48e767109063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:48:08.529706 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c7c34-b945-41df-bf8a-48e767109063 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:48:08.529978 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c7c34-b945-41df-bf8a-48e767109063 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:48:08.530242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c7c34-b945-41df-bf8a-48e767109063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:48:08.540632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c7c34-b945-41df-bf8a-48e767109063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:48:08.542081 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c7c34-b945-41df-bf8a-48e767109063 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:48:08.552894 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4289/8541] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:48:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:48:18.490485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=7 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:18.548192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=15 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:38.532102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041af091-c4d8-4a65-a141-aada4ab7f45c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:48:38.532595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041af091-c4d8-4a65-a141-aada4ab7f45c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:48:38.532699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041af091-c4d8-4a65-a141-aada4ab7f45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:48:38.553137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-041af091-c4d8-4a65-a141-aada4ab7f45c None None] MFA Rules not processed for user `4165cc6b781b4df3abd6a227170d4e46`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:48:38.554728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-041af091-c4d8-4a65-a141-aada4ab7f45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:48:38.562032 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4253/8542] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:48:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:48:39.216104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c7c34-b945-41df-bf8a-48e767109063 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:48:39.227487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c7c34-b945-41df-bf8a-48e767109063 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:48:39.232686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8ca4f-dcf7-4cef-933a-36845b232800 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:48:39.233522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8ca4f-dcf7-4cef-933a-36845b232800 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:48:39.233795 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8ca4f-dcf7-4cef-933a-36845b232800 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:48:39.234196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8ca4f-dcf7-4cef-933a-36845b232800 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:48:39.236122 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-45d8ca4f-dcf7-4cef-933a-36845b232800 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:48:39.248187 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8ca4f-dcf7-4cef-933a-36845b232800 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:48:39.249805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8ca4f-dcf7-4cef-933a-36845b232800 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:48:39.260659 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4290/8543] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:48:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:48:42.664217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041af091-c4d8-4a65-a141-aada4ab7f45c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:48:42.676964 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041af091-c4d8-4a65-a141-aada4ab7f45c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:48:42.683001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3475a5-ebe4-45ec-ba05-d3e301fb6468 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:48:42.683595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3475a5-ebe4-45ec-ba05-d3e301fb6468 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:48:42.683870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3475a5-ebe4-45ec-ba05-d3e301fb6468 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:48:42.684136 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3475a5-ebe4-45ec-ba05-d3e301fb6468 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:48:42.691851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3475a5-ebe4-45ec-ba05-d3e301fb6468 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:48:42.693485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3475a5-ebe4-45ec-ba05-d3e301fb6468 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:48:42.702930 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4254/8544] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:48:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:48:49.255557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:52.698895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:49:31.437824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8ca4f-dcf7-4cef-933a-36845b232800 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:49:31.448879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8ca4f-dcf7-4cef-933a-36845b232800 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:49:31.453982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1df73-9eda-4550-87b3-f8fce7b38c15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:49:31.454568 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1df73-9eda-4550-87b3-f8fce7b38c15 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:49:31.454846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1df73-9eda-4550-87b3-f8fce7b38c15 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:49:31.455117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1df73-9eda-4550-87b3-f8fce7b38c15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:49:31.464212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1df73-9eda-4550-87b3-f8fce7b38c15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:49:31.466119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1df73-9eda-4550-87b3-f8fce7b38c15 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:49:31.475972 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4291/8545] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:49:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:49:38.741323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3475a5-ebe4-45ec-ba05-d3e301fb6468 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:49:38.752507 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3475a5-ebe4-45ec-ba05-d3e301fb6468 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:49:38.757547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e031e5-92e7-41d1-b71c-4b0c9e3e6680 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:49:38.758127 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e031e5-92e7-41d1-b71c-4b0c9e3e6680 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:49:38.758403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e031e5-92e7-41d1-b71c-4b0c9e3e6680 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:49:38.758668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e031e5-92e7-41d1-b71c-4b0c9e3e6680 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:49:38.767189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e031e5-92e7-41d1-b71c-4b0c9e3e6680 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:49:38.768751 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e031e5-92e7-41d1-b71c-4b0c9e3e6680 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:49:38.779086 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4255/8546] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:49:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:49:41.471877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:49:48.775032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:07.449712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159bf3c-de16-4e5f-b309-9c70216a7b8f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:07.450308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159bf3c-de16-4e5f-b309-9c70216a7b8f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:07.450308 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159bf3c-de16-4e5f-b309-9c70216a7b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:07.490440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7159bf3c-de16-4e5f-b309-9c70216a7b8f None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:50:07.491653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7159bf3c-de16-4e5f-b309-9c70216a7b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:50:07.509313 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4292/8547] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 21:50:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:50:07.522190 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e031e5-92e7-41d1-b71c-4b0c9e3e6680 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:07.532118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e031e5-92e7-41d1-b71c-4b0c9e3e6680 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:07.535723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061eecf-5169-4fe6-8521-790a4d8bf6e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:07.536308 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061eecf-5169-4fe6-8521-790a4d8bf6e3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:07.536579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061eecf-5169-4fe6-8521-790a4d8bf6e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:07.536843 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061eecf-5169-4fe6-8521-790a4d8bf6e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:07.545816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061eecf-5169-4fe6-8521-790a4d8bf6e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:07.547240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061eecf-5169-4fe6-8521-790a4d8bf6e3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:07.555384 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4256/8548] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:50:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:50:07.858874 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159bf3c-de16-4e5f-b309-9c70216a7b8f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:07.870888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159bf3c-de16-4e5f-b309-9c70216a7b8f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:07.875767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cdb23-195b-476d-a651-a3bd5aaafa91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:07.876344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cdb23-195b-476d-a651-a3bd5aaafa91 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:07.876614 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cdb23-195b-476d-a651-a3bd5aaafa91 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:07.876875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cdb23-195b-476d-a651-a3bd5aaafa91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:07.885921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1cdb23-195b-476d-a651-a3bd5aaafa91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:07.887471 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1cdb23-195b-476d-a651-a3bd5aaafa91 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:07.897293 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4293/8549] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:50:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:50:17.551830 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:17.893311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=19 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:49.586267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061eecf-5169-4fe6-8521-790a4d8bf6e3 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:49.594823 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061eecf-5169-4fe6-8521-790a4d8bf6e3 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:49.598433 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6430aa4-200a-44b7-901e-18e5d794b9c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:49.599019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6430aa4-200a-44b7-901e-18e5d794b9c4 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:49.599294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6430aa4-200a-44b7-901e-18e5d794b9c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:49.599559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6430aa4-200a-44b7-901e-18e5d794b9c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:49.609451 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6430aa4-200a-44b7-901e-18e5d794b9c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:49.610868 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6430aa4-200a-44b7-901e-18e5d794b9c4 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:49.639097 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4257/8550] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:50:52.844204 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321c7ab-9231-46ba-bb24-77d79091dd89 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:52.844754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321c7ab-9231-46ba-bb24-77d79091dd89 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:52.844754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321c7ab-9231-46ba-bb24-77d79091dd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:52.887634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a321c7ab-9231-46ba-bb24-77d79091dd89 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:50:52.890140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a321c7ab-9231-46ba-bb24-77d79091dd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:50:52.902789 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4294/8551] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 21:50:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:50:52.914237 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6430aa4-200a-44b7-901e-18e5d794b9c4 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:52.924454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321c7ab-9231-46ba-bb24-77d79091dd89 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:52.928902 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6430aa4-200a-44b7-901e-18e5d794b9c4 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:52.934691 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:52.935396 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:52.935673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:52.936014 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] PATH_INFO: `/v3/users/4165cc6b781b4df3abd6a227170d4e46` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:52.937239 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4165cc6b781b4df3abd6a227170d4e46)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:52.937669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321c7ab-9231-46ba-bb24-77d79091dd89 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:52.939365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:52.943221 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6ff12-c4e0-48ac-979b-7f9dcdbf122e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:52.943829 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ff12-c4e0-48ac-979b-7f9dcdbf122e service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:52.944106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ff12-c4e0-48ac-979b-7f9dcdbf122e service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:52.944372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ff12-c4e0-48ac-979b-7f9dcdbf122e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:52.953967 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6ff12-c4e0-48ac-979b-7f9dcdbf122e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:52.955996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6ff12-c4e0-48ac-979b-7f9dcdbf122e service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:52.967795 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4295/8552] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:50:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:50:52.994470 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4165cc6b781b4df3abd6a227170d4e46'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:50:53.015977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4165cc6b781b4df3abd6a227170d4e46'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:50:53.023412 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4165cc6b781b4df3abd6a227170d4e46'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:50:53.031072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4165cc6b781b4df3abd6a227170d4e46'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:50:53.031700 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c18e430d-0f0c-4221-8c9f-4aa9429ef0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4165cc6b781b4df3abd6a227170d4e46'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:50:53.037875 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4258/8553] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:50:52 2022] DELETE /identity/v3/users/4165cc6b781b4df3abd6a227170d4e46 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:50:53.227121 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6ff12-c4e0-48ac-979b-7f9dcdbf122e service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:53.239139 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6ff12-c4e0-48ac-979b-7f9dcdbf122e service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:53.307046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:53.307794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:53.308068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:53.308324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] PATH_INFO: `/v3/projects/d42197537d0b407882fc410a5d359e1b` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:53.309492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d42197537d0b407882fc410a5d359e1b)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:53.311400 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:53.360077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd42197537d0b407882fc410a5d359e1b', 'request_id': 'req-69465c79-fdd3-44f0-ba08-08c96ec786f5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:50:53.367330 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd42197537d0b407882fc410a5d359e1b', 'request_id': 'req-69465c79-fdd3-44f0-ba08-08c96ec786f5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:50:53.374514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd42197537d0b407882fc410a5d359e1b', 'request_id': 'req-69465c79-fdd3-44f0-ba08-08c96ec786f5'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:50:53.397510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] The token cache is being invalidate because project d42197537d0b407882fc410a5d359e1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:50:53.397726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:50:53.398463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-69465c79-fdd3-44f0-ba08-08c96ec786f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:50:53.400035 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4296/8554] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:50:53 2022] DELETE /identity/v3/projects/d42197537d0b407882fc410a5d359e1b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:50:53.405570 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf25dc2-7a9a-4a5e-b947-98700be2a6e2 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:53.405844 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf25dc2-7a9a-4a5e-b947-98700be2a6e2 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:53.406105 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf25dc2-7a9a-4a5e-b947-98700be2a6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:53.453673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-aaf25dc2-7a9a-4a5e-b947-98700be2a6e2 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:50:53.454857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aaf25dc2-7a9a-4a5e-b947-98700be2a6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:50:53.513408 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4259/8555] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:50:53.577395 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4573-33ef-417a-9eca-865403bf460c None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:53.577736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4573-33ef-417a-9eca-865403bf460c None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:53.578107 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4573-33ef-417a-9eca-865403bf460c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:53.597981 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-906f4573-33ef-417a-9eca-865403bf460c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:50:53.599185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-906f4573-33ef-417a-9eca-865403bf460c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:50:53.607207 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4297/8556] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:50:53.611528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf25dc2-7a9a-4a5e-b947-98700be2a6e2 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:53.622218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf25dc2-7a9a-4a5e-b947-98700be2a6e2 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:53.626842 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b447cd-b6e4-4c9f-823e-715b631e7b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:53.627427 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b447cd-b6e4-4c9f-823e-715b631e7b08 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:53.627705 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b447cd-b6e4-4c9f-823e-715b631e7b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:53.627997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b447cd-b6e4-4c9f-823e-715b631e7b08 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:53.628635 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b447cd-b6e4-4c9f-823e-715b631e7b08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:50:53.628935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b447cd-b6e4-4c9f-823e-715b631e7b08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:53.630567 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b447cd-b6e4-4c9f-823e-715b631e7b08 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:53.643422 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4260/8557] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:50:53.647482 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4573-33ef-417a-9eca-865403bf460c None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:53.656811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4573-33ef-417a-9eca-865403bf460c None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:53.661503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db8ccd-2d8e-4de6-9228-c5523fb598ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:53.662089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db8ccd-2d8e-4de6-9228-c5523fb598ad admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:53.662364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db8ccd-2d8e-4de6-9228-c5523fb598ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:53.663457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db8ccd-2d8e-4de6-9228-c5523fb598ad admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:53.664158 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db8ccd-2d8e-4de6-9228-c5523fb598ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:53.666005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db8ccd-2d8e-4de6-9228-c5523fb598ad admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:53.703315 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4298/8558] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:50:53 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:50:53.707572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b447cd-b6e4-4c9f-823e-715b631e7b08 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:53.718002 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b447cd-b6e4-4c9f-823e-715b631e7b08 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:53.722998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:53.778211 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587afef4-9077-49e1-9ed7-cf2df5535ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:53.778808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587afef4-9077-49e1-9ed7-cf2df5535ada admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:53.779084 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587afef4-9077-49e1-9ed7-cf2df5535ada admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:53.779354 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587afef4-9077-49e1-9ed7-cf2df5535ada admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:53.779963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587afef4-9077-49e1-9ed7-cf2df5535ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:53.781922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587afef4-9077-49e1-9ed7-cf2df5535ada admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:53.803157 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4261/8559] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:50:53 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:50:53.807751 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db8ccd-2d8e-4de6-9228-c5523fb598ad admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:53.818085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db8ccd-2d8e-4de6-9228-c5523fb598ad admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:53.822906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6765e-da31-4c52-a5ea-0fbd0067fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:53.823534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6765e-da31-4c52-a5ea-0fbd0067fb8f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:53.823804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6765e-da31-4c52-a5ea-0fbd0067fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:53.824068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6765e-da31-4c52-a5ea-0fbd0067fb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:53.824694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6765e-da31-4c52-a5ea-0fbd0067fb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:53.826313 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6765e-da31-4c52-a5ea-0fbd0067fb8f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:53.828415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:53.840899 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4299/8560] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:50:53.842102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587afef4-9077-49e1-9ed7-cf2df5535ada admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:53.852261 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587afef4-9077-49e1-9ed7-cf2df5535ada admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:53.857555 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097700e7-a043-4dff-9fd5-052a71e94162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:53.858138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097700e7-a043-4dff-9fd5-052a71e94162 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:53.858410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097700e7-a043-4dff-9fd5-052a71e94162 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:53.858676 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097700e7-a043-4dff-9fd5-052a71e94162 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:53.859304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097700e7-a043-4dff-9fd5-052a71e94162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:53.860888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097700e7-a043-4dff-9fd5-052a71e94162 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:53.871521 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4262/8561] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:50:53.875588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6765e-da31-4c52-a5ea-0fbd0067fb8f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:53.886219 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6765e-da31-4c52-a5ea-0fbd0067fb8f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:53.891615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a62c5-f7fc-4c1a-a1a1-24a559c0a7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:53.892255 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a62c5-f7fc-4c1a-a1a1-24a559c0a7e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:53.892535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a62c5-f7fc-4c1a-a1a1-24a559c0a7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:53.892807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a62c5-f7fc-4c1a-a1a1-24a559c0a7e1 admin admin] PATH_INFO: `/v3/projects/b75b65002a4e4116945a865160f9ecd5/users/1e44ab28be024022bf6c8eaa84239494/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:53.907833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a62c5-f7fc-4c1a-a1a1-24a559c0a7e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75b65002a4e4116945a865160f9ecd5, user_id=1e44ab28be024022bf6c8eaa84239494, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:53.910439 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a62c5-f7fc-4c1a-a1a1-24a559c0a7e1 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:53.923974 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4300/8562] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:50:53 2022] PUT /identity/v3/projects/b75b65002a4e4116945a865160f9ecd5/users/1e44ab28be024022bf6c8eaa84239494/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:50:53.929018 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878601e-ad96-40c3-bd03-f0b8da0d8d92 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:53.929485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878601e-ad96-40c3-bd03-f0b8da0d8d92 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:53.929604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878601e-ad96-40c3-bd03-f0b8da0d8d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:53.948988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3878601e-ad96-40c3-bd03-f0b8da0d8d92 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:50:53.950262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3878601e-ad96-40c3-bd03-f0b8da0d8d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:50:54.010456 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4263/8563] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:50:54.020147 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a62c5-f7fc-4c1a-a1a1-24a559c0a7e1 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:54.021873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-116a62c5-f7fc-4c1a-a1a1-24a559c0a7e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:50:54.079225 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a62c5-f7fc-4c1a-a1a1-24a559c0a7e1 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:54.083010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20520f8-d73b-488a-8be9-582eb8533242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:54.083606 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20520f8-d73b-488a-8be9-582eb8533242 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:54.083883 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20520f8-d73b-488a-8be9-582eb8533242 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:54.084152 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20520f8-d73b-488a-8be9-582eb8533242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:54.093189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20520f8-d73b-488a-8be9-582eb8533242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:54.094605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20520f8-d73b-488a-8be9-582eb8533242 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:54.104516 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4301/8564] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:50:58.123178 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51961d1f-536a-4aec-8d0f-44d8109a9544 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:58.123661 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51961d1f-536a-4aec-8d0f-44d8109a9544 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:58.123971 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51961d1f-536a-4aec-8d0f-44d8109a9544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:58.157610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-51961d1f-536a-4aec-8d0f-44d8109a9544 None None] MFA Rules not processed for user `1e44ab28be024022bf6c8eaa84239494`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:50:58.158952 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-51961d1f-536a-4aec-8d0f-44d8109a9544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:50:58.232404 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4264/8565] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:50:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:50:58.242502 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659102f9-cf5c-4b3b-9f31-36de59522310 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:58.243000 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659102f9-cf5c-4b3b-9f31-36de59522310 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:58.243101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659102f9-cf5c-4b3b-9f31-36de59522310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:58.263050 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-659102f9-cf5c-4b3b-9f31-36de59522310 None None] MFA Rules not processed for user `1e44ab28be024022bf6c8eaa84239494`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:50:58.264231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-659102f9-cf5c-4b3b-9f31-36de59522310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:50:58.271422 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4302/8566] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:50:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:50:58.283875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51961d1f-536a-4aec-8d0f-44d8109a9544 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:50:58.285311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-51961d1f-536a-4aec-8d0f-44d8109a9544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:50:58.309500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:58.404417 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51961d1f-536a-4aec-8d0f-44d8109a9544 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:50:58.410183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f8857-95b1-4c2e-9ff2-cefdb16028e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:50:58.410719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f8857-95b1-4c2e-9ff2-cefdb16028e9 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:50:58.410992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f8857-95b1-4c2e-9ff2-cefdb16028e9 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:50:58.411257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f8857-95b1-4c2e-9ff2-cefdb16028e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:50:58.420168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f8857-95b1-4c2e-9ff2-cefdb16028e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:50:58.421989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f8857-95b1-4c2e-9ff2-cefdb16028e9 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:50:58.431992 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4265/8567] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:50:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:51:08.257117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:08.428096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=35 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:09.787736 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527fda7-787f-4808-93db-5a60e52243b2 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:09.788237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527fda7-787f-4808-93db-5a60e52243b2 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:09.788340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527fda7-787f-4808-93db-5a60e52243b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:09.807129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7527fda7-787f-4808-93db-5a60e52243b2 None None] MFA Rules not processed for user `1e44ab28be024022bf6c8eaa84239494`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:09.808322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7527fda7-787f-4808-93db-5a60e52243b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:09.815246 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4303/8568] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:10.067816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a04ee4-669c-4efb-b2f7-b272f9f6113f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:10.068338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a04ee4-669c-4efb-b2f7-b272f9f6113f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:10.068338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a04ee4-669c-4efb-b2f7-b272f9f6113f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:10.093245 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-48a04ee4-669c-4efb-b2f7-b272f9f6113f None None] MFA Rules not processed for user `176e98e72ba840c4a6546519b4c98d90`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:10.094478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-48a04ee4-669c-4efb-b2f7-b272f9f6113f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:10.101081 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4266/8569] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:51:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:10.112872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527fda7-787f-4808-93db-5a60e52243b2 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:10.114287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7527fda7-787f-4808-93db-5a60e52243b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:10.126467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527fda7-787f-4808-93db-5a60e52243b2 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:10.130010 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ba486-a2e5-431a-a5fb-76c8ccd88b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:10.130594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ba486-a2e5-431a-a5fb-76c8ccd88b77 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:10.130866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ba486-a2e5-431a-a5fb-76c8ccd88b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:10.131130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ba486-a2e5-431a-a5fb-76c8ccd88b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:10.139020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ba486-a2e5-431a-a5fb-76c8ccd88b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:10.140688 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ba486-a2e5-431a-a5fb-76c8ccd88b77 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:10.148824 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4304/8570] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:51:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:51:13.987414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a04ee4-669c-4efb-b2f7-b272f9f6113f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:13.996216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a04ee4-669c-4efb-b2f7-b272f9f6113f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:13.999972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb05f2-9803-4d32-ab1c-0a372bf6c2d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:14.000562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb05f2-9803-4d32-ab1c-0a372bf6c2d2 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:14.000835 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb05f2-9803-4d32-ab1c-0a372bf6c2d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:14.001142 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb05f2-9803-4d32-ab1c-0a372bf6c2d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:14.002832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-fccb05f2-9803-4d32-ab1c-0a372bf6c2d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:14.016545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb05f2-9803-4d32-ab1c-0a372bf6c2d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:14.018017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb05f2-9803-4d32-ab1c-0a372bf6c2d2 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:14.030074 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4267/8571] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:51:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:51:20.145261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=9 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:24.024896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:31.671022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ffeb7-60db-4836-820a-551ff066d0e6 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:31.671860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ffeb7-60db-4836-820a-551ff066d0e6 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:31.672136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ffeb7-60db-4836-820a-551ff066d0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:31.698209 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-209ffeb7-60db-4836-820a-551ff066d0e6 None None] MFA Rules not processed for user `1ca62592177d43b5b07106003c9efa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:31.699558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-209ffeb7-60db-4836-820a-551ff066d0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:31.737693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb05f2-9803-4d32-ab1c-0a372bf6c2d2 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:31.749678 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb05f2-9803-4d32-ab1c-0a372bf6c2d2 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:31.756113 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:31.756752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:31.757026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:31.757374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] PATH_INFO: `/v3/users/1e44ab28be024022bf6c8eaa84239494` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:31.758534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e44ab28be024022bf6c8eaa84239494)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:31.760548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:31.811464 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4305/8572] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 21:51:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:31.823305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ffeb7-60db-4836-820a-551ff066d0e6 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:31.825483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-209ffeb7-60db-4836-820a-551ff066d0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:31.827601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e44ab28be024022bf6c8eaa84239494'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:31.839045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ffeb7-60db-4836-820a-551ff066d0e6 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:31.840789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e44ab28be024022bf6c8eaa84239494'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:31.844433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864489b6-d49c-4ac7-ba82-1c7498d87a2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:31.845065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864489b6-d49c-4ac7-ba82-1c7498d87a2a service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:31.845669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864489b6-d49c-4ac7-ba82-1c7498d87a2a service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:31.846041 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864489b6-d49c-4ac7-ba82-1c7498d87a2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:31.848707 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e44ab28be024022bf6c8eaa84239494'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:31.855656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864489b6-d49c-4ac7-ba82-1c7498d87a2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:31.856048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e44ab28be024022bf6c8eaa84239494'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:31.856693 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e44ab28be024022bf6c8eaa84239494'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:31.857343 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864489b6-d49c-4ac7-ba82-1c7498d87a2a service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:31.863104 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4268/8573] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:51:31 2022] DELETE /identity/v3/users/1e44ab28be024022bf6c8eaa84239494 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:51:31.867768 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4306/8574] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:51:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:51:32.047603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:32.049149 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:32.119759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f712e79-c463-418f-8110-7c86185952bd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:32.124368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:32.124991 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.125299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.125568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] PATH_INFO: `/v3/projects/b75b65002a4e4116945a865160f9ecd5` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.126687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b75b65002a4e4116945a865160f9ecd5)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:32.128599 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:32.176776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b75b65002a4e4116945a865160f9ecd5', 'request_id': 'req-13096ad7-527b-4b17-8b32-ece0a778609a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:32.184350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b75b65002a4e4116945a865160f9ecd5', 'request_id': 'req-13096ad7-527b-4b17-8b32-ece0a778609a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:32.190716 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b75b65002a4e4116945a865160f9ecd5', 'request_id': 'req-13096ad7-527b-4b17-8b32-ece0a778609a'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:32.202780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] The token cache is being invalidate because project b75b65002a4e4116945a865160f9ecd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:51:32.203068 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:32.203754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-13096ad7-527b-4b17-8b32-ece0a778609a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:32.205345 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4269/8575] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:51:32 2022] DELETE /identity/v3/projects/b75b65002a4e4116945a865160f9ecd5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:51:32.210466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f7d0a-4c40-4031-85d9-b7477bc64f40 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.210746 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f7d0a-4c40-4031-85d9-b7477bc64f40 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.211014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f7d0a-4c40-4031-85d9-b7477bc64f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.232151 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-ff3f7d0a-4c40-4031-85d9-b7477bc64f40 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:32.233378 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ff3f7d0a-4c40-4031-85d9-b7477bc64f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:32.439713 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4307/8576] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:32.447832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2553af4-9d4a-41ba-84ef-5990272ebdc8 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.448108 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2553af4-9d4a-41ba-84ef-5990272ebdc8 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.448375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2553af4-9d4a-41ba-84ef-5990272ebdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.457078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:32.471875 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-c2553af4-9d4a-41ba-84ef-5990272ebdc8 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:32.473122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c2553af4-9d4a-41ba-84ef-5990272ebdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:32.481025 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4270/8577] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:32.485454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f7d0a-4c40-4031-85d9-b7477bc64f40 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:32.496500 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f7d0a-4c40-4031-85d9-b7477bc64f40 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:32.501213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107d311-b744-44d4-aebf-d2236fd34ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:32.501812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107d311-b744-44d4-aebf-d2236fd34ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.502085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107d311-b744-44d4-aebf-d2236fd34ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.502350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107d311-b744-44d4-aebf-d2236fd34ef7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.502980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107d311-b744-44d4-aebf-d2236fd34ef7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:51:32.503275 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107d311-b744-44d4-aebf-d2236fd34ef7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:32.504839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107d311-b744-44d4-aebf-d2236fd34ef7 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:32.517536 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4308/8578] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:51:32.521895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2553af4-9d4a-41ba-84ef-5990272ebdc8 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:32.531095 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2553af4-9d4a-41ba-84ef-5990272ebdc8 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:32.535640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d8294-328f-4565-bad1-1849db3d4d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:32.536213 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d8294-328f-4565-bad1-1849db3d4d42 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.536513 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d8294-328f-4565-bad1-1849db3d4d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.536776 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d8294-328f-4565-bad1-1849db3d4d42 admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.537482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d8294-328f-4565-bad1-1849db3d4d42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:32.539528 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d8294-328f-4565-bad1-1849db3d4d42 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:32.565902 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4271/8579] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:51:32 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:51:32.570065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107d311-b744-44d4-aebf-d2236fd34ef7 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:32.580600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107d311-b744-44d4-aebf-d2236fd34ef7 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:32.636808 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8e64e-f8ae-4e32-8608-15f5d0bd38c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:32.637515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8e64e-f8ae-4e32-8608-15f5d0bd38c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.637785 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8e64e-f8ae-4e32-8608-15f5d0bd38c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.638042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8e64e-f8ae-4e32-8608-15f5d0bd38c2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.638655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8e64e-f8ae-4e32-8608-15f5d0bd38c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:32.640787 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8e64e-f8ae-4e32-8608-15f5d0bd38c2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:32.649492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:32.662865 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4309/8580] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:51:32 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:51:32.666761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d8294-328f-4565-bad1-1849db3d4d42 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:32.677315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d8294-328f-4565-bad1-1849db3d4d42 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:32.681940 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f77da-0658-4ef5-b316-fde9fff01fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:32.682524 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f77da-0658-4ef5-b316-fde9fff01fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.682792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f77da-0658-4ef5-b316-fde9fff01fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.683060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f77da-0658-4ef5-b316-fde9fff01fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.684331 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f77da-0658-4ef5-b316-fde9fff01fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:32.685959 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f77da-0658-4ef5-b316-fde9fff01fc1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:32.695925 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4272/8581] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:51:32.701023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8e64e-f8ae-4e32-8608-15f5d0bd38c2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:32.711102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8e64e-f8ae-4e32-8608-15f5d0bd38c2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:32.716176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41196840-2333-48c0-9275-1806c332ddd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:32.716778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41196840-2333-48c0-9275-1806c332ddd3 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.717125 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41196840-2333-48c0-9275-1806c332ddd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.717392 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41196840-2333-48c0-9275-1806c332ddd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.718024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41196840-2333-48c0-9275-1806c332ddd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:32.719618 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41196840-2333-48c0-9275-1806c332ddd3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:32.730044 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4310/8582] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:51:32.734781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f77da-0658-4ef5-b316-fde9fff01fc1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:32.744789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f77da-0658-4ef5-b316-fde9fff01fc1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:32.750091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdd032-0704-4d3e-9686-8a903f8fea64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:32.750727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdd032-0704-4d3e-9686-8a903f8fea64 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.751007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdd032-0704-4d3e-9686-8a903f8fea64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.751278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdd032-0704-4d3e-9686-8a903f8fea64 admin admin] PATH_INFO: `/v3/projects/1e0ccb7910df4a05b7c202cf465b98bb/users/c28eaa42e61742ceb14ab645e7bddcd6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.766257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdd032-0704-4d3e-9686-8a903f8fea64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e0ccb7910df4a05b7c202cf465b98bb, user_id=c28eaa42e61742ceb14ab645e7bddcd6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:32.768909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdd032-0704-4d3e-9686-8a903f8fea64 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:32.781657 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4273/8583] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:51:32 2022] PUT /identity/v3/projects/1e0ccb7910df4a05b7c202cf465b98bb/users/c28eaa42e61742ceb14ab645e7bddcd6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:51:32.793381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89116d-b161-4d7b-98f9-73c839e5d1aa None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.793953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89116d-b161-4d7b-98f9-73c839e5d1aa None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.793953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89116d-b161-4d7b-98f9-73c839e5d1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.822016 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-df89116d-b161-4d7b-98f9-73c839e5d1aa None None] MFA Rules not processed for user `c28eaa42e61742ceb14ab645e7bddcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:32.823201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-df89116d-b161-4d7b-98f9-73c839e5d1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:32.887126 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4311/8584] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:32.897896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdd032-0704-4d3e-9686-8a903f8fea64 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:32.899616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-cbfdd032-0704-4d3e-9686-8a903f8fea64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:32.956912 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdd032-0704-4d3e-9686-8a903f8fea64 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:32.960526 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785098e-c49f-44b6-bb77-5729a778f902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:32.961158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785098e-c49f-44b6-bb77-5729a778f902 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:32.961439 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785098e-c49f-44b6-bb77-5729a778f902 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:32.961709 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785098e-c49f-44b6-bb77-5729a778f902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:32.970746 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785098e-c49f-44b6-bb77-5729a778f902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:32.972176 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785098e-c49f-44b6-bb77-5729a778f902 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:32.981355 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4274/8585] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:51:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:51:42.883754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:42.977081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:45.163029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89116d-b161-4d7b-98f9-73c839e5d1aa None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:45.229520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89116d-b161-4d7b-98f9-73c839e5d1aa None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:45.234196 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:45.234822 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:45.235116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:45.235381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] PATH_INFO: `/v3/users/c28eaa42e61742ceb14ab645e7bddcd6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:45.236515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28eaa42e61742ceb14ab645e7bddcd6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:45.238446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:45.288475 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28eaa42e61742ceb14ab645e7bddcd6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:45.296248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28eaa42e61742ceb14ab645e7bddcd6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:45.308182 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28eaa42e61742ceb14ab645e7bddcd6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:45.315483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28eaa42e61742ceb14ab645e7bddcd6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:45.316280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28eaa42e61742ceb14ab645e7bddcd6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:45.322707 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4312/8586] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:51:45 2022] DELETE /identity/v3/users/c28eaa42e61742ceb14ab645e7bddcd6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:51:45.327299 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7810c-eac6-4048-8665-55b97d7497f1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:45.327756 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7810c-eac6-4048-8665-55b97d7497f1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:45.327946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7810c-eac6-4048-8665-55b97d7497f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:45.348240 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-00c7810c-eac6-4048-8665-55b97d7497f1 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:45.349481 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-00c7810c-eac6-4048-8665-55b97d7497f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:45.409344 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4275/8587] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:45.419963 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:45.421410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:45.478126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ed642-6b60-473e-9b00-a49be246b2ac admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:45.482397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c88c52-088b-4932-8c0f-3316d711804e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:45.482993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c88c52-088b-4932-8c0f-3316d711804e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:45.483269 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c88c52-088b-4932-8c0f-3316d711804e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:45.483537 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c88c52-088b-4932-8c0f-3316d711804e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:45.492578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c88c52-088b-4932-8c0f-3316d711804e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:45.494321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c88c52-088b-4932-8c0f-3316d711804e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:45.503925 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4313/8588] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:51:45.676351 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7810c-eac6-4048-8665-55b97d7497f1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:45.677796 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-00c7810c-eac6-4048-8665-55b97d7497f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:45.690955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7810c-eac6-4048-8665-55b97d7497f1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:45.695619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:45.696246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:45.696521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:45.696792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] PATH_INFO: `/v3/projects/1e0ccb7910df4a05b7c202cf465b98bb` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:45.699092 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e0ccb7910df4a05b7c202cf465b98bb)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:45.701216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:45.754500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e0ccb7910df4a05b7c202cf465b98bb', 'request_id': 'req-3694192e-0625-4813-aec3-2a2a09883e99'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:45.762282 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e0ccb7910df4a05b7c202cf465b98bb', 'request_id': 'req-3694192e-0625-4813-aec3-2a2a09883e99'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:45.768727 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e0ccb7910df4a05b7c202cf465b98bb', 'request_id': 'req-3694192e-0625-4813-aec3-2a2a09883e99'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:45.781482 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] The token cache is being invalidate because project 1e0ccb7910df4a05b7c202cf465b98bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:51:45.781763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:45.782407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3694192e-0625-4813-aec3-2a2a09883e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:45.783973 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4276/8589] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:51:45 2022] DELETE /identity/v3/projects/1e0ccb7910df4a05b7c202cf465b98bb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:51:45.789169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a312b7-caf0-42ee-9306-eccb0f5fe2c0 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:45.789444 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a312b7-caf0-42ee-9306-eccb0f5fe2c0 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:45.789717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a312b7-caf0-42ee-9306-eccb0f5fe2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:45.809338 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-38a312b7-caf0-42ee-9306-eccb0f5fe2c0 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:45.810529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-38a312b7-caf0-42ee-9306-eccb0f5fe2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:45.827849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:45.872361 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4314/8590] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:45.880290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8f082-3033-4900-b803-4e5847132758 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:45.880593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8f082-3033-4900-b803-4e5847132758 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:45.880858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8f082-3033-4900-b803-4e5847132758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:45.901471 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-a6d8f082-3033-4900-b803-4e5847132758 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:45.903004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a6d8f082-3033-4900-b803-4e5847132758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:45.910895 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4277/8591] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:45.915124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a312b7-caf0-42ee-9306-eccb0f5fe2c0 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:45.924620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a312b7-caf0-42ee-9306-eccb0f5fe2c0 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:45.929110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef84e5-c81f-44a9-a0d6-8d21ad24e1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:45.929684 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef84e5-c81f-44a9-a0d6-8d21ad24e1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:45.929952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef84e5-c81f-44a9-a0d6-8d21ad24e1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:45.930211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef84e5-c81f-44a9-a0d6-8d21ad24e1b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:45.930834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef84e5-c81f-44a9-a0d6-8d21ad24e1b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:51:45.931126 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef84e5-c81f-44a9-a0d6-8d21ad24e1b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:45.932677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef84e5-c81f-44a9-a0d6-8d21ad24e1b2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:45.944447 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4315/8592] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:51:45.949522 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8f082-3033-4900-b803-4e5847132758 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:45.958812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8f082-3033-4900-b803-4e5847132758 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:45.963473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a99386-dab3-4303-a931-02a25d5d722d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:45.964100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a99386-dab3-4303-a931-02a25d5d722d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:45.964564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a99386-dab3-4303-a931-02a25d5d722d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:45.964885 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a99386-dab3-4303-a931-02a25d5d722d admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:45.965530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a99386-dab3-4303-a931-02a25d5d722d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:45.967426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a99386-dab3-4303-a931-02a25d5d722d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:45.993556 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4278/8593] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:51:45 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:51:45.998208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef84e5-c81f-44a9-a0d6-8d21ad24e1b2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:46.008085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef84e5-c81f-44a9-a0d6-8d21ad24e1b2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:46.066170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e15a8-4e29-4f1e-9167-c405fa533c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:46.066805 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e15a8-4e29-4f1e-9167-c405fa533c76 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:46.067064 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e15a8-4e29-4f1e-9167-c405fa533c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:46.067326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e15a8-4e29-4f1e-9167-c405fa533c76 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:46.067947 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e15a8-4e29-4f1e-9167-c405fa533c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:46.069855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e15a8-4e29-4f1e-9167-c405fa533c76 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:46.090456 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4316/8594] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:51:45 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:51:46.094801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a99386-dab3-4303-a931-02a25d5d722d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:46.105670 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a99386-dab3-4303-a931-02a25d5d722d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:46.110374 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cbbe7-96fa-4509-9fbc-8dd19d6e2114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:46.110954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cbbe7-96fa-4509-9fbc-8dd19d6e2114 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:46.111233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cbbe7-96fa-4509-9fbc-8dd19d6e2114 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:46.111500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cbbe7-96fa-4509-9fbc-8dd19d6e2114 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:46.112133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cbbe7-96fa-4509-9fbc-8dd19d6e2114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:46.113783 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cbbe7-96fa-4509-9fbc-8dd19d6e2114 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:46.124449 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4279/8595] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:51:46.128040 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e15a8-4e29-4f1e-9167-c405fa533c76 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:46.139036 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e15a8-4e29-4f1e-9167-c405fa533c76 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:46.143941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9895bc8-034d-48d1-8da5-9a66ca03fa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:46.144529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9895bc8-034d-48d1-8da5-9a66ca03fa01 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:46.144818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9895bc8-034d-48d1-8da5-9a66ca03fa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:46.145146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9895bc8-034d-48d1-8da5-9a66ca03fa01 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:46.146131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9895bc8-034d-48d1-8da5-9a66ca03fa01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:46.147732 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9895bc8-034d-48d1-8da5-9a66ca03fa01 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:46.157677 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4317/8596] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:51:46.161763 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cbbe7-96fa-4509-9fbc-8dd19d6e2114 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:46.173675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cbbe7-96fa-4509-9fbc-8dd19d6e2114 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:46.178469 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f796e4-bd0a-4062-8dd3-20995a87e265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:46.179115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f796e4-bd0a-4062-8dd3-20995a87e265 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:46.179392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f796e4-bd0a-4062-8dd3-20995a87e265 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:46.179692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f796e4-bd0a-4062-8dd3-20995a87e265 admin admin] PATH_INFO: `/v3/projects/c8be183ad7f1407abca18b0f4ec0b739/users/4c8745e030604af1be2d1d2753e01bc7/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:46.197377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f796e4-bd0a-4062-8dd3-20995a87e265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8be183ad7f1407abca18b0f4ec0b739, user_id=4c8745e030604af1be2d1d2753e01bc7, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:46.200896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f796e4-bd0a-4062-8dd3-20995a87e265 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:46.216899 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4280/8597] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:51:46 2022] PUT /identity/v3/projects/c8be183ad7f1407abca18b0f4ec0b739/users/4c8745e030604af1be2d1d2753e01bc7/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:51:46.226958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406d09e-bfd0-429a-a881-d066254d02c9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:46.227428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406d09e-bfd0-429a-a881-d066254d02c9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:46.227553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406d09e-bfd0-429a-a881-d066254d02c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:46.255696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a406d09e-bfd0-429a-a881-d066254d02c9 None None] MFA Rules not processed for user `4c8745e030604af1be2d1d2753e01bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:46.256885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a406d09e-bfd0-429a-a881-d066254d02c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:46.320389 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4318/8598] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:46.330728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f796e4-bd0a-4062-8dd3-20995a87e265 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:46.332562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-15f796e4-bd0a-4062-8dd3-20995a87e265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:46.354311 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:46.392529 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f796e4-bd0a-4062-8dd3-20995a87e265 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:46.396206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36845c8f-e8a2-4d4f-82c4-55477e24475f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:46.396794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36845c8f-e8a2-4d4f-82c4-55477e24475f service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:46.397098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36845c8f-e8a2-4d4f-82c4-55477e24475f service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:46.397360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36845c8f-e8a2-4d4f-82c4-55477e24475f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:46.406000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36845c8f-e8a2-4d4f-82c4-55477e24475f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:46.407467 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36845c8f-e8a2-4d4f-82c4-55477e24475f service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:46.416445 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4281/8599] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:51:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:51:50.015084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406d09e-bfd0-429a-a881-d066254d02c9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:50.033903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:50.078290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406d09e-bfd0-429a-a881-d066254d02c9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:50.082831 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:50.083452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:50.083725 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:50.083994 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] PATH_INFO: `/v3/users/4c8745e030604af1be2d1d2753e01bc7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:50.085136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c8745e030604af1be2d1d2753e01bc7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:50.087021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:50.132628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c8745e030604af1be2d1d2753e01bc7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:50.140235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c8745e030604af1be2d1d2753e01bc7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:50.152326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c8745e030604af1be2d1d2753e01bc7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:50.159390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c8745e030604af1be2d1d2753e01bc7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:50.159993 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c8745e030604af1be2d1d2753e01bc7'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:50.165777 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4319/8600] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:51:50 2022] DELETE /identity/v3/users/4c8745e030604af1be2d1d2753e01bc7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:51:50.171196 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120b367-1e52-458e-a348-e889d6c34a55 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:50.171674 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120b367-1e52-458e-a348-e889d6c34a55 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:50.171782 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120b367-1e52-458e-a348-e889d6c34a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:50.191478 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7120b367-1e52-458e-a348-e889d6c34a55 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:50.192689 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7120b367-1e52-458e-a348-e889d6c34a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:50.252967 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4282/8601] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:50.263580 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:50.265314 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:50.321180 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796115e8-8c77-4648-9887-b323b2e6747c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:50.325417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e20ecb-2385-4998-973e-a1e8587d4b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:50.326032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e20ecb-2385-4998-973e-a1e8587d4b47 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:50.326305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e20ecb-2385-4998-973e-a1e8587d4b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:50.326572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e20ecb-2385-4998-973e-a1e8587d4b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:50.335794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e20ecb-2385-4998-973e-a1e8587d4b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:50.337224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e20ecb-2385-4998-973e-a1e8587d4b47 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:50.347195 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4320/8602] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:51:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:51:50.827468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120b367-1e52-458e-a348-e889d6c34a55 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:50.828888 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7120b367-1e52-458e-a348-e889d6c34a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:50.843124 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120b367-1e52-458e-a348-e889d6c34a55 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:50.847771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:50.849071 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:50.849688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:50.849960 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] PATH_INFO: `/v3/projects/c8be183ad7f1407abca18b0f4ec0b739` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:50.851075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8be183ad7f1407abca18b0f4ec0b739)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:50.853004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:50.901953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8be183ad7f1407abca18b0f4ec0b739', 'request_id': 'req-848a5d20-3411-49ce-8edc-5ffc7791e882'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:50.909712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8be183ad7f1407abca18b0f4ec0b739', 'request_id': 'req-848a5d20-3411-49ce-8edc-5ffc7791e882'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:50.916162 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8be183ad7f1407abca18b0f4ec0b739', 'request_id': 'req-848a5d20-3411-49ce-8edc-5ffc7791e882'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:50.927148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] The token cache is being invalidate because project c8be183ad7f1407abca18b0f4ec0b739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:51:50.927429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:50.928053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-848a5d20-3411-49ce-8edc-5ffc7791e882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:51:50.929567 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4283/8603] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:51:50 2022] DELETE /identity/v3/projects/c8be183ad7f1407abca18b0f4ec0b739 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:51:50.935985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6d27-265a-405b-8a89-18e9881f53f9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:50.936319 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6d27-265a-405b-8a89-18e9881f53f9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:50.936582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6d27-265a-405b-8a89-18e9881f53f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:50.955928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6c9d6d27-265a-405b-8a89-18e9881f53f9 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:50.957146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6c9d6d27-265a-405b-8a89-18e9881f53f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:50.989938 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:51.017819 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4321/8604] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:51.025500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa0aac-268e-4c42-b834-297526131d78 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:51.025780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa0aac-268e-4c42-b834-297526131d78 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:51.026051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa0aac-268e-4c42-b834-297526131d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:51.047668 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-01fa0aac-268e-4c42-b834-297526131d78 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:51.048870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-01fa0aac-268e-4c42-b834-297526131d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:51.056783 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4284/8605] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:51.061051 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6d27-265a-405b-8a89-18e9881f53f9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:51.070572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6d27-265a-405b-8a89-18e9881f53f9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:51.075619 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346330ad-69f9-4d9f-bf36-56cfc732967b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:51.076197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346330ad-69f9-4d9f-bf36-56cfc732967b admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:51.076466 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346330ad-69f9-4d9f-bf36-56cfc732967b admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:51.076729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346330ad-69f9-4d9f-bf36-56cfc732967b admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:51.077390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346330ad-69f9-4d9f-bf36-56cfc732967b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:51:51.077683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346330ad-69f9-4d9f-bf36-56cfc732967b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:51.079235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346330ad-69f9-4d9f-bf36-56cfc732967b admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:51.091064 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4322/8606] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:51:51.095291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa0aac-268e-4c42-b834-297526131d78 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:51.104586 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa0aac-268e-4c42-b834-297526131d78 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:51.109404 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90644b-06b6-493c-93e8-a9bd8e6d117f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:51.110049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90644b-06b6-493c-93e8-a9bd8e6d117f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:51.110330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90644b-06b6-493c-93e8-a9bd8e6d117f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:51.110600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90644b-06b6-493c-93e8-a9bd8e6d117f admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:51.111218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90644b-06b6-493c-93e8-a9bd8e6d117f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:51.113146 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90644b-06b6-493c-93e8-a9bd8e6d117f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:51.141052 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4285/8607] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:51:51 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:51:51.145889 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346330ad-69f9-4d9f-bf36-56cfc732967b admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:51.156699 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346330ad-69f9-4d9f-bf36-56cfc732967b admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:51.215468 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482e8d1-6907-4ce8-8619-52a0e365b0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:51.216365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482e8d1-6907-4ce8-8619-52a0e365b0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:51.216646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482e8d1-6907-4ce8-8619-52a0e365b0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:51.216909 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482e8d1-6907-4ce8-8619-52a0e365b0f5 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:51.217561 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482e8d1-6907-4ce8-8619-52a0e365b0f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:51.219536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482e8d1-6907-4ce8-8619-52a0e365b0f5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:51.247101 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4323/8608] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:51:51 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:51:51.248357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90644b-06b6-493c-93e8-a9bd8e6d117f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:51.259840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90644b-06b6-493c-93e8-a9bd8e6d117f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:51.268321 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31654de2-e939-4bde-a233-cc0b2cbfdf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:51.268871 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31654de2-e939-4bde-a233-cc0b2cbfdf8c admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:51.269200 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31654de2-e939-4bde-a233-cc0b2cbfdf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:51.269475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31654de2-e939-4bde-a233-cc0b2cbfdf8c admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:51.270118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31654de2-e939-4bde-a233-cc0b2cbfdf8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:51.271738 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31654de2-e939-4bde-a233-cc0b2cbfdf8c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:51.282875 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4286/8609] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:51:51.287174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482e8d1-6907-4ce8-8619-52a0e365b0f5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:51.298998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482e8d1-6907-4ce8-8619-52a0e365b0f5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:51.303930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ad6f3-193a-489f-bc65-a0e7069c55ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:51.304683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ad6f3-193a-489f-bc65-a0e7069c55ea admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:51.304803 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ad6f3-193a-489f-bc65-a0e7069c55ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:51.305075 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ad6f3-193a-489f-bc65-a0e7069c55ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:51.305708 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ad6f3-193a-489f-bc65-a0e7069c55ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:51.307274 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ad6f3-193a-489f-bc65-a0e7069c55ea admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:51.318606 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4324/8610] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:51:51.322809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31654de2-e939-4bde-a233-cc0b2cbfdf8c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:51.333327 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31654de2-e939-4bde-a233-cc0b2cbfdf8c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:51.337927 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de046a-c3a2-427b-80cd-695d00b7c0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:51.338561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de046a-c3a2-427b-80cd-695d00b7c0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:51.338836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de046a-c3a2-427b-80cd-695d00b7c0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:51.339968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de046a-c3a2-427b-80cd-695d00b7c0cb admin admin] PATH_INFO: `/v3/projects/cd7071741b2e4f35954da54683a2b55c/users/4768c0948f1b44689210529a25211aa7/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:51.342103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:51.358424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de046a-c3a2-427b-80cd-695d00b7c0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7071741b2e4f35954da54683a2b55c, user_id=4768c0948f1b44689210529a25211aa7, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:51.361262 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de046a-c3a2-427b-80cd-695d00b7c0cb admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:51.373577 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4287/8611] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:51:51 2022] PUT /identity/v3/projects/cd7071741b2e4f35954da54683a2b55c/users/4768c0948f1b44689210529a25211aa7/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:51:51.378962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164f40f-2db5-4e46-b39c-00ea6413989e None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:51.379231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164f40f-2db5-4e46-b39c-00ea6413989e None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:51.379496 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164f40f-2db5-4e46-b39c-00ea6413989e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:51.398683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-6164f40f-2db5-4e46-b39c-00ea6413989e None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:51.399877 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-6164f40f-2db5-4e46-b39c-00ea6413989e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:51.459608 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4325/8612] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:51.470106 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de046a-c3a2-427b-80cd-695d00b7c0cb admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:51.471748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a0de046a-c3a2-427b-80cd-695d00b7c0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:51.531852 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de046a-c3a2-427b-80cd-695d00b7c0cb admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:51.535460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d46ff7-f8c7-498d-90ea-b66e4c3daf34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:51.536055 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d46ff7-f8c7-498d-90ea-b66e4c3daf34 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:51.536326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d46ff7-f8c7-498d-90ea-b66e4c3daf34 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:51.536593 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d46ff7-f8c7-498d-90ea-b66e4c3daf34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:51.546450 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d46ff7-f8c7-498d-90ea-b66e4c3daf34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:51.547876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d46ff7-f8c7-498d-90ea-b66e4c3daf34 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:51.557354 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4288/8613] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:51:55.364663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9b3be-c9a9-4437-b9c5-b1da25a1405b None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:55.365156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9b3be-c9a9-4437-b9c5-b1da25a1405b None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:55.365356 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9b3be-c9a9-4437-b9c5-b1da25a1405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:55.403705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d8f9b3be-c9a9-4437-b9c5-b1da25a1405b None None] MFA Rules not processed for user `4768c0948f1b44689210529a25211aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:51:55.405939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d8f9b3be-c9a9-4437-b9c5-b1da25a1405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:51:55.448837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:55.490275 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4326/8614] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:51:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:51:55.495272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d46ff7-f8c7-498d-90ea-b66e4c3daf34 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:55.506998 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d46ff7-f8c7-498d-90ea-b66e4c3daf34 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:55.512726 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b82bb-8cc0-4e23-9bdd-d8a00156d10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:55.513458 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b82bb-8cc0-4e23-9bdd-d8a00156d10c admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:55.513918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b82bb-8cc0-4e23-9bdd-d8a00156d10c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:55.514191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b82bb-8cc0-4e23-9bdd-d8a00156d10c admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:55.515067 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b82bb-8cc0-4e23-9bdd-d8a00156d10c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:55.517493 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b82bb-8cc0-4e23-9bdd-d8a00156d10c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:55.551518 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4289/8615] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:51:55 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:51:55.555863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9b3be-c9a9-4437-b9c5-b1da25a1405b None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:55.567913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9b3be-c9a9-4437-b9c5-b1da25a1405b None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:55.641100 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fc652-3ea5-4233-8ae9-bd5ebd2cb1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:55.641885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fc652-3ea5-4233-8ae9-bd5ebd2cb1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:55.642264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fc652-3ea5-4233-8ae9-bd5ebd2cb1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:55.642630 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fc652-3ea5-4233-8ae9-bd5ebd2cb1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:55.643380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fc652-3ea5-4233-8ae9-bd5ebd2cb1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:55.645427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fc652-3ea5-4233-8ae9-bd5ebd2cb1b5 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:55.671440 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4327/8616] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:51:55 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:51:55.675457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b82bb-8cc0-4e23-9bdd-d8a00156d10c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:55.687442 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b82bb-8cc0-4e23-9bdd-d8a00156d10c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:55.693773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2096d15-2102-4645-ac56-b6ddd5792875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:55.694874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2096d15-2102-4645-ac56-b6ddd5792875 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:55.694874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2096d15-2102-4645-ac56-b6ddd5792875 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:55.695065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2096d15-2102-4645-ac56-b6ddd5792875 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:55.695568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2096d15-2102-4645-ac56-b6ddd5792875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:55.697225 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2096d15-2102-4645-ac56-b6ddd5792875 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:55.711411 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4290/8617] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:51:55.716216 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fc652-3ea5-4233-8ae9-bd5ebd2cb1b5 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:55.727119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fc652-3ea5-4233-8ae9-bd5ebd2cb1b5 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:55.735397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0397ed4-7ae4-4e4c-a4a5-7ab47eeeeb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:55.736085 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0397ed4-7ae4-4e4c-a4a5-7ab47eeeeb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:55.736353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0397ed4-7ae4-4e4c-a4a5-7ab47eeeeb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:55.736812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0397ed4-7ae4-4e4c-a4a5-7ab47eeeeb5c admin admin] PATH_INFO: `/v3/projects/245363bb1f1744b69f35a92ae536c73f/users/54824064641441e189d7c735f010e84b/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:55.756766 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0397ed4-7ae4-4e4c-a4a5-7ab47eeeeb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=245363bb1f1744b69f35a92ae536c73f, user_id=54824064641441e189d7c735f010e84b, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:55.759682 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0397ed4-7ae4-4e4c-a4a5-7ab47eeeeb5c admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:55.778870 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4328/8618] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:51:55 2022] PUT /identity/v3/projects/245363bb1f1744b69f35a92ae536c73f/users/54824064641441e189d7c735f010e84b/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:51:55.783166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2096d15-2102-4645-ac56-b6ddd5792875 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:55.796312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2096d15-2102-4645-ac56-b6ddd5792875 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:55.859168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746fb48-1c9f-4e48-8b02-ea5692f6dee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:55.859741 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746fb48-1c9f-4e48-8b02-ea5692f6dee8 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:55.860020 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746fb48-1c9f-4e48-8b02-ea5692f6dee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:55.860289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746fb48-1c9f-4e48-8b02-ea5692f6dee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:55.860922 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746fb48-1c9f-4e48-8b02-ea5692f6dee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:55.862564 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746fb48-1c9f-4e48-8b02-ea5692f6dee8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:55.874422 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4291/8619] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:51:55.878694 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0397ed4-7ae4-4e4c-a4a5-7ab47eeeeb5c admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:51:55.890648 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0397ed4-7ae4-4e4c-a4a5-7ab47eeeeb5c admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:51:55.895735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997287-2367-480c-b2fc-7a5badf63fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:51:55.896448 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997287-2367-480c-b2fc-7a5badf63fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:51:55.896652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997287-2367-480c-b2fc-7a5badf63fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:51:55.896918 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997287-2367-480c-b2fc-7a5badf63fc0 admin admin] PATH_INFO: `/v3/projects/245363bb1f1744b69f35a92ae536c73f/users/54824064641441e189d7c735f010e84b/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:51:55.898983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7997287-2367-480c-b2fc-7a5badf63fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=245363bb1f1744b69f35a92ae536c73f, user_id=54824064641441e189d7c735f010e84b, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:51:55.902244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7997287-2367-480c-b2fc-7a5badf63fc0 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:51:55.914708 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4329/8620] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:51:55 2022] PUT /identity/v3/projects/245363bb1f1744b69f35a92ae536c73f/users/54824064641441e189d7c735f010e84b/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:52:00.859153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fb377-3e71-47d5-b300-e092834a7512 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:00.859647 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fb377-3e71-47d5-b300-e092834a7512 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:00.859749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fb377-3e71-47d5-b300-e092834a7512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:00.889760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-6f0fb377-3e71-47d5-b300-e092834a7512 None None] MFA Rules not processed for user `54824064641441e189d7c735f010e84b`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:52:00.891267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6f0fb377-3e71-47d5-b300-e092834a7512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:00.933356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:00.970148 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4292/8621] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:52:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:52:01.006020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997287-2367-480c-b2fc-7a5badf63fc0 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:52:01.008138 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f7997287-2367-480c-b2fc-7a5badf63fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:01.092191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997287-2367-480c-b2fc-7a5badf63fc0 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:52:01.095866 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a8ecd-e66c-4dde-9e6d-e639ec0e39c5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:52:01.096470 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a8ecd-e66c-4dde-9e6d-e639ec0e39c5 service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:01.096745 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a8ecd-e66c-4dde-9e6d-e639ec0e39c5 service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:01.097013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a8ecd-e66c-4dde-9e6d-e639ec0e39c5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:01.118175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:01.156045 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a8ecd-e66c-4dde-9e6d-e639ec0e39c5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:52:01.157440 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a8ecd-e66c-4dde-9e6d-e639ec0e39c5 service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:52:01.166289 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4330/8622] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:52:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:52:01.322395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749514f-cd34-4058-8cf3-4035515ff917 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:01.322775 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749514f-cd34-4058-8cf3-4035515ff917 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:01.323977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749514f-cd34-4058-8cf3-4035515ff917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:01.381441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-e749514f-cd34-4058-8cf3-4035515ff917 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:52:01.383277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e749514f-cd34-4058-8cf3-4035515ff917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:01.460843 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4293/8623] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:52:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:52:01.516640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7c7e-22b9-4a00-a018-f4c698b3343d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:01.516927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7c7e-22b9-4a00-a018-f4c698b3343d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:01.517233 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7c7e-22b9-4a00-a018-f4c698b3343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:01.569104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-93bc7c7e-22b9-4a00-a018-f4c698b3343d None None] MFA Rules not processed for user `abfea6b853134fed8ea5e32e5406bd5f`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:52:01.570331 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-93bc7c7e-22b9-4a00-a018-f4c698b3343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:01.649121 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4331/8624] 10.222.0.16 () {58 vars in 1040 bytes} [Sat Oct 15 21:52:01 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:52:01.656945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749514f-cd34-4058-8cf3-4035515ff917 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:52:01.670346 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749514f-cd34-4058-8cf3-4035515ff917 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:52:01.675388 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b628e-3a99-4baf-91d6-ea9145702cbb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:52:01.675983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b628e-3a99-4baf-91d6-ea9145702cbb service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:01.676255 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b628e-3a99-4baf-91d6-ea9145702cbb service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:01.676533 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b628e-3a99-4baf-91d6-ea9145702cbb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:01.685557 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b628e-3a99-4baf-91d6-ea9145702cbb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:52:01.687157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b628e-3a99-4baf-91d6-ea9145702cbb service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:52:01.695901 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4294/8625] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:52:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:52:10.585234 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ab32c-0db6-4df1-a9df-c735811ee612 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:10.585730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ab32c-0db6-4df1-a9df-c735811ee612 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:10.585924 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ab32c-0db6-4df1-a9df-c735811ee612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:10.604553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8e9ab32c-0db6-4df1-a9df-c735811ee612 None None] MFA Rules not processed for user `4768c0948f1b44689210529a25211aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:52:10.606362 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8e9ab32c-0db6-4df1-a9df-c735811ee612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:10.613333 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4332/8626] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:52:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:52:10.876408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b628e-3a99-4baf-91d6-ea9145702cbb service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:52:10.877893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-717b628e-3a99-4baf-91d6-ea9145702cbb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:10.934253 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b628e-3a99-4baf-91d6-ea9145702cbb service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:52:10.937841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc244d-f50a-4ff1-a9c7-5dcfdb81188f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:52:10.938430 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc244d-f50a-4ff1-a9c7-5dcfdb81188f service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:10.938704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc244d-f50a-4ff1-a9c7-5dcfdb81188f service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:10.938972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc244d-f50a-4ff1-a9c7-5dcfdb81188f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:10.940660 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-2fbc244d-f50a-4ff1-a9c7-5dcfdb81188f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:10.949847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc244d-f50a-4ff1-a9c7-5dcfdb81188f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:52:10.951265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc244d-f50a-4ff1-a9c7-5dcfdb81188f service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:52:10.959131 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4295/8627] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:52:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:52:12.442604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb6984-8350-4670-ac66-dc9d4d29d61d None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:12.443103 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb6984-8350-4670-ac66-dc9d4d29d61d None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:12.444030 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb6984-8350-4670-ac66-dc9d4d29d61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:12.467073 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-87eb6984-8350-4670-ac66-dc9d4d29d61d None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:52:12.468394 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-87eb6984-8350-4670-ac66-dc9d4d29d61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:12.476195 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4333/8628] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:52:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 21:52:12.489508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc244d-f50a-4ff1-a9c7-5dcfdb81188f service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:52:12.502743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc244d-f50a-4ff1-a9c7-5dcfdb81188f service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:52:12.507979 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb9848-8091-4651-8f97-ed10e4706608 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:52:12.508666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb9848-8091-4651-8f97-ed10e4706608 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:12.509004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb9848-8091-4651-8f97-ed10e4706608 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:12.509368 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb9848-8091-4651-8f97-ed10e4706608 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:12.518420 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb9848-8091-4651-8f97-ed10e4706608 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:52:12.520063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb9848-8091-4651-8f97-ed10e4706608 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:52:12.528772 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4296/8629] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:52:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:52:22.462359 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=67 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:22.525089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:22.734320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb6984-8350-4670-ac66-dc9d4d29d61d None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:52:22.743935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb6984-8350-4670-ac66-dc9d4d29d61d None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:52:22.749205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b20a2-a33e-458d-ae6a-6d81a8e4c0fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:52:22.749830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b20a2-a33e-458d-ae6a-6d81a8e4c0fb service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:22.750101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b20a2-a33e-458d-ae6a-6d81a8e4c0fb service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:22.750391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b20a2-a33e-458d-ae6a-6d81a8e4c0fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:22.752140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-616b20a2-a33e-458d-ae6a-6d81a8e4c0fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:22.762811 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b20a2-a33e-458d-ae6a-6d81a8e4c0fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:52:22.764242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b20a2-a33e-458d-ae6a-6d81a8e4c0fb service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:52:22.772214 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4334/8630] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:52:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:52:23.514307 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb9848-8091-4651-8f97-ed10e4706608 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:52:23.516335 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e3eb9848-8091-4651-8f97-ed10e4706608 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:23.593317 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb9848-8091-4651-8f97-ed10e4706608 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:52:23.599004 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a138130-0c6f-4af8-8b69-c865237cfa77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:52:23.599610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a138130-0c6f-4af8-8b69-c865237cfa77 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:23.599887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a138130-0c6f-4af8-8b69-c865237cfa77 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:23.600157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a138130-0c6f-4af8-8b69-c865237cfa77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:23.601916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1a138130-0c6f-4af8-8b69-c865237cfa77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:23.615122 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a138130-0c6f-4af8-8b69-c865237cfa77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:52:23.616725 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a138130-0c6f-4af8-8b69-c865237cfa77 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:52:23.627885 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4297/8631] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:52:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:52:30.327171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b20a2-a33e-458d-ae6a-6d81a8e4c0fb service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:52:30.330156 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-616b20a2-a33e-458d-ae6a-6d81a8e4c0fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:30.347662 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b20a2-a33e-458d-ae6a-6d81a8e4c0fb service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:52:30.353727 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938b971-abca-4c69-91c6-6c92781cdd0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:52:30.354752 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938b971-abca-4c69-91c6-6c92781cdd0b service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:30.355117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938b971-abca-4c69-91c6-6c92781cdd0b service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:30.355647 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938b971-abca-4c69-91c6-6c92781cdd0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:30.357469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3938b971-abca-4c69-91c6-6c92781cdd0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:52:30.375350 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938b971-abca-4c69-91c6-6c92781cdd0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:52:30.377001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938b971-abca-4c69-91c6-6c92781cdd0b service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:52:30.388657 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4335/8632] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:52:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:52:33.623051 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=34 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:39.706494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a138130-0c6f-4af8-8b69-c865237cfa77 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:52:39.719026 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a138130-0c6f-4af8-8b69-c865237cfa77 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:52:39.722568 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085d625-fc24-4c72-8a31-396ba52a5dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:52:39.723153 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085d625-fc24-4c72-8a31-396ba52a5dbd service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:52:39.723432 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085d625-fc24-4c72-8a31-396ba52a5dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:52:39.723702 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085d625-fc24-4c72-8a31-396ba52a5dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:52:39.754794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085d625-fc24-4c72-8a31-396ba52a5dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:52:39.756216 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085d625-fc24-4c72-8a31-396ba52a5dbd service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:52:39.766154 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4298/8633] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:52:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:52:40.383341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=12 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:49.761787 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=17 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:54:34.395683 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938b971-abca-4c69-91c6-6c92781cdd0b service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:54:34.397154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-3938b971-abca-4c69-91c6-6c92781cdd0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:54:34.516958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938b971-abca-4c69-91c6-6c92781cdd0b service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:54:34.522047 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f764c-d060-4ed3-9b16-363c24a32ce8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:54:34.522646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f764c-d060-4ed3-9b16-363c24a32ce8 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:54:34.522917 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f764c-d060-4ed3-9b16-363c24a32ce8 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:54:34.523183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f764c-d060-4ed3-9b16-363c24a32ce8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:54:34.524872 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-265f764c-d060-4ed3-9b16-363c24a32ce8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:54:34.536173 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f764c-d060-4ed3-9b16-363c24a32ce8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:54:34.537768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f764c-d060-4ed3-9b16-363c24a32ce8 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:54:34.547610 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4336/8634] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:54:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:54:43.688215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085d625-fc24-4c72-8a31-396ba52a5dbd service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:54:43.705886 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085d625-fc24-4c72-8a31-396ba52a5dbd service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:54:43.711100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b842c-1dcb-4679-8cdd-3c920d081962 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:54:43.711688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b842c-1dcb-4679-8cdd-3c920d081962 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:54:43.711961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b842c-1dcb-4679-8cdd-3c920d081962 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:54:43.712222 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b842c-1dcb-4679-8cdd-3c920d081962 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:54:43.713913 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-050b842c-1dcb-4679-8cdd-3c920d081962 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:54:43.725634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b842c-1dcb-4679-8cdd-3c920d081962 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:54:43.727271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b842c-1dcb-4679-8cdd-3c920d081962 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:54:43.737523 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4299/8635] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:54:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:54:44.543390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=54 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:54:53.732792 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:55:36.428309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f764c-d060-4ed3-9b16-363c24a32ce8 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:55:36.437670 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f764c-d060-4ed3-9b16-363c24a32ce8 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:55:36.441632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29692b3-6b1f-4830-905a-90cfdf0478bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:55:36.442229 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29692b3-6b1f-4830-905a-90cfdf0478bc service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:55:36.442490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29692b3-6b1f-4830-905a-90cfdf0478bc service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:55:36.442748 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29692b3-6b1f-4830-905a-90cfdf0478bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:55:36.444485 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a29692b3-6b1f-4830-905a-90cfdf0478bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:55:36.460290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29692b3-6b1f-4830-905a-90cfdf0478bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:55:36.461407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29692b3-6b1f-4830-905a-90cfdf0478bc service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:55:36.475185 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4337/8636] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:55:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:55:46.468433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:09.215813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b842c-1dcb-4679-8cdd-3c920d081962 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:09.228671 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b842c-1dcb-4679-8cdd-3c920d081962 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:09.234065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f36a1f-f048-4558-8fac-6d055b884427 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:09.234645 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36a1f-f048-4558-8fac-6d055b884427 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:09.234916 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36a1f-f048-4558-8fac-6d055b884427 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:09.235179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36a1f-f048-4558-8fac-6d055b884427 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:09.243745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f36a1f-f048-4558-8fac-6d055b884427 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:09.245339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f36a1f-f048-4558-8fac-6d055b884427 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:09.253967 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4300/8637] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:57:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:57:09.303908 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29692b3-6b1f-4830-905a-90cfdf0478bc service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:09.312646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29692b3-6b1f-4830-905a-90cfdf0478bc service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:09.316250 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0209ee4-59bb-411d-8192-476dffb68414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:09.316814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0209ee4-59bb-411d-8192-476dffb68414 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:09.317118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0209ee4-59bb-411d-8192-476dffb68414 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:09.318292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0209ee4-59bb-411d-8192-476dffb68414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:09.319900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a0209ee4-59bb-411d-8192-476dffb68414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:57:09.332716 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0209ee4-59bb-411d-8192-476dffb68414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:09.334264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0209ee4-59bb-411d-8192-476dffb68414 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:09.344847 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4338/8638] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:57:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:57:17.087552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cb03c-db88-4e8b-94ef-2a974bb4abb7 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:17.087984 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cb03c-db88-4e8b-94ef-2a974bb4abb7 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:17.088101 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cb03c-db88-4e8b-94ef-2a974bb4abb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:17.118939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-b64cb03c-db88-4e8b-94ef-2a974bb4abb7 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:57:17.120115 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-b64cb03c-db88-4e8b-94ef-2a974bb4abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:57:17.127531 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4301/8639] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:57:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:57:17.168049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0209ee4-59bb-411d-8192-476dffb68414 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:17.179622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0209ee4-59bb-411d-8192-476dffb68414 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:17.184546 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3615d-26b6-40a4-8a86-f6fa55d9522a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:17.185166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3615d-26b6-40a4-8a86-f6fa55d9522a service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:17.185441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3615d-26b6-40a4-8a86-f6fa55d9522a service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:17.185707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3615d-26b6-40a4-8a86-f6fa55d9522a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:17.194019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3615d-26b6-40a4-8a86-f6fa55d9522a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:17.195594 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3615d-26b6-40a4-8a86-f6fa55d9522a service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:17.204095 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4339/8640] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:57:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:57:17.692012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794a0f4-b119-498b-8add-43751cd92b01 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:17.692548 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794a0f4-b119-498b-8add-43751cd92b01 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:17.692839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794a0f4-b119-498b-8add-43751cd92b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:17.713073 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-7794a0f4-b119-498b-8add-43751cd92b01 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:57:17.714266 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7794a0f4-b119-498b-8add-43751cd92b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:57:17.721413 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4302/8641] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:57:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:57:17.734339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3615d-26b6-40a4-8a86-f6fa55d9522a service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:17.745245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3615d-26b6-40a4-8a86-f6fa55d9522a service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:17.750211 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bd02c-586a-4ddd-8e97-1fa06bafb819 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:17.750792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bd02c-586a-4ddd-8e97-1fa06bafb819 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:17.751065 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bd02c-586a-4ddd-8e97-1fa06bafb819 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:17.751332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bd02c-586a-4ddd-8e97-1fa06bafb819 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:17.759409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bd02c-586a-4ddd-8e97-1fa06bafb819 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:17.760960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bd02c-586a-4ddd-8e97-1fa06bafb819 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:17.769134 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4340/8642] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:57:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:57:26.321905 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680a86e-270c-4ffc-bbde-3246be15c25a None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:26.322479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680a86e-270c-4ffc-bbde-3246be15c25a None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:26.322479 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680a86e-270c-4ffc-bbde-3246be15c25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:26.343574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-f680a86e-270c-4ffc-bbde-3246be15c25a None None] MFA Rules not processed for user `4768c0948f1b44689210529a25211aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:57:26.344748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f680a86e-270c-4ffc-bbde-3246be15c25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:57:26.351818 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4303/8643] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:57:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:57:26.610454 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bd02c-586a-4ddd-8e97-1fa06bafb819 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:26.620208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bd02c-586a-4ddd-8e97-1fa06bafb819 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:26.623717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a76ea2-9604-4e46-b31d-9d23ab69689e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:26.624292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a76ea2-9604-4e46-b31d-9d23ab69689e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:26.624558 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a76ea2-9604-4e46-b31d-9d23ab69689e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:26.624817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a76ea2-9604-4e46-b31d-9d23ab69689e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:26.631849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a76ea2-9604-4e46-b31d-9d23ab69689e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:26.633268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a76ea2-9604-4e46-b31d-9d23ab69689e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:26.641108 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4341/8644] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:57:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:57:26.971996 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680a86e-270c-4ffc-bbde-3246be15c25a None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:26.984824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680a86e-270c-4ffc-bbde-3246be15c25a None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:26.990613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d5848-bd65-4512-9a95-277ed854cf83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:26.991217 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5848-bd65-4512-9a95-277ed854cf83 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:26.991492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5848-bd65-4512-9a95-277ed854cf83 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:26.991760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5848-bd65-4512-9a95-277ed854cf83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:26.993530 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a03d5848-bd65-4512-9a95-277ed854cf83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:57:27.019680 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d5848-bd65-4512-9a95-277ed854cf83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:27.021460 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d5848-bd65-4512-9a95-277ed854cf83 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:27.031438 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4304/8645] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:57:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:57:28.220094 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7aea04-a1f5-42e7-88d0-404051de8bd4 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:28.220570 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7aea04-a1f5-42e7-88d0-404051de8bd4 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:28.220890 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7aea04-a1f5-42e7-88d0-404051de8bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:28.243117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7d7aea04-a1f5-42e7-88d0-404051de8bd4 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:57:28.244367 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7d7aea04-a1f5-42e7-88d0-404051de8bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:57:28.251512 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4342/8646] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 21:57:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:57:28.264595 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d5848-bd65-4512-9a95-277ed854cf83 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:28.275785 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d5848-bd65-4512-9a95-277ed854cf83 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:28.280872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc928ce-6b51-4581-a182-efd67c1d21c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:28.281534 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc928ce-6b51-4581-a182-efd67c1d21c6 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:28.281794 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc928ce-6b51-4581-a182-efd67c1d21c6 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:28.282069 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc928ce-6b51-4581-a182-efd67c1d21c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:28.291734 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc928ce-6b51-4581-a182-efd67c1d21c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:28.293463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc928ce-6b51-4581-a182-efd67c1d21c6 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:28.302848 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4305/8647] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:57:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 21:57:37.184301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7aea04-a1f5-42e7-88d0-404051de8bd4 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:37.188816 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=28 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:37.195117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7aea04-a1f5-42e7-88d0-404051de8bd4 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:37.202515 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52e122-9908-49b1-a91f-ecfe9ec0576d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:37.203113 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52e122-9908-49b1-a91f-ecfe9ec0576d service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:37.203382 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52e122-9908-49b1-a91f-ecfe9ec0576d service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:37.203645 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52e122-9908-49b1-a91f-ecfe9ec0576d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:37.205372 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dd52e122-9908-49b1-a91f-ecfe9ec0576d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:57:37.228509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52e122-9908-49b1-a91f-ecfe9ec0576d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:37.229972 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52e122-9908-49b1-a91f-ecfe9ec0576d service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:37.237783 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4343/8648] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:57:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:57:38.268666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc928ce-6b51-4581-a182-efd67c1d21c6 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:38.274762 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=37 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:38.281504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc928ce-6b51-4581-a182-efd67c1d21c6 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:38.287474 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba079e0-ccc5-4022-9869-71c14217f895 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:38.288016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba079e0-ccc5-4022-9869-71c14217f895 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:38.288291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba079e0-ccc5-4022-9869-71c14217f895 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:38.288559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba079e0-ccc5-4022-9869-71c14217f895 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:38.297826 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba079e0-ccc5-4022-9869-71c14217f895 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:38.299753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba079e0-ccc5-4022-9869-71c14217f895 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:38.313030 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4306/8649] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:57:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:57:44.854597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52e122-9908-49b1-a91f-ecfe9ec0576d service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:44.869858 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52e122-9908-49b1-a91f-ecfe9ec0576d service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:44.875980 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079a0c-4ece-4cb9-9aac-9ae963075af4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:44.876608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079a0c-4ece-4cb9-9aac-9ae963075af4 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:44.877112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079a0c-4ece-4cb9-9aac-9ae963075af4 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:44.877592 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079a0c-4ece-4cb9-9aac-9ae963075af4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:44.891900 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d079a0c-4ece-4cb9-9aac-9ae963075af4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:44.893740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d079a0c-4ece-4cb9-9aac-9ae963075af4 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:44.906041 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4344/8650] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:57:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:57:48.307148 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:52.144339 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba079e0-ccc5-4022-9869-71c14217f895 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:57:52.153887 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba079e0-ccc5-4022-9869-71c14217f895 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:57:52.157496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9df9-f1bc-48dc-822c-5f9f700aa043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:57:52.158088 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9df9-f1bc-48dc-822c-5f9f700aa043 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:57:52.158366 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9df9-f1bc-48dc-822c-5f9f700aa043 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:57:52.158658 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9df9-f1bc-48dc-822c-5f9f700aa043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:57:52.167857 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f9df9-f1bc-48dc-822c-5f9f700aa043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:57:52.169301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f9df9-f1bc-48dc-822c-5f9f700aa043 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:57:52.179329 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4307/8651] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:57:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:57:54.901146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=16 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:58:02.174453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:17.111264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc37cc-c2cf-41cf-80de-8fce217dc2f9 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:17.111896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc37cc-c2cf-41cf-80de-8fce217dc2f9 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:17.111896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc37cc-c2cf-41cf-80de-8fce217dc2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:17.130818 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-70cc37cc-c2cf-41cf-80de-8fce217dc2f9 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:59:17.132013 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-70cc37cc-c2cf-41cf-80de-8fce217dc2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:17.146967 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4345/8652] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 21:59:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:59:17.190731 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9df9-f1bc-48dc-822c-5f9f700aa043 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:17.206833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9df9-f1bc-48dc-822c-5f9f700aa043 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:17.211816 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d86395-c15f-4808-9bb3-497da732045d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:17.212403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d86395-c15f-4808-9bb3-497da732045d service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:17.212675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d86395-c15f-4808-9bb3-497da732045d service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:17.212937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d86395-c15f-4808-9bb3-497da732045d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:17.221267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d86395-c15f-4808-9bb3-497da732045d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:17.222828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d86395-c15f-4808-9bb3-497da732045d service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:17.230847 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4308/8653] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 21:59:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 21:59:17.576607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc37cc-c2cf-41cf-80de-8fce217dc2f9 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:17.586629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc37cc-c2cf-41cf-80de-8fce217dc2f9 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:17.590365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba465b1e-9e1b-4de7-9ad1-f4116d82335c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:17.590956 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba465b1e-9e1b-4de7-9ad1-f4116d82335c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:17.591226 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba465b1e-9e1b-4de7-9ad1-f4116d82335c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:17.591488 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba465b1e-9e1b-4de7-9ad1-f4116d82335c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:17.650907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba465b1e-9e1b-4de7-9ad1-f4116d82335c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:17.652318 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba465b1e-9e1b-4de7-9ad1-f4116d82335c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:17.662295 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4346/8654] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:59:20.901356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d86395-c15f-4808-9bb3-497da732045d service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:20.903627 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-76d86395-c15f-4808-9bb3-497da732045d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:20.918563 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d86395-c15f-4808-9bb3-497da732045d service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:20.924206 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c7345-64ec-4299-b826-434a17f405c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:20.924899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c7345-64ec-4299-b826-434a17f405c8 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:20.925297 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c7345-64ec-4299-b826-434a17f405c8 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:20.925643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c7345-64ec-4299-b826-434a17f405c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:20.927409 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-df0c7345-64ec-4299-b826-434a17f405c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:20.939207 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c7345-64ec-4299-b826-434a17f405c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:20.941606 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c7345-64ec-4299-b826-434a17f405c8 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:20.952544 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4309/8655] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:59:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:59:23.891608 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba465b1e-9e1b-4de7-9ad1-f4116d82335c service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:23.912095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba465b1e-9e1b-4de7-9ad1-f4116d82335c service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:23.918399 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fbb33-d637-469a-abe4-f58a9769d2e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:23.919301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fbb33-d637-469a-abe4-f58a9769d2e3 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:23.919651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fbb33-d637-469a-abe4-f58a9769d2e3 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:23.920019 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fbb33-d637-469a-abe4-f58a9769d2e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:23.922481 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-969fbb33-d637-469a-abe4-f58a9769d2e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:23.936328 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fbb33-d637-469a-abe4-f58a9769d2e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:23.938105 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fbb33-d637-469a-abe4-f58a9769d2e3 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:23.950780 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4347/8656] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:59:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:59:23.978232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c7345-64ec-4299-b826-434a17f405c8 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:23.990191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c7345-64ec-4299-b826-434a17f405c8 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:23.995273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:23.996103 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:23.996456 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:23.996739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] PATH_INFO: `/v3/users/4768c0948f1b44689210529a25211aa7` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:23.997879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4768c0948f1b44689210529a25211aa7)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:23.999880 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:24.049349 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4768c0948f1b44689210529a25211aa7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.062485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4768c0948f1b44689210529a25211aa7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.070356 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4768c0948f1b44689210529a25211aa7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.077596 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4768c0948f1b44689210529a25211aa7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.078330 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4768c0948f1b44689210529a25211aa7'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.084359 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4310/8657] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:59:23 2022] DELETE /identity/v3/users/4768c0948f1b44689210529a25211aa7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 21:59:24.264804 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fbb33-d637-469a-abe4-f58a9769d2e3 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:24.278459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fbb33-d637-469a-abe4-f58a9769d2e3 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:24.336349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:24.336968 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:24.337270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:24.337536 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] PATH_INFO: `/v3/projects/cd7071741b2e4f35954da54683a2b55c` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:24.338621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7071741b2e4f35954da54683a2b55c)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:24.340528 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:24.385622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7071741b2e4f35954da54683a2b55c', 'request_id': 'req-1075334d-64e6-4551-b693-196795cb9e3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.392661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7071741b2e4f35954da54683a2b55c', 'request_id': 'req-1075334d-64e6-4551-b693-196795cb9e3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.399155 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7071741b2e4f35954da54683a2b55c', 'request_id': 'req-1075334d-64e6-4551-b693-196795cb9e3f'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.405514 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:24.410661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] The token cache is being invalidate because project cd7071741b2e4f35954da54683a2b55c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:59:24.412309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.413095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.418050 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4348/8658] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:59:24 2022] DELETE /identity/v3/projects/cd7071741b2e4f35954da54683a2b55c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:59:24.419227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:24.420768 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:24.435626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c017ea-d36e-41df-a77e-86ba33ccfa51 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:24.491896 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:24.492438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:24.492711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:24.492977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] PATH_INFO: `/v3/users/54824064641441e189d7c735f010e84b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:24.494118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54824064641441e189d7c735f010e84b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:24.496030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:24.541827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54824064641441e189d7c735f010e84b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.542954 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:24.557294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54824064641441e189d7c735f010e84b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.564919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54824064641441e189d7c735f010e84b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.572577 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54824064641441e189d7c735f010e84b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.573243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-c0a86d54-69ea-4b63-b1fb-dc2a456d7e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54824064641441e189d7c735f010e84b'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.579212 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4311/8659] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 21:59:24 2022] DELETE /identity/v3/users/54824064641441e189d7c735f010e84b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:59:24.758814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:24.760324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:24.776960 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075334d-64e6-4551-b693-196795cb9e3f admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:24.836518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:24.837171 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:24.837449 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:24.837719 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] PATH_INFO: `/v3/projects/245363bb1f1744b69f35a92ae536c73f` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:24.838819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=245363bb1f1744b69f35a92ae536c73f)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:24.840729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:24.886088 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '245363bb1f1744b69f35a92ae536c73f', 'request_id': 'req-a1c33852-eafa-4018-8009-6c33b94c4d83'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.894086 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '245363bb1f1744b69f35a92ae536c73f', 'request_id': 'req-a1c33852-eafa-4018-8009-6c33b94c4d83'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.900555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '245363bb1f1744b69f35a92ae536c73f', 'request_id': 'req-a1c33852-eafa-4018-8009-6c33b94c4d83'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.911015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] The token cache is being invalidate because project 245363bb1f1744b69f35a92ae536c73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 21:59:24.911297 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.911939 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-a1c33852-eafa-4018-8009-6c33b94c4d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 21:59:24.913458 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4349/8660] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:59:24 2022] DELETE /identity/v3/projects/245363bb1f1744b69f35a92ae536c73f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:59:24.918602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008d8aa-84c0-4e95-b550-0409f408b301 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:24.918889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008d8aa-84c0-4e95-b550-0409f408b301 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:24.919159 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008d8aa-84c0-4e95-b550-0409f408b301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:24.949183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d008d8aa-84c0-4e95-b550-0409f408b301 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:59:24.950392 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d008d8aa-84c0-4e95-b550-0409f408b301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:25.009476 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4312/8661] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:59:25.017724 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e85fd-da89-4a8b-86f8-11acb34ee614 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.018007 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e85fd-da89-4a8b-86f8-11acb34ee614 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.018277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e85fd-da89-4a8b-86f8-11acb34ee614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.039437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-0f4e85fd-da89-4a8b-86f8-11acb34ee614 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:59:25.041004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0f4e85fd-da89-4a8b-86f8-11acb34ee614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:25.049487 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4350/8662] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:59:25.054075 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008d8aa-84c0-4e95-b550-0409f408b301 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.063992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008d8aa-84c0-4e95-b550-0409f408b301 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.068715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce415e0-8318-4bf8-b596-798d6cf6d8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.069336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce415e0-8318-4bf8-b596-798d6cf6d8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.069615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce415e0-8318-4bf8-b596-798d6cf6d8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.069884 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce415e0-8318-4bf8-b596-798d6cf6d8d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.070521 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce415e0-8318-4bf8-b596-798d6cf6d8d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 21:59:25.070821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce415e0-8318-4bf8-b596-798d6cf6d8d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.072435 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce415e0-8318-4bf8-b596-798d6cf6d8d5 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.084517 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4313/8663] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 21:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 21:59:25.088824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e85fd-da89-4a8b-86f8-11acb34ee614 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.098869 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e85fd-da89-4a8b-86f8-11acb34ee614 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.103434 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37f673-7ef5-4074-8e40-db9fb29bb9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.104015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37f673-7ef5-4074-8e40-db9fb29bb9ba admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.104289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37f673-7ef5-4074-8e40-db9fb29bb9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.104553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37f673-7ef5-4074-8e40-db9fb29bb9ba admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.105199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37f673-7ef5-4074-8e40-db9fb29bb9ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.107061 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37f673-7ef5-4074-8e40-db9fb29bb9ba admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.134349 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4351/8664] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:59:25 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 21:59:25.138501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce415e0-8318-4bf8-b596-798d6cf6d8d5 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.149343 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce415e0-8318-4bf8-b596-798d6cf6d8d5 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.208673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693443a-d11e-4528-99c1-bb65feda9af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.209302 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693443a-d11e-4528-99c1-bb65feda9af1 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.209574 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693443a-d11e-4528-99c1-bb65feda9af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.209839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693443a-d11e-4528-99c1-bb65feda9af1 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.210461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693443a-d11e-4528-99c1-bb65feda9af1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.212399 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693443a-d11e-4528-99c1-bb65feda9af1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.233647 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4314/8665] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:59:25 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:59:25.238508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37f673-7ef5-4074-8e40-db9fb29bb9ba admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.248607 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37f673-7ef5-4074-8e40-db9fb29bb9ba admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.253791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1459d-2f13-4efa-9d39-0770357974ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.254366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1459d-2f13-4efa-9d39-0770357974ec admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.254642 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1459d-2f13-4efa-9d39-0770357974ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.254903 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1459d-2f13-4efa-9d39-0770357974ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.255520 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1459d-2f13-4efa-9d39-0770357974ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.257098 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1459d-2f13-4efa-9d39-0770357974ec admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.267243 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4352/8666] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:59:25.271590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693443a-d11e-4528-99c1-bb65feda9af1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.281935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693443a-d11e-4528-99c1-bb65feda9af1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.287497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2da44e-20f0-42ab-bf63-8bb819805a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.288093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2da44e-20f0-42ab-bf63-8bb819805a6b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.288372 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2da44e-20f0-42ab-bf63-8bb819805a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.288642 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2da44e-20f0-42ab-bf63-8bb819805a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.289312 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2da44e-20f0-42ab-bf63-8bb819805a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.290944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2da44e-20f0-42ab-bf63-8bb819805a6b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.301771 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4315/8667] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:59:25.306118 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1459d-2f13-4efa-9d39-0770357974ec admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.316167 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1459d-2f13-4efa-9d39-0770357974ec admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.320775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d74800-1dd1-416a-b20a-db767592665e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.321451 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d74800-1dd1-416a-b20a-db767592665e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.321730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d74800-1dd1-416a-b20a-db767592665e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.322398 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d74800-1dd1-416a-b20a-db767592665e admin admin] PATH_INFO: `/v3/projects/ea5345e230164cd6ae5b1195425e7ff0/users/823ad7d2979042c188525f5341febdff/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.337796 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d74800-1dd1-416a-b20a-db767592665e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea5345e230164cd6ae5b1195425e7ff0, user_id=823ad7d2979042c188525f5341febdff, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.340397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d74800-1dd1-416a-b20a-db767592665e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.354184 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4353/8668] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:59:25 2022] PUT /identity/v3/projects/ea5345e230164cd6ae5b1195425e7ff0/users/823ad7d2979042c188525f5341febdff/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:59:25.365611 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588c1f-2699-4088-846d-0e63c0ea9a27 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.365895 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588c1f-2699-4088-846d-0e63c0ea9a27 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.366167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588c1f-2699-4088-846d-0e63c0ea9a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.394965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-ac588c1f-2699-4088-846d-0e63c0ea9a27 None None] MFA Rules not processed for user `823ad7d2979042c188525f5341febdff`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:59:25.396157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ac588c1f-2699-4088-846d-0e63c0ea9a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:25.405551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:25.464585 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4316/8669] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:59:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:59:25.469116 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d74800-1dd1-416a-b20a-db767592665e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.495783 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:25.532928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d74800-1dd1-416a-b20a-db767592665e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.538411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68874c-e856-4211-aa12-a5e673a86408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.539005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68874c-e856-4211-aa12-a5e673a86408 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.539282 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68874c-e856-4211-aa12-a5e673a86408 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.539550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68874c-e856-4211-aa12-a5e673a86408 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.540178 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68874c-e856-4211-aa12-a5e673a86408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.542099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68874c-e856-4211-aa12-a5e673a86408 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.568738 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4354/8670] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 21:59:25 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:59:25.572638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588c1f-2699-4088-846d-0e63c0ea9a27 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.583383 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588c1f-2699-4088-846d-0e63c0ea9a27 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.638966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17d95f-2d88-4701-a1e5-e48c7e2e054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.639554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17d95f-2d88-4701-a1e5-e48c7e2e054f admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.639824 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17d95f-2d88-4701-a1e5-e48c7e2e054f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.640089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17d95f-2d88-4701-a1e5-e48c7e2e054f admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.640704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17d95f-2d88-4701-a1e5-e48c7e2e054f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.642662 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17d95f-2d88-4701-a1e5-e48c7e2e054f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.667333 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4317/8671] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 21:59:25 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 21:59:25.668499 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68874c-e856-4211-aa12-a5e673a86408 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.679572 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68874c-e856-4211-aa12-a5e673a86408 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.684169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed66a0a-a06d-466b-af16-837a572df552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.684757 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed66a0a-a06d-466b-af16-837a572df552 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.685029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed66a0a-a06d-466b-af16-837a572df552 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.685321 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed66a0a-a06d-466b-af16-837a572df552 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.685942 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed66a0a-a06d-466b-af16-837a572df552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.687512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed66a0a-a06d-466b-af16-837a572df552 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.698134 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4355/8672] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:59:25.702214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17d95f-2d88-4701-a1e5-e48c7e2e054f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.712552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17d95f-2d88-4701-a1e5-e48c7e2e054f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.717276 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68671647-d64d-478e-b08d-f8a3fe27b48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.717941 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68671647-d64d-478e-b08d-f8a3fe27b48b admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.718221 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68671647-d64d-478e-b08d-f8a3fe27b48b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.718487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68671647-d64d-478e-b08d-f8a3fe27b48b admin admin] PATH_INFO: `/v3/projects/6e84fcbf6bce498d87327f7520eec5d7/users/214188614819400283651340d2befb35/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.735024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68671647-d64d-478e-b08d-f8a3fe27b48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e84fcbf6bce498d87327f7520eec5d7, user_id=214188614819400283651340d2befb35, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.738326 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68671647-d64d-478e-b08d-f8a3fe27b48b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.750570 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4318/8673] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:59:25 2022] PUT /identity/v3/projects/6e84fcbf6bce498d87327f7520eec5d7/users/214188614819400283651340d2befb35/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:59:25.754672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed66a0a-a06d-466b-af16-837a572df552 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.764674 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed66a0a-a06d-466b-af16-837a572df552 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.823298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a739f4-6638-48bd-aa1b-debfd0e8c1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.823895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a739f4-6638-48bd-aa1b-debfd0e8c1ef admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.824169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a739f4-6638-48bd-aa1b-debfd0e8c1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.824463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a739f4-6638-48bd-aa1b-debfd0e8c1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.825130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a739f4-6638-48bd-aa1b-debfd0e8c1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.826707 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a739f4-6638-48bd-aa1b-debfd0e8c1ef admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.837390 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4356/8674] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 21:59:25.841840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68671647-d64d-478e-b08d-f8a3fe27b48b admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.852837 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68671647-d64d-478e-b08d-f8a3fe27b48b admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:25.857459 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6acb25-7aee-4d3f-abc7-ed710cf6d2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:25.858099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6acb25-7aee-4d3f-abc7-ed710cf6d2cd admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.858369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6acb25-7aee-4d3f-abc7-ed710cf6d2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.858636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6acb25-7aee-4d3f-abc7-ed710cf6d2cd admin admin] PATH_INFO: `/v3/projects/6e84fcbf6bce498d87327f7520eec5d7/users/214188614819400283651340d2befb35/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.860520 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6acb25-7aee-4d3f-abc7-ed710cf6d2cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e84fcbf6bce498d87327f7520eec5d7, user_id=214188614819400283651340d2befb35, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:25.863280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6acb25-7aee-4d3f-abc7-ed710cf6d2cd admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:25.873368 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4319/8675] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 21:59:25 2022] PUT /identity/v3/projects/6e84fcbf6bce498d87327f7520eec5d7/users/214188614819400283651340d2befb35/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 21:59:25.883185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ca37f-0372-4624-8fe8-0f732bb077bb None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:25.883452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ca37f-0372-4624-8fe8-0f732bb077bb None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:25.883720 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ca37f-0372-4624-8fe8-0f732bb077bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:25.913352 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a13ca37f-0372-4624-8fe8-0f732bb077bb None None] MFA Rules not processed for user `214188614819400283651340d2befb35`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:59:25.914554 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a13ca37f-0372-4624-8fe8-0f732bb077bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:25.982519 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4357/8676] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:59:25 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:59:25.997675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6acb25-7aee-4d3f-abc7-ed710cf6d2cd admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:25.999449 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bf6acb25-7aee-4d3f-abc7-ed710cf6d2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:26.068963 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:26.076189 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6acb25-7aee-4d3f-abc7-ed710cf6d2cd admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:26.082719 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44a51b-601a-4972-9f75-35bed3b2c6c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:26.083315 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44a51b-601a-4972-9f75-35bed3b2c6c4 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:26.083594 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44a51b-601a-4972-9f75-35bed3b2c6c4 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:26.083865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44a51b-601a-4972-9f75-35bed3b2c6c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:26.140740 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44a51b-601a-4972-9f75-35bed3b2c6c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:26.142407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44a51b-601a-4972-9f75-35bed3b2c6c4 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:26.152502 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4320/8677] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:59:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:59:29.132899 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ca37f-0372-4624-8fe8-0f732bb077bb None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:29.134391 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a13ca37f-0372-4624-8fe8-0f732bb077bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:29.158833 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:29.201322 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ca37f-0372-4624-8fe8-0f732bb077bb None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:29.205504 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5848aff-e649-47e0-816e-6c88d6981458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:29.206667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5848aff-e649-47e0-816e-6c88d6981458 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:29.206667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5848aff-e649-47e0-816e-6c88d6981458 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:29.207413 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5848aff-e649-47e0-816e-6c88d6981458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:29.216893 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5848aff-e649-47e0-816e-6c88d6981458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:29.218405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5848aff-e649-47e0-816e-6c88d6981458 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:29.228543 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4358/8678] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:59:29 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:59:32.147977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c173a74-b395-4640-b66b-338c4ded8b44 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:32.148429 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c173a74-b395-4640-b66b-338c4ded8b44 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:32.148646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c173a74-b395-4640-b66b-338c4ded8b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:32.169408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-3c173a74-b395-4640-b66b-338c4ded8b44 None None] MFA Rules not processed for user `823ad7d2979042c188525f5341febdff`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 21:59:32.170753 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3c173a74-b395-4640-b66b-338c4ded8b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:32.179357 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4321/8679] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 21:59:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 21:59:32.926203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5848aff-e649-47e0-816e-6c88d6981458 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:32.927839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b5848aff-e649-47e0-816e-6c88d6981458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:33.006472 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5848aff-e649-47e0-816e-6c88d6981458 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:33.012077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ca15d-c8f9-41af-9c9b-d2950803b226 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:33.012623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca15d-c8f9-41af-9c9b-d2950803b226 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:33.012897 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca15d-c8f9-41af-9c9b-d2950803b226 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:33.013199 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca15d-c8f9-41af-9c9b-d2950803b226 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:33.015286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d69ca15d-c8f9-41af-9c9b-d2950803b226 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:33.029025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ca15d-c8f9-41af-9c9b-d2950803b226 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:33.030663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ca15d-c8f9-41af-9c9b-d2950803b226 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:33.043138 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4359/8680] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:59:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 21:59:34.901745 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c173a74-b395-4640-b66b-338c4ded8b44 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:34.912795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c173a74-b395-4640-b66b-338c4ded8b44 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:34.916545 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550f6dd-3ce4-48c4-ad0e-8471a6064804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:34.917202 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550f6dd-3ce4-48c4-ad0e-8471a6064804 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:34.917485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550f6dd-3ce4-48c4-ad0e-8471a6064804 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:34.917759 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550f6dd-3ce4-48c4-ad0e-8471a6064804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:34.919519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-5550f6dd-3ce4-48c4-ad0e-8471a6064804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:34.930749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550f6dd-3ce4-48c4-ad0e-8471a6064804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:34.932259 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550f6dd-3ce4-48c4-ad0e-8471a6064804 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:34.945474 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4322/8681] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:59:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:59:42.399655 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ca15d-c8f9-41af-9c9b-d2950803b226 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:42.410001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ca15d-c8f9-41af-9c9b-d2950803b226 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:42.415235 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c710f9-80da-46f6-89e6-561b344f455b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:42.415824 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c710f9-80da-46f6-89e6-561b344f455b service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:42.416101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c710f9-80da-46f6-89e6-561b344f455b service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:42.416363 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c710f9-80da-46f6-89e6-561b344f455b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:42.418272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a9c710f9-80da-46f6-89e6-561b344f455b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:42.430622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c710f9-80da-46f6-89e6-561b344f455b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:42.432259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c710f9-80da-46f6-89e6-561b344f455b service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:42.443155 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4360/8682] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:59:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:59:44.938773 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=36 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:49.692197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550f6dd-3ce4-48c4-ad0e-8471a6064804 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 21:59:49.704107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550f6dd-3ce4-48c4-ad0e-8471a6064804 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 21:59:49.709179 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27187b8c-c75e-4dec-90a1-45d4ed5c35af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 21:59:49.709764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27187b8c-c75e-4dec-90a1-45d4ed5c35af service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 21:59:49.710039 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27187b8c-c75e-4dec-90a1-45d4ed5c35af service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 21:59:49.710298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27187b8c-c75e-4dec-90a1-45d4ed5c35af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 21:59:49.711972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-27187b8c-c75e-4dec-90a1-45d4ed5c35af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 21:59:49.723853 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27187b8c-c75e-4dec-90a1-45d4ed5c35af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 21:59:49.725428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27187b8c-c75e-4dec-90a1-45d4ed5c35af service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 21:59:49.735447 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4323/8683] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 21:59:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 21:59:52.438095 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=55 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:59.731286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:00:46.840360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c710f9-80da-46f6-89e6-561b344f455b service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:00:46.861230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c710f9-80da-46f6-89e6-561b344f455b service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:00:46.864943 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0e535-0ea3-474f-9b17-b58b0f8416a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:00:46.865569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0e535-0ea3-474f-9b17-b58b0f8416a4 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:00:46.865838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0e535-0ea3-474f-9b17-b58b0f8416a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:00:46.866102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0e535-0ea3-474f-9b17-b58b0f8416a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:00:46.868004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-78d0e535-0ea3-474f-9b17-b58b0f8416a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:00:46.887299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0e535-0ea3-474f-9b17-b58b0f8416a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:00:46.889285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0e535-0ea3-474f-9b17-b58b0f8416a4 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:00:46.905318 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4361/8684] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:00:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:00:56.898132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=10 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:04.629653 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cd1ce-660c-4fb7-91cc-141fc96999d6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:01:04.630619 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cd1ce-660c-4fb7-91cc-141fc96999d6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:01:04.630897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cd1ce-660c-4fb7-91cc-141fc96999d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:01:04.652581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d08cd1ce-660c-4fb7-91cc-141fc96999d6 None None] MFA Rules not processed for user `823ad7d2979042c188525f5341febdff`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:01:04.654100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d08cd1ce-660c-4fb7-91cc-141fc96999d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:01:04.662808 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4324/8685] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:01:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:01:05.004404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0e535-0ea3-474f-9b17-b58b0f8416a4 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:01:05.016267 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0e535-0ea3-474f-9b17-b58b0f8416a4 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:01:05.021251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacff5b3-0e69-4283-a1d4-afa6ec23b88c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:01:05.022072 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacff5b3-0e69-4283-a1d4-afa6ec23b88c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:01:05.022555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacff5b3-0e69-4283-a1d4-afa6ec23b88c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:01:05.023003 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacff5b3-0e69-4283-a1d4-afa6ec23b88c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:01:05.025615 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bacff5b3-0e69-4283-a1d4-afa6ec23b88c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:01:05.039552 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacff5b3-0e69-4283-a1d4-afa6ec23b88c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:01:05.041354 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacff5b3-0e69-4283-a1d4-afa6ec23b88c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:01:05.052587 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4362/8686] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:01:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:01:14.647072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:15.046696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:19.300906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0ec18-bfe3-4de2-ad05-6cc2bb1895b9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:01:19.301486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0ec18-bfe3-4de2-ad05-6cc2bb1895b9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:01:19.301779 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0ec18-bfe3-4de2-ad05-6cc2bb1895b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:01:19.357336 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-80e0ec18-bfe3-4de2-ad05-6cc2bb1895b9 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:01:19.359158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-80e0ec18-bfe3-4de2-ad05-6cc2bb1895b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:01:19.383343 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4325/8687] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 22:01:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:01:19.396780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacff5b3-0e69-4283-a1d4-afa6ec23b88c service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:01:19.398292 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bacff5b3-0e69-4283-a1d4-afa6ec23b88c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:01:19.415110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacff5b3-0e69-4283-a1d4-afa6ec23b88c service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:01:19.422650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3462bc9-6fba-4ec8-9bd2-62b6c09ea4fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:01:19.423257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3462bc9-6fba-4ec8-9bd2-62b6c09ea4fc service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:01:19.423729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3462bc9-6fba-4ec8-9bd2-62b6c09ea4fc service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:01:19.424001 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3462bc9-6fba-4ec8-9bd2-62b6c09ea4fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:01:19.433896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3462bc9-6fba-4ec8-9bd2-62b6c09ea4fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:01:19.435519 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3462bc9-6fba-4ec8-9bd2-62b6c09ea4fc service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:01:19.447620 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4363/8688] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:01:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:01:29.379032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=18 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:29.442247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:02:23.016724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0ec18-bfe3-4de2-ad05-6cc2bb1895b9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:02:23.026157 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0ec18-bfe3-4de2-ad05-6cc2bb1895b9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:02:23.029978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67352a34-fbdc-49bd-b2cd-4ea12e484075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:02:23.030561 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352a34-fbdc-49bd-b2cd-4ea12e484075 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:02:23.030834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352a34-fbdc-49bd-b2cd-4ea12e484075 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:02:23.031099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352a34-fbdc-49bd-b2cd-4ea12e484075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:02:23.032793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-67352a34-fbdc-49bd-b2cd-4ea12e484075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:02:23.046036 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67352a34-fbdc-49bd-b2cd-4ea12e484075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:02:23.047452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67352a34-fbdc-49bd-b2cd-4ea12e484075 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:02:23.058885 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4326/8689] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:02:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:02:33.053440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:23.320230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3462bc9-6fba-4ec8-9bd2-62b6c09ea4fc service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:03:23.333247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3462bc9-6fba-4ec8-9bd2-62b6c09ea4fc service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:03:23.339258 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6132da9-3cd6-42a8-b513-3303f7602d6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:03:23.339863 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6132da9-3cd6-42a8-b513-3303f7602d6d service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:03:23.340142 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6132da9-3cd6-42a8-b513-3303f7602d6d service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:03:23.340459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6132da9-3cd6-42a8-b513-3303f7602d6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:03:23.342366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a6132da9-3cd6-42a8-b513-3303f7602d6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:03:23.355658 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6132da9-3cd6-42a8-b513-3303f7602d6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:03:23.357946 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6132da9-3cd6-42a8-b513-3303f7602d6d service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:03:23.373209 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4364/8690] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:03:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:03:33.364508 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:35.655580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60712bfc-5a1b-4368-af91-8532472c1bb6 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:03:35.656054 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60712bfc-5a1b-4368-af91-8532472c1bb6 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:03:35.656158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60712bfc-5a1b-4368-af91-8532472c1bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:03:35.681340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-60712bfc-5a1b-4368-af91-8532472c1bb6 None None] MFA Rules not processed for user `823ad7d2979042c188525f5341febdff`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:03:35.682565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-60712bfc-5a1b-4368-af91-8532472c1bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:03:35.696720 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4327/8691] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:03:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:03:36.068410 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6132da9-3cd6-42a8-b513-3303f7602d6d service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:03:36.079721 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6132da9-3cd6-42a8-b513-3303f7602d6d service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:03:36.083828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8e21c-ac49-4110-811a-a16e643037df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:03:36.084463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8e21c-ac49-4110-811a-a16e643037df service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:03:36.084941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8e21c-ac49-4110-811a-a16e643037df service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:03:36.085263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8e21c-ac49-4110-811a-a16e643037df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:03:36.087311 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80b8e21c-ac49-4110-811a-a16e643037df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:03:36.098442 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8e21c-ac49-4110-811a-a16e643037df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:03:36.099786 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8e21c-ac49-4110-811a-a16e643037df service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:03:36.108586 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4365/8692] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:03:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:03:45.675575 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:45.676990 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60712bfc-5a1b-4368-af91-8532472c1bb6 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:03:45.691083 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60712bfc-5a1b-4368-af91-8532472c1bb6 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:03:45.696454 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9efcbd-293f-4f58-b32f-003249bb196f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:03:45.697076 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9efcbd-293f-4f58-b32f-003249bb196f service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:03:45.697348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9efcbd-293f-4f58-b32f-003249bb196f service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:03:45.697615 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9efcbd-293f-4f58-b32f-003249bb196f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:03:45.699334 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ca9efcbd-293f-4f58-b32f-003249bb196f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:03:45.711324 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9efcbd-293f-4f58-b32f-003249bb196f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:03:45.712932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9efcbd-293f-4f58-b32f-003249bb196f service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:03:45.723908 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4328/8693] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:03:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:03:46.104293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:55.719504 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:04:45.426465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8e21c-ac49-4110-811a-a16e643037df service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:04:45.427888 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-80b8e21c-ac49-4110-811a-a16e643037df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:04:45.501868 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8e21c-ac49-4110-811a-a16e643037df service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:04:45.506905 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003d7a5-2d82-499d-a5df-4bca7b8bcb74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:04:45.507494 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d7a5-2d82-499d-a5df-4bca7b8bcb74 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:04:45.507765 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d7a5-2d82-499d-a5df-4bca7b8bcb74 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:04:45.508025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d7a5-2d82-499d-a5df-4bca7b8bcb74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:04:45.516369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003d7a5-2d82-499d-a5df-4bca7b8bcb74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:04:45.517966 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003d7a5-2d82-499d-a5df-4bca7b8bcb74 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:04:45.528003 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4366/8694] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:04:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:04:52.074440 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9efcbd-293f-4f58-b32f-003249bb196f service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:04:52.124050 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9efcbd-293f-4f58-b32f-003249bb196f service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:04:52.130626 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32d880-75fd-4e97-b58c-bddb4d37a1a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:04:52.131224 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32d880-75fd-4e97-b58c-bddb4d37a1a6 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:04:52.131503 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32d880-75fd-4e97-b58c-bddb4d37a1a6 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:04:52.131774 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32d880-75fd-4e97-b58c-bddb4d37a1a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:04:52.140384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32d880-75fd-4e97-b58c-bddb4d37a1a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:04:52.142841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32d880-75fd-4e97-b58c-bddb4d37a1a6 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:04:52.152711 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4329/8695] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:04:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:04:52.352923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003d7a5-2d82-499d-a5df-4bca7b8bcb74 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:04:52.354360 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d003d7a5-2d82-499d-a5df-4bca7b8bcb74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:04:52.371979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003d7a5-2d82-499d-a5df-4bca7b8bcb74 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:04:52.382153 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3e039-aa80-44ff-bcff-fa4179235c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:04:52.382826 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3e039-aa80-44ff-bcff-fa4179235c1c service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:04:52.383102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3e039-aa80-44ff-bcff-fa4179235c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:04:52.383369 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3e039-aa80-44ff-bcff-fa4179235c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:04:52.385324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-51f3e039-aa80-44ff-bcff-fa4179235c1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:04:52.397179 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3e039-aa80-44ff-bcff-fa4179235c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:04:52.398620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3e039-aa80-44ff-bcff-fa4179235c1c service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:04:52.407998 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4367/8696] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:04:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:04:52.504304 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32d880-75fd-4e97-b58c-bddb4d37a1a6 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:04:52.515269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32d880-75fd-4e97-b58c-bddb4d37a1a6 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:04:52.520777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b940ab-1929-4b73-a270-a6e221652ef3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:04:52.521415 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b940ab-1929-4b73-a270-a6e221652ef3 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:04:52.521687 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b940ab-1929-4b73-a270-a6e221652ef3 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:04:52.521953 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b940ab-1929-4b73-a270-a6e221652ef3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:04:52.523659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f7b940ab-1929-4b73-a270-a6e221652ef3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:04:52.535514 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b940ab-1929-4b73-a270-a6e221652ef3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:04:52.537100 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b940ab-1929-4b73-a270-a6e221652ef3 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:04:52.546852 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4330/8697] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:04:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:04:52.720650 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3e039-aa80-44ff-bcff-fa4179235c1c service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:04:52.724085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b940ab-1929-4b73-a270-a6e221652ef3 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:04:52.732710 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3e039-aa80-44ff-bcff-fa4179235c1c service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:04:52.733494 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b940ab-1929-4b73-a270-a6e221652ef3 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:04:52.737699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789a129-6779-4ac7-8e95-6e39152330a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:04:52.738285 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789a129-6779-4ac7-8e95-6e39152330a3 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:04:52.738490 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e9b0b-2e76-44c7-83ba-b5ed40632baa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:04:52.738630 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789a129-6779-4ac7-8e95-6e39152330a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:04:52.738813 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789a129-6779-4ac7-8e95-6e39152330a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:04:52.739071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9b0b-2e76-44c7-83ba-b5ed40632baa service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:04:52.739341 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9b0b-2e76-44c7-83ba-b5ed40632baa service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:04:52.739597 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9b0b-2e76-44c7-83ba-b5ed40632baa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:04:52.740866 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f789a129-6779-4ac7-8e95-6e39152330a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:04:52.748844 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e9b0b-2e76-44c7-83ba-b5ed40632baa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:04:52.750509 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e9b0b-2e76-44c7-83ba-b5ed40632baa service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:04:52.755028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789a129-6779-4ac7-8e95-6e39152330a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:04:52.756445 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789a129-6779-4ac7-8e95-6e39152330a3 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:04:52.760624 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4368/8698] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:04:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:04:52.768319 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4331/8699] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:04:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:04:52.832374 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e9b0b-2e76-44c7-83ba-b5ed40632baa service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:04:52.843206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e9b0b-2e76-44c7-83ba-b5ed40632baa service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:04:52.849535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b8101-ae01-4d3b-83ed-c016a36aa051 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:04:52.850145 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b8101-ae01-4d3b-83ed-c016a36aa051 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:04:52.850406 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b8101-ae01-4d3b-83ed-c016a36aa051 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:04:52.850672 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b8101-ae01-4d3b-83ed-c016a36aa051 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:04:52.852370 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-0c8b8101-ae01-4d3b-83ed-c016a36aa051 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:04:52.864459 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b8101-ae01-4d3b-83ed-c016a36aa051 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:04:52.866130 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b8101-ae01-4d3b-83ed-c016a36aa051 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:04:52.876526 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4369/8700] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:04:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:04:53.710316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789a129-6779-4ac7-8e95-6e39152330a3 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:04:53.719476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789a129-6779-4ac7-8e95-6e39152330a3 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:04:53.723365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6403004-be56-41df-bcc5-69a6fe120146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:04:53.723977 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6403004-be56-41df-bcc5-69a6fe120146 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:04:53.724254 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6403004-be56-41df-bcc5-69a6fe120146 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:04:53.724552 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6403004-be56-41df-bcc5-69a6fe120146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:04:53.731699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6403004-be56-41df-bcc5-69a6fe120146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:04:53.733167 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6403004-be56-41df-bcc5-69a6fe120146 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:04:53.741779 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4332/8701] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:04:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:05:02.871855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=50 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:05:03.738229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=38 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:05:06.177176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b8101-ae01-4d3b-83ed-c016a36aa051 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:05:06.186409 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b8101-ae01-4d3b-83ed-c016a36aa051 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:05:06.190649 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cb6fa-d5d0-4dab-9fed-2dee4942ea00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:05:06.191247 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cb6fa-d5d0-4dab-9fed-2dee4942ea00 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:05:06.191526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cb6fa-d5d0-4dab-9fed-2dee4942ea00 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:05:06.191794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cb6fa-d5d0-4dab-9fed-2dee4942ea00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:05:06.193629 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c66cb6fa-d5d0-4dab-9fed-2dee4942ea00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:05:06.204023 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66cb6fa-d5d0-4dab-9fed-2dee4942ea00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:05:06.205540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66cb6fa-d5d0-4dab-9fed-2dee4942ea00 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:05:06.213434 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4370/8702] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:05:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:05:16.209930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:05:47.304172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6403004-be56-41df-bcc5-69a6fe120146 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:05:47.316618 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6403004-be56-41df-bcc5-69a6fe120146 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:05:47.321831 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce600e38-af29-48d7-b0e0-3344584b56d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:05:47.322426 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce600e38-af29-48d7-b0e0-3344584b56d6 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:05:47.322698 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce600e38-af29-48d7-b0e0-3344584b56d6 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:05:47.322987 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce600e38-af29-48d7-b0e0-3344584b56d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:05:47.324733 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ce600e38-af29-48d7-b0e0-3344584b56d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:05:47.337375 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce600e38-af29-48d7-b0e0-3344584b56d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:05:47.339186 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce600e38-af29-48d7-b0e0-3344584b56d6 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:05:47.350660 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4333/8703] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:05:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:05:52.143491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cb6fa-d5d0-4dab-9fed-2dee4942ea00 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:05:52.153700 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cb6fa-d5d0-4dab-9fed-2dee4942ea00 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:05:52.157730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eab4ec-cab3-40a5-b83a-91cf6a08b5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:05:52.158326 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eab4ec-cab3-40a5-b83a-91cf6a08b5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:05:52.158601 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eab4ec-cab3-40a5-b83a-91cf6a08b5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:05:52.158870 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eab4ec-cab3-40a5-b83a-91cf6a08b5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:05:52.168136 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eab4ec-cab3-40a5-b83a-91cf6a08b5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:05:52.169792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eab4ec-cab3-40a5-b83a-91cf6a08b5f6 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:05:52.181403 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4371/8704] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:05:52 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:05:55.147962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca4f69-f028-4f4f-ac68-750200ec0dfc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:05:55.148501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca4f69-f028-4f4f-ac68-750200ec0dfc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:05:55.148501 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca4f69-f028-4f4f-ac68-750200ec0dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:05:55.169780 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-08ca4f69-f028-4f4f-ac68-750200ec0dfc None None] MFA Rules not processed for user `823ad7d2979042c188525f5341febdff`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:05:55.171015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-08ca4f69-f028-4f4f-ac68-750200ec0dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:05:55.179137 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4334/8705] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:05:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:05:55.219695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eab4ec-cab3-40a5-b83a-91cf6a08b5f6 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:05:55.221200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-f0eab4ec-cab3-40a5-b83a-91cf6a08b5f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:05:55.323986 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eab4ec-cab3-40a5-b83a-91cf6a08b5f6 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:05:55.327922 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48a6c8-b538-41ca-a240-3df60edc201d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:05:55.328540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48a6c8-b538-41ca-a240-3df60edc201d service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:05:55.328819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48a6c8-b538-41ca-a240-3df60edc201d service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:05:55.329132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48a6c8-b538-41ca-a240-3df60edc201d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:05:55.349259 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48a6c8-b538-41ca-a240-3df60edc201d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:05:55.350984 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48a6c8-b538-41ca-a240-3df60edc201d service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:05:55.365018 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4372/8706] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:05:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:05:56.145810 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca4f69-f028-4f4f-ac68-750200ec0dfc None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:05:56.160227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca4f69-f028-4f4f-ac68-750200ec0dfc None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:05:56.171541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133a3b3-ebe6-41dd-8c11-f3cdf240bef3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:05:56.172448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133a3b3-ebe6-41dd-8c11-f3cdf240bef3 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:05:56.172967 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133a3b3-ebe6-41dd-8c11-f3cdf240bef3 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:05:56.173500 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133a3b3-ebe6-41dd-8c11-f3cdf240bef3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:05:56.187062 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133a3b3-ebe6-41dd-8c11-f3cdf240bef3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:05:56.188909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133a3b3-ebe6-41dd-8c11-f3cdf240bef3 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:05:56.205080 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4335/8707] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:05:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:06:05.359516 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=41 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:06:06.198364 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=15 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:07:24.512778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48a6c8-b538-41ca-a240-3df60edc201d service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:07:24.522185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48a6c8-b538-41ca-a240-3df60edc201d service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:07:24.526380 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be82a8c-155f-4acb-9434-c11c2f2140ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:07:24.526973 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be82a8c-155f-4acb-9434-c11c2f2140ac service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:07:24.527248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be82a8c-155f-4acb-9434-c11c2f2140ac service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:07:24.527513 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be82a8c-155f-4acb-9434-c11c2f2140ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:07:24.537929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be82a8c-155f-4acb-9434-c11c2f2140ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:07:24.539512 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be82a8c-155f-4acb-9434-c11c2f2140ac service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:07:24.550386 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4373/8708] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:07:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:07:34.545623 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:20.237918 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133a3b3-ebe6-41dd-8c11-f3cdf240bef3 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:08:20.252569 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133a3b3-ebe6-41dd-8c11-f3cdf240bef3 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:08:20.259114 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8127069-674a-45e0-9590-872466cd6b46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:08:20.260931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8127069-674a-45e0-9590-872466cd6b46 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:08:20.261382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8127069-674a-45e0-9590-872466cd6b46 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:08:20.261832 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8127069-674a-45e0-9590-872466cd6b46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:08:20.272133 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8127069-674a-45e0-9590-872466cd6b46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:08:20.273769 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8127069-674a-45e0-9590-872466cd6b46 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:08:20.284808 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4336/8709] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:08:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:08:24.476836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be82a8c-155f-4acb-9434-c11c2f2140ac service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:08:24.490539 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be82a8c-155f-4acb-9434-c11c2f2140ac service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:08:24.496213 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ece85-2e1c-4ccf-b021-6659a613a5fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:08:24.496891 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ece85-2e1c-4ccf-b021-6659a613a5fe service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:08:24.497205 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ece85-2e1c-4ccf-b021-6659a613a5fe service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:08:24.497476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ece85-2e1c-4ccf-b021-6659a613a5fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:08:24.499202 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-232ece85-2e1c-4ccf-b021-6659a613a5fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:08:24.525549 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ece85-2e1c-4ccf-b021-6659a613a5fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:08:24.528276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ece85-2e1c-4ccf-b021-6659a613a5fe service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:08:24.543450 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4374/8710] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:08:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:08:30.280134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:30.559729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae449a-8cee-4ad9-b02f-cea1558970d1 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:08:30.560008 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae449a-8cee-4ad9-b02f-cea1558970d1 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:08:30.560278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae449a-8cee-4ad9-b02f-cea1558970d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:08:30.582078 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d4ae449a-8cee-4ad9-b02f-cea1558970d1 None None] MFA Rules not processed for user `823ad7d2979042c188525f5341febdff`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:08:30.583301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d4ae449a-8cee-4ad9-b02f-cea1558970d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:08:30.592876 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4337/8711] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:08:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:08:30.627484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443a3e3-e11d-466f-8e64-4e4dc26ccd04 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:08:30.628062 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443a3e3-e11d-466f-8e64-4e4dc26ccd04 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:08:30.628170 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443a3e3-e11d-466f-8e64-4e4dc26ccd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:08:30.660731 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7443a3e3-e11d-466f-8e64-4e4dc26ccd04 None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:08:30.662046 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7443a3e3-e11d-466f-8e64-4e4dc26ccd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:08:30.671324 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4375/8712] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 22:08:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:08:30.683697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae449a-8cee-4ad9-b02f-cea1558970d1 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:08:30.693877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae449a-8cee-4ad9-b02f-cea1558970d1 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:08:30.698016 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda42494-1fc3-4b82-9b3d-6a6b3a81d506 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:08:30.698877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda42494-1fc3-4b82-9b3d-6a6b3a81d506 service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:08:30.699353 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda42494-1fc3-4b82-9b3d-6a6b3a81d506 service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:08:30.699621 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda42494-1fc3-4b82-9b3d-6a6b3a81d506 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:08:30.711558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda42494-1fc3-4b82-9b3d-6a6b3a81d506 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:08:30.713357 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda42494-1fc3-4b82-9b3d-6a6b3a81d506 service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:08:30.726966 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4338/8713] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:08:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:08:40.654669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=18 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:40.721269 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=9 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:45.094503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443a3e3-e11d-466f-8e64-4e4dc26ccd04 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:08:45.106702 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443a3e3-e11d-466f-8e64-4e4dc26ccd04 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:08:45.112327 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a93b3-9558-4b39-9cdf-ced66b408295 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:08:45.114806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a93b3-9558-4b39-9cdf-ced66b408295 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:08:45.114806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a93b3-9558-4b39-9cdf-ced66b408295 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:08:45.114806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a93b3-9558-4b39-9cdf-ced66b408295 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:08:45.123859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a93b3-9558-4b39-9cdf-ced66b408295 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:08:45.125469 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a93b3-9558-4b39-9cdf-ced66b408295 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:08:45.136854 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4376/8714] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:08:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:08:55.132651 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:09:51.433212 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda42494-1fc3-4b82-9b3d-6a6b3a81d506 service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:09:51.502929 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda42494-1fc3-4b82-9b3d-6a6b3a81d506 service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:09:51.507978 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cf774-32b3-4316-8aba-06476e2eaa33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:09:51.508579 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf774-32b3-4316-8aba-06476e2eaa33 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:09:51.508861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf774-32b3-4316-8aba-06476e2eaa33 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:09:51.509165 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf774-32b3-4316-8aba-06476e2eaa33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:09:51.511021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-954cf774-32b3-4316-8aba-06476e2eaa33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:09:51.523260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954cf774-32b3-4316-8aba-06476e2eaa33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:09:51.524965 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954cf774-32b3-4316-8aba-06476e2eaa33 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:09:51.537659 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4339/8715] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:09:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:09:52.137605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a93b3-9558-4b39-9cdf-ced66b408295 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:09:52.139168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7e6a93b3-9558-4b39-9cdf-ced66b408295 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:09:52.218971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a93b3-9558-4b39-9cdf-ced66b408295 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:09:52.224433 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e3345-6054-4c9a-b1f7-dff89d40a40e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:09:52.225027 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e3345-6054-4c9a-b1f7-dff89d40a40e service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:09:52.225333 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e3345-6054-4c9a-b1f7-dff89d40a40e service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:09:52.225599 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e3345-6054-4c9a-b1f7-dff89d40a40e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:09:52.227418 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b60e3345-6054-4c9a-b1f7-dff89d40a40e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:09:52.305999 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e3345-6054-4c9a-b1f7-dff89d40a40e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:09:52.307525 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e3345-6054-4c9a-b1f7-dff89d40a40e service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:09:52.325514 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4377/8716] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:09:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:09:55.219344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cf774-32b3-4316-8aba-06476e2eaa33 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:09:55.295862 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cf774-32b3-4316-8aba-06476e2eaa33 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:09:55.300229 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadefef-3707-4a48-9263-21290ff8b05b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:09:55.300988 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadefef-3707-4a48-9263-21290ff8b05b service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:09:55.301147 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadefef-3707-4a48-9263-21290ff8b05b service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:09:55.301365 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadefef-3707-4a48-9263-21290ff8b05b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:09:55.303405 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8cadefef-3707-4a48-9263-21290ff8b05b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:09:55.328201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadefef-3707-4a48-9263-21290ff8b05b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:09:55.329728 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadefef-3707-4a48-9263-21290ff8b05b service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:09:55.339023 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4340/8717] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:09:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:09:56.995432 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e3345-6054-4c9a-b1f7-dff89d40a40e service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:09:57.007807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e3345-6054-4c9a-b1f7-dff89d40a40e service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:09:57.012855 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e13abe-2292-44fc-98bb-6a4dda06d456 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:09:57.013492 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e13abe-2292-44fc-98bb-6a4dda06d456 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:09:57.013767 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e13abe-2292-44fc-98bb-6a4dda06d456 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:09:57.014032 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e13abe-2292-44fc-98bb-6a4dda06d456 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:09:57.016031 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a5e13abe-2292-44fc-98bb-6a4dda06d456 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:09:57.028368 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e13abe-2292-44fc-98bb-6a4dda06d456 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:09:57.029989 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e13abe-2292-44fc-98bb-6a4dda06d456 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:09:57.041232 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4378/8718] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:09:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:10:01.952777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadefef-3707-4a48-9263-21290ff8b05b service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:01.954457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-8cadefef-3707-4a48-9263-21290ff8b05b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:02.064704 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=100 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:02.073519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadefef-3707-4a48-9263-21290ff8b05b service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:02.078203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:02.078829 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:02.079107 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:02.079377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] PATH_INFO: `/v3/users/823ad7d2979042c188525f5341febdff` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:02.080665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=823ad7d2979042c188525f5341febdff)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:02.082703 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:02.131000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '823ad7d2979042c188525f5341febdff'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.143616 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '823ad7d2979042c188525f5341febdff'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.151144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '823ad7d2979042c188525f5341febdff'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.158483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '823ad7d2979042c188525f5341febdff'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.159134 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '823ad7d2979042c188525f5341febdff'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.165368 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4341/8719] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 22:10:01 2022] DELETE /identity/v3/users/823ad7d2979042c188525f5341febdff => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 22:10:02.170177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49075eb-012e-43e1-bd68-4349ea759fd5 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:02.170664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49075eb-012e-43e1-bd68-4349ea759fd5 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:02.170839 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49075eb-012e-43e1-bd68-4349ea759fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:02.200951 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-a49075eb-012e-43e1-bd68-4349ea759fd5 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:10:02.202212 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a49075eb-012e-43e1-bd68-4349ea759fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:02.239921 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:02.261255 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4379/8720] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:10:02.270997 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:02.272441 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:02.329572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2899c1-5877-4644-b935-48458b137ce0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:02.333438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c271-11f4-4ccf-8b03-9634ba7ff97e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:02.334065 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c271-11f4-4ccf-8b03-9634ba7ff97e service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:02.334344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c271-11f4-4ccf-8b03-9634ba7ff97e service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:02.334612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c271-11f4-4ccf-8b03-9634ba7ff97e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:02.343752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c271-11f4-4ccf-8b03-9634ba7ff97e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:02.345214 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c271-11f4-4ccf-8b03-9634ba7ff97e service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:02.354912 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4342/8721] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:10:02.522344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49075eb-012e-43e1-bd68-4349ea759fd5 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:02.523773 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a49075eb-012e-43e1-bd68-4349ea759fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:02.542892 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49075eb-012e-43e1-bd68-4349ea759fd5 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:02.548185 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:02.549397 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:02.549742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:02.550042 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] PATH_INFO: `/v3/projects/ea5345e230164cd6ae5b1195425e7ff0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:02.551253 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea5345e230164cd6ae5b1195425e7ff0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:02.553177 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:02.599758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea5345e230164cd6ae5b1195425e7ff0', 'request_id': 'req-04367bc2-1769-44d7-838c-bf5be163a525'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.607035 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea5345e230164cd6ae5b1195425e7ff0', 'request_id': 'req-04367bc2-1769-44d7-838c-bf5be163a525'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.614298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea5345e230164cd6ae5b1195425e7ff0', 'request_id': 'req-04367bc2-1769-44d7-838c-bf5be163a525'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.625335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] The token cache is being invalidate because project ea5345e230164cd6ae5b1195425e7ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 22:10:02.625622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.626365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.627945 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4380/8722] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 22:10:02 2022] DELETE /identity/v3/projects/ea5345e230164cd6ae5b1195425e7ff0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:10:02.631935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c271-11f4-4ccf-8b03-9634ba7ff97e service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:02.642197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c271-11f4-4ccf-8b03-9634ba7ff97e service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:02.699665 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:02.700195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:02.700468 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:02.700735 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] PATH_INFO: `/v3/users/214188614819400283651340d2befb35` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:02.714486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=214188614819400283651340d2befb35)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:02.716486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:02.720915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:02.763172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '214188614819400283651340d2befb35'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.775657 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '214188614819400283651340d2befb35'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.782874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '214188614819400283651340d2befb35'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.790267 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '214188614819400283651340d2befb35'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.790911 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-722069f0-2e16-4398-961e-4a124734b199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '214188614819400283651340d2befb35'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:02.796751 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4343/8723] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 22:10:02 2022] DELETE /identity/v3/users/214188614819400283651340d2befb35 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:10:03.070654 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:03.072068 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:03.085934 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04367bc2-1769-44d7-838c-bf5be163a525 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:03.143621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:03.144263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.144542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.144810 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] PATH_INFO: `/v3/projects/6e84fcbf6bce498d87327f7520eec5d7` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.155280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e84fcbf6bce498d87327f7520eec5d7)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:03.157230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:03.205441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e84fcbf6bce498d87327f7520eec5d7', 'request_id': 'req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:03.213084 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e84fcbf6bce498d87327f7520eec5d7', 'request_id': 'req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:03.215054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:03.220896 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e84fcbf6bce498d87327f7520eec5d7', 'request_id': 'req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:03.232457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] The token cache is being invalidate because project 6e84fcbf6bce498d87327f7520eec5d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 22:10:03.232743 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:03.233428 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-1d7a571f-192c-4eb5-b9e5-640a5f199d3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:10:03.234924 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4381/8724] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 22:10:03 2022] DELETE /identity/v3/projects/6e84fcbf6bce498d87327f7520eec5d7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:10:03.240673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1f22a-a720-4f09-8823-c608ee619865 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.241180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1f22a-a720-4f09-8823-c608ee619865 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.241288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1f22a-a720-4f09-8823-c608ee619865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.291389 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-aae1f22a-a720-4f09-8823-c608ee619865 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:10:03.292610 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-aae1f22a-a720-4f09-8823-c608ee619865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:03.351152 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4344/8725] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:10:03.359161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458dcc5-1c3a-4c02-8525-58bc65cb5526 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.359450 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458dcc5-1c3a-4c02-8525-58bc65cb5526 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.359730 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458dcc5-1c3a-4c02-8525-58bc65cb5526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.380244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-7458dcc5-1c3a-4c02-8525-58bc65cb5526 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:10:03.381497 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7458dcc5-1c3a-4c02-8525-58bc65cb5526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:03.389430 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4382/8726] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:10:03.394371 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1f22a-a720-4f09-8823-c608ee619865 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:03.404063 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1f22a-a720-4f09-8823-c608ee619865 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:03.408876 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f1dd5-1f76-4e6b-b2fa-f45b9563bfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:03.409443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f1dd5-1f76-4e6b-b2fa-f45b9563bfec admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.409925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f1dd5-1f76-4e6b-b2fa-f45b9563bfec admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.410637 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f1dd5-1f76-4e6b-b2fa-f45b9563bfec admin admin] PATH_INFO: `/v3/domains` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.411711 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f1dd5-1f76-4e6b-b2fa-f45b9563bfec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 22:10:03.412019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f1dd5-1f76-4e6b-b2fa-f45b9563bfec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:03.413947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f1dd5-1f76-4e6b-b2fa-f45b9563bfec admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:03.427112 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4345/8727] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 22:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 22:10:03.431498 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458dcc5-1c3a-4c02-8525-58bc65cb5526 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:03.441493 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458dcc5-1c3a-4c02-8525-58bc65cb5526 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:03.446162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f852f-5439-4c3e-938b-3d2e909503dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:03.446747 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f852f-5439-4c3e-938b-3d2e909503dc admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.447015 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f852f-5439-4c3e-938b-3d2e909503dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.447277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f852f-5439-4c3e-938b-3d2e909503dc admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.447884 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f852f-5439-4c3e-938b-3d2e909503dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:03.449840 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f852f-5439-4c3e-938b-3d2e909503dc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:03.487605 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4383/8728] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 22:10:03 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 22:10:03.491754 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f1dd5-1f76-4e6b-b2fa-f45b9563bfec admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:03.502215 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f1dd5-1f76-4e6b-b2fa-f45b9563bfec admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:03.550172 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:03.563692 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd526b29-3279-40da-839e-6fe28afcbea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:03.565000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526b29-3279-40da-839e-6fe28afcbea9 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.565363 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526b29-3279-40da-839e-6fe28afcbea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.565636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526b29-3279-40da-839e-6fe28afcbea9 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.566656 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd526b29-3279-40da-839e-6fe28afcbea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:03.569154 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd526b29-3279-40da-839e-6fe28afcbea9 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:03.592771 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4346/8729] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 22:10:03 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 22:10:03.596774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f852f-5439-4c3e-938b-3d2e909503dc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:03.607287 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f852f-5439-4c3e-938b-3d2e909503dc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:03.611971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0d438-04f6-4b3b-b669-27234ab28bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:03.612557 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0d438-04f6-4b3b-b669-27234ab28bbe admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.612834 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0d438-04f6-4b3b-b669-27234ab28bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.613141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0d438-04f6-4b3b-b669-27234ab28bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.613771 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0d438-04f6-4b3b-b669-27234ab28bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:03.615345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0d438-04f6-4b3b-b669-27234ab28bbe admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:03.625314 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4384/8730] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 22:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 22:10:03.629814 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd526b29-3279-40da-839e-6fe28afcbea9 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:03.639801 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd526b29-3279-40da-839e-6fe28afcbea9 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:03.645001 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cc87c-ebad-4a87-8e86-b4cd16e7271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:03.645602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cc87c-ebad-4a87-8e86-b4cd16e7271b admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.645874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cc87c-ebad-4a87-8e86-b4cd16e7271b admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.646137 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cc87c-ebad-4a87-8e86-b4cd16e7271b admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.646789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cc87c-ebad-4a87-8e86-b4cd16e7271b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:03.648360 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cc87c-ebad-4a87-8e86-b4cd16e7271b admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:03.658814 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4347/8731] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 22:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 22:10:03.663465 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0d438-04f6-4b3b-b669-27234ab28bbe admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:03.674484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0d438-04f6-4b3b-b669-27234ab28bbe admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:03.679096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3718642-b421-406e-bd13-57e68f5d082e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:03.679735 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3718642-b421-406e-bd13-57e68f5d082e admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.680011 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3718642-b421-406e-bd13-57e68f5d082e admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.680280 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3718642-b421-406e-bd13-57e68f5d082e admin admin] PATH_INFO: `/v3/projects/5ccb8936213a46b4804d7bbb43d4042b/users/6ddf65a9073b4abba4c8d09fed8a1ea6/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.695541 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3718642-b421-406e-bd13-57e68f5d082e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccb8936213a46b4804d7bbb43d4042b, user_id=6ddf65a9073b4abba4c8d09fed8a1ea6, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:03.698244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3718642-b421-406e-bd13-57e68f5d082e admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:03.712381 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4385/8732] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 22:10:03 2022] PUT /identity/v3/projects/5ccb8936213a46b4804d7bbb43d4042b/users/6ddf65a9073b4abba4c8d09fed8a1ea6/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:10:03.717535 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a3676-60f6-4f2a-a2f2-2317b527a3bc None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.717869 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a3676-60f6-4f2a-a2f2-2317b527a3bc None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.718138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a3676-60f6-4f2a-a2f2-2317b527a3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.737587 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-d74a3676-60f6-4f2a-a2f2-2317b527a3bc None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:10:03.738836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d74a3676-60f6-4f2a-a2f2-2317b527a3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:03.797690 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4348/8733] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:10:03.807503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3718642-b421-406e-bd13-57e68f5d082e admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:03.809230 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-a3718642-b421-406e-bd13-57e68f5d082e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:03.866484 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3718642-b421-406e-bd13-57e68f5d082e admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:03.870102 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5441b5-a700-4fe8-b77d-ff83f0eee94a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:03.870686 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5441b5-a700-4fe8-b77d-ff83f0eee94a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:03.870953 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5441b5-a700-4fe8-b77d-ff83f0eee94a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:03.871240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5441b5-a700-4fe8-b77d-ff83f0eee94a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:03.880779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5441b5-a700-4fe8-b77d-ff83f0eee94a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:03.882210 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5441b5-a700-4fe8-b77d-ff83f0eee94a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:03.891435 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4386/8734] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:10:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:10:07.923247 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac48237-73a1-4942-aecb-5f9b2326bdd9 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:07.924181 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac48237-73a1-4942-aecb-5f9b2326bdd9 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:07.924672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac48237-73a1-4942-aecb-5f9b2326bdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:07.953138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-9ac48237-73a1-4942-aecb-5f9b2326bdd9 None None] MFA Rules not processed for user `6ddf65a9073b4abba4c8d09fed8a1ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:10:07.954272 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9ac48237-73a1-4942-aecb-5f9b2326bdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:08.022221 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4349/8735] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:10:08.026664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5441b5-a700-4fe8-b77d-ff83f0eee94a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:08.038335 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5441b5-a700-4fe8-b77d-ff83f0eee94a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:08.043714 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda04f7e-5d4f-4707-965f-d1b50db1a762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:08.044556 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda04f7e-5d4f-4707-965f-d1b50db1a762 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:08.044661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda04f7e-5d4f-4707-965f-d1b50db1a762 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:08.044935 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda04f7e-5d4f-4707-965f-d1b50db1a762 admin admin] PATH_INFO: `/v3/projects` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:08.045794 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda04f7e-5d4f-4707-965f-d1b50db1a762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:08.054543 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda04f7e-5d4f-4707-965f-d1b50db1a762 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:08.089618 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4387/8736] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 22:10:08 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 22:10:08.090387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac48237-73a1-4942-aecb-5f9b2326bdd9 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:08.102452 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac48237-73a1-4942-aecb-5f9b2326bdd9 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:08.160565 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:08.168695 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42348e36-81d1-47e3-8a09-062a4a5f66b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:08.169340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42348e36-81d1-47e3-8a09-062a4a5f66b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:08.169667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42348e36-81d1-47e3-8a09-062a4a5f66b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:08.169879 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42348e36-81d1-47e3-8a09-062a4a5f66b8 admin admin] PATH_INFO: `/v3/users` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:08.170508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42348e36-81d1-47e3-8a09-062a4a5f66b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:08.172457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42348e36-81d1-47e3-8a09-062a4a5f66b8 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:08.199004 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4350/8737] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 22:10:08 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 22:10:08.203055 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda04f7e-5d4f-4707-965f-d1b50db1a762 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:08.234132 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda04f7e-5d4f-4707-965f-d1b50db1a762 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:08.243523 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399eb6fd-7456-4f56-b350-2d6441065607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:08.244263 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eb6fd-7456-4f56-b350-2d6441065607 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:08.244620 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eb6fd-7456-4f56-b350-2d6441065607 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:08.244895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eb6fd-7456-4f56-b350-2d6441065607 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:08.245578 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399eb6fd-7456-4f56-b350-2d6441065607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:08.247191 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399eb6fd-7456-4f56-b350-2d6441065607 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:08.259559 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4388/8738] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 22:10:08.263821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42348e36-81d1-47e3-8a09-062a4a5f66b8 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:08.281300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42348e36-81d1-47e3-8a09-062a4a5f66b8 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:08.286679 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903e168-87c8-4198-9829-f03fcf68fdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:08.287688 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903e168-87c8-4198-9829-f03fcf68fdcc admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:08.287972 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903e168-87c8-4198-9829-f03fcf68fdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:08.288243 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903e168-87c8-4198-9829-f03fcf68fdcc admin admin] PATH_INFO: `/v3/projects/3fffb5042c4b456b9c7668049530cf74/users/8f807ece945c48f3aa84184c7d9dae40/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:08.306414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903e168-87c8-4198-9829-f03fcf68fdcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fffb5042c4b456b9c7668049530cf74, user_id=8f807ece945c48f3aa84184c7d9dae40, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:08.309271 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903e168-87c8-4198-9829-f03fcf68fdcc admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:08.323852 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4351/8739] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 22:10:08 2022] PUT /identity/v3/projects/3fffb5042c4b456b9c7668049530cf74/users/8f807ece945c48f3aa84184c7d9dae40/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:10:08.327964 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399eb6fd-7456-4f56-b350-2d6441065607 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:08.338906 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399eb6fd-7456-4f56-b350-2d6441065607 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:08.386302 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:08.405427 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1dc9a7-bc6b-4e2b-ad69-2ddc51be6fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:08.406044 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dc9a7-bc6b-4e2b-ad69-2ddc51be6fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:08.406324 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dc9a7-bc6b-4e2b-ad69-2ddc51be6fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:08.406590 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dc9a7-bc6b-4e2b-ad69-2ddc51be6fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:08.407231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1dc9a7-bc6b-4e2b-ad69-2ddc51be6fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:08.409215 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1dc9a7-bc6b-4e2b-ad69-2ddc51be6fc4 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:08.420196 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4389/8740] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 22:10:08.424781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903e168-87c8-4198-9829-f03fcf68fdcc admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:08.435685 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903e168-87c8-4198-9829-f03fcf68fdcc admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:08.440766 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20af0c2-3d47-4ca2-8935-2bf0fa843497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:08.441465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20af0c2-3d47-4ca2-8935-2bf0fa843497 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:08.441739 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20af0c2-3d47-4ca2-8935-2bf0fa843497 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:08.442058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20af0c2-3d47-4ca2-8935-2bf0fa843497 admin admin] PATH_INFO: `/v3/projects/3fffb5042c4b456b9c7668049530cf74/users/8f807ece945c48f3aa84184c7d9dae40/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:08.443983 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20af0c2-3d47-4ca2-8935-2bf0fa843497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fffb5042c4b456b9c7668049530cf74, user_id=8f807ece945c48f3aa84184c7d9dae40, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:08.446712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20af0c2-3d47-4ca2-8935-2bf0fa843497 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:08.458669 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4352/8741] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 22:10:08 2022] PUT /identity/v3/projects/3fffb5042c4b456b9c7668049530cf74/users/8f807ece945c48f3aa84184c7d9dae40/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 22:10:12.916807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c93d3-0ca4-47ad-a2dd-05ed8f24f175 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:12.917365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c93d3-0ca4-47ad-a2dd-05ed8f24f175 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:12.917365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c93d3-0ca4-47ad-a2dd-05ed8f24f175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:12.945574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-185c93d3-0ca4-47ad-a2dd-05ed8f24f175 None None] MFA Rules not processed for user `8f807ece945c48f3aa84184c7d9dae40`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:10:12.946885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-185c93d3-0ca4-47ad-a2dd-05ed8f24f175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:13.020203 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4390/8742] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:10:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:10:13.037968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20af0c2-3d47-4ca2-8935-2bf0fa843497 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:13.040491 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c20af0c2-3d47-4ca2-8935-2bf0fa843497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:13.122848 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20af0c2-3d47-4ca2-8935-2bf0fa843497 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:13.128840 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8674546-02ba-417e-9750-3fce47721f65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:13.129483 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674546-02ba-417e-9750-3fce47721f65 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:13.129758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674546-02ba-417e-9750-3fce47721f65 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:13.130028 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674546-02ba-417e-9750-3fce47721f65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:13.191541 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8674546-02ba-417e-9750-3fce47721f65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:13.193188 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8674546-02ba-417e-9750-3fce47721f65 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:13.203589 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4353/8743] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:10:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:10:14.307184 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecdbd8-6545-404f-9c94-86220fb98516 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:14.307667 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecdbd8-6545-404f-9c94-86220fb98516 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:14.307784 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecdbd8-6545-404f-9c94-86220fb98516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:14.326711 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-4eecdbd8-6545-404f-9c94-86220fb98516 None None] MFA Rules not processed for user `6ddf65a9073b4abba4c8d09fed8a1ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:10:14.327898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4eecdbd8-6545-404f-9c94-86220fb98516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:14.335603 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4391/8744] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:10:14.653549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8674546-02ba-417e-9750-3fce47721f65 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:14.655854 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d8674546-02ba-417e-9750-3fce47721f65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:14.720414 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:14.735961 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8674546-02ba-417e-9750-3fce47721f65 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:14.740035 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bb386-61f0-425a-b323-a054c508b933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:14.740933 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bb386-61f0-425a-b323-a054c508b933 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:14.741633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bb386-61f0-425a-b323-a054c508b933 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:14.741836 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bb386-61f0-425a-b323-a054c508b933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:14.743865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-983bb386-61f0-425a-b323-a054c508b933 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:14.762382 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bb386-61f0-425a-b323-a054c508b933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:14.763861 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bb386-61f0-425a-b323-a054c508b933 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:14.773468 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4354/8745] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:10:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:10:23.359819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecdbd8-6545-404f-9c94-86220fb98516 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:23.362305 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4eecdbd8-6545-404f-9c94-86220fb98516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:23.382220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecdbd8-6545-404f-9c94-86220fb98516 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:23.388303 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95e055-9fff-400a-a407-4fcfe914adc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:23.389245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95e055-9fff-400a-a407-4fcfe914adc2 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:23.389346 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95e055-9fff-400a-a407-4fcfe914adc2 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:23.389646 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95e055-9fff-400a-a407-4fcfe914adc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:23.391503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cf95e055-9fff-400a-a407-4fcfe914adc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:10:23.405168 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95e055-9fff-400a-a407-4fcfe914adc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:23.406846 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95e055-9fff-400a-a407-4fcfe914adc2 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:23.420081 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4392/8746] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:10:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:10:24.769889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:26.036715 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bb386-61f0-425a-b323-a054c508b933 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:10:26.045549 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bb386-61f0-425a-b323-a054c508b933 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:10:26.049344 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df5413-2a7d-47fd-904b-ab9fde5110dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:10:26.049932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df5413-2a7d-47fd-904b-ab9fde5110dd service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:10:26.050348 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df5413-2a7d-47fd-904b-ab9fde5110dd service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:10:26.050620 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df5413-2a7d-47fd-904b-ab9fde5110dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:10:26.059537 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df5413-2a7d-47fd-904b-ab9fde5110dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:10:26.060968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df5413-2a7d-47fd-904b-ab9fde5110dd service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:10:26.070846 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4355/8747] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:10:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:10:33.415034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=52 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:36.066724 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:12:54.296257 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95e055-9fff-400a-a407-4fcfe914adc2 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:12:54.315644 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95e055-9fff-400a-a407-4fcfe914adc2 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:12:54.319256 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65753c-2221-46fa-a9f3-1165d1be0242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:12:54.319845 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65753c-2221-46fa-a9f3-1165d1be0242 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:12:54.320119 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65753c-2221-46fa-a9f3-1165d1be0242 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:12:54.320387 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65753c-2221-46fa-a9f3-1165d1be0242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:12:54.322112 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ea65753c-2221-46fa-a9f3-1165d1be0242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:12:54.349096 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65753c-2221-46fa-a9f3-1165d1be0242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:12:54.350510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65753c-2221-46fa-a9f3-1165d1be0242 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:12:54.371530 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4393/8748] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:12:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 22:13:04.368405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=19 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:13:11.257310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc309c7-9978-4398-b0aa-a4d988399c29 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:11.257841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc309c7-9978-4398-b0aa-a4d988399c29 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:11.257841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc309c7-9978-4398-b0aa-a4d988399c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:11.311289 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-edc309c7-9978-4398-b0aa-a4d988399c29 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:13:11.312516 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-edc309c7-9978-4398-b0aa-a4d988399c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:11.381120 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4356/8749] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 22:13:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:13:11.427565 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65753c-2221-46fa-a9f3-1165d1be0242 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:13:11.429157 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-ea65753c-2221-46fa-a9f3-1165d1be0242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:11.518463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65753c-2221-46fa-a9f3-1165d1be0242 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:13:11.523628 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2657d4-d748-4b20-943b-1e9896abf787 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:13:11.524272 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2657d4-d748-4b20-943b-1e9896abf787 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:11.524544 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2657d4-d748-4b20-943b-1e9896abf787 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:11.524819 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2657d4-d748-4b20-943b-1e9896abf787 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:11.533474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2657d4-d748-4b20-943b-1e9896abf787 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:13:11.535059 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2657d4-d748-4b20-943b-1e9896abf787 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:13:11.543887 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4394/8750] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 22:13:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 22:13:21.378250 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=40 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:13:21.540024 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=41 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:13:24.506098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74012c-2dca-43d8-8ced-ff403378da0f None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:24.506600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74012c-2dca-43d8-8ced-ff403378da0f None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:24.506600 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74012c-2dca-43d8-8ced-ff403378da0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:24.545350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-4d74012c-2dca-43d8-8ced-ff403378da0f None None] MFA Rules not processed for user `6225c81d2620463886d8a936b59784df`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:13:24.546546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-4d74012c-2dca-43d8-8ced-ff403378da0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:24.555814 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4357/8751] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Oct 15 22:13:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:13:24.567677 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2657d4-d748-4b20-943b-1e9896abf787 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:13:24.569181 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-cf2657d4-d748-4b20-943b-1e9896abf787 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:24.627248 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2657d4-d748-4b20-943b-1e9896abf787 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:13:24.630817 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8ca0d-7ba9-4a01-bd72-a92ae7808ebf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:13:24.631417 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8ca0d-7ba9-4a01-bd72-a92ae7808ebf service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:24.631691 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8ca0d-7ba9-4a01-bd72-a92ae7808ebf service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:24.631955 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8ca0d-7ba9-4a01-bd72-a92ae7808ebf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:24.643009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8ca0d-7ba9-4a01-bd72-a92ae7808ebf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:13:24.644463 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8ca0d-7ba9-4a01-bd72-a92ae7808ebf service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:13:24.656200 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4395/8752] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:13:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 22:13:24.722367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74012c-2dca-43d8-8ced-ff403378da0f None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:13:24.731607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74012c-2dca-43d8-8ced-ff403378da0f None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:13:24.735180 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0446803-0675-4b75-aa29-03ebd5be1f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:13:24.735760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0446803-0675-4b75-aa29-03ebd5be1f61 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:24.736032 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0446803-0675-4b75-aa29-03ebd5be1f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:24.736295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0446803-0675-4b75-aa29-03ebd5be1f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:24.737992 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f0446803-0675-4b75-aa29-03ebd5be1f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:24.748465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0446803-0675-4b75-aa29-03ebd5be1f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:13:24.750015 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0446803-0675-4b75-aa29-03ebd5be1f61 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:13:24.758076 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4358/8753] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:13:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:13:24.832339 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8ca0d-7ba9-4a01-bd72-a92ae7808ebf service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:13:24.841782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8ca0d-7ba9-4a01-bd72-a92ae7808ebf service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:13:24.845349 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be196e-f628-447e-bae3-cf0382671a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:13:24.845929 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be196e-f628-447e-bae3-cf0382671a52 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:24.846201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be196e-f628-447e-bae3-cf0382671a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:24.846495 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be196e-f628-447e-bae3-cf0382671a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:24.848159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c4be196e-f628-447e-bae3-cf0382671a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:24.861639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be196e-f628-447e-bae3-cf0382671a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:13:24.863106 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be196e-f628-447e-bae3-cf0382671a52 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:13:24.874951 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4396/8754] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:13:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 22:13:24.935820 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0446803-0675-4b75-aa29-03ebd5be1f61 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:13:24.937294 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f0446803-0675-4b75-aa29-03ebd5be1f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:25.005900 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0446803-0675-4b75-aa29-03ebd5be1f61 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:13:25.011021 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e583b-5989-49ca-b9c2-e3c1d4b20c3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:13:25.011672 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e583b-5989-49ca-b9c2-e3c1d4b20c3a service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:25.011949 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e583b-5989-49ca-b9c2-e3c1d4b20c3a service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:25.012218 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e583b-5989-49ca-b9c2-e3c1d4b20c3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:25.013981 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-6a7e583b-5989-49ca-b9c2-e3c1d4b20c3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:25.025578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e583b-5989-49ca-b9c2-e3c1d4b20c3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:13:25.027723 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e583b-5989-49ca-b9c2-e3c1d4b20c3a service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:13:25.037371 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4359/8755] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:13:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:13:26.463166 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d85176-84b3-4e9d-b4af-157b9ce7da93 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:26.463685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d85176-84b3-4e9d-b4af-157b9ce7da93 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:26.463685 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d85176-84b3-4e9d-b4af-157b9ce7da93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:26.485127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-92d85176-84b3-4e9d-b4af-157b9ce7da93 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:13:26.486301 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-92d85176-84b3-4e9d-b4af-157b9ce7da93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:26.493835 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4397/8756] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 22:13:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:13:26.505795 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e583b-5989-49ca-b9c2-e3c1d4b20c3a service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:13:26.516989 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e583b-5989-49ca-b9c2-e3c1d4b20c3a service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:13:26.522017 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c3623-cb8f-4c56-8a6b-a45d67dfc172 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:13:26.522605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c3623-cb8f-4c56-8a6b-a45d67dfc172 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:26.522877 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c3623-cb8f-4c56-8a6b-a45d67dfc172 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:26.523144 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c3623-cb8f-4c56-8a6b-a45d67dfc172 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:26.531898 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c3623-cb8f-4c56-8a6b-a45d67dfc172 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:13:26.533523 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c3623-cb8f-4c56-8a6b-a45d67dfc172 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:13:26.542009 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4360/8757] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 22:13:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 22:13:32.933789 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d85176-84b3-4e9d-b4af-157b9ce7da93 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:13:32.944131 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d85176-84b3-4e9d-b4af-157b9ce7da93 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:13:32.948507 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d55601-7118-4116-9caa-163012aeeb3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:13:32.949140 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d55601-7118-4116-9caa-163012aeeb3a service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:32.949414 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d55601-7118-4116-9caa-163012aeeb3a service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:32.949675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d55601-7118-4116-9caa-163012aeeb3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:32.952127 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e0d55601-7118-4116-9caa-163012aeeb3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:32.967089 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d55601-7118-4116-9caa-163012aeeb3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:13:32.968634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d55601-7118-4116-9caa-163012aeeb3a service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:13:32.982594 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4398/8758] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:13:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:13:36.538387 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=53 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:13:42.975457 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=41 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:13:45.791855 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b23c76-1bcc-4d52-8748-b9a2b2276485 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:45.792406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b23c76-1bcc-4d52-8748-b9a2b2276485 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:45.792406 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b23c76-1bcc-4d52-8748-b9a2b2276485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:45.812089 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-58b23c76-1bcc-4d52-8748-b9a2b2276485 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:13:45.813301 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-58b23c76-1bcc-4d52-8748-b9a2b2276485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:13:45.820434 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4361/8759] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 22:13:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:13:45.833754 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d55601-7118-4116-9caa-163012aeeb3a service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:13:45.846141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d55601-7118-4116-9caa-163012aeeb3a service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:13:45.851169 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32508905-b65d-4a17-ac6c-0e714c9c1fe6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:13:45.851758 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32508905-b65d-4a17-ac6c-0e714c9c1fe6 service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:13:45.852029 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32508905-b65d-4a17-ac6c-0e714c9c1fe6 service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:13:45.852289 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32508905-b65d-4a17-ac6c-0e714c9c1fe6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:13:45.860518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32508905-b65d-4a17-ac6c-0e714c9c1fe6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:13:45.862101 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32508905-b65d-4a17-ac6c-0e714c9c1fe6 service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:13:45.870226 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4399/8760] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 22:13:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 22:13:55.806708 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:13:55.866875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:14:56.437485 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b23c76-1bcc-4d52-8748-b9a2b2276485 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:14:56.438878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-58b23c76-1bcc-4d52-8748-b9a2b2276485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:14:56.513631 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b23c76-1bcc-4d52-8748-b9a2b2276485 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:14:56.518932 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a60f4-108b-47e4-bc2b-278dbfebf708 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:14:56.519540 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a60f4-108b-47e4-bc2b-278dbfebf708 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:14:56.519812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a60f4-108b-47e4-bc2b-278dbfebf708 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:14:56.520081 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a60f4-108b-47e4-bc2b-278dbfebf708 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:14:56.521821 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-ce2a60f4-108b-47e4-bc2b-278dbfebf708 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:14:56.532939 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a60f4-108b-47e4-bc2b-278dbfebf708 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:14:56.534909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a60f4-108b-47e4-bc2b-278dbfebf708 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:14:56.545010 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4362/8761] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:14:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:14:59.676873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32508905-b65d-4a17-ac6c-0e714c9c1fe6 service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:14:59.717873 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32508905-b65d-4a17-ac6c-0e714c9c1fe6 service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:14:59.723004 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44183622-fcaa-40f6-905d-b0077cb8d9cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:14:59.723582 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44183622-fcaa-40f6-905d-b0077cb8d9cc service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:14:59.723851 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44183622-fcaa-40f6-905d-b0077cb8d9cc service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:14:59.724111 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44183622-fcaa-40f6-905d-b0077cb8d9cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:14:59.725835 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-44183622-fcaa-40f6-905d-b0077cb8d9cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:14:59.737244 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44183622-fcaa-40f6-905d-b0077cb8d9cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:14:59.738848 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44183622-fcaa-40f6-905d-b0077cb8d9cc service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:14:59.748378 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4400/8762] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:14:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:15:06.541030 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=32 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:07.430652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a60f4-108b-47e4-bc2b-278dbfebf708 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:15:07.439942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a60f4-108b-47e4-bc2b-278dbfebf708 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:15:07.444425 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97784dc-96b0-4357-acdc-d0c5d255e1ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:15:07.445056 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97784dc-96b0-4357-acdc-d0c5d255e1ea service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:15:07.445355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97784dc-96b0-4357-acdc-d0c5d255e1ea service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:15:07.445625 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97784dc-96b0-4357-acdc-d0c5d255e1ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:15:07.447496 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c97784dc-96b0-4357-acdc-d0c5d255e1ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:15:07.457502 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97784dc-96b0-4357-acdc-d0c5d255e1ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:15:07.458915 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97784dc-96b0-4357-acdc-d0c5d255e1ea service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:15:07.467666 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4363/8763] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:15:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:15:09.744371 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=20 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:17.464265 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:22.939053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44183622-fcaa-40f6-905d-b0077cb8d9cc service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:15:22.949128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44183622-fcaa-40f6-905d-b0077cb8d9cc service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:15:22.952760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172c35d-549e-49c2-b0eb-a7fb940a70fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:15:22.953381 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172c35d-549e-49c2-b0eb-a7fb940a70fb service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:15:22.953653 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172c35d-549e-49c2-b0eb-a7fb940a70fb service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:15:22.953916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172c35d-549e-49c2-b0eb-a7fb940a70fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:15:22.962518 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172c35d-549e-49c2-b0eb-a7fb940a70fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:15:22.963916 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172c35d-549e-49c2-b0eb-a7fb940a70fb service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:15:22.972880 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4401/8764] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:15:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:15:23.813367 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97784dc-96b0-4357-acdc-d0c5d255e1ea service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:15:23.829477 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97784dc-96b0-4357-acdc-d0c5d255e1ea service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:15:23.836257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd706a3-f116-42b0-9eeb-bf5c4c6db52f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:15:23.836865 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd706a3-f116-42b0-9eeb-bf5c4c6db52f service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:15:23.837183 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd706a3-f116-42b0-9eeb-bf5c4c6db52f service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:15:23.837453 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd706a3-f116-42b0-9eeb-bf5c4c6db52f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:15:23.839712 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-9dd706a3-f116-42b0-9eeb-bf5c4c6db52f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:15:23.853673 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd706a3-f116-42b0-9eeb-bf5c4c6db52f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:15:23.855581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd706a3-f116-42b0-9eeb-bf5c4c6db52f service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:15:23.868682 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4364/8765] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:15:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 22:15:28.745261 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b13a-6393-4652-81b5-9ec4a98ef724 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:15:28.745837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b13a-6393-4652-81b5-9ec4a98ef724 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:15:28.745837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b13a-6393-4652-81b5-9ec4a98ef724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:15:28.766353 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-d2e4b13a-6393-4652-81b5-9ec4a98ef724 None None] MFA Rules not processed for user `6ddf65a9073b4abba4c8d09fed8a1ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:15:28.767635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-d2e4b13a-6393-4652-81b5-9ec4a98ef724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:15:28.774678 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4402/8766] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:15:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:15:30.580804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd706a3-f116-42b0-9eeb-bf5c4c6db52f service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:15:30.590273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd706a3-f116-42b0-9eeb-bf5c4c6db52f service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:15:30.593890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c352ee-5d4a-4f57-a62f-f6eba5d035dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:15:30.594476 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c352ee-5d4a-4f57-a62f-f6eba5d035dd service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:15:30.594747 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c352ee-5d4a-4f57-a62f-f6eba5d035dd service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:15:30.595012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c352ee-5d4a-4f57-a62f-f6eba5d035dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:15:30.602839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c352ee-5d4a-4f57-a62f-f6eba5d035dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:15:30.604377 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c352ee-5d4a-4f57-a62f-f6eba5d035dd service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:15:30.613652 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4365/8767] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:15:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:15:31.060542 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4b13a-6393-4652-81b5-9ec4a98ef724 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:15:31.073206 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4b13a-6393-4652-81b5-9ec4a98ef724 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:15:31.078109 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5807c-f735-4e83-8090-e300e62fc610 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:15:31.078715 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5807c-f735-4e83-8090-e300e62fc610 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:15:31.079018 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5807c-f735-4e83-8090-e300e62fc610 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:15:31.079277 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5807c-f735-4e83-8090-e300e62fc610 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:15:31.081124 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-58f5807c-f735-4e83-8090-e300e62fc610 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:15:31.092632 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5807c-f735-4e83-8090-e300e62fc610 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:15:31.094208 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5807c-f735-4e83-8090-e300e62fc610 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:15:31.104458 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4403/8768] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:15:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:15:32.810118 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c352ee-5d4a-4f57-a62f-f6eba5d035dd service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:15:32.820465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c352ee-5d4a-4f57-a62f-f6eba5d035dd service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:15:32.824659 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba0f1e-17f0-4ec7-b9e6-47b1d5255a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:15:32.825634 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba0f1e-17f0-4ec7-b9e6-47b1d5255a2b service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:15:32.826096 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba0f1e-17f0-4ec7-b9e6-47b1d5255a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:15:32.826546 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba0f1e-17f0-4ec7-b9e6-47b1d5255a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:15:32.828597 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-d6ba0f1e-17f0-4ec7-b9e6-47b1d5255a2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:15:32.839290 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba0f1e-17f0-4ec7-b9e6-47b1d5255a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:15:32.840825 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba0f1e-17f0-4ec7-b9e6-47b1d5255a2b service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:15:32.850853 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4366/8769] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:15:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:15:33.541446 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5807c-f735-4e83-8090-e300e62fc610 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:15:33.555483 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5807c-f735-4e83-8090-e300e62fc610 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:15:33.562260 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535ebac-69b9-4c71-8372-b91ccdee99e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:15:33.563300 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535ebac-69b9-4c71-8372-b91ccdee99e0 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:15:33.563529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535ebac-69b9-4c71-8372-b91ccdee99e0 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:15:33.563958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535ebac-69b9-4c71-8372-b91ccdee99e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:15:33.567134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-7535ebac-69b9-4c71-8372-b91ccdee99e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:15:33.579775 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535ebac-69b9-4c71-8372-b91ccdee99e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:15:33.582276 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535ebac-69b9-4c71-8372-b91ccdee99e0 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:15:33.593402 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4404/8770] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:15:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:15:40.076209 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba0f1e-17f0-4ec7-b9e6-47b1d5255a2b service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:15:40.090878 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba0f1e-17f0-4ec7-b9e6-47b1d5255a2b service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:15:40.096942 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befbad0-8f7a-468a-ae80-c3c414938451 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:15:40.097581 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befbad0-8f7a-468a-ae80-c3c414938451 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:15:40.097849 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befbad0-8f7a-468a-ae80-c3c414938451 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:15:40.098112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befbad0-8f7a-468a-ae80-c3c414938451 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:15:40.107195 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befbad0-8f7a-468a-ae80-c3c414938451 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:15:40.108770 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befbad0-8f7a-468a-ae80-c3c414938451 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:15:40.119705 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4367/8771] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:15:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:15:42.168571 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535ebac-69b9-4c71-8372-b91ccdee99e0 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:15:42.178696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535ebac-69b9-4c71-8372-b91ccdee99e0 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:15:42.182336 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974ee27-dc3a-4380-8d9a-e0e47772af5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:15:42.182915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974ee27-dc3a-4380-8d9a-e0e47772af5b service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:15:42.183192 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974ee27-dc3a-4380-8d9a-e0e47772af5b service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:15:42.183455 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974ee27-dc3a-4380-8d9a-e0e47772af5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:15:42.192979 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974ee27-dc3a-4380-8d9a-e0e47772af5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:15:42.194436 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974ee27-dc3a-4380-8d9a-e0e47772af5b service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:15:42.204379 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4405/8772] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:15:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:15:50.115059 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=24 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:52.199913 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=27 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:17:55.199012 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befbad0-8f7a-468a-ae80-c3c414938451 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:17:55.208608 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befbad0-8f7a-468a-ae80-c3c414938451 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:17:55.212085 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a472b1-42b7-4559-90ab-1a08ac47e2ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:17:55.212663 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a472b1-42b7-4559-90ab-1a08ac47e2ff service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:17:55.212931 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a472b1-42b7-4559-90ab-1a08ac47e2ff service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:17:55.213227 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a472b1-42b7-4559-90ab-1a08ac47e2ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:17:55.223087 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a472b1-42b7-4559-90ab-1a08ac47e2ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:17:55.224484 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a472b1-42b7-4559-90ab-1a08ac47e2ff service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:17:55.235245 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4368/8773] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:17:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:18:05.230652 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:38.271604 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974ee27-dc3a-4380-8d9a-e0e47772af5b service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:38.280203 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974ee27-dc3a-4380-8d9a-e0e47772af5b service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:38.283904 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be81bd3-0f6e-4400-ab6a-495c8a8c810e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:38.284476 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be81bd3-0f6e-4400-ab6a-495c8a8c810e service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:38.284779 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be81bd3-0f6e-4400-ab6a-495c8a8c810e service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:38.285054 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be81bd3-0f6e-4400-ab6a-495c8a8c810e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:38.295962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be81bd3-0f6e-4400-ab6a-495c8a8c810e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:38.297474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be81bd3-0f6e-4400-ab6a-495c8a8c810e service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:38.309301 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4406/8774] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:19:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:19:40.903463 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a472b1-42b7-4559-90ab-1a08ac47e2ff service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:40.912277 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a472b1-42b7-4559-90ab-1a08ac47e2ff service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:40.915936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcdd22-70e7-4a6d-b13b-012a26da5ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:40.916508 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcdd22-70e7-4a6d-b13b-012a26da5ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:40.916781 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcdd22-70e7-4a6d-b13b-012a26da5ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:40.917066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcdd22-70e7-4a6d-b13b-012a26da5ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:40.979288 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcdd22-70e7-4a6d-b13b-012a26da5ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:40.980729 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcdd22-70e7-4a6d-b13b-012a26da5ec1 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:40.990375 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4369/8775] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:19:44.225836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be81bd3-0f6e-4400-ab6a-495c8a8c810e service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:44.238535 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be81bd3-0f6e-4400-ab6a-495c8a8c810e service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:44.245146 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69040eb2-938d-4b2b-9dc7-289b1a173af2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:44.245760 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69040eb2-938d-4b2b-9dc7-289b1a173af2 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:44.246034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69040eb2-938d-4b2b-9dc7-289b1a173af2 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:44.246298 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69040eb2-938d-4b2b-9dc7-289b1a173af2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:44.249726 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-69040eb2-938d-4b2b-9dc7-289b1a173af2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:44.280422 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69040eb2-938d-4b2b-9dc7-289b1a173af2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:44.282452 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69040eb2-938d-4b2b-9dc7-289b1a173af2 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:44.316191 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4407/8776] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:19:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:19:47.561771 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcdd22-70e7-4a6d-b13b-012a26da5ec1 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:47.573447 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcdd22-70e7-4a6d-b13b-012a26da5ec1 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:47.578584 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83526a6e-683a-4714-b433-857234b7b6f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:47.579273 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83526a6e-683a-4714-b433-857234b7b6f4 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:47.579613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83526a6e-683a-4714-b433-857234b7b6f4 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:47.579969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83526a6e-683a-4714-b433-857234b7b6f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:47.581946 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-83526a6e-683a-4714-b433-857234b7b6f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:47.594305 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83526a6e-683a-4714-b433-857234b7b6f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:47.595969 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83526a6e-683a-4714-b433-857234b7b6f4 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:47.607275 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4370/8777] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:19:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:19:47.627581 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69040eb2-938d-4b2b-9dc7-289b1a173af2 service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:47.639574 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69040eb2-938d-4b2b-9dc7-289b1a173af2 service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:47.645914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:47.646603 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:47.646878 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:47.647154 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] PATH_INFO: `/v3/users/6ddf65a9073b4abba4c8d09fed8a1ea6` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:47.648246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddf65a9073b4abba4c8d09fed8a1ea6)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:47.650828 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:47.697526 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddf65a9073b4abba4c8d09fed8a1ea6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:47.704660 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddf65a9073b4abba4c8d09fed8a1ea6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:47.717270 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddf65a9073b4abba4c8d09fed8a1ea6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:47.731952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddf65a9073b4abba4c8d09fed8a1ea6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:47.732588 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddf65a9073b4abba4c8d09fed8a1ea6'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:47.738750 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4408/8778] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 22:19:47 2022] DELETE /identity/v3/users/6ddf65a9073b4abba4c8d09fed8a1ea6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:19:47.915093 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83526a6e-683a-4714-b433-857234b7b6f4 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:47.927007 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83526a6e-683a-4714-b433-857234b7b6f4 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:47.983428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:47.984046 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:47.984322 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:47.984590 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] PATH_INFO: `/v3/projects/5ccb8936213a46b4804d7bbb43d4042b` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:47.985699 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ccb8936213a46b4804d7bbb43d4042b)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:47.987603 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:48.033097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ccb8936213a46b4804d7bbb43d4042b', 'request_id': 'req-0afd5732-eb72-4c74-a8a2-626a30b10bd4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.040345 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ccb8936213a46b4804d7bbb43d4042b', 'request_id': 'req-0afd5732-eb72-4c74-a8a2-626a30b10bd4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.046847 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ccb8936213a46b4804d7bbb43d4042b', 'request_id': 'req-0afd5732-eb72-4c74-a8a2-626a30b10bd4'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.057490 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] The token cache is being invalidate because project 5ccb8936213a46b4804d7bbb43d4042b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 22:19:48.057777 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.058403 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.059933 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4371/8779] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 22:19:47 2022] DELETE /identity/v3/projects/5ccb8936213a46b4804d7bbb43d4042b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:19:48.064052 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:48.065534 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:48.078932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec45e5-8922-45ac-b97f-e09947871142 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:48.135678 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:48.136310 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:48.136591 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:48.136850 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] PATH_INFO: `/v3/users/8f807ece945c48f3aa84184c7d9dae40` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:48.137996 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f807ece945c48f3aa84184c7d9dae40)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:48.139879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:48.185895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:48.201377 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f807ece945c48f3aa84184c7d9dae40'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.209776 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f807ece945c48f3aa84184c7d9dae40'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.222930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f807ece945c48f3aa84184c7d9dae40'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.231894 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f807ece945c48f3aa84184c7d9dae40'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.232622 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4190f70c-ccd5-45b7-948f-2d307debb3f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f807ece945c48f3aa84184c7d9dae40'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.238896 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4409/8780] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 22:19:48 2022] DELETE /identity/v3/users/8f807ece945c48f3aa84184c7d9dae40 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:19:48.741890 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:48.743737 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:48.756558 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd5732-eb72-4c74-a8a2-626a30b10bd4 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:48.766027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:48.817488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:48.818109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:48.818475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:48.818743 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] PATH_INFO: `/v3/projects/3fffb5042c4b456b9c7668049530cf74` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:48.819841 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fffb5042c4b456b9c7668049530cf74)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:48.822350 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:48.869858 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fffb5042c4b456b9c7668049530cf74', 'request_id': 'req-61b4dbea-6336-40a7-bb41-a7ec17196222'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.877158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fffb5042c4b456b9c7668049530cf74', 'request_id': 'req-61b4dbea-6336-40a7-bb41-a7ec17196222'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.883511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fffb5042c4b456b9c7668049530cf74', 'request_id': 'req-61b4dbea-6336-40a7-bb41-a7ec17196222'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.894612 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] The token cache is being invalidate because project 3fffb5042c4b456b9c7668049530cf74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 22:19:48.894893 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.895511 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-61b4dbea-6336-40a7-bb41-a7ec17196222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:19:48.896987 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4372/8781] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 22:19:48 2022] DELETE /identity/v3/projects/3fffb5042c4b456b9c7668049530cf74 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:19:48.902780 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd161730-92d2-40be-bc2a-e74127931532 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:48.903240 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd161730-92d2-40be-bc2a-e74127931532 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:48.903404 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd161730-92d2-40be-bc2a-e74127931532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:48.923764 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-dd161730-92d2-40be-bc2a-e74127931532 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:19:48.924971 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-dd161730-92d2-40be-bc2a-e74127931532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:48.982995 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4410/8782] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:19:48.990697 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd87e6-3255-4908-a3f5-7b9fdd2c917c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:48.990980 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd87e6-3255-4908-a3f5-7b9fdd2c917c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:48.991249 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd87e6-3255-4908-a3f5-7b9fdd2c917c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:49.011566 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-67cd87e6-3255-4908-a3f5-7b9fdd2c917c None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:19:49.012764 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-67cd87e6-3255-4908-a3f5-7b9fdd2c917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:49.020735 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4373/8783] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:19:49.025550 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd161730-92d2-40be-bc2a-e74127931532 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:49.035190 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd161730-92d2-40be-bc2a-e74127931532 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:49.039807 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be88626-3bde-40fb-9cfe-ee8a2cf8ffbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:49.040491 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be88626-3bde-40fb-9cfe-ee8a2cf8ffbc admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:49.040705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be88626-3bde-40fb-9cfe-ee8a2cf8ffbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:49.040974 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be88626-3bde-40fb-9cfe-ee8a2cf8ffbc admin admin] PATH_INFO: `/v3/domains` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:49.041652 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be88626-3bde-40fb-9cfe-ee8a2cf8ffbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 22:19:49.041952 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be88626-3bde-40fb-9cfe-ee8a2cf8ffbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:49.043527 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be88626-3bde-40fb-9cfe-ee8a2cf8ffbc admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:49.055238 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4411/8784] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Oct 15 22:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 22:19:49.059492 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd87e6-3255-4908-a3f5-7b9fdd2c917c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:49.068686 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd87e6-3255-4908-a3f5-7b9fdd2c917c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:49.073943 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a2dd3-152b-4c12-939a-688e49350c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:49.074519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a2dd3-152b-4c12-939a-688e49350c8d admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:49.074789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a2dd3-152b-4c12-939a-688e49350c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:49.075053 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a2dd3-152b-4c12-939a-688e49350c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:49.075666 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a2dd3-152b-4c12-939a-688e49350c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:49.077761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a2dd3-152b-4c12-939a-688e49350c8d admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:49.105357 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4374/8785] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 22:19:49 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 22:19:49.109600 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be88626-3bde-40fb-9cfe-ee8a2cf8ffbc admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:49.120320 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be88626-3bde-40fb-9cfe-ee8a2cf8ffbc admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:49.176268 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0c31b-f5de-4fec-86aa-42d178e07cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:49.176865 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0c31b-f5de-4fec-86aa-42d178e07cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:49.177175 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0c31b-f5de-4fec-86aa-42d178e07cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:49.177445 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0c31b-f5de-4fec-86aa-42d178e07cf2 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:49.178067 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0c31b-f5de-4fec-86aa-42d178e07cf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:49.179958 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0c31b-f5de-4fec-86aa-42d178e07cf2 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:49.201936 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4412/8786] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 22:19:49 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 22:19:49.205994 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a2dd3-152b-4c12-939a-688e49350c8d admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:49.217246 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a2dd3-152b-4c12-939a-688e49350c8d admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:49.221881 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7740035-6b67-4e46-9d90-eaf227fe3c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:49.222465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7740035-6b67-4e46-9d90-eaf227fe3c4f admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:49.222744 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7740035-6b67-4e46-9d90-eaf227fe3c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:49.223010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7740035-6b67-4e46-9d90-eaf227fe3c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:49.223650 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7740035-6b67-4e46-9d90-eaf227fe3c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:49.225278 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7740035-6b67-4e46-9d90-eaf227fe3c4f admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:49.236051 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4375/8787] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 22:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 22:19:49.240176 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0c31b-f5de-4fec-86aa-42d178e07cf2 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:49.250612 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0c31b-f5de-4fec-86aa-42d178e07cf2 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:49.255264 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff2778-ce8a-4b08-ad7b-9efe1b74a41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:49.255853 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff2778-ce8a-4b08-ad7b-9efe1b74a41f admin admin] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:49.256128 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff2778-ce8a-4b08-ad7b-9efe1b74a41f admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:49.256393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff2778-ce8a-4b08-ad7b-9efe1b74a41f admin admin] PATH_INFO: `/v3/roles` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:49.257020 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff2778-ce8a-4b08-ad7b-9efe1b74a41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:49.258729 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff2778-ce8a-4b08-ad7b-9efe1b74a41f admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:49.269638 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4413/8788] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 22:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 22:19:49.273800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7740035-6b67-4e46-9d90-eaf227fe3c4f admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:49.284280 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7740035-6b67-4e46-9d90-eaf227fe3c4f admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:49.288956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06130b-f959-4646-86e4-0f31aab82ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:49.289613 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06130b-f959-4646-86e4-0f31aab82ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:49.289889 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06130b-f959-4646-86e4-0f31aab82ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:49.290338 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06130b-f959-4646-86e4-0f31aab82ec6 admin admin] PATH_INFO: `/v3/projects/f02d6a82dfc941c9a0f19bce198d76ab/users/4879d2bce837495baf7411ee27934ece/roles/b5b286eb4fb94285914766a260e99693` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:49.305228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06130b-f959-4646-86e4-0f31aab82ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02d6a82dfc941c9a0f19bce198d76ab, user_id=4879d2bce837495baf7411ee27934ece, role_id=b5b286eb4fb94285914766a260e99693)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:49.308519 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06130b-f959-4646-86e4-0f31aab82ec6 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:49.321544 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4376/8789] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 22:19:49 2022] PUT /identity/v3/projects/f02d6a82dfc941c9a0f19bce198d76ab/users/4879d2bce837495baf7411ee27934ece/roles/b5b286eb4fb94285914766a260e99693 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:19:49.326791 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee04127-63cf-4922-980b-c32253649642 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:49.327134 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee04127-63cf-4922-980b-c32253649642 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:49.327405 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee04127-63cf-4922-980b-c32253649642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:49.346669 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-eee04127-63cf-4922-980b-c32253649642 None None] MFA Rules not processed for user `f9ecef8e2dba4f429d37c67479eb320e`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:19:49.347875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-eee04127-63cf-4922-980b-c32253649642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:49.360797 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:49.408412 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4414/8790] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:19:49.418384 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06130b-f959-4646-86e4-0f31aab82ec6 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:49.420319 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-7f06130b-f959-4646-86e4-0f31aab82ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:49.462160 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:49.479601 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06130b-f959-4646-86e4-0f31aab82ec6 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:49.483395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27098d11-ea18-459a-ae3c-468052e51b4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:49.484097 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27098d11-ea18-459a-ae3c-468052e51b4c service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:49.484373 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27098d11-ea18-459a-ae3c-468052e51b4c service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:49.484643 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27098d11-ea18-459a-ae3c-468052e51b4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:49.494752 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27098d11-ea18-459a-ae3c-468052e51b4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:49.496233 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27098d11-ea18-459a-ae3c-468052e51b4c service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:49.505811 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4377/8791] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:19:53.189108 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b28c29-ea74-4eba-bf48-5c1ae1646684 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:53.189927 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b28c29-ea74-4eba-bf48-5c1ae1646684 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:53.190201 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b28c29-ea74-4eba-bf48-5c1ae1646684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:53.220875 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-e0b28c29-ea74-4eba-bf48-5c1ae1646684 None None] MFA Rules not processed for user `4879d2bce837495baf7411ee27934ece`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:19:53.222069 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e0b28c29-ea74-4eba-bf48-5c1ae1646684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:53.291761 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4415/8792] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:19:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:19:53.296509 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27098d11-ea18-459a-ae3c-468052e51b4c service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:53.306800 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27098d11-ea18-459a-ae3c-468052e51b4c service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:53.311798 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8985db-9813-4c2e-bb24-34c32dd9536a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:53.312390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8985db-9813-4c2e-bb24-34c32dd9536a admin admin] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:53.312667 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8985db-9813-4c2e-bb24-34c32dd9536a admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:53.312935 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8985db-9813-4c2e-bb24-34c32dd9536a admin admin] PATH_INFO: `/v3/projects` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:53.313602 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8985db-9813-4c2e-bb24-34c32dd9536a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:53.315517 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8985db-9813-4c2e-bb24-34c32dd9536a admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:53.343038 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4378/8793] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Oct 15 22:19:53 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 22:19:53.347222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b28c29-ea74-4eba-bf48-5c1ae1646684 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:53.359245 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b28c29-ea74-4eba-bf48-5c1ae1646684 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:53.433220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6a07d-b591-4495-8490-eef6d5863907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:53.433837 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6a07d-b591-4495-8490-eef6d5863907 admin admin] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:53.434104 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6a07d-b591-4495-8490-eef6d5863907 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:53.434364 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6a07d-b591-4495-8490-eef6d5863907 admin admin] PATH_INFO: `/v3/users` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:53.434998 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6a07d-b591-4495-8490-eef6d5863907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:53.436885 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6a07d-b591-4495-8490-eef6d5863907 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:53.464841 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4416/8794] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Oct 15 22:19:53 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 22:19:53.468897 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8985db-9813-4c2e-bb24-34c32dd9536a admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:53.482515 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8985db-9813-4c2e-bb24-34c32dd9536a admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:53.488473 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d8634-efe9-4d1b-9c32-15494980dac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:53.489102 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d8634-efe9-4d1b-9c32-15494980dac0 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:53.489376 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d8634-efe9-4d1b-9c32-15494980dac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:53.489646 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d8634-efe9-4d1b-9c32-15494980dac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:53.490291 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d8634-efe9-4d1b-9c32-15494980dac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:53.491906 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d8634-efe9-4d1b-9c32-15494980dac0 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:53.504755 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4379/8795] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 22:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 22:19:53.509560 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6a07d-b591-4495-8490-eef6d5863907 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:53.520365 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6a07d-b591-4495-8490-eef6d5863907 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:53.525914 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6070c-d800-4b77-bb09-a7f1dcc596ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:53.526598 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6070c-d800-4b77-bb09-a7f1dcc596ad admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:53.527022 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6070c-d800-4b77-bb09-a7f1dcc596ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:53.527293 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6070c-d800-4b77-bb09-a7f1dcc596ad admin admin] PATH_INFO: `/v3/projects/acb14dbd0d644c0e84c8f2c15f3360f0/users/ca3addd5bcc9407b87519e47e107002d/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:53.544522 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6070c-d800-4b77-bb09-a7f1dcc596ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb14dbd0d644c0e84c8f2c15f3360f0, user_id=ca3addd5bcc9407b87519e47e107002d, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:53.547251 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6070c-d800-4b77-bb09-a7f1dcc596ad admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:53.563523 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4417/8796] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 22:19:53 2022] PUT /identity/v3/projects/acb14dbd0d644c0e84c8f2c15f3360f0/users/ca3addd5bcc9407b87519e47e107002d/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:19:53.567091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d8634-efe9-4d1b-9c32-15494980dac0 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:53.591112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d8634-efe9-4d1b-9c32-15494980dac0 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:53.653851 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706bed0-c780-4340-bfac-d65e44128ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:53.654465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706bed0-c780-4340-bfac-d65e44128ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:53.654742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706bed0-c780-4340-bfac-d65e44128ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:53.655010 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706bed0-c780-4340-bfac-d65e44128ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:53.655651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706bed0-c780-4340-bfac-d65e44128ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:53.657298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706bed0-c780-4340-bfac-d65e44128ef1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:53.667999 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4380/8797] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 22:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 22:19:53.672547 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6070c-d800-4b77-bb09-a7f1dcc596ad admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:53.684340 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6070c-d800-4b77-bb09-a7f1dcc596ad admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:53.689014 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5ee18-3379-4722-832f-92eec6484f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:53.689778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5ee18-3379-4722-832f-92eec6484f72 admin admin] REQUEST_METHOD: `PUT` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:53.690049 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5ee18-3379-4722-832f-92eec6484f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:53.690315 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5ee18-3379-4722-832f-92eec6484f72 admin admin] PATH_INFO: `/v3/projects/acb14dbd0d644c0e84c8f2c15f3360f0/users/ca3addd5bcc9407b87519e47e107002d/roles/fab66a1a6ba64c9583182c78978b218e` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:53.692242 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5ee18-3379-4722-832f-92eec6484f72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb14dbd0d644c0e84c8f2c15f3360f0, user_id=ca3addd5bcc9407b87519e47e107002d, role_id=fab66a1a6ba64c9583182c78978b218e)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:53.694907 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5ee18-3379-4722-832f-92eec6484f72 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:53.708784 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4418/8798] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Oct 15 22:19:53 2022] PUT /identity/v3/projects/acb14dbd0d644c0e84c8f2c15f3360f0/users/ca3addd5bcc9407b87519e47e107002d/roles/fab66a1a6ba64c9583182c78978b218e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:19:58.136407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1b956-34e9-43fb-aba7-e8ae52c5291c None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:58.136407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1b956-34e9-43fb-aba7-e8ae52c5291c None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:58.136407 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1b956-34e9-43fb-aba7-e8ae52c5291c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:58.175428 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-bcb1b956-34e9-43fb-aba7-e8ae52c5291c None None] MFA Rules not processed for user `ca3addd5bcc9407b87519e47e107002d`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:19:58.176995 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bcb1b956-34e9-43fb-aba7-e8ae52c5291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:58.254366 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4381/8799] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:19:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:19:58.271792 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5ee18-3379-4722-832f-92eec6484f72 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:58.275638 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8eb5ee18-3379-4722-832f-92eec6484f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:58.281345 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:58.362656 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5ee18-3379-4722-832f-92eec6484f72 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:58.368740 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6092c1d-4d98-405d-ac78-45a6638a5886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:58.369390 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6092c1d-4d98-405d-ac78-45a6638a5886 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:58.369664 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6092c1d-4d98-405d-ac78-45a6638a5886 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:58.369930 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6092c1d-4d98-405d-ac78-45a6638a5886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:58.430854 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6092c1d-4d98-405d-ac78-45a6638a5886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:58.432661 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6092c1d-4d98-405d-ac78-45a6638a5886 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:58.442767 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4419/8800] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:19:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 22:19:59.378937 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1b956-34e9-43fb-aba7-e8ae52c5291c None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:19:59.380386 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-bcb1b956-34e9-43fb-aba7-e8ae52c5291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:59.417604 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:59.459559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1b956-34e9-43fb-aba7-e8ae52c5291c None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:19:59.465197 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59604b3d-11ad-451f-b29e-d860d5616164 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:19:59.466058 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59604b3d-11ad-451f-b29e-d860d5616164 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:59.466488 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59604b3d-11ad-451f-b29e-d860d5616164 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:59.466750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59604b3d-11ad-451f-b29e-d860d5616164 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:59.468683 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-59604b3d-11ad-451f-b29e-d860d5616164 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:59.482669 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59604b3d-11ad-451f-b29e-d860d5616164 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:19:59.484310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59604b3d-11ad-451f-b29e-d860d5616164 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:19:59.495548 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4382/8801] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:19:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:19:59.521919 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d04e40-c9d9-465e-b0f0-e53cd5216174 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:19:59.522462 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d04e40-c9d9-465e-b0f0-e53cd5216174 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:19:59.522579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d04e40-c9d9-465e-b0f0-e53cd5216174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:19:59.544749 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-25d04e40-c9d9-465e-b0f0-e53cd5216174 None None] MFA Rules not processed for user `4879d2bce837495baf7411ee27934ece`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:19:59.546114 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-25d04e40-c9d9-465e-b0f0-e53cd5216174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:19:59.553342 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4420/8802] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:19:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:20:05.654098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59604b3d-11ad-451f-b29e-d860d5616164 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:20:05.665655 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59604b3d-11ad-451f-b29e-d860d5616164 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:20:05.670761 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4505d0a-ccbb-4a62-b2a8-1b495a25a8b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:20:05.671352 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4505d0a-ccbb-4a62-b2a8-1b495a25a8b7 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:20:05.671636 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4505d0a-ccbb-4a62-b2a8-1b495a25a8b7 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:20:05.671904 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4505d0a-ccbb-4a62-b2a8-1b495a25a8b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:20:05.673638 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f4505d0a-ccbb-4a62-b2a8-1b495a25a8b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:20:05.685111 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4505d0a-ccbb-4a62-b2a8-1b495a25a8b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:20:05.686722 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4505d0a-ccbb-4a62-b2a8-1b495a25a8b7 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:20:05.696551 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4383/8803] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:20:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:20:09.538099 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=61 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:20:11.176299 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d04e40-c9d9-465e-b0f0-e53cd5216174 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:20:11.177814 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-25d04e40-c9d9-465e-b0f0-e53cd5216174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:20:11.247249 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d04e40-c9d9-465e-b0f0-e53cd5216174 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:20:11.250830 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27260d1a-c01a-4913-820f-33e2e4a69e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:20:11.251424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27260d1a-c01a-4913-820f-33e2e4a69e90 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:20:11.251697 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27260d1a-c01a-4913-820f-33e2e4a69e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:20:11.251962 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27260d1a-c01a-4913-820f-33e2e4a69e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:20:11.261503 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27260d1a-c01a-4913-820f-33e2e4a69e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:20:11.262910 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27260d1a-c01a-4913-820f-33e2e4a69e90 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:20:11.272239 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4421/8804] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:20:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:20:15.692310 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=25 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:20:21.268028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=31 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:21:18.165287 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4505d0a-ccbb-4a62-b2a8-1b495a25a8b7 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:21:18.176424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4505d0a-ccbb-4a62-b2a8-1b495a25a8b7 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:21:18.180527 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c5686-c590-482f-9801-396fc9fcffd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:21:18.181201 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c5686-c590-482f-9801-396fc9fcffd1 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:21:18.181487 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c5686-c590-482f-9801-396fc9fcffd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:21:18.181750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c5686-c590-482f-9801-396fc9fcffd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:21:18.183562 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-496c5686-c590-482f-9801-396fc9fcffd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:21:18.193793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c5686-c590-482f-9801-396fc9fcffd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:21:18.195316 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c5686-c590-482f-9801-396fc9fcffd1 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:21:18.204154 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4384/8805] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:21:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:21:28.200461 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:11.275756 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e31be-be44-4ffa-9874-6586b8902b07 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:22:11.276231 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e31be-be44-4ffa-9874-6586b8902b07 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:22:11.276332 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e31be-be44-4ffa-9874-6586b8902b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:22:11.305529 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-8c5e31be-be44-4ffa-9874-6586b8902b07 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:22:11.306705 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8c5e31be-be44-4ffa-9874-6586b8902b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:22:11.362873 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4422/8806] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 22:22:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:22:11.409623 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c5686-c590-482f-9801-396fc9fcffd1 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:22:11.411043 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-496c5686-c590-482f-9801-396fc9fcffd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:22:11.482248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c5686-c590-482f-9801-396fc9fcffd1 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:22:11.487605 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa621814-56bc-4d72-ad30-3be350fc3e10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:22:11.488256 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621814-56bc-4d72-ad30-3be350fc3e10 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:22:11.488536 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621814-56bc-4d72-ad30-3be350fc3e10 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:22:11.488806 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621814-56bc-4d72-ad30-3be350fc3e10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:22:11.497828 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa621814-56bc-4d72-ad30-3be350fc3e10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:22:11.499465 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa621814-56bc-4d72-ad30-3be350fc3e10 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:22:11.507585 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4385/8807] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 22:22:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 22:22:13.890563 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e31be-be44-4ffa-9874-6586b8902b07 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:22:13.892424 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-8c5e31be-be44-4ffa-9874-6586b8902b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:22:13.914761 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e31be-be44-4ffa-9874-6586b8902b07 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:22:13.921698 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0179f-0353-4334-9ee7-0b9a0853a343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:22:13.922306 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0179f-0353-4334-9ee7-0b9a0853a343 service nova] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:22:13.922576 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0179f-0353-4334-9ee7-0b9a0853a343 service nova] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:22:13.923200 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0179f-0353-4334-9ee7-0b9a0853a343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:22:13.924836 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-98f0179f-0353-4334-9ee7-0b9a0853a343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:22:13.943634 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0179f-0353-4334-9ee7-0b9a0853a343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:22:13.945665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0179f-0353-4334-9ee7-0b9a0853a343 service nova] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:22:13.958837 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4423/8808] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:22:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:22:21.119166 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb7226-5069-4a48-bd5a-b1e74ffae070 None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:22:21.119760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb7226-5069-4a48-bd5a-b1e74ffae070 None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:22:21.119760 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb7226-5069-4a48-bd5a-b1e74ffae070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:22:21.139962 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0bbb7226-5069-4a48-bd5a-b1e74ffae070 None None] MFA Rules not processed for user `4879d2bce837495baf7411ee27934ece`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:22:21.141168 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0bbb7226-5069-4a48-bd5a-b1e74ffae070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:22:21.147919 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4386/8809] 10.222.0.16 () {58 vars in 958 bytes} [Sat Oct 15 22:22:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:22:21.482141 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0179f-0353-4334-9ee7-0b9a0853a343 service nova] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:22:21.491510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0179f-0353-4334-9ee7-0b9a0853a343 service nova] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:22:21.495063 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b282e-7004-4b6d-9104-8e3041e25e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:22:21.495639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b282e-7004-4b6d-9104-8e3041e25e50 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:22:21.495932 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b282e-7004-4b6d-9104-8e3041e25e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:22:21.496217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b282e-7004-4b6d-9104-8e3041e25e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:22:21.497995 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-bc1b282e-7004-4b6d-9104-8e3041e25e50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:22:21.507309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b282e-7004-4b6d-9104-8e3041e25e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:22:21.508695 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b282e-7004-4b6d-9104-8e3041e25e50 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:22:21.516839 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4424/8810] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:22:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:22:21.850955 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb7226-5069-4a48-bd5a-b1e74ffae070 None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:22:21.865158 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb7226-5069-4a48-bd5a-b1e74ffae070 None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:22:21.870232 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a18a35-e8f2-4e35-bdf0-4a1b6daed344 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:22:21.870874 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a18a35-e8f2-4e35-bdf0-4a1b6daed344 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:22:21.871098 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a18a35-e8f2-4e35-bdf0-4a1b6daed344 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:22:21.871408 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a18a35-e8f2-4e35-bdf0-4a1b6daed344 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:22:21.873863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e3a18a35-e8f2-4e35-bdf0-4a1b6daed344 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:22:21.885867 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a18a35-e8f2-4e35-bdf0-4a1b6daed344 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:22:21.887539 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a18a35-e8f2-4e35-bdf0-4a1b6daed344 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:22:21.900292 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4387/8811] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:22:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:22:23.082183 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198659f7-a39a-42db-a93d-5e8ce41b1f4f None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:22:23.082665 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198659f7-a39a-42db-a93d-5e8ce41b1f4f None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:22:23.082768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198659f7-a39a-42db-a93d-5e8ce41b1f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:22:23.102605 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-198659f7-a39a-42db-a93d-5e8ce41b1f4f None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:22:23.104097 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-198659f7-a39a-42db-a93d-5e8ce41b1f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:22:23.111417 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4425/8812] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 22:22:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:22:23.123892 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a18a35-e8f2-4e35-bdf0-4a1b6daed344 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:22:23.135044 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a18a35-e8f2-4e35-bdf0-4a1b6daed344 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:22:23.140748 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0d499-063c-41ae-ab2e-a207cd4ff75a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:22:23.141369 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0d499-063c-41ae-ab2e-a207cd4ff75a service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:22:23.141651 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0d499-063c-41ae-ab2e-a207cd4ff75a service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:22:23.141919 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0d499-063c-41ae-ab2e-a207cd4ff75a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:22:23.150177 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0d499-063c-41ae-ab2e-a207cd4ff75a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:22:23.151755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0d499-063c-41ae-ab2e-a207cd4ff75a service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:22:23.160985 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4388/8813] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 22:22:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 22:22:33.097774 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=44 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:33.157547 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=47 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:34.091997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198659f7-a39a-42db-a93d-5e8ce41b1f4f None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:22:34.101898 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198659f7-a39a-42db-a93d-5e8ce41b1f4f None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:22:34.106859 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f29ce-a89f-47c0-b8f1-514994f93074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:22:34.107467 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f29ce-a89f-47c0-b8f1-514994f93074 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:22:34.107739 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f29ce-a89f-47c0-b8f1-514994f93074 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:22:34.108005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f29ce-a89f-47c0-b8f1-514994f93074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:22:34.109940 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-469f29ce-a89f-47c0-b8f1-514994f93074 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:22:34.119849 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f29ce-a89f-47c0-b8f1-514994f93074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:22:34.121285 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f29ce-a89f-47c0-b8f1-514994f93074 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:22:34.129205 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4426/8814] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:22:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:22:34.777072 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0d499-063c-41ae-ab2e-a207cd4ff75a service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:22:34.794793 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0d499-063c-41ae-ab2e-a207cd4ff75a service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:22:34.800834 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e979c-5e02-4eb8-9d50-c527489d4d21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6225c81d2620463886d8a936b59784df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:22:34.801486 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e979c-5e02-4eb8-9d50-c527489d4d21 service nova] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:22:34.801789 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e979c-5e02-4eb8-9d50-c527489d4d21 service nova] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:22:34.802060 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e979c-5e02-4eb8-9d50-c527489d4d21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:22:34.803809 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-1b6e979c-5e02-4eb8-9d50-c527489d4d21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:22:34.825755 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e979c-5e02-4eb8-9d50-c527489d4d21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:22:34.827413 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e979c-5e02-4eb8-9d50-c527489d4d21 service nova] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:22:34.839028 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4389/8815] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:22:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:22:44.125713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:44.834150 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=6 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:24:46.198510 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f862d4-9866-4304-a93a-f292b1dfe1dc None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:24:46.198982 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f862d4-9866-4304-a93a-f292b1dfe1dc None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:24:46.199071 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f862d4-9866-4304-a93a-f292b1dfe1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:24:46.258723 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-c6f862d4-9866-4304-a93a-f292b1dfe1dc None None] MFA Rules not processed for user `af8591ebfa174c91853b673360326a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:24:46.259923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-c6f862d4-9866-4304-a93a-f292b1dfe1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:24:46.319761 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4427/8816] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Oct 15 22:24:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:24:46.326005 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e979c-5e02-4eb8-9d50-c527489d4d21 service nova] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:24:46.335833 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e979c-5e02-4eb8-9d50-c527489d4d21 service nova] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:24:46.340966 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc6358-835e-4861-a794-8668da7448a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:24:46.341572 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc6358-835e-4861-a794-8668da7448a9 service glance] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:24:46.341839 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc6358-835e-4861-a794-8668da7448a9 service glance] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:24:46.342099 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc6358-835e-4861-a794-8668da7448a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:24:46.343882 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-a6cc6358-835e-4861-a794-8668da7448a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:24:46.374378 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc6358-835e-4861-a794-8668da7448a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:24:46.375947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc6358-835e-4861-a794-8668da7448a9 service glance] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:24:46.396895 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4390/8817] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:24:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:24:54.227640 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc473fa-fa97-474e-b0fc-e2afd1d9e392 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:24:54.228129 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc473fa-fa97-474e-b0fc-e2afd1d9e392 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:24:54.228224 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc473fa-fa97-474e-b0fc-e2afd1d9e392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:24:54.286009 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-1bc473fa-fa97-474e-b0fc-e2afd1d9e392 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:24:54.287709 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-1bc473fa-fa97-474e-b0fc-e2afd1d9e392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:24:54.305009 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4428/8818] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Oct 15 22:24:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:24:54.345355 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc6358-835e-4861-a794-8668da7448a9 service glance] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:24:54.368013 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc6358-835e-4861-a794-8668da7448a9 service glance] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:24:54.373027 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b98644-f891-42c7-9456-c746d8f9aac4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:24:54.373648 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98644-f891-42c7-9456-c746d8f9aac4 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:24:54.373923 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98644-f891-42c7-9456-c746d8f9aac4 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:24:54.374193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98644-f891-42c7-9456-c746d8f9aac4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:24:54.382443 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b98644-f891-42c7-9456-c746d8f9aac4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:24:54.384019 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b98644-f891-42c7-9456-c746d8f9aac4 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:24:54.392659 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4391/8819] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 22:24:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 22:24:57.853553 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc473fa-fa97-474e-b0fc-e2afd1d9e392 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:24:57.864290 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc473fa-fa97-474e-b0fc-e2afd1d9e392 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:24:57.868115 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b9cf7-6347-4c0d-8a19-6944124769ca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af8591ebfa174c91853b673360326a9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:24:57.868717 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b9cf7-6347-4c0d-8a19-6944124769ca service glance] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:24:57.868997 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b9cf7-6347-4c0d-8a19-6944124769ca service glance] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:24:57.869286 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b9cf7-6347-4c0d-8a19-6944124769ca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:24:57.878555 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b9cf7-6347-4c0d-8a19-6944124769ca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:24:57.880411 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b9cf7-6347-4c0d-8a19-6944124769ca service glance] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:24:57.896625 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4429/8820] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:24:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:25:04.125742 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408ef1b-1da8-4f67-9bd4-491de55bb97e None None] REQUEST_METHOD: `POST` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:04.126248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408ef1b-1da8-4f67-9bd4-491de55bb97e None None] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:04.126340 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408ef1b-1da8-4f67-9bd4-491de55bb97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:04.156531 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.auth.core [None req-0408ef1b-1da8-4f67-9bd4-491de55bb97e None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:25:04.157956 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-0408ef1b-1da8-4f67-9bd4-491de55bb97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:25:04.186425 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4392/8821] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 22:25:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 22:25:04.199291 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b9cf7-6347-4c0d-8a19-6944124769ca service glance] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:04.240923 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b9cf7-6347-4c0d-8a19-6944124769ca service glance] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:04.246034 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa0786-7525-4af6-b95a-e6e26feabf7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:04.246621 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa0786-7525-4af6-b95a-e6e26feabf7e service swift] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:04.246895 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa0786-7525-4af6-b95a-e6e26feabf7e service swift] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:04.247159 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa0786-7525-4af6-b95a-e6e26feabf7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:04.255880 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa0786-7525-4af6-b95a-e6e26feabf7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:04.257464 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa0786-7525-4af6-b95a-e6e26feabf7e service swift] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:04.265760 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4430/8822] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 22:25:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 22:25:05.604950 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408ef1b-1da8-4f67-9bd4-491de55bb97e None None] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:05.631864 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408ef1b-1da8-4f67-9bd4-491de55bb97e None None] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:05.635870 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5b9c9-ba81-4eee-b45b-5b55394da917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:05.636607 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5b9c9-ba81-4eee-b45b-5b55394da917 service neutron] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:05.636944 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5b9c9-ba81-4eee-b45b-5b55394da917 service neutron] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:05.637300 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5b9c9-ba81-4eee-b45b-5b55394da917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:05.639112 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-72b5b9c9-ba81-4eee-b45b-5b55394da917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:25:05.654640 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5b9c9-ba81-4eee-b45b-5b55394da917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:05.656193 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5b9c9-ba81-4eee-b45b-5b55394da917 service neutron] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:05.669024 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4393/8823] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:25:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:25:07.421990 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa0786-7525-4af6-b95a-e6e26feabf7e service swift] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:07.444021 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa0786-7525-4af6-b95a-e6e26feabf7e service swift] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:07.449123 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada6b9b-a001-41e2-9e3c-0ae2aae6fb67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:07.449712 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada6b9b-a001-41e2-9e3c-0ae2aae6fb67 service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:07.449985 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada6b9b-a001-41e2-9e3c-0ae2aae6fb67 service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:07.450252 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada6b9b-a001-41e2-9e3c-0ae2aae6fb67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:07.459625 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada6b9b-a001-41e2-9e3c-0ae2aae6fb67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:07.461351 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada6b9b-a001-41e2-9e3c-0ae2aae6fb67 service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:07.471410 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4431/8824] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:25:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:25:07.784228 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5b9c9-ba81-4eee-b45b-5b55394da917 service neutron] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:07.785675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-72b5b9c9-ba81-4eee-b45b-5b55394da917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:25:07.800263 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5b9c9-ba81-4eee-b45b-5b55394da917 service neutron] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:07.805457 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face052c-3e4d-4747-81d7-c5486a0311b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:07.806025 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face052c-3e4d-4747-81d7-c5486a0311b1 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:07.806295 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face052c-3e4d-4747-81d7-c5486a0311b1 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:07.806559 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face052c-3e4d-4747-81d7-c5486a0311b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:07.815086 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face052c-3e4d-4747-81d7-c5486a0311b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:07.816682 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face052c-3e4d-4747-81d7-c5486a0311b1 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:07.826428 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4394/8825] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:25:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:25:09.245237 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c70e11-5447-45dd-93a1-6c50c48c3d90 None None] REQUEST_METHOD: `POST` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:09.245693 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c70e11-5447-45dd-93a1-6c50c48c3d90 None None] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:09.245781 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c70e11-5447-45dd-93a1-6c50c48c3d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:09.265344 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.auth.core [None req-08c70e11-5447-45dd-93a1-6c50c48c3d90 None None] MFA Rules not processed for user `7a9c40e53f72418381298190ca330e14`. Rule list: `[]` (Enabled: `True`). {{(pid=99099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 22:25:09.266540 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-08c70e11-5447-45dd-93a1-6c50c48c3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:25:09.274436 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4432/8826] 10.222.0.16 () {58 vars in 967 bytes} [Sat Oct 15 22:25:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 22:25:09.286749 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face052c-3e4d-4747-81d7-c5486a0311b1 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:09.298812 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face052c-3e4d-4747-81d7-c5486a0311b1 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:09.304260 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892df93-be45-4fa0-9375-f2f1f97759c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abfea6b853134fed8ea5e32e5406bd5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:09.304872 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892df93-be45-4fa0-9375-f2f1f97759c3 service swift] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:09.305257 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892df93-be45-4fa0-9375-f2f1f97759c3 service swift] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:09.305532 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892df93-be45-4fa0-9375-f2f1f97759c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:09.313925 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892df93-be45-4fa0-9375-f2f1f97759c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:09.315497 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892df93-be45-4fa0-9375-f2f1f97759c3 service swift] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:09.323714 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4395/8827] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Oct 15 22:25:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 22:25:13.574005 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c70e11-5447-45dd-93a1-6c50c48c3d90 None None] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:13.578415 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=100 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:25:13.585197 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c70e11-5447-45dd-93a1-6c50c48c3d90 None None] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:13.588902 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c3246-2de2-4a8d-8cdc-d216325fd479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176e98e72ba840c4a6546519b4c98d90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:13.589949 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3246-2de2-4a8d-8cdc-d216325fd479 service neutron] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:13.590407 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3246-2de2-4a8d-8cdc-d216325fd479 service neutron] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:13.590879 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3246-2de2-4a8d-8cdc-d216325fd479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:13.674586 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c3246-2de2-4a8d-8cdc-d216325fd479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:13.676053 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c3246-2de2-4a8d-8cdc-d216325fd479 service neutron] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:13.700681 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4433/8828] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:25:16.835909 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892df93-be45-4fa0-9375-f2f1f97759c3 service swift] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:16.843379 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=64 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:25:16.852805 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892df93-be45-4fa0-9375-f2f1f97759c3 service swift] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:16.858554 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff7116-cef4-4e7a-b686-3ec71dfad540 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:16.859286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff7116-cef4-4e7a-b686-3ec71dfad540 service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:16.859675 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff7116-cef4-4e7a-b686-3ec71dfad540 service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:16.860048 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff7116-cef4-4e7a-b686-3ec71dfad540 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:16.862091 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-c5ff7116-cef4-4e7a-b686-3ec71dfad540 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:25:16.874947 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff7116-cef4-4e7a-b686-3ec71dfad540 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:16.876633 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff7116-cef4-4e7a-b686-3ec71dfad540 service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:16.888209 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4396/8829] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:25:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:25:20.187992 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c3246-2de2-4a8d-8cdc-d216325fd479 service neutron] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:20.200393 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c3246-2de2-4a8d-8cdc-d216325fd479 service neutron] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:20.206431 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac9267-2f5b-4b23-9f49-7b3964497faf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:20.207162 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac9267-2f5b-4b23-9f49-7b3964497faf service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:20.207437 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac9267-2f5b-4b23-9f49-7b3964497faf service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:20.207806 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac9267-2f5b-4b23-9f49-7b3964497faf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:20.209569 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-35ac9267-2f5b-4b23-9f49-7b3964497faf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:25:20.226161 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac9267-2f5b-4b23-9f49-7b3964497faf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:20.227768 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac9267-2f5b-4b23-9f49-7b3964497faf service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:20.238595 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4434/8830] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:25:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:25:20.254109 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff7116-cef4-4e7a-b686-3ec71dfad540 service placement] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:20.266578 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff7116-cef4-4e7a-b686-3ec71dfad540 service placement] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:20.273808 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:20.274309 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:20.274475 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:20.274899 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] PATH_INFO: `/v3/users/4879d2bce837495baf7411ee27934ece` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:20.276066 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4879d2bce837495baf7411ee27934ece)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:20.278024 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:20.330395 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4879d2bce837495baf7411ee27934ece'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.346411 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4879d2bce837495baf7411ee27934ece'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.359936 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4879d2bce837495baf7411ee27934ece'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.375424 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4879d2bce837495baf7411ee27934ece'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.376191 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4879d2bce837495baf7411ee27934ece'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.383193 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4397/8831] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 22:25:20 2022] DELETE /identity/v3/users/4879d2bce837495baf7411ee27934ece => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:25:20.573532 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac9267-2f5b-4b23-9f49-7b3964497faf service placement] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:20.584613 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac9267-2f5b-4b23-9f49-7b3964497faf service placement] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:20.644474 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:20.645160 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:20.645441 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:20.645713 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] PATH_INFO: `/v3/projects/f02d6a82dfc941c9a0f19bce198d76ab` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:20.646827 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f02d6a82dfc941c9a0f19bce198d76ab)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:20.648812 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:20.695782 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f02d6a82dfc941c9a0f19bce198d76ab', 'request_id': 'req-4b6e730d-677a-4c33-b72a-81b545fe18d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.703220 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f02d6a82dfc941c9a0f19bce198d76ab', 'request_id': 'req-4b6e730d-677a-4c33-b72a-81b545fe18d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.709815 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f02d6a82dfc941c9a0f19bce198d76ab', 'request_id': 'req-4b6e730d-677a-4c33-b72a-81b545fe18d3'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.714635 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:25:20.735742 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] The token cache is being invalidate because project f02d6a82dfc941c9a0f19bce198d76ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 22:25:20.736028 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.736696 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.738188 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4435/8832] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 22:25:20 2022] DELETE /identity/v3/projects/f02d6a82dfc941c9a0f19bce198d76ab => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:25:20.742784 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:20.744248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:25:20.757968 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7dabc-6aea-4033-8a3a-e13b799f381c admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:20.814138 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:20.814758 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:20.815049 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:20.815298 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] PATH_INFO: `/v3/users/ca3addd5bcc9407b87519e47e107002d` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:20.816410 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca3addd5bcc9407b87519e47e107002d)` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:20.819203 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:20.864323 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca3addd5bcc9407b87519e47e107002d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.875551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:25:20.877248 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca3addd5bcc9407b87519e47e107002d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.885390 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca3addd5bcc9407b87519e47e107002d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.892804 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca3addd5bcc9407b87519e47e107002d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.893448 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.notifications [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca3addd5bcc9407b87519e47e107002d'} {{(pid=99100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:20.899414 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4398/8833] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 22:25:20 2022] DELETE /identity/v3/users/ca3addd5bcc9407b87519e47e107002d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:25:21.071174 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:25:21.072675 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:25:21.085639 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e730d-677a-4c33-b72a-81b545fe18d3 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:25:21.145957 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9ecef8e2dba4f429d37c67479eb320e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e66271254b4380bfa7d7e1ccea54cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:25:21.146579 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:25:21.146847 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:25:21.147110 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] PATH_INFO: `/v3/projects/acb14dbd0d644c0e84c8f2c15f3360f0` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:25:21.148222 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb14dbd0d644c0e84c8f2c15f3360f0)` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:25:21.150189 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:25:21.199983 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb14dbd0d644c0e84c8f2c15f3360f0', 'request_id': 'req-e93a50f5-a797-4296-afa6-9677a914b9b9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:21.207077 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb14dbd0d644c0e84c8f2c15f3360f0', 'request_id': 'req-e93a50f5-a797-4296-afa6-9677a914b9b9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:21.213928 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb14dbd0d644c0e84c8f2c15f3360f0', 'request_id': 'req-e93a50f5-a797-4296-afa6-9677a914b9b9'} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:21.223941 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] The token cache is being invalidate because project acb14dbd0d644c0e84c8f2c15f3360f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 22:25:21.224217 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:21.224838 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.notifications [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 22:25:21.226380 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4436/8834] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 22:25:21 2022] DELETE /identity/v3/projects/acb14dbd0d644c0e84c8f2c15f3360f0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 22:25:30.895920 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:25:31.220915 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:30:09.408580 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] Authenticating user token {{(pid=99100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:30:09.410000 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:30:09.492286 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954a74d-0ad9-4460-bf2b-1077ffa061c1 admin admin] Validating token access rules against request {{(pid=99100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:30:09.551945 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efc68b-3e32-4c7c-a3e9-90cf70a180af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:30:09.552551 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efc68b-3e32-4c7c-a3e9-90cf70a180af service placement] REQUEST_METHOD: `GET` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:30:09.552827 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efc68b-3e32-4c7c-a3e9-90cf70a180af service placement] SCRIPT_NAME: `/identity` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:30:09.553128 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efc68b-3e32-4c7c-a3e9-90cf70a180af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:30:09.554863 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.fernet_utils [None req-e3efc68b-3e32-4c7c-a3e9-90cf70a180af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:30:09.566438 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efc68b-3e32-4c7c-a3e9-90cf70a180af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:30:09.568045 ncdv-master-1 devstack@keystone.service[99100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efc68b-3e32-4c7c-a3e9-90cf70a180af service placement] RBAC: Authorization granted {{(pid=99100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:30:09.577803 ncdv-master-1 devstack@keystone.service[99100]: [pid: 99100|app: 0|req: 4399/8835] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:30:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:30:16.649025 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] Authenticating user token {{(pid=99099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 22:30:16.650487 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:30:16.665734 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a50f5-a797-4296-afa6-9677a914b9b9 admin admin] Validating token access rules against request {{(pid=99099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 22:30:16.671246 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ed028-5408-4f08-99e4-7bca61a4145e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ca62592177d43b5b07106003c9efa3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '178e554a27104762a2d5b21906024be0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 22:30:16.672117 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ed028-5408-4f08-99e4-7bca61a4145e service placement] REQUEST_METHOD: `GET` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 22:30:16.672396 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ed028-5408-4f08-99e4-7bca61a4145e service placement] SCRIPT_NAME: `/identity` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 22:30:16.672663 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ed028-5408-4f08-99e4-7bca61a4145e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 22:30:16.675309 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.fernet_utils [None req-618ed028-5408-4f08-99e4-7bca61a4145e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 22:30:16.687778 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ed028-5408-4f08-99e4-7bca61a4145e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 22:30:16.689366 ncdv-master-1 devstack@keystone.service[99099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ed028-5408-4f08-99e4-7bca61a4145e service placement] RBAC: Authorization granted {{(pid=99099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 22:30:16.699046 ncdv-master-1 devstack@keystone.service[99099]: [pid: 99099|app: 0|req: 4437/8836] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Oct 15 22:30:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 22:30:19.573750 ncdv-master-1 devstack@keystone.service[99100]: DEBUG dbcounter [-] [99100] Writing DB stats keystone:SELECT=63 {{(pid=99100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:30:26.694860 ncdv-master-1 devstack@keystone.service[99099]: DEBUG dbcounter [-] [99099] Writing DB stats keystone:SELECT=6 {{(pid=99099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}